Rebuild the xDB index in Solr

This topic describes how to rebuild the xDB Search index if you are using the Solr search provider. Before rebuilding, ensure that your environment meets the following requirements:

  • There are two xConnect Solr cores - the live core (for example, “xdb”) and the rebuild core, which has a _rebuild suffix (for example, “xdb_rebuild”)
  • Both cores are based on the basic_configs
  • xConnect schema modifications have been applied to both cores

Important

Rebuilding the reporting database does not trigger a rebuild of the index. You must rebuild the index separately.

Create a rebuild core

If you do not have a rebuild core, complete the following steps. If you have a rebuild core, skip this section.

  1. Go to the server where Solr is installed.

  2. Navigate to the folder that hosts the core - for example: C:\Solr-6.1.0_https\server\solr.

  3. Create a copy of your core - for example, xdb - and rename it xdb_rebuild.

  4. Open the xdb_rebuild folder and delete the following:

    1. The core.properties file
    2. The entire data folder
  5. Open a browser and go to the Solr UI - for example, https://localhost:8986/solr/#/ (note that the URL uses HTTPS).

  6. Click Core Admin in the left-hand menu:

  1. Click Add Core and type in the name and path of your core rebuild core - leave other properties unchanged:

Rebuild the core

You cannot trigger an index rebuild from a user interface or via the xConnect Client API. You must explicitly trigger the rebuild process in one of the following ways:

Invoke the rebuild request command

To invoke the rebuild request command:

  1. Go to the server where the xConnect Search Indexer is running and open command line.
  2. Navigate to the folder that contains the service.
  3. Run XConnectSearchIndexer -rr (alternatively XConnectSearchIndexer -requestrebuild). The command registers a small document in the live core signaling that the rebuild should be started. The xConnect Search Indexer will notice the document and start the rebuild process.

Manually post a rebuild document

Post the following document directly to the live core in Solr (for example, using the Powershell Invoke-WebRequest cmdlet):

{"id":"xdb-rebuild-status","lastupdated_dt":"2017-05-24T10:15:47.603436Z","rebuildstate_l":1,"progresstotal_d":0.0,"progresscurrent_d":0.0}

Confirm rebuild

When the rebuild is complete, the cores are swapped and the rebuilt core becomes the live core. When you swap cores, names are swapped but physical folder locations are not swapped:

Name Before Rebuild Name After Rebuild Core Location
xdb (live) xdb_rebuild C:\Solr-6.1.0_https\server\solr\xdb
xd_rebuild xdb (live) C:\Solr-6.1.0_https\server\solr\xdb_rebuild

At this point, the rebuild core is no longer in use. However, it still has all data that was in the live core before the swap.

Manually swapping cores back

If you need to swap back (not rebuild a second time), you must first make sure the cores were recently swapped. Swapping back will only work if:

  • The xDB Collection provider’s change tracking data still holds all changes since the cores were swapped.
  • xConnect search can be offline while the swapped back core gets up to date by pulling the above changes. The smaller the time window/changes since the swap, the less time this would take.

Assuming you meet these requirements, proceed with the following instructions. If you do not meet these requirements, you will not be able to swap cores.

  1. Stop the indexer.

Warning

Failing to stop the indexer before swapping back the cores may cause the indexer to skip data.

  1. Swap the cores by using the swap command in Solr directly.
  2. Start the indexer again.

Deleting the old core

If you are confident that the rebuild completed successfully, you can remove all data from the _rebuild core by deleting or clearing the core.

Note

Make sure you delete the old core if your reason for rebuilding was to remove all PII sensitive data from the index. The old core will still contain this data.