cqrs – How to detect broken indexes in Lucene?


I am the main author of Squidex, a headless CMS (https://github.com/squidex/squidex).

Squidex has a CQRS like architecture. So when you create a content a new ContentCreated event is created. An event consumer listens to all content events and populates lucene indexes.

Unfortunately there is only a single event consumer for the whole application and not one event consumer per project. But some installations include the Squidex cloud offering could have dozens to thousands of projects.

Lets say the user makes a search and does not retrieve the expected results. This could happen because somehow an event has been missed. Of course it would be a bug that an event is not handled by the event consumer, but if it happens it needs to be handled properly afterwards.

But I cannot just restart the event consumer, because then it would start from beginning for all events and projects. So it would be great if I can somehow detect that the index is very likely broken and just recreate this index. The question is “how?”. The index structure is not corrupt and the number of index entries could be correct, for example when an update event has been missed.

Of course I can also add something to the UI to recreate the full text index per project manually, but it would be great if it would just work.