How Informer Deletes ElasticSearch Indices

When Informer Queries run, the results are indexed and stored within ElasticSearch as indices.

Informer performs a routine cleanup of ElasticSearch with a default Job called Auto System Cleanup. By default, this Job runs once every 24 hours at 11:59 PM server time. The firing time can be adjusted to correspond with planned maintenance windows. If the Informer server is offline during the configured fire time, the Job executes on startup after any preceding Jobs in the queue finish. It is best not to change the interval for this Job unless directed by Support.

Informer removes indices marked for deletion when the Auto System Cleanup Job runs, which calls the /datasets/_vacuum route. This route can be called directly if API licensing is enabled.

Indices become marked for deletion by setting a dataExpiresAt value on the record.

• For Dataset Queries using the Append update mode, this value is set when using the Refresh or Clear data actions. Previously indexed data then has a date-time recorded for dataExpiresAt.
• For Ad hoc Queries, the results-expiration value is configured under the Queries section of the Settings page. This configuration determines the required idle time for the results before the dataExpiresAt record is set, marking the data for cleanup.

The vacuum process removes any expired indices, along with any orphaned indices, based on whether the dataExpiresAt flag is on or before when the process runs. Indices that meet the criteria but are marked as in-progress of indexing are excluded from this cleanup.

Note: In a multi-Tenant deployment, each Tenant has its own Auto System Cleanup Job.

The orphaned and expired indices can also be inspected and cleaned up manually from System Health.