24.4 Set Retention Policies for the Data

The Watchdog service in the database monitors system storage capacity. If the capacity exceeds a certain threshold then Watchdog tells the database to start deleting the oldest partitions until disk usage drops below the threshold. By default, the Watchdog threshold is 95% of capacity. To prevent the purging of needed data, you can use storage groups to set retention policies for deleting specific data.

When setting the policies for storage group retention and disk space utilization, do not allow your storage group utilization to increase above 90%. As storage groups near 99% utilization, they start running out of disk space, which reduces the performance of searches due to increasing fragmentation.

For more information about Watchdog, see the Administrator’s Guide to ArcSight Platform on the ArcSight documentation site.

24.4.1 Delete Old Data

Events are stored in their assigned storage groups either in the ArcSight database. Over time, the storage system can retain unneeded or outdated data. To preserve space in the database and improve data retrieval from storage groups, you can configure the database to remove events older than a certain number of months. For example, your data retention policy might expect data older than 24 months to be purged. This process deletes data from the database.

Search automatically applies all deletion settings on the first day of the month at 2:10 a.m.

  1. Create or modify a storage group.

  2. For Delete Data Older than, enter the age of data, in months, that you want to be deleted.

    Ensure that your retention policy takes into consideration the maximum size of your storage groups and database. If a storage group fills up, the oldest events could be purged automatically to make room for incoming events, even if the older events are within the retention period.

  3. Select SAVE.

  4. Apply your changes.