EDR searches with a large number of conditions take a long time or fail
Technical Articles ID:
KB96113
Last Modified: 2023-03-03 07:30:49 Etc/GMT
Environment
Endpoint Detection and Response (EDR)
ePolicy Orchestrator (ePO) 5.x
Problem
Historical searches take more than two hours to complete before failing.
Large searches also fail for search queries with 5,000 or more hash conditions, over a search period of three days or more.
Cause
Your search contains more queries than the maximum query clause count of 1,024. This count is set by default by Elastic Search 6.8.3 in the Activity Catalog. The number of clauses in the Elastic Search isn't a static value, and depends on the query along with the heap size and the processing capacity available for the search.
In this situation, with 5,000 or more hashes in the search query, 5,000 or more match conditions are present as clauses in the query. These hashes and conditions, along with the heap size and number of processors, violate this setting. You then see the slowness, errors, and failures.
Solution
Don't exceed the limit of 1,024 search conditions in your searches. Searches within this limit work as expected for all durations (up to 30 days) in current production environments.
Perform smaller searches, with at most 1,024 search conditions, to not exceed the current backend and server limitations.
At the time of article authoring, almost all servers lack the capacity and resources required to support and successfully complete searches with 5,000+ conditions.
NOTE: You can increase the value of the max clause count setting in the Elastic Search configuration. Making this change isn't recommended as per the Elastic Search documentation. This limit is in place to prevent searches from becoming too large and consuming too much CPU and memory. Increasing this value can lead to performance degradations and memory issues.
|