How to troubleshoot high memory usage on systems with our products
Technical Articles ID:
KB74951
Last Modified: 2023-01-06 10:31:52 Etc/GMT
Environment
Endpoint Security (ENS) Threat Prevention 10.x
Microsoft PerfMon
Microsoft PoolMon
Summary
To identify and understand the memory performance issues, we recommend that you use PerfMon and PoolMon, in addition to Windows Task Manager.
Use Windows Task Manager to monitor memory:
- Press Ctrl+Alt+Delete and select Task Manager.
- Click the Performance tab.
- Monitor the following over time:
- Under Physical Memory (K), verify to see whether the Available value decreases. If it decreases, you might have a memory leak.
- Under Kernel Memory (K), observe changes in Paged and Non-paged memory to identify whether it's a kernel-paged or non-paged memory leak.
- If you identify a leak, click the Processes tab, and select View.
- Select Columns and enable the following:
- Page Faults
- Virtual Memory Size
- Paged Pool
- Non-paged Pool
- Handle Count
- Thread Count
- On the Processes tab, click Mem Usage to bring the process using the most memory to the top.
NOTE: If you identify a process that's using high memory and not releasing it, use the following information to help troubleshoot the issue. You might also need to provide a process dump to help identify the cause.
PoolMon and PerfMon
For a more in-depth and accurate analysis, run PoolMon and PerfMon at the same time.
PoolMon
- Prepare to run PoolMon:
- Poolmon.exe is contained in the Microsoft Windows Driver Kit (WDK). You can download it from the Microsoft WDK site.
- Install PoolMon on the computer that you want to test by following the Microsoft product instructions.
- Run PoolMon. The following example outlines a procedure for using PoolMon to detect a memory leak:
- Press Windows+R, type cmd, and press Enter.
- Go to the PoolMon directory.
- Type the following command and press Enter:
IMPORTANT: To obtain the most accurate results, follow the instructions below accurately.
Starting PoolMon changes the data, so you must let it run until it reaches a steady state and the data is reliable.
poolmon -b -p -r -n <filename>.log
Let PoolMon run for at least a few hours; sometimes it might need to run for a few days.
- Stop PoolMon, wait for 30 minutes, and then restart PoolMon.
IMPORTANT: Repeat it every 30 minutes for at least two hours.
- If needed, use the following script to take multiple snapshots over time:
@ECHO off
:LOOP
ECHO %DATE %TIME% >>filename.log
Poolmon -b -p -r -n filename.log
Ping -n seconds 127.0.0.1 >NULL
GOTO LOOP
NOTE: For the seconds value, we recommend every 15 minutes.
- When data collection is complete, examine the following values for each tag, and note any that continually increase:
- Diff (allocations minus free bytes)
- Bytes (number of bytes allocated minus number of bytes freed)
- Examine the allocations that are increasing, and determine whether the bytes are now freed. Allocations that have still not been freed, or have continued to increase in size, are the likely cause.
For more details about PoolMon use, see the Microsoft PoolMon site.
PerfMon
PerfMon offers several methods to save captured data. But, we use the Microsoft Binary Performance Log (BLG) format to troubleshoot performance issues.
- Press Windows+R, type cmd, and press Enter.
- Type the following command and press Enter:
perfmon
- Click Data Collector Sets, User Defined.
- Right-click User Defined, select New, and select Data Collector Set.
- Type a name (for example, Trellix <date_timestamp>), select Create manually, and click OK.
- Under Create data logs, select only Performance counter, and click Next.
- Click Add. In the next page from the drop-down list, select Processor, select <All instances>, and click Add.
- Select Memory from the drop-down list and select <All instances>, then click Add.
- Select Process from the drop-down list, and select <All instances> from the next drop-down list.
- Click Add, OK, Next, Next, then select Start this data collector set now, and then click Finish.
Wait for the data to run long enough to capture the information and collect the log while reproducing the issue.
- Right-click User Defined, right-click the <log name>, and click Stop.
- Retrieve the log from C:\Perflogs\Admin\examplename\computername_date-time\DataColletor01.blg.
For more details about PerfMon use, see the Microsoft PerfMon site.
|