This article provides basic information about the minimum data collection steps to troubleshoot common MOVE AV issues.

Make sure that all logs you collect are from the same system that experiences the issue. You must collect all the logs at the same time. Use the logging data time stamps to troubleshoot the problem.

You can't use mismatched logs from different systems or logs collected at different times for troubleshooting. It might result in the need to recollect all minimum data collection logs.
 
IMPORTANT: The following files are required for Technical Support:

• Minimum Escalation Requirements (MER) files with debug logging for MOVE AV are required for all issues.
  • For information about debug logging, see Verify whether MOVE AV debug logging is enabled.
  • For information about how to download the MER files, see KB59385 - How to use MER tools with supported Trellix products.

Depending on the issue, the following Microsoft tools might be required:

Click to expand the section you want to view:

How to use Windows Performance Recorder (WPR)

 

1. Click Start, type cmd.exe in the search bar, right-click cmd.exe from the list, and select Run as administrator.
2. Type wprui.exe and press Enter to start WPR.
    
   NOTE: If the program doesn’t run or isn’t found, you must first install it. WPR is part of the Windows Performance Toolkit. It’s available from the Windows SDK, or the Windows Assessment and Deployment Kit:
   • For Windows SDK, see the SDK web page.
   • For Windows Assessment and Deployment Kit, see the WPR web page.
      
3. Choose to use a Performance Scenario and other settings, as recommended in the following table:

   Performance Issue	Performance Scenario	Logging Mode	Profiles to Include	Number Of Iterations
   Slow boot or logon	Boot	File	First-level Triage, CPU usage File I/O activity, Minifilter I/O Activity	At least 1
   High CPU usage	General	File	First-level Triage, CPU usage File I/O activity, Minifilter I/O Activity	N/A
   Application is slow or unresponsive	General	File	First-level Triage, CPU usage File I/O activity, Minifilter I/O Activity	N/A

   
   WPR data is collected to allow for deeper analysis of the problem. The Minifilter I/O Activity option is one of the most important, and is located under the "Scenario Analysis" section.
   
   The use of WPR places extra strain on the system, which can change or mask the original problem you want to investigate. Collect two data sets with different detail levels. Use the Light setting to show the issue, and the Verbose setting to allow a data set suitable for deeper analysis. Capture at least 30 seconds.
   
   When possible, collect a WPR log without the issue while you perform the same task, for comparative purposes. A data set without ENS present is needed to establish the benchmark for expected performance.

How to prepare Process Monitor:

 

1. Prepare Process Monitor:
   1. Download Process Monitor.
   2. Extract Procmon.exe to the Desktop.
2. Run Process Monitor:
   When you’re ready to start Process Monitor, use the option below that the relevant data collection section requires.
   • To enable the Process Monitor boot logging option, if required by the relevant data collection section:
     1. Open the Process Monitor console.
     2. Click Options.
     3. Click Enable Boot Logging.
     4. Click OK on the pop-up window. The next time a reboot occurs, a boot trace log is created.
     5. To save the log, run Process Monitor again and click File, Save (select All Events and use the native PML format).
         
   • To immediately start Process Monitor:
     1. Run Procmon.exe. It automatically starts to capture process information.
     2. To stop Process Monitor, press Ctrl+E or click File and deselect Capture Events. Press Ctrl+E again to resume data collection.
     3. To save the log, click File, Save (select All Events and use the native PML format).

Verify whether MOVE AV​ debug logging is enabled

Enable MOVE AV debug logging:

A) Debug logging for MOVE AV Agentless:

1. Log on to the MOVE Agentless SVA Console:
   
   Logon: svaadmin
   Password: <ENTER_PASSWORD>
    
2. Use the following commands to enable debug level logging:
   
   sudo /opt/McAfee/move/bin/chloglevel DEBUG DEBUG DEBUG
   [sudo] password for svaadmin: <ENTER PASSWORD>
   
   NOTE: This command requires sudo rights, and the MOVE services are restarted when it executes.
    
3. Replicate the issue.
4. To disable debug level logging, type the following at the command prompt and press Enter:
   
   sudo /opt/McAfee/move/bin/chloglevel WARN WARN WARN
   [sudo] password for svaadmin: <PASSWORD>

B) Debug Logging for MOVE AV Multi-Platform:

1. At the command prompt, type the following command and press Enter:
   
   mvadm loglevel enable <module_name> <log level types>

Examples:

• mvadm loglevel enable SCAN ALL
• mvadm loglevel enable ALL ALL

2. To disable debug logging, type the following at the command prompt and press Enter:
   
   mvadm loglevel disable <module_name> <log level>

Examples:

• mvadm loglevel disable SCAN ALL
• mvadm loglevel disable ALL ALL

NOTE: This command clears the specified log level types, or all types for module MODULE_NAME or for all modules.

3. To set the debug log level back to default settings, type the following at the command prompt and press Enter:
   
   mvadm loglevel disable ALL INFO DETAIL FNENTRY FNEXIT

Slow boot, startup, or logon

Perform the steps in this section if the symptoms are any of the following:

• Slow boot or startup
• Slow logon

Data collection steps for AMTrace and Process Monitor:

1. Start Process Monitor and enable the boot logging option.
2. Reboot the system.
3. Reproduce the issue.
4. Log on to the system.
5. Open Process Monitor and save the boot log.

Data collection steps for WPR:

1. Run WPR.
2. Configure the Boot Performance Scenario.
3. Start the capture.
4. Reboot the system.
5. Reproduce the issue.
6. Log on to the system.
7. Allow the WPR: Boot Trace to finish.
8. Close WPR.
9. Capture the saved ETL files.

Slow performance (reproducible)

Perform the steps in this section if the symptoms listed below are reproducible:

• Slow application startup
• Slow application performance
• Slow system performance

Data collection steps for AMTrace and Process Monitor:

1. Start Process Monitor.
2. Reproduce the issue.
3. Stop Process Monitor and save the log.

Data collection steps for WPR:

1. Run WPR.
2. Configure the General Performance Scenario.
3. Start the trace.
4. Reproduce the issue.
5. Stop the trace.
6. Close WPR.
7. Capture the saved ETL file.

Slow performance (random)

Perform the steps in this section if the symptoms occur randomly and are any of the following:

• Slow application startup
• Slow application performance
• Slow system performance

Data collection steps for WPR:

1. Run WPR.
2. Configure the General Performance Scenario with Memory as the Logging Mode.
3. Start the trace.
4. Reproduce the issue.
5. Save the trace as soon as possible after you reproduce the issue.
6. Close WPR.
7. Capture the saved ETL file.

System hangs, deadlock, or Bug Check (blue screen)

Perform the steps in this section if the symptoms are any of the following:

• System hangs, or a deadlock occurs
• System Bug Check (blue screen)

Data collection steps for a system hang or deadlock:

1. Configure the system to create a full memory.dmp. For details, see KB56023 - Create a memory dump for analysis by Technical Support.
2. Configure the system to allow for a keyboard crash. For details, see the Microsoft article Forcing a System Crash from the Keyboard.
    
3. Create the dump file when the issue occurs. Generally, the longer you can wait before you generate the dump file, the easier it is to identify the hang condition in the dump.

Data collection steps for a system Bug Check:

1. Configure the system to create a full memory.dmp. For details, see KB56023 - Create a memory dump for analysis by Technical Support.
2. Collect the full dump file when the system Bug Check (blue screen) occurs.

Data collection steps for FLTMC output:

1. Open an administrative command prompt.
2. Type fltmc.
3. Collect the output.

Application hangs, deadlock, or crash

Perform the steps in this section if the symptoms are any of the following:

• Application hangs, or deadlock (not responding and doesn’t recover)
• Application crash

Data collection steps for an application hang or deadlock:

1. Download the Microsoft ProcDump tool. 
2. Extract ProcDump to the Desktop.
3. Open an administrative command prompt, and change directory to C:\Users\username\Desktop\Procdump.
4. Run the command: procdump -ma <process name>
5. Collect the created dump file, which is located in the Procdump folder.

Data collection steps for an application crash:

1. Download the Microsoft ProcDump tool.
2. Extract ProcDump to the Desktop.
3. Open an administrative command prompt, and change directory to C:\Users\username\Desktop\Procdump.
4. Run the command: procdump -ma -e <process name>
    
   NOTE: The -e switch instructs ProcDump to generate a dump the next time the process crashes.
    
5. Wait for the process to crash again.
6. Collect the created dump file, which is located in the Procdump folder.

Memory leak (user or kernel)

Perform the steps in this section if the symptoms involve a User Mode or Application memory leak.

Collect three User Mode or Application crash dumps for analysis.

1. Download the Microsoft ProcDump tool.
2. Extract ProcDump to the Desktop.
3. Identify the process name that is leaking memory.
4. Enable a stack trace on the leaking process.
   See KB91252 - How to enable a stack trace using the gflags.exe utility.
5. Wait for the suspect process to show high memory usage.
6. Open an administrative command prompt, and change directory to C:\Users\username\Desktop\Procdump.
7. Run the following command: procdump -ma <process name>
8. Collect the created dump file, which is located in the Procdump folder.
9. Repeat the steps and collect three User Mode or Application crash dumps for analysis.
10. Disable the stack trace on the process once all crash dump files are collected.

NOTE: You need to perform memory leak data collection over time, because the memory leak exhibits its behavior over time. For the best data collection results, we recommend that you reboot the system, and then start the following data collection. This sequence allows the data set to show the existence of the memory leak, over time, as it manifests on the system.

Perform the steps in this section if there’s a suspected kernel memory leak involving a Trellix process:

1. Familiarize yourself with Poolmon and Perfmon usage and configuration as described in KB74951 - How to troubleshoot high memory use on systems with VirusScan Enterprise 8.8.x.
2. Configure the system to create a full memory.dmp. For details, see KB56023 - Create a memory dump for analysis by Technical Support.
3. Configure the system to allow for a keyboard crash. See this Microsoft article for more information.
4. Reboot the system reported to show a memory leak.
5. Start the Poolmon and Perfmon data collection.
6. Wait for the system to show high memory usage.
7. Stop Poolmon and Perfmon, and collect the resulting data.
8. Force the system to perform a bug check while the high memory usage is still exhibited.
9. Collect the memory dump.

Issues related to Device Guard or Credential Guard

Perform the steps in this section if the symptoms involve Device Guard or Credential Guard:

1. Collect the appropriate ENS data for the experienced symptom, as outlined in this article.
2. Also, collect an ETW (Event Tracing for Windows) trace with the following command, executed in an administrative command prompt:
   
   @echo off
   ECHO These commands enable tracing:
   @echo on
   logman create trace "base_DeviceGuard" -ow -o c:base_DeviceGuard.etl -p "Microsoft-Windows-DeviceGuard" 0xffffffffffffffff 0xff -nb 16 16 -bs 1024 -mode Circular -f bincirc -max 4096 -ets
   @echo off
   echo
   ECHO Reproduce your issue and enter any key to stop tracing
   @echo on
   pause
   logman stop "base_DeviceGuard" -ets
   @echo off
   echo Tracing has been captured and saved successfully at c:base_DeviceGuard.etl
   pause

MOVE AV components fail to install

Perform the data collection steps in this section if one or more components don't install.

NOTE: Ensure that you collect the data during the installation of MOVE AV.

1. Start Process Monitor.
2. Re-create the issue.
3. Run the MOVE AV installation.
4. Stop Process Monitor and save the log.
5. Collect a MER file (run as Administrator).

Issues that involve the Threat Intelligence Exchange (TIE) module

Perform the steps in this section if the symptoms are related to TIE:

1. Collect the appropriate data based on the symptoms outlined in this article.
2. Collect the TIE Server log on the TIE Server appliance at /var/McAfee/tieserver/logs/tieserver.log.