Troubleshooting duplicate system tree entries in ePolicy Orchestrator
Technical Articles ID:
KB93591
Last Modified: 2023-08-11 06:27:55 Etc/GMT
Last Modified: 2023-08-11 06:27:55 Etc/GMT
Environment
ePolicy Orchestrator (ePO) 5.x
Summary
In ePO, you might have a situation where you have multiple entries for a given system in the ePO system tree. These entries are commonly known as duplicate systems. This article describes the different types of duplicate systems, how they get created in the system tree, and how to troubleshoot the problem.
Managed versus Unmanaged (First type)
When dealing with duplicate systems in ePO, the first question to ask is "Are all duplicate entries shown as Managed or do some of them show as Unmanaged?"
Multiple rows for a single system tree entry (Second type)
The second type of duplicate system is not strictly speaking a duplicate at all. It's a scenario wherein the system tree displays multiple rows for the same system, even though there's only one actual system in ePO.
The symptoms for this scenario are as follows:
In the case above, ePO thinks that the system has both VirusScan Enterprise (VSE) 8.8.0.2232 and 8.8.0.2233 installed, which is impossible. You can work around this problem by removing the relevant column, in the case, the 'Product Version (VirusScan Enterprise)' column, from the system tree view.
NOTE: This is only a cosmetic fix, and doesn't address the underlying problem. To resolve the root cause correctly, see KB93129 - Duplicate managed products entries shown for a single system.
New system tree entries created when they mustn't be (Third type)
The third type of duplicate system is the most complex to diagnose. This issue occurs when a managed system already has an entry in the system tree, which then communicates with ePO, rather than updating the existing entry. In this situation, ePO thinks it's a new system, and creates an entry for it.
The giveaway symptoms for this scenario are as follows:
NOTE: The image above shows that the last communication and Agent GUID values are all different.
To properly troubleshoot duplicate systems for this scenario, you need to understand how ePO decides when it needs to create a new entry for a system.
Every time a system communicates with ePO, ePO has two courses of action:
NOTE: This is how earlier versions of ePO used to work. In environments wherein systems were being frequently rebuilt, this behavior caused unacceptable numbers of duplicate entries to be created. So, ePO's behavior was changed.
Another check was added as the first step in the 'create a new entry' path. When a system communicates with ePO, the first message it sends contains (among other things) the following:
The above is an important point. The MAC address check is only looking to see if the specific MAC address reported by the client exists anywhere else in the database. ePO only stores one MAC address for any given system and updates it every time the system communicates. The MAC address stored by ePO is the address reported by the client the last time it communicated.
There are certain scenarios wherein multiple systems can have the same MAC address. Common examples are listed below:create a new entry ' path. If the system actually has an entry in the system tree, the duplicate is created.
Example:
For a duplicate to be created, the system doesn't have to report a new MAC address, just a different one from the last time it communicated with ePO.
Special case with VDI mode clients
For non-persistent VDI environments, the agent can operate in a special mode known as VDI mode. This mode allows ePO to make sure that the VDI client always uses the same GUID, which enables ePO to track the specific client through the deprovision and provision cycle.
For VDI mode details, see KB87654 - McAfee Agent provisioning and deployment on Virtual Desktop Infrastructure systems.
Summary:
Troubleshooting
Now, we understand how ePO decides when to create a new entry. We can now troubleshoot why they're being created.
As we know, for ePO to decide to create a new entry, it must not know the GUID. So the first question is, "Why did the client GUID change?"
Here are the most common ways that a client creates a new GUID:
Example: We know that a new GUID is created in either of the following scenarios:
First, make sure that ePO isn't configured to skip the MAC check:
Managed versus Unmanaged (First type)
When dealing with duplicate systems in ePO, the first question to ask is "Are all duplicate entries shown as Managed or do some of them show as Unmanaged?"
- Managed
A system only shows as Managed in ePO, if it has an agent installed and it has communicated at least once with ePO.
- Unmanaged
These systems are entries created in the system tree via another method; for example, by an Active Directory sync task, or manually adding via the New Systems feature.
When a system communicates with ePO for the first time, ePO tries to match it to an existing unmanaged entry in the tree. When an unmanaged entry exists, but ePO is unable to match it, it creates a new entry in the system tree. You now have a duplicate system with two entries, one of which is unmanaged.
For further details, see the ePO Product Guide in the following section: How a system is added to the System Tree.
To address the issue above, you must review how the systems are being added to the tree.
Example: If you're using an Active Directory sync task, verify that the synchronization settings are configured correctly. The option Systems that exist elsewhere in the system tree can create duplicate entries.
To address the issue above, you must review how the systems are being added to the tree.
Example: If you're using an Active Directory sync task, verify that the synchronization settings are configured correctly. The option Systems that exist elsewhere in the system tree can create duplicate entries.
Multiple rows for a single system tree entry (Second type)
The second type of duplicate system is not strictly speaking a duplicate at all. It's a scenario wherein the system tree displays multiple rows for the same system, even though there's only one actual system in ePO.
The symptoms for this scenario are as follows:
- The agent GUID and the last communication time columns for all entries are the same.
- If you select the checkbox for one of the systems, the other systems are automatically selected as well.
In the case above, ePO thinks that the system has both VirusScan Enterprise (VSE) 8.8.0.2232 and 8.8.0.2233 installed, which is impossible. You can work around this problem by removing the relevant column, in the case, the 'Product Version (VirusScan Enterprise)' column, from the system tree view.
NOTE: This is only a cosmetic fix, and doesn't address the underlying problem. To resolve the root cause correctly, see KB93129 - Duplicate managed products entries shown for a single system.
New system tree entries created when they mustn't be (Third type)
The third type of duplicate system is the most complex to diagnose. This issue occurs when a managed system already has an entry in the system tree, which then communicates with ePO, rather than updating the existing entry. In this situation, ePO thinks it's a new system, and creates an entry for it.
The giveaway symptoms for this scenario are as follows:
- The Agent GUID and last communication time are different for each entry.
- If you select the checkbox for one of the systems, the other systems aren't automatically selected.
- Only one of the entries is active. For example, when the client system communicates, the last communication time is only updated on one of the duplicate systems.
NOTE: The image above shows that the last communication and Agent GUID values are all different.
To properly troubleshoot duplicate systems for this scenario, you need to understand how ePO decides when it needs to create a new entry for a system.
Every time a system communicates with ePO, ePO has two courses of action:
- The system is already known to ePO. In this case, ePO needs to update its properties, and see if there are any new policies or tasks.
- The system isn't known to ePO. In this case, ePO must create a new entry for it in the system tree.
- The first question is "Is this system a VDI mode client?"
- If the answer is Yes, ePO treats the system differently. This topic is covered in more detail later in this article.
- If the answer is Yes, ePO treats the system differently. This topic is covered in more detail later in this article.
- If the system isn't a VDI client, the next most important question is "is the Agent GUID already in the database?"
- If the GUID is in the database, the system is already known to ePO, and follows the 'update properties' course of action.
- If the GUID is not in the database, it might be a new system, so ePO starts to follow the '
create a new entry ' path.
- The agent is uninstalled and reinstalled, or reinstalled with the command option
/forceinstall . - The system is rebuilt from an image.
- The system has a hardware change. For example, the hard-drive is taken from one laptop and placed in another chassis. This action would cause the
SMBiosUUID to be different the next time the agent starts. As a result, the agent creates a new Agent GUID.
NOTE: This is how earlier versions of ePO used to work. In environments wherein systems were being frequently rebuilt, this behavior caused unacceptable numbers of duplicate entries to be created. So, ePO's behavior was changed.
Another check was added as the first step in the '
- Agent GUID
- MAC address detected by the Agent
The above is an important point. The MAC address check is only looking to see if the specific MAC address reported by the client exists anywhere else in the database. ePO only stores one MAC address for any given system and updates it every time the system communicates. The MAC address stored by ePO is the address reported by the client the last time it communicated.
There are certain scenarios wherein multiple systems can have the same MAC address. Common examples are listed below:
- Systems that connect via a Virtual Private Network (VPN) concentrator
- Systems that present a virtual MAC address, such as server clusters, with teamed Network Interface Cards (NICs).
- First, you can temporarily disable the MAC check globally in ePO, by setting a registry key and restarting the ePO services. This method is most commonly used in the teamed NIC scenario. The MAC address check is disabled, and both nodes of the teamed NIC cluster are allowed to communicate with ePO. Because they have different GUIDs, each node has an entry created in the system tree. Then, the MAC address check is re-enabled so that other systems that change GUID don't create duplicate entries.
To disable the MAC address check, see KB57886 - Unable to see both nodes of a cluster in the ePolicy Orchestrator directory when they share a MAC address.
- For systems that connect via VPN, you can configure ePO to selectively skip the MAC check based on the Organization Unique Identifier (OUI).
NOTE: The OUI is the fist six characters of the MAC address, used to identify the vendor of the network device.
You can add multiple OUIs to ePO. When a system with an unknown GUID communicates with ePO, the system's OUI is compared with the list in the database. If a match is found, ePO skips the MAC check and creates a new entry for the system. For more details, see the Add Virtual MAC Vendor in the ePO Product Guide.
Example:
- Imagine we have a laptop with an agent installed. The agent is communicating with ePO without any problems.
- It has an agent GUID of 1111111111.
- It's connected to the network via Wi-Fi, with a MAC address of 11:11:11:11:11:11. At this point, in the ePO database, this system has a GUID of 1111111111. Because the last time it communicated, it was using the wireless NIC, ePO has recorded the MAC address for this system as 11:11:11:11:11:11.
- Suddenly, it suffers a fatal software crash, and your IT department has to reimage it.
- The IT engineer connects the laptop to the network using the laptop's wired NIC. The NIC has a MAC address of 22:22:22:22:22:22. The engineer reinstalls the operating system and agent.
- As it's a brand-new install, the first time the agent starts, it generates a new agent GUID of 2222222222 and communicates with ePO.
- ePO first searches the database for an entry with a GUID of 2222222222. It doesn't find one, so it continues to the MAC check.
- ePO then searches the database for an entry with a MAC address of 22:22:22:22:22:22. It doesn't find one, so ePO creates a new entry with GUID 2222222222 and MAC address 22:22:22:22:22:22. Now, we have a duplicate entry, two entries for the same system, only one of which is active.
For a duplicate to be created, the system doesn't have to report a new MAC address, just a different one from the last time it communicated with ePO.
Special case with VDI mode clients
For non-persistent VDI environments, the agent can operate in a special mode known as VDI mode. This mode allows ePO to make sure that the VDI client always uses the same GUID, which enables ePO to track the specific client through the deprovision and provision cycle.
For VDI mode details, see KB87654 - McAfee Agent provisioning and deployment on Virtual Desktop Infrastructure systems.
Summary:
- When a VDI client shuts down, the agent sends a specific message to ePO.
- ePO then tags that system as deprovisioned in the database.
- When VDI-mode agents communicate with ePO, they inform ePO that they're in VDI mode at the start of the communication.
- ePO checks to see if there's an entry in the database that has the same fully qualified domain name. ePO then tags it as deprovisioned.
- If it doesn't find one, ePO creates a new entry for the system.
- ePO doesn't look to see if the GUID or MAC address exists. In a non-persistent VDI environment, both aren't relevant.
- If there's anything that prevents ePO from receiving the deprovision message, the next time that client is provisioned, ePO creates a duplicate entry.
- The most common cause is if the VDI client isn't cleanly shut down. For example, if it crashes, or the host forcibly shuts down the system.
Troubleshooting
Now, we understand how ePO decides when to create a new entry. We can now troubleshoot why they're being created.
As we know, for ePO to decide to create a new entry, it must not know the GUID. So the first question is, "Why did the client GUID change?"
Here are the most common ways that a client creates a new GUID:
- Agent was uninstalled and reinstalled.
- Agent was upgraded using the
/forceinstall switch, or deployed from ePO with theForce installation over existing version option selected. - Agent detected a new
SMBiosUUID , indicating a hardware change. For example, the hard drive was moved from one laptop to another. - Agent health check failed, which caused it to regenerate its database files and create a new Agent GUID.
Example: We know that a new GUID is created in either of the following scenarios:
- The system has been rebuilt.
- The agent has been uninstalled and reinstalled.
- Look in the agent installation logs (
frminst.log ).- Was the agent recently installed with the
/forceinstall option? - If so, we don't recommend general use of this option. It must only be used under certain circumstances, such as if you're trying to upgrade the agent to a new location with the
/instdir switch.
- Was the agent recently installed with the
- Look in the
mascvc log file.- Look for evidence of the GUID changing.
- For example, if the MA
healthcheck fails, you would see an error message that saysmaconfig.Error: MA databases integrity check failed .
First, make sure that ePO isn't configured to skip the MAC check:
- Make sure that the agent hasn't been installed in VDI mode by mistake.
- View the system properties for the system in ePO. Look at the bottom of the list of properties where the VDI property is located. It must be Yes if the agent is installed in VDI mode.
- If the agent isn't supposed to be in VDI mode, uninstall it, and reinstall it, without using the
/EnableVDIMode switch. - Make sure that the MAC search has not been disabled globally. For details, see KB57886 - Unable to see both nodes of a cluster in the ePolicy Orchestrator directory when they share a MAC address.
NOTE: The setting above applies on a per-agent-handler basis. So, if you have multiple Agent Handlers, look at them all and make sure that the MAC search isn't disabled.
- If the system isn't connecting via VPN, verify that the OUI of the agent's MAC address is added to ePO's Virtual MAC Vendors list by mistake. If it has, remove the relevant OUI.
- Examine the ePO audit log, and enter the client system name in the Quick Find filter. You see an
ASCI - New System event when ePO created the entry. The details of the event you see is the MAC address that the client reported. - Compare the details with the MAC address in the system properties for the most recent duplicate entry. This entry is the MAC address used by the system before the duplicate was created.
This help explain why ePO was unable to match the MAC address. For example, the previous entry might have the MAC address for the wireless adapter, and the audit log entry has the address of the docking station adapter.
- To conclude, by far, the most common cause of duplicate entries in ePO is the unnecessary creation of a new GUID on the client systems.
- In turn, the most common cause of this issue is reinstalling the agent using the
/forceinstall switch, seen as part of a scheduled server task or logon script. - The key to understanding where duplicate entries are coming from is to first to discover why the GUID is changing. Then, identifying the source of the problem is greatly simplified.
- The McAfee Agent /forceinstall switch must not be used unless indicated in a KB article to resolve an issue.
- Using the /forceinstall switch as a regular process can cause issues with the McAfee Agent, which leads to duplicate entries in the ePolicy Orchestrator system tree. It might lead to systems getting an incorrect policy assignment.
- Change the installation directory
Or - Downgrade the McAfee Agent
Affected Products
Languages:
This article is available in the following languages: