Jun 07 2017

Print this Post

VMM is unable to refresh Hyper-V hosts

I recently had to endure a painful issue with VMM and Hyper-V while working on a VMWare to Hyper-V migration project.

First let me tell you about the architecture,

  • 2 Hyper-V clusters made of Lenovo blade servers. Each cluster in its own enclosure.
  • 1 VMM cluster with 2 nodes as VMs. One node on each Hyper-V cluster.

Sounds amazing, except when the VMM service is hosted on the second node (VMM2) and is unable to refresh any of the hosts in cluster 2, the cluster it lives in.

Every time I tried to host the service on it, I got this ugly error,

Error (2916) VMM is unable to complete the request. The connection to the agent 'srvhv5.example.com' was lost. WinRM: URL: [http://srvhv5.example.com:5985], Verb: [ENUMERATE], Resource: [http://schemas.microsoft.com/wbem/wsman/1/wmi/root/standardcimv2/MSFT_NetAdapter], Filter: [] Unknown error (0x80338126)

After several days of professional head banging, it occurred to me that maybe VMM2 just hates its host and wants to move out, so I live migrated it to Hyper-V Cluster 1 and voila! Everything just works.

Apparently either VMM1 and VMM2 are in an unorthodox relationship and want to live together, or a hardware issue on Cluster 2 is causing the mess. A little more digging and I found that although the blade models are identical, the enclosures and network cards are not. Accordingly the network configuration on cluster 2 is completely different.

Cluster 1 had a 4x10G NIC. One port was assigned to MGT, one assigned to HB and LM. And two were teamed into a beautiful 20Gbps VM network uplink.

Cluster 2 had a 2x10G NIC split via vNICs into 3 NICs for each port. Then one vNIC from each port teamed for MGT, one for HB and LM, and one for VM network.

Hmm…so I thought maybe if we simplify this configuration things would work better. Although we lose some of the redundancy offered by teaming.

I ended up configuring the NIC as follows,

  • Port 1: 1x10G vNIC dedicated for VM network.
  • Port 2: 2x5G vNICs, one for MGT and one for HB and LM.

A few more minutes I was ready to give VMM2 a tour around its new home and she was happy, refreshing the hosts went smoothly.

Theoretically I believe the teaming of vNICs should just work. But apparently, packets were going dizzy and dedicating a port just for VMs paid well.

That’s all for today, happy head banging fellow geeks.

About the author

Walid AlMoselhy

Permanent link to this article: http://almoselhy.azurewebsites.net/2017/06/vmm-is-unable-to-refresh-hyper-v-hosts/

Leave a Reply

Your email address will not be published. Required fields are marked *