Xeon 5500 CPUs and EVC in vSphere

In early April when I traveled to Houston to meet with HP about the ProLiant G6 servers—a trip which resulted in this blog post—I got into a discussion with representatives from HP and Intel regarding the Xeon 5500 CPUs and Enhanced VMotion Compatibility (EVC).

As you probably already know, EVC builds upon hardware extensions provided by AMD and Intel (AMD in the AMD-V extensions, Intel in FlexMigration) that allows a CPU to “alter” its personality or identification so as to allow live migration (i.e., VMotion) between CPUs that would otherwise be incompatible. Without these hardware extensions, you would have had to resort to unsupported CPU masking, which I’ve also discussed (here’s a summary of articles I’ve written on the topic). With the hardware extensions and EVC, the hypervisor can now take over the process of creating custom CPU masks that take all CPUs in the cluster down to the “lowest common denominator” (i.e., EVC masks off all features except those that are common to all CPUs in the cluster).

Ed Haletky provides a good overview of VMware EVC, in case you need more information.

Anyway, that’s enough background information. When I was in Houston, HP and Intel were discussing all the great new features that were available in the Xeon 5500. Knowing that introducing new CPU features generally meant introducing VMotion boundaries (like I describe here), I asked this question: “What happens to all these new features when you put a Xeon 5500-based server into a cluster with older servers and use EVC to guarantee VMotion compatibility?” My thought was that EVC would downgrade the Xeon 5500 by masking new features, thus negating many of the performance benefits offered by the Xeon 5500.

Obviously, the architectural changes introduced by the Xeon 5500 (specifically, Quick Path Interconnect) wouldn’t be affected, but what about Extended Page Tables (EPT)? Or VMDq? Or FlexPriority?

There were folks there from HP and Intel, and they were all stumped. They had to call in a few experts, but finally we got an answer from an Intel engineer on the VMware Alliance Team who was able to provide more information:

… The only features that would be disabled are some new instructions that are accessible to end-user applications living in Ring 3. For NHM-EP those would be the SSE4.2 and SSE4.1 instructions… No other features would be compromised. All the features of NHM-EP that provide performance and are under the direct control of the OS or hypervisor in Ring 0 will be fully usable: EPT, HT, NuMA, FlexPriority, VMDq, VT-x, VT-d, etc.

So there you have it: you can mix Xeon 5500-based servers in with older servers running previous generations of Xeon CPUs without sacrificing the performance benefits gained from the Xeon 5500.

Tags: , , , , ,

13 comments

  1. joe’s avatar

    Good info to know! Building clusters around CPUs is very resource consuming. Any plans to implement a lab and verify the performance assertions by Intel?

  2. Nicholas Weaver’s avatar

    I had this same question and I am very glad you found out.

  3. slowe’s avatar

    I have a vendor that is supposed to be sending me some servers with Xeon 5500 CPUs; if that actually happens, then I may be able to verify it myself. I can’t make any guarantees.

  4. Max’s avatar

    Very interesting and encouraging.
    I would really, really like to hear AMD’s answer to the equivalent question.

  5. Fletch’s avatar

    Does the same apply for the Dell R900′s currently selling with the (6 core) Xeon 73xx and 74xx series?
    Can I count on EVC not dumbing these CPUs down to much to make vMotion compatible with the R900′s we bought almost 2 years ago?

    Which of these SSE4.2, SSE4.1 Ring 3 instructions might my VMs miss the most?

    http://en.wikipedia.org/wiki/SSE4

    thanks

  6. Alvaro Munoz’s avatar

    Dear friends…

    Here is a Proof of Concept by Intel that could be useful for all of you about this topic.

    http://www.intel.co.jp/in/business/pdfs/VirtualizationXeon5500.pdf

    Regards!

  7. Eric Daly’s avatar

    I have heard conflicting reports on some forums that new Intel 5500 will not work with EVC Vmotion on 5300 (5335, 5345). Anyone have EVC on these with ESX 3.5 U3?

  8. slowe’s avatar

    Eric Daly,

    EVC with the Xeon 5500 isn’t supported with ESX/ESXi 3.x. You’ll have to upgrade to vSphere in order to get EVC support for your Xeon 5500-based servers.

  9. Itzik’s avatar

    Hi Scott,
    was it ever verified

  10. Matt Mancini aka @vmexplorer’s avatar

    I recently tested EVC with 53xx to 55xx and it worked without issue under ESXi 4.1 – more information here.. http://vmexplorer.com/2011/01/05/test-lab-%e2%80%93-the-plan-and-layout-with-xsigo-juniper-iomega-vmware-and-hpdell-servers/

    I plan to post the final results Day 6 wich should include EVC notes…

  11. jason burroughs’s avatar

    what about 5570′s and 5670′s? we have 8 5570′s and just bought 8 5670′s ( these are 8 Ciscso blades of each) and are worried about losing features of the new ones).

  12. slowe’s avatar

    Itzik, I personally haven’t verified the results, nor am I sure there is an easy way to do so.

    Matt, thanks for posting the link!

    Jason, you can most certainly use EVC to ease compatibility between 5500 and 5600. I would imagine that you will see similar results here—new Ring 3 instructions will be lost, but core functionality will remain the same. Good luck, and let us know what happens!

  13. scott’s avatar

    Many Thanks for the post on vsphere.

    Thanks
    http://www.vmexplore.com

Comments are now closed.