Stretched Clusters, Distance vMotion and L2 Adjacency

Beth Pariseau recently published an article discussing the practical value of long-distance vMotion, partially in response to EMC’s announcement of VPLEX Geo at EMC World 2011. In that article, Beth quotes some text from a tweet I posted as well as some text from Chad Sakac’s recent post on VPLEX Geo. However, there are a couple inaccuracies from Beth’s article that I really feel need to be cleared up:

  1. Long-distance vMotion and stretched clusters are not the same thing.
  2. L2 adjacency for virtual machines is not the same as L2 adjacency for the vMotion interfaces.

Regarding point #1, in her article, Beth implies that Chad’s statement “Stretched vSphere clusters over [long] distances are, as of right now, still not supported” is a statement that long-distance vMotion is not supported. Long-distance vMotion, over distances with latencies of less than 5 ms round trip time (RTT), is fully supported. What’s not supported is a stretched cluster, which is not a prerequisite for long-distance vMotion (as I pointed out in the stretched clusters presentation Beth also referenced). If you want to do long-distance vMotion, you don’t need to set up a stretched cluster, so statements of support for stretched clusters cannot be applied as statements of support for long-distance vMotion. Let’s not confuse the two, as they are separate and distinct.

Regarding point #2, the L2 adjacency for virtual machines (VMs) is absolutely necessary for distance vMotion. As I explained here, it is possible to use a Layer 3 protocol to handle the actual VMkernel (vMotion) traffic, but the VMs themselves still require Layer 2 adjacency. If you don’t maintain a single Layer 2 domain for the VMs, then VMs would have to change their IP addresses on a live migration. That’s REALLY BAD and it completely breaks live migration. Once again, there is a very separate and distinct behavior that you’re trying to modify with large L2 domains.

Am I off? Speak your mind in the comments below.

Tags: , , , , ,

11 comments

  1. Beth Pariseau’s avatar

    Hi Scott,

    I appreciate your feedback. The article does state that distance vMotion under 100 km is supported with VPlex Local and Metro. However, the focus of my piece is on longer-distance vMotion with VPlex Geo.

    Regarding your second point, I would greatly appreciate some further clarification: how does one achieve Layer 2 adjacency over long distance without stretched clustering? Any direction you could provide on this point would be tremendously helpful.

    Thanks again,

    Beth

  2. Beth Pariseau’s avatar

    Just to let you know, the post has been updated. Thanks for your clarifications, Scott!

  3. slowe’s avatar

    Thanks Beth! I appreciate the dialog.

  4. Jivetolkein’s avatar

    Just to round it up – following the link about L2 adjacency for vMotion traffic, you’ve made the point that vMotion traffic CAN be routed, but it isn’t supported – as this is last years article, and chances are I’ve missed something, is it supported yet? Any ‘we’re looking at it’ whispers that are public?

    I can’t say that I’ve seen a need too yet, as the L2 adjacency requirement for the VMs themselves usually means I can get the vMotion network from the network brainiacs …

  5. slowe’s avatar

    Jivetolkien, at this point I don’t know that the vMotion L2 requirement (for the VMkernel ports, not for VM traffic) is even still in the support statement.

  6. Duncan’s avatar

    I’ve asked around for this a while back (http://www.yellow-bricks.com/2010/08/19/layer-2-adjacency-for-vmotion-vmkernel/). It has never been officially tested and as such there is no official support statement. I have found that it works, as one would expect, but again no official statement.

  7. Duncan’s avatar

    By the way, can we stop quoting from Twitter? This will only lead to misinformation and people locking their twitter streams and blocking specific users.

  8. Jivetolkein’s avatar

    Thaks guys, won’t be hanging my hat on such a design then till I hear different then.

  9. Eli Ben-Shoshan’s avatar

    You can achieve L2 adjacency for the VM networks with OTV possibly. We are looking at doing something like this in the near future.

  10. slowe’s avatar

    Eli, you’re right that you can use OTV to establish L2 adjacency and I’d love to hear how your implementation turns out. Thanks for commenting!

  11. Eyal Azran’s avatar

    You can still use L3 DCI (Data Center Interconnect) and use vmotion using LAM, which stands for Local Area Mobility (old cisco feature). When the vm migrate to the remote site, the local router recognize the vm has different IP subnet than configured on the router and generates host specific static rouete for the vm. The route can then be redistributed into OSPF or other dynamic routing protocols which informs the whole network about the change.

Comments are now closed.