Network Overlays vs. Network Virtualization

Dan Wendlandt said something today in the NVP deep dive session (liveblog of the session here) that really crystallized something for me. I thought perhaps I might be the only one that was seeing a trend, but Dan’s comment leads me to believe there are others seeing this trend as well. Here’s the quote, taken from my liveblog of the session:

It is important to note, as Dan does, that a tunneling protocol alone is not network virtualization.

There’s a lot of buzz in the industry about network virtualization and network overlays, and often those terms are used interchangeably. People talk about the need for multi-tenancy and address space isolation, point to network overlays like VXLAN, NVGRE, and STT as the answer to all our problems, and in so doing they (inadvertently) conflate network overlays with network virtualization. Network overlays and network virtualization aren’t the same thing, and people that use them interchangeably probably don’t fully understand what’s involved.

<aside>By the way, if you’re not familiar with the idea of network overlays, I’d recommend reading this, this, and this to get you started. There’s plenty more out there, but those three articles will at least prime the pump, I think.</aside>

Network overlays are great for address space isolation (for example, isolating duplicate MAC addresses, duplicate IP addresses, or duplicate VLAN IDs). As such, network overlays can be an important part of network virtualization. You need more than a network overlay, though, to have network virtualization; you also need virtualized network services (like NAT, firewalls, ACLs, QoS, routing, and the like) and you need a control plane (else how would you coordinate the various pieces within the network virtualization solution?). The overlay protocol is just one piece of the puzzle, so using “network overlay” interchangeably with “network virtualization” is incorrect.

As always, I welcome the input of those more educated/knowledgeable than I am. If you’re a networking expert (or a virtual networking expert), feel free to speak up in the comments and correct my misunderstanding or misconceptions (please disclose vendor affiliations). I’m always open to deepening my knowledge—and helping others with their understanding and knowledge along the way.

Tags: , ,

  1. Chris Bennett’s avatar

    Nice post Scott.

    This reminds me of the ‘routing protocol’ versus ‘routed protocol’ distinction Cisco impress on CCNA candidates.

    I think the issue is “network overlay” != “overlay protocol / tunneling protocol”, so statements like:

    “It is important to note, as Dan does, that a tunneling protocol alone is not network virtualization.”
    “The overlay protocol is just one piece of the puzzle”

    are true, while statements like:

    “Network overlays and network virtualization aren’t the same thing”
    “You need more than a network overlay, though, to have network virtualization”

    are ambiguous because I think you mean protocol for every use of the term ‘network overlay’.

    Wikipedia’s first sentence suggests it’s a reasonably broad term:
    http://en.wikipedia.org/wiki/Network_overlay (redirects to Overlay Network)
    “An overlay network is a computer network which is built on the top of another network.”

    My guess is some people’s use of ‘network overlay’ is in the context of a specific protocol they are talking about / referring to, and that’s why protocol is omitted from some of the content that is published / communicated.

  2. slowe’s avatar

    Chris, thanks for taking the time to comment. One place where I disagree with you is in regard to the ambiguous statements using only the term “network overlay (or “overlay network”). I still maintain that “network overlay/overlay network != network virtualization” for all the same reasons. The mere presence of an overlay network does not equate to network virtualization, IMHO.

  3. Chris Bennett’s avatar

    “The mere presence of an overlay network does not equate to network virtualization, IMHO”

    I’m coming from a networking background on this – I would view the use of MPLS, VLANS, QinQ, L2TPv3, DMVPN, IPSEC, VRF-lite and any other protocol acronym that facilitates the creation of an an overlay/tunneled network, to all be forms of network virtualisation in and of themselves. What makes any of the above types of overlay networks not a form of network virtualisation?

  4. Jon Hudson’s avatar

    Great points. Though I must say that what I like even more than this clarity is the overall trend I am seeing of tighter definitions as we get closer and closer to what we are really after.

    Just as NfV offers a nice way to define a virtualized network function from something like openflow being a programatic approach, distinguishing the difference between network virtualization and network overlay is significant and understanding that brings a sharper view to the solution space.

    For example, network overlays have been around _forever_. Many things are considered overlays including LISP, TRILL, L2/L3VPN etc, but I would not call them network virtualization. That does not mean they can’t become part of a virtualized network, it just means on their own they are not.

    And yes, sure there is always the danger of debating what the definition of “it” is. (for example; is network virtualization different than a virtualized network =)

    However in the end posts like this help all of us clarify what exactly it is we are all so excited about, which helps other people understand it so they too can get excited.

    I think this is one of big advantages of you joining VMware, as you grok and write we all read and grok.

    Scott Lowe, the Pide Piper of Virtual Networking ;-)

  5. slowe’s avatar

    Chris, I think the answer to your question lies in the definition of network virtualization, which (paraphrased) means “a faithful and accurate reproduction of the physical network that is fully isolated and provides both location independence and physical network state independence.” Tools like MPLS, VLANs, QinQ, L2TPv3, L2VPN/L3VPN, etc., are all tools that could be used as part of a network virtualization solution. In my mind, a comparison could be made here between server and network virtualization—overlay protocols could be likened to the KVM kernel module. Both are important, but both also require an array of other components to create the full virtualization effect (in KVM’s instance, it needs QEMU, libvirt, etc.; in network virtualization, it needs virtualized network services and a control plane). Does that help at all?

    Jon, good to hear from you! Thanks for taking the time to comment. I do agree that clearer definitions help everyone. Of course, then you go and muddy the waters by mentioning NfV. :-) However, I think that you used it in the same context I would use it, so it’s all good. BTW, with regard to “virtualized network” and “network virtualization”: I would say that a virtualized network is an instance of an entity created by network virtualization, like a VM is an instance of an entity created by compute (server) virtualization.

    keep the comments coming—this is a great dialogue!

  6. Jon Hudson’s avatar

    Chris and Scott, I think you two are illustrating exactly the confusion!

    So lets take the definition that Scott posted “a faithful and accurate reproduction of the physical network that is fully isolated and provides both location independence and physical network state independence.” (which i like)

    That definition allows for both the virtualization of the traffic via protocols (what I would consider an Overlay to be) and also the virtualization of a network device or function (that NfV is).

    the challenge here is we are talking about several things in all this.

    1.) a virtual network from the view of the client or user.
    2.) a virtual network from the view of the host or provider.
    3.) a virtual instance of a networking device.

    I think the distinction may come down to the problem being solved.

    First off I agree Scott, Network Virtualization is the higher object that results in a virtualized network .

    For Overlays, I think the value being everything from scale to security to multi-tenancy. This is all from the point of view of the user on the network.

    NfV I think the value is the same as server virtualization, it is time to deployment, it’s physical device consolidation, it’s linked clones and shared resources. It’s “Right click, Router” . This is from the point of view from the admin/provider/operator.

    So you could argue that Network Virtualization is the goal and that components of network virtualization may be Overlays, may be Virtualized Network devices.

    The challenge becomes almost one of potency. For example is a drink with .002% alcohol an alcoholic drink? I think we’d all agree that 8% alcohol does make a drink. I know some folks that expect much more than that when asking for a drink.

    If you have a network made up of virtualized devices (vSwitches, vRouters, vFirewals etc) and also made up of Overlays (VLAN, MPLS, TRILL, L2/L3VPN, VxLAN, STT, NVGRE, DOVE, QinQ) what percentage of a network needs to be virtualized before it’s a Virtual Network? Remember what VLAN* stands for =)

    *might be a clue there, perhaps an overlay is a virtual LAN and a virtualized network contains many virtual LANs with routing between them?

    Does it just have to be virtualized from the point of view of the client or user? Or does it also have be virtualized from the point of view of the operator or administrator.

    To take Scott’s analogy about KVM a step deeper into the past. What bout BSD Jails? Or Solaris Containers? Were those VMs?

    The more I think about it I think we may take Scotts definition (fully isolated physical, location and state independence) and tests it against a use case.

    Can I take a stateful copy of my entire network, copy it, zip it up, send it to tech support, have them use that to recreate my exact network and debug my problem?

    So perhaps it comes down to something like this?

    A virtualized network:

    MAY include overlays
    MAY include tunnels
    MAY include virtual devices
    MAY include multitenancy
    MUST provide total physical independence
    MUST provide total location independence
    MUST provide total stateful independence

    Meaning that just because your network has overlays and virtual devices in it, does not mean it’s a virtualized network.

    If you can copy, tar, gzip and scp a stateful copy of your entire network somewhere else and recreate it exactly. Then you do.

    I think I may have just created a test that means NO ONE has a virtualized network?

    Hmm…that was not my goal…grrr….I need a drink….

  7. David Pasek’s avatar

    Scott, excellent write-up. As always. First of all I absolutely agree that good definitions, terminology, and conceptual view of particular layer is fundamental to fully understand any technology or system. Modern hardware infrastructure is complex and complexity is growing year on year. Programming has the same history. Who programs in assembler nowadays? Why we use object oriented programming more then 20 years? The answer is … to avoid complexity and have control on system behavior. In software MVC model is often use and it stands for Model-View-Controller. Model is logical representation of something we want to run in software, View is as simplified presentation to end user and controller is engine behind the scene. The same concept apply to SDI (Software Defined Infrastructure) where SDN (Software Defined Network) is another example of the same story. VMware did excellent job with infrastructure abstraction. Everything in VMware vSphere is object. Better to say managed object which has some properties and methods. So it is the model. vSphere Client or Web Client or vCLI or PowerCLI are different user interfaces into the system. So it is View. And who is Controller? Controller is vCenter because it orchestrates system behavior. vCenter controller includes prepackaged behavior (out-of-the-box) but it can be extended by custom scripts and orchestrated externally for example by vCenter Orchestrator. That’s what I really love VMware vSphere. And it is from the begining architected to purely represent hardware infrastructure in software constructs.

    Now back to Network Virtualization. In my opinion Network Overlay (for example VXLAN) is mandatory component to abstract L2 from physical switches and have it in software. Particular Network overlay protocol must be implemented in “Network Hypervisor” which is software L2 switch. But “Network Hypervisor” has to implement also other protocols and components to be classified as “Network Virtualization” and not only as another software vSwitch. What Scott already mentioned is that networking is not just L2 but also L3-7 so other network services must be available to speak about full “Network Virtualization”. Am I correct Scott?And I feel the open question in this post … who is the controller of “Network Virtualization”? :-)

  8. slowe’s avatar

    Jon, great comment. I see your point about the “how virtualized” must a network be before it is considered a virtualized network. Is it only an overlay? Is it only VLANs? Is it only virtualized network services, aka NfV? I think your “May/Must” set of conditions are actually quite good and help distill this down to the essential definition. Thanks for that!

    David, I like your idea of applying MVC to virtualization, especially network virtualization. The Controller, in your model, is the control plane that I referenced in an earlier comment—meaning that your solution MAY have overlays and it MAY have virtualized network services (using Jon’s terminology), but it MUST provide these other properties, and in order to provide those properties a control plane is needed. This is where a lot of the innovation in network virtualization is occurring (in the control plane).

  9. Donny’s avatar

    Some of this seems to be splitting hairs, but I was enthralled in the discussion none the less.

    For me, the definition is very clear. Having built a number of large compute grids and datacenters, there is a significant network design activity involved throughout the process. In the end there is a documented design and architecture from demarc to host interface.

    Using this as the baseline, when a “tenant” can design and implement a full network solution to support requirements and operations without interacting with hardware, you have a virtualized network.

    This implementation may be a simple as a subnet and router. It may be as complex as FW/IPS/Route/Packet Capture. Overlays may be the how, but the service of network transport and management is the what. Automation may coordinate physical configurations, virtual objects, and policy management.

    To sum up, a virtualized network is one which can be instantiated, operated, and removed without physical asset interaction by the network manager.

  10. Jon Hudson’s avatar

    I work in the IETF, splitting hairs is what we DO ;-)

    I like this VERY much

    “…. a virtualized network is one which can be instantiated, operated, and removed without physical asset interaction by the network manager.”

    Good stuff!

  11. David Pasek’s avatar

    Yes. I also like Donny’s simple definition of virtual network.

    “…. a virtualized network is one which can be instantiated, operated, and removed without physical asset interaction by the network manager.”

    and network manager is OK with such operations :-)

  12. jason’s avatar

    Hi Scott,

    I guess to define network virtualization, we need to define a network. Does it means that a network is not a network without QOS/NAT/etc ?

    By the same rules, what defines a virtual machine ? If I create a VM with without virtual USB, is it a VM ? What about one without a vnic ? Or one without a vhdd ? Well, if I create one with a vCPU or vRAM, I know its definitely not a VM, so that effectively define for me the minimum requirements.

    By the same token, then I would consider a overlay network as network virtualization.

  13. Iben Rodriguez’s avatar

    Hi Scott,

    After some time working with SDN more please clarify your statement:
    you also need virtualized network services (like NAT, firewalls, ACLs, QoS, routing, and the like) and you need a control plane (else how would you coordinate the various pieces within the network virtualization solution?).

    With SDN and NFV aren’t there actually 3 planes we worry about?
    1) data plane – this is where the packets go back and forth
    2) control plane – what packets can go where
    3) management plane – coordinating the components interactions with each other and with people

    I think there might be some confusion on all this. What is your perspective?