VLAN Trunking to Guest Domains with Open vSwitch

In other articles, I’ve talked about how to use Open vSwitch (OVS) with VLANs to place guest domains (VMs) into a particular VLAN. In this article, I want to show you how to pass VLAN tags all the way into the guest domain—in other words, how to do VLAN trunking to guest domains using OVS. To do this, we’re going to leverage the OVS-libvirt integration I referenced in this post on using VLANs with OVS and libvirt.

For this to work, you must have an operating system in the guest domain that is capable of recognizing and using the VLAN tags that are being passed to it by OVS. In this article, I’ll use Ubuntu 12.04 as the OS in the guest domain. For other operating systems, the commands and/or procedures to configure VLAN support appropriately will probably differ, so keep that in mind.

There are two parts to making this work:

  1. Configuring OVS (manually or via libvirt) to pass VLAN tags to the guest OS.
  2. Configuring the guest domain’s installed OS to take advantage of the VLAN tags being passed up by OVS.

Let’s look at each of these parts separately. We’ll start with configuring OVS, either manually or via libvirt, to pass the VLAN tags up to the guest domain.

Configuring OVS to Pass VLAN Tags to the Guest Domain

There are two ways to accomplish this: you can do it manually, or you can do it via OVS integration with recent builds of libvirt.

Manually Configuring OVS

To configure OVS manually, you would need to:

  1. Identify which vnet port you want to configure for VLAN trunking
  2. Configure the vnet port to trunk the VLANs.

To identify which vnet port needs to be modified, you’ll want to figure out the guest domain interface(s) that is/are connected to the vnet port. You can do this by using this command (substitute the desired vnet port name in place of vnet0 in the following command):

ovs-vsctl list interface vnet0

In the output of the command, look for the external_ids line; it will contain an entry called “attached-mac”, and that represents the MAC address of the interface in the guest domain OS attached to this particular vnet port. You can compare this to the output of ip addr list or ifconfig -a in Ubuntu to find a matching MAC address in the guest domain. Correlating the two values allows you to determine which guest domain is attached to which vnet port, and then you can modify the correct vnet port appropriately.

You’d modify the vnet port using this command:

ovs-vsctl set port vnet0 trunks=20,30,40

You’d want to substitute the appropriate values for vnet0 and the VLAN IDs that you want passed up to the guest domain. Once you’ve made the change, you can verify the changes using this command (replacing vnet0 with the correct port):

ovs-vsctl list port vnet0

Note that if you want the guest domain to receive both untagged (native VLAN) traffic as well as tagged (trunked) traffic, there is an additional setting you must set:

ovs-vsctl set port vnet0 vlan_mode=native-untagged

With this setting in place, the OS installed into the guest domain will be able to communicate over the untagged (native) VLAN as well as using VLAN tags.

Using libvirt Integration

If the manual method of configuring OVS seems a bit cumbersome, using the libvirt integration makes it much easier.

Basically, you’ll follow the configuration outlined in this blog post to create a libvirt network that corresponds to an OVS bridge. Here’s an example of the XML code to accomplish this task:

Of particular interest for what we’re trying to accomplish here is the very last section, the portgroup named “vlan-all.” Note that for this specific portgroup, the vlan element has a property that specifies it is a trunk, and then there are multiple tag elements that list each VLAN ID that will be trunked across this network into the guest domain.

Using this configuration, when we create the guest domain and specify that it is attached to the network named “vlan-all” (matching the portgroup in the libvirt network definition), libvirt will automatically configure OVS appropriately (it will set the trunks value for that domain’s OVS port).

However, it will not configure the OVS port to allow untagged traffic as well (only tagged traffic will be passed). If you want the guest domain to receive untagged traffic also, you must set the vlan_mode value manually as outlined above.

Configuring the Guest Domain to Use VLAN Tags

Once you’ve followed the steps outlined above and have OVS configured correctly, then you’re ready to configure the OS in the guest domain. Keep in mind that I’m using Ubuntu 12.04 in this post, but you’re welcome to use any operating system that supports VLAN tags.

Assuming that eth0 is the interface in the guest domain that is receiving tagged traffic from OVS, this snippet in /etc/network/interfaces will create and configure a VLAN interface:

Technically, the “raw-vlan-device” line isn’t needed because the parent device name is in the name of the VLAN device, but I like to include it for completeness and ease of debugging. (Your mileage may vary, of course.) The number on the end of the eth0 (for example, eth0.20) corresponds to the VLAN ID (VLAN 20, in this case) being passed up by OVS.

You can repeat this configuration for multiple VLAN interfaces.

Use Case

I’ll have to admit that I can’t immediately think of some useful use cases for this sort of configuration. At first glance, you might think that it would be useful in situations where you need logical separation, but I think there are better ways than VLANs to accomplish this task (and those ways are probably simpler). I primarily set out to document this in order to better solidify my knowledge of how OVS works and is configured. However, I’d be happy to hear from others on what they think might be interesting or useful use cases for this sort of configuration. Feel free to add your thoughts in the comments below. Courteous comments are always welcome!

Tags: , , , ,

  1. Derwin Warren’s avatar

    Hi Scott, I notice that there is a lot of hard-coding in your XML in reference to port-group names. Does this XML support substitution parameters? For instance, if a customer has a port-group names that do not follow the traditional “vlan-{vlan-id}” syntax. For example, say I want my port-groups to be “vlan-101-{ip-address/subnet}”? Can the XML provide a substitution parameter to pull this information from the ipconfig or vnet settings? The point being to not have to manually modify or hard-code the XML in case changes occur in the future which make us have to touch this file.

    Great post.

  2. Will Dennis’s avatar

    Hi Scott,

    As far as use cases, we use(d) VLANs trunked to a Linux server for applications such as NIS and CUPS that use broadcast-based discovery on a LAN. (CUPS now uses mDNS for this, and NIS is basically historical at this point.) There may be some facility for interconnecting KVM/Xen hosts as well, that have VMs on multiple VLANs.

    Thanks for another great article!
    Will

  3. slowe’s avatar

    Derwin, to my knowledge there is no support for substitution parameters in the libvirt XML syntax.

  4. slowe’s avatar

    Will, thanks for your comment. I’m thinking another potential use case might be VLAN interfaces going into a Linux-based VM for firewalling, NAT, etc.

  5. b0b0lf0’s avatar

    hi.

    When you configure the guest domain to use vlan tags you use eth0, i assume that eth0 is part of ovsbr0?, or im wrong?.

  6. Manu’s avatar

    Hi Scott, I am new to this openvswitch. And trying to configure Hypervisor to hypervisor tunnel with NVP controller/service node. Tunnels comes up between hypervisors and service node. Also, I could successfully ping hypervisor1-VM to hypervisor-2-VM.
    Now i am trying to enable VLAN Tagging. As soon as i enable VLAN tagging on VMs, traffic stops flowing.
    I feel the hypervisor is not sending the packets out of the external bridge interface. Is there any configuration i am missing.
    I followed all your openvswitch blogs and tried.

    Sample config:
    root@ubuntu124-68181:~# ovs-vsctl show
    760e9a7f-2c47-4180-a990-34cf5b2e6b42
    Manager “ssl:192.168.0.13:6632″
    is_connected: true
    Bridge “br1″
    Port “eth5″
    Interface “eth5″
    Port “br1″
    trunks: [0, 2, 10]
    Interface “br1″
    type: internal
    Bridge “br0″
    Port “br0″
    Interface “br0″
    type: internal
    Port “eth0″
    Interface “eth0″
    Bridge br-int
    Controller “ssl:192.168.0.13:6633″
    is_connected: true
    Controller “unix:ovs-l3d.mgmt”
    is_connected: true
    fail_mode: secure
    Port “vnet0″
    Interface “vnet0″
    Port br-int
    trunks: [2]
    Interface br-int
    type: internal
    Port “vnet1″
    tag: 2
    trunks: [2]
    Interface “vnet1″
    Port “vxlan83951892″
    Interface “vxlan83951892″
    type: vxlan
    options: {key=flow, remote_ip=”5.1.1.20″, tos=inherit}
    Port “vxlan83951922″
    Interface “vxlan83951922″
    type: vxlan
    options: {key=flow, remote_ip=”5.1.1.50″, tos=inherit}
    ovs_version: “1.11.0.26211″
    root@ubuntu124-68181:~#

  7. thota’s avatar

    Hi Scott, Thanks for the wonderful article. I have got a similar requirement, where in I have to trunk more than one VLAN to the guest domain. The host has one 10Gbps NIC (named eth11 on my machine) card and the tagged data comes from the external switch to the host on this NIC. Now, I have to switch this traffic to the the guest VM.

    I want to trunk traffic with vlan tags 200 and 201 to the guest. Below is my environment.
    - Bridge with the name br-int
    - Virtual machine attached the bridge using tap interface (tapa2d511ef-37), which is added to bridge.
    - Added eth11 port to the bridge br-int

    output of “ovs-vsctl show”
    [root@compute1 ~]# ovs-vsctl show
    0bfed3e4-5f13-4726-a6a5-073fd47f1500
    Bridge br-int
    Port “tapa2d511ef-37″
    trunks: [200, 201]
    Interface “tapa2d511ef-37″
    Port br-int
    Interface br-int
    type: internal
    Port “eth11″
    trunks: [200, 201]
    Interface “eth11″
    ovs_version: “1.11.0″

    - Lastly created vlan sub interfaces on the host (eth11.200 and eth11.201) and also on the virtual machine (eth0.200 and eth0.201).
    - Assigned ip addresses to eth11.200 and eth0.200 with same subnet.
    I’m not able to ping eth0.200 from the host. What could be the mistake in my configuration.

    However, I could trunk to virtual machine using another “internal” interface. I created another port on the bridge, say “vlanx” using command
    ovs-vsctl add-port br-int vlanx — set interface vlanx type=internal
    ovs-vsctl set port vlanx trunks=200,201 vlan_mode=native-untagged

    and then created two vlan subif for vlanx, vlanx.200 and vlanx.201.
    With this configuration I could ping eth0.200 and eth0.201 (vlanx.200 has an IP with subnet similar to eth0.200)

    Lastly, If I assign IP address to br-int, I could ping eth0 interface in the virtual machine (untagged packets)

    I don’t understand the reason for this behavior. Could you please explain this. Thanks in advance.

Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>