Some Insight into Open vSwitch Configuration

As you may already know, I’ve been working with Open vSwitch (OVS) for a few weeks now, trying to wrap my head around how this open source project works. One thing that I really struggled with—and still struggle with, to a certain extent—is a lack of user-friendly documentation. While there are a few posts that provide some basic instructions, I haven’t found any good articles that provide a bit more depth and more explanation. Maybe it’s just me, but I like to know why I’m typing certain commands, and how those commands work.

What is relatively well-documented are tasks like creating a bridge, adding ports to a bridge, or creating a bond. These tasks use commands like ovs-vsctl add-br or ovs-vsctl add-port. These commands are pretty easy to understand, reasonably well-documented in the help screens and the manpages, and provide decent error messages back to the user if the syntax is wrong. What isn’t quite so well-documented are tasks like VLANs or LACP, and that’s where I was struggling.

(OK, rant over.)

Anyway, I think that I’ve finally made some progress, and I wanted to share what I’ve found with you. (At some point in the near future, I intend to post some “intro to OVS basics” posts to help others that might be learning OVS for the first time like me.)

So, let’s say that you want to set the VLANs that are allowed across an OVS port. By default, all OVS ports are VLAN trunks, but based on my experience you still need to set the VLANs that are allowed across the trunk before they actually act like trunks. If you’re familiar with Cisco switches, think of this process as using the switchport trunk allowed vlans command.

To set the VLAN trunks for a given port, use this command:

ovs-vsctl set port <port name> trunks=<list of VLAN IDs>

Obviously, you’ll want to substitute the correct port name and VLAN IDs in that command. If you’re unsure of the port name to use, use ovs-vsctl show to review the OVS configuration and determine which port(s) to configure.

If you want to configure an OVS port as a VLAN access port, use this command:

ovs-vsctl set port <port name> tag=<VLAN ID>

Again, it should go without stating, but you’ll want to substitute the correct values for your environment in the command above.

Here’s one more example before I provide some explanation. Let’s say that you have a bond (a NIC team or a link aggregate) and you want to enable LACP on that bond. You’d run this command:

ovs-vsctl set port <port name> lacp=active

The “port name” in this case would be something like bond0, which is a bond you’ve created using ovs-vsctl add-bond command. Your physical switch must be properly configured as well in order to support LACP on the appropriate physical switch ports.

What are these commands doing, exactly? This is the part that I couldn’t find documented (well) anywhere. I saw lots of references to ovs-vsctl parameters like set interface or set port, but no clear explanation of what these commands were doing, or why. Almost every single example I saw also used these commands during the process of creating a bridge, bond, port, or interface (not afterward). What if you needed to modify the values after the object is created? Do you have to delete the object and recreate it?

Oddly enough, it was this post (which has nothing to do with VLANs, trunks, or LACP, but instead focuses on sFlow) that sparked my understanding. I was reading the post on how to configure OVS for sFlow while also reviewing the manpage for ovs-vsctl when I had the epiphany: these objects (bridge, port, bond, interface) are tables in the OVSDB, so you need to use the OVSDB-related parameters for ovs-vsctl in order to modify their properties.

Looking at the ovs-vsctl manpage (or the --help screen), you can see that there are several DB-related commands. Here’s the generic form:

ovs-vsctl <command> <table name> <record name> <setting=value>

In this generic command, <command> would be something like set, get, or list, and the <table name> would be replaced by a specific OVSDB table. For example, one such table is port. Let’s plug a specific command and a specific table into the generic form:

ovs-vsctl set port <record name> <setting=value>

(Did something just click with you?)

We could continue plugging specific items into the generic form to arrive at a command like this:

ovs-vsctl set port bond0 trunks=10,20,30,40,50

The trick, of course, is knowing what values to substitute into the command to manipulate the OVS database in the right way. Fortunately, there are a couple of commands that can help.

To see all the OVS bridges and their settings, use this command:

ovs-vsctl list bridge

To see all the OVS ports and their settings, use this command:

ovs-vsctl list port

Finally, to see all the OVS interfaces and their settings, use this command:

ovs-vsctl list interface

You can add a specific record to the above commands; for example, to see the settings for a port named bond0:

ovs-vsctl list port bond0

This will show you the settings that are available for that particular record; you can then use ovs-vsctl set as described earlier to set the value for a setting. This is how you configure VLAN trunks (by setting the value of the trunks setting for a particular port) or enable LACP (by setting the value of the LACP setting for a particular port). These commands can be run when a record is created, like this:

ovs-vsctl add-bond br0 bond0 eth0 eth1 lacp=active trunks=10,11,12

Or you can run the commands after the record/object is created, like this:

ovs-vsctl set port bond0 lacp=active trunks=10,11,12

Hopefully, this additional information and insight—which seems so simple now that I understand it—will prove helpful to others.

If there are errors or inaccuracies in my information, please speak up in the comments and correct me. This will also help other readers. All courteous comments are welcome!

Tags: , , ,

  1. Peter Phaal’s avatar

    Great article. It should be very helpful to anyone familiarizing themselves with the Open vSwitch – I struggled for a day to figure out the LAG configuration.

    I noticed your reference to my article on configuring sFlow. The Open vSwitch supports both sFlow and NetFlow and anyone looking to monitor virtual switch performance might be interested in a more recent article describing how to configure both methods of monitoring and comparing the results you can expect:

    http://blog.sflow.com/2011/10/comparing-sflow-and-netflow-in-vswitch.html

  2. Farrukh’s avatar

    I have been trying to understand OVS myself and I couldn’t agree more with you.

    Your article is very very helpful. Thank you!

  3. Brent Salisbury’s avatar

    Hi Scott, Great post! You will save have saved me hours of digging around for proper tagging of a LAG group. Thanks for taking the time to share!
    -Brent

  4. Giancarlo’s avatar

    Great practice manual for VLANs ad OVS, thanks a lot!!!

  5. Ravindra’s avatar

    Hello Scott, Thanks for the information… it is really well explained.

  6. Matt’s avatar

    In response to “What are these commands doing, exactly? This is the part that I couldn’t find documented (well) anywhere. I saw lots of references to ovs-vsctl parameters like set interface or set port, but no clear explanation of what these commands were doing, or why.”

    I was stymied by the same problem, in part due to the poor manpage formatting on the Open vSwitch website. The ovs-vswitchd.conf.db(5) manpage is what you’re looking for — except you’ll want to look at it using good old-fashioned “man” rather than online.

  7. Prashant’s avatar

    Hello Scott Its really greate explanation of OVS configuration sepcially the about command explanation..and
    I have on que. the physical port which we are going to add under bridge
    dosen’t have the ip but some can we create vlan under that port with the ip or need some explnation about that…???

  8. slowe’s avatar

    All, thank you for the feedback. I’m glad to hear that this is useful.

    Matt, thanks—I’ll give that a try, and see if its helpful.

    Prashant, have a look at this post (http://blog.scottlowe.org/2012/10/31/layer-3-routing-with-open-vswitch/); it has the information you need on VLAN interfaces with OVS.

  9. Prashant’s avatar

    thank you!!

  10. Fred Hsu’s avatar

    I just got around to reading this, and it totally clicked! Thanks for the wonderful explanation! It makes total sense to think of everything as DB operations instead of “normal” switch CLI commands.

  11. Ashraf Khalid’s avatar

    Hi,

    I had this requirement which I fail to translate using openvswitch. Kindly help.

    eth2 —— —— bond0.609——-br-ext (10.4.129.10)
    |——bond0—–|
    eth6——- ——bond0.555——–br-int (172.17.100.10)

    I need to create:

    1. bond0 (active-backup using eth2,eth6; and trunks=555,609),

    2. VLAN interfaces on top of the bond0 (bond0.609 for VLAN 609, and bond.55 for VLAN 609)

    3. VLAN bridges (br-ext and br-int)

    4. Assign the relevant IP address for the VLAN to these bridge

    I could manage the same using Linux native bridge and bonding, but unable to successfully translate the same using openvswitch.

    I do not want to use native linux bonding, and want to use ovs-vsctl as much as possible to configure the physical interfaces, bond or bridge.

    Kindly help!

    Thank you..

  12. Sebastian’s avatar

    Hi Scott,

    Thank you so much to share your learning on OVS, and to touch up on topics which are less documented.

    I am a newbee to OVS, and had some queries..

    1. Do we need to create all the ports/bonds/bridge (fake or real) under the main bridge?

    2. How to create VLAN interfaces on top of a bond device (OVSBond)?

    3. Is there an alternate to Fake bridges for connecting multiple virtual machines to the same vlan? I mean like a VLAN interface interface, and then create bridge on top of each such vlan interfaces?

    4. Is there any article that describes the differences (conceptual) between Linux native bridging and bridging using OVS?

    5. I have a requirement which I want to address:

    i) I want to create OVSBond (bond0) using physical interfaces (eth1, eth2)

    ii) Then create OVSPort (bond0.100) with VLAN TAG as 100, and OVSPort (bond0.200) with VLAN TAG as 200 respectively on top of the OVSBond (bond0).

    iii) Now I need to create bridge on top of each of these VLAN interfaces (bond0.100 & bond0.200) as br-100 and br-200 respectively.

    iv) Assign IP Address to br-100 and br-200 (Both needs access to the external world, and I want to assign IP address to test the connection)

    v) Finally I intent to connect my Virtual Machines which need connectivity to the respective VLAN to ONLY talk to other virtual machines under the same network. That is, VMs using interfaces on br-100 should not communicate with br-200, and viceversa.

    Hope you can help me regarding this…

    kind regards,
    Sebastian

  13. brandon’s avatar

    Thank you for explaining this!!

  14. Andrew Bruce’s avatar

    Hey Scott! Longtime fan and you got me through the VCP5 – thanks for your hard work. Quick question on OVS: can I bridge one OVS to another? Is that a horrible idea to begin with?

    Thanks – keep up the great work, Sir!
    Andy

  15. slowe’s avatar

    Hi Andrew, thanks for the feedback, I really appreciate it! You can definitely link OVS bridges together—see my blog post(s) on OVS patch ports.

Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>