Installing KVM and Open vSwitch on Ubuntu

This is the first of a number of posts in which I’ll be discussing Ubuntu Linux, KVM, and the Open vSwitch (OVS). I hope that you find the posts helpful.

Let me start this post by providing this disclaimer: I’m still very early in the learning curve with KVM and OVS, so I can’t promise you that this post will be absolutely perfect. It is, however, a pretty decent starting point. I’ve gone through this process several times, and there is only one sticking point that I haven’t yet resolved (which I’ll describe later on).

I’m going to assume that you are reasonably familiar with Ubuntu, as that’s the platform I’m using for all my testing (specifically, Ubuntu Server 12.04 LTS with a 64-bit kernel). I’m also assuming that you are prefacing all these commands with sudo or that you’ve attained root privileges via the method of your choice.

We’ll start this process by ensuring that our Ubuntu installation is up-to-date:

# apt-get update && apt-get dist-upgrade

In my case, this brought me up to Ubuntu 12.04.1, running the 3.2.0-29 kernel.

Next, we’ll install KVM and a couple associated packages:

# apt-get install kvm libvirt-bin virtinst

This command will install KVM, libvirt, a couple of command-line utilities, and a whole bunch of other required packages.

Next, to prepare the system for the installation of OVS, we’ll remove the default libvirt bridge (named virbr0). This ensures that the bridge compatibility portion of OVS can load without any potential conflicts:

# virsh net-destroy default
# virsh net-autostart --disable default

At this point, because we aren’t using the default Linux bridge, we can remove the ebtables package using the following command. (Note: I am not 100% sure that this step is necessary, but almost all of the guides I followed included this step. If anyone has more information, please let me know in the comments.)

# aptitude purge ebtables

You’re now ready to install OVS (this should be all on one line; I’ve added a line breaks—noted by a backslash—for readability):

# apt-get install openvswitch-controller openvswitch-brcompat \
openvswitch-switch openvswitch-datapath-source

This command installs the various OVS components and—as with KVM—a large number of required packages. Depending on the speed of your Internet connection, you might want to go get a cup of coffee.

Once the OVS packages are installed, we’ll need to enable bridge compatibility mode for OVS. To enable bridge compatibility mode, edit /etc/default/openvswitch-switch and change this line:

#BRCOMPAT=no

Change it by removing the hash mark (uncommenting it) and specifying yes, like this:

BRCOMPAT=yes

At this point, you can try starting OVS with service openvswitch-switch start, but you might get an error about a module being in the wrong format or not built for your kernel. If you removed the default libvirt bridge (virbr0) using the virsh commands earlier, then you should be OK, but if you get an error about the module being in the wrong format go ahead and run this command to build and install the necessary module:

# module-assistant auto-install openvswitch-datapath

Once this process finishes, let’s go ahead and reboot. After the system reboots and comes back up, check the status of KVM and OVS with the following commands.

To check KVM, use this command:

# virsh -c qemu:///system list

This will return a list of running VMs (which is probably empty on your system).

To check OVS, first use this command to see if the kernel modules loaded properly:

# lsmod | grep brcom

This should return an entry containing brcompat_mod and openvswitch_mod. Follow that with this command:

service openvswitch-switch status

The output should show several OVS processes in the running state.

Assuming that everything is working properly, you should now be able to run ovs-vsctl show and get a very simple response that basically indicates that OVS has no configuration.

We are now nearing the end; only a couple of steps remain. The last thing we need to do is create an OVS bridge that will allow KVM guests to communicate with the outside world. This is a two-step process. The first step is to use ovs-vsctl to create a bridge and add at least one physical interface:

# ovs-vsctl add-br br0
# ovs-vsctl add-port br0 eth0

Obviously, you’ll want to substitute the correct physical interface for eth0 in the commands above. After running these commands, then running ovs-vsctl show will return something like this:

bc12c8d2-6900-42dd-9c1c-30e8ecb99a1b
    Bridge "br0"
        Port "eth0"
            Interface "eth0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "1.4.0+build0"

The final step is to edit /etc/network/interfaces so that the bridge comes up automatically. This is where I ran into problems. No matter what I tried, I could get the bridge to come up, but the physical interface attached to the bridge would not come up automatically. I could easily manually bring it up (using ifconfig eth0 up), but it wouldn’t come up automatically on boot. If anyone has any ideas, I’m open to them.

In any case, here’s the /etc/network/interfaces that I used (IP addresses and domain names have been changed to protect the innocent):

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static

# The OVS bridge interface
auto br0
iface br0 inet static
address 192.168.1.200
network 192.168.1.0
netmask 255.255.255.0
broadcast 192.168.1.255
gateway 192.168.1.1
bridge_ports eth0
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
dns-search mydomain.local
dns-nameservers 192.168.1.1 192.168.1.2

I tried several different variations for eth0, but none of them would bring the interface up automatically when I boot the system. Everything else seems to work just fine, so I’ve settled with manually bringing the interface up after a reboot until I figure out how to fix it. Suggestions are welcome.

In a future post (very soon), I’ll be talking more about OVS and what it looks like when you boot up KVM guests on OVS, so stay tuned for that.

As always, thoughts, corrections, questions, and clarifications are welcome—just speak up in the comments below. Thanks!

Tags: , , , , ,

  1. Ariel Antigua’s avatar

    I’m not yet playing with OVS but on a few systems I’m running Proxmox and the configuration for eth0 is as follow:

    iface eth0 inet manual

    auto vmbr0
    iface vmbr0 inet static
    address 172.22.114.145
    netmask 255.255.255.240
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

    Maybe the trick is putting eth0 on manual, did you try that already?
    Can you list the configuration that you make to eth0?

  2. Brian’s avatar

    Nice – I’ll be following your adventures with this one, and playing the home version of the game :-)

  3. Paul’s avatar

    You can run your script automatically by simply putting this in the “iface eth0″ section of /etc/network/interfaces:
    up /sbin/ifconfig eth0 up

  4. Sergio Guzman’s avatar

    Hello,

    I haven’t tested OVS yet, but I’ve had success with the following in /etc/network/interfaces:

    auto eth0
    iface eth0 inet manual
    up ifconfig $IFACE up

    auto br0
    iface br0 inet static
    address 192.168.100.200
    netmask 255.255.255.0
    gateway 192.168.100.1
    network 192.168.100.0
    broadcast 192.168.100.255
    bridge_ports eth0
    bridge_stp off
    bridge_maxwait 0
    bridge_fd 0
    bridge_hello 0
    up ifconfig $IFACE up

  5. Arie’s avatar

    Scott, to bring eth0 up automatically on boot, I added “address 0.0.0.0″ below the “iface eth0″ line.

  6. Arie’s avatar

    To clarify, here is my interfaces file looks like:

    # eth1 is to physical interface that attached to br0
    auto eth1
    iface eth1 inet static # static, not manual
    address 0.0.0.0

    auto br0
    iface br0 inet static
    address x.x.x.x
    netmask x.x.x.x
    gateway x.x.x.x

  7. Brad Hedlund’s avatar

    So I tried this on Ubuntu Server 12.10 … and I crashed and burned big time. The brcompat module wouldn’t load to save my life.

    But it worked like a charm on 12.04.

    Thanks Scott!

  8. slowe’s avatar

    Brad, I’ve had similar experiences with Ubuntu Server 12.10, and have not yet been able to find a workaround. As soon as I figure out how to make it work, I’ll post more information here.

  9. Umair Hoodbhoy’s avatar

    Ubuntu 12.10 runs kernel 3.5. According to http://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=FAQ;hb=HEAD that kernel version is only supported by Open vSwitch 1.9.x, which isn’t out yet. Right now the latest is Open vSwitch 1.7.1, so your best bet is Ubuntu 12.04, which has kernel version 3.2

  10. Paul C’s avatar

    For those having issues with Ubuntu 12.10, Here’s how I got this working -

    http://tech.paulcz.net/2012/12/configuring-kvm-openvswitch-for-ubuntu.html

  11. Andreas Kempf’s avatar

    Problem on

    Description: Ubuntu 12.10
    Release: 12.10

    brcompat module is the Kernel

    Kernel “Linux 3.5.0-17-generic #28″ works

  12. deepankar’s avatar

    Hi
    I am try to learn about openflow and openvswitch. The setup which I am trying to create

    Controller
    /\
    / \
    / \
    / \
    Ovs Br1———–Ovs Br2
    | |
    | |
    VM1 VM2

    The setup is on single Ubuntu host machine. The OVS Br1 and Br2 is connected using patch cable and VM1(Qemu-kvm) and VM2(Qemu-kvm) are connected to respective Bridges using tap interface created using tunctl. VM1 and VM2 are different subnet. I want ping from VM1 to VM2. How should I do it using openflow controllers.

    Thanks in Advance
    Deepankar

  13. Tom Fu’s avatar

    Hi, do you test openvswitch on ubuntu desktop 12.04, I follow your steps on ubuntu desktop 12.04, and can’t start openvswitch module. Thx!

  14. banagha’s avatar

    followed all the steps but ovswitch not running
    service openvswitch-switch status
    o/p ovsdb-server is not running
    ovs-vswitchd is not running
    ovs-brcompatd is not running

    plz help

  15. apoorvemohan’s avatar

    Hi

    This is a really usefull link. Thanks scott.
    I followed the instructions mentioned above and after the completion on the entire process I rebooted my system and ran ovs-vsctl show when the system was up. It gave me this(connection to /var/run/openvswitch/db.sock failed: No such file or directory) error that occued due to bridge-utils package that I had installed earlier.

    After googling I found this(http://comments.gmane.org/gmane.linux.network.openvswitch.general/370) link that helped me recover from the error.


    Thanks
    Apoorve

  16. Gopal’s avatar

    I’m running Ubu12.04 64 bit under vmware workstation. KVM is installed successfully and it can create VMs. Tried following this blog to install OVS. After I do:

    virsh net-autostart –disable default

    and

    aptitude purge ebtables

    I can no longer access the network. When I try to install OVS after that, the apt-get cannot connect to the network. The eth0 has no address. Now I’m stuck big time. Have no idea how to reverse the process. Tried “sudo dhclient eth0″ and “virsh net-autostart –network default” and virsh net-start default (which gives error: network is already active)… but no go. My VM has become a virtual brick. :-( Any idea what to do and how does one install OVS if you have disabled the network (I thought disabling virbr0 should not have had any effect on eth0) ?

    regards
    GA

  17. Shweta’s avatar

    Hi!

    I followed this post to install OVS on Ubuntu 12.04. ovs-brcompatd is not running. When I restarted OVS using service openvswitch-switch start, it shows

    /usr/share/openvswitch/scripts/ovs-ctl: 75: /usr/share/openvswitch/scripts/ovs-ctl: -e: not found
    * Inserting brcompat module
    * could not load brcompat module, disabling bridge compatibility
    /usr/share/openvswitch/scripts/ovs-ctl: 75: /usr/share/openvswitch/scripts/ovs-ctl: -e: not found
    * Inserting brcompat module
    * could not load brcompat module, disabling bridge compatibility
    * ovsdb-server is already running
    * ovs-vswitchd is already running

    Please help me on this.

    Regards,
    Shweta

  18. Kenneth’s avatar

    I have it working and it startup the brigde automatically. This is my network/interfaces:
    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).

    # The loopback network interface
    auto lo
    iface lo inet loopback

    # The primary network interface
    auto eth0
    iface eth0 inet manual
    up ifconfig $IFACE 0.0.0.0 up
    down ifconfig @IFACE down

    auto br0
    iface br0 inet static
    address 192.168.0.10
    netmask 255.255.255.0
    network 192.168.0.0
    broadcast 192.168.0.255
    gateway 192.168.0.1
    #bridge_ports eth0

    I hope it helped

  19. Adrian’s avatar

    Hi everybody!

    I’m trying to start openvswitch.
    When I write: service openvswitch-switch status

    it returns me that all is ok:
    ovsdb-server is running with pid 1723
    ovs-vswitchd is running with pid 1737
    ovs-brcompatd is running with pid 1740

    BUT, when I write: ovs-vsctl show
    it returns me an error:
    00004|reconnect|WARN|unix:/usr/local/var/run/openvswitch/db.sock: connection attempt failed (No such file or directory)

    I have done the bridge fact, the ebtables command and the BRCOMPAT=yes tasks.

    Why is not working? could you help me? THANKS A LOT!!

  20. slowe’s avatar

    Adrian, you might need to use sudo with that command (or run it as root). That error looks like what I see when I don’t have the right permissions.

  21. Adrián’s avatar

    @slowe Finally I could. It was not a “sudo” problem. It was a bridge problem and a task with ovsdb-server starting

  22. roy’s avatar

    Hi Adrian, Can you tell me the details about your solution? I am facing the same problem.

  23. Adrián’s avatar

    @roy hi roy!! I only run all with sudo privileges, so this it did not be the problem.
    First of running “ovs-vsctl show” or “sudo ovs-vsctl set-controller br-int tcp:192.168.1.208:6633″ for instance, I run what you have here in this link:

    https://github.com/homework/openvswitch/blob/master/INSTALL.Linux on the line “ovsdb:
    ovsdb-server –remote=punix:/usr/local/var/run/openvswitch/db.sock \……………”
    :)

  24. roy’s avatar

    Hi Adrian, Thanks for your reply. I could not solve the problem with the approach though. I changed the OVS version to 1.10 and built from source as debian package. Now it is working fine.

  25. balaji’s avatar

    kernel is :3.8.0-35-generic

    # service openvswitch-switch status
    ovsdb-server is running with pid 7001
    ovs-vswitchd is running with pid 7018
    ovs-brcompatd is not running

  26. slowe’s avatar

    Giuliano, you are correct—in more recent versions of OVS, Linux bridge compatibility is not supported. This could cause problems with other tools (older versions of LXC, for example) that don’t natively support OVS.

    Balaji, sounds like everything is working fine. As I just mentioned, Linux bridge compatibility (ovs-brcompatd) was removed/deprecated in newer versions of OVS. I suspect this is what you’re seeing here, though without knowing your version of OVS I can’t say for certain. Does ovs-vsctl work? (Try “sudo ovs-vsctl show” and see what it returns.)

  27. Milstein’s avatar

    1. My first question is : I want to know after successful installation when I restart my Ubuntu ? What I need to do restart the server and db because after restart using Mininet Sudo mn command it says the ovs-server is not started? is it what everyone need to do everytime?

    2. My Second question is how to completely remove ovs once started all its component so that Mininet can run sudo mn command and don;t use OVS as switch again.

    Thanks please help !

  28. slowe’s avatar

    Milstein, I haven’t used OVS with Mininet so I can’t really help you. Sorry!

  29. mark’s avatar

    FATAL: Module brcompat not found.
    * Inserting brcompat module
    * could not load brcompat module, disabling bridge compatibility
    FATAL: Module brcompat not found.
    * Inserting brcompat module
    * could not load brcompat module, disabling bridge compatibility
    * ovsdb-server is already running
    * ovs-vswitchd is already running

  30. slowe’s avatar

    Mark, depending on the version of OVS you’re using, the brcompat module may no longer be supported.

  31. George Jones’s avatar

    After installation


    # lsmod | grep brcom

    Does not show anything. (no entry containing brcompat_mod and
    openvswitch_mod), But


    service openvswitch-switch status

    shows


    ovsdb-server is running with pid 1679
    ovs-vswitchd is running with pid 1718

    This is on


    gmj@ubuntu:~$ dpkg -l | grep openv
    ii libopenvg1-mesa-lts-saucy 9.2.1-1ubuntu3~precise1 free implementation of the OpenVG API -- runtime
    ii openvswitch-brcompat 1.4.6-0ubuntu1.12.04.2 Open vSwitch bridge compatibility support
    ii openvswitch-common 1.4.6-0ubuntu1.12.04.2 Open vSwitch common components
    ii openvswitch-controller 1.4.6-0ubuntu1.12.04.2 Open vSwitch controller implementation
    ii openvswitch-datapath-dkms 1.4.6-0ubuntu1.12.04.2 Open vSwitch datapath module source - DKMS version
    ii openvswitch-datapath-source 1.4.6-0ubuntu1.12.04.2 Open vSwitch datapath module source - module-assistant version
    ii openvswitch-pki 1.4.6-0ubuntu1.12.04.2 Open vSwitch public key infrastructure dependency package
    ii openvswitch-switch 1.4.6-0ubuntu1.12.04.2 Open vSwitch switch implementations
    gmj@ubuntu:~$ uname -a
    Linux ubuntu 3.11.0-26-generic #45~precise1-Ubuntu SMP Tue Jul 15 04:02:35 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

    Is this expected behavior?

  32. Gilbert Standen’s avatar

    This post was really awesome and helpful in my work. I have done some work on using openvswitch with kvm and I posted that work here at my blog:
    https://sites.google.com/site/nandydandyoracle/home/kvm-openvswitch-1
    Thanks!

  33. Gerardo’s avatar

    @gopal are you using more than one NIC? I think you need two different ethernet cards (if you don’t have two physical network cards, you could probably create a virtual one). The Openvswitch FAQ says “A physical Ethernet device that is part of an Open vSwitch bridge should not have an IP address. If one does, then that IP address will not be fully functional.”

Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>