Virtualization

You are currently browsing articles tagged Virtualization.

One of the cool things about libvirt is the ability to work with multiple hypervisors and virtualization technologies, including Linux containers using LXC. In this post, I’m going to show you how to use libvirt with LXC, including leveraging libvirt to help automate attaching containers to Open vSwitch (OVS).

If you aren’t familiar with Linux containers and LXC, I invite you to have a look at my introductory post on Linux containers and LXC. It should give you enough background to make this post make sense.

To use libvirt with an LXC container, there are a couple of basic steps:

  1. Create the container using standard LXC user-space tools.
  2. Create a libvirt XML definition for the container.
  3. Define the libvirt container domain.
  4. Start the libvirt container domain.

The first part, creating the container, is pretty straightforward:

lxc-create -t ubuntu -n cn-02

This creates a container using the Ubuntu template and calls it cn–01. As you may recall from my introductory LXC post, this creates the container’s configuration and root filesystem in /var/lib/lxc by default. (I’m assuming you are using Ubuntu 12.04 LTS, as I am.)

Once you have the container created, you next need to get it into libvirt. Libvirt uses a standard XML-based format for defining VMs, containers, networks, etc. At first, I thought this might be the most difficult section, but thanks to this page I was able to create a template XML configuration.

Here’s the template I was able to create:

(If you can’t see the embedded code above, please click here.)

Simply take this XML template and save it as something like lxc-template.xml or similar. Then, after you’ve created your container using lxc-create as above, you can easily take this template and turn it into a specific container configuration with only one command. For example, suppose you created a container named “cn–02″ (as I did with the command I showed earlier). If you wanted to customize the XML template, just use this simple Unix/Linux command:

sed 's/REPLACE/cn-02/g' lxc-template.xml > cn-02.xml

Once you have a container-specific libvirt XML configuration, then defining it in libvirt is super-easy:

virsh -c lxc:// define cn-02.xml

Then start the container:

virsh -c lxc:// start cn-02

And connect to the container’s console:

virsh -c lxc:// console cn-02

When you’re done with the container’s console, press Ctrl-] (that’s Control and right bracket at the same time); that will return you to your host.

Pretty handy, eh? Further, since you’re now controlling your containers via libvirt, you can leverage libvirt’s networking functionality as well—which means that you can easily create libvirt virtual networks backed by OVS and automatically attach containers to OVS for advanced networking configurations. You only need to create an OVS-backed virtual network like I describe in this post on VLANs with OVS and libvirt.

I still need to do some additional investigation and testing to see how the networking configuration in the container’s config file interacts with the networking configuration in the libvirt XML file. For example, how do you define multiple network interfaces? Can you control the name of the veth pairs that show up in the host? I don’t have any answers for these questions (yet). If you know the answers, feel free to speak up in the comments!

All courteous feedback and interaction is welcome, so I invite you to start (or join) the discussion via the comments below.

Tags: , , , , , ,

This post will show you how to use Open vSwitch (OVS) to connect LXC containers on different hosts together using GRE tunnels. This process is quite similar to using OVS and GRE tunnels to connect VMs, and even has some similarity to connecting network namespaces over a GRE tunnel.

If you’re not familiar with LXC, my introductory post on LXC might be useful. I also found this post on networking with LXC to be extremely helpful, as was this overview of LXC on Ubuntu 12.04. As you might guess, I used Ubuntu 12.04.3 LTS for my testing; note that if you are using a different distribution, the specific commands and/or file locations may differ.

At a high level, the process for creating this configuration looks like this:

  • Create your containers using virtual Ethernet (veth) networking.
  • Create a OVS bridge that does not contain any physical interfaces and attach a GRE interface configured to connect to a remote container host.
  • Attach the containers’ veth interfaces to this new OVS bridge.

Let’s take a look at each of these steps in a bit more detail.

Create the Containers

My LXC introduction post covers this in a fair amount of detail, but I’ll walk you through the steps again just for the sake of completeness. Additionally, I wanted to use containers with two separate interfaces: a “public” interface that could reach the outside world, and a “private” interface for inter-container communications.

To create the containers, you can simply use lxc-create to create the basic container. For example, if you are also using Ubuntu 12.04 64-bit for your testing, then creating a container with the same release and architecture would require only this command (note you might need to prepend sudo to this command, depending on your specific configuration):

lxc-create -t ubuntu -n <container name>

Once the base container is created, you can then edit the container configuration (which is found at /var/lib/lxc/container name/config by default) to add the second interface. You’d add this text to the configuration:

lxc.network.type = veth
lxc.network.flags = up
lxc.network.veth.pair = c1eth1
lxc.network.ipv4 = 10.10.10.10/24

In my specific testing, I left the container’s first network interface attached to the default lxcbr0 Linux bridge, which provides connectivity to external networks via NAT. If you wanted bridged connectivity, you’d need to set that up separately. Also, feel free to substitute a different name for c1eth1 above; this just provides a user-recognizable name for the host-facing side of the veth pair that provides connectivity for the container.

After you’ve created and configured the container(s) appropriately, start the container with lxc-start -n <container name> to ensure that it boots successfully and without any unexpected errors. Then shut it down and start it again with the console detached, like so:

lxc-start -d -n <container name>

Now that the container is up and running, you’re ready to configure OVS.

Create and Configure the OVS Bridge

If you’re not familiar with connecting things with GRE tunnels using OVS, I suggest you take a look here and here. You will probably also find this post on examining OVS traffic patterns to be helpful in understanding why you will configure OVS in this particular way.

Create the OVS bridge you’ll use for inter-container connectivity using ovs-vsctl, like this (feel free to substitute whatever name you want for “br-int” in the command below):

ovs-vsctl add-br br-int

Add a GRE interface to this bridge:

ovs-vsctl add-port br-int gre0

Then configure the GRE interface appopriately:

set interface gre0 type=gre options:remote_ip=192.168.1.100

The IP address specified above should point to a reachable interface on the remote host where the other container(s) is/are running. Also, you are welcome to use a different name for the port and interface than what I used (I used gre0), but be sure the port and interface match (as far as I know you can’t use one name for the port and a different name for the interface).

If you are connecting only two hosts with containers, then you’ll repeat this configuration on the second host, except that its GRE tunnel will point back to the first host (the first host specifies the IP address of the second host, the second host specifies the IP address of the first host).

If you are connecting more than two hosts, you’ll either want to a) manually build a non-looping topology of GRE tunnels, or b) build a full mesh of GRE tunnels and enable STP, as outlined here.

Once you’ve created the OVS bridge and the GRE interface, you’re ready for the final step: connecting the containers to OVS.

Attach the Containers to OVS

The final step is to connect the container’s veth interface to OVS. If you’re using an early enough version of OVS that still contains the Linux bridge compatibility code, then you can just specify the name of the OVS bridge with the lxc.network.link directive in the container’s configuration file. Newer versions of OVS, however, lack the Linux bridge compatibility layer, so this doesn’t work. In this case, you’ll need to manually attach the veth interfaces to OVS.

To manually attach the container’s veth interface to OVS, you must first identify the veth interface. This is why I used the lxc.network.veth.pair configuration directive in the container to assign a user-recognizable name. If you didn’t use that directive, you can use the ip link list command to list the interfaces.

Once you’ve identified the correct veth interface, simply add it to OVS with this command:

ovs-vsctl add-port br-int c1eth1

I used the c1eth1 name in the command above; you’d substitute the correct name for the appropriate veth interface for your container(s).

Once that is done, you should be able to ping over the GRE tunnel using the IP addresses assigned to the inter-container communication interfaces (which should be recognized as eth1 in the containers). Congratulations—you’ve just connected containers on two different hosts together over a GRE tunnel!

There are a couple of things to note about this configuration:

  • Note that we only used a single interface on the host. Obviously, we could have used more (for example, we could have use eth0 for host management and eth1 for host-to-host GRE tunnel connections), but using only a single host interface allows us to use this method in public cloud environments that only provide a single interface.
  • While the GRE tunnel provides encapsulation, it does not provide encryption.
  • The process of attaching the containers’ veth interface to OVS is manual right now, so that won’t scale in an environment of any size. I’m exploring ways to help automate that process. One possible avenue is the use of libvirt, so stay tuned for posts describing any progress I make in that area.

If you have any questions, corrections, suggestions, or clarifications, please feel free to post them in the comments below. All feedback is welcome and appreciated.

Tags: , , , , ,

In this post, I’m going to provide a brief introduction to working with Linux containers via LXC. Linux containers are getting a fair amount of attention these days (perhaps due to Docker, which leverages LXC on the back-end) as a lightweight alternative to full machine virtualization such as that provided by “traditional” hypervisors like KVM, Xen, or ESXi.

Both full machine virtualization and containers have their advantages and disadvantages. Full machine virtualization offers greater isolation at the cost of greater overhead, as each virtual machine runs its own full kernel and operating system instance. Containers, on the other hand, generally offer less isolation but lower overhead through sharing certain portions of the host kernel and operating system instance. In my opinion full machine virtualization and containers are complementary; each offers certain advantages that might be useful in specific situations.

Now that you have a rough idea of what containers are, let’s take a closer look at using containers with LXC. I’m using Ubuntu 12.04.3 LTS for my testing; if you’re using something different, keep in mind that certain commands may differ from what I show you here.

Installing LXC is pretty straightforward, at least on Ubuntu. To install LXC, simply use apt-get:

apt-get install lxc

Once you have LXC installed, your next step is creating a container. To create a container, you’ll use the lxc-create command and supply the name of the container template as well as the name you want to assign to the new container:

lxc-create -t <template> -n <container name>

You’ll need Internet access to run this command, as it will download (via your configured repositories) the necessary files to build a container according to the template you specified on the command line. For example, to use the “ubuntu” template and create a new container called “cn–01″, the command would look like this:

lxc-create -t ubuntu -n cn-01

Note that the “ubuntu” template specified in this command has some additional options supported; for example, you can opt to create a container with a different release of Ubuntu (it defaults to the latest LTS) or a different architecture (it defaults to the host’s architecture).

Once you have at least one container created, you can list the containers that exist on your host system:

lxc-list

This will show you all the containers that have been created, grouped according to whether the container is stopped, frozen (paused), or running.

To start a container, use the lxc-start command:

lxc-start -n <container name>

Using the lxc-start command as shown above is fine for initial testing of your container, to ensure that it boots up as you expect. However, you won’t want to run your containers long-term like this, as the container “takes over” your console with this command. Instead, you want the container to run in the background, detached from the console. To do that, you’ll add the “-d” parameter to the command:

lxc-start -d -n <container name>

This launches your container in the background. To attach to the console of the container, you can use the lxc-console command:

lxc-console -n <container name>

To escape out of the container’s console back to the host’s console, use the “Ctrl-a q” key sequence (press Ctrl-a, release, then press q).

You can freeze (pause) a container using the lxc-freeze command:

lxc-freeze -n <container name>

Once frozen, you can unfreeze (resume) a container just as easily with the lxc-unfreeze command:

lxc-unfreeze -n <container name>

You can also make a clone (a copy) of a container:

lxc-clone -o <existing container> -n <new container name>

On Ubuntu, LXC is configured by default to start containers in /var/lib/lxc. Each container will have a directory there. In a container’s directory, that container’s configuration will be stored in a file named config. I’m not going to provide a comprehensive breakdown of the settings available in the container’s configuration (this is a brief introduction), but I will call out a few that are worth noting in my opinion:

  • The lxc.network.type option controls what kind of networking the container will use. The default is “veth”; this uses virtual Ethernet pairs. (If you aren’t familiar with veth pairs, see my post on Linux network namespaces.)
  • The lxc.network.veth.pair configuration option controls the name of the veth interface created in the host. By default, a container sees one side of the veth pair as eth0 (as would be expected), and the host sees the other side as either a random name (default) or whatever you specify here. Personally, I find it useful to rename the host interface so that it’s easier to tell which veth interface goes to which container, but YMMV.
  • lxc.network.link specifies a bridge to which the host side of the veth pair should be attached. If you leave this blank, the host veth interface is unattached.
  • The configuration option lxc.rootfs specifies where the container’s root file system is stored. By default it is /var/lib/lxc/<container name>/rootfs.

There are a great deal of other configuration options, naturally; check out man 5 lxc.conf for more information. You may also find this Ubuntu page on LXC to be helpful; I certainly did.

I’ll have more posts on Linux containers in the future, but this should suffice to at least help you get started. If you have any questions, any suggestions for additional resources other readers should consider, or any feedback on the post, please add your comment below. I’d love to hear from you (courteous comments are always welcome).

Tags: , , , ,

In this post, I’m going to provide an update on using GRE tunnels with Open vSwitch (OVS) to include more than 2 hosts. I previously showed you how to use GRE tunnels with OVS to connect VMs on different hypervisor hosts, but in my testing I didn’t use this technique with more than two hypervisors. A few readers posted comments to that article asking how to extend the solution to more than 2 hypervisors, but I hadn’t had the time to test anything more.

Now, as a result of some related work I’ve been doing, I have an update on using this technique for more than two hosts. If you didn’t read the post on using GRE tunnels with OVS, go back and read that now. Also, be sure to read my post on examining OVS traffic patterns, as this is also useful information. Finally, note that this information applies to any use of GRE tunnels with OVS, not just GRE tunnels with OVS on hypervisors.

Let’s say you have three hosts:

  1. HostA, with an IP address of 10.1.1.1
  2. HostB, with an IP address of 10.1.1.2
  3. HostC, with an IP address of 10.1.1.3

To connect entities (VMs, containers, etc.) on these hosts using GRE tunnels, you’d need to manually configure OVS on each of hosts:

  • On HostA, you’d need a GRE tunnel to HostB (10.1.1.2) and a GRE tunnel to HostC (10.1.1.3)
  • On HostB, you’d need a GRE tunnel to HostA (10.1.1.1) and one to HostC (10.1.1.3)
  • On HostC, you’d need two GRE tunnels, one to HostA (10.1.1.1) and one to HostB (10.1.1.2)

I won’t repeat the specific commands to create those tunnels here, as it is well explained in my earlier article. What this creates is a virtual topology like this:

GRE tunnel full mesh

What you’ll find when you try this yourself is that everything works fine when there are just two hosts; this is what I also found when I first wrote the article. When you add the third host, though, you’ll find—assuming you created a full mesh of GRE tunnels—is that everything stops working.

Here’s how to fix that. Run this command on each of the hosts running OVS:

ovs-vsctl set bridge <bridge name> stp_enable=true

Yes, that’s right: looping is the culprit here. Look back at the topology figure above. In the physical world, a topology like that using switches (without STP) would take down your network because of a bridging loop. The same applies here as well. In both cases (physical or virtual) you have two choices: you can either not create a full mesh topology (you could use a star topology or something if you wanted) or you run STP. It’s up to you.

Assuming you turn on STP, then you’ll find after a few minutes that you’ll be able to happily ping between VMs on these hypervisors.

I do want to share one final note before I wrap up. STP is needed in this instance because we are relying on OVS in MAC learning mode (just like a physical switch). If we were to add an OpenFlow controller to this mix and push flow rules down to OVS, OVS would stop using MAC learning, and we would no longer need STP in order to build full-mesh topologies of tunnels.

Feel free to post any questions or comments below. All courteous comments are welcome!

Tags: , , , ,

Welcome to part 7 of the Learning NVP blog series, in which I will discuss transitioning from a focus on NVP to looking at NSX.

If you’re just now joining me for this series, here’s what’s transpired thus far:

When I first started this series back in May of this year, I said this:

Before continuing, it might be useful to set some context around NVP and NSX… The architecture I’m describing here will also be applicable to NSX, which VMware announced in early March. Because NSX will leverage NVP’s architecture, spending some time with NVP now will pay off with NSX later.

Well, the “later” that I referenced is now upon us. I had hoped to be much farther along with this blog series by now, but it has proven more difficult than I had anticipated to get this content written and published. Given that NSX officially GA’d last week at VMworld EMEA in Barcelona, I figured it was time to make the transition from NVP to NSX.

The way I’ll handle the transition from talking NVP to discussing VMware NSX is through an upgrade. I have a completely virtualized environment that is currently running all the NVP components: three controllers, NVP Manager, three nested hypervisors running Ubuntu+KVM+OVS, two gateways, and a service node. (I know, I know—I haven’t written about service nodes yet. Sorry.) The idea is to take you through the upgrade process, upgrading my environment from NVP 3.1.1 to NVP 3.2.1 and then to NSX 4.0.0. From that point forward, the series will change from “Learning NVP” to “Learning NSX”, and I’ll continue with discussing all the topics that I have planned. These include (among others):

  • Deploying service nodes
  • Using an L2 gateway service
  • Using an L3 gateway service
  • Enabling distributed east-west routing
  • Many, many more topics…

Unfortunately, my travel schedule over the next few weeks is pretty hectic, which will probably limit my ability to move quickly on performing and documenting the upgrade process. Nevertheless, I will press forward as quickly as possible, so stay tuned to the site for more updates as soon as I’m able to get them published.

Questions? Comments? Feel free to add them below. All I ask for is common courtesy and disclosure of vendor affiliations, where applicable. Thanks!

Tags: , , , ,

Welcome to part 6 of the Learning NVP blog series. In this part, I’m going to show you how to add an NVP gateway appliance to your NVP environment. In future posts, you’ll use this NVP gateway to host either L2 or L3 gateway services (more on those in a moment). First, though, let’s take a quick recap of what’s transpired so far:

In this part, I’m going to walk you through setting up an NVP gateway appliance. If you’ll recall from our introductory high-level architecture overview, the role of the gateway is to provide L2 (switched/bridged) and L3 (routed) connectivity between logical networks and physical networks. So, adding a gateway would then enable you to extend the logical network you created in part 4 to include either L2 or L3 connectivity to the outside world.

<aside>Many of you have probably seen some of the announcements from VMworld about NSX integrations from various networking suppliers (Arista, Brocade, Dell, and Juniper, for example). These announcements will allow NSX—which I’ve said before will leverage a great deal of NVP’s architecture—to use these hardware devices as L2 gateways, providing bridged/switched connectivity between logical networks and physical networks.</aside>

This post will focus only on getting the gateway appliance set up; in future posts, I’ll show you how to actually add the L2 or L3 connectivity to your logical network.

Building the NVP Gateway

The NVP gateway software is distributed as an ISO, like the NVP controller software. You’d typically install this software on a bare metal server, though with recent releases of NVP it is supported to install the gateway into a VM (refer to the latest NVP release notes for more details). As with the NVP controllers and NVP Manager, the gateway is built on Ubuntu 12.04, and the installer process is completely automated. Once you boot from the ISO, the installation will proceed automatically; when completed, you’ll be left at the login prompt.

Configuring the NVP Gateway

Once the NVP gateway software is installed, configuring the gateway is really straightforward. In fact, it feels a lot like configuring NVP controllers (I suspect this is by design). Here are the steps:

  1. Set the password for the admin user (optional, but highly recommended).

  2. Set the hostname for the gateway appliance (also optional, but strongly recommended).

  3. Configure the network interfaces; you’ll need management, transport, and external connectivity. (I’ll explain those in more detail later.)

  4. Configure DNS and NTP settings.

Let’s take a closer look at these steps. The first step is to set the password for the admin user, which you can accomplish with this command:

set user admin password

From here, you can proceed with setting the hostname for the gateway:

set hostname <hostname>

(So far, these commands should be pretty familiar. They are the same commands used when you set up the NVP controllers and NVP Manager.)

The next step is configure network connectivity; you’ll start by listing the available network interfaces with this command:

show network interfaces

As you’ve seen with the other NVP appliances, the NVP gateway software builds an Open vSwitch (OVS) bridge for each physical interface. In the case of a gateway, you’ll need at least three interfaces—a management interface, a transport network interface, and an external network interface. The diagram below provides a bit more context around how these interfaces are used:

NVP gateway appliance interfaces

Since these interfaces have very different responsibilities, it’s important that you properly configure them. Otherwise, things won’t work as expected. Take the time to identify which interface listed in the show network interfaces output corrsponds to each function. You’ll first want to establish management connectivity, so that should be the first interface to configure. Assuming that breth1 (the bridge matching the physical eth2 interface) is your management interface, you’ll configure it using this command:

set network interface breth1 ip config static 192.168.1.12 255.255.255.0

You’ll want to repeat this command for the other interfaces in the gateway, assigning appropriate IP addresses to each of them.

You may also need to configure the routing for the gateway. Check the routing table(s) with this command:

show network routes

If there is no default route, you can set one using this command:

add network route 0.0.0.0 0.0.0.0 <Default gateway IP address>

Once the appropriate network connectivity has been established, then you can proceed with the next step: adding DNS and NTP servers. Here are the commands for this step:

add network dns-server <DNS server IP address>
add network ntp-server <NTP server IP address>

If you accidentally fat-finger an IP address or hostname along the way, use the remove network dns-server or remove network ntp-server command to remove the incorrect entry, then re-add it correctly with the commands above.

Congrats! The NVP gateway appliance is now up and running. You’re ready to add it to NVP. Once it’s added to NVP, you’ll be able to use the gateway appliance to add gateway services to your logical networks.

Adding the Gateway to NVP

To add the new gateway appliance to NVP, you’ll use NVP Manager (I showed you how to set up NVP Manager in part 3 of the series). Once you’ve opened a web browser, navigated to the NVP Manager web UI, and logged in, then you can start the process of adding the gateway to NVP.

  1. Once you’re logged into NVP Manager, click on the Dashboard link across the top. (If you’re already at the Dashboard, you can skip this step.)

  2. In the Summary of Transport Components box, click the Connect & Add Transport Node button. This will open the Connect to Transport Node dialog box.

  3. Supply the management IP address of the gateway appliance, along with the appropriate username and password, then click Connect.

  4. After a moment, the Connect to Transport Node dialog box will show details of the gateway appliance, such as the interfaces, the bridges, the NIC bonds (if any), and the gateway’s SSL certificate. Click Configure at the bottom of the dialog box to continue.

  5. Supply a display name (something like nvp-gw–01) and, optionally, one or more tags. Click Next.

  6. Unless you know you need to select any of the options on the next screen (I’ll try to cover them in a later blog post), just click Next.

  7. On the final screen, you’ll need to establish connectivity to a transport zone. You’ll want to select the appropriate interface (in my example environment, it was breth2) and the appropriate encapsulation protocol (STT is generally recommended for connectivity back to hypervisors). Then select the appropriate transport zone from the drop-down list. In the end, you’ll have a screen that looks something like this (note that your interfaces, IP addresses, and transport zone information will likely be different):

  8. Adding a gateway to NVP

  9. Click Save to finish the process. The number of gateways listed in the Summary of Transport Components box should increment by 1 in the Registered column. However, the Active column will remain unchanged—that’s because there’s one more step needed.

  10. Back on the gateway appliance itself, run this command (you can use the IP address of any controller in the NVP controller cluster):

  11. set switch manager-cluster <NVP controller IP address>
  12. Back in NVP Manager, refresh the Summary of Transport Components box (there’s a small refresh icon in the corner), and you’ll see the Active column update to show the gateway appliance is now registered and active in NVP.

That’s it—you’re all done adding a gateway appliance to NVP. In future posts, you’ll leverage the gateway appliance to add L2 (bridged) and L3 (routed) connectivity in and out of logical networks. First, though, I’ll need to address the transition from NVP to NSX, so look for that coming soon. In the meantime, feel free to post any questions, thoughts, or suggestions in the comments below. I welcome all courteous comments (even if you disagree with something I’ve said!).

Tags: , , , ,

Welcome to Technology Short Take #36. In this episode, I’ll share a variety of links from around the web, along with some random thoughts and ideas along the way. I try to keep things related to the key technology areas you’ll see in today’s data centers, though I do stray from time to time. In any case, enough with the introduction—bring on the content! I hope you find something useful.

Networking

  • This post is a bit older, but still useful in the event if you’re interested in learning more about OpenFlow and OpenFlow controllers. Nick Buraglio has put together a basic reference OpenFlow controller VM—this is a KVM guest with CentOS 6.3 with the Floodlight open source controller.
  • Paul Fries takes on defining SDN, breaking it down into two “flavors”: host dominant and network dominant. This is a reasonable way of grouping the various approaches to SDN (using SDN in the very loose industry sense, not the original control plane-data plane separation sense). I’d like to add to Paul’s analysis that it’s important to understand that, in reality, host dominant and network dominant systems can coexist. It’s not at all unreasonable to think that you might have a fabric controller that is responsible for managing/optimizing traffic flows across the physical transport network/fabric, and an overlay controller—like VMware NSX—that integrates tightly with the hypervisor(s) and workloads running on those hypervisors to create and manage logical connectivity and logical network services.
  • This is an older post from April 2013, but still useful, I think. In his article titled “OpenFlow Test Deployment Options“, Brent Salisbury—a rock star new breed network engineer emerging in the new world of SDN—discusses some practical deployment strategies for deploying OpenFlow into an existing network topology. One key statement that I really liked from this article was this one: “SDN does not represent the end of networking as we know it. More than ever, talented operators, engineers and architects will be required to shape the future of networking.” New technologies don’t make talented folks who embrace change obsolete; if anything, these new technologies make them more valuable.
  • Great post by Ivan (is there a post by Ivan that isn’t great?) on flow table explosion with OpenFlow. He does a great job of explaining how OpenFlow works and why OpenFlow 1.3 is needed in order to see broader adoption of OpenFlow.

Servers/Hardware

  • Intel announced the E5 2600 v2 series of CPUs back at Intel Developer Forum (IDF) 2013 (you can follow my IDF 2013 coverage by looking at posts with the IDF2013 tag). Kevin Houston followed up on that announcement with a useful post on vSphere compatibility with the E5 2600 v2. You can also get more details on the E5 2600 v2 itself in this related post by Kevin as well. (Although I’m just now catching Kevin’s posts, they were published almost immediately after the Intel announcements—thanks for the promptness, Kevin!)
  • blah

Security

Nothing this time around, but I’ll keep my eyes posted for content to share with you in future posts.

Cloud Computing/Cloud Management

Operating Systems/Applications

  • I found this refresher on some of the most useful apt-get/apt-cache commands to be helpful. I don’t use some of them on a regular basis, and so it’s hard to remember the specific command and/or syntax when you do need one of these commands.
  • I wouldn’t have initially considered comparing Docker and Chef, but considering that I’m not an expert in either technology it could just be my limited understanding. However, this post on why Docker and why not Chef does a good job of looking at ways that Docker could potentially replace certain uses for Chef. Personally, I tend to lean toward the author’s final conclusions that it is entirely possible that we’ll see Docker and Chef being used together. However, as I stated, I’m not an expert in either technology, so my view may be incorrect. (I reserve the right to revise my view in the future.)

Storage

  • Using Dell EqualLogic with VMFS? Better read this heads-up from Cormac Hogan and take the recommended action right away.
  • Erwin van Londen proposes some ideas for enhancing FC error detection and notification with the idea of making hosts more aware of path errors and able to “route” around them. It’s interesting stuff; as Erwin points out, though, even if the T11 accepted the proposal it would be a while before this capability showed up in actual products.

Virtualization

That’s it for this time around, but feel free to continue to conversation in the comments below. If you have any additional information to share regarding any of the topics I’ve mentioned, please take the time to add that information in the comments. Courteous comments are always welcome!

Tags: , , , , , , , , , , , ,

As most of you probably know, I visit quite a few VMUG User Conferences around the United States and around the world. I’d probably do even more if my calendar allowed, because it’s truly an honor for me to have the opportunity to help educate the VMware user community. I know I’m not alone in that regard; there are numerous VMware “rock stars” (not that I consider myself a “rock star”) out there who also work tirelessly to support the VMware community. One need not look very far to see some examples of these types of individuals: Mike Laverick, William Lam, Duncan Epping, Josh Atwell, Nick Weaver, Alan Renouf, Chris Colotti, Cody Bunch, or Cormac Hogan are all great examples. (And I’m sure there are many, many more I’ve forgotten!)

However, one thing that has consistently been a topic of discussion among those of us who frequent VMUGs has been this question: “How do we get users more engaged in VMUG?” VMUG is, after all, the VMware User Group. And while all of us are more than happy to help support VMUG (at least, I know I am), we’d also like to see more user engagement—more customers speaking about their use cases, their challenges, the things they’ve learned, and the things they want to learn. We want to see users get connected with other users, to share information and build a community-based body of knowledge. So how can we do that?

As I see it, there is a variety of reasons why users don’t volunteer to speak:

  • They might be afraid of public speaking, or aren’t sure how good they’ll be.
  • They feel like the information they could share won’t be helpful or useful to others.
  • They aren’t sure how to structure their presentation to make it informative yet engaging.

We (meaning a group of us that support a lot of these events) have tossed around a few ideas, but nothing has ever really materialized. Today I hope to change all that. Today, I’m announcing that I will personally help mentor 5 different VMware users who are willing to step up and volunteer to speak for the first time at a local VMUG meeting in the near future.

So what does this mean?

  • I will help you select a topic on which to speak (in coordination with your local VMUG leader).
  • I will provide guidance and feedback on gathering your content.
  • I will review and provide feedback and suggestions for improving your presentation.
  • If desired, I will provide tips and tricks for public speaking.

And I’m calling on others within the VMUG community who are frequent speakers to do the same. I think that Mike Laverick might have already done something like this; perhaps the others have as well. If so, that’s awesome. If not, I challenge you, as someone viewed in a technical leadership role within the VMware and VMUG communities, to use that leadership role in a way that I hope will reinvigorate and renew user involvement and participation in the VMware/VMUG community.

If you’re one of the 5 people who’s willing to take me up on this offer, the first step is contact me and your local VMUG leader and express your interest. Don’t have my e-mail address? Here’s your first challenge: it’s somewhere on this site.

If you’re already a frequent speaker at VMUGs and you, too, want to help mentor other speakers, you can either post a comment here to that effect (and provide people with a way of getting in touch with you), or—if you have your own blog—I encourage you to make the same offer via your own site. Where possible, I’ll try to update this (or you can use trackbacks) so that readers have a good idea of who out there is willing to provide assistance to help them become the next VMUG “rock star” presenter.

Good luck, and I look forward to hearing from you!

UPDATE: A few folks have noted that all the names I listed above are VMware employees, so I’ve added a couple others who are not. Don’t read too much into that; it was all VMware employees because I work at VMware, too, and they’re the ones I communicate with frequently. There are lots of passionate and dedicated VMUG supporters out there—you know who you are!

Also, be sure to check the comments; a number of folks are volunteering to also mentor new speakers.

Tags: , ,

This is session EDCS008, “Virtualizing the Network to Enable a Software-Defined Infrastructure (SDI).” The speakers are Brian Johnson (@thehevy on Twitter) from Intel and Jim Pinkerton from Microsoft. Brian is a Solution Architect; Jim is a Windows Server Architect. If you’ve ever been in one of Brian’s presentations, you know he does a great job of really diving deep in some of this stuff. (Can you tell I’m a fan?)

Brian starts the session with a review of how the data center has evolved over the last 10 years or so, driven by the widespread adoption of compute virtualization, increased CPU capacity, and the adoption of 10Gb Ethernet. This naturally leads to a discussion of software-defined networking (SDN) as a means whereby the network can evolve to keep up the rapid pace of change and innovation in other areas of the data center. Why is this a big deal? Brian draws the comparison between property management and how IT is shaping:

  • A rental house is pretty easy to manage. One tenant, infrequent change, long-term investments.
  • An apartment means more tenants, but still relatively infrequent change.
  • A hotel means lots of tenants and the ability to handle frequent change and lots of room turnover.

The connection here is VMs—we’re now running lots of VMs, and the VMs change regularly. The infrastructure needs to be ready to handle this rapid pace of change.

At this point, Jim Pinkerton of Microsoft takes over to discuss how Windows Server thinks about this issue and these challenges. According to Jim, the world has moved beyond virtualization—it now needs the ability to scale and secure many workloads cost-effectively. You need greater automation, and you need to support any type of application. Jim talks about private clouds, hosting (IaaS-type services), and public clouds. He points out that MTTR (Mean Time to Repair) is a more important metric than MTBF (Mean Time Between Failures).

Driven by how the data center is evolving (the points in the previous paragraph), the network needs to be evolved:

  • Deliver networking as part of a pooled, automated infrastructure
  • Ensure multitenant isolation, scale, and performance
  • Expand data center capacity seamlessly as per business needs
  • Reduce operational complexity

Out of these design principles comes SDN, according to Pinkerton. Key attributes of SDN, according to Microsoft, are flexibility, control, and automation. At this point Pinkerton digresses into a discussion of SMB3 and its performance characteristics over 10Gb Ethernet—which, frankly, is completely unrelated to the topic of the presentation. After a few slides of discussing SMB3 with very little relevance to the rest of the discussion, Pinkerton moves back into a discussion of the virtual switch found in Windows Server 2012 R2.

Brian now takes over again, focusing on virtual switch performance and behavior. East-west traffic between VMs can hit 60–70Gbps, because it all happens inside the server. How do we maintain that traffic performance when we see east-west traffic between servers? We can deploy more interfaces, which is commonly seen. Moving to 10Gb Ethernet is another solution. Intel needed to add features to their network controllers—features like stateless offloads, virtual machine queues, and SR-IOV support—in order to drive performance for multiple 10Gb Ethernet interfaces. SR-IOV can help address some performance and utilization concerns, but this presents a problem when working with network virtualization. If you’re bypassing the hypervisor, how do you get on the virtual network?

Brian leaves this question for now to talk about how network virtualization with overlays helps address some of the network provisioning concerns that exist today. He provides an example of how using overlays—he uses NVGRE, since this is a joint presentation with Microsoft—can allow tenants (customers, internal business units, etc.) to share private address spaces and eliminate many manual VLAN configuration tasks. He makes the point that network virtualization is possible without SDN, but SDN makes it much easier and simpler to manage and implement network virtualization.

One drawback of overlays is that many network interface cards (NICs) today don’t “understand” the overlays, and therefore can’t perform certain hardware offloads that help optimize traffic and utilization. However, Brian shows a next-gen Intel NIC that will understand network overlays and will be able to perform offloads (like LSO, RSS, and VMQ) on encapsulated traffic.

This leads Brian to a discussion of Intel Open Network Platform (ONP), which encompasses two aspects:

  1. Intel ONP Switch reference design (aka “Seacliff Trail”), which leverages Intel silicon to support SDN and network Virtualization
  2. Intel ONP Server reference design, which shows how to optimize virtual switching using Intel’s Data Plane Development Kit (DPDK)

The Intel ONP Server reference design (sorry, can’t remember the code name) actually uses Open vSwitch (OVS) as a core part of its design.

Intel ONP includes something called FlexPipe (this is part of the Intel FM6700 chipset) to enable faster innovation and quicker support for encapsulation protocols (like NVGRE, VXLAN, and whatever might come next). The Intel ONP Switch supports serving as a bridge to connect physical workloads into virtual networks that are encapsulated, and being able to do this at full line rate using 40Gbps uplinks.

At this point, Brian and Jim wrap up the session and open up for questions and answers.

Tags: , , , ,

This is a liveblog of Intel Developer Forum (IDF) 2013 session EDCS003, titled “Enhancing OpenStack with Intel Technologies for Public, Private, and Hybrid Cloud.” The presenters are Girish Gopal and Malini Bhandaru, both with Intel.

Gopal starts off by showing the agenda, which will provide an overview of Intel and OpenStack, and then dive into some specific integrations in the various OpenStack projects. The session will wrap up with a discussion of Intel’s Open IT Cloud, which is based on OpenStack. Intel is a Gold Member of the OpenStack Foundation, has made contributions to a variety of OpenStack projects (tools, features, fixes and optimizations), has built its own OpenStack-based private cloud, and is providing additional information and support via the Intel Cloud Builders program.

Ms. Bhandaru takes over to provide an overview of the OpenStack architecture. (Not surprisingly, they use the diagram prepared by Ken Pepple.) She tells attendees that Intel has contributed bits and pieces to many of the various OpenStack projects. Next, she dives a bit deeper into some OpenStack Compute-specific contributions.

The first contribution she mentions is Trusted Compute Pools (TCP), which was enabled in the Folsom release. TCP relies upon the Trusted Platform Module (TPM), which in turn builds on Intel TXT and Trusted Boot. Together with the Open Attestation (OAT) SDK (available from https://github.com/OpenAttestation/OpenAttestation), Intel has contributed a “Trust Filter” for OpenStack Compute as well as a “Trust Filter UI” for OpenStack Dashboard. These components allow for hypervisor/compute node attestation to ensure that the underlying compute nodes have not been compromised. Users can then request that their instances are scheduled onto trusted nodes.

Intel has also done work on TCP plus Geo-Tagging. This builds on TCP to enforce policies about where instances are allowed to run. This includes a geo attestation service and Dashboard extensions to support that functionality. This work has not yet been done, but is found in current OpenStack blueprints.

In addition to trust, Intel has done work on security with OpenStack. Intel’s work focuses primarily around key management. Through collaboration with Rackspace, Mirantis, and some others, Intel has proposed a new key management service for OpenStack. This new service would rely upon good random number generation (which Intel strengthened in the Xeon E5 v2 release announced earlier today), secure storage (to encrypt the keys), careful integration with OpenStack Identity (Keystone) for authentication and access policies, extensive logging and auditing, high availability, and a pluggable-backend (similar to Cinder/Neutron). This would allow encryption of Swift objects, Glance images, and Cinder volumes. The key manager project is called Barbican (https://github.com/cloudkeep/barbican) and provides integration with OpenStack Identity. In the future, they are looking at creation and certification of private-public pairs, software support for periodic background tasks, KMIP support, and potential AES-XTS support for enhanced performance. This will also leverage Intel’s AES-NI support in newer CPUs/chipsets.

Intel also helped update the OpenStack Security Guide (http://docs.openstack.org/sec/).

Next, Intel talks about how they have worked to expose hardware features into OpenStack. This would allow for greater flexibility with the Nova scheduler. This involves work in libvirt as well as OpenStack, so that OpenStack can be aware of CPU functionality (which, in turn, might allow cloud providers to charge extra for “premium images” that offer encryption support in hardware). The same goes for exposing PCI Express (PCIe) Accelerator support into OpenStack as well.

Gopal now takes over and moves the discussion into storage in OpenStack. With regard to block storage via Cinder, Intel has incorporated support to filter volumes based on availability zone, capabilities, capacity, and other features so that volumes are allocated more intelligently based on workload and type of service required. By granting greater intelligence to how volumes are allocated, cloud service providers can offer differentiated (read: premium priced) services for block storage. This work is enabled in the Grizzly release.

In addition to block storage, many OpenStack environments also leverage Swift for object storage. Intel is focused on enabling erasure coding to Swift, which would enable reduced storage requirements in Swift deployments. Initially, erasure coding will be used for “cold” objects (objects that aren’t accessed or updated frequently); this helps preserve the service level for “hot” objects. Erasure coding would replace triple replication to reduce storage requirements in the Swift capacity tier. (Note that this something I also discussed with SwiftStack a couple weeks ago during VMworld.)

Intel has also developed something called COSBench, which is an open source tool that can be used to measure cloud object storage performance. COSBench is available at https://github.com/intel-cloud/cosbench.

At this point, Gopal transitions to networking in OpenStack. This discussion focuses primarily around Intel Open Network Platform (ONP). There’s another session that will go deeper on this topic; I expect to attend that session and liveblog it as well.

The networking discussion is very brief; perhaps because there is a dedicated session for that topic. Next up is Intel’s work with OpenStack Data Collection (Ceilometer), which includes work to facilitate the transformation and collection of data from multiple publishers. In addition, Intel is looking at enhanced usage statistics to affect compute scheduling decisions (essentially this is utilization-based scheduling).

Finally, Gopal turns to a discussion of Intel IT Open Cloud, which is a private cloud within Intel. Intel is now at 77% virtualized, with 80% of all new servers being deployed in the cloud. It’s less than an hour to deploy instances. Intel estimates a savings of approximately $21 million so far. Where is Intel IT Open Cloud headed? Intel IT is looking at using all open source software for Intel IT Open Cloud (this implies that it is not built with open source software today). There is another session on Intel IT Open Cloud tomorrow that I will try to attend.

At this point, Gopal summarizes all of the various Intel contributions to OpenStack (I took a picture of this I posted via Twitter) and ends the session.

Tags: , , , , , ,

« Older entries § Newer entries »