Welcome to part 12 of the Learning NSX blog series, in which I continue the discussion around integration between OpenStack and VMware NSX, and in which I’ll provide more details about how exactly to integrate them.

If you are just now joining the series, I encourage you to visit the Learning NVP/NSX page, where you can find links to all the posts in the series. While you’ll want to be caught up on all the posts (they do build on one another in various ways), in particular you’ll want to make sure you’ve read part 11. Part 11 covers the basics of VMware NSX-OpenStack integration, and explains how the various components of OpenStack Neutron and VMware NSX will interact.

Once you understand how the different components of Neutron and NSX will interact, getting NSX integrated into OpenStack Neutron isn’t too terribly difficult. The basic steps look like this:

  1. Install the VMware NSX plugin for Neutron.
  2. Configure VMware NSX for Neutron.
  3. Configure Neutron for VMware NSX.

Let’s take a more in-depth look at each of these steps.

Installing the VMware NSX plugin for Neutron

VMware distributes a set of compiled binary packages for OpenStack Neutron plus the VMware NSX plugin from the VMware NSX support portal (available to VMware NSX customers). Source code is also available, if you’d prefer that. These builds provided by VMware represent the latest fixes to both Neutron and NSX based off the “official” OpenStack Neutron releases. A single download contains all the different components of Neutron that you need (it’s a tarred and gzipped file that you just unpack).

Once you have the packages (I’ll assume you’re using Ubuntu and therefore have downloaded and will use the Debian packages), then you can just use dpkg -i to install the appropriate package(s) on the appropriate node(s). Recall from part 11 that when implementing Neutron with VMware NSX, you’ll need both a Neutron server as well as a network node running the DHCP and metadata agents. Here’s a breakdown of which packages need to be installed on which nodes:

  • On the Neutron server, you’d install neutron-common, neutron-server, python-neutron, and neutron-plugin-nicira.
  • On the Neutron network node, you’d install (at a minimum) neutron-common, neutron-dhcp-agent, and neutron-metadata-agent. If you wanted LBaaS support, you’d also install neutron-lbaas-agent. You could optionally install the Python client with python-neutron as well.

From here, you would proceed with setting up OpenStack Neutron as outlined in a variety of places, including the official OpenStack docs. If you do choose to use the official docs to get Neutron configured, here’s how the breakdown of the instructions map to the setup you’d need to build for use with VMware NSX:

  • The “Installing networking support on a dedicated controller node” section contains information for setting up Neutron on an OpenStack controller that does not run any of the underlying agents. Typically, this system would also run the API servers for some of the other OpenStack services as well (like Nova, Cinder, or Glance).
  • The “Installing networking support on a dedicated network node” section contains the information for setting up a network node that would run the DHCP and metadata agents. Recall that you don’t need the L3 agent, since that is handled by NSX. It might include the LBaaS agent, if you need that functionality.
  • The “Installing networking support on a dedicated compute node” section has the information for setting up your OpenStack compute nodes to interact with Neutron appropriately. Note that you don’t need to install an agent on the compute nodes; adding the compute nodes to NSX (as described in part 4) establishes the necessary communication between the NSX controllers and the compute nodes.

This helps you get Neutron up and running; in the “Configuring Neutron for VMware NSX” section below, I’ll provide additional specifics around how to configure Neutron to communicate with VMware NSX. For now, let’s make sure that VMware NSX is ready for Neutron.

Configuring VMware NSX for Neutron

VMware NSX was designed to be cloud platform-agnostic, so there isn’t a whole lot that needs to be done here. There are, however, a few tasks you’ll want to make sure you’ve done inside VMware NSX:

  1. You’ll want to ensure that you’ve added at least one NSX gateway appliance to your installation. (Part 6 describes how to add an NVP/NSX gateway appliance.)
  2. You’ll want to ensure that you’ve added an L3 gateway service, as described in part 9 of the series. The L3 gateway service replaces the L3 agent in OpenStack Neutron and is therefore necessary to provide routed/NAT’d connectivity into or out of logical (tenant) networks. Use NSX Manager to get the UUID of the L3 gateway service you’ve added; we’ll need that when we configure Neutron.
  3. You’ll need to make sure you’ve already created a transport zone, as described in part 4 and explained in greater detail in part 5. Use NSX Manager to get the UUID of the transport zone that you want Neutron to use when creating overlay networks; you’ll need that when configuring the NSX plugin for Neutron.

Configuring Neutron for VMware NSX

Now, let’s get into some nitty gritty specifics on how we configure OpenStack Neutron to interact with VMware NSX.

Most of the configuration is done within the NSX-specific configuration files, but there are two settings in neutron.conf on the controller node (where the Neutron API server is running) that you’ll want to set:

  • You’ll absolutely want to set the core_plugin value to neutron.plugins.nicira.NeutronPlugin.NvpPluginV2. (Note that in future releases of the NSX plugin, the name may change from “nicira” and “Nvp” to “vmware” and “Nsx”.)
  • You’ll probably also want to set allow_overlapping_ips to True so that Nova metadata works as you would expect. (I’ll have more on that in a moment.)

The bulk of the rest of the configuration is found in nvp.ini, which is typically found in the /etc/neutron/plugins/nicira directory. Here are the relevant settings that you’ll want to configure:

  • You’ll want to set nvp_user and nvp_password appropriately for your VMware NSX installation.
  • Populate the nvp_controllers line with the addresses of the NSX controllers in your environment, in the form “W.X.Y.Z:443″. Separate the controllers’ IP addresses with commas.
  • Place the UUID of the transport zone that you want Neutron to use when creating overlay networks as the value for default_tz_uuid.
  • Place the UUID of the L3 gateway service that you want Neutron to use when creating logical routers with external gateways as the value for the default_l3_gw_service_uuid entry.
  • In the [database] section, make sure there is an appropriate MySQL connection entry for the Neutron database (assuming you are using MySQL). An example connection entry might look like “mysql://neutron:[email protected]/neutron” (or similar).
  • There are a couple different ways to provide Nova metadata to the instances; I prefer using a special metadata access network (I’ll likely talk more about that in a future post). To use this configuration, set metadata_mode to “access_network” and set enable_metadata_access_network to True. (You may also need to set metadata_dhcp_host_route to False.)

That should be all the settings you need on the controller node. However, you’ll also need to slightly configure the DHCP and metadata agents on the network node:

  • In the dhcp_agent.ini file, set enable_isolated_metadata and enable_metadata_network to True. If your Linux distribution supports network namespaces (Ubuntu does), then also set use_namespaces to True.
  • The metadata agent does not require any special configuration above and beyond what is needed to get Neutron running.

Once you restart all relevant services so that they pick up the new settings, you should have Neutron talking to VMware NSX correctly. To test if everything is working correctly, use the Neutron CLI to create a logical network:

neutron net-create test-network

In my environment (running OpenStack Havana and NSX 4.0.0), that produced output that looked like this:

Neutron CLI output

If all is working as expected, then you should see a matching logical switch listed in NSX Manager:

Logical switches in NSX Manager

(Click the image above for a larger version.)

It may be obvious, but you’ll note that the ID returned by the neutron net-create command matches the UUID listed in NSX Manager. You’ll also note that the os_tid tag assigned to the logical switch in NSX Manager matches the tenant ID of the tenant who owns the logical switch in OpenStack. Finally, you’ll note that the bound transport zone’s UUID will match the UUID you specified in nvp.ini as I outlined earlier.

That’s it—you now have VMware NSX integrated with OpenStack Neutron!

In the next post, I’ll revisit the topic of logical networking and logical switches within VMware NSX, something I first discussed fairly early in the series. Once I’ve reviewed some concepts and established a firm foundation, future posts will take a look at how to take advantage of a very cool feature within VMware NSX: the distributed logical router.

In the meantime, feel free to post any questions, clarifications, or thoughts in the comments below. Please include any vendor affiliations, where applicable; otherwise, all courteous comments are welcome!

Tags: , , , , , ,

Technology and Travel

Cody Bunch recently posted a quick round-up of what he carries when traveling, and just for fun I thought I’d do the same. Like Cody, I don’t know that I would consider myself a road warrior, but I have traveled a pretty fair amount. Here’s what I’m currently carrying when I hit the road:

  • Light laptop and tablet: After years of carrying around a 15″ MacBook Pro, then going down to a 13″ MacBook Pro, I have to say I’m pretty happy with the 13" MacBook Air that I’m carrying now. Weight really does make a difference. I’m still toting the full-size iPad, but will probably switch to an iPad mini later in the year to save a bit more weight.
  • Bag: I settled on the Timbuktu Commute messenger bag (see my write-up) and I’m quite pleased with it. A good bag makes a big difference when you’re mobile.
  • Backup battery: I’m carrying the NewTrent PowerPak 10.0 (NT100H). It may not be the best product out there, but it’s worked pretty well for me. It’s not too heavy and not too big, and will charge both phones and tablets.
  • Noise-canceling earphones: The Bose QC20 earphones (in-ear) are awesome. Naturally they let in a bit more noise than the bigger on-ear QC15 headphones, but the added noise is worth the tremendous decrease in size and weight.

On the software side, I’ll definitely echo Cody’s recommendation of Little Snitch; it’s a excellent product that I’ve used for years. You might also consider enabling the built-in firewall (see this write-up for enabling pf on OS X Mountain Lion; haven’t tried on Mavericks yet) for an added layer of network protection.

What about you, other road warriors out there? What are you carrying these days?

Update: Thanks to Ivan Pepelnjak, who pointed out that I had inadvertently swapped out the product names for the Bose earphones and headphones. That’s been corrected!

Tags: , ,

Most IT vendors agree that more extensive use of automation and orchestration in today’s data centers are beneficial to customers. The vendors may vary in their approach to providing this automation and orchestration—some may prefer to do it in software (VMware would be one of these, along with other software companies like Microsoft and Red Hat), while others want to do it in hardware. There are advantages and disadvantages to each approach, naturally, and customers need to evaluate the various solutions against their own requirements to find the best fit.

However, the oft-overlooked problem that more extensive use of automation and orchestration creates is one of control—specifically, how customers can control this automation and orchestration according to their own specific policy. A recent post on the Network Heresy site discusses the need for policy in fully automated IT environments:

However, fully automated IT management is a double-edged sword. While having people on the critical path for IT management was time-consuming, it provided an opportunity to ensure that those resources were managed sensibly and in a way that was consistent with how the business said they ought to be managed. In other words, having people on the critical path enabled IT resources to be managed according to business policy. We cannot simply remove those people without also adding a way of ensuring that IT resources obey business policy—without introducing a way of ensuring that IT resources retain the same level of policy compliance.

VMware, along with a number of other companies, has launched an open source effort to address this challenge: finding a way to enable customers to manage their resources according to their business policy, and do so in a cloud-agnostic way. This effort is called Congress, and it has received some attention from those who think it’s a critical project). I’m really excited to be involved in this project, and I’m also equally excited to be working with some extremely well-respected individuals across a number of different companies (this is most definitely not a VMware-only project). I believe that creating an open source solution to the policy problem will further the cause of cloud computing and the transformation of our industry. I strongly urge you to read this first post, titled “On Policy in the Data Center: The policy problem”, and stay tuned for future blog posts that will dive into even greater detail. Exciting times are ahead!

Tags: , ,

Crossing the Threshold

Last week while attending the CloudStack Collaboration Conference in my home city of Denver, I had a bit of a realization. I wanted to share it here in the hopes that it might serve as an encouragement for others out there.

Long-time readers know that one of my projects over the last couple of years has been to become more fluent in Linux (refer back to my 2012 project list and my 2013 project list). I gave myself a B+ for my efforts last year, feeling that I had made good progress over the course of the year. Even so, I still felt like there was still so much that I needed to learn. As so many of us are inclined to do, I was more focused on what I still hadn’t learned instead of taking a look at what I had learned.

This is where last week comes in. Before the conference started, I participated in a couple of “mini boot camps” focused on CloudStack and related tools/clients/libraries. (You may have seen some of my tweets about tools like cloudmonkey, Apache libcloud, and awscli/ec2stack.) As I worked through the boot camps, I could hear the questions that other attendees were asking as well as the tasks with which others were struggling. Folks were wrestling with what I thought were pretty simple tasks; these were not, after all, very complex exercises. So the lab guide wasn’t complete or correct; you should be able to figure it out, right?

Then it hit me. I’m a Linux guy now.

That’s right—I had crossed the threshold between “working on being a Linux guy” and “being a Linux guy.” It’s not that I know everything there is to know (far from it!), but that the base level of knowledge had finally accrued to a level where—upon closer inspection—I realized that I was fluent enough that I could perform most common tasks without a great deal of effort. I knew enough to know what to do when something didn’t work, or wasn’t configured properly, and the general direction in which to look when trying to determine exactly what was going on.

At this point you might be wondering, “What does that have to do with encouraging me?” That’s a fair question.

As IT professionals—especially those on the individual contributor (IC) track instead of the management track—we are tasked with having to constantly learn new products, new technologies, and new methodologies. Because the learning never stops (and that isn’t a bad thing, in my humble opinion), we tend to focus on what we haven’t mastered. We forget to look at what we have learned, at the progress that we have made. Maybe, like me, you’re on a journey of learning and education to move from being a specialist in one type of technology to a practitioner of another type. If that’s the case, perhaps it’s time you stop saying “I will be a <new technology> person” and say “I am a <new technology> person.” Perhaps it’s time for you to cross the threshold.

Tags: , ,

Welcome to Technology Short Take #40. The content is a bit light this time around; I thought I’d give you, my readers, a little break. Hopefully there’s still some useful and interesting stuff here. Enjoy!

Networking

  • Bob McCouch has a nice write-up on options for VPNs to AWS. If you’re needing to build out such a solution, you might want to read his post for some additional perspectives.
  • Matthew Brender touches on a networking issue present in VMware ESXi with regard to VMkernel multi-homing. This is something others have touched on before (including myself, back in 2008—not 2006 as I tweeted one day), but Matt’s write-up is concise and to the point. You’ll definitely want to keep this consideration in mind for your designs. Another thing to consider: vSphere 5.5 introduces the idea of multiple TCP/IP stacks, each with its own routing table. As the ability to use multiple TCP/IP stacks extends throughout vSphere, it’s entirely possible this limitation will go away entirely.
  • YAOFC (Yet Another OpenFlow Controller), interesting only because it focuses on issues of scale (tens of thousands of switches with hundreds of thousands of endpoints). See here for details.

Servers/Hardware

  • Intel recently announced a refresh of the E5 CPU line; Kevin Houston has more details here.

Security

  • This one slipped past me in the last Technology Short Take, so I wanted to be sure to include it here. Mike Foley—whom I’m sure many of you know—recently published an ESXi security whitepaper. His blog post provides more details, as well as a link to download the whitepaper.
  • The OpenSSL “Heartbleed” vulnerability has captured a great deal of attention (justifiably so). Here’s a quick article on how to assess if your Linux-based server is affected.

Cloud Computing/Cloud Management

  • I recently built a Windows Server 2008 R2 image for use in my OpenStack home lab. This isn’t as straightforward as building a Linux image (no surprises there), but I did find a few good articles that helped along the way. If you find yourself needing to build a Windows image for OpenStack, check out creating a Windows image on OpenStack (via Gridcentric) and building a Windows image for OpenStack (via Brent Salisbury). You might also check out Cloudbase.it, which offers a version of cloud-init for Windows as well as some prebuilt evaluation images. (Note: I was unable to get the prebuilt images to download, but YMMV.)
  • Speaking of building OpenStack images, here’s a “how to” guide on building a Debian 7 cloud image for OpenStack.
  • Sean Roberts recently launched a series of blog posts about various OpenStack projects that he feels are important. The first project he highlights is Congress, a policy management project that has recently gotten a fair bit of attention (see a reference to Congress at the end of this recent article on the mixed messages from Cisco on OpFlex). In my opinion, Congress is a big deal, and I’m really looking forward to seeing how it evolves.
  • I have a related item below under Virtualization, but I wanted to point this out here: work is being done on a VIF driver to connect Docker containers to Open vSwitch (and thus to OpenStack Neutron). Very cool. See here for details.
  • I love that Cody Bunch thinks a lot like I do, like this quote from a recent post sharing some links on OpenStack Heat: “That generally means I’ve got way too many browser tabs open at the moment and need to shut some down. Thus, here comes a huge list of OpenStack links and resources.” Classic! Anyway, check out the list of Heat resources, you’re bound to find something useful there.

Operating Systems/Applications

  • A short while back I had a Twitter conversation about spinning up a Minecraft server for my kids in my OpenStack home lab. That led to a few other discussions, one of which was how cool it would be if you could use Heat autoscaling to scale Minecraft. Then someone sends me this.
  • Per the Microsoft Windows Server Team’s blog post, the Windows Server 2012 R2 Udpate is now generally available (there’s also a corresponding update for Windows 8.1).

Storage

  • Did you see that EMC released a virtual edition of VPLEX? It’s being called the “data plane” for software-defined storage. VPLEX is an interesting product, no doubt, and the introduction of a virtual edition is intriguing (but not entirely unexpected). I did find it unusual that the release of the virtual edition signalled the addition of a new feature called “MetroPoint”, which allows two sites to replicate back to a single site. See Chad Sakac’s blog post for more details.
  • This discussion on MPIO and in-guest iSCSI is a great reminder that designing solutions in a virtualized data center (or, dare I say it—a software-defined data center?) isn’t the same as designing solutions in a non-virtualized environment.

Virtualization

  • Ben Armstrong talks briefly about Hyper-V protected networks, which is a way to protect a VM against network outage by migrating the VM to a different host if a link failure occurs. This is kind of handy, but requires Windows Server clustering in order to function (since live migration in Hyper-V requires Windows Server clustering). A question for readers: is Windows Server clustering still much the same as it was in years past? It was a great solution in years past, but now it seems outdated.
  • At the same time, though, Microsoft is making some useful networking features easily accessible in Hyper-V. Two more of Ben’s articles show off the DHCP Guard and Router Guard features available in Hyper-V on Windows Server 2012.
  • There have been a pretty fair number of posts talking about nested ESXi (ESXi running as a VM on another hypervisor), either on top of ESXi or on top of VMware Fusion/VMware Workstation. What I hadn’t seen—until now—was how to get that working with OpenStack. Here’s how Mathias Ewald made it work.
  • And while we’re talking nested hypervisors, be sure to check out William Lam’s post on running a nested Xen hypervisor with VMware Tools on ESXi.
  • Check out this potential way to connect Docker containers with Open vSwitch (which then in turn opens up all kinds of other possibilities).
  • Jason Boche regales us with a tale of a vCenter 5.5 Update 1 upgrade that results in missing storage providers. Along the way, he also shares some useful information about Profile-Driven Storage in general.
  • Eric Gray shares information on how to prepare an ESXi ISO for PXE booting.
  • PowerCLI 5.5 R2 has some nice new features. Skip over to Alan Renouf’s blog to read up on what is included in this latest release.

I should close things out now, but I do have one final link to share. I really enjoyed Nick Marshall’s recent post about the power of a tweet. In the post, Nick shares how three tweets—one with Duncan Epping, one with Cody Bunch, and one with me—have dramatically altered his life and his career. It’s pretty cool, if you think about it.

Anyway, enough is enough. I hope that you found something useful here. I encourage readers to contribute to the discussion in the comments below. All courteous comments are welcome.

Tags: , , , , , , , , , , ,

Some time ago, I introduced you to the idea of Linux network namespaces, and provided an overview of some of the commands needed to interact with network namespaces. In this post, I’ll follow up on that post with some additional information on using network namespaces with other types of network interfaces.

In the previous network namespaces article, I mentioned (incorrectly, it turns out) that you had to use virtual Ethernet (veth) interfaces in order to connect a namespace to the physical network:

It turns out you can only assign virtual Ethernet (veth) interfaces to a network namespace.

I say “incorrectly” because you are able to assign more than just virtual Ethernet interfaces to a Linux network namespace. I’m not sure why I arrived at that conclusion, because subsequent testing—using Ubuntu 12.04 LTS as with the original testing—showed no problem assigning physical interfaces to a particular network namespace. The VMs I used for the subsequent testing are using a different kernel (the 3.11 kernel instead of whatever I used in the previous testing), so it’s entirely possible that’s the difference. If I get the opportunity, I’ll try with earlier kernel builds to see if that makes any difference.

In any case, assigning other types of network interfaces to a network namespace is just like assigning veth interfaces. First, you create the network namespace:

ip netns add <new namespace name>

Then, you’d assign the interface to the namespace:

ip link set <device name> netns <namespace name>

For example, if you wanted to assign eth1 to the “blue” namespace, you’d run this:

ip link set eth1 netns blue

Please note that I haven’t found a way to unassign an interface from a network namespace other than deleting the namespace entirely.

If you want to assign a VLAN interface to a namespace, the process is slightly different. You’ll have to create the namespace first, as with physical and veth interfaces, but you’ll also have to create the VLAN interface in the “default” namespace first, then move it over to the desired namespace.

For example, first you’d create the namespace “red”:

ip netns add red

Then you’d create the VLAN interface for VLAN 100 on physical interface eth1:

ip link add link eth1 name eth1.100 type vlan id 100

The generic form of this command is this:

ip link add link <physical device> name <VLAN device name> type vlan id <VLAN ID>

Note that you can’t use ip netns exec to run the command to create the VLAN interface in the network namespace directly; it won’t work because the parent interface upon which the VLAN interface is based doesn’t exist in the namespace. So you’ll create the VLAN interface in the default namespace first, then move it over.

Once the VLAN interface is created, then move it to the target namespace:

ip link set eth1.100 netns red

One (sort of) interesting thing I noted in my testing was that link status and IP addresses don’t move between namespaces. Therefore, don’t bother assigning an IP address or setting the link state of an interface before you move it to the final namespace, because you’ll just have to do it again.

To make the interface functional inside the target namespace, you’ll use ip netns exec to target the specific configuration commands against the desired namespace. For example, if the VLAN interface eth1.100 exists in the namespace “blue”, you’d run these commands:

ip netns exec blue ip addr add 10.1.1.1/24 dev eth1.100

That adds the IP address to the interface in the namespace; then you’d use this command to set the link status to up:

ip netns exec blue ip link set eth1.100 up

Then you can test network connectivity with our good friend ping like this:

ip netns exec blue ping -c 4 10.1.1.2

(Obviously, you’d want to substitute an appropriate IP address there for your specific configuration and environment.)

I hope this additional information on working with Linux network namespaces is useful. As always, I invite and encourage any questions, thoughts, or corrections in the comments below. All courteous comments are welcome!

Tags: , ,

In this post, I’m going to talk about a way to modify QCOW2-based images without having to boot up a full-blown virtual machine based on the image. I found this process while trying to change the behavior of cloud-init on the official Ubuntu cloud images from Canonical.

While I had successfully built my own Ubuntu 12.04 LTS image for my OpenStack-based home lab (and it worked just fine), I was keen to use Canonical’s official cloud images as they were much smaller (around 250MB versus 1.5GB for my home-built image). Given that disk space is at a bit of a premium in my home lab, using the smaller images would be quite beneficial. The problem was that cloud-init wasn’t configured to operate in a way that worked for my home lab environment.

The question, of course, was…how do I do this? I tried a couple different approaches:

  1. I installed VirtualBox on one of my OS X systems, because I knew VirtualBox supported QCOW2 images. Unfortunately, trying to launch a VM using one of the official cloud images as a root disk failed. (I presume this is because the cloud images assume the use of the virtio block driver, which VirtualBox doesn’t support.)

  2. Next, I installed a plain jane Ubuntu VM in VirtualBox and attached the official cloud image QCOW2 file as a second hard disk. That didn’t work either; the Ubuntu VM refused to recognize the filesystem(s) on the QCOW2 image.

I was about at my wits’ end when I found this post. Aha! I’d heard of guestfish before, but I hadn’t thought to try it in this instance. Guestfish is part of the larger libguestfs project.

So I tried the instructions in the post I’d found. Unfortunately, it failed the first time I tried, but I later—thanks to qemu-img check—found that the Ubuntu 12.04 cloud image I’d downloaded was somehow damaged/corrupted. Using a new download of the image was successful.

Here are the steps I used to modify the contents of a QCOW2 disk image using guestfish:

  1. Starting from a vanilla Ubuntu 12.04 LTS install, use apt-get update and apt-get install guestfish to install guestfish. (You’ll probably need/want to use sudo with both of these commands.)

  2. During the installation of guestfish, you can select to create a “virtual appliance” that guestfish uses. If you choose not to do this during installation, you can always do it later with sudo update-guestfs-appliance.

  3. Once guestfish is installed and the virtual appliance is created/updated, launch guestfish itself with sudo guestfish. Note that if you don’t have root privileges or don’t use sudo, guestfish will still launch and act like everything is working—but it will fail. Once guestfish is up and running, you’ll be sitting at a “> <fs>” prompt.

  4. Type add <QCOW2 image filename> (obviously substituting the full path and correct filename for the QCOW2 image you’d like to modify).

  5. Type run and press Enter. You’ll see a progress bar that updates the status of the guestfs virtual appliance. Once it’s complete, you’ll be returned to the “> <fs>” prompt.

  6. Enter list-filesystems. Guestfish should return a list of filesystems on the QCOW2 disk image. In the case of the official Canonical cloud images, only “/dev/vda1″ is returned.

  7. To gain access to the filesystem in the QCOW2 image, enter mount <device specification> <guestfish mount point>. For example, when I was modifying the Canonical cloud images, I used mount /dev/vda1 /. Keep in mind you’re not mounting to the system on which guestfish is running; you’re mounting to the guestfish virtual appliance.

  8. Now just edit the files in the QCOW2 image using whatever method you prefer. In my case, I needed to edit /etc/cloud/cloud.cfg, so I just used vi to edit the file and make whatever changes were needed.

  9. When you’re done, type exit to quit guestfish and commit changes to the QCOW2 image.

And that’s it! Using this technique I was able to quickly and easily modify the Ubuntu cloud images so that cloud-init behaved in the way that I wanted.

Hopefully this information helps someone else. If you have any questions or corrections, I invite you to share in the comments below.

Tags: , , ,

Welcome to part 11 of the Learning NSX blog series, in which I provide a high-level overview of the basics on integrating VMware NSX into an OpenStack deployment using OpenStack Neutron. In case you’re just now catching up on this blog series, I encourage you to visit my Learning NVP/NSX page, which has a brief summary of all the posts in the series.

Now that I shown you how to build all the different components of NSX (NSX controllers, NSX Manager, gateway appliances, and service nodes), it’s time to add integration into a cloud management platform. VMware NSX was designed to be integrated into a cloud management platform. Because VMware NSX offers a full-featured RESTful API, you could—in theory—integrate NSX into just about any cloud management platform. However, for the purposes of this series, I’ll limit the discussion to focus on how one would integrate VMware NSX with OpenStack via the OpenStack Neutron (formerly “Quantum”) project, which provides virtual networking functionality for OpenStack-based clouds.

The challenge in discussing OpenStack-NSX integration is that one must first understand the basics of OpenStack Neutron before looking at how to integrate VMware NSX into Neutron. Therefore, the primary goal of this post in the series is to provide an overview of Neutron, and then discuss how NSX integrates into Neutron. The next post in the series will provide more in-depth technical details on exactly how the integration is configured.

Let’s start with a generic overview of OpenStack Neutron and its components.

Examining OpenStack Neutron Components

OpenStack Neutron itself has a number of different components:

  • The Neutron server, which supplies the Neutron API and is typically—but not required to be—deployed co-resident on an OpenStack “controller” node (not to be confused with an NSX controller).
  • The Neutron DHCP agent, which provides DHCP services for the various logical networks created in Neutron (at least, whenever DHCP is enabled for a logical network).
  • The Neutron L3 agent, which provides L3 (routed) connectivity for Neutron logical networks. This includes both L3 between logical networks as well as L3 in and out of logical networks.
  • The Neutron metadata agent, which is responsible for providing connectivity between instances and the Nova metadata service (for customizing instances appropriately).
  • Finally, for many Open vSwitch (OVS)-based installations, there is the Neutron OVS agent, which provides a way to program OVS to do what Neutron needs it to do.

The Neutron DHCP, L3, and metadata agents are typically—but not required to be—installed on a so-called “network” node. This network node thus provides DHCP services to the various logical networks (often using Linux network namespaces, if the Linux distribution supports them), routed connectivity in and out of logical networks (once again with network namespaces, Linux bridges, and iptables rules), and metadata service connectivity.

From a traffic flow perspective, traffic from one subnet in a logical network to another subnet in a logical network must hairpin through the network node (where the L3 agent resides). Traffic headed out of a logical network must flow through the network node, where iptables rules will perform the necessary network address translation (NAT) functions associated with the use of floating IPs in an OpenStack environment. And, as I’ve already mentioned, the network node provides DHCP services to all the logical networks as well.

This is, by necessity, a very high-level overview of OpenStack Neutron and the core components. Let’s now take a look at how these components are affected when you choose to use VMware NSX with OpenStack Neutron.

Reviewing OpenStack Neutron With NSX

When you’re using VMware NSX as the mechanism behind OpenStack Neutron, some of Neutron’s functionality is provided by NSX itself:

  • You no longer need the L3 agent. L3 connectivity is provided by the NSX gateway appliances and logical gateway services (refer to part 6 and part 9 of the series, respectively).
  • You won’t need the OVS agent on the hypervisors, because the NSX controllers are responsible for configuring/programming OVS on transport nodes. More details on this interaction is provided in part 4 of the series. (Transport nodes is a generic term referring to nodes that participate in the data plane, such as hypervisors, gateways, and service nodes.)
  • You will still need the Neutron server, the DHCP agent, and the metadata agent.

Therefore, if you choose to deploy the OpenStack Neutron components in a “typical” fashion, you’d have a setup something like this:

  • An OpenStack “controller” node would host the Neutron server, which provides the API with which other parts of OpenStack will interact. This node generally does not have OVS installed, but would have the NSX plugin for Neutron installed. This plugin implements the integration between OpenStack and NSX, and will communicate with NSX via the NSX northbound RESTful API.
  • An OpenStack “network” node would host the Neutron DHCP agent and the Neutron metadata agent. This node would have OVS installed, and would be registered into NSX as a hypervisor (even though it is not a hypervisor and will not host any VMs).

At this point, you should have a pretty good understanding of how, at a high level, NSX integrates with and affects OpenStack Neutron. In the next post in the series, I’ll provide more details on exactly how to configure the integration between VMware NSX and OpenStack Neutron.

Tags: , , , , , ,

If you’ve been following trends in the “cloud” world, you’ve probably heard of Docker. In this post I’m going to provide a quick introduction to Docker, which might be helpful if you’re wondering what Docker is and why it’s garnered so much attention.

The best way to describe Docker is to use the phrase from the Docker web site—Docker is “an open source project to pack, ship and run any application as a lightweight container.” The idea is to provide a comprehensive abstraction layer that allows developers to “containerize” or “package” any application and have it run on any infrastructure. The use of container here refers more to the consistent, standard packaging of applications rather than referring to any underlying technology (a distinction that will be important in a moment). The most common analogy used to help people understand Docker is saying that Docker containers are like shipping containers: they provide a standard, consistent way of shipping just about anything. Docker containers provide a standard, consistent way of packaging just about any application. (At least, that’s the idea.)

So what are the underlying technologies that make up Docker?

  • Docker leverages LXC (Linux Containers), which encompasses Linux features like cgroups and namespaces for strong process isolation and resource control. If you’re not familiar with LXC, I suggest having a look at my introductory post on LXC. Docker’s use of LXC is why I mentioned earlier that Docker’s use of container is more about consistent/standard packaging than any specific technology. While Docker currently leverages LXC, the Docker architecture isn’t necessarily limited to LXC. In theory, Docker could leverage KVM to do the same things it does today with LXC. (Update: While in the process of finishing this post, Docker released version 0.9, which does offer the ability to replace LXC with a different back-end “engine” called libcontainer.)
  • Docker leverages a copy-on-write filesystem (currently AUFS, but other filesystems are being investigated). This allows Docker to instantiate containers (using that term to refer to Docker’s construct, not the LXC construct) very quickly—instead of having to make full copies of whatever files comprise a container, Docker can basically use “pointers” back to existing files. It seems to me this is a big part of what makes Docker so useful.
  • Perhaps due in part to the use of a copy-on-write file system, Docker containers are easily “linked” to other containers. Some people refer to this as stacking containers (continuing the shipping container analogy); to me, it makes more sense to talk about layering one container on top of another. For example, you might create a container that is based on a base Ubuntu image, and then in turn create another container that is based on the first container.
  • Docker uses a “plain text” configuration language (I suppose you could think of it as a domain-specific language, or DSL) to control the configuration of an application container, such as what files should (or should not) be included, what network ports should be open, and what processes/applications should be running inside the container. (I’ll provide an example of a Docker file in just a few moments.)

Now that you have an idea of the basics, let’s take a deeper look. For the purposes of this article, I’m using Ubuntu 12.04 LTS as the Linux platform. Please keep in mind that different distributions may use different commands.

Installing Docker

To install Docker on Ubuntu, it’s a pretty straightforward process.

  1. First, you’ll want to make sure you are running at least the 3.8 kernel. (Newer versions of 12.04 LTS, such as 12.04.4, appear to be installing the 3.11 kernel by default, so you may not have to do this step.) To install the 3.8 kernel (in case you’re running something older), just run these two commands and reboot:
sudo apt-get update
sudo apt-get install linux-image-generic-lts-raring \
linux-headers-generic-lts-raring
  1. Next, add the Docker repository key to your local Apt keychain:
sudo apt-key adv --keyserver keyserver.ubuntu.com \
--recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
  1. Add the Docker repository to your list of repositories:
sudo sh -c "echo deb http://get.docker.io/ubuntu docker main \
> /etc/apt/sources.list.d/docker.list"
  1. Finally, install Docker with an apt-get combo:
sudo apt-get update
sudo apt-get install lxc-docker

Note that this automatically installs LXC, but an “older” version (0.7.5). I haven’t tested to see if Docker will work with the newer alpha release of LXC 1.0.0 that is available in the precise-backports repository (see this post for more information).

Launching a Simple Docker Container

Now that Docker is installed, you can launch a quick Docker container like this:

sudo docker run -i -t ubuntu /bin/bash

If everything is working, the first time you run this command it will automatically download an Ubuntu image (as specified by “ubuntu” on that command line) and create a simple Docker container that just runs the bash shell. You’ll get dropped into an odd prompt that will look something like this (the numbers will change from system to system):

root@4a2f737d6e2e:/#

You’re now running in an Ubuntu environment, but it’s a clean environment—there are no daemons running, nothing listening on various network ports, nothing else except the bash shell. If you run ifconfig, you’ll see that eth0 has an IP address, but that’s it. When you type exit, the container will terminate and you’ll be returned to the prompt for your host system.

Now, let’s take a look at what happened as a result of running that command. If you run sudo docker images, you will see a list of available Docker images that were downloaded to your local system. They’ll all be listed as “ubuntu” in the first column, and it’s the second column that will differentiate them—you’ll see listings for 12.04, 12.10, 13.04, 13.10, etc. You could run a specific Ubuntu image with a command like this:

sudo docker run -i -t ubuntu:13.10 /bin/bash

If you run this command, you’ll note that you’re launched into this new container very quickly. This is one of the strengths of Docker: once an image has been downloaded, launching containers based on that image is very fast.

One other thing you’ll note about Docker is that the containers you start up are ephemeral—meaning that changes you make to the container aren’t persistent. Try this: start up a container (you can use a simple container like the examples I’ve provided already), create a file, and then exit the container. Start the container again, and you’ll see that the file you created previously no longer exists. There are workarounds for this, but for the sake of keeping this post to a manageable size I’ll save discussing those workarounds for another time.

Launching More Complex Docker Containers

So what if you need a Docker container that does need to have a daemon running and listening on a network port? It’s great that you can run a container for the bash shell, but that’s not particularly useful. What if you wanted to run a MySQL container, an SSH container, or a web server container?

To do something like that, you’ll need to utilize a Dockerfile. A Dockerfile is a plain text file that tells Docker exactly how to construct a Docker image. (I mentioned earlier that Docker had its own DSL; this is what I’m talking about.) Here’s an example Dockerfile:

FROM ubuntu:12.04
MAINTAINER Joe Shmoe "[email protected]"
RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
RUN apt-get update
RUN apt-get install -y nginx
EXPOSE 80
CMD ["nginx"]

The syntax for a Dockerfile is INSTRUCTION arguments, where “INSTRUCTION” refers to commands like FROM, RUN, or CMD. Here are a few quick notes:

  • The FROM command is what enables Docker containers to be based on other Docker containers. In this case, the Docker container is based on the Ubuntu image; specifically, the Ubuntu 12.04 image.
  • The RUN commands execute various commands within the new container. You’d use RUN instructions to install packages, modify files, etc.
  • The EXPOSE command exposes a listening port on a Docker container.
  • The CMD instruction tells what command/daemon to run when the container launches.

I don’t have room here for a full Dockerfile tutorial, but Docker themselves has a great one. Suffice it to say that if you want anything more complex than a simple bash shell, you’re probably going to need to use a Dockerfile.

Once you have a working Dockerfile (just as a heads-up, the above example doesn’t actually work—sorry), you can create a Docker image from that Dockerfile using the docker build command. Once you have a working image, you can then base other Docker containers on that image (using the FROM instruction in a subsequent Dockerfile).

Exploring Other Docker Commands

There are also some other Docker commands you’ll find useful:

  • The docker images command lists all the Docker images available on the system. This includes images that you’ve downloaded from elsewhere as well as images you’ve created.
  • As I mentioned earlier, docker build allows you to create your own images from a Dockerfile.
  • To see which Docker containers are running, use docker ps.
  • The docker rmi command removes Docker images.
  • Finally, just run docker help if you need to see what other commands are available.

Trying Out Docker

You don’t need a lot to try out Docker; a simple Ubuntu 12.04 LTS VM running on VirtualBox, VMware Fusion, or VMware Workstation will work fine. Nested virtualization support isn’t required when using LXC, and the Docker containers themselves are pretty lightweight. To try out Docker, you only need to create a VM and follow the instructions provided earlier for installing Docker. Once you’ve got that, you’re ready to explore it for yourself.

For additional information or another perspective on Docker, I recommend taking a look at this presentation on SlideShare by James Turnbull (formerly with Puppet Labs, now with Docker).

As always, all courteous questions, thoughts, suggestions, or corrections are welcome.

Tags: , , ,

Welcome to Technology Short Take #39, in which I share a random assortment of links, articles, and thoughts from around the world of data center-related technologies. I hope you find something useful—or at least something interesting!

Networking

  • Jason Edelman has been talking about the idea of a Common Programmable Abstraction Layer (CPAL). He introduces the idea, then goes on to explore—as he puts it—the power of a CPAL. I can’t help but wonder if this is the right level at which to put the abstraction layer. Is the abstraction layer better served by being integrated into a cloud management platform, like OpenStack? Naturally, the argument then would be, “Not everyone will use a cloud management platform,” which is a valid argument. For those customers who won’t use a cloud management platform, I would then ask: will they benefit from a CPAL? I mean, if they aren’t willing to embrace the abstraction and automation that a cloud management platform brings, will abstraction and automation at the networking layer provide any significant benefit? I’d love to hear others’ thoughts on this.
  • Ethan Banks also muses on the need for abstraction.
  • Craig Matsumoto of SDN Central helps highlight a recent (and fairly significant) development in networking protocols—the submission of the Generic Network Virtualization Encapsulation (Geneve) proposal to the IETF. Jointly authored by VMware, Microsoft, Red Hat, and Intel, this new protocol proposal attempts to bring together the strengths of the various network virtualization encapsulation protocols out there today (VXLAN, STT, NVGRE). This is interesting enough that I might actually write up a separate blog post about it; stay tuned for that.
  • Lee Doyle provides an analysis of the market for network virtualization, which includes some introductory information for those who might be unfamiliar with what network virtualization is. I might contend that Open vSwitch (OVS) alone isn’t an option for network virtualization, but that’s just splitting hairs. Overall, this is a quick but worthy read if you are trying to get started in this space.
  • Don’t think this “software-defined networking” thing is going to take off? Read this, and then let me know what you think.
  • Chris Margret has a nice dissection of how bash completion works, particularly in regards to the Cumulus Networks implementation.

Servers/Hardware

  • Via Kevin Houston, you can get more details on the Intel E7 v2 and new blade servers based on the new CPU. x86 marches on!
  • Another interesting tidbit regarding hardware: it seems as if we are now seeing the emergence of another round of “hardware offloads.” The first round came about around 2006 when Intel and AMD first started releasing their hardware assists for virtualization (Intel VT and AMD-V, respectively). That technology was only “so-so” at first (VMware ESX continued to use binary translation [BT] because it was still faster than the hardware offloads), but it quickly matured and is now leveraged by every major hypervisor on the market. This next round of hardware offloads seems targeted at network virtualization and related technologies. Case in point: a relatively small company named Netronome (I’ve spoken about them previously, first back in 2009 and again a year later), recently announced a new set of network interface cards (NICs) expressly designed to provide hardware acceleration for software-defined networking (SDN), network functions virtualization (NFV), and network virtualization solutions. You can get more details from the Netronome press release. This technology is actually quite interesting; I’m currently talking with Netronome about testing it with VMware NSX and will provide more details as that evolves.

Security

  • Ben Rossi tackles the subject of security in a software-defined world, talking about how best to integrate security into SDN-driven architectures and solutions. It’s a high-level article and doesn’t get into a great level of detail, but does point out some of the key things to consider.

Cloud Computing/Cloud Management

  • “Racker” James Denton has some nice articles on OpenStack Neutron that you might find useful. He starts out with discussing the building blocks of Neutron, then goes on to discuss building a simple flat network, using VLAN provider networks, and Neutron routers and the L3 agent. And if you need a breakdown of provider vs. tenant networks in Neutron, this post is also quite handy.
  • Here’s a couple (first one, second one) of quick walk-throughs on installing OpenStack. They don’t provide any in-depth explanations of what’s going on, why you’re doing what you’re doing, or how it relates to the rest of the steps, but you might find something useful nevertheless.
  • Thinking of building your own OpenStack cloud in a home lab? Kevin Jackson—who along with Cody Bunch co-authored the OpenStack Cloud Computing Cookbook, 2nd Edition—has three articles up on his home OpenStack setup. (At least, I’ve only found three articles so far.) Part 1 is here, part 2 is here, and part 3 is here. Enjoy!
  • This post attempts to describe some of the core (mostly non-technical) differences between OpenStack and OpenNebula. It is published on the OpenNebula.org site, so keep that in mind as it is (naturally) biased toward OpenNebula. It would be quite interesting to me to see a more technically-focused discussion of the two approaches (and, for that matter, let’s include CloudStack as well). Perhaps this already exists—does anyone know?
  • CloudScaling recently added a Google Compute Engine (GCE) API compatibility module to StackForge, to allow users to leverage the GCE API with OpenStack. See more details here.
  • Want to run Hyper-V in your OpenStack environment? Check this out. Also from the same folks is a version of cloud-init for Windows instances in cloud environments. I’m testing this in my OpenStack home lab now, and hope to have more information soon.

Operating Systems/Applications

Storage

Virtualization

  • Brendan Gregg of Joyent has an interesting write-up comparing virtualization performance between Zones (apparently referring to Solaris Zones, a form of OS virtualization/containerization), Xen, and KVM. I might disagree that KVM is a Type 2 hardware virtualization technology, pointing out that Xen also requires a Linux-based dom0 in order to function. (The distinction between a Type 1 that requires a general purpose OS in a dom0/parent partition and a Type 2 that runs on top of a general purpose OS is becoming increasingly blurred, IMHO.) What I did find interesting was that they (Joyent) run a ported version of KVM inside Zones for additional resource controls and security. Based on the results of his testing—performed using DTrace—it would seem that the “double-hulled virtualization” doesn’t really impact performance.
  • Pete Koehler—via Jason Langer’s blog—has a nice post on converting in-guest iSCSI volumes to native VMDKs. If you’re in a similar situation, check out the post for more details.
  • This is interesting. Useful, I’m not so sure about, but definitely interesting.
  • If you are one of the few people living under a rock who doesn’t know about PowerCLI, Alan Renouf is here to help.

It’s time to wrap up; this post has already run longer than usual. There was just so much information that I want to share with you! I’ll be back soon-ish with another post, but until then feel free to join (or start) the conversation by adding your thoughts, ideas, links, or responses in the comments below.

Tags: , , , , , , , , , , , ,

« Older entries § Newer entries »