Reducing the Friction: Using Keyboard Shortcuts

Friday, June 14, 2013 in General, Macintosh, Personal by slowe | No comments

One thing that I personally have found extremely helpful in making common tasks easier—a key aspect of the “reducing the friction” idea I’ve been discussing—is the use of keyboard shortcuts. In this post, I want to talk about how you might be able to maximize your use of keyboard shortcuts to help improve your personal efficiency.

In my view, there are three aspects to using keyboard shortcuts more extensively:

  1. Using a keyboard-based launcher/search tool
  2. Using a system-wide hotkey tool
  3. Using (and/or customizing) application-specific keyboard shortcuts

Let’s take a quick look at each of these.

Keyboard-Based Launcher/Search Tool

I don’t think it’s any secret that I’m a fan of Quicksilver, regarded by many to be the quintessential tool for OS X in this category. There are other tools in this category, of course; examples include LaunchBar and Alfred. The key value that all these tools add to an existing system is the ability to quickly and easily search for something or launch an application without having to take your hands off the keyboard. Most of these tools also take an “object-oriented” approach that lets you use them to open URLs, open documents or folders, perform web searches, access browser bookmarks, and more.

System-Wide Hotkey Tool

While tools such as Quicksilver (or LaunchBar or Alfred) have some of this functionality as well, I’m specifically thinking of a utility like FastScripts. FastScripts allow you to bind AppleScripts to keyboard shortcuts, either on a system-wide basis or on an application-specific basis. There are other tools similar to this, but this is one with which I am very familiar (I think Keyboard Maestro offers similar functionality, plus other features as well.)

Here are some potential ways you could use this tool to help reduce the friction:

  • Perhaps you want to be able to compose a new e-mail message from any application on your system. You could write an AppleScript that launches Mail.app and composes a new message, then use FastScripts to create a system-wide shortcut key. Now, in whatever application you’re in, you can quickly and easily bring up a window to send a new e-mail message—all without taking your hands off the keyboard or having to switch between applications.
  • Maybe you want to make it even easier to launch a particular application. You could write a very simple AppleScript that launches the application, then use FastScripts or its equivalent to create a system-wide hotkey to launch the app. (This sort of functionality is especially useful if you aren’t using a tool like Quicksilver, LaunchBar, or Alfred.)
  • Like with an application, an AppleScript to open a commonly-accessed folder is pretty simple. Think about being able to use a quick keyboard shortcut to access folders that you frequently use, without having to navigate through the folder structure or switch applications to get there. Handy!

Application-Specific Keyboard Shortcuts

There are two parts to this particular aspect of using keyboard shortcuts:

  1. Using something like FastScripts, but on an application-specific basis
  2. Using OS X’s System Preferences application to customize keyboard shortcuts more to your liking

Use case #1 I’ve kind of already described in the previous section, so I won’t go into it again. There are lots of potential use cases here—creating a keyboard shortcut for an AppleScript that archives the selected messages is one example that might help accelerate your e-mail processing workflow.

Use case #2 is a bit more interesting, I think. A little-known fact is that you can use System Preferences to create new keyboard shortcuts for specific features within applications as well as customize existing keyboard shortcuts. Here’s an example. In some of my applications, the Export command (for saving a file in a different format) used Option-Command-E, but in other applications it didn’t. Since keyboard shortcuts are a little like muscle memory, using the same keyboard shortcut across multiple applications makes it easier and simpler. So, I used System Preferences so that more of the applications I use regularly have the same keyboard shortcut (where possible) for the Export function.

I hope some of these ideas are useful, or that they spark new ideas you might have about how you might reduce the friction in your own workflows. I’d love to hear any ideas or suggestions that other people have, so please speak up in the comments below. Thanks!

Tags: , ,

Resources for Networking and Network Virtualization

Wednesday, June 12, 2013 in Networking, Virtualization by slowe | 1 comment

Good friend (and now colleague) John Troyer asked me via e-mail if I had a “reading list” for networking and network virtualization. I didn’t have a formally-prepared “reading list,” so I just shared with John some of the resources that I personally use. John’s response was: “You should blog this.” So, here it is—a list of resources for networking and network virtualization.

  • Brent Salisbury, aka @networkstatic, has plenty of good material on his site.
  • Jason Edelman (@jedelman8 on Twitter) is also worth reading.
  • Network Heresy is a great site. In the spirit of transparency, note that Network Heresy is maintained by folks from VMware/Nicira, such as Dr. Bruce Davie, Martin Casado, and others.
  • Brad Hedlund (@bradhedlund if you’d like to follow him on Twitter) is another great networking and network virtualization resource. Since Brad works with me at VMware, the focus of most of his material is on network virtualization.
  • Ivan Pepelnjak (@ioshints) is a legend among networking folk—I’ve learned a ton from reading his stuff.
  • Greg Ferro (@etherealmind) also shares tons of great networking material on his site.
  • I’ve also found some useful material at Nick Buraglio’s site.
  • Finally, we’re hoping to make VMware’s new Network Virtualization blog a useful destination for networking and network virtualization material. There are also some additional sites on the blogroll there.

Naturally, any list like this is incomplete (my apologies if you weren’t included, I’m sure it was an oversight not an intentional omission!), so I invite readers to share resources they’ve found useful in the comments below. Courteous comments are always welcome, although I do ask for disclosure of vendor affiliations where applicable.

Tags: ,

The Symbiosis Between Automation and Engineering

Monday, June 10, 2013 in General by slowe | 5 comments

A little while ago Steve Beaver wrote a post titled “Is Automation Killing The Engineering?” In the post, Steve ponders whether the increased use of automation in today’s data centers is killing engineering knowledge. The argument, as I understand it, says that because tasks are becoming increasingly automated, data center professionals are increasingly less knowledgeable about how things actually work. But is this a valid argument? Is automation killing engineering?

I think there are at least two aspects to this idea that are worth exploring:

  1. On one hand, increased attention to and use of automation is quite likely enabling some IT professionals to do things they weren’t able to do manually. For example, I might not know very much about GlusterFS, but using a Puppet module for GlusterFS I could get it installed and configured without actually having to learn how it is done. (I think this is the trend that Steve is picking up on in his post.)

  2. On the other hand, an increased focus on automation and configuration management is enabling IT professionals to more quickly accomplish things that otherwise would have taken more time and effort. Using the same example from before, while I might know exactly how to configure GlusterFS and get it up and running manually, using a Puppet module to do so makes the process quicker, more scalable, and the results more consistent.

It’s up to us, as an industry and a profession, to ensure that we find—and maintain—the right balance between depth of engineering knowledge and extent of automation in our data centers. While the meme says “Automate all the things!”, we have to ask ourselves, what does it make sense to automate? Further, we have to ensure that we are holding ourselves accountable to know how the automation works. See, the pendulum can swing too far in either direction. We can carefully engineer and hand-craft our solutions (snowflake servers, anyone?), but that’s not the most scalable approach. We can also blindly use automation tools without understanding how things work—in which case the solution might be up and running, but is it running well? Is it optimized? Is it actually meeting the requirements it needs to meet? We don’t know, because we’ve allowed the pendulum to swing too far in the opposite direction.

I believe there is a careful symbiosis between engineering and automation that we must maintain. Yes, we should use automation—it’s a force multiplier that brings scalability and consistency. However, in order to ensure these consistent configurations are the correct configurations, we need to have the right depth of engineering knowledge. We must ensure that we understand both how the automation tools work as well as what the tools are doing.

In other words, we have to live by this phrase I posted to Twitter some time ago (the individual tweet is long gone in the “stream of consciousness” that is Twitter, so I can’t include a link):

“You can’t automate something if you don’t understand it.”

So, my advice to you: Start with automating what you know. Codify your existing engineering knowledge. Then, use the time freed up by those efforts to expand your knowledge, allowing you to automate new things once you understand them. This, I think, allows us to strike the correct balance between automation and engineering.

Disagree with what I’ve said here? That’s OK—share your views, perspectives, or thoughts in the comments below. All courteous comments are welcome!

Tags: ,

Reducing the Friction: Automating File Management

Friday, June 7, 2013 in General, Macintosh, Personal by slowe | 1 comment

One of the key things you can do to help improve your efficiency is “reduce the friction”—make common tasks easier, simpler, faster. (This isn’t my phrase or idea, by the way, but I’ve forgotten exactly where I first saw it.) In my last Reducing the Friction post, I talked about how I use AppleScript to help automate part of my blog publishing process. In this post, I’ll share how I help automate some common file management tasks.

The tool that I use here is Hazel, a nifty rules-based file management tool. Hazel uses a set of “if-then” rules to automatically perform tasks on a file based on a set of conditions. I use Hazel for a number of things, but the primary use case I’ll highlight here is automatically archiving files. I use this same technique in two different ways:

  1. To automatically archive files I no longer need/want
  2. To automatically archive published blog posts

The basic rule is the same in both instances, although the trigger is different. Here’s a screenshot of the Hazel rule for the first usage, archiving files I no longer need/want:

Hazel rule screenshot

Hazel rule for archiving files

Let me break that down real quick:

  • The rule condition states that a file must have been assigned the blue color label in the Finder (which, in my system, is what I use to mark a file as no longer needed/wanted) in order for the rule to be fired.
  • Once the rule is fired, then Hazel will move the file to the Archive-Pending folder and clear the color label from the file.

So, to archive files to my Archive-Pending folder (which I periodically move off my laptop to my home NAS for longer-term storage), I simply assign it the blue color label and Hazel does the rest. By the way, you’ll note that this rule is attached to the ~/Documents folder.

I use the same basic rule, with one small tweak, for the second usage I listed (automatically archiving published blog posts):

Hazel rule screenshot

Hazel rule for archiving published blog posts

As you can see, the rule actions are the same (move the file to the Archive-Pending folder and clear the color label), but the trigger is different. In this particular case, I use the green color label, which I’ve named “Finished” in my Finder preferences, to trigger the rule action. This rule is attached to my Blog-Drafts folder.

So, when I’m done with a blog post, I mark the Markdown file with the Finished (green) label, and Hazel automatically moves the post to my Archive-Pending folder. This helps ensure that the Blog-Drafts folder only contains unpublished/in-progress drafts, and also helps ensure that I have an archive of all my blog posts (an archive that is separate from backups of the WordPress database). This second reason, by the way, is one of the reasons I use the publishing process I use instead of just using the WordPress editor (although that is a perfectly fine way of doing it, if that works for you).

So what’s next? Hazel has the ability to watch a file’s activity (based on last opened or last modified dates), so in theory I could have Hazel start automatically moving “old” or “inactive” files to the Archive-Pending folder as well. I’m a bit hesitant to do that right now, but it is an option I might explore in the future.

I’d love to hear any other thoughts readers might have on “reducing the friction” through automating parts of the file management process (either with Hazel or with other tools). Feel free to add your thoughts, ideas, or suggestions in the comments below.

Tags: ,

Technology Short Take #33

Tuesday, June 4, 2013 in Networking, Security, Storage, Virtualization by slowe | 1 comment

Welcome to Technology Short Take #33, the latest in my irregularly-published series of articles discussing various data center technology-related links, articles, rants, thoughts, and questions. I hope that you find something useful here. Enjoy!

Networking

  • Tom Nolle asks the question, “Is virtualization reality even more elusive than virtual reality?” It’s a good read; the key thing that I took away from it was that SDN, NFV, and related efforts are great, but what we really need is something that can pull all these together in a way that customers (and providers) reap the benefits.
  • What happens when multiple VXLAN logical networks are mapped to the same multicast group? Venky explains it in this post. Venky also has a great write-up on how the VTEP (VXLAN Tunnel End Point) learns and creates the forwarding table.
  • This post by Ranga Maddipudi shows you how to use App Firewall in conjunction with VXLAN logical networks.
  • Jason Edelman is on a roll with a couple of great blog posts. First up, Jason goes off on a rant about network virtualization, briefly hitting topics like the relationship between overlays and hardware, the role of hardware in network virtualization, the changing roles of data center professionals, and whether overlays are the next logical step in the evolution of the network. I particularly enjoyed the snippet from the post by Bill Koss. Next, Jason dives a bit deeper on the relationship between network overlays and hardware, and shares his thoughts on where it does—and doesn’t—make sense to have hardware terminating overlay tunnels.
  • Another post by Tom Nolle explores the relationship—complicated at times—between SDN, NFV, and the cloud. Given that we define the cloud (sorry to steal your phrase, Joe) as elastic, pooled resources with self-service functionality and ubiquitous access, I can see where Tom states that to discuss SDN or NFV without discussing cloud is silly. On the flip side, though, I have to believe that it’s possible for organizations to make a gradual shift in their computing architectures and processes, so one almost has to discuss these various components individually, because to tie them all together makes it almost impossible. Thoughts?
  • If you haven’t already introduced yourself to VXLAN (one of several draft protocols used as an overlay protocol), Cisco Inferno has a reasonable write-up.
  • I know Steve Jin, and he’s a really smart guy. I must disagree with some of his statements regarding what software-defined networking is and is not and where it fits, written back in April. I talked before about the difference between network virtualization and SDN, so no need to mention that again. Also, the two key flaws that Steve identifies—single point of failure and scalability—aren’t flaws with SDN/network virtualization, but rather flaws in an implementation of said technologies, IMHO.

Servers/Hardware

  • Correction from the last Technology Short Take—I incorrectly stated that the HP Moonshot offerings were ARM-based, and therefore wouldn’t support vSphere. I was wrong. The servers (right now, at least) are running Intel Atom S1260 CPUs, which are x86-based and do offer features like Intel VT-x. Thanks to all who pointed this out, and my apologies for the error!
  • I missed this on the #vBrownBag series: designing HP Virtual Connect for vSphere 5.x.

Security

Cloud Computing/Cloud Management

  • Hyper-V as hypervisor with OpenStack Compute? Sure, see here.
  • Cody Bunch, who has been focusing quite a bit on OpenStack recently, has a nice write-up on using Razor and Chef to automate an OpenStack build. Part 1 is here; part 2 is here. Good stuff—keep it up, Cody!
  • I’ve mentioned in some of my OpenStack presentations (see SpeakerDeck or Slideshare) that a great place to start if you’re just getting started is DevStack. Here, Brent Salisbury has a nice write-up on using DevStack to install OpenStack Grizzly.

Operating Systems/Applications

  • Boxen, a tool created by GitHub to manage their OS X Mountain Lion laptops for developers, looks interesting. Might be a useful tool for other environments, too.
  • If you use TextMate2 (I switched to BBEdit a little while ago after being a long-time TextMate user), you might enjoy this quick post by Colin McNamara on Puppet syntax highlighting using TextMate2.

Storage

  • Anyone have more information on Jeda Networks? They’ve been mentioned a couple of times on GigaOm (here and here), but I haven’t seen anything concrete yet. Hey, Stephen Foskett, if you’re reading: get Jeda Networks to the next Tech Field Day.
  • Tim Patterson shares some code from Luc Dekens that helps check VMFS version and block sizes using PowerCLI. This could come in quite handy in making sure you know how your datastores are configured, especially if you are in the midst of a migration or have inherited an environment from someone else.

Virtualization

  • Interested in using SAML and Horizon Workspace with vCloud Director? Tom Fojta shows you how.
  • If you aren’t using vSphere Host Profiles, this write-up on the VMware SMB blog might convince you why you should and show you how to get started.
  • Michael Webster tackles the question: is now the best time to upgrade to vSphere 5.1? Read the full post to see what Michael has to say about it.
  • Duncan points out an easy error to make when working with vSphere HA heartbeat datastores in this post. Key takeaway: sometimes the fix is a lot simpler than we might think at first. (I know I’m guilty of making things more complicated than they need to be at times. Aren’t we all?)
  • Jon Benedict (aka “Captain KVM”) shares a script he wrote to help provide high availability for RHEV-M.
  • Chris Wahl has a nice write-up on using log shipping to protect your vCenter database. It’s a bit over a year old (surprised I missed it until now), and—as Chris points out—log shipping doesn’t protect the database (primary and secondary copies) against corruption. However, it’s better than nothing (which I suspect it what far too many people are using).

Other

  • If you aspire to be a writer—whether that be a blogger, author, journalist, or other—you might find this article on using the DASH method for writing to be helpful. The six tips at the end of the article are especially helpful, I think.

Time to wrap this up for now; the rest will have to wait until the next Technology Short Take. Until then, feel free to share your thoughts, questions, or rants in the comments below. Courteous comments are always welcome!

Tags: , , , , , , , , , , , , , ,

Reducing the Friction: BBEdit to MarsEdit

Friday, May 31, 2013 in General, Macintosh, Personal by slowe | 6 comments

In some of the presentations that I give on productivity and efficiency, one of the things I mention is reducing the friction; that is, making processes more streamlined so they’re easier to perform. In this post, I’m going to describe one way I reduced the friction for producing and publishing blog posts using BBEdit, TextSoap, MarsEdit, and some AppleScript.

It’s no secret that I’ve become a huge fan of Markdown, the human-readable plain text “markup” format created by Jon Gruber. The vast majority of all my content is now created in Markdown, and then converted to RTF (to share with my Office-using co-workers), PDF (for broader publication), or HTML (for publishing online). Until very recently, my blog publishing process looked something like this:

  1. Write the blog post in Markdown, using TextMate.
  2. Using a built-in Markdown binary in TextMate, convert the Markdown into HTML.
  3. Run the raw HTML through TextSoap (very handy tool) to remove smart quotes and curly apostrophes.
  4. Paste the parsed HTML into MarsEdit for publication to my blog.

While it seems complicated, it wasn’t terribly complicated—but it wasn’t as seamless as it could be. So, I set out to improve the process. The first big change was a switch from TextMate to BBEdit, which is more extensible (and kept up to date by the developer). That change allowed me to do two things:

  • Switch from the built-in Markdown support in TextMate to using a separate (and more up-to-date) MultiMarkdown binary maintained by Fletcher Penny.
  • Introduce an AppleScript (BBEdit has outstanding support for AppleScript) to automate some portion of the process.

My first pass at automating the process just got me back to where I was before—writing Markdown in BBEdit, converting to HTML, cleaning the HTML with TextSoap, and pasting into MarsEdit. Not too impressive, but acceptable, and a process with which I was familiar. I stuck with that process for a while, primarily because it was a known entity. A couple of days ago, though, I asked myself: Can I do better? Can I be more efficient?

So, my second pass at automating the process is much more comprehensive. The AppleScript I wrote as a result of challenging myself to reduce the friction does the following:

  • Takes the Markdown from BBEdit and converts it to HTML.
  • Using the HTML produced by the standalone MultiMarkdown binary, it then calls TextSoap to (in the background) clean the HTML according to a custom cleaner I’d created (the custom cleaner, called “Replace HTML Entities,” just replaces curly quotes and curly apostrophes, which don’t translate well on my site).
  • Creates a new, blank blog post in MarsEdit, into which it pastes the cleaned HTML as the body of the post.

I store the script in BBEdit’s scripts folder, which means I can invoke the script easily from within BBEdit.

Here’s the AppleScript itself (click here if the script doesn’t show up):

Now, I can write my Markdown in BBEdit, invoke the script, and get dropped out to HTML code sitting in a new blog post in MarsEdit. All I need to do to publish the post at that point is supply the metadata (tags, categories, title, excerpt) and click Send to Blog. Done. (I used this process for this post, in fact.) How’s that for reduced friction?

Tags: , , ,

A Use Case for Policy Routing with KVM and Open vSwitch

Thursday, May 30, 2013 in Linux, Networking, Virtualization by slowe | 2 comments

In an earlier post, I provided an introduction to policy routing as implemented in recent versions of Ubuntu Linux (and possibly other distributions as well), and I promised that in a future post I would provide a practical application of its usage. This post looks at that practical application: how—and why—you would use Linux policy routing in an environment running OVS and a Linux hypervisor (I’ll assume KVM for the purposes of this post).

Before I get into the “how,” let’s first discuss the “why.” Let’s assume that you have a KVM+OVS environment and are leveraging tunnels (GRE or other) for some guest domain traffic. Recall from my post on traffic patterns with Open vSwitch that tunnel traffic is generated by the OVS process itself, and therefore is controlled by the Linux host’s IP routing table with regard to which interfaces that tunnel traffic will use. But what if you need the tunnel traffic to be handled differently than the host’s management traffic? What if you need a default route for tunnel traffic that uses one interface, but a different default route for your separate management network that uses its own interface? This is why you would use policy routing in this configuration. Using source routing (i.e., policy routing based on the source of the traffic), you could easily define a table for tunnel traffic that has its own default route while still allowing management traffic to use the host’s default routing table.

Let’s take a look at how it’s done. In this example, I’ll make the following assumptions:

  • I’ll assume that you’re running host management traffic through OVS, as I outlined here. I’ll use the name mgmt0 to refer to the management interface that’s running through OVS for host management traffic. We’ll use the IP address 192.168.100.10 for the mgmt0 interface.
  • I’ll assume that you’re running tunnel traffic through an OVS interface interface named tep0. (This helps provide some consistency with my walk-through on using GRE tunnels with OVS.) We’ll use the IP address 192.168.200.10 for the tep0 interface.
  • I’ll assume that the default gateway on each subnet uses the .1 address on that subnet.

With these assumptions out of the way, let’s look at how you would set this up.

First, you’ll create a custom policy routing table, as outlined here. I’ll use the name “tunnel” for my new table:

echo 200 tunnel >> /etc/iproute2/rt_tables

Next, you’ll need to modify /etc/network/interfaces for the tep0 interface so that a custom policy routing rule and custom route are installed whenever this interface is brought up. The new configuration stanza would look something like this:

(If the configuration stanza doesn’t appear above, click here.)

Finally, you’ll want to ensure that mgmt0 is properly configured in /etc/network/interfaces. No special configuration is required there, just the use of the gateway directive to install the default route. Ubuntu will install the default route into the main table automatically, making it a “system-wide” default route that will be used unless a policy routing rule dictates otherwise.

With this configuration in place, you now have a system that:

  • Can communicate via mgmt0 with other systems in other subnets via the default gateway of 192.168.100.1.
  • Can communicate via tep0 to establish tunnels with other hypervisors in other subnets via the 192.168.200.1 gateway.

This configuration requires only the initial configuration (which could, quite naturally, be automated via a tool like Puppet) and does not require using additional routes as the environment scales to include new subnets for other hypervisors (either for management or tunnel traffic). Thus, organizations can use recommended practices for building scalable L3 networks with reasonably-sized L2 domains without sacrificing connectivity to/from the hypervisors in the environment.

(By the way, this is something that is not easily accomplished in the vSphere world today. ESXi has only a single routing table for all VMkernel interfaces, which means that management traffic, vMotion traffic, VXLAN traffic, etc., are all bound by that single routing table. To achieve full L3 connectivity, you’d have to install specific routes into the VMkernel routing table on each ESXi host. When additional subnets are added for scale, each host would have to be touched to add the additional route.)

Hopefully this gives you an idea of how Linux policy routing could be effectively used in environments leveraging virtualization, OVS, and overlay protocols. Feel free to add your thoughts, idea, corrections, or questions in the comments below. Courteous comments are always welcome! (Please disclose vendor affiliations where applicable.)

Tags: , , , , ,

A Quick Introduction to Linux Policy Routing

Wednesday, May 29, 2013 in Linux, Networking by slowe | 3 comments

In this post, I’m going to introduce you to policy routing as implemented in recent versions of Ubuntu Linux (and possibly other Linux distributions as well, but I’ll be using Ubuntu 12.04 LTS). Policy routing actually allows us a great deal of flexibility in how we direct traffic out of a Linux host; I’ll discuss a rather practical application of this configuration in a future blog post. For now, though, let’s just focus on how to configure policy routing.

There are a couple parts involved in policy routing:

  • Policy routing tables: Linux comes with three by default: local (which cannot be modified or deleted), main, and default. Somewhat unintuitively, routes added to the system without a routing table specified go to the main table, not the default table.
  • Policy routing rules: Again, Linux comes with three rules, one for each of the default routing tables.

In order for us to leverage policy routing for our purposes, we need to do three things:

  1. We need to create a custom policy routing table.
  2. We need to create one or more custom policy routing rules.
  3. We need to populate the custom policy routing table with routes.

Let’s look at each of these steps separately.

Creating a Custom Policy Routing Table

The first step is to create a custom policy routing table. Each table is represented by an entry in the file /etc/iproute2/rt_tables, so creating a new table is generally accomplished using a command like this:

echo 200 custom >> /etc/iproute2/rt_tables

This creates the table with the ID 200 and the name “custom”. You’ll reference this name later as you create the rules and populate the table with routes, so make note of it. Because this entry is contained in the rt_tables file, it will be persistent across reboots.

Creating Policy Routing Rules

The next step is to create the policy routing rules that will tell the system which table to use to determine the correct route. In this particular case, I’m going to use the source address (i.e., the originating address for the traffic) as the determining factor in the rule. This is a common application of policy routing, and for that reason it’s often referred to as source routing.

To create the policy routing rule, use this command:

ip rule add from <source address> lookup <table name>

Let’s say that we wanted to create a rule that told the system to use the “custom” table we created earlier for all traffic originating from the source address 192.168.30.200. The command would look like this:

ip rule add from 192.168.30.200 lookup custom

You can see all the policy routing rules that are currently in effect using this command:

ip rule list

As I mentioned in the beginning of this article, there are default rules that govern the use of the local, main, and default tables (these are the built-in tables). Once you’ve added your rule, you should see it listed there as well.

There is a problem here, though: rules created this way are ephemeral and will disappear when the system is restarted (or when the networking is restarted). To make the rules persist, add a line like this to /etc/network/interfaces:

post-up ip rule add from 192.168.30.200 lookup custom

You’d want to place this line in the configuration stanza that configures the interface with the address 192.168.30.200. With this line in place, the rule should persist across reboots or across network restarts.

Populating the Routing Table

Once we have the custom policy routing table created and a rule defined that directs the system to use it, we need to populate the table with the correct routes. The generic command to do this is the ip route add command, but with a specific table parameter added.

Using our previous example, let’s say we wanted to add a default route that was specific to traffic originating from 192.168.30.200. We’ve already created a custom policy routing table, and we have a rule that directs the system to use that table for traffic originating from that address. To add a new default route specifically for that interface, you’d use this command:

ip route add default via 192.168.30.1 dev eth1 table custom

Naturally, you’d want to substitute the correct default gateway for 192.168.30.1 and the correct interface for eth1 in the above command, but this should give you the right idea. Of course, you don’t have to use default routes; you could install specific routes into the custom policy routing table as well. This also works on VLAN sub-interfaces, so you could create per-VLAN routing tables:

ip route add default via 192.168.30.1 dev eth0.30 table vlan30

This command installs a default route for the 192.168.30.x interface on VLAN 30, using a table named “vlan30″ (note that the table needs to created before you can add routes to it, as far as I can tell).

As with the policy routing tables, routes added this way are not persistent, so you’ll want to make them persistent by adding a line like this to your /etc/network/interfaces configuration file:

post-up ip route add default via 192.168.30.1 dev eth1 table custom

This will ensure that the appropriate routes are added to the appropriate policy routing table when the corresponding network interface is brought up.

Summary

There’s a great deal more functionality possible in policy routing, but this at least gives you the basics you need to understand how it works. In a future post, I’ll provide a specific use case where this functionality could be put to work. In the meantime, feel free to share any corrections, clarifications, questions, or thoughts in the comments below.

Tags: , ,

VLAN Trunking to Guest Domains with Open vSwitch

Tuesday, May 28, 2013 in Linux, Networking by slowe | 4 comments

In other articles, I’ve talked about how to use Open vSwitch (OVS) with VLANs to place guest domains (VMs) into a particular VLAN. In this article, I want to show you how to pass VLAN tags all the way into the guest domain—in other words, how to do VLAN trunking to guest domains using OVS. To do this, we’re going to leverage the OVS-libvirt integration I referenced in this post on using VLANs with OVS and libvirt.

For this to work, you must have an operating system in the guest domain that is capable of recognizing and using the VLAN tags that are being passed to it by OVS. In this article, I’ll use Ubuntu 12.04 as the OS in the guest domain. For other operating systems, the commands and/or procedures to configure VLAN support appropriately will probably differ, so keep that in mind.

There are two parts to making this work:

  1. Configuring OVS (manually or via libvirt) to pass VLAN tags to the guest OS.
  2. Configuring the guest domain’s installed OS to take advantage of the VLAN tags being passed up by OVS.

Let’s look at each of these parts separately. We’ll start with configuring OVS, either manually or via libvirt, to pass the VLAN tags up to the guest domain.

Configuring OVS to Pass VLAN Tags to the Guest Domain

There are two ways to accomplish this: you can do it manually, or you can do it via OVS integration with recent builds of libvirt.

Manually Configuring OVS

To configure OVS manually, you would need to:

  1. Identify which vnet port you want to configure for VLAN trunking
  2. Configure the vnet port to trunk the VLANs.

To identify which vnet port needs to be modified, you’ll want to figure out the guest domain interface(s) that is/are connected to the vnet port. You can do this by using this command (substitute the desired vnet port name in place of vnet0 in the following command):

ovs-vsctl list interface vnet0

In the output of the command, look for the external_ids line; it will contain an entry called “attached-mac”, and that represents the MAC address of the interface in the guest domain OS attached to this particular vnet port. You can compare this to the output of ip addr list or ifconfig -a in Ubuntu to find a matching MAC address in the guest domain. Correlating the two values allows you to determine which guest domain is attached to which vnet port, and then you can modify the correct vnet port appropriately.

You’d modify the vnet port using this command:

ovs-vsctl set port vnet0 trunks=20,30,40

You’d want to substitute the appropriate values for vnet0 and the VLAN IDs that you want passed up to the guest domain. Once you’ve made the change, you can verify the changes using this command (replacing vnet0 with the correct port):

ovs-vsctl list port vnet0

Note that if you want the guest domain to receive both untagged (native VLAN) traffic as well as tagged (trunked) traffic, there is an additional setting you must set:

ovs-vsctl set port vnet0 vlan_mode=native-untagged

With this setting in place, the OS installed into the guest domain will be able to communicate over the untagged (native) VLAN as well as using VLAN tags.

Using libvirt Integration

If the manual method of configuring OVS seems a bit cumbersome, using the libvirt integration makes it much easier.

Basically, you’ll follow the configuration outlined in this blog post to create a libvirt network that corresponds to an OVS bridge. Here’s an example of the XML code to accomplish this task:

Of particular interest for what we’re trying to accomplish here is the very last section, the portgroup named “vlan-all.” Note that for this specific portgroup, the vlan element has a property that specifies it is a trunk, and then there are multiple tag elements that list each VLAN ID that will be trunked across this network into the guest domain.

Using this configuration, when we create the guest domain and specify that it is attached to the network named “vlan-all” (matching the portgroup in the libvirt network definition), libvirt will automatically configure OVS appropriately (it will set the trunks value for that domain’s OVS port).

However, it will not configure the OVS port to allow untagged traffic as well (only tagged traffic will be passed). If you want the guest domain to receive untagged traffic also, you must set the vlan_mode value manually as outlined above.

Configuring the Guest Domain to Use VLAN Tags

Once you’ve followed the steps outlined above and have OVS configured correctly, then you’re ready to configure the OS in the guest domain. Keep in mind that I’m using Ubuntu 12.04 in this post, but you’re welcome to use any operating system that supports VLAN tags.

Assuming that eth0 is the interface in the guest domain that is receiving tagged traffic from OVS, this snippet in /etc/network/interfaces will create and configure a VLAN interface:

Technically, the “raw-vlan-device” line isn’t needed because the parent device name is in the name of the VLAN device, but I like to include it for completeness and ease of debugging. (Your mileage may vary, of course.) The number on the end of the eth0 (for example, eth0.20) corresponds to the VLAN ID (VLAN 20, in this case) being passed up by OVS.

You can repeat this configuration for multiple VLAN interfaces.

Use Case

I’ll have to admit that I can’t immediately think of some useful use cases for this sort of configuration. At first glance, you might think that it would be useful in situations where you need logical separation, but I think there are better ways than VLANs to accomplish this task (and those ways are probably simpler). I primarily set out to document this in order to better solidify my knowledge of how OVS works and is configured. However, I’d be happy to hear from others on what they think might be interesting or useful use cases for this sort of configuration. Feel free to add your thoughts in the comments below. Courteous comments are always welcome!

Tags: , , , ,

Learning NVP, Part 1: High-Level Architecture

Tuesday, May 21, 2013 in Networking, Virtualization by slowe | 12 comments

This blog post kicks off a new series of posts describing my journey to become more knowledgeable about the Nicira Network Virtualization Platform (NVP). NVP is, in my opinion, an awesome platform, but there hasn’t been a great deal of information shared about the product, how it works, how you configure it, etc. That’s something I’m going to try to address in this series of posts. In this first post, I’ll start with a high-level description of the NVP architecture. Don’t worry—more in-depth information will come in future posts.

Before continuing, it might be useful to set some context around NVP and NSX. This series of posts will focus on NVP—a product that is available today and is currently in use in production. The architecture I’m describing here will also be applicable to NSX, which VMware announced in early March. Because NSX will leverage NVP’s architecture, spending some time with NVP now will pay off with NSX later. Make sense?

Let’s start with a figure. The diagram below graphically illustrates the NVP architecture at a high level:

High-level NVP architecture diagram

The key components of the NVP architecture include:

  • A scale-out controller cluster: The NVP controllers handle computing the network topology and providing configuration and flow information to create logical networks. The controllers support a scale-out model for high availability and increased scalability. The controller cluster supplies a northbound REST API that can be consumed by cloud management platforms such as OpenStack or CloudStack, or by home-grown cloud management systems.
  • A programmable virtual switch: NVP leverages Open vSwitch (OVS), an independent open source project with contributors from across the industry, to fill this role. OVS communicates with the NVP controller clusters to receive configuration and flow information.
  • Southbound communications protocols: NVP uses two open communications protocols to communicate southbound to OVS. For configuration information, NVP leverages OVSDB; for flow information, NVP uses OpenFlow. The management (OVSDB) communication between the controller cluster and OVS is encrypted using SSL.
  • Gateways: Gateways provide the “on-ramp” to enter or exit NVP logical networks. Gateways can provide either L2 gateway services (to bridge NVP logical networks onto physical networks) as well as L3 gateway services (to route between NVP logical networks and physical networks). In either case, the gateways are also built using a scale-out model that provides high availability and scalability for the L2 and L3 gateway services they host.
  • Encapsulation protocol: To provide full independence and isolation of logical networks from the underlying physical networks, NVP uses encapsulation protocols for transporting logical network traffic across physical networks. Currently, NVP supports both Generic Routing Encapsulation (GRE) and Stateless Transport Tunneling (STT), with additional encapsulation protocols planned for future releases.
  • Service nodes: To offload the handling of BUM (Broadcast, Unknown Unicast, and Multicast) traffic, NVP can optionally leverage one or more service nodes. Note that service nodes are optional; customers can choose to have BUM traffic handled locally on each hypervisor node. (Note that service nodes are not shown in the diagram above.)

Now that you have an idea of the high-level architecture, let me briefly outline how the rest of this series will be organized. The basic outline of this series will roughly correspond to how NVP would be deployed in a real-world environment.

  1. In the next post (or two), I’ll be focusing on getting the controller cluster built and diving a bit deeper into the controller cluster architecture.
  2. Once the controller cluster is up and running, I’ll take a look at getting NVP Manager up and running. NVP Manager is an application that consumes the northbound REST APIs from the controller cluster in order to view and manage NVP logical networks and NVP components. In most cases, this function is part of a cloud management platform (such as OpenStack or CloudStack), but using NVP Manager here allows me to focus on NVP instead of worrying about the details of the cloud management platform itself.
  3. The next step will be to bring hypervisor nodes into NVP. I’ll focus on using nodes running KVM, but keep in mind that Xen is also supported by NVP. If time (and resources) permit, I may try to look at bringing up Xen-based hypervisor nodes as well. Because NVP leverages OVS as the edge virtual switch, I’ll naturally be discussing some OVS-related tasks and topics as well.
  4. Following the addition of hypervisor nodes into NVP, I’ll look at creating a simple logical network, and we’ll examine how this logical network works with the underlying physical network.
  5. To add more flexibility to our logical network, we need to be able to bring physical resources into NVP logical networks. To enable that functionality, we’ll need to add gateways and gateway services to our configuration, so I’ll discuss gateways and L2 gateway services, how they work, and how we add them to an NVP configuration.
  6. The next step is to enable L3 (routing) functionality within NVP, and that is enabled by L3 gateway services. I’ll spend some time talking about the L3 gateway services, their architecture, adding them to NVP, and including L3 functionality in an NVP logical network. I’ll also explore distributed L3 routing, where the L3 routing is actually distributed across hypervisors in an NVP environment (this is a new feature just added in NVP 3.1).
  7. Now that we have both L2 and L3 gateway services in NVP, I’ll take a look at building more intricate logical networks.

Beyond that, it’s hard to say where the series will go. I’ll likely also take a look at some of NVP’s security features, and examine a few more complex NVP use cases. If there are additional topics you’d like to see beyond what I’ve outlined above, please feel free to speak up in the comments below.

I’m excited about this journey to learn NVP in more detail, and I’m looking forward to taking all of you along with me. Ready? Let’s go!

Tags: , , , , ,

« Older entries