CLI

You are currently browsing articles tagged CLI.

Using GRE Tunnels with Open vSwitch

Tuesday, May 7, 2013 in Interoperability, Linux, Networking, Virtualization by slowe | 15 comments

I’m back with another “how to” article on Open vSwitch (OVS), this time taking a look at using GRE (Generic Routing Encapsulation) tunnels with OVS. OVS can use GRE tunnels between hosts as a way of encapsulating traffic and creating an overlay network. OpenStack Quantum can (and does) leverage this functionality, in fact, to help separate different “tenant networks” from one another. In this write-up, I’ll walk you through the process of configuring OVS to build a GRE tunnel to build an overlay network between two hypervisors running KVM.

Naturally, any sort of “how to” such as this always builds upon the work of others. In particular, I found a couple of Brent Salisbury’s articles (here and here) especially useful.

This process has 3 basic steps:

  1. Create an isolated bridge for VM connectivity.
  2. Create a GRE tunnel endpoint on each hypervisor.
  3. Add a GRE interface and establish the GRE tunnel.

These steps assume that you’ve already installed OVS on your Linux distribution of choice. I haven’t explicitly done a write-up on this, but there are numerous posts from a variety of authors (in this regard, Google is your friend).

We’ll start with an overview of the topology, then we’ll jump into the specific configuration steps.

Reviewing the Topology

The graphic below shows the basic topology of what we have going on here:

Topology overview

We have two hypervisors (CentOS 6.3 and KVM, in my case), both running OVS (an older version, version 1.7.1). Each hypervisor has one OVS bridge that has at least one physical interface associated with the bridge (shown as br0 connected to eth0 in the diagram). As part of this process, you’ll create the other internal interfaces (the tep and gre interfaces, as well as the second, isolated bridge to which VMs will connect. You’ll then create a GRE tunnel between the hypervisors and test VM-to-VM connectivity.

Creating an Isolated Bridge

The first step is to create the isolated OVS bridge to which the VMs will connect. I call this an “isolated bridge” because the bridge has no physical interfaces attached. (Side note: this idea of an isolated bridge is fairly common in OpenStack and NVP environments, where it’s usually called the integration bridge. The concept is the same.)

The command is very simple, actually:

ovs-vsctl add-br br2

Yes, that’s it. Feel free to substitute a different name for br2 in the command above, if you like, but just make note of the name as you’ll need it later.

To make things easier for myself, once I’d created the isolated bridge I then created a libvirt network for it so that it was dead-easy to attach VMs to this new isolated bridge.

Configuring the GRE Tunnel Endpoint

The GRE tunnel endpoint is an interface on each hypervisor that will, as the name implies, serve as the endpoint for the GRE tunnel. My purpose in creating a separate GRE tunnel endpoint is to separate hypervisor management traffic from GRE traffic, thus allowing for an architecture that might leverage a separate management network (which is typically considered a recommended practice).

To create the GRE tunnel endpoint, I’m going to use the same technique I described in my post on running host management traffic through OVS. Specifically, we’ll create an internal interface and assign it an IP address.

To create the internal interface, use this command:

ovs-vsctl add-port br0 tep0 -- set interface tep0 type=internal

In your environment, you’ll substitute br2 with the name of the isolated bridge you created earlier. You could also use a different name than tep0. Since this name is essentially for human consumption only, use what makes sense to you. Since this is a tunnel endpoint, tep0 made sense to me.

Once the internal interface is established, assign it with an IP address using ifconfig or ip, whichever you prefer. I’m still getting used to using ip (more on that in a future post, most likely), so I tend to use ifconfig, like this:

ifconfig tep0 192.168.200.20 netmask 255.255.255.0

Obviously, you’ll want to use an IP addressing scheme that makes sense for your environment. One important note: don’t use the same subnet as you’ve assigned to other interfaces on the hypervisor, or else you can’t control that the GRE tunnel will originate (or terminate) on the interface you specify. This is because the Linux routing table on the hypervisor will control how the traffic is routed. (You could use source routing, a topic I plan to discuss in a future post, but that’s beyond the scope of this article.)

Repeat this process on the other hypervisor, and be sure to make note of the IP addresses assigned to the GRE tunnel endpoint on each hypervisor; you’ll need those addresses shortly. Once you’ve established the GRE tunnel endpoint on each hypervisor, test connectivity between the endpoints using ping or a similar tool. If connectivity is good, you’re clear to proceed; if not, you’ll need to resolve that before moving on.

Establishing the GRE Tunnel

By this point, you’ve created the isolated bridge, established the GRE tunnel endpoints, and tested connectivity between those endpoints. You’re now ready to establish the GRE tunnel.

Use this command to add a GRE interface to the isolated bridge on each hypervisor:

ovs-vsctl add-port br2 gre0 -- set interface gre0 type=gre \
options:remote_ip=<GRE tunnel endpoint on other hypervisor>

Substitute the name of the isolated bridge you created earlier here for br2 and feel free to use something other than gre0 for the interface name. I think using gre as the base name for the GRE interfaces makes sense, but run with what makes sense to you.

Once you repeat this command on both hypervisors, the GRE tunnel should be up and running. (Troubleshooting the GRE tunnel is one area where my knowledge is weak; anyone have any suggestions or commands that we can use here?)

Testing VM Connectivity

As part of this process, I spun up an Ubuntu 12.04 server image on each hypervisor (using virt-install as I outlined here), attached each VM to the isolated bridge created earlier on that hypervisor, and assigned each VM an IP address from an entirely different subnet than the physical network was using (in this case, 10.10.10.x).

Here’s the output of the route -n command on the Ubuntu guest, to show that it has no knowledge of the “external” IP subnet—it knows only about its own interfaces:

ubuntu:~ root$ route -n
Kernel IP routing table
Destination  Gateway       Genmask        Flags Metric Ref Use Iface
0.0.0.0      10.10.10.254  0.0.0.0        UG    100    0   0   eth0
10.10.10.0   0.0.0.0       255.255.255.0  U     0      0   0   eth0

Similarly, here’s the output of the route -n command on the CentOS host, showing that it has no knowledge of the guest’s IP subnet:

centos:~ root$ route -n
Kernel IP routing table
Destination  Gateway        Genmask        Flags Metric Ref Use Iface
192.168.2.0  0.0.0.0        255.255.255.0  U     0      0   0   tep0
192.168.1.0  0.0.0.0        255.255.255.0  U     0      0   0   mgmt0
0.0.0.0      192.168.1.254  0.0.0.0        UG    0      0   0   mgmt0

In my case, VM1 (named web01) was given 10.10.10.1; VM2 (named web02) was given 10.10.10.2. Once I went through the steps outlined above, I was able to successfully ping VM2 from VM1, as you can see in this screenshot:

VM-to-VM connectivity over GRE tunnel

(Although it’s not shown here, connectivity from VM2 to VM1 was obviously successful as well.)

“OK, that’s cool, but why do I care?” you might ask.

In this particular context, it’s a bit of a science experiment. However, if you take a step back and begin to look at the bigger picture, then (hopefully) something starts to emerge:

  • We can use an encapsulation protocol (GRE in this case, but it could have just as easily been STT or VXLAN) to isolate VM traffic from the physical network and from other VM traffic. (Think multi-tenancy.)
  • While this process was manual, think about some sort of controller (an OpenFlow controller, perhaps?) that could help automate this process based on its knowledge of the VM topology.
  • Using a virtualized router or virtualized firewall, I could easily provide connectivity into or out of this isolated (encapsulated) private network. (This is probably something I’ll experiment with later.)
  • What if we wrapped some sort of orchestration framework around this, to help deploy VMs, create networks, add routers/firewalls automatically, all based on the customer’s needs? (OpenStack Networking, anyone?)

Anyway, I hope this is helpful to someone. As always, I welcome feedback and suggestions for improvement, so feel free to speak up in the comments below. Vendor disclosures, where appropriate, are greatly appreciated. Thanks!

Tags: , , , , ,

Technology Short Take #29

Monday, February 4, 2013 in Networking, Security, Storage, Virtualization by slowe | 6 comments

Welcome to Technology Short Take #29! This is another installation in my irregularly-published series of links, thoughts, rants, and raves across various data center-related fields of technology. As always, I hope you find something useful here.

Networking

  • Who out there has played around with Mininet yet? Looks like this is another tool I need to add to my toolbox as I continue to explore networking technologies like OpenFlow, Open vSwitch, and others.
  • William Lam has a recent post on some useful VXLAN commands found in ESXCLI with vSphere 5.1. I’m a CLI fan, so I like this sort of stuff.
  • I still have a lot to learn about OpenFlow and networking, but this article from June of last year (it appears to have been written by Ivan Pepelnjak) discusses some of the potential scalability concerns around early versions of the OpenFlow protocol. In particular, the use of OpenFlow to perform granular per-flow control when there are thousands (or maybe only hundreds) of flows presents a scalability challenge (for now, at least). In my mind, this isn’t an indictment of OpenFlow, but rather an indictment of the way that OpenFlow is being used. I think that’s the point Ivan tried to make as well—it’s the architecture and how OpenFlow is used that makes a difference. (Is that a reasonable summary, Ivan?)
  • Brad Hedlund (who will be my co-worker starting on 2/11) created a great explanation of network virtualization that clearly breaks down the components and explains their purpose and function. Great job, Brad.
  • One of the things I like about Open vSwitch (OVS) is that it is so incredibly versatile. Case in point: here’s a post on using OVS to connect LXC containers running on different hosts via GRE tunnels. Handy!

Servers/Hardware

  • Cisco UCS is pretty cool in that it makes automation of compute hardware easier through such abstractions as server profiles. Now, you can also automate UCS with Chef. I traded a few tweets with some Puppet folks, and they indicated they’re looking at this as well.
  • Speaking of Puppet and hardware, I also saw a mention on Twitter about a Puppet module that will manage the configuration of a NetApp filer. Does anyone have a URL with more information on that?
  • Continuing the thread on configuration management systems running on non-compute hardware (I suppose this shouldn’t be under the “Servers/Hardware” section any longer!), I also found references to running CFEngine on network apliances and running Chef on Arista switches. That’s kind of cool. What kind of coolness would result from even greater integration between an SDN controller and a declarative configuration management tool? Hmmm…

Security

  • Want full-disk encryption in Ubuntu, using AES-XTS-PLAIN64? Here’s a detailed write-up on how to do it.
  • In posts and talks I’ve given about personal productivity, I’ve spoken about the need to minimize “friction,” that unspoken drag that makes certain tasks or workflows more difficult and harder to adopt. Tal Klein has a great post on how friction comes into play with security as well.

Cloud Computing/Cloud Management

  • If you, like me, are constantly on the search for more quality information on OpenStack and its components, then you’ll probably find this post on getting Cinder up and running to be helpful. (I did, at least.)
  • Mirantis—recently the recipient of $10 million in funding from various sources—posted a write-up in late November 2012 on troubleshooting some DNS and DHCP service configuration issues in OpenStack Nova. The post is a bit specific to work Mirantis did in integrating an InfoBlox appliance into OpenStack, but might be useful in other situation as well.
  • I found this article on Packstack, a tool used to transform Fedora 17/18, CentOS 6, or RHEL 6 servers into a working OpenStack deployment (Folsom). It seems to me that lots of people understand that getting an OpenStack cloud up and running is a bit more difficult than it should be, and are therefore focusing efforts on making it easier.
  • DevStack is another proof point of the effort going into make it easier to get OpenStack up and running, although the focus for DevStack is on single-host development environments (typically virtual themselves). Here’s one write-up on DevStack; here’s another one by Cody Bunch, and yet another one by the inimitable Brent Salisbury.

Operating Systems/Applications

  • If you’re interested in learning Puppet, there are a great many resources out there; in fact, I’ve already mentioned many of them in previous posts. I recently came across these Example42 Puppet Tutorials. I haven’t had the chance to review them myself yet, but it looks like they might be a useful resource as well.
  • Speaking of Puppet, the puppet-lint tool is very handy for ensuring that your Puppet manifest syntax is correct and follows the style guidelines. The tool has recently been updated to help fix issues as well. Read here for more information.

Storage

  • Greg Schulz (aka StorageIO) has a couple of VMware storage tips posts you might find useful reading. Part 1 is here, part 2 is here. Enjoy!
  • Amar Kapadia suggests that adding LTFS to Swift might create an offering that could give AWS Glacier a real run for the money.
  • Gluster interests me. Perhaps it shouldn’t, but it does. For example, the idea of hosting VMs on Gluster (similar to the setup described here) seems quite interesting, and the work being done to integrate KVM/QEMU with Gluster also looks promising. If I can ever get my home lab into the right shape, I’m going to do some testing with this. Anyone done anything with Gluster?
  • Erik Smith has a very informative write-up on why FIP snooping is important when using FCoE.
  • Via this post on ten useful OpenStack Swift features, I found this page on how to build the “Swift All in One,” a useful VM for learning all about Swift.

Virtualization

  • There’s no GUI for it, but it’s kind of cool that you can indeed create VM anti-affinity rules in Hyper-V using PowerShell. This is another example of how Hyper-V continues to get more competent. Ignore Microsoft and Hyper-V at your own risk…
  • Frank Denneman takes a quick look at using user-defined NetIOC network resource pools to isolate and protect IP-based storage traffic from within the guest (i.e., using NFS or iSCSI from within the guest OS, not through the VMkernel). Naturally, this technique could be used to “protect” or “enhance” other types of important traffic flows to/from your guest OS instances as well.
  • Andre Leibovici has a brief write-up on the PowerShell module for the Nicira Network Virtualization Platform (NVP). Interesting stuff…
  • This write-up by Falko Timme on using BoxGrinder to create virtual appliances for KVM was interesting. I might have to take a look at BoxGrinder and see what it’s all about.
  • In case you hadn’t heard, OVF 2.0 has been announced/released by the DMTF. Winston Bumpus of VMware’s Office of the CTO has more information in this post. I also found the OVF 2.0 frequently asked questions (FAQs) to be helpful. Of course, the real question is how long it will be before vendors add support for OVF 2.0, and how extensive that support actually is.

And that’s it for this time around! Feel free to share your thoughts, suggestions, clarifications, or corrections in the comments below. I encourage your feedback, and thanks for reading.

Tags: , , , , , , , , , , , , , , ,

Using Mock to Build Libvirt 1.0.1 RPMs for CentOS 6.3

Tuesday, January 22, 2013 in Linux, Virtualization by slowe | No comments

In this third post on using Mock to build RPMs for CentOS 6.3, I’m going to show you how to use Mock to build RPMs for Libvirt 1.0.1 that you can install on CentOS. As you’ll see later, this post builds on the previous two posts (one on using Mock to build RPMs for sanlock 2.4 and one on using Mock to build RPMs for libssh2 1.4.1).

Here’s a quick overview of the process:

  1. Set up Mock and the environment.
  2. Install prerequisites into the Mock environment.
  3. Build the Libvirt RPMs.

Let’s take a closer look at each of these steps.

Setting Up Mock and the Environment

The first phase in the process is to set up Mock and the environment for building the RPMs. Fortunately, this is relatively simple.

First, activate EPEL. My preferred method for activating the EPEL repository is to download the RPM, then use yum localinstall to install it, like this:

wget http://fedora.mirrors.pair.com/epel/6/i386/\
epel-release-6-8.noarch.rpm
yum localinstall epel-release-6-8.noarch.rpm

(Note that I’ve line-wrapped the URL with a backslash to make it more readable. That line-wrapped command actually works just as it is in the shell as well.)

Next, you’ll need to install Mock and related RPM-building tools:

yum install fedora-packager

Third, create a dedicated user for building RPMs. I use “makerpm” as my username, but you could use something else. Just make sure that the name makes sense to you, and that the new user is a member of the mock group:

useradd makerpm -G mock
passwd makerpm

From this point on, you’ll want to be running as this user you just created, so switch to that user with su - makerpm. This ensures that the RPMs are built under this dedicated user account.

The final step in setting up Mock and the build environment is to run the following command while running as the dedicated user you created:

rpmdev-setuptree

Now you’re ready to move on to the next phase: installing prerequisites into the Mock environment.

Installing Prerequisites Into the Mock Environment

One of the great things about Mock is that it creates an isolated chroot into which it installs all the necessary prerequisites for a particular package. This helps ensure that the package’s dependencies are managed correctly. However, if you are trying to build a package where dependencies don’t exist in the repositories, then you have to take a few additional steps. When you’re trying to build libvirt 1.0.1 RPMs for use with CentOS 6.3, you’ll find yourself in exactly this situation. Libvirt has dependencies on newer versions of sanlock-devel and libssh2-devel than are available in the repositories.

Fortunately, there is a workaround—and here’s where those other posts on Mock will come in handy. Use the instructions posted here to build RPMs for sanlock 2.4, and use the instructions here to build RPMs for libssh2 1.4.1.

Once the RPMs are built (and they should build without any major issues, based on my testing), then use these commands to get them into the isolated Mock environment (I’ve line-wrapped here with backslashes for readability):

mock -r epel-6-x86_64 --init
mock -r epel-6-x86_64 --install \
~/rpmbuild/RPMS/sanlock-lib-2.4-3.el6.x86_64.rpm \
~/rpmbuild/RPM/sanlock-devel-2.4-3.el6.x86_64.rpm \
~/rpmbuild/RPMS/libssh2-1.4.1-2.el6.x86_64.rpm \
~/rpmbuild/RPMS/libssh2-devel-1.4.1-2.el6.x86_64.rpm

This will install these packages into the Mock environment, not onto the general Linux installation.

Once you’ve gotten these packages compiled and installed, then you’re ready for the final phase: building the libvirt RPMs.

Building the Libvirt RPMs

As in my earlier post on compiling Libvirt 1.0.1 RPMs for CentOS 6.3, this final step is almost anti-climactic. That’s good, though, because it means you’ve done all the previous steps perfectly.

First, fetch the source RPM from the libvirt HTTP server:

wget http://libvirt.org/sources/libvirt-1.0.1-1.fc17.src.rpm

Next, move the source RPM into the ~/rpmbuild/SRPMS directory:

mv libvirt-1.0.1-1.fc17.src.rpm ~/rpmbuild/SRPMS

Finally, run Mock to rebuild the RPMs:

mock -r epel-6-x86_64 --no_clean ~/rpmbuild/SRPMS/libvirt-1.0.1-1.fc17.src.rpm

Note that the --no-clean parameter is required here to prevent Mock from cleaning out the chroot and getting rid of the packages you installed into the environment earlier.

This command should run without any errors or problems, and produce a set of RPMs (typically) found in /var/lib/mock/epel-6-x86_64/results. You can then take these RPMs and install them on another CentOS 6.3 system using yum localinstall.

Testing the RPMs

To verify that everything worked as expected, I tested the RPMs using these steps:

  1. Using a clean CentOS 6.3 VM (built using the “Minimal Desktop” option), I used yum groupinstall to install the Virtualization, Virtualization Client, Virtualization Platform, and Virtualization Tools groups. This installed version 0.9.10 of libvirt.

  2. I then installed the updated version of libvirt using yum localinstall. I had to specify the dependencies manually on the command line; I anticipate that had I been using a real repository, this would not have been the case. The updated libvirt, sanlock, and libssh2 packages all installed correctly.

  3. I started the libvirtd service (it worked), and ran virsh --version. It returned version 1.0.1.

I imagine there might be more comprehensive/better ways of testing the RPMs that I built, but they seemed to work fine on my end. If anyone has any other suggestions for how we can double-check to ensure the packages are working correctly, feel free to speak up in the comments below. I also welcome any other corrections, suggestions, or questions in the comments. Courteous comments are always welcome.

Tags: , , ,

Using Mock to Build libssh2 1.4.1 RPMs for CentOS 6.3

Tuesday, January 22, 2013 in Linux by slowe | 1 comment

As with the related post on using Mock to rebuild sanlock 2.4 for CentOS 6.3, this post might seem a bit odd. Don’t worry—I’ll tie it into something else very soon. In this post, I’ll show you how to use Mock to build RPMs for libssh2 1.4.1 for use with CentOS 6.3.

The information in this post is based on information found in two other very helpful pages:

Using Mock to test package builds
How to rebuild a package from Fedora or EPEL for RHEL, CentOS, or SL

I tested these instructions on a newly-built CentOS 6.3 VM, installed using the “Minimal Desktop” option. I haven’t tested it on other RHEL variants or other versions, so keep that in mind.

First, you’ll want to activate EPEL. You’ll do that by downloading the RPM and using yum localinstall to install it. You can also use rpm to install it directly from the URL, but I prefer using yum localinstall. I’ve line-wrapped the EPEL URL with a backslash for readability.

wget http://fedora.mirrors.pair.com/epel/6/i386/\
epel-release-6-8.noarch.rpm
yum localinstall epel-release-6-8.noarch.rpm

Once EPEL is installed, then install Mock and related tools:

yum install fedora-packager

This will download and install Mock and related tools such as rpmbuild.

Next, create a user under which you’ll run all these commands, and make sure this account is a member of the mock group:

useradd makerpm -G mock
passwd makerpm

From here on, you’ll want to be running as this user you just created, so switch to that user with su - makerpm.

The first step while running as the user you created is to setup the RPM build environment:

rpmdev-setuptree

Now that the directory structure is created, use wget to download the source RPM for libssh2 1.4.1-2 from the Fedora 17 release repository (the URL is line-wrapped here for readability):

wget http://dl.fedoraproject.org/pub/fedora/linux/releases/17\
/Everything/source/SRPMS/l/libssh2-1.4.1-2.fc17.src.rpm

Move the source RPM to the rpmbuild/SRPMS directory:

mv libssh2-1.4.1-2.fc17.src.rpm ~/rpmbuild/SRPMS

And, finally, rebuild the RPMs with mock:

mock --rebuild ~/rpmbuild/SRPMS/libssh2-1.4.1-2.fc17.src.rpm

Assuming everything completes successfully (it did on my CentOS 6.3 VM), then you’ll end up with a group of RPMs found in /var/lib/mock/epel-6-x86_64/results (the exact directory will vary based on OS version and build; I was using 64-bit CentOS 6.3). You should then be able to install those RPMs onto a CentOS 6.3 system using yum localinstall and the prerequisites will be managed properly.

Have fun!

Tags: , ,

Using Mock to Build Sanlock 2.4 RPMs for CentOS 6.3

Tuesday, January 22, 2013 in Linux by slowe | No comments

The topic of this post might seem a bit strange, but it will all make sense later. In this post, I’ll show you how to use Mock to build RPMs for sanlock 2.4 for use with CentOS 6.3.

The information in this post is based on information found in two other very helpful pages:

Using Mock to test package builds
How to rebuild a package from Fedora or EPEL for RHEL, CentOS, or SL

I tested these instructions on a newly-built CentOS 6.3 VM, installed using the “Minimal Desktop” option. I haven’t tested it on other RHEL variants or other versions, so keep that in mind.

First, you’ll want to activate EPEL. You’ll do that by downloading the RPM and using yum localinstall to install it. You can also use rpm to install it directly from the URL, but I prefer using yum localinstall. (Note that the URL for the EPEL RPM is line-wrapped here for readability.)

wget http://fedora.mirrors.pair.com/epel/6\
i386/epel-release-6-8.noarch.rpm
yum localinstall epel-release-6-8.noarch.rpm

Once EPEL is installed, then install Mock and related tools:

yum install fedora-packager

This will download and install Mock and related tools such as rpmbuild.

Next, create a user under which you’ll run all these commands, and make sure this account is a member of the mock group:

useradd makerpm -G mock
passwd makerpm

From here on, you’ll want to be running as this user you just created, so switch to that user with su - makerpm.

The first step while running as the user you created is to setup the RPM build environment:

rpmdev-setuptree

Now that the directory structure is created, use wget to download the source RPM for sanlock 2.4-3 from the Fedora 17 update repository (the URL is line-wrapped here for readability):

wget http://dl.fedoraproject.org/pub/fedora/linux/updates\
/17/SRPMS/sanlock-2.4-3.fc17.src.rpm

Move the source RPM to the rpmbuild/SRPMS directory:

mv sanlock-2.4-3.fc17.src.rpm ~/rpmbuild/SRPMS

And, finally, rebuild the RPMs with mock:

mock --rebuild ~/rpmbuild/SRPMS/sanlock-2.4-3.fc17.src.rpm

Assuming everything completes successfully (it did on my CentOS 6.3 VM), then you’ll end up with a group of RPMs found in /var/lib/mock/epel-6-x86_64/results (the exact directory will vary based on OS version and build; I was using 64-bit CentOS 6.3). You should then be able to install those RPMs onto a CentOS 6.3 system using yum localinstall and the prerequisites will be managed properly.

Enjoy!

Tags: , ,

Building Libvirt 1.0.1 RPMs for CentOS 6.3

Wednesday, January 16, 2013 in Linux, Virtualization by slowe | 5 comments

In previous articles, I’ve shown you how to compile libvirt 0.10.1 on CentOS 6.3, but—as several readers have pointed out in the comments to that and other articles—compiling packages from source may not be the best long-term approach. Not only does it make it difficult to keep the system up-to-date, it also makes automating the configuration of the host rather difficult. In this post, I’ll show you how to rebuild a source RPM for libvirt 1.0.1 so that it will install (and work) under CentOS 6.3. (These instructions should work for RHEL 6.3, too, but I haven’t tested them.)

Overview

The process for rebuilding a source RPM isn’t too terribly difficult, assuming that you can get the dependencies worked out. Here’s a quick look at the steps involved:

  1. Create a set of build directories for source RPMs.
  2. Download the source RPM and install all prerequisites/dependencies onto the system.
  3. Rebuild the source RPM for the destination system.

Let’s take a look at each of these steps in a bit more detail.

Create the Build Environment

The CentOS wiki has a great page for how to set up an RPM build environment. I won’t repeat all the steps here (refer to the wiki page instead), but here’s a quick summary of what’s involved:

  1. Install the necessary packages (typically you’ll need the rpmbuild, redhat-rpm-config, make, and gcc packages). You might also need certain development libraries, but this will vary according the source RPMs you’re rebuilding (more on that in the next section).
  2. Create the necessary directories under your home directory.

I’ll assume that you’ve followed the steps outlined in the CentOS wiki to set up your environment appropriately before continuing with the rest of this process.

Download the Source RPM and Install Prerequisites

The libvirt 1.0.1 source RPMs are available directly from the libvirt HTTP server, easily downloaded with wget:

wget http://libvirt.org/sources/libvirt-1.0.1-1.fc17.src.rpm

You can just download the source RPM to your home directory. Before you can build a new RPM from the source RPM, though, you’ll first need to install all the various prerequisites that libvirt requires. Most of them can be installed easily using yum with a command like this:

yum install xhtml1-dtds augeas libudev-devel \
libpci-access-devel yajl-devel, libpcap-devel libnl-devel \
avahi-devel radvd ebtables qemu-img iscsi-initiator-utils \
parted-devel device-mapper-devel numactl-devel netcfg-devel \
systemtap-sdt-devel scrub numad libblkid-devel

There are two dependencies, though—sanlock and libssh2—that require versions newer than what are available in the CentOS/EPEL repositories. For those, you’ll need to recompile your own RPMs. Fortunately, this is pretty straightforward. The next two sections provide more details on getting these prerequisites handled.

Building an RPM for sanlock

To build a CentOS 6.3 RPM for version 2.4 of sanlock (the minimum version needed by libvirt 1.0.1), first use wget to download a Fedora 17 version of the source RPM. I’ve wrapped the URL with a backslash for readability:

wget http://dl.fedoraproject.org/pub/fedora/linux/updates/17/SRPMS\
/sanlock-2.4-3.fc17.src.rpm

Next, install an prerequisite library using yum install libaio-devel.

Finally, use rpmbuild to rebuild the sanlock source RPM:

rpmbuild --rebuild sanlock-2.4-3.fc17.src.rpm

This process should proceed without any problems. The resulting RPMs that are created will be found in ~/rpmbuild/RPMS/x86_64 (assuming you are, as I am, using a 64-bit build of CentOS).

Building the RPMs, however, isn’t enough—you need to install them so that you can build the libvirt RPMs. So install the sanlock-devel and sanlock-lib RPMs using yum locainstall (do this command from the directory where the RPMs reside):

yum localinstall sanlock-devel-* sanlock-lib-*

That should take care of the sanlock dependency.

Building an RPM for libssh2

To build a CentOS 6.3 RPM for version 1.4.1 of libssh2 (libvirt 1.0.1 requires at least version 1.3.0), first download the source RPM using wget (I’ve wrapped the URL here for readability):

wget http://dl.fedoraproject.org/pub/fedora/linux/releases\
/17/Everything/source/SRPMS/l/libssh2-1.4.1-2.fc17.src.rpm

(That’s a lowercase L in the URL just after SRPMS.)

Once you have the source RPM downloaded, then just rebuild the source RPM:

rpmbuild --rebuild libssh2-1.4.1-2.fc17.src.rpm

Then install the resulting RPMs using yum localinstall:

yum localinstall libssh2-1.4.1-* libssh2-devel-*

That takes care of the last remaining dependency. You’re now ready to compile the RPMs for libvirt 1.0.1.

Build the Libvirt RPM

This part is almost anticlimactic. Just use the rpmbuild command:

rpmbuild --rebuild libvirt-1.0.1-1.fc17.src.rpm

If you’ve successfully installed all the necessary prerequisites, then the RPM compilation process should proceed without any issues.

Once the RPM compilation process is complete, you’ll find libvirt 1.0.1 RPMs in the ~/rpmbuild/RPMS/x86_64 directory (assuming a 64-bit version of CentOS) which you can easily install with yum localinstall, or post to your own custom repository.

I hope this post helps someone. If you have any questions, or if you spot an error, please speak up in the comments below. All courteous comments are welcome!

Tags: , , ,

Why a CLI for Dropbox on Mac OS X

Saturday, December 8, 2012 in Macintosh by slowe | 11 comments

I was thinking about a command-line interface (CLI) for Dropbox, and how I personally would take advantage of such a feature. So, after failing to find any indication that the Mac OS X Dropbox client contained a CLI, tonight on Twitter I made this comment:

Too bad the #Mac version of @Dropbox doesn’t have a CLI.

Shortly thereafter, I received this response:

@scott_lowe @Dropbox What would you do with it if you did?

I posted a response (which you can see if you follow either of the Twitter links above), but I realized that my response really needed a bit more background.

Like many people in IT today, I’m pretty mobile. I have a home office, but I also travel a fair amount. My laptop, a 2011 13″ MacBook Pro with 8GB of RAM and a 512GB SSD, is my primary computer. The problem is this: when I’m in my home office, I want my laptop to be configured a certain way, but when I’m traveling, I need it configured a different way. For example, when I’m in my home office, I want Synergy running so that I can connect to the Synergy server on my Mac Pro workstation. When I’m not in the home office, Synergy should not be running. So how do I get the computer to automatically reconfigure itself? The answer is quite simple, actually: an app called ControlPlane.

ControlPlane is a handy little application that performs a set of actions based on a context. A context is defined as a set of conditions, like (as in my situation) being connected to my 24″ Apple Cinema Display and being connected via Ethernet to a network using my home network’s IP addressing scheme. If all those conditions are met, then it’s quite likely I’m in my home office—meaning I’m in that particular context—and ControlPlane should perform a set of actions to reconfigure my laptop. Similarly, if those conditions aren’t true, then it’s quite likely I’m not in my home office—meaning I’m in a roaming or traveling context—and therefore my computer should be configured a different way. Handy, right?

To put some specifics on this idea, then, here’s how I use ControlPlane:

  • I have two contexts, one called Docked and one called Roaming. Docked is only for when I’m connected to my 24″ Apple Cinema Display in my actual home office, wired up via Ethernet (not wireless), and have an IP address off my home network’s subnet. When those conditions are true, I’m “docked” and I need Synergy running so that I can share keyboard and mouse between my Mac Pro workstation and my laptop.
  • Any other time, I’m not “docked” and should be in the Roaming context. In the Roaming context, Synergy should not be running.
  • When I enter the Docked context, ControlPlane should launch Synergy, if it’s not already running, and then issue a Growl notification that the computer is entering the Docked context.
  • When I leave the Docked context (meaning I’m entering the Roaming context), then ControlPlane should kill Synergy (if it’s running), and post a Growl notification.

ControlPlane is capable of much, much more, but (for now) this is sufficient. At some point in the future, I might have it mount network drives (or maybe my EncFS filesystem).

I said all that to finally come back to the comment that started all this: if Dropbox had a CLI (or AppleScript support, but that’s probably too much to ask for), then I could use ControlPlane to automate/manipulate the behavior of Dropbox as part of my contexts. For example, I could define another context—say, Disconnected—in which there are no active network interfaces. In that context, I’d like Dropbox to pause syncing. Then, when I enter another context, either Roaming or Docked, then Dropbox should continue syncing. However, without some sort of non-GUI access to Dropbox, this isn’t possible (to my knowledge).

Anyway, that’s what I was thinking. Courteous comments (or questions) are always invited and encouraged, so feel free to speak out below.

Tags: , , ,

Advanced Spotlight Queries in the Mac GUI

Wednesday, July 25, 2012 in Macintosh by slowe | 6 comments

I mentioned on Twitter a couple days ago that I was mulling a switch from EagleFiler to a pure file system-based approach leveraging OpenMeta tags. Long-time readers may recall that it was only September of last year that I switched to EagleFiler from Yojimbo, my previous “anything bucket”. My decision to switch away from EagleFiler is not a reflection on the application itself; it’s a great application. For me, it just seemed as if EagleFiler was duplicating functionality I could already get with just the file system, so what’s the point? Plus, as I increasingly keep data synchronized across multiple systems, EagleFiler wasn’t as friendly to my data synchronization solutions as I would have liked. A file system-based approach leveraging OpenMeta tags is perfectly happy with both Dropbox and Unison.

The OpenMeta tags are really the key; using OpenMeta tags, I can set up saved searches that easily let me drill down to only certain subsets of files regardless of where on my computer those files might be stored. However, the problem that I was running into was that the Mac OS X GUI did not provide a way for me to find all untagged files (obviously, the key to making tags work effectively is using tags everywhere). For that, I had to drop out to the OS X command line and use the mdfind command, like this:

mdfind -onlyin ~ "kOMUserTags != '*'"

The kOMUserTags is the name of the extended attribute in which OpenMeta tags are stored. This particular query finds files that don’t have any OpenMeta tags. Unfortunately, there didn’t seem to be any way to bring this query (and the query results) into the GUI.

After some further experimentation, I grew frustrated with how things were progressing (or not progressing, in this instance) and thought I’d try to dig up some documentation on the syntax for mdfind. I didn’t find the documentation—but I did find this page, which gave me two important pieces of information:

  1. It showed me the syntax to query on both filename as well as extended attributes at the same time.

  2. It showed me how to bring these queries into the Finder GUI.

Let’s take a look at the first one. To query on both filename and one or more extended attributes, you can’t use the -name parameter to mdfind. Instead, you have to use this syntax:

mdfind -onlyin ~ "(kMDItemFSName == '*.txt') && (kOMUserTags != '*')"

That query will show you all the text files that don’t have any OpenMeta tags applied. Obviously, you could customize the filename specification or the particular extended attributes you wanted to search, but you get the idea.

The second thing I found was how to bring these advanced queries into the Finder GUI. The trick is in enabling “Raw Query” on the pop-up menu in the Spotlight search menu:

  1. Open a Spotlight search window by pressing Cmd-F while Finder is active.
  2. Click the “Kind” pop-up menu and select “Other…”.
  3. It may take a moment, but in a bit a window will open with all of the various types of Spotlight metadata. Scroll through the list to find Raw, then place a checkmark in the “In Menu” column. (By the way, this is also how to add Tags to the menu for use in Spotlight searches.)

Now, in a Spotlight search window (and that includes new Smart Folders), you can click the pop-up, select Raw Query, and then type your raw mdfind query like the one above, and it will be displayed in the Finder. Cool! Using this Raw Query support allows me to build extremely sophisticated Smart Folders.

For example, I’ve already shown you the query to list all untagged files (very first example in this post) and the query to list all untagged files of a certain type (second example). Here’s the query to list files that have one tag but not another tag:

mdfind -onlyin ~ "(kOMUserTags == 'Tag1') && (kOMUserTags != 'Tag2')"

You could, naturally, combine this with a filename filter to get even more specific. I hope this helps someone else out there fine tune their Spotlight searches as well!

Tags: , ,

Some Useful UNIX Commands on your Mac

Monday, July 25, 2011 in Macintosh, UNIX by slowe | 9 comments

Over the last day or so I’ve been messing around at the UNIX command line on my Mac, trying to find a workaround for a VPN policy that doesn’t allow split tunneling. (Just as a stupid side question, what is the security issue with split tunneling, anyway?) Along the way, I uncovered some handy commands for gathering information about the networking configuration of your Mac.

I can’t take credit for all of these; most of them were shared with me by Matt Cowger, fellow VCDX and vSpecialist.

If anyone has any additional commands they’d like to share, I encourage you to add them to the comments on this post. Enjoy!

To find the IP address of the default gateway:

netstat -nr -f inet | grep default | grep en | awk '{print $2}'

To find the interface name of the default route:

netstat -nr -f inet | grep default | grep en | awk '{print $6}'

To find the IP address assigned to the interface for the default gateway:

ORGGWIF=`netstat -nr -f inet | grep default | grep en | awk '{print $6}'`
ifconfig $ORGGWIF | grep "inet " | awk '{print $2}'

To find the default gateway network:

ORGGWIF=`netstat -nr -f inet | grep default | grep en | awk '{print $6}'`
netstat -I $ORGGWIF -n | grep -v : | grep $ORGGWIF | awk '{print $3}'

To find the subnet mask for the default gateway network:

ORGGWIF=`netstat -nr -f inet | grep default | grep en | awk '{print $6}'`
system_profiler SPNetworkDataType | grep -A 15 $ORGGWIF | grep "Subnet Masks" | awk '{print $3}'

To convert the subnet mask into CIDR format:

ORGGWIF=`netstat -nr -f inet | grep default | grep en | awk '{print $6}'`
ORGGWMASK=`system_profiler SPNetworkDataType | grep -A 15 $ORGGWIF | grep "Subnet Masks" | awk '{print $3}'`
echo obase=2.$ORGGWMASK | tr . \; | bc | tr -d 0\\n | wc -c | awk '{print $1}'

To determine the wireless SSID to which your Mac is currently associated:

/System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport -I | grep SSID | tail -n 1 | awk '{print $2}'

CLI gurus and wizards are encouraged to share other useful commands in the comments below. Thanks!

Tags: , , ,

Using GNU Screen with SSH

Thursday, June 23, 2011 in Macintosh, UNIX by slowe | 12 comments

This post is probably old news to experienced UNIX sys admins, but I thought the information might be useful to less knowledgeable folks like me. I also hope that the resulting conversation will help uncover even more knowledge we can all put to good use.

I’ve messed around with the screen utility off and on for a while. One thing I’d never quite figured out, though, was how using screen helped with SSH sessions. I kept seeing references to using screen to help keep things running when you needed to disconnect from an SSH session. That seems like a useful feature, so I decided to dig into it and see what I could figure out.

In the end, what I figured out was this:

  • I needed to install screen on the remote host(s). In my case, the remote hosts were OpenBSD (I removed the secret back doors), so a quick pkg_add corrected that issue.
  • I had to recreate my .screenrc file on the remote host(s). Fortunately, my .screenrc is very simple—it only enables the ability to use the iTerm2/Terminal scrollbar to scroll back and increases the scrollback buffer—so that was no big deal.

With these changes in place, you can then use this command to connect to a remote host:

ssh -t <server.domain.name> screen -R

On the first connection, this command will create a new screen session. When you’re done with this SSH session and want to disconnect, just detach from the screen session (typically using Ctrl-a d). That also disconnects the SSH session, but here’s the kicker: your screen session is still running—as are any processes you had running in that session.

When you go to reconnect, use the same command again and it will reconnect you to your existing screen session and you’ll be right back where you left off. Pretty handy!

<aside>By the way, the -t in the SSH command is necessary; without it, you’ll get a “Must be connected to a terminal” error message and it won’t work properly.<aside>

I’m sure this barely scratches the surface of the useful tricks one could perform using screen, so I challenge any and all readers to submit other useful tricks in the comments below. Or, if there is a better way of doing what I’m discussing in this article, please speak up!

Tags: , ,

« Older entries