Exploring RHEL-OVS Integrations

Thursday, February 7, 2013 in Linux, Networking by slowe | 2 comments

In this post, I’m going to explore some integrations between Red Hat Enterprise Linux (RHEL)/RHEL variants and Open vSwitch (OVS). This post will lay the foundation for a future post describing OVS automation using Puppet.

Over the past few weeks, I’ve been rebuilding my home lab to focus on some core projects/technologies for the upcoming year (more on that in this post). As part of the rebuild effort, I’ve rebuilt my hosts to run CentOS 6.3 x64, KVM, Libvirt, GlusterFS, and OVS, and have crafted Puppet code to help install (and configure, in some cases) most of these components. This post, as with some others, is an offshoot of the lab rebuild work.

(In case you were wondering, the other posts that have come out of the home lab rebuild project include building Libvirt RPMs, using Mock to build Sanlock RPMs, using Mock to build libssh2 RPMs, using Mock to build Libvirt RPMs, and using Puppet for user accounts and configuration files. I’m sure there will be more.)

The information in this post will build upon some base OVS knowledge, so if you aren’t familiar with OVS, then you might want to go back and read some of my earlier OVS posts (these will at least get you started):

Some Insight into Open vSwitch Configuration
Link Aggregation and LACP with Open vSwitch
VLANs with Open vSwitch Fake Bridges
Running Host Management on Open vSwitch
Layer 3 Routing with Open vSwitch

Finally, before we get into the real meat of the information, I’d like to point out that the information in this post builds upon the information already included in the OVS documentation (see the README.RHEL file in the distribution tarball). I’m merely attempting to provide some additional context and examples on how these can be used.

Now that the preliminaries are out of the way…

The basic gist of the RHEL-OVS integration is support for the use of the scripts in /etc/sysconfig/network-scripts to do some configuration of OVS itself. For example, creating bridges, establishing bonds, configuring internal ports—all these tasks can be done using an ifcfg-<name> script.

Let’s start with a very basic example. Let’s say that you want to create a single OVS bridge named ovsbr0. To do that, you’d create a file named ifcfg-ovsbr0 in the /etc/sysconfig/network-scripts directory, and make the contents look like this:

Now let’s say that you want to create a link aggregate on that bridge. To create a link aggregate on ovsbr0 (the bridge you just created), create a file named ifcfg-bond0 in the same directory, and make its contents look like this:

Note that this is an LACP-enabled link aggregate, so you’ll also have to configure your physical switch appropriately.

Finally, suppose that you want to create an internal interface (it will appear as a physical interface to Linux, but is hosted on OVS) across which you can run your management traffic. No problem! Just create a file named ifcfg-mgmt0 and make the contents of the file look like this:

Each of these scripts will create the corresponding structures/configurations within OVS. There is a drawback, however. In order for these changes to take effect, you must restart the network (perform a service network restart). This doesn’t appear to be an OVS limitation of any sort; if you’ve read any of the other OVS posts, you know that you can make these changes live via the ovs-vsctl command. Instead, it simply appears to be a limitation of the fact that these scripts are only evaluated during a network stop/start event.

Once these scripts have been evaluated, though, you should end up with a configuration that looks something like this (UUIDs have been changed to protect the innocent):

Given the limitation, I can imagine that a natural question to ask would be, “Why use this integration, which requires a network restart, when you could just make the configuration changes yourself?” Excellent question. I see this as a way of establishing a “baseline” configuration for OVS, upon which you could (dynamically) add all the other configurations you need. In addition, because the base OVS configuration exists as a set of files, this opens up some other interesting possibilities. I’ll explore those possibilities in a future post.

As always, courteous comments are welcome, so feel free to add your questions, thoughts, corrections, or clarifications in the comments below.

Tags: , ,

Looking Ahead: My 2013 Projects

Thursday, February 7, 2013 in General, Linux, Networking, Personal by slowe | 7 comments

In early January 2012, I posted a list of my 2012 projects. Those projects included learning to script in Perl, learning German, becoming more familiar with Xen/Open vSwitch/OpenStack, and pursuing CCNP. In late June 2012, my mid-year project update let you know that I was dropping the Perl efforts, but sticking it out with German, Xen/OVS/OpenStack, and CCNP. (I later dropped Xen in favor of KVM.) Finally, in early January 2013 I graded myself on my progress on my 2012 goals. In this post, I’d like to take lessons learned from my 2012 projects and apply them toward my list of 2013 projects.

As a quick summary, here are some key “lessons learned” from my 2012 projects:

  • The synergy of the projects does make a difference. I should try to find projects that are related and that support one another in some fashion.
  • Projects need to be more tightly defined. Projects that are overly broad will have mixed results.
  • There are still some fundamental building blocks that I need to reinforce.

With the conclusions listed above in mind, here is my list of 2013 projects and—in some cases—some proposed goals for which I’ll be aiming.

  1. Continue to learn German. Obviously, this is a continuation of my 2012 project to learn to speak German. This year, I’ll need to re-evaluate ways to enhance my motivation, and find additional ways to reinforce the information I’m learning. I don’t have any specific goals in mind for this project, although that is something I’ll be evaluating over the course of the year (I do agree that clear goals—something I didn’t establish last year—can help with progress.)

  2. Reinforce base Linux knowledge. This is one of the “fundamental building blocks” that I needed to reinforce. I’ll be focusing on Red Hat Enterprise Linux (RHEL) and RHEL variants, since that seems to be what’s most commonly found in enterprise deployments today. My primary goal here is to be able to do most anything that I need to do without constantly having to look up a “how to” guide; a secondary goal that I’m considering (but haven’t decided upon) would be to look at a Red Hat-focused certification (like RHCE).

  3. Continue using Puppet for automation. This will be partly tied to #2. I’ve mentioned elsewhere that you can’t automate something you don’t fully understand, so as my base Linux knowledge is reinforced I will also be seeking ways to automate tasks with Puppet. As with my base Linux knowledge, my primary goal here is to be able to do most anything that I need to do with Puppet. I might consider Puppet certification, but I think that’s a long shot.

  4. Expand and deepen data center networking fundamentals. Making progress on CCNP was one of my 2012 goals, but it didn’t really fit in with the rest of my focus areas (which were almost all centered in the data center). To help improve the synergy of projects, then, I’ve decided that my networking focus should be DC-oriented. This also ties in nicely with my new job role at Nicira/VMware. My primary goal here is simply to be more knowledgeable, although over the course of the year I will consider pursuing CCNA-DC.

So, that’s my list of 2013 projects. I’m sure that I’ll have a few side projects here and there as well (for example, there is a book project and at least one video training series project on the books for 2013), but the projects listed above represent the majority of my technical focus over the coming year. I’d love to hear your feedback—am I missing something important? Are there ways I could further improve the synergy of the projects? As always, your honest and courteous comments are welcome.

Tags: , , , , ,

Technology Short Take #29

Monday, February 4, 2013 in Networking, Security, Storage, Virtualization by slowe | 6 comments

Welcome to Technology Short Take #29! This is another installation in my irregularly-published series of links, thoughts, rants, and raves across various data center-related fields of technology. As always, I hope you find something useful here.

Networking

  • Who out there has played around with Mininet yet? Looks like this is another tool I need to add to my toolbox as I continue to explore networking technologies like OpenFlow, Open vSwitch, and others.
  • William Lam has a recent post on some useful VXLAN commands found in ESXCLI with vSphere 5.1. I’m a CLI fan, so I like this sort of stuff.
  • I still have a lot to learn about OpenFlow and networking, but this article from June of last year (it appears to have been written by Ivan Pepelnjak) discusses some of the potential scalability concerns around early versions of the OpenFlow protocol. In particular, the use of OpenFlow to perform granular per-flow control when there are thousands (or maybe only hundreds) of flows presents a scalability challenge (for now, at least). In my mind, this isn’t an indictment of OpenFlow, but rather an indictment of the way that OpenFlow is being used. I think that’s the point Ivan tried to make as well—it’s the architecture and how OpenFlow is used that makes a difference. (Is that a reasonable summary, Ivan?)
  • Brad Hedlund (who will be my co-worker starting on 2/11) created a great explanation of network virtualization that clearly breaks down the components and explains their purpose and function. Great job, Brad.
  • One of the things I like about Open vSwitch (OVS) is that it is so incredibly versatile. Case in point: here’s a post on using OVS to connect LXC containers running on different hosts via GRE tunnels. Handy!

Servers/Hardware

  • Cisco UCS is pretty cool in that it makes automation of compute hardware easier through such abstractions as server profiles. Now, you can also automate UCS with Chef. I traded a few tweets with some Puppet folks, and they indicated they’re looking at this as well.
  • Speaking of Puppet and hardware, I also saw a mention on Twitter about a Puppet module that will manage the configuration of a NetApp filer. Does anyone have a URL with more information on that?
  • Continuing the thread on configuration management systems running on non-compute hardware (I suppose this shouldn’t be under the “Servers/Hardware” section any longer!), I also found references to running CFEngine on network apliances and running Chef on Arista switches. That’s kind of cool. What kind of coolness would result from even greater integration between an SDN controller and a declarative configuration management tool? Hmmm…

Security

  • Want full-disk encryption in Ubuntu, using AES-XTS-PLAIN64? Here’s a detailed write-up on how to do it.
  • In posts and talks I’ve given about personal productivity, I’ve spoken about the need to minimize “friction,” that unspoken drag that makes certain tasks or workflows more difficult and harder to adopt. Tal Klein has a great post on how friction comes into play with security as well.

Cloud Computing/Cloud Management

  • If you, like me, are constantly on the search for more quality information on OpenStack and its components, then you’ll probably find this post on getting Cinder up and running to be helpful. (I did, at least.)
  • Mirantis—recently the recipient of $10 million in funding from various sources—posted a write-up in late November 2012 on troubleshooting some DNS and DHCP service configuration issues in OpenStack Nova. The post is a bit specific to work Mirantis did in integrating an InfoBlox appliance into OpenStack, but might be useful in other situation as well.
  • I found this article on Packstack, a tool used to transform Fedora 17/18, CentOS 6, or RHEL 6 servers into a working OpenStack deployment (Folsom). It seems to me that lots of people understand that getting an OpenStack cloud up and running is a bit more difficult than it should be, and are therefore focusing efforts on making it easier.
  • DevStack is another proof point of the effort going into make it easier to get OpenStack up and running, although the focus for DevStack is on single-host development environments (typically virtual themselves). Here’s one write-up on DevStack; here’s another one by Cody Bunch, and yet another one by the inimitable Brent Salisbury.

Operating Systems/Applications

  • If you’re interested in learning Puppet, there are a great many resources out there; in fact, I’ve already mentioned many of them in previous posts. I recently came across these Example42 Puppet Tutorials. I haven’t had the chance to review them myself yet, but it looks like they might be a useful resource as well.
  • Speaking of Puppet, the puppet-lint tool is very handy for ensuring that your Puppet manifest syntax is correct and follows the style guidelines. The tool has recently been updated to help fix issues as well. Read here for more information.

Storage

  • Greg Schulz (aka StorageIO) has a couple of VMware storage tips posts you might find useful reading. Part 1 is here, part 2 is here. Enjoy!
  • Amar Kapadia suggests that adding LTFS to Swift might create an offering that could give AWS Glacier a real run for the money.
  • Gluster interests me. Perhaps it shouldn’t, but it does. For example, the idea of hosting VMs on Gluster (similar to the setup described here) seems quite interesting, and the work being done to integrate KVM/QEMU with Gluster also looks promising. If I can ever get my home lab into the right shape, I’m going to do some testing with this. Anyone done anything with Gluster?
  • Erik Smith has a very informative write-up on why FIP snooping is important when using FCoE.
  • Via this post on ten useful OpenStack Swift features, I found this page on how to build the “Swift All in One,” a useful VM for learning all about Swift.

Virtualization

  • There’s no GUI for it, but it’s kind of cool that you can indeed create VM anti-affinity rules in Hyper-V using PowerShell. This is another example of how Hyper-V continues to get more competent. Ignore Microsoft and Hyper-V at your own risk…
  • Frank Denneman takes a quick look at using user-defined NetIOC network resource pools to isolate and protect IP-based storage traffic from within the guest (i.e., using NFS or iSCSI from within the guest OS, not through the VMkernel). Naturally, this technique could be used to “protect” or “enhance” other types of important traffic flows to/from your guest OS instances as well.
  • Andre Leibovici has a brief write-up on the PowerShell module for the Nicira Network Virtualization Platform (NVP). Interesting stuff…
  • This write-up by Falko Timme on using BoxGrinder to create virtual appliances for KVM was interesting. I might have to take a look at BoxGrinder and see what it’s all about.
  • In case you hadn’t heard, OVF 2.0 has been announced/released by the DMTF. Winston Bumpus of VMware’s Office of the CTO has more information in this post. I also found the OVF 2.0 frequently asked questions (FAQs) to be helpful. Of course, the real question is how long it will be before vendors add support for OVF 2.0, and how extensive that support actually is.

And that’s it for this time around! Feel free to share your thoughts, suggestions, clarifications, or corrections in the comments below. I encourage your feedback, and thanks for reading.

Tags: , , , , , , , , , , , , , , ,

Puppet, User Accounts, and Configuration Files

Tuesday, January 29, 2013 in Linux by slowe | No comments

A short while ago I posted an article that described how to use Puppet for account management. In that article, I showed you how to use virtual user resources to manage user accounts on Puppet-managed systems. In this post, I’m going to expand on that configuration so that we can also manage the initial account configuration files, and do so in the proper order.

One of the things the configuration in my first post didn’t handle was the Puppet configuration of the files in /etc/skel and making sure those files were in place before the user accounts were created. As a result, it was possible that the user account could be created on a system before the /etc/skel files were updated, and then that user account would have “unmanaged” copies of the initial configuration files. Further Puppet agent runs wouldn’t correct the problem, because the files in /etc/skel are only copied over when the account is created. If the account has already been created, then it’s too late—the files in /etc/skel must be managed before the accounts are. To fix the issue, you have to ensure that the resources are processed in a specific manner. In this post, I’ll show you how to manage that.

There are two parts to extending the Puppet accounts module to also manage some configuration files:

  1. Add a subclass to manage the files.
  2. Create a dependency between the virtual user resources and this new subclass.

Let’s look at each of these.

Adding a Subclass

To add a subclass to manage the configuration files, I created config.pp and placed it in the manifests folder for the accounts module. Here’s a simplified look at the contents of that file:

This is pretty straightforward Puppet code; it creates a managed file resource and specifies that the file be sourced from the Puppet master server. The full and actual accounts::config subclass that I’m using has a number of managed file resources, including files in /etc/skel, but I’ve omitted that here for the sake of brevity. (The other file resources that are defined look very much like the example shown, so I didn’t see any point in including them.) The config.pp also uses some values from an accounts::params subclass and some conditionals to manage different files on different operating systems.

To really put the subclass to work, though, we have to include it elsewhere. So, in the accounts module’s init.pp, we add a line that simply states include accounts::config. However, the problem that occurs if you stop there is the problem I described earlier: Puppet might create the user account before it places the file resources under management, and then the user account won’t get the updated/managed files.

To fix that, we create a dependency.

Creating a Dependency

Before running into this situation, I was pretty familiar with creating dependencies. For example, if you were defining a class for a particular daemon to run on Linux, you might use the Puppet package-file-service “trifecta”, and you might include a dependency, like this (entirely fictional) example. Note in this example that the file resource is dependent on the package resource, and the service resource is dependent on the file resource (as denoted by the capitalized Package and File instances):

(My apologies if my syntax for this fictional example isn’t perfect—I didn’t run it through puppet-lint.)

The problem in this particular case, though, is that I didn’t need a dependency on a single file; I needed a dependency on a whole group of files. To further complicate matters, the files on which the dependency existed might change between operating systems. For example, I might (and do) have different files on RHEL/CentOS than on Ubuntu/Debian. So how to accomplish this? The answer is actually quite simple: create a dependency on the subclass, not the individual resources.

So, without the dependency, the code to define the virtual users looked like this:

With the dependency, the code to define the virtual users looks like this:

The only difference between the two (other than changes in the comments at the top) is the addition of the require statement, which creates a dependency not on a single resource but instead to an entire subclass—the accounts::config subclass, which in turn has a variety of file resources that are managed according to operating system.

It’s such a simple solution I can’t believe I didn’t see it at first, and when it was pointed out to me (via the #puppet IRC channel, thanks), I had a “Duh!” moment. Even though it is a simple and straightforward solution, if I overlooked it then others might overlook it as well—a problem that hopefully this blog post will help fix.

As always, I welcome feedback from readers, so feel free to weigh in with questions, clarifications, or corrections. Courteous comments are always welcome!

Tags: , , ,

New Year, New Challenges, New Opportunities

Friday, January 25, 2013 in General, Networking, Virtualization by slowe | 31 comments

I suppose there’s no sense in beating around the bush. As the blog post title indicates, I’m taking on a new set of challenges (and a new set of opportunities) in 2013—and the way to do that is in a new role with a new company. So, effective 2/8/2013, I am leaving EMC Corporation to join the former Nicira group at VMware, working directly for Martin Casado. I’ll be working with folks like Brad Hedlund (see his announcement here), Bruce Davie, and Teemu Koponen. I’m truly awed by the talent on this team.

My time at EMC over the last three years has been great, and my choice to leave was a difficult choice to make. The decision to leave does not reflect anything bad about EMC, but rather reflects the magnitude of the opportunities for personal and professional growth that lie ahead with VMware’s virtual networking group. There is a saying among my former team at EMC that goes like this: “Once a vSpecialist, always a vSpecialist.” I don’t agree with this statement, because it implies a sense of permanence—something those of us in IT simply can’t afford to have. You must change, you must evolve, you must become something more than what you were in the past, or you will become irrelevant. While I appreciate my time at EMC—both my time as a vSpecialist and my time within the ESG CTO’s office—the time for growth and evolution has come. This move will help me further evolve and grow. I’ve always been interested in networking, but this will be the first time it will be the primary focus for me, and I’m really looking forward to expanding my knowledge, learning new concepts and ideas, and leveraging my existing experience and expertise with virtualization in new and exciting ways.

Although there are great opportunities ahead, there are also a few challenges. I’m not relocating (I love Denver too much!), but my travel schedule will ramp up quite a bit. Travel has been down for me for the last several months (since I left the vSpecialist team), but in the new role my travel will go back up again as I’ll be meeting with the rest of the virtual networking team in Palo Alto, meeting with strategic customers and partners, supporting community events (expect to see me at VMUG events), and educating field sales resources on virtual networking and why it’s important. Undoubtedly the increased travel will have an impact on Crystal and the rest of the family, and I appreciate everyone’s thoughts and prayers as we sort that out.

One other challenge will come from a shift in “allegiance.” I experienced a similar effect when I joined EMC. When I was with ePlus (it seems so long ago!) I was able to maintain reasonably good relationships with different storage vendors as well as different networking vendors. When I joined EMC, the other storage vendors no longer wanted to work with me. I suppose I can understand that. I was able, though, to continue maintain reasonably good relationship with various networking vendors (and even a few other virtualization vendors). I suspect now, though, that my shift to VMware will alter that landscape again. I can only hope the relationships I’ve established with colleagues at “competing” organizations (real or perceived competition) aren’t negatively affected too much.

Long-time readers know that several transitions have occurred over the nearly 8 years that I’ve been writing here. As I’ve done for the last 8 years, I’ll continue to post as much useful, relevant, and interesting content here as I’m able. Will there be a shift in focus? Possibly; I can’t promise there won’t be. Still, I’ll strive to keep sharing as much as I’m able as together we grow, change, and evolve along with the IT industry. Thanks for the support, and I hope that it continues.

Courteous comments are always welcome, so if you have questions or thoughts you want to share, feel free to speak up below.

Tags: , , ,

How I Build the Technology Short Takes

Wednesday, January 23, 2013 in General, Personal by slowe | 1 comment

Shortly after I published Technology Short Take #27, a reader asked me on Twitter how I managed the information that goes into each Technology Short Take (TST) article. Although I’ve blogged about my productivity setup before, that post is now over two years old and horribly out of date. Given that I need to post a more current description of my tools and workflow, I thought I’d take a brief moment to answer the reader’s question. Here’s how I go about building the Technology Short Take articles.

I’ve mentioned before that I have three “layers” of tools: a consumption layer, an organization layer, and a creation layer. I’ll break down the process according to these layers.

The Consumption Layer

This is where I go about actually finding the content that gets pulled into a TST post. There’s nothing terribly unique here; I have a collection of RSS feeds which I subscribe and I get content from people I follow on Twitter. I will also visit various Usenet newsgroups and certain IRC channels (only on irc.freenode.net) from time to time.

If you’re interested in seeing the RSS feeds to which I subscribe, here’s an OPML list of all my subscriptions.

The majority of the content I consume is web-based, so when I find an article that I want to use in a TST post, I’ll save that as a Safari web archive. I wish there was a more platform-independent method, but as yet I haven’t found a good solution. Once I’ve saved a page for future use, then we move into the organization layer.

The Organization Layer

As content is discovered via any of the various consumption layer tools, I then need to get that content “sucked” into my primary organization layer tool. I use a really, really fancy tool—it’s called the file system.

When I save a web page that I’m planning on including in a TST article, I generally save it, by default, to the Desktop. I have a program named Hazel that watches the Desktop and Downloads folders for web archive files, and automatically moves them to a WebArchives folder. From there, I use a couple of saved Spotlight searches to identify newly-created web archives that don’t have a source URL or don’t have any OpenMeta tags assigned. For these newly-created web archives, I use the Spotlight comments field to store the source URL, and I use an application named Tagger to assign OpenMeta tags.

Once a web archive has its source URL and OpenMeta tags assigned, then I have a group of saved Spotlight searches the group files together by topic: virtualization, storage, OpenStack, Open vSwitch, etc. This makes it super easy for me to find web archives—or other files—related to a particular topic. All these saved searches are built on queries involving the OpenMeta tags.

Content will remain here until either a) I use it in a TST article and no longer need it; or b) I use it in a TST article but feel it’s worth keeping for future reference. I might keep content for quite a long time before I use it. Case in point: the Q-tools stuff from Dave Gray that eventually found its way into some of my VMUG presentations was something I found in 2009 (it was published in 2008).

The Creation Layer

After collecting content for a while, a scheduled, recurring OmniFocus action pops up reminding me to write the next TST post. At this point, I go back to my organization layer tools (saved Spotlight searches and content folders) to pull out the various pieces of information that I want to include. I write the post in Markdown using TextMate, building off a skeleton template that has all the content headers already in place.

Using the saved searches I described above, I’ll search through my content to see what I want to include in the TST post. When an item is included in a TST blog post, I’ll write my thoughts about the article or post, then grab the source URL from the Spotlight comments to make a hyperlink to the content. If the content is useful and informative, I might keep it around; otherwise, I’ll generally delete the saved web archive or bookmark file. I repeat this process, going through all my saved content, until I feel that the TST post is sufficiently full of content.

Then, because it’s all written in Markdown, I convert the post to HTML and actually publish it to the site using the excellent MarsEdit application. TextMate makes this incredibly easy with just a few keystrokes.

And that’s it! That’s the “mystery” behind the Technology Short Take articles. Feel free to post any questions or thoughts you have about my workflow and tools in the comments below. Courteous comments are always welcome.

Tags: , ,

Using Mock to Build Libvirt 1.0.1 RPMs for CentOS 6.3

Tuesday, January 22, 2013 in Linux, Virtualization by slowe | No comments

In this third post on using Mock to build RPMs for CentOS 6.3, I’m going to show you how to use Mock to build RPMs for Libvirt 1.0.1 that you can install on CentOS. As you’ll see later, this post builds on the previous two posts (one on using Mock to build RPMs for sanlock 2.4 and one on using Mock to build RPMs for libssh2 1.4.1).

Here’s a quick overview of the process:

  1. Set up Mock and the environment.
  2. Install prerequisites into the Mock environment.
  3. Build the Libvirt RPMs.

Let’s take a closer look at each of these steps.

Setting Up Mock and the Environment

The first phase in the process is to set up Mock and the environment for building the RPMs. Fortunately, this is relatively simple.

First, activate EPEL. My preferred method for activating the EPEL repository is to download the RPM, then use yum localinstall to install it, like this:

wget http://fedora.mirrors.pair.com/epel/6/i386/\
epel-release-6-8.noarch.rpm
yum localinstall epel-release-6-8.noarch.rpm

(Note that I’ve line-wrapped the URL with a backslash to make it more readable. That line-wrapped command actually works just as it is in the shell as well.)

Next, you’ll need to install Mock and related RPM-building tools:

yum install fedora-packager

Third, create a dedicated user for building RPMs. I use “makerpm” as my username, but you could use something else. Just make sure that the name makes sense to you, and that the new user is a member of the mock group:

useradd makerpm -G mock
passwd makerpm

From this point on, you’ll want to be running as this user you just created, so switch to that user with su - makerpm. This ensures that the RPMs are built under this dedicated user account.

The final step in setting up Mock and the build environment is to run the following command while running as the dedicated user you created:

rpmdev-setuptree

Now you’re ready to move on to the next phase: installing prerequisites into the Mock environment.

Installing Prerequisites Into the Mock Environment

One of the great things about Mock is that it creates an isolated chroot into which it installs all the necessary prerequisites for a particular package. This helps ensure that the package’s dependencies are managed correctly. However, if you are trying to build a package where dependencies don’t exist in the repositories, then you have to take a few additional steps. When you’re trying to build libvirt 1.0.1 RPMs for use with CentOS 6.3, you’ll find yourself in exactly this situation. Libvirt has dependencies on newer versions of sanlock-devel and libssh2-devel than are available in the repositories.

Fortunately, there is a workaround—and here’s where those other posts on Mock will come in handy. Use the instructions posted here to build RPMs for sanlock 2.4, and use the instructions here to build RPMs for libssh2 1.4.1.

Once the RPMs are built (and they should build without any major issues, based on my testing), then use these commands to get them into the isolated Mock environment (I’ve line-wrapped here with backslashes for readability):

mock -r epel-6-x86_64 --init
mock -r epel-6-x86_64 --install \
~/rpmbuild/RPMS/sanlock-lib-2.4-3.el6.x86_64.rpm \
~/rpmbuild/RPM/sanlock-devel-2.4-3.el6.x86_64.rpm \
~/rpmbuild/RPMS/libssh2-1.4.1-2.el6.x86_64.rpm \
~/rpmbuild/RPMS/libssh2-devel-1.4.1-2.el6.x86_64.rpm

This will install these packages into the Mock environment, not onto the general Linux installation.

Once you’ve gotten these packages compiled and installed, then you’re ready for the final phase: building the libvirt RPMs.

Building the Libvirt RPMs

As in my earlier post on compiling Libvirt 1.0.1 RPMs for CentOS 6.3, this final step is almost anti-climactic. That’s good, though, because it means you’ve done all the previous steps perfectly.

First, fetch the source RPM from the libvirt HTTP server:

wget http://libvirt.org/sources/libvirt-1.0.1-1.fc17.src.rpm

Next, move the source RPM into the ~/rpmbuild/SRPMS directory:

mv libvirt-1.0.1-1.fc17.src.rpm ~/rpmbuild/SRPMS

Finally, run Mock to rebuild the RPMs:

mock -r epel-6-x86_64 --no_clean ~/rpmbuild/SRPMS/libvirt-1.0.1-1.fc17.src.rpm

Note that the --no-clean parameter is required here to prevent Mock from cleaning out the chroot and getting rid of the packages you installed into the environment earlier.

This command should run without any errors or problems, and produce a set of RPMs (typically) found in /var/lib/mock/epel-6-x86_64/results. You can then take these RPMs and install them on another CentOS 6.3 system using yum localinstall.

Testing the RPMs

To verify that everything worked as expected, I tested the RPMs using these steps:

  1. Using a clean CentOS 6.3 VM (built using the “Minimal Desktop” option), I used yum groupinstall to install the Virtualization, Virtualization Client, Virtualization Platform, and Virtualization Tools groups. This installed version 0.9.10 of libvirt.

  2. I then installed the updated version of libvirt using yum localinstall. I had to specify the dependencies manually on the command line; I anticipate that had I been using a real repository, this would not have been the case. The updated libvirt, sanlock, and libssh2 packages all installed correctly.

  3. I started the libvirtd service (it worked), and ran virsh --version. It returned version 1.0.1.

I imagine there might be more comprehensive/better ways of testing the RPMs that I built, but they seemed to work fine on my end. If anyone has any other suggestions for how we can double-check to ensure the packages are working correctly, feel free to speak up in the comments below. I also welcome any other corrections, suggestions, or questions in the comments. Courteous comments are always welcome.

Tags: , , ,

Using Mock to Build libssh2 1.4.1 RPMs for CentOS 6.3

Tuesday, January 22, 2013 in Linux by slowe | 1 comment

As with the related post on using Mock to rebuild sanlock 2.4 for CentOS 6.3, this post might seem a bit odd. Don’t worry—I’ll tie it into something else very soon. In this post, I’ll show you how to use Mock to build RPMs for libssh2 1.4.1 for use with CentOS 6.3.

The information in this post is based on information found in two other very helpful pages:

Using Mock to test package builds
How to rebuild a package from Fedora or EPEL for RHEL, CentOS, or SL

I tested these instructions on a newly-built CentOS 6.3 VM, installed using the “Minimal Desktop” option. I haven’t tested it on other RHEL variants or other versions, so keep that in mind.

First, you’ll want to activate EPEL. You’ll do that by downloading the RPM and using yum localinstall to install it. You can also use rpm to install it directly from the URL, but I prefer using yum localinstall. I’ve line-wrapped the EPEL URL with a backslash for readability.

wget http://fedora.mirrors.pair.com/epel/6/i386/\
epel-release-6-8.noarch.rpm
yum localinstall epel-release-6-8.noarch.rpm

Once EPEL is installed, then install Mock and related tools:

yum install fedora-packager

This will download and install Mock and related tools such as rpmbuild.

Next, create a user under which you’ll run all these commands, and make sure this account is a member of the mock group:

useradd makerpm -G mock
passwd makerpm

From here on, you’ll want to be running as this user you just created, so switch to that user with su - makerpm.

The first step while running as the user you created is to setup the RPM build environment:

rpmdev-setuptree

Now that the directory structure is created, use wget to download the source RPM for libssh2 1.4.1-2 from the Fedora 17 release repository (the URL is line-wrapped here for readability):

wget http://dl.fedoraproject.org/pub/fedora/linux/releases/17\
/Everything/source/SRPMS/l/libssh2-1.4.1-2.fc17.src.rpm

Move the source RPM to the rpmbuild/SRPMS directory:

mv libssh2-1.4.1-2.fc17.src.rpm ~/rpmbuild/SRPMS

And, finally, rebuild the RPMs with mock:

mock --rebuild ~/rpmbuild/SRPMS/libssh2-1.4.1-2.fc17.src.rpm

Assuming everything completes successfully (it did on my CentOS 6.3 VM), then you’ll end up with a group of RPMs found in /var/lib/mock/epel-6-x86_64/results (the exact directory will vary based on OS version and build; I was using 64-bit CentOS 6.3). You should then be able to install those RPMs onto a CentOS 6.3 system using yum localinstall and the prerequisites will be managed properly.

Have fun!

Tags: , ,

Using Mock to Build Sanlock 2.4 RPMs for CentOS 6.3

Tuesday, January 22, 2013 in Linux by slowe | No comments

The topic of this post might seem a bit strange, but it will all make sense later. In this post, I’ll show you how to use Mock to build RPMs for sanlock 2.4 for use with CentOS 6.3.

The information in this post is based on information found in two other very helpful pages:

Using Mock to test package builds
How to rebuild a package from Fedora or EPEL for RHEL, CentOS, or SL

I tested these instructions on a newly-built CentOS 6.3 VM, installed using the “Minimal Desktop” option. I haven’t tested it on other RHEL variants or other versions, so keep that in mind.

First, you’ll want to activate EPEL. You’ll do that by downloading the RPM and using yum localinstall to install it. You can also use rpm to install it directly from the URL, but I prefer using yum localinstall. (Note that the URL for the EPEL RPM is line-wrapped here for readability.)

wget http://fedora.mirrors.pair.com/epel/6\
i386/epel-release-6-8.noarch.rpm
yum localinstall epel-release-6-8.noarch.rpm

Once EPEL is installed, then install Mock and related tools:

yum install fedora-packager

This will download and install Mock and related tools such as rpmbuild.

Next, create a user under which you’ll run all these commands, and make sure this account is a member of the mock group:

useradd makerpm -G mock
passwd makerpm

From here on, you’ll want to be running as this user you just created, so switch to that user with su - makerpm.

The first step while running as the user you created is to setup the RPM build environment:

rpmdev-setuptree

Now that the directory structure is created, use wget to download the source RPM for sanlock 2.4-3 from the Fedora 17 update repository (the URL is line-wrapped here for readability):

wget http://dl.fedoraproject.org/pub/fedora/linux/updates\
/17/SRPMS/sanlock-2.4-3.fc17.src.rpm

Move the source RPM to the rpmbuild/SRPMS directory:

mv sanlock-2.4-3.fc17.src.rpm ~/rpmbuild/SRPMS

And, finally, rebuild the RPMs with mock:

mock --rebuild ~/rpmbuild/SRPMS/sanlock-2.4-3.fc17.src.rpm

Assuming everything completes successfully (it did on my CentOS 6.3 VM), then you’ll end up with a group of RPMs found in /var/lib/mock/epel-6-x86_64/results (the exact directory will vary based on OS version and build; I was using 64-bit CentOS 6.3). You should then be able to install those RPMs onto a CentOS 6.3 system using yum localinstall and the prerequisites will be managed properly.

Enjoy!

Tags: , ,

Building Libvirt 1.0.1 RPMs for CentOS 6.3

Wednesday, January 16, 2013 in Linux, Virtualization by slowe | 5 comments

In previous articles, I’ve shown you how to compile libvirt 0.10.1 on CentOS 6.3, but—as several readers have pointed out in the comments to that and other articles—compiling packages from source may not be the best long-term approach. Not only does it make it difficult to keep the system up-to-date, it also makes automating the configuration of the host rather difficult. In this post, I’ll show you how to rebuild a source RPM for libvirt 1.0.1 so that it will install (and work) under CentOS 6.3. (These instructions should work for RHEL 6.3, too, but I haven’t tested them.)

Overview

The process for rebuilding a source RPM isn’t too terribly difficult, assuming that you can get the dependencies worked out. Here’s a quick look at the steps involved:

  1. Create a set of build directories for source RPMs.
  2. Download the source RPM and install all prerequisites/dependencies onto the system.
  3. Rebuild the source RPM for the destination system.

Let’s take a look at each of these steps in a bit more detail.

Create the Build Environment

The CentOS wiki has a great page for how to set up an RPM build environment. I won’t repeat all the steps here (refer to the wiki page instead), but here’s a quick summary of what’s involved:

  1. Install the necessary packages (typically you’ll need the rpmbuild, redhat-rpm-config, make, and gcc packages). You might also need certain development libraries, but this will vary according the source RPMs you’re rebuilding (more on that in the next section).
  2. Create the necessary directories under your home directory.

I’ll assume that you’ve followed the steps outlined in the CentOS wiki to set up your environment appropriately before continuing with the rest of this process.

Download the Source RPM and Install Prerequisites

The libvirt 1.0.1 source RPMs are available directly from the libvirt HTTP server, easily downloaded with wget:

wget http://libvirt.org/sources/libvirt-1.0.1-1.fc17.src.rpm

You can just download the source RPM to your home directory. Before you can build a new RPM from the source RPM, though, you’ll first need to install all the various prerequisites that libvirt requires. Most of them can be installed easily using yum with a command like this:

yum install xhtml1-dtds augeas libudev-devel \
libpci-access-devel yajl-devel, libpcap-devel libnl-devel \
avahi-devel radvd ebtables qemu-img iscsi-initiator-utils \
parted-devel device-mapper-devel numactl-devel netcfg-devel \
systemtap-sdt-devel scrub numad libblkid-devel

There are two dependencies, though—sanlock and libssh2—that require versions newer than what are available in the CentOS/EPEL repositories. For those, you’ll need to recompile your own RPMs. Fortunately, this is pretty straightforward. The next two sections provide more details on getting these prerequisites handled.

Building an RPM for sanlock

To build a CentOS 6.3 RPM for version 2.4 of sanlock (the minimum version needed by libvirt 1.0.1), first use wget to download a Fedora 17 version of the source RPM. I’ve wrapped the URL with a backslash for readability:

wget http://dl.fedoraproject.org/pub/fedora/linux/updates/17/SRPMS\
/sanlock-2.4-3.fc17.src.rpm

Next, install an prerequisite library using yum install libaio-devel.

Finally, use rpmbuild to rebuild the sanlock source RPM:

rpmbuild --rebuild sanlock-2.4-3.fc17.src.rpm

This process should proceed without any problems. The resulting RPMs that are created will be found in ~/rpmbuild/RPMS/x86_64 (assuming you are, as I am, using a 64-bit build of CentOS).

Building the RPMs, however, isn’t enough—you need to install them so that you can build the libvirt RPMs. So install the sanlock-devel and sanlock-lib RPMs using yum locainstall (do this command from the directory where the RPMs reside):

yum localinstall sanlock-devel-* sanlock-lib-*

That should take care of the sanlock dependency.

Building an RPM for libssh2

To build a CentOS 6.3 RPM for version 1.4.1 of libssh2 (libvirt 1.0.1 requires at least version 1.3.0), first download the source RPM using wget (I’ve wrapped the URL here for readability):

wget http://dl.fedoraproject.org/pub/fedora/linux/releases\
/17/Everything/source/SRPMS/l/libssh2-1.4.1-2.fc17.src.rpm

(That’s a lowercase L in the URL just after SRPMS.)

Once you have the source RPM downloaded, then just rebuild the source RPM:

rpmbuild --rebuild libssh2-1.4.1-2.fc17.src.rpm

Then install the resulting RPMs using yum localinstall:

yum localinstall libssh2-1.4.1-* libssh2-devel-*

That takes care of the last remaining dependency. You’re now ready to compile the RPMs for libvirt 1.0.1.

Build the Libvirt RPM

This part is almost anticlimactic. Just use the rpmbuild command:

rpmbuild --rebuild libvirt-1.0.1-1.fc17.src.rpm

If you’ve successfully installed all the necessary prerequisites, then the RPM compilation process should proceed without any issues.

Once the RPM compilation process is complete, you’ll find libvirt 1.0.1 RPMs in the ~/rpmbuild/RPMS/x86_64 directory (assuming a 64-bit version of CentOS) which you can easily install with yum localinstall, or post to your own custom repository.

I hope this post helps someone. If you have any questions, or if you spot an error, please speak up in the comments below. All courteous comments are welcome!

Tags: , , ,

« Older entries § Newer entries »