Puppet

You are currently browsing articles tagged Puppet.

This is a liveblog of a session on using Puppet to build a highly available (HA) installation of OpenStack. The presenter is Boris Renski of Mirantis.

Boris believes that you need to know many things in order to successfully create the build architecture for an OpenStack deployment:

  • Linux
  • Networking
  • Virtualization
  • Python
  • Ruby
  • Puppet
  • Cobbler (or Razor)
  • mCollective/Salt

Boris introduces Fuel, which is an automation library for OpenStack (that is supposedly easy enough for a goat to use—a play on Mirantis’ geographical location and the Borat movies).

Fuel essentially includes the following items:

  • An OpenStack reference architecture with HA
  • Puppet manifests for deploying OpenStack
  • Cobbler-based bare metal provisioning
  • OpenStack packages
  • Support for CentOS, RHEL, and Ubuntu
  • Support for OpenStack Essex, Folsom, and (in May) Grizzly
  • A detailed configuration guide

Fuel supports a number of different deployment configurations: single node (pretty straightforward, much like DevStack); multi-node (including compact Swift and standalone Swift); and multi-node HA (with compact and standalone Swift and Quantum elements). “Swift compact” is for when Swift will be used only as back-end storage for VMs. “Quantum compact” is running Quantum on the controller node, even with high availability.

Fuel was specifically created in the form of a library so that users could easily modify and adopt the scripts to fit their particular OpenStack deployment. This gives users more flexibility when using Fuel to deploy OpenStack.

For the HA architecture:

  • They use HAProxy for most of the OpenStack services
  • For the message queue, they use an active/active RabbitMQ cluster
  • For the database, they use an active/active Galera MySQL cluster (this forces a minimum of four physical nodes)
  • The architecture uses keepalived for VIP (virtual IP) management

The overall process for deploying OpenStack with Fuel goes like this:

  1. Build the Fuel “master node,” which runs Cobbler and Puppet master
  2. Enter hardware info into Cobbler
  3. Cobbler installs the base OS (CentOS, RHEL, Ubuntu)
  4. Puppet picks up the node and installs/configures OpenStack

Next, Boris goes into a more light-hearted section on how he taught a goat to use Fuel. For us humans, this means the “Fuel portal,” which provides step-by-step instructions on using Fuel. They (Mirantis) also created “Fuel Web,” which is an easy GUI for Fuel. A private beta for Fuel Web is starting today.

Boris now turns the stage over to Roman, who shows a live demo of using Fuel to turn up a 2-node OpenStack deployment. Overall, Fuel looks like a very interesting and useful tool.

Looking ahead, what’s on the Fuel roadmap? Roman wants to add screens for the management of disks and NICs, which don’t exist in Fuel Web today. There’s also no support for Cinder in the web UI today, which is another item they’d like to add in future releases. They are also considering some level of monitoring and performance metrics for the OpenStack environments deployed using Fuel. Finally, they want to extend Fuel to help with OpenStack upgrades as well.

Fuel is available for download at http://fuel.mirantis.com.

Tags: , ,

Welcome to Technology Short Take #31, my irregularly published series that takes a look at links, posts, articles, and thoughts from around the web related to core data center technologies. I hope that you find something useful!

Networking

  • Umair Hoodbhoy speculates in this post that the inclusion of Cisco’s ONE Controller in the recently-announced “Daylight” effort could mean the end for Big Switch’s Floodlight. (Umair’s play on words—”in Daylight there is no need for Floodlights”—is cute.)
  • Of course, Big Switch recently moved to “diversify,” if you will, away from just Floodlight with the introduction of Switch Light. As usual, Brent Salisbury has an excellent write-up on Switch Light, so I recommend reading his post. Switch Light seems like a good idea—more competition is always good, isn’t that what people say?—but I wonder how much cooperation Big Switch will get from the major networking vendors with regards to OpenFlow interoperability now that Big Switch is competing even more directly with them via Switch Light.
  • I think I might have mentioned this before (sorry if so), but here’s a good write-up on using the Edge Gateway CLI for monitoring and troubleshooting. Nice.
  • Greg Ferro examines a potential SDN use case (an OpenFlow use case) in the form of enterprise firewall migrations.
  • Just getting started in the networking field? Last year, Brent Salisbury put together a couple of great posts that help “refresh the basics” of networking. Part 1 covers Ethernet, IP, and TCP headers in Wireshark captures; part 2 pulls that together to show how the headers encapsulate in the OSI stack. If you’re not already familiar with this information, this is good reading.

Servers/Hardware

Nothing this time around, but I’ll stay alert for information I can include in the next Technology Short Take!

Security

  • Mounting guest disk images on the host? That’s a no-no from a security perspective—see here to learn why.
  • Mike Foley shared recently that the release candidate of the vSphere 5 Security Hardening Guide has been released. Check it out here.

Cloud Computing/Cloud Management

  • I haven’t had the chance to actually try it out myself, but Blueprint looks interesting. As the website describes it, it’s designed to “reverse engineer” servers so that you can migrate them into a configuration management system like Chef or Puppet.
  • Looking for a decent high-level overview of OpenStack and how it works? Check out this article titled “In a nutshell: How OpenStack works”. (As an aside, I think it’s awesome how Ken Pepple’s diagrams show up in all sorts of places. One day I hope my material proves as useful to folks.)
  • If you use Puppet for configuration management and want to deploy GlusterFS, be sure to check out this Puppet Forge module. I’ve tested it and it works as advertised.
  • This is an older article (published in May of last year), and it’s a bit on the lengthy side, but I like the tack the author uses. He describes cloud as the synthesis of many different forms of innovation within IT, pulling together things like open source, virtualization, distributed programming, NoSQL, DevOps/NoOps, distributed teams, dynamic languages, and Big Data (among others). He then goes on to provide examples of how organizations building or leveraging clouds are synthesizing these various independent technological innovations together. If you have a few minutes (as I said, it’s a bit on the lengthy side), I’d recommend reading it.

Operating Systems/Applications

  • This series is a bit older, but an interesting one nevertheless. Brian McClain, who was one of the presenters in a Cloud Foundry/BOSH session I liveblogged at VMworld 2012, has his own personal blog and posted a series of articles on using BOSH with vSphere. I hadn’t really considered how one might use BOSH for deploying (and managing) multi-VM applications on vSphere, but Brian provides some practical examples. Part 1 of the series is here, followed by part 2, part 3, part 4, and part 5.
  • Like using Markdown on OS X? You might find these handy.
  • Ah, the good old days of DOS…reborn as FreeDOS.
  • Go ahead, read up on YAML. You know you want to. Well, YAML is used in both Hiera (can be used with Puppet) and BOSH, after all.
  • Here’s another interesting tool that I haven’t had the opportunity to actually test myself. Oz looks like it could be quite useful—especially in virtualized/cloud computing environments—but I’m struggling to determine why I should use Oz instead of OS-specific mechanisms (like a kickstart file). If anyone has used Oz and can shed some light on this question, I’d appreciate it.
  • You may have heard that I recently switched from TextMate to BBEdit as my default OS X text editor (and therefore the tool whereby I do most of my content generation). As part of the switch, I found this to be helpful. (I might post a separate entry about the switch, if enough people seem interested in reading about it.)

Storage

Virtualization

That’s it for this time. I have plenty more links I wanted to share, but I figured I’d better not let this post get any longer. As always, courteous comments are welcome, so I invite you to participate in the conversation by adding your thoughts below.

Tags: , , , , , , , , , ,

Technology Short Take #30

Welcome to Technology Short Take #30. This Technology Short Take is a bit heavy on the networking side, but I suppose that’s understandable given my recent job change. Enjoy!

Networking

  • Ben Cherian, Chief Strategy Officer for Midokura, helps make a case for network virtualization. (Note: Midokura makes a network virtualization solution.) If you’re wondering about network virtualization and why there is a focus on it, this post might help shed some light. Given that it was written by a network virtualization vendor, it might seem a bit rah-rah, so keep that in mind.
  • Brent Salisbury has a fantastic series on OpenFlow. It’s so good I wish I’d written it. He starts out by discussing proactive vs. reactive flows, in which Brent explains that OpenFlow performance is less about OpenFlow and more about how flows are inserted into the hardware. Next, he tackles the concerns over the scale of flow-based forwarding in his post on coarse vs. fine flows. I love this quote from that article: “The second misnomer is, flow based forwarding does not scale. Bad designs are what do not scale.” Great statement! The third post in the series tackles what Brent calls hybrid SDN deployment strategies, and Brent provides some great design considerations for organizations looking to deploy an SDN solution. I’m looking forward to the fourth and final article in the series!
  • Also, if you’re looking for some additional context to the TCAM considerations that Brent discusses in his OpenFlow series, check out this Packet Pushers blog post on OpenFlow switching performance.
  • Another one from Brent, this time on Provider Bridging and Provider Backbone Bridging. Good explanation—it certainly helped me.
  • This article by Avi Chesla points out a potential security weakness in SDN, in the form of a DoS (Denial of Service) attack where many switching nodes request many flows from the central controller. It appears to me that this would only be an issue for networks using fine-grained, reactive flows. Am I wrong?
  • Scott Hogg has a nice list of 9 common Spanning Tree mistakes you shouldn’t make.
  • Schuberg Philis has a nice write-up of their CloudStack+NVP deployment here.

Servers/Hardware

  • Alex Galbraith recently posted a two-part series on what he calls the “NanoLab,” a home lab built on the Intel NUC (“Next Unit of Computing”). It’s a good read for those of you looking for some very quiet and very small home lab equipment, and Alex does a good job of providing all the details. Check out part 1 here and part 2 here.
  • At first, I thought this article was written from a sarcastic point of view, but it turns out that Kevin Houston’s post on 5 reasons why you may not want blade servers is the real deal. It’s nice to see someone who focuses on blade servers opening up about why they aren’t necessarily the best fit for all situations.

Security

  • Nick Buraglio has a good post on the potential impact of Arista’s new DANZ functionality on tap aggregation solutions in the security market. It will be interesting to see how this shapes up. BTW, Nick’s writing some pretty good content, so if you’re not subscribed to his blog I’d reconsider.

Cloud Computing/Cloud Management

  • Although this post is a bit older (it’s from September of last year), it’s still an interesting comparison of both OpenStack and CloudStack. Note that the author apparently works for Mirantis, which is a company that provides OpenStack consulting services. In spite of that fact, he manages to provide a reasonably balanced approach to comparing the two cloud management platforms. Both of them (I believe) have had releases since this time, so some of the points may not be valid any longer.
  • Are you a CloudStack fan? If so, you should probably check out this collection of links from Aaron Delp. Aaron’s focused a lot more on CloudStack now that he’s at Citrix, so he might be a good resource if that is your cloud management platform of choice.

Operating Systems/Applications

  • If you’re just now getting into the whole configuration management scene where tools like Puppet, Chef, and others play, you might find this article helpful. It walks through the difference between configuring a system imperatively and configuring a system declaratively (hint: Puppet, Chef, and others are declarative). It does presume a small bit of programming knowledge in the examples, but even as a non-programmer I found it useful.
  • Here’s a three-part series on beginning Puppet that you might find helpful as well (Part 1, Part 2, and Part 3).
  • If you’re a developer-type person, I would first ask why you’re reading my site, then I’d point you to this post on the AMQP, MQTT, and STOMP messaging protocols.

Storage

Virtualization

  • Although these posts are storage-related, the real focus is on how the storage stack is implemented in a virtualization solution, which is why I’m putting them in this section. Cormac Hogan has a series going titled “Pluggable Storage Architecture (PSA) Deep Dive” (part 1 here, part 2 here, part 3 here). If you want more PSA information, you’d be hard-pressed to find a better source. Well worth reading for VMware admins and architects.
  • Chris Colotti shares information on a little-known vSwitch advanced setting that helps resolve an issue with multicast traffic and NICs in promiscuous mode in this post.
  • Frank Denneman reminds everyone in this post that the concurrent vMotion limit only goes to 8 concurrent vMotions when vSphere detects the NIC speed at 10Gbps. Anything less causes the concurrent limit to remain at 4. For those of you using solutions like HP VirtualConnect or similar that allow you to slice and dice a 10Gb link into smaller links, this is a design consideration you’ll want to be sure to incorporate. Good post Frank!
  • Interested in some OpenStack inception? See here. How about some oVirt inception? See here. What’s that? Not familiar with oVirt? No problem—see here.
  • Windows Backup has native Hyper-V support in Windows Server 2012. That’s cool, but are you surprised? I’m not.
  • Red Hat and IBM put out a press release today on improved I/O performance with RHEL 6.4 and KVM. The press release claims that a single KVM guest on RHEL 6.4 can support up to 1.5 million IOPS. (Cue timer until next virtualization vendor ups the ante…)

I guess I should wrap things up now, even though I still have more articles that I’d love to share with readers. Perhaps a “mini-TST”…

In any event, courteous comments are always welcome, so feel free to speak up below. Thanks for reading and I hope you’ve found something useful!

Tags: , , , , , , , , , , , ,

In this post, I’m going to show you a way to automate the configuration of Open vSwitch (OVS) using Puppet. While this method will work, it is not without its drawbacks (which I’ll explain later in the post). I freely admit that there might be better ways of accomplishing this task; this is one simple way that I discovered.

I wish I could say that this method of automating OVS with Puppet was clever, but—to be honest—it’s really more of a brutish hack that anything else. In an earlier post, I described some integrations between RHEL and OVS that allows you to use interface configuration files in /etc/sysconfig/network-scripts to configure portions of OVS. For example, you could use this interface configuration file to create an OVS internal interface for management traffic:

Further, based on a number of the Puppet-related posts I’ve written (this one, for example), you probably know that Puppet has the ability to enforce the presence and contents of file-based resources.

So, Puppet can create and manage files, and files can be used to configure OVS. You can probably see where this is going. That’s right—if we use Puppet to manage interface configuration scripts, we can automate the configuration of OVS (only on systems running RHEL or RHEL variants, naturally).

Here’s a snippet of Puppet code that could be used to automate the configuration of OVS to use an internal interface for management traffic:

As I said, this is a bit of a brutish hack—not an elegant solution, but one that works. Naturally, because this builds on the RHEL-OVS integrations, you’ll need to do an ifdown and/or ifup to make the change(s) effective. (One could likely use an exec statement in the Puppet manifest to run these commands for you.) Another drawback is that this only works on RHEL and RHEL variants, whereas both OVS and Puppet are far more widely supported. Still, it might come in handy in some situations.

If you have corrections, clarifications, or suggestions, please feel free to speak up in the comments below. Courteous comments are both encouraged and welcomed!

Tags: , , , , ,

In an earlier post on using Puppet to manage user accounts, I showed how I was using virtual user resources to manage user accounts on Puppet-managed nodes. Today, I ran into a situation where I wanted one of these virtual user resources to have a different group membership on one class of nodes than on another class of nodes. In this post, I’ll show you how I managed to modify the group membership of a realized virtual user resource.

(Disclaimer: There are probably better, more elegant ways of solving this problem. The purpose of this post is to help stimulate discussion and encourage learning, not to say this is the only way to do something. Keep that in mind as you read.)

To refresh your memory, this is how the virtual user resources were being realized on individual nodes:

Note the use of capitalized “Account::Virtual”, which means we are referring to a specific, defined (virtually defined, at least) resource.

My first attempt was to try overriding, where I refer back to the realized user resource using a similar syntax, then specify a particular attribute (the groups attribute, in this case). I’d used this technique in the past with non-virtual resources (example here), but—unfortunately—it didn’t seem to work with virtual resources.

A web search took me to this Google Groups thread, where I found two critical pieces of information:

  1. Realizing apparently doesn’t allow overrides.
  2. You can use the collection function to modify realized resources.

Armed with that information, I added this code to the nodes.pp manifest to modify the group membership for a realized virtual resource on specific nodes:

That worked! On the nodes where this code was added, the account named “johndoe” was added to the group “othergroup” successfully.

Great, so this solution works, but really the bigger questions are these:

  • Are there better, more elegant ways of solving this particular problem?
  • What drawbacks are there to this solution?

Yes, I could modify the accounts::virtual class to include supplementary group memberships, and that’s probably a better long-term solution. However, that requires more testing, since you’re modifying code that affects user accounts across a number of systems. This solution, on the other hand, is pretty quick and relatively safe.

Thoughts? Ideas? I’d love to hear your feedback, so speak up in the comments below.

Tags: , ,

In early January 2012, I posted a list of my 2012 projects. Those projects included learning to script in Perl, learning German, becoming more familiar with Xen/Open vSwitch/OpenStack, and pursuing CCNP. In late June 2012, my mid-year project update let you know that I was dropping the Perl efforts, but sticking it out with German, Xen/OVS/OpenStack, and CCNP. (I later dropped Xen in favor of KVM.) Finally, in early January 2013 I graded myself on my progress on my 2012 goals. In this post, I’d like to take lessons learned from my 2012 projects and apply them toward my list of 2013 projects.

As a quick summary, here are some key “lessons learned” from my 2012 projects:

  • The synergy of the projects does make a difference. I should try to find projects that are related and that support one another in some fashion.
  • Projects need to be more tightly defined. Projects that are overly broad will have mixed results.
  • There are still some fundamental building blocks that I need to reinforce.

With the conclusions listed above in mind, here is my list of 2013 projects and—in some cases—some proposed goals for which I’ll be aiming.

  1. Continue to learn German. Obviously, this is a continuation of my 2012 project to learn to speak German. This year, I’ll need to re-evaluate ways to enhance my motivation, and find additional ways to reinforce the information I’m learning. I don’t have any specific goals in mind for this project, although that is something I’ll be evaluating over the course of the year (I do agree that clear goals—something I didn’t establish last year—can help with progress.)

  2. Reinforce base Linux knowledge. This is one of the “fundamental building blocks” that I needed to reinforce. I’ll be focusing on Red Hat Enterprise Linux (RHEL) and RHEL variants, since that seems to be what’s most commonly found in enterprise deployments today. My primary goal here is to be able to do most anything that I need to do without constantly having to look up a “how to” guide; a secondary goal that I’m considering (but haven’t decided upon) would be to look at a Red Hat-focused certification (like RHCE).

  3. Continue using Puppet for automation. This will be partly tied to #2. I’ve mentioned elsewhere that you can’t automate something you don’t fully understand, so as my base Linux knowledge is reinforced I will also be seeking ways to automate tasks with Puppet. As with my base Linux knowledge, my primary goal here is to be able to do most anything that I need to do with Puppet. I might consider Puppet certification, but I think that’s a long shot.

  4. Expand and deepen data center networking fundamentals. Making progress on CCNP was one of my 2012 goals, but it didn’t really fit in with the rest of my focus areas (which were almost all centered in the data center). To help improve the synergy of projects, then, I’ve decided that my networking focus should be DC-oriented. This also ties in nicely with my new job role at Nicira/VMware. My primary goal here is simply to be more knowledgeable, although over the course of the year I will consider pursuing CCNA-DC.

So, that’s my list of 2013 projects. I’m sure that I’ll have a few side projects here and there as well (for example, there is a book project and at least one video training series project on the books for 2013), but the projects listed above represent the majority of my technical focus over the coming year. I’d love to hear your feedback—am I missing something important? Are there ways I could further improve the synergy of the projects? As always, your honest and courteous comments are welcome.

Tags: , , , , ,

Welcome to Technology Short Take #29! This is another installation in my irregularly-published series of links, thoughts, rants, and raves across various data center-related fields of technology. As always, I hope you find something useful here.

Networking

  • Who out there has played around with Mininet yet? Looks like this is another tool I need to add to my toolbox as I continue to explore networking technologies like OpenFlow, Open vSwitch, and others.
  • William Lam has a recent post on some useful VXLAN commands found in ESXCLI with vSphere 5.1. I’m a CLI fan, so I like this sort of stuff.
  • I still have a lot to learn about OpenFlow and networking, but this article from June of last year (it appears to have been written by Ivan Pepelnjak) discusses some of the potential scalability concerns around early versions of the OpenFlow protocol. In particular, the use of OpenFlow to perform granular per-flow control when there are thousands (or maybe only hundreds) of flows presents a scalability challenge (for now, at least). In my mind, this isn’t an indictment of OpenFlow, but rather an indictment of the way that OpenFlow is being used. I think that’s the point Ivan tried to make as well—it’s the architecture and how OpenFlow is used that makes a difference. (Is that a reasonable summary, Ivan?)
  • Brad Hedlund (who will be my co-worker starting on 2/11) created a great explanation of network virtualization that clearly breaks down the components and explains their purpose and function. Great job, Brad.
  • One of the things I like about Open vSwitch (OVS) is that it is so incredibly versatile. Case in point: here’s a post on using OVS to connect LXC containers running on different hosts via GRE tunnels. Handy!

Servers/Hardware

  • Cisco UCS is pretty cool in that it makes automation of compute hardware easier through such abstractions as server profiles. Now, you can also automate UCS with Chef. I traded a few tweets with some Puppet folks, and they indicated they’re looking at this as well.
  • Speaking of Puppet and hardware, I also saw a mention on Twitter about a Puppet module that will manage the configuration of a NetApp filer. Does anyone have a URL with more information on that?
  • Continuing the thread on configuration management systems running on non-compute hardware (I suppose this shouldn’t be under the “Servers/Hardware” section any longer!), I also found references to running CFEngine on network apliances and running Chef on Arista switches. That’s kind of cool. What kind of coolness would result from even greater integration between an SDN controller and a declarative configuration management tool? Hmmm…

Security

  • Want full-disk encryption in Ubuntu, using AES-XTS-PLAIN64? Here’s a detailed write-up on how to do it.
  • In posts and talks I’ve given about personal productivity, I’ve spoken about the need to minimize “friction,” that unspoken drag that makes certain tasks or workflows more difficult and harder to adopt. Tal Klein has a great post on how friction comes into play with security as well.

Cloud Computing/Cloud Management

  • If you, like me, are constantly on the search for more quality information on OpenStack and its components, then you’ll probably find this post on getting Cinder up and running to be helpful. (I did, at least.)
  • Mirantis—recently the recipient of $10 million in funding from various sources—posted a write-up in late November 2012 on troubleshooting some DNS and DHCP service configuration issues in OpenStack Nova. The post is a bit specific to work Mirantis did in integrating an InfoBlox appliance into OpenStack, but might be useful in other situation as well.
  • I found this article on Packstack, a tool used to transform Fedora 17/18, CentOS 6, or RHEL 6 servers into a working OpenStack deployment (Folsom). It seems to me that lots of people understand that getting an OpenStack cloud up and running is a bit more difficult than it should be, and are therefore focusing efforts on making it easier.
  • DevStack is another proof point of the effort going into make it easier to get OpenStack up and running, although the focus for DevStack is on single-host development environments (typically virtual themselves). Here’s one write-up on DevStack; here’s another one by Cody Bunch, and yet another one by the inimitable Brent Salisbury.

Operating Systems/Applications

  • If you’re interested in learning Puppet, there are a great many resources out there; in fact, I’ve already mentioned many of them in previous posts. I recently came across these Example42 Puppet Tutorials. I haven’t had the chance to review them myself yet, but it looks like they might be a useful resource as well.
  • Speaking of Puppet, the puppet-lint tool is very handy for ensuring that your Puppet manifest syntax is correct and follows the style guidelines. The tool has recently been updated to help fix issues as well. Read here for more information.

Storage

  • Greg Schulz (aka StorageIO) has a couple of VMware storage tips posts you might find useful reading. Part 1 is here, part 2 is here. Enjoy!
  • Amar Kapadia suggests that adding LTFS to Swift might create an offering that could give AWS Glacier a real run for the money.
  • Gluster interests me. Perhaps it shouldn’t, but it does. For example, the idea of hosting VMs on Gluster (similar to the setup described here) seems quite interesting, and the work being done to integrate KVM/QEMU with Gluster also looks promising. If I can ever get my home lab into the right shape, I’m going to do some testing with this. Anyone done anything with Gluster?
  • Erik Smith has a very informative write-up on why FIP snooping is important when using FCoE.
  • Via this post on ten useful OpenStack Swift features, I found this page on how to build the “Swift All in One,” a useful VM for learning all about Swift.

Virtualization

  • There’s no GUI for it, but it’s kind of cool that you can indeed create VM anti-affinity rules in Hyper-V using PowerShell. This is another example of how Hyper-V continues to get more competent. Ignore Microsoft and Hyper-V at your own risk…
  • Frank Denneman takes a quick look at using user-defined NetIOC network resource pools to isolate and protect IP-based storage traffic from within the guest (i.e., using NFS or iSCSI from within the guest OS, not through the VMkernel). Naturally, this technique could be used to “protect” or “enhance” other types of important traffic flows to/from your guest OS instances as well.
  • Andre Leibovici has a brief write-up on the PowerShell module for the Nicira Network Virtualization Platform (NVP). Interesting stuff…
  • This write-up by Falko Timme on using BoxGrinder to create virtual appliances for KVM was interesting. I might have to take a look at BoxGrinder and see what it’s all about.
  • In case you hadn’t heard, OVF 2.0 has been announced/released by the DMTF. Winston Bumpus of VMware’s Office of the CTO has more information in this post. I also found the OVF 2.0 frequently asked questions (FAQs) to be helpful. Of course, the real question is how long it will be before vendors add support for OVF 2.0, and how extensive that support actually is.

And that’s it for this time around! Feel free to share your thoughts, suggestions, clarifications, or corrections in the comments below. I encourage your feedback, and thanks for reading.

Tags: , , , , , , , , , , , , , , ,

A short while ago I posted an article that described how to use Puppet for account management. In that article, I showed you how to use virtual user resources to manage user accounts on Puppet-managed systems. In this post, I’m going to expand on that configuration so that we can also manage the initial account configuration files, and do so in the proper order.

One of the things the configuration in my first post didn’t handle was the Puppet configuration of the files in /etc/skel and making sure those files were in place before the user accounts were created. As a result, it was possible that the user account could be created on a system before the /etc/skel files were updated, and then that user account would have “unmanaged” copies of the initial configuration files. Further Puppet agent runs wouldn’t correct the problem, because the files in /etc/skel are only copied over when the account is created. If the account has already been created, then it’s too late—the files in /etc/skel must be managed before the accounts are. To fix the issue, you have to ensure that the resources are processed in a specific manner. In this post, I’ll show you how to manage that.

There are two parts to extending the Puppet accounts module to also manage some configuration files:

  1. Add a subclass to manage the files.
  2. Create a dependency between the virtual user resources and this new subclass.

Let’s look at each of these.

Adding a Subclass

To add a subclass to manage the configuration files, I created config.pp and placed it in the manifests folder for the accounts module. Here’s a simplified look at the contents of that file:

This is pretty straightforward Puppet code; it creates a managed file resource and specifies that the file be sourced from the Puppet master server. The full and actual accounts::config subclass that I’m using has a number of managed file resources, including files in /etc/skel, but I’ve omitted that here for the sake of brevity. (The other file resources that are defined look very much like the example shown, so I didn’t see any point in including them.) The config.pp also uses some values from an accounts::params subclass and some conditionals to manage different files on different operating systems.

To really put the subclass to work, though, we have to include it elsewhere. So, in the accounts module’s init.pp, we add a line that simply states include accounts::config. However, the problem that occurs if you stop there is the problem I described earlier: Puppet might create the user account before it places the file resources under management, and then the user account won’t get the updated/managed files.

To fix that, we create a dependency.

Creating a Dependency

Before running into this situation, I was pretty familiar with creating dependencies. For example, if you were defining a class for a particular daemon to run on Linux, you might use the Puppet package-file-service “trifecta”, and you might include a dependency, like this (entirely fictional) example. Note in this example that the file resource is dependent on the package resource, and the service resource is dependent on the file resource (as denoted by the capitalized Package and File instances):

(My apologies if my syntax for this fictional example isn’t perfect—I didn’t run it through puppet-lint.)

The problem in this particular case, though, is that I didn’t need a dependency on a single file; I needed a dependency on a whole group of files. To further complicate matters, the files on which the dependency existed might change between operating systems. For example, I might (and do) have different files on RHEL/CentOS than on Ubuntu/Debian. So how to accomplish this? The answer is actually quite simple: create a dependency on the subclass, not the individual resources.

So, without the dependency, the code to define the virtual users looked like this:

With the dependency, the code to define the virtual users looks like this:

The only difference between the two (other than changes in the comments at the top) is the addition of the require statement, which creates a dependency not on a single resource but instead to an entire subclass—the accounts::config subclass, which in turn has a variety of file resources that are managed according to operating system.

It’s such a simple solution I can’t believe I didn’t see it at first, and when it was pointed out to me (via the #puppet IRC channel, thanks), I had a “Duh!” moment. Even though it is a simple and straightforward solution, if I overlooked it then others might overlook it as well—a problem that hopefully this blog post will help fix.

As always, I welcome feedback from readers, so feel free to weigh in with questions, clarifications, or corrections. Courteous comments are always welcome!

Tags: , , ,

Welcome to Technology Short Take #27! This is my usual collection of links, thoughts, rants, and ideas about data center-related technologies. Here’s hoping you find something useful!

Networking

  • If you’re interested in learning more about OpenFlow and software-defined networking but need to do this on a shoestring budget in your home lab, a number of guides have been written to help out. I haven’t personally used any of these guides yet, but I’m working my way in that direction. (I needed to fill in some other knowledge gaps first.) First up is Brent Salisbury’s how to build an SDN lab without needing OpenFlow hardware. Brent is creating some fantastic content that I’ve found extremely useful. His earlier post on getting started with OpenFlow and Open vSwitch tutorial lab is also quite good. Another good resource is Dan Hersey’s guide to building an SDN-based private cloud in an hour. I encourage you to have a look at these posts if you’re at all interested in any of these technologies.
  • Bruce Davie and Martin Casado (with Nicira, now part of VMware) have written a post comparing the VXLAN and STT tunneling protocols. Not unsurprisingly, one of the key advantages of STT that’s highlighted is its improved performance due to TSO support in NIC hardware. VXLAN, on the other hand, is seeing broader adoption across multiple vendors. There’s no mention of NVGRE (or just plain GRE).
  • Related to the bare metal provisioning work (see below under “Servers/Hardware”), Mirantis also detailed some bare-metal networking stuff they’ve done for OpenStack in relation to the use of bare metal nodes.

Servers/Hardware

  • Mirantis published an article discussing a framework they built for bare-metal provisioning with OpenStack that allows OpenStack to place workloads onto bare-metal nodes instead of onto a hypervisor. It’s interesting work, but unfortunately it looks like this work won’t be returned to the community (it was developed for one or more of their clients). There are also a few follow-up posts, such as this one on placement control and multi-tenancy isolation and this one on preparing images for bare metal nodes. Also see the “Networking” section above for a related post on the networking aspects involved.

Security

I don’t have anything for this area this time around, but I’ll stay alert for articles to add next time. Feel free to share something in the comments!

Cloud Computing/Cloud Management

  • I might have mentioned this before, but Ken Pepple’s OpenStack Folsom architecture post is just awesome. It’s well worth reading and reviewing in depth.
  • This OpenStack-on-Debian HOWTO is a bit older (and probably out of date), but it does give a decent overview of the components that are involved and—via the configuration—how they relate to each other. While the details for installing a current version of OpenStack are likely to be different now, you might still find this conceptually helpful.
  • These articles are a bit long in the tooth, but CSS Corp has a useful series of articles on bundling various Linux distributions for use with OpenStack: bundling CentOS, bundling CentOS with VNC, bundling Debian, and bundling OpenSUSE. It would be interesting to me to see how much of this, if any, could be automated with something like Puppet. If any enterprise Puppet experts want to give it a go, I’d be happy to publish a guest blog post for you with full details on how it’s done.
  • Much like there are some great “how to’s” on how to run an SDN lab (see the Networking section earlier), there are also some great write-ups on doing the same for OpenStack. For example, Cody Bunch published this article on running OpenStack Private Cloud on ESXi, and Brent Salisbury (there he is again!) posted an older guide to OpenStack Essex on Ubuntu on VirtualBox as well as a newer guide to OpenStack DevStack on Fusion.

Operating Systems/Applications

Storage

  • I don’t fully understand all the details involved, but this post on changes in block protocol scalability in Xen outlines what sounds like good progress in improving efficiency.
  • This article is a bit older, published at the start of October, but it talks about an interesting project (product?) by Qlogic called “Mt. Rainier.” (Stu Miniman of Wikibon has more information here as well.) Apparently, “Mt. Rainier” will allow customers to combine PCIe-based SSD storage inside servers into a “virtual SAN” (now there’s an original and not over-used term). The really interesting aspect, in my opinion, is the use of “Mt. Rainier” to create shared caches across servers. Is this the beginning of the data center fractal edge?

Virtualization

  • Big news in the QEMU world: In the QEMU 1.3 release, the QEMU-KVM and QEMU projects have been merged. Why is this important? It’s first necessary to understand the relationship between QEMU and KVM. KVM is the set of kernel modules that leverage hardware virtualization functionality inside Intel and AMD CPUs, and it makes possible the virtualization of closed-source operating systems like Windows. QEMU, on the other hand, is needed to emulate everything else that a VM needs: networking, storage, USB, keyboard, mouse, etc. Both KVM and QEMU are needed for a full virtualization solution. Until the 1.3 release, QEMU (without hardware acceleration via KVM) was one branch, and QEMU-KVM (with KVM hardware acceleration) was a separate branch. The QEMU 1.3 release completes an effort to merge both efforts into a single development tree.
  • The merge of QEMU and QEMU-KVM isn’t the only cool thing happening with QEMU; also included in the 1.3 release is GlusterFS integration. This integration dramatically improves GlusterFS performance by allowing QEMU’s block layer to communicate directly with the Gluster backend without going through the userspace FUSE components.
  • Erik Scholten of VMGuru.nl has posted a good hypervisor feature comparison document. It includes RHEV 3.1 in the comparison, even though RHEV 3.1 wasn’t released (was still in beta) at the time the comparison was written.
  • Speaking of RHEV: apparently RHEV 3.1 was released yesterday (Wednesday, December 4, 2012), although I haven’t been able to find any sort of official press release or announcement.
  • Debunking an argument I’ve heard quite a bit is this article by Frank Denneman on using SIOC with multiple datastores backed by a single pool of disks.
  • Need to compact a virtual hard disk in Windows 8/Windows Server 2012? Ben Armstrong shows how here.
  • I enjoyed this article by Josh Townsend on using SUSE Studio and HAProxy to create a (free) open source load balancing solution for VMware View.

That’s it for this time around; no need to overwhelm you with too much information! Besides, I have to keep a few items around for Technology Short Take #28…

As always, comments, thoughts, rants, or corrections are welcome below.

Tags: , , , , , , , , , , , , ,

It’s been over four months since my last post on open source Puppet, when I posted some updated multi-OS Puppet configurations. I’m back from a long Puppet hiatus with some information on using Puppet for managing user accounts.

It was pretty tough coming back to Puppet after a four-month break (during which time I’ve been spending plenty of time on KVM, Open vSwitch, and libvirt), and it was almost like learning Puppet again from scratch. Fortunately, with the help of a few folks from Twitter and the #puppet IRC channel, I was able to get managing user accounts with Puppet to work (and learned a little bit in the process).

The key to making it work is the idea of a virtual resource. In Puppet, you are normally only allowed to define a resource once; you’d then apply that same resource against multiple systems. However, if you define a user resource for one system, you’re then prevented from defining the same user resource for another system—the resource has already been defined. With a virtual resource, though, you can define the resource once (as a virtual resource; denoted with an “@” in front of the resource). Once a virtual resource has been defined, you can then realize (or instantiate) that resource as many times as needed across multiple systems. (At least, that’s how I understand it. I’m still wrapping my head around some of this stuff.) The Puppet Labs site does a much better job of explaining virtual resources.

Now that you’re armed with that little bit of background information, let’s get started on making this work. There are 2 basic steps involved:

  1. Create a class with a defined type that creates the virtual user resources.
  2. Realize the virtual user resources where needed.

Let’s tackle that first step: creating the class and defined type.

Essentially what you’re going to do to create the class and defined type is create a module. As such, we’ll want to follow Puppet’s guidelines for module structures. In my case, I created a directory for the class (I called mine accounts), with all the suggested subdirectories for a module, even if most of them weren’t needed. From there, I only need to create two files, both in the accounts/manifests directory: init.pp and virtual.pp.

I’ll come to the init.pp in a moment; somewhat counter-intuitively, we’ll start with the virtual.pp manifest first. Here’s the contents of that manifest:

I used the latest version of puppet-lint to ensure that all stylistic recommendations were followed properly. I derived this code from this example, which—once I wrapped my head around the concept—I found quite helpful. Note that there are no virtual resources here; those come in just a moment.

Now that the defined type is done, we can use it to actually create the virtual user resources. We’ll actually do that in the accounts/manifests/init.pp file, like this:

You’ll just repeat that snippet of test as many times as necessary to create a virtual “accounts::virtual” resource for each user account you want to manage within Puppet. Each “accounts::virtual” resource includes a user account, a group account, and a home directory, but we manage all those individual resources as one object through the class definition.

Once you’re ready to actually instantiate an virtual resource, you do that with a snippet of code like this:

Let’s break that down real quick:

  • This is a standard node definition, typically found in nodes.pp or similar.
  • Note that the accounts class (module) is included; this is the class (module) we created earlier, where we created the virtual user resources. This makes the virtual user resources “available” to this node.
  • Then we realize (or actually instantiate/create) the user account on this node with the realize (Accounts::Virtual['johndoe']) statement. Pay attention to the capitalized Accounts::Virtual in this code—this refers back to a resource that has already been defined. In this case, it’s the virtual resource you defined in the init.pp file in the class you established earlier.

With this structure in place, when the node named “server.domain.net” runs Puppet and connects to the Puppet master, it will create the realized resources—user account, group account, and home directory—specified in the node definition. Pretty cool, huh?

I freely admit that I’m still relatively new to Puppet, so I’m sure there are numerous ways this approach could be improved. I tested this code on both CentOS 6.3 as well as Ubuntu 12.04, and it seems to work fine on both platforms. Feel free to submit suggestions for improvement, corrections, or clarifications in the comments below.

Tags: , , ,

« Older entries