http://linux.sys-con.com/node/803618
http://opensource.sys-con.com/node/803618

Cool! I hope the article is useful to others.

As a quick follow-up to that article, you may recall that I ran into a strange issue with OpenVPN and mt-daapd prior to the home network rebuild. I just finished installing OpenVPN last night and, anticipating the problem, did some digging to see how I’d fix the problem this time around. Turns out there’s nothing to worry about; Avahi skips point-to-point interfaces by default, and OpenVPN tags its interfaces as point-to-point. So, everything works as expected.

• Ubuntu and Mac OS X Integration
• OpenVPN and mt-daapd
• Viscosity, a Mac OpenVPN Client
• VLAN Interfaces with OpenBSD 4.1
• Soaked

So, here’s my guide for creating a bootable ESXi USB flash drive from Mac OS X.

1. Download the ESXi installable ISO.
2. Double-click the ISO to mount it (an icon will appear on your desktop). From there, navigate the contents of the ISO image to find VMware-VMvisor-big-3.5.0-xxxx.i386.dd.bz2 and copy it out of the ISO image into a separate folder.
3. Insert the USB flash drive into an available USB port. Mac OS X will mount the drive and an icon will appear on your desktop.
4. Open the Terminal and type the following command:
    
   diskutil list
    
   On my system, the USB drive was listed as /dev/disk1, but your mileage may vary. It should be pretty easy to tell which device is the USB drive, as the first partition (i.e., /dev/disk1s1) will have a label that matches the name of the icon on the desktop.
5. Type this command:
    
   diskutil unmountDisk /dev/disk1
    
   Replace /dev/disk1 in the command above with the appropriate entry for your system, as identified in the previous command. The icon for the USB flash drive will disappear from your desktop.
6. Run this command:
    
   bzcat <path to VMware-VMvisor file> | dd of=/dev/disk1
    
   Replace /dev/disk1 in the command above with the appropriate entry for your system, as identified by the “diskutil list” command in step 4.

When the process completes—you’ll know because the Terminal prompt will return—use this command to “eject” the USB flash drive and make it safe for physical removal:

diskutil eject /dev/disk1

Again, replace /dev/disk1 with the appropriate device for your system.

At this point, the USB flash drive should be ready to roll. Insert it into a compatible server and virtualize away!

In the process of creating this guide, I found the following sites to be extremely helpful:

mechanised.com: How to Create Your Own Bootable ESXi USB Stick
Create ISO on Mac OS X 10.4 | Martin Bergek
Installing VMware ESXi on a USB memory stick using Ubuntu

For Mac users, the special sauce is the “diskutil” command. Unmounting the USB drive from the Finder also made the underlying BSD device, i.e., /dev/disk1, disappear. Without unmounting it in Finder, the device is reported as “busy” and can’t be accessed (even via root). By using diskutil, we are able to make the device accessible.

• Dead PowerBook
• Replacing Office 2008 Icons
• Ubuntu and Mac OS X Integration
• Quick Note on ESX and ESXi Storage Multipathing
• Using vmware-vim-cmd to Modify a Portgroup

For more complete information on some of the new features that are slated to be included in Windows Server 2008 R2, check out the announcement on the Windows Server Division Weblog.

• Virtual Iron Tackles Power Usage with New Release
• Citrix Announces XenServer 5
• Walking a Fine Line
• Follow Up on the VDC-OS Announcement
• VI 3.5 Expected in December

Since my job has me both in and out of the office quite a bit, I’ve had to adapt most of my processes to work with only my MacBook Pro’s built-in display. While I do have a second monitor that I use when I’m actually at the office, I’ve grown quite accustomed to Exposé and the ability to quickly see all my windows (F9), only the windows for the application I’m currently using (F10), or quickly gain access to my desktop (F11). Unlike Danny, this “kool-aid drinking fan boy mactard” actually doesn’t mind the OS X UI in the least; in fact, I find that it tends to get out of my way the most. That’s not a slight against Linux or Windows; the OS is a tool, just like any other, and users need to use the right tool for them. For me, the OS X UI makes sense and works well. Your mileage may vary, of course.

I don’t use Spaces, the Mac’s virtual desktop functionality, because…well, it’s awful. Besides, having gotten so accustomed to being able to quickly and easily navigate windows I’ve found that I don’t need the extra desktops. I used to be a huge virtual desktops fan (just read some old entries here in the Macintosh category), but after getting the hang of using Exposé I just can’t get back into using virtual desktops.

Anyway, enough of that. In my efforts to manage the daily information flows, I use a few key applications:

• Apple Mail with MailTags and Mail Act-On: The combination of Apple Mail with MailTags and Mail Act-On allows me to quickly and easily process e-mail messages by tagging them and filing them with only a few keystrokes. In early 2008 I resolved to keep my Inbox empty, and these tools have been a key part of actually managing to do that.
• NetNewsWire: NNW manages all my RSS feed subscriptions. When I review new items in NNW, I only superficially scan the headlines across all the subscriptions. Items that look like they are worth a deeper investigation get added to my OmniFocus inbox for reading later. I currently don’t track any Twitter searches via RSS, but that may change later. We’ll see.
• OmniFocus: I use OF to manage all my obligations. Anything that takes more than a couple of minutes to handle gets dumped into OF for processing later. A couple of AppleScripts automate the process of getting items into OF from NNW (via a Quicksilver trigger) and from Apple Mail into OF (via Mail Act-On). This allows me to quickly and easily process only two inboxes—my e-mail inbox and my RSS inbox—and track everything inside OmniFocus. I also keep OF on my Mac synchronized with OmniFocus on my iPhone.
• For Twitter, I’m using a program called NatsuLion, which also has a matching iPhone application. I may dump this for another application, as I’m looking now for an easier (read: more automated) way to share more information via Twitter. It would be great to be able to use an AppleScriptable Twitter client to which I could push an NNW headline, for example.

Along with those applications, I typically have several Remote Desktop sessions, a number of browser windows, iCal, my blogging client, TextMate, and Office 2008 running at any given moment. Between this core group of applications, I find that I’m able to manage—and sometimes rise above—the information deluge.

• Filling the Gap With a Trigger
• Getting Things Done on my Mac, Part II
• Virtual Desktops on Mac OS X
• Follow Me on Twitter
• VDI Announcements at VMworld Europe 2008

• We’ll start off today’s list with some Hyper-V links. First up is this article on how to manually add a VM configuration to Hyper-V. It would be interesting to me to know some of the technical details—i.e., the design decisions that led Microsoft to architect things in this way—that might explain why this process is, in my opinion, so complicated. Was it scalability? Manageability? If anyone knows, please share your information in the comments.
• It looks like this post by John Howard on how to resolve event ID 4096 with Hyper-V is also closely related.
• This blog post brings to light a clause in Microsoft’s licensing policy that forces organizations to use Windows Server 2008 CALs when accessing a Windows Server 2003-based VM hosted on Hyper-V. In the spirit of disclosure, it’s important to note that this was written by VMware, but an independent organization apparently verified the licensing requirements. So, while you may get Hyper-V at no additional cost (not free) with Windows Server 2008, you’ll have to pay to upgrade your CALs to Windows Server 2008 in order to access any Windows Server 2003-based VMs on those Hyper-V hosts. Ouch.
• Wrapping up this edition’s Microsoft virtualization coverage is this post by Ben Armstrong warning Hyper-V users about the use of physical disks with VMs. Apparently, it’s possible to connect a physical disk to both the Hyper-V parent partition as well as a guest VM, and…well, bad things can happen when you do that. The unfortunate part is that Hyper-V doesn’t block users from doing this very thing.
• Daniel Feller asks the question, “Am I the only one who has trouble understanding Cloud Computing?” No, Daniel, you’re not the only one—I’ve written before about how amorphous and undefined cloud computing is. In this post over at the Citrix Community site, Daniel goes on to indicate that cloud computing’s undefined nature is actually its greatest strength:
   

  As I see it, Cloud Computing is a big white board waiting for organizations to make their requirements known. Do you want a Test/QA environment to do whatever? This is cloud computing. Do you want someone to deliver office productivity applications for you? That is cloud computing. Do you want to have all of your MP3s stored on an Internet storage repository so you can get to it from any device? That is also cloud computing.

  Daniel may be right there, but I still insist that there need to be well-defined and well-understood standards around cloud computing in order for cloud computing to really see broad adoption. Perhaps cloud computing is storing my MP3s on the Internet, but what happens when I want to move to a different MP3 storage provider? Without standards, that becomes quite difficult, perhaps even impossible. I’m not the only one who thinks this way, either; check this post by Geva Perry. Until some substance appears in all these clouds, people are going to hold off.

• Rodney Haywood shared a useful command to use with VMware HA in this post about blades and VMware HA. He points out that it’s a good idea to spread VMware HA primary nodes across multiple blade chassis so that the failure of a single chassis does not take down all the primary nodes. One note about the using the “ftcli” command is that you’ll need to set the FT_DIR environment variable first using “export FT_DIR=/opt/vmware/aam” (assuming you’re using bash as the shell on VMware ESX). Otherwise, the advice to spread clusters across chassis as well as to ensure that primary agents are spread across chassis is advice that should be followed.
• Joshua Townsend has a good post at VMtoday.com about using PowerShell and SQL queries to determine the amount of free space within guest VMs. As he states in his post, this can often impact the storage design significantly. It seems to me that there used to be a plug-in for vCenter that added this information, but I must be mistaken as I can no longer find it. Oh, and one of Eric Siebert’s top 10 lists also points out a free utility that will provide this information as well.
• I don’t have a record of where this information turned up, but this article from NetApp (NOW login required) on troubleshooting NFS performance was quite helpful. In particular, it linked to this VMware KB article that provides in-depth information on how to identify IRQ sharing that’s occurring between the Service Console and the VMkernel. Good stuff.
• Want more information on scaling a VMware View installation? Greg Lato posts a notice about the VMware View Reference Architecture Kit, available from VMware, that provides more information on some basic “building blocks” in creating a large-scale View implementation. I’ve only had the opportunity to skim through the documents thus far, but I like what I’ve seen thus far. Chad also mentions the Reference Architecture Kit on his site as well.
• Duncan at Yellow Bricks posts yet another useful “in the trenches” post about VMFS-3 heap size. If your VMware ESX server is handling more than 4TB of open VMDK files, then it’s worth having a look at this VMware KB article.
• The idea of “virtual routing” is an interesting idea, but I share the thoughts of one of the commenters in that technologies like VMotion/XenMotion/live migration may not be able to respond quickly enough to changing network patterns to be effective. Perhaps it’s just my server-centric view showing itself, but it seems more “costly” (in terms of effort) to move servers around to match traffic flow than to just route the traffic accordingly.
• CrossBow looks quite cool, but I’m having a hard time understanding the real business value. I am quite confident that my lack of understanding about CrossBow is simply a reflection of the fact that I don’t know enough about Solaris Containers or how Xen handles networking, but can someone help me better understand? What is the huge deal with Crossbow?
• Jason Boche shares some information with us about how to increase the number of simultaneous VMotion operations per host. That information could be quite handy in some cases.
• I had high hopes for this document on VMFS best practices, but it fell short of my hopes. I was looking for hard guidelines on when to use isolation vs. consolidation, strong recommendations on VMFS volume sizes and the number of VMs to host in a VMFS volume, etc. Instead, I got an overview of what VMFS is and how it works—not what I needed.
• Users interested in getting started with PowerShell with VMware Infrastructure should have a look at this article by Scott Herold. It’s an excellent place to start.
• Here’s a list of some of the basic things you should do on a “golden master” template for Windows Server VMs. I actually disagree with #15, preferring instead to let Windows manage the time at the guest OS level. The only other thing I’d add: be sure your VMDK is aligned to the underlying storage. Otherwise, this is a great checklist to follow.

I think that should just about do it for this post. Comments are welcome!

• Virtualization Short Take #19
• You’ve Got to be Kidding Me
• Is Power the Key to Controlling the Cloud?
• Virtualization Short Take #14
• Virtualization Short Take #20

So, here are my proposed sessions for Virtualization Congress 2009 in Las Vegas:

VMware ESX Networking Scenarios: Even after extensive coverage across the Internet, there are still a lot of details about VMware ESX networking that users just don’t understand. This is especially true for new/relatively new adopters of VMware’s technology. The idea behind this session is to provide these users with an in-depth review of various VMware ESX networking scenarios and the pros and cons of each configuration.

Real-World Considerations with Integrating Advanced Storage Array Functionality in Virtualized Environments: Yes, I know the title is a mouthful. This session takes a closer look at advanced storage array functionality like space-efficient clones, replication, snapshots, and the like to try to understand what considerations—if any—there are when using these features in virtualized environments. Being able to take a snapshot is great, but what if the snapshot is corrupt? It won’t do you much good, will it?

So, if either of these sessions looks interesting to you or you think it would be useful to conference attendees, I’d appreciate your support by voting for them. Thanks!

• Storage Array Snapshots with VMware
• Hyper-V Released
• VMworld 2006 Day 3 and Wrap-Up
• 3183: VMware SRM on NetApp Storage
• TA2644: Networking I/O Virtualization

OK, this isn’t world-shattering news or a discovery of paramount importance, but I do find this a bit odd given that with the Linux-based Service Console most everything should be entirely case-sensitive. However, I found that with esxcfg-advcfg only part of the command needs to be properly capitalized.

Consider these examples. If I wanted to increase the maximum number of NFS datastores on VMware ESX, I would use this command:

esxcfg-advcfg -s 32 /NFS/MaxVolumes

This command will also work (note the capitalization, or lack thereof):

esxcfg-advcfg -s 32 /NFS/maxvolumes

But this command won’t work; it will report that it can’t find the parameter:

esxcfg-advcfg -s 32 /nfs/maxvolumes

I noticed the same behavior with a couple of other settings, so I’m reasonably confident that it’s not just this one setting. It appears that esxcfg-advcfg only cares if the first portion of the path is properly capitalized, and the rest of the parameter is case insensitive.

Interesting, no?

• ESX Firewall Oddity
• Identifying ESX Server NICs in Blades
• ESX iSCSI Basic Configuration from the CLI
• Quick Note on ESX and ESXi Storage Multipathing
• Important Note Regarding VMware over NFS

Thanks to this post by Cody Bunch and this Twitter response by Hal Rottenberg, I cobbled together this PowerShell command:

get-datacenter "Name" | get-vm | get-networkadapter | where-object { $_.networkname -like "OldPortGroup" } | set-networkadapter -networkname "NewPortGroup" -Confirm:$false

It worked like a champ! Obviously, you could limit the scope of this command by filtering the VMs that are returned with a wildcard pattern on the Get-VM command.

Thanks to Cody and Hal for their assistance!

• Semi-Automatic Security Groups
• VMware ESX Networking Articles
• ESX Server and the Native VLAN
• Enumerating Universal Group Membership
• Listing Groups in Active Directory

My primary goals for this “home network rebuild” were the following:

1. Rebuild the home server with a newer version of Linux, and possibly switch to a different distribution.
2. Continue to provide DNS, DHCP, HTTP, and HTTP proxying/content filtering services to the home network.
3. Continue to provide file sharing services via Server Message Block/Common Internet File System (SMB/CIFS) for Windows-based systems on the home network.
4. Continue to have a shared music library available via Digital Audio Access Protocol (DAAP, aka iTunes) available to all systems on the home network.
5. Provide file sharing services to Macs on the network via AppleTalk Filing Protocol (AFP) over TCP.

Ideally, I also wanted to enable Time Machine backups from my Mac laptop to the home server.

After doing a fair amount of research, I settled on the use of Ubuntu 8.04 LTS (”Hardy Heron”) for the server build. I didn’t go with Ubuntu 8.10 (”Intrepid Ibex”) simply because a) I already had 8.04.1 downloaded and burned to a CD; and b) Hardy Heron is an LTS release, so I should have better support over the long term.

I won’t bore readers with the details of the rebuild, but it took about a day or two to get a larger hard drive installed, Ubuntu installed and configured, and services running like DHCP (including some static reservations for certain computers, like my laptop and my iPhone), DNS (using MaraDNS, much easier to figure out than BIND), Apache, and Squid with SquidGuard. At this point, I’d completed tasks #1 and #2.

On to task #3. This was pretty simple and straightforward and easily accomplished via Samba, with nothing really unique to document here. The one interesting thing that I did find was a way to map the long usernames that Mac OS X uses (like “Bob Jones”) to a short username (like “bjones”). I used this command in the /etc/samba/smb.conf file:

username map = /etc/samba/usermap.conf

In this file, I simply placed lines that mapped the long usernames to the short usernames. Since Mac OS X defaults to the long username when connecting to the server, this allows me to simply type in a password and connect. I searched for hours trying to find a way to have Mac OS X supply my current password to the Samba server so that I wouldn’t get prompted, but could not find any information. If anyone knows the trick, I’d love to hear about it. After configuring a few shares, setting Linux permissions and the umask, and then testing from both my Mac laptop and a Windows laptop, task #3 was finished.

Task #4, providing an iTunes-compatible music server, was also really straightforward and easy. For this, I again selected Firefly Media Server, formerly mt-daapd, which I’d used before with great success. Again, nothing unusual or unique to document here, except for the potential interaction with Avahi (more on that later).

The final task was installing Netatalk to provide AFP over TCP file sharing services for Macs on the network. Fortunately for me, one of the sites I’d been using to help in my project pointed me to this blog post, which had a prebuilt package for Netatalk that included the necessary SSL support that Mac OS X requires. That saved me the trouble of compiling Netatalk from source. Following the steps in the Kremalicious article as well as information from this guide, I configured Netatalk to present a volume to use for Time Machine backups. It was at this point that I noticed a strange interaction with Avahi.

Avahi is a multicast DNS (what Apple calls Bonjour) server for Linux. It’s responsible for advertising services to multicast DNS-enabled systems, such as other Linux systems running Avahi or Macs. I’d installed Avahi earlier and used some service definitions from this article and this blog post to advertise Samba and HTTP. In addition, after installing Firefly, I’d noticed that Firefly starting advertising its presence automatically through Avahi with no service definition required.

Upon installing Netatalk, I also noticed that Netatalk started advertising automatically via Avahi as well, but using the IP address of the server. In order to be able to control how Netatalk advertises via Avahi, I had to change this line in /etc/avahi/avahi-daemon.conf:

enable-dbus=no

The suggestion for this change came from this thread on the Ubuntu Forums. Upon making the change and restarting Avahi, the odd Netatalk entry went away, but so did Firefly! To advertise both Netatalk and Firefly, I added a couple of files to /etc/avahi/services:

afpd.service:

<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM “avahi-service.dtd”>
<service-group>
<name replace-wildcards=”yes”>Intrepid Time Machine</name>
<service>
<type>_afpovertcp._tcp</type>
<port>548</port>
</service>
<service>
<type>_device-info._tcp</type>
<port>0</port>
<txt-record>model=AirPort</txt-record>
</service>
</service-group>

daapd.service:

<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM “avahi-service.dtd”>
<service-group>
<name replace-wildcards=”yes”>Home Music Server</name>
<service>
<type>_daap._tcp</type>
<port>3689</port>
</service>
</service-group>

After placing these two files into /etc/avahi/services, the new services starting advertising immediately. By the way, you’ll note the extra “device-info” entry in afpd.service; that sets the icon that will be used by Macs when they discover this service. I made mine look like a Time Capsule by using the setting listed above.

During this work with Avahi, I uncovered a couple of interesting things:

• I found that restarting the Avahi daemon is actually more problematic than just leaving it alone; in order to make it start advertising services again after a restart, you’ll have to open one of the files in /etc/avahi/services and then close it again. No changes are necessary to the file, but opening it will kickstart Avahi into service advertisement.
• Advertising SMB/CIFS and AFP together with the same name caused my Mac to ignore the SMB/CIFS services and only use AFP. I had to separate SMB/CIFS and AFP into different entries. Since I was using AFP really only for Time Machine backups and SMB/CIFS for everything else, it wasn’t really a big deal.
• Advertising SMB/CIFS and RFB (Screen Sharing, as outlined here) works fine together.

At this point, task #5 was pretty much complete. I had originally envisioned providing file sharing services to the same locations via both AFP and SMB/CIFS, but in the end—partially because of the odd issue with AFP and SMB/CIFS being advertised together as described above—settled for using AFP only for Time Machine and SMB/CIFS for everything else.

Along the way, I also configured screen sharing as outlined here, and it seems to work just fine. I have to leave an account logged in to the Ubuntu server, but I can just lock the screen when I’m not logged in remotely.

The last step was to enable Time Machine backups to the Ubuntu server via AFP. First, the hack to enable non-Time Capsule network backups (this should be all on one line):

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

I was then able to select the Ubuntu-hosted AFP volume for Time Machine backups. Attempting to run a Time Machine, backup, however, reported an error about being “unable to create the disk image”. Fortunately, a number of different articles pointed to the use of hdiutil to create the disk image, and that seemed to fix the problem. Time Machine is now backing up to the AFP volume, although I suspect I still have a few issues to work through (for example, it looks as though I have to keep the Time Machine AFP volume mounted in order for automatic backups to run).

So, when everything is said and done, I was able to achieve all my stated goals. The only outstanding issue that I haven’t yet figured out yet centers on automatic logins; for both AFP and SMB/CIFS, I get prompted for a password when connecting, even though I keep my password synchronized (manually) between my Mac and the Ubuntu server. Any tips on how to resolve that would certainly be appreciated.

Along the way, I found the following sites to be quite helpful. I’m sure there are others that I used along the way, and I apologize if I’ve failed to extend credit where credit is due.

Limit size of Time Machine backups on Time Capsule
Set up Time Machine on a NAS in three easy steps
Make Ubuntu a Perfect Mac File Server and Time Machine Volume
Five Guides on How to Integrate Ubuntu into a Mac OS X Network
Time Machine Wireless Backups without Time Capsule

• My Ubuntu-Mac Article is Popular, I Guess
• Greater AD Integration via NFS and Automounts
• OpenVPN and mt-daapd
• Hanging Around #vmware
• Mac OS X and .local Domains

One Year Ago on blog.scottlowe.org

LACP with Cisco Switches and NetApp VIFs
Hyper-V Architectural Issue
Latest VDI Article Published

Two Years Ago on blog.scottlowe.org

Bookmark Spam?
Personal Computing as a Collection of VMs?
Application Agnosticism

Three Years Ago on blog.scottlowe.org

Mac OS X and .local Domains
WMF Flaw Exploit Grows Worse
Complete Linux-AD Authentication Details

• Comment Spam
• Bookmark Spam?
• VMware Fusion 2.0 Beta 1
• Personal Computing as a Collection of VMs?
• Comments Available Again