Backup

You are currently browsing articles tagged Backup.

In late 2009, I posted a how-to on making Snow Leopard work with an Iomega ix4-200d for Time Machine backups. I’ll recommend you refer back to that article for full details, but the basic steps are as follows:

  1. Use the hdiutil command to create the sparse disk image with the correct name (a concatenation of the computer’s name and the MAC address for the Ethernet interface).
  2. Create a special file inside the sparse disk image (the com.apple.TimeMachine.MachineID.plist file).
  3. Put the sparse disk image on the TimeMachine share on the ix4-200d (if you didn’t create it there).
  4. Set up Time Machine as normal.

In the comments to the original article, a few people suggested that newer firmware revisions to the Iomega ix4-200d eliminated the need for this process. However, in setting up my wife’s new 13″ MacBook Pro, I found that this process is still necessary. Even though my Iomega ix4-200d is now running the latest available firmware (the 2.1.38.xxx revision), her MacBook Pro—running Mac OS X 10.6.7 with all latest updates—would not work with the Iomega until I manually created the sparse disk image and populated it with the com.apple.TimeMachine.MachineID.plist file. Once I followed those steps, the laptop immediately started backing up.

So, it would seem that even with the latest available firmware on the ix4-200d, it’s still necessary to follow the steps I outlined in my previous article in order to make Time Machine work.

Tags: , , ,

EMC Data Domain made a couple of significant product announcements today. First, EMC announced the EMC Data Domain Global Deduplication Array (GDA). Second, EMC also announced a doubling of the logical capacity for the high-end Data Domain DD880.

The full press release for the GDA announcement is available here. The “speeds and feeds” of the new multi-controller architecture of the GDA include throughput of up to 12.8 TB/hour, up to 270 concurrent backup operations, and up to 14.2 PB (yes, petabytes) of logical storage capacity. That’s pretty impressive, if you ask me.

As for the doubling of capacity on the DD880, the full press release has all the details. In addition to greater capacity on the DD880 (up to 7.1 PB of logical storage capacity), the announcement also unveiled Data Domain encryption. This represents the industry’s first encryption of data at rest on deduplicated storage. Encryption occurs inline using administrator-selected 128-bit or 256-bit Advanced Encryption Storage (AES) algorithms.

Take a look at the full press releases for both these announcements for more information.

Tags: , , ,

Chad Sakac of EMC (visit his weblog here) recently sent me an Iomega ix4-200d storage unit. Considering that I didn’t need it for my VMware lab—which runs both EMC and NetApp storage arrays—I pressed it into service at home. I can’t tell you how useful it’s been, especially the built-in Time Machine support. Making my three-year-old MacBook Pro running Leopard (Mac OS X 10.5) work with the Iomega’s Time Machine implementation was a piece of cake: point it at the ix4-200d and you’re all set. It’s been great.

Recently, though, I picked up a new MacBook Pro running Snow Leopard. Unfortunately, due to some “under the covers” changes from Leopard to Snow Leopard, making your Snow Leopard-based Mac use the ix4-200d for Time Machine backups isn’t quite so straightforward. Thankfully, due to these instructions, I was able to make it work without too much effort. Here’s how.

First, you’ll need to create a sparse disk image using the following command from Terminal.app:

hdiutil create -size 500G -fs HFS+J -volname 'Time Machine' -type SPARSEBUNDLE <filename>.sparsebundle

Leopard required that the name of the sparse disk image be a concatenation of the computer’s name and the MAC address of the Ethernet interface (en0) on the system. (By the way, you have to use the MAC address of en0 even if you are performing wireless backups over en1.) It appears that Snow Leopard does not have this restriction. I followed it anyway, just in case. hdiutil will create it in whatever directory you are in when you run the command. If you aren’t in the TimeMachine directory on your ix4-200d (which would have a path of /Volumes/TimeMachine on your local Mac), then you’ll need to copy the sparse disk image later. That’s OK.

Second, you’ll need to create a file named com.apple.TimeMachine.MachineID.plist and copy that into the sparse disk image. The contents of this file should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.backupd.HostUUID</key>
<string>System UUID Here</string>
</dict>
</plist>

You’ll need to put your system’s UUID here; you can get this value from System Profiler. Once you’ve created this file with the appropriate values, copy it into the sparse disk image you created. I did this with the Finder and it works fine, but if you want to use Terminal.app you can use this command:

cp com.apple.TimeMachine.MachineID.plist <filename>.sparsebundle

Finally, copy—if necessary—the sparse disk image from wherever you created it to the TimeMachine share on the Iomega ix4-200d. Make sure you’ve mounted the TimeMachine share via AFP and then copy the sparsebundle over using this command:

cp -pfr <filename>.sparsebundle /Volumes/TimeMachine/<filename>.sparsebundle

After you’ve completed all these steps, you can go into the Time Machine preferences and activate Time Machine against the ix4-200d. It worked seamlessly for me on Mac OS X 10.6.2, but your mileage may vary.

I also found this site and this site helpful in confirming the necessary contents of the com.apple.TimeMachine.MachineID.plist file, which is really the key to making it work in Snow Leopard.

I hope someone finds this useful!

Tags: , , ,

I was reading this article by Eric Gray about CSV and followed his link to this ZDNet post about VMFS-3. I was a little taken aback.

One of the first questions that popped into my head was, “Why not use VMware Converter?” After all, it’s the tool that was expressly designed to do exactly what Jason Perlow was attempting to do in the DR exercise with his customer: import data back into a VMware vSphere environment.

Further—and I could be wrong here—but it seems to me that I’ve accessed USB storage devices from the Service Console before. Assuming that I’m correct (again, a big assumption, and one that I’ll test in the lab today), why not just use vmkfstools to import the VMDKs back into a VMFS datastore? And yes, Jason does provide a passing reference to ESXi, so perhaps he was using ESXi at the DR site and therefore didn’t have access to the Service Console.

I wasn’t there in the DR exercise, so there may have been other mitigating circumstances of which I am not aware. In addition, using VMware Converter wouldn’t have addressed the lengthy time taken to copy the data across the network which, as I understand it, was one of the primary complaints in Jason’s article.

In the end, though, why is Jason complaining about VMFS-3 and the time taken to restore a bunch of VMware vSphere virtual machines when the real root of the problem was the “consumer grade hard disks that you buy at the local Staples or Best Buy hooked up to some random Linux server”? He would have faced the exact same problem if he were trying to restore large Exchange databases from these commodity 1TB hard disks, or if he were trying to restore large Oracle or SQL databases from commodity 1TB hard disks. Does that mean Microsoft and Oracle are at fault as well? In this case, since VMware is the new “whipping boy” in the data center, VMware—specifically, VMFS-3—gets blamed.

I fail to see how better VMFS interoperability would have helped. Even if he’d been able to run VMs off USB-attached consumer grade hard disks, would you have wanted to? Then, VMware vSphere would have been blamed for “awful, horrible performance”. Like I said, VMware is the new whipping boy, and everything is VMware’s fault. Hey, at least the networking guys are happy—it’s no longer the network’s fault!

And while this wouldn’t have helped in Jason’s situation (he was trying to get data into VMFS, not get data out of VMFS), there is this open source VMFS driver available. So VMFS-3 is as much the “black box” that he lets on, in my opinion.

Let me know what you think: am I way off here?

Tags: , , ,

PHD Virtual Technologies, maker of the esXpress backup solution, are today announcing the availability of the latest version of their backup product. esXpress version 3.6 offers support for VMware vSphere 4 and, according to PHD Virtual, also offers improvements for all versions of VMware ESX since version 3.0.2.

esXpress 3.6 features a new deduplication engine that provides substantial new performance levels, based on information provided by PHD Virtual. For example:

  • File-level restores are now up to four times faster
  • Image-level restores are up to twice as fast
  • Initial backups are up to twice as fast

esXpress performs source-side deduplication at the block level to help reduce backup storage requirements and help keep WAN traffic to a minimum when backing up over a WAN.

It’s unclear to me whether esXpress leverages any new vSphere-specific features, like Changed Block Tracking, in the new release. PHD Virtual has committed to providing a review copy for me to run in the lab; once I’ve had the chance to “kick the tires,” so to speak, I’ll post more information here.

More details are available from PHD Virtual’s web site.

Tags: , , , ,

Reader Kyle Ross shared with me a potential issue with VMware’s new backup product, VMware Data Recovery. Others within the VMware blogging scene have also covered this, but I wanted to mention it as well so that others didn’t run into the problem. Here’s Kyle’s write-up:

I was made aware of a serious (in my opinion) bug with VDR during a call with VMware support that I haven’t seen discussed anywhere. This is an internally known issue that causes snapshots to build up on VM’s that are members of VDR backup jobs.

During the backup process a new snapshot is created and VDR updates the snapshot descriptor file (vm_name-000001.vmdk) to mark the snapshot as un-removable. The bug is introduced when the backup process completes, it fails to mark the snapshot as removable causing them to remain.

The tricky part of the problem is that the snapshots are not visible through the vSphere Client, nor are they listed in apps like ‘RVTools’ that use the VMware CLI to gather data. They could potentially be listed in the new datastore views but I didn’t think to look there before I resolved it in my environment. I ran across them by logging into the service console and running the following command to list all the delta files on the datastores attached to the server.

find /vmfs/volumes/ -name \*delta\*

In my environment I noticed numerous VM’s with multi-gigabyte delta files that I couldn’t account for via snapshots listed in the GUI. Here is the solution I was given by VMware. Via the service console, browse to the location of the VMDK files for the affected VM. Run this command to identify the descriptors that need to be corrected, replacing ‘virtual_machine_name’ with the actual name of the VM.

grep –I ddb.dele *virtual_machine_name*-000???.vmdk

This command will quickly identify the delta files that are marked as non-deletable. The workaround is to edit the affected VMDK descriptor files and change “ddb.deletable” from “false” to “true”. You will probably also need to edit the root VMDK file and change this field as well, otherwise you may be left with one open snapshot. Note that due to a change in how ESX 4 performs file locking, you will probably need to SSH into the host that is currently running the VM to edit these files. Once you have edited all the files, create a new snapshot for the VM either via the GUI or command line. Then issue the “Delete All” snapshots command to force ESX to combine all the files and close all the visible and hidden snapshots.

As soon as more information is available, I’ll post it here. If any other readers have more information to share, please speak up in the comments.

Tags: , , ,

This session is titled “Best Practices for Architecting VMware Consolidated Backup Enabled Solutions,” and it’s being led by a couple of different presenters, one of whom is Dan Anderson, who also led the VCB session on Partner Day and leads the VCB labs (and led the VCB labs last year at VMworld 2006).  So far, it looks like this session won’t be a repeat of Monday’s Partner Day session, which is good.

The session started out with a review of VCB architecture, VCB components, and the interplay between the various portions of the VCB solution—the VCB proxy server, the pre-freeze and post-thaw scripts, file-level backups versus full VM backups, etc.  This stuff I already knew and had already been covered in detail in previous sessions.

The discussion then moved into a mention of the various items that affect the design of a VCB solution.  This would include things like the SAN architecture (Fibre Channel vs. iSCSI), software components (VCB version, VC version, ESX version, third-party backup software version), recovery mechanism, backup types, etc.  Dan mentioned again the command-line interface for VMware Converter that was mentioned on Monday; I really need to dig up that information so that I can explore that possibility in greater detail.

It was stated again today that there are bad candidates for VMware snapshots—high I/O, high transaction VMs are bad candidates for snapshots, and therefore are bad candidates for use with VCB.  (Recall that VCB uses snapshots to unlock the base VMDK for use by the VCB proxy, so a VM that is a bad candidate for snapshots is therefore a bad candidate for VCB-enabled backup solutions.)

I ended up leaving the session early because it turned out that a great deal more of the information that John and Dan were presenting was identical to the information that was presented on Monday at the partner session.  This is not a reflection on the presenters, or the session materials; I’d just already seen most, if not all, of the materials before.

Tags: , , , ,

I had the opportunity to attend three Partner Day sessions today, all three from the “Advanced In-Depth Technical Track.”

The first was this morning, and it was focused on an in-depth technical review of VMware Consolidated Backup.  I’ve written a few articles about VMware Consolidated Backup before:

The presenter was very knowledgeable; in fact, he was the one running the VCB labs at VMworld (and ran the labs at VMworld last year, which I attended).  I got some great information from the session, and when I have more time I’ll compile that information here.  Some of the key points I took away from the session included information on a command-line interface (CLI) for VMware Converter that allows for automated restores of VCB full VM backups using Converter (I’m really excited about looking into that one); good information on the minimal permissions needed for the user that logs into VirtualCenter and whose login information is hard-coded in config.js (this is a big security concern for many customers); and some RDM compatibility mode issues (RDM in virtual compatibility mode versus physical compatibility mode).  It was a great session; I thoroughly enjoyed it.

Unfortunately, the next session (the first session after lunch) was not nearly as useful.  Although included in the same in-depth track, there was a remarkable lack of technical information.  Fortunately, it was only 45 minutes long.

The final session of the day was on VMware’s VDI and ACE solutions, including a first look at VDM (Virtual Desktop Manager) 2.0, which VMware just publicly announced earlier today.  The presenter, Tommy Walker of VMware, was a great presenter and I enjoyed the presentation.  I’m hoping to be able to catch up with Tommy later this week to conduct some in-depth comparisons of VDM with other brokers, such as Leostream (a broker with which I’ve worked fairly extensively—and which, by the way, just released version 5.0 of their broker).

As soon as I have some additional time, I’ll try to post some additional information about these sessions and some of the in-depth technical details presented.

Tags: , , , ,

Strange VCB Error

While in the process of verifying the operation of VMware Consolidated Backup (version 1.0.3) today using the command-line vcbMounter.exe utility, I kept receiving an error from vcbMounter and the full VM backups would fail.  Nothing seemed obvious at first, so I added the “-L 6” parameter to the command line, which was something like this:

vcbmounter -h vcserver.example.com -u username -p password
-r e:\mnt -a ipaddr:10.1.1.107 -t fullvm -L 6

Nothing terribly complicated there, just a simple full VM backup of the VM whose IP address is 10.1.1.107.  (For those of you that aren’t familiar with the vcbMounter.exe command-line syntax, it looks worse than it actually is.  Trust me.)  Upon running this command with the increased logging, I kept getting these errors:

[2007-08-07 12:13:47.418 'App' 2144 warning] Could not
obtain inquiry page 128 for device on path 0, target 4, lun 0
 
[2007-08-07 12:13:47.418 'App' 2144 warning] Sending SCSI inquiry
failed: Unknown error. (No proper error code was returned.)
 
[2007-08-07 12:13:47.418 'App' 2144 warning] Could not
obtain inquiry page 128 for device on path 0, target 5, lun 0
 
[2007-08-07 12:13:47.418 'App' 2144 warning] Sending SCSI inquiry
failed: Unknown error. (No proper error code was returned.)

The odd thing was, target ID 4 and target ID 5 were local SCSI targets, not anything SAN-related.  In fact, they were the system (C:) and data (D:) drives that had been created when the server was built and Windows Server 2003 was installed.

Google turned up nothing obvious, so I decided to try running the command directly against an ESX server.  The modified command now looked like this:

vcbmounter -h esxserver.example.com -u root -p password
-r e:\mnt -a ipaddr:10.1.1.107 -t fullvm -L 6

The operation still failed, but now I had a critical piece of missing information:

[2007-08-07 12:29:43.798 'BlockList' 2052 error] Your VirtualCenter or the ESX server hosting the virtual machine you are dealing with needs to be upgraded to work with this version of VCB. (VCB attempted to invoke the method “acquireLeaseExt” on a remote object of type “vim.host.DiskManager”, but this method is unknown to this object type.)

Aha!  A quick review of the environment showed that the ESX host this particular VM was hosted on was indeed running version 3.0.1.  With a quick VMotion to a nearby host running ESX Server 3.0.2 and a repeat of the command (changed to target the new host, obviously), and the backup operation worked.  I moved the guest back to the original host again, and the operation failed again.  This pattern held true regardless of whether the vcbMounter.exe command targeted the VirtualCenter server (which was running version 2.0.2) or the ESX Server. Anytime the VM was hosted on the ESX server running 3.0.1, the command failed.

<aside>Now why didn’t the error message just say that the first time, instead of complaining about local SCSI disks?</aside>

A quick review of the VCB 1.0.3 release notes turns up this fairly brief blurb:

VMware Consolidated Backup (VCB) 1.0.3 is compatible with VirtualCenter 2.0.2 and ESX Server 3.0.2 (or newer) only. This release is not supported if used with older version of ESX Server and VirtualCenter.

OK, fair enough—I should have more closely read the release notes before getting too deep into the testing.  But what this means is that customers won’t be able to start using VCB 1.0.3 until all their hosts have been upgraded to ESX Server 3.0.2 and VirtualCenter has been upgraded to 2.0.2.  I don’t know at this time if VCB 1.0.2 will work against both the newer and older versions; if not, that will put organizations in a situations where they may end up with two different sets of VCB proxy servers: one set to support the hosts running ESX Server 3.0.1, and another set to support hosts running ESX Server 3.0.2.  And that doesn’t even take VirtualCenter into consideration!

Anyone out there testing VCB 1.0.2 against the newer releases of ESX and VC?  This will help tell us if we can leverage the existing VCB infrastructure until after all the hosts have been upgraded, and then upgrade VCB, or if a parallel VCB infrastructure will have to be established to support the newer of ESX and VC.

Tags: , , , ,

I think I’ve mentioned before that I use NetNewsWire for subscribing to various RSS and Atom feeds for various topics.  Of course, given my interest in virtualization, VMware and VMware-related tags are things that are included in my feed list, and quite often I come across a link from a tag search or del.icio.us subscription that I find interesting.  It may not justify a dedicated blog post about that particular item, though, because it may not be anything about which I can contribute useful information.  After all, there’s enough useless junk floating around the Internet as it is; why should I contribute to the problem?

Likewise, I typically won’t post a link to my del.icio.us bookmarks unless it’s something that I know I’ll need to reference again, and these tidbits of information sometimes aren’t useful reference articles.  So, instead of blogging about these or just posting them to del.icio.us, I thought I’d post the links here.  It seems like a reasonable compromise between the two, if you ask me.

There you go.  Enjoy!

Tags: , , , , ,

« Older entries