You are currently browsing articles tagged Snapshots.

Sanbolic is continuing to differentiate its clustered file system, Melio FS, in advance of the rudimentary clustered file system Microsoft plans on introducing in Windows Server 2008 R2. In an announcement last week, Sanbolic announced support for fully journaled snapshots. This functionality allows any server accessing the clustered file system to invoke a snapshot. The new snapshot functionality provides support for VSS and “full industry standard APIs,” although I’m not really sure what those “full industry standard APIs” are exactly.

You can download the full press release describing the new functionality here.

Separately, Sanbolic also announced that Melio FS fully supports Microsoft System Center Virtual Machine Manager 2008; more information on that is also available.

Now, if only Sanbolic would port Melio FS to VMware ESX/ESXi, then we could have some really interesting discussions. Snapshot functionality built into the shared file system, anyone?

Tags: , , , ,

This session provided information on enhancements to NetApp’s cloning functionality. These enhancements are due to be released along with Data ONTAP 7.3.1, which is expected out in December. Of course, that date may shift, but that’s the expected release timeframe.

The key focus of the session was new functionality that allows for Data ONTAP to clone individual files without a backing Snapshot. This new functionality is an extension of NetApp’s deduplication functionality, and is enabled by changes within Data ONTAP that enable block sharing, i.e., the ability for a single block to appear in multiple files or in multiple places in the same file. The number of times these blocks appear is tracked using a reference count. The actual reference count is always 1 less than the number of times the block appears. A block which is not shared has no reference count; a block that is shared in two locations has a reference count of 1. The maximum reference count is 255, so that means a single block is allowed to be shared up to 256 times within a single FlexVol. Unfortunately, there’s no way to view the reference count currently, as it’s stored in the WAFL metadata.

As with other cloning technologies, the only space that is required is for incremental changes from the base. (There is small overhead for metadata as well.) This functionality is going to be incorporated into the FlexClone license and will likely be referred to as “file-level FlexClone”. I suppose that cloning volumes with be referred to as “volume FlexClone” or similar.

This functionality will be command-line driven, but only from advanced mode (must do a “priv set adv” in order to access the commands). The commands are described below.

To clone a file or a LUN (the command is the same in both cases):

clone start <src_path> <dst_path> -n -l

To check the status of a cloning process or stop a cloning process, respectively:

clone status
clone stop

Existing commands for Snapshot-backed clones (“lun clone” or “vol clone”) will remain unchanged.

File-level cloning will integrate with Volume SnapMirror (VSM) without any problems; the destination will be an exact copy of the source, including clones. Not so for Qtree SnapMirror (QSM) and SnapVault, which will re-inflate the clones to full size. Users will need to run deduplication on the destination to try to regain the space. Dump/restores will work like QSM or SnapVault.

Now for the limitations, caveats and the gotchas:

  • Users can’t run single-file SnapRestore and a clone process at the same time.
  • Users can’t clone a file or a LUN that exists only in a Snapshot. The file or LUN must exist in the active file system.
  • ACLs and streams are not cloned.
  • The “clone” command does not work in a vFiler context.
  • Users can’t use synchronous SnapMirror with a volume that contains cloned files or LUNs.
  • Volume SnapRestore cannot run while cloning is in progress.
  • SnapDrive does not currently support this method of cloning. It’s anticipated that SnapManager for Virtual Infrastructure (SMVI) will be the first to leverage this functionality.
  • File-level FlexClone will be available for NFS only at first. Although it’s possible to clone data regions within a LUN, support is needed at the host level that isn’t present today.
  • Because blocks can only be shared 256 times (within a file or across files), it’s possible that some blocks in a clone will be full copies. This is especially true if there are lots of clones. Unfortunately, there is no easy way to monitor or check this. “df -s” can show space savings due to cloning, but that isn’t very granular.
  • There can be a maximum of 16 outstanding clone operations per FlexVol.
  • There is a maximum of 16TB of shared data among all clones. Trying to clone more than that results in full copies.
  • The maximum volume size for being able to use cloning is the same as for deduplication.

Obviously, VMware environments—VDI in particular—are a key use case for this sort of technology. (By the way, in case no one has yet connected the dots, this is the technology that I discussed here). To leverage this functionality, NetApp will update a tool known as the Rapid Cloning Utility (RCU; described in more detail in TR-3705) to take full advantage of file-level FlexCloning after Data ONTAP 7.3.1 is released. Note that the RCU is available today, but it only uses volume-level FlexClone.

Tags: , , , , , , , ,

Storage Short Take #4

Last week I provided a list of virtualization-related items that had made their way into my Inbox in some form or another; today I’ll share storage-related items with you in Storage Short Take #4! This post will also be cross-published to the Storage Monkeys Blogs.

  • Stephen Foskett has a nice round-up of some of the storage-related changes available to users in VMWare ESX 3.5 Update 3. Of particular note to many users is the VMDK Recovery Tool. Oh, and be sure to have a look at Stephen’s list of top 10 innovative enterprise storage hardware products. He invited me to participate in creating the list, but I just didn’t feel like I would have been able to contribute anything genuinely useful. Storage is an area I enjoy, but I don’t think I’ve risen to the ranks of “storage guru” just yet.
  • And in the area of top 10 storage lists, Marc Farley shares his list of top 10 network storage innovations as well. I’ll have to be honest—I recognize more of these products than I did ones on Stephen’s list.
  • Robin Harris of StorageMojo provides some great insight into the details behind EMC’s Atmos cloud storage product. I won’t even begin to try to summarize some of that information here as it’s way past my level, but it’s fascinating reading. What’s also interesting to me is that EMC chose to require users to use an API to really interact with the Atmos (more detailed reasons why provided here by Chad Sakac), while child company VMware is seeking to prevent users from having to modify their applications to take advantage of “the cloud.” I don’t necessarily see a conflict between these two approaches as they are seeking to address two different issues. Actually, I see similarities between EMC’s Atmos approach and Microsoft’s Azure approach, both which require retooling applications to take advantage of the new technology.
  • Speaking of Chad, here’s a recent post on how to add storage to the Celerra Virtual Appliance.
  • Andy Leonard took up a concern about NetApp deduplication and volume size limits a while back. The basic gist of the concern is that in its current incarnation, NetApp deduplication limits the size of the volume that can be deduplicated. If the size of the volume ever exceeds that limit, it can’t be deduplicated—even if the volume is subsequently resized back within the limit. With that in mind, users must actively track deduplication space savings so that, in the event they need to undo the deduplication, they don’t inadvertently lose the ability to deduplicate because they exceeded the size limit. Although Larry Freeman aka “Dr Dedupe” responded in the comments to Andy’s post, I don’t think that he actually addressed the problem Andy was trying to state. Although the logical data size can grow to 16TB within a deduplicated volume, you’ll still need to watch deduplication space savings if you think you might need to undo the deduplication for whatever reason. Otherwise, you could exceed the volume size limitations and lose the ability to deduplicate that volume.
  • And while we are on the subject of NetApp, a blog post by Beth Pariseau from earlier in the year recently caught my attention; it was in regards to NetApp Snapshots in LUN environments. I’ve discussed a little bit of this before in my post about managing space requirements with LUNs. The basic question: how much additional space is recommended—or required—when using Snapshots and LUNs? Before the advent of Snapshot auto-delete and volume autogrow, the mantra from NetApp was “2x + delta”—two times the size of the LUN plus changes. With the addition of these features, deduplication, and additional thin provisioning functionality, NetApp has now moved their focus to “1x + Delta”—the size of the LUN plus space needed for changes. It’s not surprising to me that there is confusion in this area, as NetApp themselves has worked so hard to preach “2x + Delta” and now has to go back and change their message. Bottom line: You’re going to need additional space for storing Snapshots of your LUNs, and the real amount is determined by your change rate, how many Snapshots you will keep, and for how long you will keep them. 20% might be enough, or you might need 120%. It all depends upon your applications and your business needs.
  • If you’re into Solaris ZFS, be sure to have a look at this NFS performance white paper by Sun. It provides some good details on recent changes to how NFS exports are implemented in conjunction with ZFS.

That’s it for this time around, but feel free to share any interesting links and your thoughts on them in the comments!

Tags: , , , , , , , , , ,

VMware has released Update 2 for VMware Infrastructure 3 version 3.5, which includes updates to VMware ESX, VMware ESXi, VirtualCenter, and VMware Consolidated Backup (VCB). Check the Release Notes for the full details; I won’t reproduce them here, but instead I’ll just point out the particularly interesting details.

Also reporting this information (at the time of this writing) are David from, Rich from VM /ETC, and Duncan at Yellow Bricks. Rich’s post also highlights in red the features that he finds most significant.

Some of the features and/or functionality added in Update 2 that I find most notable include:

  • The biggest, in my mind, is VSS quiescing support. This allows VMware snapshots to leverage VSS for more consistent snapshots. Microsoft had been using the lack of VSS support as a key argument against VMware; this tackles that issue head-on. Also surf over to Duncan’s site and see his post about enabling VSS snapshot support in VMware Tools.
  • Users can now hot extend a virtual disk (extend a virtual disk while the VM is running).
  • Users can clone a virtual machine while it is up and running (live cloning). There is now no need to shut down a VM in order to clone it. I suspect this functionality will have some very interesting repercussions from an operational perspective, and may serve as the basis for future functionality as well.
  • VMware now officially introduces Enhanced VMotion Compatibility (EVC), which leverages Intel FlexMigration and AMD-V Extended Migration support. This functionality automatically configures CPUs within a cluster to be VMotion-compatible and won’t allow you to add hosts to a cluster that can’t be configured via EVC to be compatible.

This doesn’t even touch on any of the other numerous features that are supported. Again, go check the Release Notes or one of the linked blogs above for complete details.

The introduction of new features that reduce service interruption—namely, hot extending virtual disks and live VM cloning—is exactly the move that VMware needs to take to further differentiate their virtualization solution from competitors’ solutions. I’ve stated time and time again that innovation in the virtualization space will continue to set VMware apart from the competition.

Tags: , , , , , ,

A new article of mine has been published on! This article discusses the use of storage array snapshots with VMware, specifically focusing on ensuring that storage array snapshots are consistent and usable:

When used in conjunction with a VMware infrastructure, storage array-based snapshots are touted for their ability to create point-in-time pictures of virtual machines (VMs) for business continuity, disaster recovery and backups. While this can be true, it’s important to understand how virtualization affects storage array snapshot use. Incorrect usage can render storage array snapshots unreliable and generally defunct.

The article provides a few guidelines on making sure that storage array snapshots are usable. Keep in mind, too, that some storage array vendors have applications that are specifically designed to help with this particular issue. NetApp, for example, has SnapManager for Virtual Infrastructure; this product is specifically designed to address this problem (among other problems). I would imagine that other vendors also offer a software solution to this problem, but I’m not particularly familiar with those. I’d love to hear from readers as to their experience or knowledge with any such software solutions.

Tags: , , ,

A couple of days ago my good friend Nick Triantos—formerly of Storage Foo fame, now writing at the Storage Nuts & Bolts Blog—published a short piece of a new version of Open Systems SnapVault (OSSV) that offers new functionality when used with VMware Infrastructure 3 (VI3).

For those that aren’t familiar with OSSV, it’s basically a way to bring NetApp-style snapshots to non-NetApp storage. For example, it’s pretty common to use OSSV to take snapshots of a server that is not attached to a NetApp SAN and store those snapshots on a NetApp SAN. Think of a remote office/branch office kind of scenario, for example, where the branch server can be easily backed up using incremental snapshots to a NetApp storage system in the main location. It’s pretty handy technology, to be honest.

After reading Nick’s piece on OSSV, one question popped in my mind: is he talking about running OSSV on the guest, or running OSSV on the ESX host? So I contacted Nick, obtained clarification, and wanted to post that clarification here. (BTW, thanks, Nick, for taking the time to answer my questions and allowing me to discuss this here.)

Nick’s article primarily focuses on the use of OSSV at the ESX level, running within the ESX Console OS (or Service Console). In this respect, OSSV will be taking snapshots of the VMDK files via the Service Console and replicating them over to a NetApp storage system. As with running OSSV in other environments, the snapshots are block-level incremental snapshots and thus only the changed blocks will be replicated across the wire when a snapshot is taken and shipped off to the destination storage system.

What’s apparently new here is that NetApp has optimized the OSSV code so that the initial baseline will only capture the utilized portions of a VMDK file. Keep in mind that ESX, by default, uses thick-provisioned disks, so that a 30GB virtual disk takes up 30GB on the physical storage. OSSV 2.6, the new version, understands that if only 10GB of that 30GB VDMK is actually being utilized, it will only replicate 10GB of data. That is really important in reducing the overhead required for the initial baseline transfer. Thereafter, OSSV operates as usual by capturing and replicating only changed blocks.

This is a nice addition to OSSV, and it greatly increases the usefulness of OSSV in VI3 environments. But there is a drawback…has anyone figured it out yet?

If you guessed that taking block-level incremental snapshots of the VMDK files via the Service Console means that we lose file-level granularity within the guest file system, you would be correct! What does this mean? Basically, it means that if you have to restore a VMDK from an OSSV snapshot, you have to restore the entire VMDK. You can’t restore individual files within a guest from an OSSV snapshot taken while OSSV is running at the ESX layer.

Before you start knocking OSSV as worthless, however, consider that VCB suffers the same limitation when creating image-level backups. Also keep in mind that file-level backups are only possible with VCB when the guest is running Microsoft Windows, so you’re forced to use image-level backups with any other operating system. Also keep in mind that file-level backups with VCB are slower than image-level backups. With these considerations in mind, it becomes clearer that the limitation is not inherent to OSSV per se, but rather a limitation of technologies operating at the ESX layer.

Of course, there are a number of workarounds to this; one way is to attach the restored VMDK to a different VM, then pull the individual files out that are needed. If you do need file-level granularity from within the guest OS (such as the ability to quickly and easily restore a specific file within a guest), then you can always run OSSV inside the guest and replicate block-level incremental snapshots from the guest over to a target NetApp storage system. Just be sure to keep these distinct procedures in mind as you plan backups of VMs using OSSV.

As always, I encourage you to ask questions, make comments, or add your thoughts below.

UPDATE: Keep in mind that OSSV is VMotion aware, meaning that block-level incrementals are preserved even after a VMotion operation. This is true even within DRS/HA clusters; you just have to install the OSSV agent on all ESX servers within the cluster. The real question: what effect will Storage VMotion have?

Tags: , , , , ,

It’s that time again, friends, time for another Virtualization Short Take!

  • OpenSolaris on Fusion: As expected, Solaris/OpenSolaris fans are experimenting with OpenSolaris on Fusion. Apparently, it runs rather well.
  • Brian Madden had an interesting thought about Thinstall (now ThinApp) plus WINE to eliminate Windows. In the end, Brian feels like many companies will just want to deal with the larger vendors, and won’t be willing to support this kind of “cobbled together” solution. The idea of using ThinApp on WINE on a non-Windows operating system is a pretty cool idea, but it may be a bit early for its time.
  • Microsoft Hyper-V made it to RC1, apparently ahead of schedule. I wonder if they will try to make RTM in time for TechEd in Florida in June? In addition, Microsoft also released information about how they are “eating their own dog food” and using Hyper-V for the MSDN and TechNet web sites.
  • Citrix has released XenDesktop 2.0, their VDI solution. Alessandro has a fairly complete breakdown of the components involved in the solution and the various editions under which it will be released. A lot of these components are pre-existing products that are being rebundled into XenDesktop; XenApp (Presentation Server) and Provisioning Server (Ardence) are two examples. VMware came out with a competitive response almost immediately, and Gareth dissected that response on DABCC. Having not actually installed XenDesktop yet, I don’t know how integrated—or not integrated—the various components are, so I’ll reserve judgment until later. I have my beefs with VDM; in particular, I don’t like how it mandates VM provisioning in order to use pools. I hope that Leostream’s removal of their P>V product as reported by Alessandro doesn’t portend dark days for Leostream.
  • According to Tony Asaro at Virtual Iron, Citrix’s release of XenDesktop signals the beginning of a “shift” in focus from server virtualization to desktop virtualization. One must consider this comment in the context of who is providing the comment; Virtual Iron is, of course, a competitor in the server virtualization market whose product is also based on the Xen hypervisor. Besides, even if that is true, so what? Citrix has made an existence out of focusing on client-side application delivery. This would be completely logical, in my mind, and would allow Citrix to focus on an area where they are strong instead of competing in a market where they are weak.
  • Lou Springer brings us a method of connecting to a VM’s console using VNC over SSH from Mac OS X. I’d seen references to using this with VMware Server, but didn’t know that it worked with VI3. Thanks, Lou! (Lou’s trick was based on information from this VMware KB article, by the way.)
  • From IPMer, here’s some information on using VMware Converter to assist with VM snapshots. This was picked up by Rich over at VM /ETC and also included in the first-ever VMware Communities Roundtable podcast (which I’ve downloaded but not yet had the opportunity to actually review yet).

That’s it for today. I hope that everyone has a great Memorial Day. Don’t forget to thank a veteran or active serviceman/servicewoman for your freedom!

Tags: , , , , , , , , ,

A short while ago, I discussed that VMDKs on NFS may start out thin provisioned, but will lose that thin provisioned status over time. Operations like cloning and Storage VMotion will cause these thin provisioned disks to become thick (fully provisioned) disks, and you lose one of the benefits of running VMware on NFS.

Fortunately, if you’re running a NetApp storage system as your NFS server, you can preserve the thin provisioned status of these VMDKs by leveraging NetApp’s single file SnapRestore functionality. This article describes how that works.

There’s a couple of caveats here:

  1. This technique only helps with making new VMs from an existing VM. SnapRestore won’t help preserve thin provisioned status after a Storage VMotion operation.
  2. This isn’t integrated with VirtualCenter, so you won’t be able to take advantage of VirtualCenter’s integration with Sysprep and such.

As a result of #2 above, then, you’ll need to first prepare your source VM by running Sysprep inside the VM (assuming it is a Windows-based VM) and then allowing Sysprep to shut down the VM. Once that’s accomplished, then you can proceed.

The first step is to take a snapshot of the volume containing the already prepared VM:

snap create <vol-name> <snapshot-name>

Next, create a new VM in VirtualCenter, but do not create a virtual disk for the VM. This will create the VM configuration and associated files and the directory on the NFS datastore.

Third, run a SnapRestore operation to restore both the .vmdk file and the -flat.vmdk files. You have to restore both in order for this to work; keep in mind that the .vmdk file is just a header file and the -flat.vmdk is the actual disk file. The commands would look something like this:

snap restore -t file -s <snapshot-name> -r <new filename and path> <original filename and path>

As an example, let’s say you had a VM named template01 and you wanted to clone the disks for template01 to a new VM called newvm01, and these are stored on a volume called nfsvol. After you’ve run Sysprep on template01, taken the Snapshot and called it base_snapshot, and created newvm01 without a virtual disk, you’d run this command:

snap restore -t file -s base_snapshot -r /vol/nfsvol/newvm01/newvm01.vmdk /vol/nfsvol/template01/template01.vmdk

That would restore the .vmdk (header) file; then you’d restore the actual virtual disk file:

snap restore -t file -s base_snapshot -r /vol/nfsvol/newvm01/newvm01-flat.vmdk /vol/nfsvol/template01/template01-flat.vmdk

Once this process is complete—and it may take some time depending upon the size of the files being restored—you should see both the .vmdk and the -flat.vmdk files listed in the Datastore Browser.

“But wait a minute, Scott,” you say. “The -flat.vmdk no longer looks thin provisioned. You lied! This process doesn’t work.”

Indeed, it will look like it is no longer thin provisioned. Trust me; there’s one more step and then all will make sense. If you log into the ESX Server and open the .vmdk file in vi, you’ll see that it references the old file name of the -flat.vmdk. Edit that to reflect the new, restored file name, save your changes, and go back to the Datastore Broswer again. Refresh the display, and all should be well.

Why does the Datastore Browser work that way? Beats me. You’ll also find that running an “ls -la” on an NFS datastore from the Service Console will show you -flat.vmdk files that appear to be thick provisioned. The only way I’ve found to see if they are thin provisioned is to use the Datastore Browser. It’s a VMware thing, I suppose.

The last and final step is to edit the settings for the VM you created earlier and add the new virtual disk to that VM. Then you can boot up that VM and proceed with whatever customization steps, if any, are needed.

In the near future I plan to test another possible method of preserving the VMDK’s thin provisioned status, a method that is storage agnostic. Look for details of that testing here.

Tags: , , , , , ,

For readers that didn’t already know this, Snapshots on NetApp-hosted CIFS shares are automatically and transparently recognized by Windows versions that support the “Previous Versions” tab. I believe this functionality is native on Windows Server 2003 and an add-on for Windows XP, but I’m not 100% certain. Either way, this means that users can easily recover files from NetApp Snapshots from within the Windows GUI.

Here’s a screenshot of the Previous Versions support on an out-of-the-box server running Windows Server 2003 R2:

NetApp Snapshots as Previous Versions

As you can see in the screenshot, the Previous Versions tab automatically shows the NetApp Snapshots. No configuration is required; this is after a vanilla installation of Windows Server 2003 R2 and all applicable updates from Windows Update.

To NetApp veterans, this is nothing new, but I thought some new NetApp users out there might find information about this functionality and integration useful.

Tags: , , , ,

Once again, here’s my take a few virtualization-related stories that have passed through my computer in the last few days:

  • OK, this first one isn’t technically related to virtualization, but it was too good to pass up. Is there anyone besides me and The Register who thinks NetApp’s new logo is…um…well, not as good as the previous one?
  • A new blog war is brewing between VMware and Citrix, and this time I had nothing to do with it: VMware apparently launched the first volley in discussing the value of ESX Server’s memory overcommitment and page sharing functionality; Citrix’s Roger Klorese then responded and Simon Crosby chimed in as well. I would completely agree with Roger’s and Simon’s comments, except for this one statement in Eric’s original post:

    We created and powered on 512MB Windows XP VMs running a light workload [emphasis mine] and kept adding them until the server couldn’t take any more.

    Since Eric stated the parameters of the test involved lightly loaded workstations, Roger’s comments about heavy workloads don’t apply. Besides, any engineer worth his/her weight isn’t going to overcommit a production workload like that, and this analysis shows that some overcommitment can produce notable financial results.

  • CIO Magazine recently published a list of 10 virtualization risks hiding in your company. It’s a pretty interesting list, although it’s worthwhile to note that this list was produced by a VP of Marketing for Embotics and therefore is heavily slanted toward the risks that his company’s products can help mitigate.
  • This is interesting and novel, but that’s about it. (UPDATE: The creator of the 37migrations VI plugin, Schley Andrew Kutz, wrote me to state that there is no point in 37migrations; it’s just for fun. So stop trying to find a deeper meaning in it, OK?)
  • There’s apparently a problem with using Sysprep in VirtualCenter 2.5 with Windows Server 2003 SP2. A Microsoft hotfix is available.
  • Speaking of NetApp, they’ve been generating some buzz around their SnapManager for Virtual Infrastructure (SMVI) product, yet another unreleased product. I echo Duncan’s thoughts about the VC plugin!
  • Gabe shares some information he’s gathered about VMsafe, the recently announced security APIs from VMware.
  • Alessandro shares his thoughts about Microsoft’s virtualization strategy following the announcement of Microsoft’s purchase of Kidaro. My question is this: was VMware’s announcement of offline VDI functionality at VMworld Europe 2008 because they had an inkling of Microsoft’s moves, or is Microsoft’s purchase a result of VMware’s announcement?

That’s it for today. Join in the discussion by adding your 2 cents in the comments below!

Tags: , , , , , , , , ,

« Older entries