blog.scottlowe.org

A colleague at work just turned me on to the news that Brocade will be buying Foundry for $3 billion. This sets the stage for Brocade to compete even more directly with Cisco, not only in storage networking but now also in Ethernet networking.

Thanks for the heads-up, Greg!

My Virtualization Short Take series seems to be reasonably popular, so I thought I’d expand into another area that interests me: storage. Here is Storage Short Take #1! If this proves helpful or useful to readers, I’ll continue the series on an irregular basis.

• If you’re new to SnapMirror, especially synchronous SnapMirror, this synchronous SnapMirror configuration guide may prove very helpful to you.
• Good friend Nick Triantos discusses NetApp’s Storage Recovery Adapter (SRA) for VMware Site Recovery Manager (SRM). While the discussion is specific to NetApp, it’s a good example of how the storage vendors are responsible for implementing vendor-specific functionality in the SRA. The other storage vendors supporting SRM are responsible for the same things for their storage arrays and storage array functionality.
• Isn’t this the truth—everyone and their brother has “storage virtualization” functionality built into their products these days. Frankly, I’m tired of hearing about it.
• If you’re running VMs on NFS on NetApp storage, you’ll want to note this knowledge base article (NOW login required). It notes that a SCSI disk timeout increase may need to be set in order to accommodate cluster failover times.
• I recently came across this white paper from Emulex and Cisco regarding the use of N_Port ID Virtualization (NPIV) in a VMware environment. Personally, I found it to be a bit light on the technical details and a bit heavy on the marketing side, but otherwise useful.
• This tool from NetApp (NOW login required) can help with approximating storage I/O. It’s not perfect, but it might help provide some rough estimates. I’m sure other vendors have similar tools; readers are encouraged to share links to those tools in the comments.

Well, that’s it for now. I’d love to hear feedback (good or bad) from readers as to whether this is even remotely useful or interesting.

Sanbolic, whose Melio FS product I discussed a short while ago, announced today the availability of their Kayo file system. The official press release is here in PDF format. Quoting from the press release:

Sanbolic today announced that Windows Server 2008 Hyper-V virtual machines can now be stored on a single shared storage area network (SAN) storage volume using Sanbolic Kayo File System. The virtual machines can then be moved independently between physical host servers using Quick Migration because all host servers have shared access to the virtual machines files. Kayo FS will be price at $299 per host server and sold in a 5 license bundle.

Kayo FS is described as “VMFS for Hyper-V,” providing file level shared access to a shared SAN volume. This is distinguished from Sanbolic’s advanced file system, Melio FS, which provides byte-range locking and can provide concurrent access to application data on a SAN. The use of either Kayo FS or Melio FS resolves a key problem with Hyper-V deployments that want to take advantage of Quick Migration functionality, and that is that each VM would require its own LUN.

The introduction of Kayo FS also removes the key objection to the use of Melio FS for Hyper-V deployments: price. Kayo FS will be priced much lower than Melio FS; this means organizations adopting Hyper-V will be much more likely to swallow the cost of Kayo FS vs. Melio FS.

Here’s Virtualization Short Take #12, a collection of links I’ve gathered over the last week or so and my thoughts on them. Enjoy!

• For those that missed it in the Release Notes, VMware added support for Storage VMotion and 10Gb Ethernet with iSCSI SANs, as outlined in this VI Team blog entry. I went back and reviewed the Release Notes and didn’t see this listed anywhere, so this is news to me. Of course, I already knew that Storage VMotion worked just fine with iSCSI, but this added formal support for iSCSI.
• Virtualfuture.info published some good recommendations for running Citrix in a VI3 environment. If you run Citrix Presentation Server…er, XenApp…in a VI3 environment, these tuning tips may prove quite handy.
• VMware’s Virtual Reality blog posted an entry on some of the architectural advantages of VMware Infrastructure in comparison to the two leading competitors, Xen (any Xen-based solution) and Hyper-V. Many of the things listed as advantages by VMware are severe points of contention with the other vendors, such as the direct vs. indirect I/O model. Ultimately, time will tell which model was the best; I honestly don’t know enough about the deep dark internals to really state which is better. One thing I am glad to see pointed out is the true comparison of hypervisor sizes; Microsoft can say all they want that Hyper-V is only 600K in size and therefore is the “thinnest” hypervisor, but the truth of the matter is that Hyper-V can’t run without Windows Server 2008 in the parent partition. As a result, it doesn’t really matter how “thin” Hyper-V is, does it?
• Via Mike Laverick, I learned that Microsoft may have brought up the whole 64-bit hypervisor vs. 32-bit hypervisor argument yet again. Mike used a snippet from this Microsoft Virtualization Team Blog entry; in reading it myself, I don’t get quite the same 64-bit vs. 32-bit that Mike picked up. That’s good, because I didn’t want to have to go there again. Personally, the tone I picked up from the whole article was one of educating people far too accustomed to Virtual Server/VirtualPC and trying to educate them on how Hyper-V is different.
• Virtualization analyst Chris Wolf recently posted an entry in which he questioned if Apple would capitalize on the opportunity that virtualization is creating. It’s an interesting scenario, one that is similar to a scenario that I discussed a couple of years ago in a piece titled “Application Agnosticism.” In that article, I suggested that seamless host-guest interactions with virtualization software (now implemented by VMware as Unity and by Parallels as Coherence) would usher in a new wave of computing. I suggested that Mac OS X was ahead of the curve because of its ability to run native OS X applications, UNIX applications, X11 applications, Windows applications via WINE (or the commercial variant CrossOver Office), and applications from any other operating system via virtualization. Sounds like I may have been a bit ahead of my time!
• Chad continues discussing VMware HA with another post on some additional configuration options for HA. Also check out the comments with links to even more information on HA’s advanced configuration options.
• This VMware KB article has some good information on getting LUN identification information. The breakdown of the command-line output from esxcfg-mpath is particularly helpful (and for that reason I’ve added it to my del.icio.us bookmarks).
• Rich of VM /ETC shares with us a “Doh!” moment he had when he saw this simple method for identifying VMs with snapshots. Sometimes it’s the simplest solutions that evade us the longest. Here’s what I want to know: Aaron, what exactly does “/HEADDESK” mean, anyway?
• This article at SearchNetworking.com brings to light some of the challenges networking professionals face with server virtualization. I do agree with one point made in the article regarding the mapping of applications—what the end users really care about—to the networking infrastructure. VMware’s support for CDP in recent versions of VMware Infrastructure is a step in the right direction, but there is still more work to do for sure. I’m not so sure about the rest of the points in the article, but I may be an exception to the norm; I was a CCNA for a while (on track for CCNP) and have done my fair share of Cisco configurations, so I’m no stranger to the networking world. The use of VLANs to ease configuration in a server virtualization environment seems just second nature to me. Also, I did note that the author indicated that “server administrators sometimes inappropriately configure the switches to create a loop” (referring to vSwitches in ESX). How exactly does that happen? I’ve never seen a way to link two vSwitches together without using a VM.

As always, readers’ thoughts are welcome in the comments!

I’ll do my best to liveblog as much content from this session as possible, but I arrived late (you’ll understand why soon) and my battery is running low. Sorry all!

I arrived in the session as the speaker was reviewing some Hyper-V architecture with regards to the placement of the storage drivers and the use of Virtualization Service Clients (VSCs) and Virtualization Service Providers (VSPs). I think I covered those in my earlier liveblogging session from Day 1; if I didn’t, I’ll revise this post with more information later.

One key limitation with virtualized servers with regards to storage is how to handle Fibre Channel; you can’t really have multiple VMs using the same HBA because they would all share the same WWPN (World Wide Port Name), and SAN zoning is all built on the use of WWPNs. NPIV (N_Port ID Virtualization) is the answer here, which allows an HBA to register multiple fabric addresses. Each VM can use one of those multiple addresses. This allows us to return to zoning on a per server/per VM basis and eliminates this problem.

Next the presenter discussed the idea of virtual fabrics, a T.11 standard implemented by Cisco as VSAN and by Brocade as LSAN. Each HBA can only reside in a single virtual fabric, but traffic can be routed to other fabrics as needed.

Regarding virtual HBAs and fabric QoS, this is handled on a per-initiator WWPN basis. (I guess this means we can apply QoS to individual VMs when using NPIV, then?) I’m not exactly sure why the presenter is discussing this particular topic; it doesn’t seem to tie back to the main topic of storage connectivity for virtual machines.

So what is required to do virtual HBAs (again, I’m assuming this means using NPIV to present a virtual HBA to a Hyper-V hosted VM)? Next month, a newer version of Emulex VMPilot and the Storport Miniport driver will provide support for Windows Server 2003 SP2 and Hyper-V on Windows Server 2008. This will be compatible with 4Gpbs HBA from Emulex. Depending upon the HBA, the number of virtual ports (VPorts) may be limited (only 16 VPorts on midrange 4Gbps HBAs, for example).

There is no performance impact for using virtual HBAs, by the way. The number is so small as to defy measurement.

You also need FC switch support for NPIV. This requirement only applies to “edge switches” where NPIV-enabled HBAs will actually connect. There are also no other requirements on storage devices.

Best practices for storage access:

• For consolidating file/print servers or desktops, place all the VMs on a single shared LUN. (Note that this prevents Quick Migration.) You can’t use fabric QoS on a per-VM basis also, because all the VMs share the same WWPN of the physical HBA.
• For virtualizing application servers, use a dedicated LUN for each VM. This will allow the use of Quick Migration and can used virtual HBA, LUNs can be masked only to this specific VM, and you can create single-initiator zones using the virtual HBA and the applicable storage targets.

System Center Virtual Machine Manager (SCVMM) supports VDS (Virtual Disk Service) for Fibre Channel and iSCSI. VMM allows partners to extend built-in functionality by exposing NPIV WMI interfaces and allowing NPIV WMI methods to enumerate, create, and delete VPorts. In addition, VMM automatically manages VM and VPort migration in SANs.

Emulex VMPilot is the UI for SAN configuration with VMM. The current version is version 1.1; version 1.2 will be released next month and will provide support for Hyper-V. VMPilot with VMM enables VPort creation according to the T.11 NPIV standard, provides a graphical and command line interface, and provides automatic generation of virtual WWPNs. (I need to stop by the Emulex booth downstairs to get more information on this.) VMPilot also supplies a VMM PRO pack to report HBA health up to System Center Operations Manager and then back to SCVMM.

Recapping the benefits:

• LUN optimization through VM to LUN assignment
• Fabric QoS and prioritization at the VM level
• Single-initiator zoning possible, returns to a storage best practice
• Array-level LUN masking to control LUN access on a per-VM basis
• Accelerated VM migration (not sure about this one)
• VSAN integration and routing
• Eliminates duplication of storage administration tasks

I think that’s about going to do it for this liveblog; my battery is almost gone. I’ll be back with more information as soon as possible!

The recent couple of articles I wrote about using NetApp deduplication—in particular, the article on using NetApp deduplication with block storage—have raised some questions that are probably worth addressing. Although NetApp deduplication works just fine with block-based storage, there are some considerations with regards to how the LUNs should be provisioned when deduplication will come into play.

Fortunately for me, someone at NetApp decided that it would be a good idea to document the five basic configurations of using NetApp deduplication with block storage. As seen in this comment to an earlier article, Larry Freeman points out this document on the NetApp Communities site (has anyone else noticed the similarity between VMware Communities and NetApp Communities?) that outlines the 5 basic configurations and where the freed blocks go in each configuration. Excellent—that saves me some work!

The most common configurations I’ve seen are configurations D (LUN not space reserved, space guarantee set to Volume) and E (LUN not space reserved, space guarantee set to None). Customers like to see the LUNs “shrink” after deduplication, and this is the only way to make that happen.

The only things we need now are for NetApp to a) remove the volume size limitations and b) get us deduplication at the aggregate level. Then we’d really be set!

I came across this great article on the HP Fibre Channel VirtualConnect module tonight. Excellent work! Combine this with my TechTarget article on Ethernet networking with HP VirtualConnect and you have some good resources for using VirtualConnect in your environment.

Building on my earlier article on setting up NetApp deduplication, I wanted to follow up with some information on using NetApp deduplication with block storage (LUNs presented via Fibre Channel or iSCSI).

For the most part, using NetApp deduplication with block storage is a lot like I described earlier:

• You (obviously) still need the NearStore and deduplication (A-SIS) licenses installed on the controller(s).
• You will still turn deduplication on using the “sis on” command for the FlexVol containing the LUNs.
• Limitations on the size of the FlexVol still apply.
• You use the “sis status” command to check on the status of deduplication, and the “sis config” command to see the deduplication schedule.

OK, so what’s different? Well, it has to do with how LUNs are provisioned on a NetApp storage system. I’ve blogged before about managing LUN space requirements on a NetApp, and about using LUN clones vs. FlexClones. That second article, in particular, really goes into detail on how LUNs are implemented on top of NetApp’s file system, WAFL. Since LUNs are represented by WAFL as a single file, they are also normally “space reserved,” meaning that the maximum size of the LUN is allocated at the time of creation. If you create a 50GB LUN, then Data ONTAP creates a 50GB file right away. (For readers out there who are well-versed in NetApp storage, I know that’s a bit of a simplification, but bear with me.)

What does this have to do with deduplication? Great question. If the LUN is space reserved—meaning that the maximum size of the LUN is allocated up front and remains allocated to the LUN—then the file that represents the LUN won’t ever decrease in size to reflect deduplication savings, and deduplication therefore does you absolutely no good whatsoever. This is not to say that deduplication doesn’t work, just that it won’t help you at all.

Fortunately, there’s an easy fix for this. When creating the LUN, simply uncheck the box marked “Space Reserved” and allow Data ONTAP to allocate space to the LUN out of the containing FlexVol on an as-needed basis. Because the file that represents the LUN can grow in size, it can also shrink in size, and deduplication will cause the file that represents the LUN to decrease in size. This then allows you to provision additional LUNs from the same FlexVol to take advantage of the space savings resulting from deduplication.

I know that seems a bit confusing; I’ll probably post another article with some more in-depth discussions of the details. (Either that, or I’ll encourage my NetApp readers to chime in below in the comments.)

So, in summary, when using NetApp deduplication with block storage:

• you’ll setup and configure deduplication on the FlexVol containing your LUN(s) just like described in my earlier article;
• you’ll uncheck the “Space Reserved” checkbox when creating the LUNs to be deduplicated;
• you won’t see the space savings from the host’s perspective and therefore can’t store more data in that LUN than the size of the LUN; but
• you will be able to provision additional LUNs in that same FlexVol that can be presented back to host for additional storage.

I hope this helps clarify some of the questions or issues surrounding the use of NetApp deduplication with block storage. Feel free to add information, experiences with deduplication and block storage, or ask additional questions in the comments below.

UPDATE: There are some additional considerations about how to provision LUNs along with NetApp deduplication that warrant a more in-depth discussion. Look for a follow-up post within the next few days.

VMware’s done a great job of rolling together the VMware news and views with their VMware blog aggregator, Planet V12n. Does anyone know of something equivalent for NetApp? Where is “Planet NetApp”?

If you’ve worked with Network Appliance storage before, you’re probably already familiar with the idea of snap reserve (storage space set aside to accommodate for Snapshots) and fractional reserve (used with LUNs).  I’m going to hold the in-depth discussion of why you need snap reserve and fractional reserve for a different day, but I did want to pass on these commands that were shared with me by a colleague of mine.  These Data ONTAP commands, available with Data ONTAP 7.2 or later (some commands are available in Data ONTAP 7.1), will help you manage the space requirements for LUNs on a NetApp storage area network (SAN).

I’ll try to explain the commands along the way, but I would recommend you review the documentation available from the NOW site for more complete information.

vol options <volname> fractional_reserve 0

This command sets the fractional reserve to zero percent, down from the default of 100 percent.  Note that fractional reserve only applies to LUNs, not to NAS storage presented via CIFS or NFS.

snap autodelete <volname> trigger snap_reserve

This sets the trigger at which Data ONTAP will begin deleting Snapshots.  In this case, Snapshots will start getting deleted when the snap reserve for the volume gets nearly full.  The current size of the snap reserve can be viewed for a particular volume with the “snap reserve <volname>” command.

snap autodelete <volname> defer_delete none

This command instructs Data ONTAP not to exhibit any preference in the types of Snapshots that are deleted.  Options for this command include “user_created” (delete user-created Snapshot copies last) or “prefix” (Snapshot copies with a specified prefix string).

snap autodelete <volname> target_free_space 10

With this setting in place, Snapshots will be deleted until there is 10% free space in the volume.

snap autodelete <volname> on

Now that the Snapshot autodelete options have been configured, this command will actually turn the functionality on.

vol options <volname> try_first snap_delete

When a FlexVol runs into an issue with space, this option tells Data ONTAP to first try to delete Snapshots in order to free up space.  This command works in conjunction with the next command:

vol autosize <volname> on

This enables Data ONTAP to automatically grow the size of a FlexVol if the need arises.  This command works hand-in-hand with the previous command; Data ONTAP will first try to delete Snapshots to free up space, then grow the FlexVol according to the autosize configuration options.  Between these two options—Snapshot autodelete and volume autogrow—you can reduce the fractional reserve from the default of 100 and still make sure that you don’t run into problems taking Snapshots of your LUNs.

If you have a NOW login, you can get more information on Snapshot autodelete here; more information on volume autogrow is available here.  Be aware that SnapDrive may require different settings in order to accommodate its functionality, as it moves LUN management out of the storage system and onto the host.  Finally, the values presented here are only examples; be sure to use values that are appropriate for your environment.

Credit for compiling this list goes to my colleague Chauncey Willard.  Good work!