ONTAP

You are currently browsing articles tagged ONTAP.

3010: A MultiStore Primer

Wednesday, April 8, 2009 in Storage, Virtualization by slowe | 18 comments

This session describes NetApp’s MultiStore functionality. MultiStore is the name given to NetApp’s functionality for secure logical partitioning of network and storage resources. The presenters for the session are Roger Weeks, TME with NetApp, and Scott Gelb with Insight Investments.

When using MultiStore, the basic building block is the vFiler. A vFiler is a logical construct within Data ONTAP that contains a lightweight instance of the Data ONTAP multi-protocol server. vFilers provide the ability to securely partition both storage resources and network resources. Storage resources are partitioned at either the FlexVol or Qtree level; it’s recommended to use FlexVols instead of Qtrees. (The presenters did not provide any further information beyond that recommendation. Do any readers have more information?) On the network side, the resources that can be logically partitioned are IP addresses, VLANs, VIFs, and IPspaces (logical routing tables).

Some reasons to use vFilers would include storage consolidation, seamless data migration, simple disaster recovery, or better workload management. MultiStore integrates with SnapMirror to provide some of the functionality needed for some of these use cases.

MultiStore uses vFiler0 to denote the physical hardware, and vFiler0 “owns” all the physical storage resources. You can create up to 64 vFiler instances, and active/active clustered configurations can support up to 130 vFiler instances (128 vFilers plus 2 vFiler0 instances) during a takeover scenario.

Each vFiler stores its configuration in a separate FlexVol (it’s own root vol, if you will). All the major protocols are supported within a vFiler context: NFS, CIFS, iSCSI, HTTP, and NDMP. Fibre Channel is not supported; you can only use Fibre Channel with vFiler0. This is due to the lack of NPIV support within Data ONTAP 7. (It’s theoretically possible, then, that if/when NetApp adds NPIV support to Data ONTAP that Fibre Channel would be supported within vFiler instances.)

Although it is possible to move resources between vFiler0 and a separate vFiler instance, doing so may impact client connections.

Managing vFilers appears to be the current weak spot; you can manage vFiler instances using the Data ONTAP CLI, but vFiler instances don’t have an interactive shell. Therefore, you have to direct commands to vFiler instances via SSH or RSH or using the vFiler context in vFiler0. You access the vFiler context by prepending the “vfiler” keyword to the commands at the CLI in vFiler0. Operations Manager 3.7 and Provisioning Manager can manage vFiler instances; FilerView can start, stop, or delete individual vFiler instances but cannot direct commands to an individual vFiler. If you need to manage CIFS on a vFiler instance, you can use the Computer Management MMC console to connect remotely to that vFiler instance to manage shares and share permissions, just as you can with vFiler0 (assuming CIFS is running within the vFiler, of course).

IPspaces are a logical routing construct that allow each vFiler to have its own routing table. For example, you may have a DMZ vFiler and an internal vFiler, each with their own, separate routing table. Up to 101 IPspaces are supported per controller. You can’t delete the default IPspace, as it’s the routing table for vFiler0. It is recommended to use VLANs and/or VIFs with IPspaces as a best practice.

One of the real advantages of using MultiStore and vFilers is the data migration and disaster recovery functionality that it enables when used in conjunction with SnapMirror. There are two sides to this:

  • “vfiler migrate” allows you to move an entire vFiler instance, including all data and configuration, from one physical storage system to another physical storage system. You can keep the same IP address or change the IP address. All other network identification remains the same: NetBIOS name, host name, etc., so the vFiler should look exactly the same across the network after the migration as it did before the migration.
  • “vfiler dr” is similar to “vfiler migrate” but uses SnapMirror to keep the source and target vFiler instances in sync with each other.

It makes sense, but you can’t use “vfiler dr” or “vfiler migrate” on vFiler0 (the physical storage system). My own personal thought regarding “vfiler dr”: what would this look like in a VMware environment using NFS? There could be some interesting possibilities there.

With regard to security, a Matasano security audit was performed and the results showed that there were no vulnerabilities that would allow “data leakage” between vFiler instances. This means that it’s OK to run a DMZ vFiler and an internal vFiler on the same physical system; the separation is strong enough.

Other points of interest:

  • Each vFiler adds about 400K of system memory, so keep that in mind when creating additional vFiler instances.
  • You can’t put more load on a MultiStore-enabled system than a non-MultiStore-enabled system. The ability to create logical vFilers doesn’t mean the physical storage system can suddenly handle more IOPS or more capacity.
  • You can use FlexShare on a MultiStore-enabled system to adjust priorities for the FlexVols assigned to various vFiler instances.
  • As of Data ONTAP 7.2, SnapMirror relationships created in a vFiler context are preserved during a “vfiler migrate” or “vfiler dr” operation.
  • More enhancements are planned for Data ONTAP 7.3, including deduplication support, SnapDrive 5.0 or higher support for iSCSI with vFiler instances, SnapVault additions, and SnapLock support.

Some of the potential use cases for MultiStore include file services consolidation (allows you to preserve file server identification onto separate vFiler instances), data migration, and disaster recovery. You might also use MultiStore if you needed support for multiple Active Directory domains with CIFS.

UPDATE: Apparently, my recollection of the presenters’ information was incorrect, and FTP is not a protocol supported with vFilers. I’ve updated the article accordingly.

Tags: , , , , , , , , , , , ,

Storage Alignment Document

Tuesday, March 31, 2009 in Storage, Virtualization by slowe | 17 comments

NetApp has recently released TR-3747, Best Practices for File System Alignment in Virtual Environments. This document addresses the situations in which file system alignment is necessary in environments running VMware ESX/ESXi, Microsoft Hyper-V, and Citrix XenServer. The authors are Abhinav Joshi (he delivered the Hyper-V deep dive at Insight last year), Eric Forgette (wrote the Rapid Cloning Utility, I believe), and Peter Learmonth (a well-recognized name from the Toasters mailing list), so you know there’s quite a bit of knowledge and experience baked into this document.

There are a couple of nice tidbits of information in here. For example, I liked the information on using fdisk to set the alignment of a guest VMDK from the ESX Service Console; that’s a pretty handy trick! I also thought the tables which described the different levels at which misalignment could occur were quite useful. (To be honest, though, it took me a couple of times reading through that section to understand what information the authors were trying to deliver.)

Anyway, if you’re looking for more information on storage alignment, the different levels at which it may occur, and the methods used to fix it at each of these levels, this is an excellent resource that I strongly recommend reading. Does anyone have any pointers to similar documents from other storage vendors?

Tags: , , , , , , , , , ,

NetApp VIF Member Limitations

Monday, February 2, 2009 in Networking, Storage by slowe | 24 comments

I’ve written a couple of times about NetApp virtual interfaces (VIFs), which are Data ONTAP’s name for a link aggregate using either EtherChannel or dynamic LACP. The earlier articles I wrote are:

Cisco Link Aggregation and NetApp VIFs
LACP with Cisco Switches and NetApp VIFs

I came across an issue today of which I was not aware. I’ve been working on a new NetApp deployment with a fellow engineer that called for a number of different VIFs to be created: one for CIFS traffic, one for NFS traffic, and one for SnapMirror traffic. (Yes, I know the SnapMirror VIF won’t really use more than one link because it’s all point-to-point traffic; it’s primarily for redundancy.) There were some really strange network issues going on, like losing connectivity to the default gateway one moment and then network connectivity being restored the next. We were having a hard time troubleshooting the problem until one of the network engineers casually commented that it looked like the static LACP bundles (the aggregated links represented by the VIFs on the NetApp storage array) weren’t really coming up.

That comment lead to a deeper inspection of the NetApp VIFs and eventually a case with NetApp. The end result was that we learned that multimode VIFs can’t span built-in NICs and add-in NICs. Since the FAS3000 series has a limited number of built-in NICs, we’d installed two additional quad-port NICs and then, as was customary, created VIFs spanning the built-in NICs and the add-in NICs for maximum redundancy. Well, that doesn’t work!

Once we reconfigured the Cisco switches (these were Cisco Catalyst 3750 switches uplinked via 10 Gigabit Ethernet to Catalyst 6509 switches) so that the link aggregates only contained add-in NICs or built-in NICs but not both, the connections came up fully and the network connectivity issues disappeared.

So, when creating multimode VIFs, be sure to only include NICs from add-in cards or the built-in NICs, but not both.

Tags: , , ,

2031: Enhancements to NetApp Cloning Technology

Wednesday, December 3, 2008 in Storage by slowe | No comments

This session provided information on enhancements to NetApp’s cloning functionality. These enhancements are due to be released along with Data ONTAP 7.3.1, which is expected out in December. Of course, that date may shift, but that’s the expected release timeframe.

The key focus of the session was new functionality that allows for Data ONTAP to clone individual files without a backing Snapshot. This new functionality is an extension of NetApp’s deduplication functionality, and is enabled by changes within Data ONTAP that enable block sharing, i.e., the ability for a single block to appear in multiple files or in multiple places in the same file. The number of times these blocks appear is tracked using a reference count. The actual reference count is always 1 less than the number of times the block appears. A block which is not shared has no reference count; a block that is shared in two locations has a reference count of 1. The maximum reference count is 255, so that means a single block is allowed to be shared up to 256 times within a single FlexVol. Unfortunately, there’s no way to view the reference count currently, as it’s stored in the WAFL metadata.

As with other cloning technologies, the only space that is required is for incremental changes from the base. (There is small overhead for metadata as well.) This functionality is going to be incorporated into the FlexClone license and will likely be referred to as “file-level FlexClone”. I suppose that cloning volumes with be referred to as “volume FlexClone” or similar.

This functionality will be command-line driven, but only from advanced mode (must do a “priv set adv” in order to access the commands). The commands are described below.

To clone a file or a LUN (the command is the same in both cases):

clone start <src_path> <dst_path> -n -l

To check the status of a cloning process or stop a cloning process, respectively:

clone status
clone stop

Existing commands for Snapshot-backed clones (“lun clone” or “vol clone”) will remain unchanged.

File-level cloning will integrate with Volume SnapMirror (VSM) without any problems; the destination will be an exact copy of the source, including clones. Not so for Qtree SnapMirror (QSM) and SnapVault, which will re-inflate the clones to full size. Users will need to run deduplication on the destination to try to regain the space. Dump/restores will work like QSM or SnapVault.

Now for the limitations, caveats and the gotchas:

  • Users can’t run single-file SnapRestore and a clone process at the same time.
  • Users can’t clone a file or a LUN that exists only in a Snapshot. The file or LUN must exist in the active file system.
  • ACLs and streams are not cloned.
  • The “clone” command does not work in a vFiler context.
  • Users can’t use synchronous SnapMirror with a volume that contains cloned files or LUNs.
  • Volume SnapRestore cannot run while cloning is in progress.
  • SnapDrive does not currently support this method of cloning. It’s anticipated that SnapManager for Virtual Infrastructure (SMVI) will be the first to leverage this functionality.
  • File-level FlexClone will be available for NFS only at first. Although it’s possible to clone data regions within a LUN, support is needed at the host level that isn’t present today.
  • Because blocks can only be shared 256 times (within a file or across files), it’s possible that some blocks in a clone will be full copies. This is especially true if there are lots of clones. Unfortunately, there is no easy way to monitor or check this. “df -s” can show space savings due to cloning, but that isn’t very granular.
  • There can be a maximum of 16 outstanding clone operations per FlexVol.
  • There is a maximum of 16TB of shared data among all clones. Trying to clone more than that results in full copies.
  • The maximum volume size for being able to use cloning is the same as for deduplication.

Obviously, VMware environments—VDI in particular—are a key use case for this sort of technology. (By the way, in case no one has yet connected the dots, this is the technology that I discussed here). To leverage this functionality, NetApp will update a tool known as the Rapid Cloning Utility (RCU; described in more detail in TR-3705) to take full advantage of file-level FlexCloning after Data ONTAP 7.3.1 is released. Note that the RCU is available today, but it only uses volume-level FlexClone.

Tags: , , , , , , , ,

A Couple More Articles Published

Sunday, November 2, 2008 in Networking, Storage, Virtualization by slowe | 1 comment

So a couple more articles I wrote have been published by SearchVMware.com:

Deduplication enhancements improve NetApp on VMware ESX

NetApp storage area network (SAN) deduplication is useful for reducing storage requirements and saving space out of the box. When combined with a VMware Infrastructure 3 (VI3) or a virtual desktop environment, however, a few simple configurations can offer additional space savings and minimize the overhead of running deduplication on a storage array.

Physical network design options for VI3

VMware ESX offers outstanding support for a wide variety of networking configurations. Users have the option of using network interface card (NIC) teaming with or without physical switch support; numerous VLAN configurations, three of which are described in more detail in VST, EST and VGT tagging tips; support for both active and standby NICs, including per-port group active/standby NICs; and jumbo frame support. With all these options, it can be daunting to find the right configuration for your environment. In this article, we take a closer look at some network design decisions and how they play into the physical side of the network.

As usual, these articles have been added to my Delicious.com bookmark list and tagged with “Articles”; you can view my full list of articles here. (Oh, I added a video I shot while at VMworld 2008 to that list as well, in case you were wondering why it’s there.)

Tags: , , , , , ,

Storage Short Take #2

Tuesday, August 5, 2008 in Storage by slowe | 5 comments

Here’s another installation of Storage Short Takes, with news and views of the storage world. Readers, if you have any more sources of information about storage happenings, please let me know in the comments. I’d love to expand the coverage of storage in these posts.

  • I came across this EMC CLARiiON-focused blog the other day. Although it hasn’t been updated since March of this year and the volume of posts is quite low, the quality of the posts seems pretty good. (I’m new to EMC storage, though, so what this guy is saying might be way off and I’d never know.) The blog post about LUN alignment plus VMDK alignment was particularly helpful.
  • Stephen Foskett has a good overview of just the storage-related fixes and enhancements found in Update 2 of VMware Infrastructure 3 version 3.5.
  • EMC recently introduced the CLARiiON CX4 series of storage arrays (more information here), and Chad Sakac blogged about it in relation to VMware and other virtualization technologies. Looks pretty impressive; I may have to take Chad up on his offer of hardware for the lab…
  • NetApp has apparently released Data ONTAP 7.3. Check NOW for more details on the specific features added in this release.

Thanks for reading!

UPDATE: I had a typo in the URL for the NOW site above; that has been corrected. Thanks, Jeremiah!

Tags: , , , , ,

FlexClones Versus Deduplication with VMware Infrastructure

Tuesday, August 5, 2008 in Storage, Virtualization by slowe | 8 comments

A number of times over the last few months, I’ve run into situations where NetApp’s FlexClone technology was being heavily pitched to customers interested in deploying, or expanding their deployment of, VMware Infrastructure.

In case you aren’t familiar with the use of NetApp FlexClones in conjunction with VMware Infrastructure, have a look at these earlier articles of mine:

How to Provision VMs Using NetApp FlexClones
NetApp FlexClones with VMware, Part 1
NetApp FlexClones with VMware, Part 2
LUN Clones vs. FlexClones

Now, after you’ve read all those articles (you did read them, didn’t you?), it should be fairly clear that using FlexClones can be very advantageous. However, those advantages come with some tradeoffs as well, most notably in the complete and total lack of integration with VMware Infrastructure itself.

This lack of integration means that users can’t use VirtualCenter templates, because the cloning is taking place at the storage array instead of within VMware Infrastructure. This also means that customers can’t apply customization specifications during the cloning process, so users will need to create their own Sysprep answer files and manually Sysprep the VMs before invoking the FlexClone process. Users are required to create scripts and tools to do simple things like using the VM name for the guest OS name during cloning.

<aside>Lest anyone think I’m picking on NetApp here, let me state that this would apply to any storage vendor that offers pointer-based copies. As long as the use of those pointer-based copies (or even deep copies, for that matter) is not integrated within VirtualCenter, then they will suffer the same problems.</aside>

Deduplication, on the other hand, works seamlessly with VMware Infrastructure. This is primarily because the details of the deduplication are completely hidden; it all occurs “inside the box.” Nothing needs to be configured within VirtualCenter; no VMs need to be modified. The NetApp storage system handles the details of the deduplication process itself, and VMware Infrastructure just consumes the storage.

Looking at these two technologies in that light, one might ask: why use FlexClones at all? If deduplication works seamlessly with VMware Infrastructure and FlexClones don’t, then why bother? To be honest, there are some instances where FlexClones make sense—even with the lack of integration. Consider some of the examples listed below.

  • In instances where a user needs to deploy lots of VMs in a very rapid fashion, FlexClones are much better. If time-to-deployment is the #1 driving factor, then FlexClones are the way to go. This could be particularly applicable and useful in VDI situations, as long as the broker doesn’t mandate handling provisioning itself (like VDM does).
  • In environments where provisioning and re-provisioning occurs on a frequent, regular basis, then FlexClones make sense. Even though large numbers of VMs aren’t being provisioned, the time saved on frequent re-provisioning via FlexClones will not be insignificant.
  • In situtations where there isn’t sufficient storage for the VMs before they are deduplicated, FlexClones may be a better option. Deduplication is post-process, meaning that storage will be needed for the full datasets until deduplication runs. In situations where that isn’t an option, then FlexClones can provide the same end benefit.

Personally, I’m of the opinion that unless an organization meets one of these criteria, then that organization should look to deduplication instead of FlexClones. Of course, that’s just my personal opinion, and I’m open to hear what others have to say about the matter. NetApp gurus, feel free to weigh in.

Tags: , , , , , ,

Provisioning LUNs for Use with Deduplication

Tuesday, May 20, 2008 in Storage by slowe | No comments

The recent couple of articles I wrote about using NetApp deduplication—in particular, the article on using NetApp deduplication with block storage—have raised some questions that are probably worth addressing. Although NetApp deduplication works just fine with block-based storage, there are some considerations with regards to how the LUNs should be provisioned when deduplication will come into play.

Fortunately for me, someone at NetApp decided that it would be a good idea to document the five basic configurations of using NetApp deduplication with block storage. As seen in this comment to an earlier article, Larry Freeman points out this document on the NetApp Communities site (has anyone else noticed the similarity between VMware Communities and NetApp Communities?) that outlines the 5 basic configurations and where the freed blocks go in each configuration. Excellent—that saves me some work!

The most common configurations I’ve seen are configurations D (LUN not space reserved, space guarantee set to Volume) and E (LUN not space reserved, space guarantee set to None). Customers like to see the LUNs “shrink” after deduplication, and this is the only way to make that happen.

The only things we need now are for NetApp to a) remove the volume size limitations and b) get us deduplication at the aggregate level. Then we’d really be set!

Tags: , , ,

Using NetApp Deduplication with Block Storage

Thursday, April 24, 2008 in Storage by slowe | 18 comments

Building on my earlier article on setting up NetApp deduplication, I wanted to follow up with some information on using NetApp deduplication with block storage (LUNs presented via Fibre Channel or iSCSI).

For the most part, using NetApp deduplication with block storage is a lot like I described earlier:

  • You (obviously) still need the NearStore and deduplication (A-SIS) licenses installed on the controller(s).
  • You will still turn deduplication on using the “sis on” command for the FlexVol containing the LUNs.
  • Limitations on the size of the FlexVol still apply.
  • You use the “sis status” command to check on the status of deduplication, and the “sis config” command to see the deduplication schedule.

OK, so what’s different? Well, it has to do with how LUNs are provisioned on a NetApp storage system. I’ve blogged before about managing LUN space requirements on a NetApp, and about using LUN clones vs. FlexClones. That second article, in particular, really goes into detail on how LUNs are implemented on top of NetApp’s file system, WAFL. Since LUNs are represented by WAFL as a single file, they are also normally “space reserved,” meaning that the maximum size of the LUN is allocated at the time of creation. If you create a 50GB LUN, then Data ONTAP creates a 50GB file right away. (For readers out there who are well-versed in NetApp storage, I know that’s a bit of a simplification, but bear with me.)

What does this have to do with deduplication? Great question. If the LUN is space reserved—meaning that the maximum size of the LUN is allocated up front and remains allocated to the LUN—then the file that represents the LUN won’t ever decrease in size to reflect deduplication savings, and deduplication therefore does you absolutely no good whatsoever. This is not to say that deduplication doesn’t work, just that it won’t help you at all.

Fortunately, there’s an easy fix for this. When creating the LUN, simply uncheck the box marked “Space Reserved” and allow Data ONTAP to allocate space to the LUN out of the containing FlexVol on an as-needed basis. Because the file that represents the LUN can grow in size, it can also shrink in size, and deduplication will cause the file that represents the LUN to decrease in size. This then allows you to provision additional LUNs from the same FlexVol to take advantage of the space savings resulting from deduplication.

I know that seems a bit confusing; I’ll probably post another article with some more in-depth discussions of the details. (Either that, or I’ll encourage my NetApp readers to chime in below in the comments.)

So, in summary, when using NetApp deduplication with block storage:

  • you’ll setup and configure deduplication on the FlexVol containing your LUN(s) just like described in my earlier article;
  • you’ll uncheck the “Space Reserved” checkbox when creating the LUNs to be deduplicated;
  • you won’t see the space savings from the host’s perspective and therefore can’t store more data in that LUN than the size of the LUN; but
  • you will be able to provision additional LUNs in that same FlexVol that can be presented back to host for additional storage.

I hope this helps clarify some of the questions or issues surrounding the use of NetApp deduplication with block storage. Feel free to add information, experiences with deduplication and block storage, or ask additional questions in the comments below.

UPDATE: There are some additional considerations about how to provision LUNs along with NetApp deduplication that warrant a more in-depth discussion. Look for a follow-up post within the next few days.

Tags: , , , ,

NetApp Blog Aggregator?

Sunday, April 20, 2008 in Storage by slowe | 2 comments

VMware’s done a great job of rolling together the VMware news and views with their VMware blog aggregator, Planet V12n. Does anyone know of something equivalent for NetApp? Where is “Planet NetApp”?

Tags: , , , ,

« Older entries