NetApp RCU and VSC

I spent some time last week at the NetApp RTP office getting a special sneak preview of a couple of software products getting announced at VMworld 2009 in San Francisco. One of these is the Rapid Cloning Utility (RCU); the other is the Virtual Storage Console (VSC). Both of these software products are intended to plug into vCenter Server to provide more centralized access to both storage and virtualization management.

The Rapid Cloning Utility (RCU) has actually been around for a while, but it wasn’t an officially supported tool. This new version of RCU changes that; it now becomes a free tool but an officially supported tool for NetApp customers with active support agreements. The primary purpose of the RCU is to automate the use of NetApp’s FlexClone functionality for rapidly provisioning virtual machines. The scope and scale of virtual desktop deployments lends itself well to the use of RCU, but RCU could also be applicable in server virtualization environments as well.

Some of the functionality brought to the table by the RCU includes:

  • Full support for cloning block-based datastores accessed via Fibre Channel or iSCSI
  • Full support for file-level FlexCloning on NFS datastores
  • Automated import of virtual machines into View Manager, where applicable
  • Bulk import into XenDesktop, where applicable
  • Ability to store virtual machine swap file in separate VMDK in a separate datastore
  • Support for MetroCluster

In addition, the RCU incorporates support for deduplication (options to enable deduplication and report on it from within vCenter Server), NFS datastore resizing, creating/deleting/cloning block-based datastores, and support for basic role-based administration control (RBAC) in that user permissions are checked before tasks are launched. In addition, use of the RCU mitigates many of the drawbacks I’ve discussed in the past with regard to use array-based cloning in virtualized environments. All in all, it’s a useful addition to vCenter Server for environments using NetApp storage.

The Virtual Storage Console (VSC) replaces the old NetApp Host Utilities Kit (HUK), which NetApp used to fine-tune and configure certain host and HBA parameters. Now those same settings can be managed from within vCenter Server. The VSC also provides access to NetApp’s mbrscan and mbralign tools, which are designed to identify and correct problems with VMDK alignment.

Both of these utilities, if I recall correctly, require that you are running the very latest version of Data ONTAP.

Tags: , , ,

20 comments

  1. Chad Sakac’s avatar

    These are very good utilities. I would be remiss (as an EMCer) if I didn’t point people interested in this idea, but are EMC customers that there are several analagous vCenter plugins for EMC.

    The EMC analogue to the VSC is a vCenter plugin called the EMC Storage Viewer launched at VMworld Europe 2009 in Feb.. It is freely available on Powerlink (details are on my blog here: http://virtualgeek.typepad.com/virtual_geek/2009/04/where-to-get-the-emc-storage-viewer-vcenter-plugin.html)

    The EMC analogue to RCU is a vCenter plugin called the VDI Deployment Tool launched at VMworld Europe 2009 in Feb.. It is freely available on Powerlink (details are on my blog here: – http://virtualgeek.typepad.com/virtual_geek/2009/03/vmworld-europe-2009-emc-post-show-report.html

    The EMC Analogue to SMVI (snapmanager for Virtual Infrastructure) is EMC Replication Manager.

    Personally, I think the RCU is a bit better than our existing deployment tool (VMworld is coming up :-) but on the others, I think rational folks could debate back and forth which are better.

    (we have a couple others as well at EMC with no NetApp analog – the SRM failback vcenter plugin most notably – with a few others coming. Like VSC, we plan to bring these plugins together over time via a common framework model).

    It’s fun to have a strong competitor to parry back and forth with – in the end, it’s good for customers!

  2. slowe’s avatar

    When I get the NS-960 up and running (should be this week), I plan to have a look at your utilities as well!

  3. dlove’s avatar

    As an EMC’er, I would like to see a properly functioning webserver aka Navisphere that wasn’t based on embedded Windows XP and so slow it makes administering the array painful at best. EMC’s answer? upgrade to DMX… We have CX3-80, CX3-40, CX-700 and they all do the same thing, horrible performance for practically half a mil a piece :)

    I’d even settle for a global hot spare and for that I’d have to get into a CX4, wow such innovators!

  4. amanoj’s avatar

    This is great news. I have been eagerly waiting for the new release of this tool. My environment is utilizing vSphere 4.0 with DataStores hosted on a NetApp 3140 Cluster. Do you know if this is a licensed product? (Meaning is it free from NetApp, or $$$?)

  5. Eric Forgette’s avatar

    Hi Scott,
    The previous version of the RCU is fully supported as well.

    Hi Amanoj,
    Both products are free to NetApp customers. Flexclone is required to use the RCU.

    Thanks,
    -Eric

  6. Brian  Gracely’s avatar

    @amanoj – RCU is a free distribution via the NetApp NOW site, it doesn’t require any licensing. To utilize FlexCloning, that does require a FlexClone license for ONTAP. VSC also includes the MBR* tools for proper VM alignment &/or migration for misalignment.

  7. Peter Learmonth’s avatar

    Amanoj: There is no charge for RCU or VSC, however there are some other products they leverage which require a license. RCU uses FlexClone. Single VM cloning only works with NFS (for now), so you would need the NFS license for that. RCU can clone whole datastores with NFS or VMFS/LUN-based datastores.

    Chad: IMHO, the EMC SRM failback “plugin” is an incomplete product at best. All it does is turn the EMC mirror around. It does nothing to create or reverse any failover workflows within SRM itself, which is the more complicated part – at least as compared to SnapMirror management. So, while having a “failback plugin” for SRM for EMC replication looks good in PowerPoint – big deal. SnapMirror is so easy to turn around we hardly need a plugin – but that doesn’t mean there won’t ever be one.

  8. Vaughn’s avatar

    Sorry to be late to the party – a few points and thoughts

    1. EMC products often only work on singular arrays and/or protocols.

    The EMC Storage Viewer is similar to the NetApp VSC with a few ‘minor’ differences including… the VSC works on every platform running Data ONTAP, with every storage protocol, and actually changes the configurations of the ESSX/ESXi hosts to their optimal values.

    The EMC VDI tool is similar to the NetApp RCU with a few ‘minor’ differences including… the ECM tool clones 2 VMs per LUN, only supports Celerra, and iSCSI. The NetApp RCU works on every platform running Data ONTAP, with every storage protocol, can clone any number of VM and datastores, imports directly into View Manager and XenDesktop, and all clones both Virtual Servers along with Virtual Desktops are pre-deduplicated.

    Is the EMC Replication Manager similar to SMVI? I don’t know. Can it use its offsite backup disk with SRM? Does testing a SRM recovery plan require additional storage or disrupt replication?

    2. The VSC & RCU are both free and SMVI is agressively priced!

    3. The EMC failback plug-in does automate the return of services to the primary site. Come check out the NetApp booth at VMworld for more on this type of functionality! Sorry – I can’t disclose everything here!

  9. Chad Sakac’s avatar

    @Peter – that’s not all it does (and this is why I try to NOT comment on what NetApp does/doesnt do).

    - It correlates VMs and failover replication sessions by integrating with the vCenter APIs.
    - It reverses replication (easy with SnapMirror and Celerra Replicator)
    - It handles promotion, resignaturing and renaming datastore handing in ESX
    - It handles registration of VMs in vCenter

    The next version (targeted to be synchronous with the NFS support in SRM) adds:
    - NFS support
    - IP address handling
    - some sequencing

    @dlove – thank you for the feedback. global hot-sparing has been in anything since FLARE 26 – going back several years, and is a free non-disruptive upgrade. And I’m sorry for anything that has been less than perfect. Re GUI performance and handling – Stay tuned – won’t be long. Now – the CX700 series is in essence 4-5 years old at this point (equivalent to using a FAS900 series platform) – you can contact me directly if you would like, and I’m happy to personally help you.

  10. slowe’s avatar

    For readers who need a score card, and since some of our commenters are forgetting to provide vendor affiliation:

    Chad Sakac – Works for EMC
    Vaughn Stewart, James Burke, Peter Learmonth, Brian Gracely, Eric Forgette – Work for NetApp

    Hope this helps!

  11. Vaughn’s avatar

    Can you add to your score card: Scott Lowe with E-Plus – HP Elite and NetApp Star Partner!

  12. Chad Sakac’s avatar

    Answers inline. I always try to be as transparent as I can. Will you?

    ** I really want to state something up front. I DO NOT WANT TO DO THIS. This is being negative on a respected competitor, but you have consistently poked and goaded. my STRONG preference is for the customer to challenge each vendor they consider to make their TOTAL argument why they are the right choice – FOR THEM. **

    Sigh – here it goes.

    VAUGHN: 1. EMC products often only work on singular arrays and/or protocols.

    CHAD: True. Our platforms have relative strengths and weaknesses in various use cases. Some customer need platforms for all sorts of use cases. Example: constant non-disruptive operations, very high active port scaling, huge numbers of storage objects, replicas, and mainframe support – NO ONE does this in midrange platforms – these are characteristics of high-end arrays – ergo HDS USP, IBM DS8000, EMC Symmetrix). NetApp calls this a weakness – always. I think a more rational person looks at it and says: “sometimes it’s a strength, sometimes it’s a weakness. Depends on the customer requirements”.

    What is rarely discussed by NetApp – is not all functionality is common across protocols. Example – VM-level replicas is NFS only – not iSCSI/FC/FCoE. Another Example? MultiStore (is that the right name?) is iSCSI/NFS only. For a LONG time (I don’t think this is still the case?) Metrocluster didn’t support FC use cases. Vaughn – I’m fully disclosing – quid pro quo? What other NetApp features/functions have a protocol restriction?

    VAUGHN: The EMC Storage Viewer is similar to the NetApp VSC with a few ‘minor’ differences including… the VSC works on every platform running Data ONTAP, with every storage protocol, and actually changes the configurations of the ESX/ESXi hosts to their optimal values.

    CHAD: EMC Storage Viewer works on EMC CLARiiON, Symmetrix, and Celerra – every EMC platform. There is currently MORE visibility in CLARiiON and Symm than there is with NFS datastores, but the correlation of objects is in there.

    Customer feedback (at least to date) has been that they don’t want the VMware administrator provisioning storage with the exception of triggering snapshot events and triggering VM failover. We’re pushing that, but getting pushback from the customers. I personally think that over time, we need to make storage 100% invisible.

    VAUGHN: The EMC VDI tool is similar to the NetApp RCU with a few ‘minor’ differences including… the EMC tool clones 2 VMs per LUN, only supports Celerra, and iSCSI. The NetApp RCU works on every platform running Data ONTAP, with every storage protocol, can clone any number of VM and datastores, imports directly into View Manager and XenDesktop, and all clones both Virtual Servers along with Virtual Desktops are pre-deduplicated.

    CHAD: First of all, the current VDI tool has no restriction on the number of VMs per datastore. It imports into VMware View Manager. It currently is iSCSI and Celerra only. RCU 2.1 is coming at VMworld, right? The next version of our tool (which is being renamed since it’s not just about VDI) enables: NFS and iSCSI (still Celerra only), VM level offloaded snapshot/clone, datastore level replicas, and some (we’re pushing the same customer boundary here) provisioning actions (similar to the RCU 2.1 functions you discussed and will launch at VMworld). Based on feedback, we may be broadening the function. BTW – if people want to see it (including NetApp folks), you can come by the booth.

    VAUGHN: Is the EMC Replication Manager similar to SMVI? I don’t know. Can it use its off-site backup disk with SRM? Does testing a SRM recovery plan require additional storage or disrupt replication?

    CHAD: I think they are similar. Some of your input is incorrect, some is correct. Can you use off-site replica with SRM? No. Testing an SRM recovery plan does not disrupt replication or require additional storage. So – here’s the ball back to you. Does SMVI handle Exchange? SQL Server? Oracle? (all as VMs if you want?) RM does. Can NetApp do this? Sure, with SME, SMO, SMS, etc. Ironically, many tools, for a single purpose. This is a case where there is a single tool, across EMC products and use cases to manage replicas – snapshots, clones, continuous data protection. We will be demonstrating this at the booth – so if people want to see it (including asking ANY questions – you can come by the booth!)

    VAUGHN: 2. The VSC & RCU are both free and SMVI is aggressively priced!

    CHAD: EMC Storage Viewer, the VDI plugin (and our SRM failback plugin) are free, and Replication Manager is considerably less expensive (in my experience) than SMVI, PARTICULARLY when you need SME, SMO, SMS, SMVI, (ergo you have Exchange, SQL Server, Oracle) etc.

    VAUGHN: 3. The EMC failback plug-in does automate the return of services to the primary site. Come check out the NetApp booth at VMworld.

    CHAD: our intent with the tool was not to replace SRM, and you know as I do what the SRM roadmap is (this is on it). BTW – don’t assume that our SRM failback tool is static, and we won’t be showing OUR update (including multiple platforms). But hey, since we’re talking about NOW (you talk about your futures, we can talk about ours too) – do you have an SRM failback tool that does what our’s does? If we want to compare futures to futures, I can play that game too. We will be doing failback also in the plugin (if people want that). Personally, I think that VMware’s plans for SRM are much more mature, and long term, these will both have less value (as there are very many state transitions that are possible in this scenarios). Working hard to make sure we’re in lockstep with VMware, but if you want to see where we’re going with our SRM failback plugin, again – come by the booth :-)

    CHAD: I’ve also publicly shown here that the next generation of Navisphere, a free software upgrade, actually has the array element manager connect to vCenter APIs directly – this gives the same visibility and end-to-end picture that the EMC Storage Viewer gives to the VMware admin to the storage admin. We’ve done the same in Recoverpoint. I’ve shown how ESX hosts (with all their details) auto-register. We have a compliance and change control plugin. We integrate with the vStorage API for data protection (Avamar, Networker). We integrate with VMsafe APIs (RSA DLP) Not trying to say “ergo, NetApp is bad” (they aren’t) – just that there are areas where each of us is ahead as we battle back and forth around deep VMware integration.

    Echoing your comment, Vaughn… come check out the EMC booth at VMworld for more on this type of functionality! Sorry – I can’t disclose everything here!

    So, on a note of civility – I’ll finish the same way I started at the beginning of the comment thread: “It’s fun to have a strong competitor to parry back and forth with – in the end, it’s good for customers!”

  13. Chad Sakac’s avatar

    Oh one other disclosure – currently there is an SnapManager for Sharepoint. CURRENTLY (again, stressing the CURRENTLY) EMC Replication Manager doesn’t cover integrated Sharepoint use cases.

    I believe that NetApp OEMs and integrates something for SnapManager for Sharepoint (but DON’T consider me an authority on NetApp products – PLEASE).

  14. ricdanger’s avatar

    Any idea on the VSC release date?

  15. Chad Sakac’s avatar

    Correction to my prior disclosure on Sharepoint – I was incorrect. Sharepoint is supported.

    Some details on how Replication Manager provides application-level consistency (similar question was asked on the parallel thread on Vaughn’s blog):

    App consistency is done in ways (and just one more reason why I think they are similar) that are similar to the SnapManager family (warning – I don’t claim to be an expert on SnapManager – and NetApp folks who know more – please correct me)

    Exchange – uses VSS (and includes Ontrack PowerControls for mailbox restore – just like SME with the mailbox recovery option – which is the same Ontrack software).

    SQL Server – uses VDI (SQL Server supports VSS and VDI, VDI remains the preferred method as it covers some scenarios VSS does not)

    Oracle – uses the Hot-Backup mechanism.

    VMware – integrates with the ESX snapshot mechanism in the pre-post array replica mechanism (for IO consistency), which in turn uses the VMware tools (which actually use a bit of EMC code) for VSS handling inside the guest. This guest-level consistency is INDEPENDENT of the application-level consistency mechanisms listed below. For very VERY dense datastores (many VMs) this mechanism can be turned off (exchanging faster jobs for lack of guaranteed VM-level consistency). I believe SMVI does something similar. BTW – not related to Replication Manager, our upcoming update of the vCenter plugin I referred to in this thread that does this at VM-level on NFS datastores only – just like RCU – uses a similar mechanism for VM-level consistency. You can see this at VMworld.

    Sharepoint – uses VSS for correlation across the sharepoint farm, VDI for the SQL Server elements (and includes Ontrack PowerControls for individual item restore – I believe the SnapManager equivalent also OEMs a 3rd party for single item restore).

    That last one (Sharepoint) is in the latest version that also adds NFS datastore support in the VMware use case (previous versions were VMFS only).

    The funny thing is I work hard to keep pretty plugged into things here at EMC, and even I didn’t know about the Sharepoint piece until it crossed my path in Outlook today. This highlights the hazard of these negative FUD campaigns. it’s SO easy to be wrong, so easy to be right – but only in the most transient fashion as these move SO fast.

    That’s my original intent with this comment thread. There’s so much goodness on use-case integration from NetApp, so much from EMC, and it’s moving SO fast – it’s hard to even keep up with your OWN stuff. People shouldn’t listen to what EMC people say about others – and the reverse is also true in the opposite direction with NetApp.

    Customers should push vendors to demonstrate how their solutions work for THEM.

  16. slowe’s avatar

    All,

    ePlus is indeed an HP partner and a NetApp STAR partner–and a Cisco partner, an IBM partner, a Symantec partner, a VMware partner, a Microsoft partner, a Sun partner, and–yes–an EMC partner. First and foremost, however, ePlus is a customer advocate. The right solution is the best solution for the customer, not the best solution for our partners.

    Of course, this is not a company blog, I don’t speak for the company, and the company does not necessarily endorse anything I have to say out here…

  17. Chris Waltham’s avatar

    I downloaded the official VSC from now.netapp.com a few minutes ago. Not sure when it was released, but it’s clearly GA now.

    For those of us that live behind firewalls, I had to open TCP port 7 (yes, really! it’s the TCP echo port) between my workstation & our vCenter Server.

  18. shaun’s avatar

    lol… loving the handbags at dawn stuff! Very funny, seems to be a common thread on many a VMware blog. Scott, just looking at current failback options for SRM here @ PPD prior to the new SRM version that I know now have this option out of the box. As you know we are using NetApp Filers was the failback plug-in ever created or not? Any info on failback greaty appreciated.
    Thanks
    Shaun

  19. slowe’s avatar

    Hi Shaun, good to hear from you. I don’t know if NetApp has a failback plug-in for the FAS storage arrays or their newly-acquired Engenio arrays. Obviously, I’m a bit out of touch with the latest developments in NetApp’s camp. :-)

    Can a reader who is more current with NetApp’s offerings provide some additional information?

Comments are now closed.