Rich Brambley (of VM /ETC), Duncan Epping (of Yellow Bricks), and I were having a brief discussion on Twitter late last week about storage multipathing. Rich initiated the discussion with this update, which prompted me to respond and thus started the conversation. Rich and I continued the conversation via e-mail (Twitter isn’t exactly the best medium for that kind of exchange), and prompted by his questions I did some digging. Here’s what I came up with.
This NetApp KB article (NOW login required to view) is kind of what kick-started the entire thing. In that article, NetApp recommends that users set the preferred path for every LUN anytime an ESXi server is rebooted. This behavior seemed curious to me, so I inquired with some industry contacts to see where this recommendation may have originated.
One of my contacts responded that the vicfg-mpath command, part of the Remote CLI used to manage ESXi, is broken in that it lists the target’s WWN not the target’s WWPN. Apparently this will be addressed in a future version of the vicfg-mpath tool. To compound the issue, setting preferred paths from VirtualCenter is apparently not persistent across reboots because the VML LUN name is not used. Hence, when working with ESXi, neither method of setting the preferred path—vicfg-mpath or VirtualCenter—can provide persistent settings. Hence, the recommendation in the NetApp KB article.
But what about VMware ESX? The issue about using VirtualCenter remains, but esxcfg-mpath allows users to use the VML LUN name during the command to ensure that paths are persistent across reboots. In fact, there’s a note in the help screen of esxcfg-mpath (”esxcfg-mpath -h”) that mentions the fact that VMHBA names are not persistent across reboots, and users should use VML LUN names to be sure of consistency.
So there you have it—when using ESXi you’ll want to set the preferred path for a LUN when you reboot, but if you using ESX you can avoid that trouble by using the VML LUN name in the esxcfg-mpath command. As clear as mud, right?
Tags: ESX, ESXi, NetApp, SAN, Storage, Virtualization, VMware
-
I guess it’s better to wait for the vStorage multipathing solution by the vendors and/or the roundrobin to be fully supported.
-
I don’t know about the rest of you, but I don’t want to have to re-configure paths after every reboot.
-
Round-robin with ALUA is the answer for asymmetric active-active arrays. For symmetric active-active arrays that report all paths as Active/Optimized then active-active loadbalancing alone will do the job.
-
You want active/active loadbalancing with ALUA support for asymmetric active/active arrays that reports paths as Active/Optimized, Active/Unoptimized. All of major storage arrays VMware’s been deployed today already provide ALUA support.
For symmetric active/active arrays that report all paths as Active/Optimized round robin or any other load balancing algorithm alone will do.
-
For ESX and Netapp , doesn’t the NetApp host utilities kit for ESX automatically pick the optimized preferred path?
-
Yes. The Host Utilities Kit uses the vml thus preferred/active paths are consistent across reboots. it;s easy and it’s free
-
Wade, as Nick points out, the Host Utilities Kit will ensure that preferred/active paths are consistent. This is one reason to be sure to include the NetApp Host Utilities Kit in your VMware ESX builds.
-
Guys,
Although I’m a little late weighing in on this issue, I never-the-less have some pretty bad news on the most bizarre behavior regarding ESX multi-pathing, Active Preferred Paths and HBA FC Storage. I work at a Shop with nearly 50 ESX 3.5 Update 2 Hosts spread across 7 HA/DRS Clusters Targeted at NetApp Filers, so naturally we have the v3.1 Host Attach kit installed.
Most of the Filers are in Single-Image Mode, but the issue pertains to Partner Mode as well. The issue is that I have to check for Path optimization every week on the Hosts as not only does the Active Path flip from a Primary to a Proxy Target, but the Preferred designation flips as well so the Path never fails back to a Primary Target Path.
If I don’t stay on top of it, the Filers send a boatload of FCP Partner-Path Misconfigured Notifications which is not nice. I’ve been watching this since ESX v3.01, and it can happen on a Host reboot, rescan or, all by itself.
It’s only documented that the Preferred Path designation is persistant across reboots…It doesn’t actually work that way!
-
Has anybody managed to get this to work? I’m using esxcfg-mpath on ESX to change the load balancing policy for 3 LUNs (would be a single one if ESX didn’t have a 2TB limit per LUN), and even though I’m specifying the LUNs using the VML name, the policy still gets reset back to “Fixed” when I reboot.
I guess I can just put this into a script that gets run when ESX boots, but it would be nice if ESX stored this and remembered it (actually, running esxcfg-mpath -r restores the policy I set, but why doesn’t it do it automatically when booting?).
-
Well, it looks like the problem of ESX not remembering the path settings after rebooting is caused (at least in my case) by the fact that it tries to restore those settings before the SW iSCSI has been initialized, so it obviously can’t find any of the LUNs. Is this some kind of hint from VMware telling us to go out and buy a QLogic adapter? I’m already surprised that we can’t specify static targets when using the software initiator, but this is just plain silly. What happens to people running ESXi who can’t run scripts during startup?
Site Sponsors
10 comments
Comments feed for this article
Trackback link: http://blog.scottlowe.org/2008/10/27/quick-note-on-esx-and-esxi-storage-multipathing/trackback/