Understanding NPIV and NPV

Two technologies that seem to have come to the fore recently are NPIV (N_Port ID Virtualization) and NPV (N_Port Virtualization). Judging just by the names, you might think that these two technologies are the same thing. While they are related in some aspects and can be used in a complementary way, they are quite different. What I’d like to do in this post is help explain these two technologies, how they are different, and how they can be used. I hope to follow up in future posts with some hands-on examples of configuring these technologies on various types of equipment.

First, though, I need to cover some basics. This is unnecessary for those of you that are Fibre Channel experts, but for the rest of the world it might be useful:

  • N_Port: An N_Port is an end node port on the Fibre Channel fabric. This could be an HBA (Host Bus Adapter) in a server or a target port on a storage array.
  • F_Port: An F_Port is a port on a Fibre Channel switch that is connected to an N_Port. So, the port into which a server’s HBA or a storage array’s target port is connected is an F_Port.
  • E_Port: An E_Port is a port on a Fibre Channel switch that is connected to another Fibre Channel switch. The connection between two E_Ports forms an Inter-Switch Link (ISL).

There are other types of ports as well—NL_Port, FL_Port, G_Port, TE_Port—but for the purposes of this discussion these three will get us started. With these definitions in mind, I’ll start by discussing N_Port ID Virtualization (NPIV).

N_Port ID Virtualization (NPIV)

Normally, an N_Port would have a single N_Port_ID associated with it; this N_Port_ID is a 24-bit address assigned by the Fibre Channel switch during the FLOGI process. The N_Port_ID is not the same as the World Wide Port Name (WWPN), although there is typically a one-to-one relationship between WWPN and N_Port_ID. Thus, for any given physical N_Port, there would be exactly one WWPN and one N_Port_ID associated with it.

What NPIV does is allow a single physical N_Port to have multiple WWPNs, and therefore multiple N_Port_IDs, associated with it. After the normal FLOGI process, an NPIV-enabled physical N_Port can subsequently issue additional commands to register more WWPNs and receive more N_Port_IDs (one for each WWPN). The Fibre Channel switch must also support NPIV, as the F_Port on the other end of the link would “see” multiple WWPNs and multiple N_Port_IDs coming from the host and must know how to handle this behavior.

Once all the applicable WWPNs have been registered, each of these WWPNs can be used for SAN zoning or LUN presentation. There is no distinction between the physical WWPN and the virtual WWPNs; they all behave in exactly the same fashion and you can use them in exactly the same ways.

So why might this functionality be useful? Consider a virtualized environment, where you would like to be able to present a LUN via Fibre Channel to a specific virtual machine only:

  • Without NPIV, it’s not possible because the N_Port on the physical host would have only a single WWPN (and N_Port_ID). Any LUNs would have to be zoned and presented to this single WWPN. Because all VMs would be sharing the same WWPN on the one single physical N_Port, any LUNs zoned to this WWPN would be visible to all VMs on that host because all VMs are using the same physical N_Port, same WWPN, and same N_Port_ID.
  • With NPIV, the physical N_Port can register additional WWPNs (and N_Port_IDs). Each VM can have its own WWPN. When you build SAN zones and present LUNs using the VM-specific WWPN, then the LUNs will only be visible to that VM and not to any other VMs.

Virtualization is not the only use case for NPIV, although it is certainly one of the easiest to understand.

<aside>As an aside, it’s interesting to me that VMotion works and is supported with NPIV as long as the RDMs and all associated VMDKs are in the same datastore. Looking at how the physical N_Port has the additional WWPNs and N_Port_IDs associated with it, you’d think that VMotion wouldn’t work. I wonder: does the HBA on the destination ESX/ESXi host have to “re-register” the WWPNs and N_Port_IDs on that physical N_Port as part of the VMotion process?</aside>

Now that I’ve discussed NPIV, I’d like to turn the discussion to N_Port Virtualization (NPV).

N_Port Virtualization

While NPIV is primarily a host-based solution, NPV is primarily a switch-based technology. It is designed to reduce switch management and overhead in larger SAN deployments. Consider that every Fibre Channel switch in a fabric needs a different domain ID, and that the total number of domain IDs in a fabric is limited. In some cases, this limit can be fairly low depending upon the devices attached to the fabric. The problem, though, is that you often need to add Fibre Channel switches in order to scale the size of your fabric. There is therefore an inherent conflict between trying to reduce the overall number of switches in order to keep the domain ID count low while also needing to add switches in order to have a sufficiently high port count. NPV is intended to help address this problem.

NPV introduces a new type of Fibre Channel port, the NP_Port. The NP_Port connects to an F_Port and acts as a proxy for other N_Ports on the NPV-enabled switch. Essentially, the NP_Port “looks” like an NPIV-enabled host to the F_Port on the other end. An NPV-enabled switch will register additional WWPNs (and receive additional N_Port_IDs) via NPIV on behalf of the N_Ports connected to it. The physical N_Ports don’t have any knowledge this is occurring and don’t need any support for it; it’s all handled by the NPV-enabled switch.

Obviously, this means that the upstream Fibre Channel switch must support NPIV, since the NP_Port “looks” and “acts” like an NPIV-enabled host to the upstream F_Port. Additionally, because the NPV-enabled switch now looks like an end host, it no longer needs a domain ID to participate in the Fibre Channel fabric. Using NPV, you can add switches and ports to your fabric without adding domain IDs.

So why is this functionality useful? There is the immediate benefit of being able to scale your Fibre Channel fabric without having to add domain IDs, yes, but in what sorts of environments might this be particularly useful? Consider a blade server environment, like an HP c7000 chassis, where there are Fibre Channel switches in the back of the chassis. By using NPV on these switches, you can add them to your fabric without having to assign a domain ID to each and every one of them.

Here’s another example. Consider an environment where you are mixing different types of Fibre Channel switches and are concerned about interoperability. As long as there is NPIV support, you can enable NPV on one set of switches. The NPV-enabled switches will then act like NPIV-enabled hosts, and you won’t have to worry about connecting E_Ports and creating ISLs between different brands of Fibre Channel switches.

I hope you’ve found this explanation of NPIV and NPV helpful and accurate. In the future, I hope to follow up with some additional posts—including diagrams—that show how these can be used in action. Until then, feel free to post any questions, thoughts, or corrections in the comments below. Your feedback is always welcome!

Disclosure: Some industry contacts at Cisco Systems provided me with information regarding NPV and its operation and behavior, but this post is neither sponsored nor endorsed by anyone.

Tags: , , , , ,

  1. Jason Boche’s avatar

    I still feel VMware’s NPIV implementation is a little weak and there are a lot of hoops to jump through in order to get it to work. It bothers me that both the both the WWPN for (all) the ESX host HBAs and the virutal WWPN for the VM need to be zoned on the fabric switch(es) and presented to the storage controller(s). Not to mention, the slightest disruption can “break” the NPIV chain causing the NPIV port on the fabric switch to go offline and the fabric path reverts back directly to the traditional model. The only use case I’ve been able to come up with is the ability to track fabric/storage utilization on an individual VM basis.

    Fibre channel RDM disk presented to a VM through NPIV shows up as a standard SCSI drive in a VM. I wish that NPIV would present a virtualized HBA with the WWID visible inside the VM. Now that would be something.

    Jas

  2. slowe’s avatar

    Jason,

    You’re correct. VMware’s current implementation of NPIV is definitely weak, error-prone, and less than useful. I’m looking forward to a more powerful implementation of NPIV.

  3. Fabio’s avatar

    Great Article :)

    Thank you!!

  4. Charles’s avatar

    From a historical perspective, it’s worth mentioning that NPIV was initially developed by Emulex, IBM, and McDATA (now Brocade) to provide more scalable access to Fibre Channel storage from mainframe Linux Virtual Machine instances by allowing the assignment of a virtual WWN to each Linux OS partition.

    Since then, Brocade has developed something called “Access Gateway” that leverages NPIV to present open systems Fibre Channel host connections as logical devices to SAN fabrics. This, in effect, eliminates switch-to-switch interoperability problems by allowing servers to seamlessly connect to Brocade, McDATA, Cisco, or other SAN fabrics. Additional benefits are that switches in Access Gateway mode do not consume a Domain ID and do not require zoning (they use the fabric configuration).

    Reference: http://www.brocade.com/downloads/documents/white_papers/Virtualizing_Embedded_Switches_Access_Gateway_WP_01.pdf

  5. slowe’s avatar

    Charles,

    The “Access Gateway” you’re describing sounds exactly like NPV—no domain ID, no zoning, and eliminates switch-to-switch interoperability issues. Is there a difference between Access Gateway and NPV?

  6. Stuart Miniman’s avatar

    Scott,
    It’s worth mentioning that NPV mode is standard configuration for most FCoE solutions today. This helps with the E_port interoperability that you discuss.

  7. Sledge’s avatar

    Just thiinking, if NPV can be ‘ab’used for SAN migrations betweend different vendors……

  8. Mike La Spina’s avatar

    Hi Scott,

    I liked your article, its well written.

    I thought it would be prudent to mention that Cisco’s “NPV mode” is really an F_node port expander fabric with many of the desirable features we need for virtualized environments.

    Regards,

    Mike

  9. Brad Hedlund’s avatar

    Scott,
    I really like your plain spoken writing style. A very accurate post BTW. Keep’em coming! [HIGH FIVE]

    Cheers,
    Brad

  10. Michael Rottlaender’s avatar

    Hi Charles, your Question about “Access Gateway” came also to my mind while reading the article. I think it is the same. The only important prerequisite of our Bladeswitch in Access Gateway mode is the NPIV Support on the external switch and it just does the same as described for NPV.
    @Scott: thanks for the good explanation!

  11. Radhakrishnan’s avatar

    Hi Scott,
    Thanks, It’s a wonderful explanation to easily understand about NPV as well as NPIV. One thing I am not clear how the link established between N port and NP port from the N-port perspective? is it a point to point(FC link) port or loop port?.. if in case of loop port are there any Hub in-between N-port(host) and NP port(switch)?

    Thanks,
    Krish

  12. rickster’s avatar

    i was really excited when i first heard of NPIV … we are using vSCSI on redundant VIO servers in a fully virtualized environment on Power6…but a real show-stopper was …you would have downtime if you have to update your drivers! but your article is really GOOD anyway!

    i think NPIV is perfect for TSM …:) like placing a TSM storage agent on every p6 …

  13. Atul’s avatar

    Great article with such a simple explanation.

  14. slowe’s avatar

    Atul, I’m glad you found the article helpful. Thanks for your feedback!

  15. sc’s avatar

    First, let me say great article.
    Second, I’m just getting into this, and have never used NPIV until now. We’ve always had physical HBA’s with the actual WWN used in zoning, in a 1 to 1 relationship.

    My question is this:
    For NPIV to work properly, should you zone using just the virtual WWN’s or do you have to use both the physical and virtual WWN’s?

    We are using Cisco 9509′s and IBM AIX VIO Power7 machines

  16. slowe’s avatar

    Sc, thanks for the feedback; I’m glad you found the article helpful. Every vendor’s implementation of NPIV support is slightly different; in VMware vSphere, for example, you have to zone to both the physical WWN of the host on which the VM is running as well as the virtual WWN assigned via NPIV. I believe—although I am not an expert with AIX—that this is not the case with other implementations.

  17. sc’s avatar

    Thanks for the rsponse, I’ll try just the virtual WWN’s and see how it goes.

  18. Steve’s avatar

    Awesome~
    Now this is consumable data.
    Thank you Scott!

  19. Lutieri’s avatar

    very nice and helpful article! it was very nice written. congrats and thanks

  20. Narayan’s avatar

    Nice article but i have a question

    can we cascade multiple npv switches ?? what i mean is is it possible to have something like this

    HPc7000 (in NPV) — Cisco MDS (in NPV) — Brocade (in NPIV) ?

  21. slowe’s avatar

    Narayan, yes, I believe that you can nest multiple NPV switches behind NPIV. However, there is a limit on the number of VN_ports that can be supported with NPIV, so you’re limited in how many levels you can nest.

  22. Anugrah’s avatar

    Amazing article. Hope to see some more like this.

  23. Lawrence Foo’s avatar

    Hi Scott,

    I came across an FAQ on Brocade.

    On page 4, it mention that the Brocade Access Gateway is indeed use to reduce the domain ID. Not sure whether it is configured as NPV or NPIV?

    http://www.brocade.com/downloads/documents/faqs/Switches_FAQ_External_8_20_09_Final.pdf

  24. slowe’s avatar

    Lawrence, Access Gateway is using NPV (note that the text indicates that switches running Access Gateway will connect as N_Ports instead of E_Ports). The upstream switch to which a switch running Access Gateway will connect will be using NPIV.

    Hope this helps!

  25. ET’s avatar

    thanks for this great article.
    one question, does this scenario work ?
    FC switch support NPIV – one NPV-enabled switch – HBA support NPIV for VMs.

  26. slowe’s avatar

    ET, if I’m understanding your question correctly, you can have multiple layers of NPIV-NPV working together. There are limits on how many layers/levels you could connect, but the simple example you described should work.

  27. Mohammad’s avatar

    Very plain English to understand the concempt of NPIV and NPV.Thanks Scott.
    Mohammad

  28. Hernán J. Larrea’s avatar

    Hey great article, I’m about to configure some access gateweays and was wondering how all this works.

    Thanks!!
    Hernán

  29. Ashwin’s avatar

    This has been very useful.. Thank you.

  30. Scott’s avatar

    I echo the previous comments. This is a well written, easy to understand article explaining NPV and NPIV, and helped me grasp how IBM POWER VIOS works with NPIV. Thank you.

  31. TopTech’s avatar

    Thanks Scott for your persistent goodness to mankind :)

    Totally off topic question.

    I have 4x8GB host ports on my DS 5100 storage unit. Can I bundle them to my MDS 9148 and get an aggregate of 32GB?

    Because I can’t make up my mind to eliminate MDS and go directly in Nexus 5548UP. This would be one of the deciding factors.

    Thanks

  32. Chris’s avatar

    Scott,
    Great explanation. If your books are as well written as this, I’m looking forward to reading one.

  33. Ram’s avatar

    Reading through this article was more like listening to a colleague talking. Thank you Scott. Thanks to the other comments as well. I understand the concepts because of your simple style of explaining.

  34. uday’s avatar

    Excellent explanation..!!
    Thank you

  35. Tibo’s avatar

    Hi Scott,

    Many thanks for this great article.

    You’re right, in VMware VSphere environment, we need to zone to both the physical WWN of the host on which the VM is running as well as the virtual WWN assigned via NPIV.

    It is a great way to spread the VMs on multiple storage targets. (for load-balancing purpose, or strategic architecture… etc.)

    However… when we lost the path from the virtual WWN to the storage target (i.e. : GBIC failed… etc.), the traffic is definitely sent by the physical WWN, and unfortunately… Even when the failed path is rebuilt, there is no way to use the virtual path…
    :(

  36. Eric’s avatar

    Very nice article. Clear and easy to understand after your explanation.
    thanks!

  37. Kenny’s avatar

    No better way to make things simple!

  38. Kenny’s avatar

    You make things look easier than they are, so for someone having doubts, this clarifies a lot, thanks for the way you explain all…

  39. Comtaq’s avatar

    Really was great . finnaly could undrestand what is these ports , u are really great , man !!!!

  40. Sid’s avatar

    Thanks!! This helped me understand at one shot!!

  41. sisira’s avatar

    Simple and superb! so can we be sure brocade AG and cisco NPV are the same?

  42. Drew Schlussel’s avatar

    Outstanding article. It will be interesting to see how Hyper-V takes advantage of NPIV in Server 12…

  43. Eric K. Miller’s avatar

    I’m trying to figure out whether vSphere 5 will allow a virtual WWN to be presented as a target, not an initiator, using something like NexentaStor in a VM. Has anyone tried this?

  44. Shamim’s avatar

    Hi Charles,
    Brocade Access Gateway is developed based on NPV. brocade switches like 200E, the mode can be changed to ag mode and then it acts as many N port conencted to fewer F port.

    It reduces the number of F port required. As NPV help us achieve this.

  45. slowe’s avatar

    Erik M, to my knowledge this is not something that is possible with vSphere today. If anyone knows differently, please let me know.

  46. Joe Smith’s avatar

    Scott, pardon the stupid question, but one thing I’m confused about is why the two different terminologies? NPIV is when an N_Port registers with the fabric doing a FLOGI and then allows subsequent VMs to log in by converting their FLOGI to an FDISC. This way one N_Port can be assigned multiple WWPNs and N_Port IDs by the fabric.

    And it seems that NPV is the exact same thing. The difference it seems is that when an HBA uses this “trick,” its called NPIV. When a switchport uses this trick its called NPV.

    Am I missing something?

    Thanks

  47. Joe Smith’s avatar

    Scott – disappointed you havent answered my question, but I figured it out. NPV is a CISCO term and Brocade calls it Access Gateway, which to me is less confusing and makes more sense.

    This is the problem with a lot of Blogs out there: Cisco puts out its white papers and uses their terminology and perception of the world, and everyone laps it up and regurgitates it to the rest of the Internet community without realizing what they are doing.

  48. slowe’s avatar

    Joe, there have been several comments (either on this article or others) that confirmed the terminology difference with regard to Brocade’s Access Gateway and Cisco’s NPV. I don’t know which term is more “industry standard,” but I’m glad that you were able to get the information you need.

1 · 2 ·