FCoE

You are currently browsing articles tagged FCoE.

What Does FCoE Have To Do With VM Mobility?

Monday, January 16, 2012 in Storage, Virtualization by slowe | 2 comments

I recently read over Stephen Foskett’s Eight Unresolved Questions About FCoE article, in which he outlines eight topics that should be addressed/prioritized in order to make FCoE, in his words, “truly world-class”.

One of these topics was virtual machine mobility. Here’s what Stephen wrote under the heading “How Do You Handle Virtual Machine Mobility?”:

As I described recently, virtual machine mobility is a major technical challenge for existing networks. The VMware proposal, the VXLAN, seems to be gaining traction right now. But this is only a solution for data networking. How will FCoE SANs handle virtual machine mobility? This remains unresolved as far as I can tell, though Ethernet switch vendors have come up with their own answers. Brocade demonstrated just such a solution at Networking Field Day 2, and I know that others have answers as well. But will there be an interoperable industry solution?

My response to Stephen’s question is pretty simple: VM mobility isn’t a problem for FCoE. VM mobility is a problem for all storage, regardless of protocol.

In my view, tying VM mobility to FCoE is simply buzzword hype. Let’s look at this from a technical perspective—what is so different about FCoE compared to FC, iSCSI, or NFS with regard to VM mobility? Nothing! Regardless of storage protocol, when you move a VM from one data center to another data center, the VM’s storage is still going to sit back in the original data center. The protocol whereby we access that data doesn’t change that fact.

Yes, you can route the IP-based protocols (iSCSI and NFS), which might give you some additional flexibility. But the migrated VM’s storage is still going to sit in the original data center—the only thing you’ve accomplished is introduced some additional latency by adding routers into the mix.

As you can see, VM mobility and storage isn’t an FCoE problem—it’s a storage problem, plain and simple. Protocols can’t fix it. What’s needed to fix it is a fundamental shift in how storage is designed. We need distributed file systems (not just clustered file systems), distributed caching algorithms, greater integration between the block storage and the distributed file systems, greater hypervisor awareness of the underlying storage topology, and more efficient data movement mechanisms. Until then, we can rage all we want about how this protocol or that protocol doesn’t address VM mobility, but we’re simply missing the mark.

Tags: , , ,

Setting up Software FCoE on vSphere 5

Tuesday, November 29, 2011 in Networking, Storage, Virtualization by slowe | 6 comments

One of the features added in vSphere 5 was a software FCoE initiator. This is a feature that has gotten relatively little attention, other than a brief mention here and there. I’m not entirely sure why the software FCoE initiator in vSphere 5 hasn’t gotten more attention, but this past week I had the opportunity to work with the software FCoE initiator in a bit more detail. In this post I’m going to describe in a bit more detail how to set up the software FCoE initiator; in future posts, I hope to be able to provide some performance and troubleshooting information.

Prerequisites

In order to use the software FCoE initiator, you must have a network interface card (NIC) that supports the FCoE offloads. The Intel X520 NICs certainly do; these are the cards that I used in my testing. (Disclaimer: Intel provided me a pair of X520 NICs at no charge to use in this testing.) There might be others, I don’t know; the VMware HCL doesn’t seem to provide an easy way of identifying those NICs that do support the FCoE offloads vs. those NICs that don’t.

If you have a NIC that doesn’t support FCoE offloads, you won’t even be able to add a software FCoE initiator:

If, on the other hand, your NIC does support FCoE offloads, you’ll see the option to add a software FCoE initiator, like this:

Obviously, you’ll want to be sure that your NIC does support FCoE offloads before continuing.

Setting Up Networking for Software FCoE

Before you can actually set up a software FCoE initiator, you’ll first need to configure your networking appropriately. There is one important piece of information you’ll want to be sure to have before you continue: the ID of the VLAN for FCoE traffic.

If you’ve read my article on setting up FCoE on a Nexus 5000, then you know that—on a Nexus 5000 series switch, anyway—you must map the FCoE VSAN to a VLAN (using the fcoe vsan XXX command). You’re going to need that VLAN ID, so make sure that you have it. In a dual fabric setup, you’re going to have two different VLAN IDs. You’ll need both.

Once you have those VLAN IDs, you can then proceed with the networking setup:

  1. Create a VMkernel port on your ESXi host. When creating the VMkernel port, when prompted for VLAN ID specify the FCoE VLAN on fabric A.
  2. I don’t know why (and maybe this will be fixed in a future release), but you’ll need to assign an IP address to each VMkernel port. The IP address will not be used, so just pick an address on a subnet that you don’t use. (Don’t use a subnet that you are using elsewhere on the ESXi host or you could run into IP routing weirdness—remember that the VMkernel uses the first interface it finds for a particular route.)
  3. After you’ve created the VMkernel port, modify the NIC teaming policy to only use the physical uplink that is physically connected to fabric A. This might require a bit of sleuthing and/or using CDP/LLDP data to ensure that you have the right uplink selected.
  4. Create a second VMkernel port on your ESXi host, this time specifying the FCoE VLAN ID for fabric B and modifying the NIC teaming policy
    When creating the VMkernel ports, specify the appropriate VLAN IDs—one for fabric A, one for fabric B. Modify the NIC teaming policy to only use the physical uplink connected to fabric B, again using physical tracing and CDP/LLDP data as needed to verify it.

At this point, you should now have two VMkernel ports, each with separate (unused) IP addresses and configured for separate VLAN IDs and separate physical uplinks. The VLAN IDs and the physical uplinks should correspond to the FCoE fabrics (fabric A and fabric B).

Setting Up the Software FCoE Initiator

With the networking in place, you can actually add the software FCoE initiator using the “Add…” button on the Storage Adapters view of the Configuration tab. This opens up the Add Software Adapter dialog box I showed you earlier, where you can select “Add Software FCoE Adapter” and click OK.

That option opens the following dialog box:

You’ll note that the VLAN ID is 0 and isn’t changeable. I couldn’t find any way to enable this field, and in my testing it proved unnecessary to change it (it worked anyway). Select the appropriate physical uplink and click OK. You’ll do this twice—once for fabric A, and again for fabric B. After you’ve done this twice, you’ll have two software FCoE adapters:

For each of these two software FCoE adapters, you’ll see a node WWN and a port WWN listed. You can use these values in creating the zones and zonesets (see here for more information). First, though, you’ll want to be sure that the software FCoE adapter is actually talking to the fabric correctly; the best way to do that is to use the show flogi data command (on a Nexus 5000; other vendors’ switches will use a slightly different command). The outcome of the show flogi data command will look something like this:

In this screenshot (taken from an SSH session into a Nexus 5010), you can see that a device has logged into vfc1009 on VSAN 300. If you compare the port name and node name, you’ll see that they match one of the software FCoE adapters shown earlier. This is only one of the two fabrics; a matching result was seen from the other fabric, showing that both software FCoE adapters successfully logged into their respective fabrics.

At this point, the rest of the configuration consists of creating the appropriate device aliases (if desired); defining zones and zonesets; and presenting storage from the storage array(s) to the initiator(s). Since these are topics that are fairly well understood and well documented elsewhere, I won’t rehash them again here.

In future posts, I hope to be able to do provide some performance and some troubleshooting information. However, it will likely be early 2012 before I have that opportunity. In the meantime, if anyone has any additional information they’d like to share—including any corrections or clarifications—please feel free to add them to the comments below.

Tags: , , , ,

Technology Short Take #17

Monday, November 28, 2011 in Linux, Macintosh, Microsoft, Networking, Storage, Virtualization by slowe | 1 comment

Welcome to Technology Short Take #17, another of my irregularly-scheduled collections of various data center technology-related links, thoughts, and comments. Here’s hoping you find something useful!

Networking

  • I think it was J Metz of Cisco that posted this to Twitter, but this is a good reference to the various 10 Gigabit Ethernet modules.
  • I’ve spoken quite a bit about stretched clusters and their potential benefits. For an opposing view—especially regarding the use of stretched clusters as a disaster avoidance solution—check out this article. It’s a nice counterpoint, especially from the perspective of the network.
  • Anyone know anything about sFlow?
  • Here’s a good post on VXLAN that has some useful information. I’d just like to point out that VXLAN is really only intended to address Layer 2 communications “within” a vApp or a collection of VMs (perhaps a single organization’s VMs), and doesn’t do anything to address Layer 3 routing/accessibility for clients (or “consumers”) attempting to connect to those systems. For that, you’ll still need—at least today—technologies like OTV, LISP, and others.
  • A quick thought that I’m still exploring: what’s the impact of OpenFlow on technologies like VXLAN, NVGRE, and others? Does SDN eliminate the need for these technologies? I’d be curious to hear your thoughts.

Servers/Operating Systems

  • If you’ve adopted Mac OS X Lion 10.7, you might have noticed some problems connecting to older servers/NAS devices running AFP (AppleTalk Filing Protocol). This Apple KB article describes a fix. Although I’m running Snow Leopard now, I was running Lion on a new MacBook Pro and I can attest that this fix does work.
  • This Microsoft KB article describes how to extend the Windows Server 2008 evaluation period. I’ve found this useful for Windows Server 2008 instances in the lab that I need for longer 60 days but that I don’t necessarily want to activate (because they are transient).

Storage

  • Jason Boche blogged about a way to remove stubborn hosts from Unisphere. I’ve personally never seen this problem, but it’s nice to know how to address it should it occur.
  • Who would’ve thought that an HDD could serve as a cache for an SSD? Shouldn’t it be the other way around? Normally, that would probably be the case, but as described here there are certain instances and ways in which using an HDD as a cache for an SSD can improve performance.
  • Scott Drummonds wraps up his 3 part series on flash storage in part 3, which contains information on sizing flash storage. If you haven’t been reading this series, I’d recommend giving it a look.
  • Scott also weighs in on the flash as SSD vs. flash on PCIe discussion. I’d have to agree that interfaces are important, and the ability of the industry to successfully leverage flash on the PCIe bus is (today) fairly limited.
  • Henri updated his VNXe blog series with a new post on EFD and RR performance. No real surprises here, although I do have one question for Henri: is that your car in the blog header?

Virtualization

  • Interested in setting up host-only networking on VMware Fusion 4? Here’s a quick guide.
  • Kenneth Bell offers up some quick guidelines on when to deploy MCS versus PVS in a XenDesktop environment. MCS vs. PVS is a topic of some discussion on the vSpecialist mailing list as they have very different IOPs requirements and I/O profiles.
  • Speaking of VDI, Andre Leibovici has two articles that I wanted to point out. First, Andre does a deep dive on Video RAM in VMware View 5 with 3D; this has tons of good information that is useful for a VDI architect. (The note about the extra .VSWP overhead, for example, is priceless.) Andre also has a good piece on VDI and Microsoft Outlook that’s worth reading, laying out the various options for Outlook-related storage. If you want to be good at VDI, Andre is definitely a great resource to follow.
  • Running Linux in your VMware vSphere environment? If you haven’t already, check out Bob Plankers’ Linux Virtual Machine Tuning Guide for some useful tips on tuning Linux in a VM.
  • Seen this page?
  • You’ve probably already heard about Nick Weaver’s new “Uber” tool, a new VM alignment tool called UBERAlign. This tool is designed to address VM alignment, a problem with how guest file systems are formatted within a VMDK. For more information, see Nick’s announcement here.
  • Don’t disable DRS when you’re using vCloud Director. It’s as simple as that. (If you want to know why, read Chris Colotti’s post.)
  • Here’s a couple of great diagrams by Hany Michael on vCloud Director management pods (both public cloud and private cloud management).
  • People automatically assume that “virtualization” means consolidating multiple workloads onto a single physical server. However, virtualization is really just a layer of abstraction, and that layer of abstraction can be used in a variety of ways. I spoke about this in early 2010. This article (written back in March of 2011) by Brad Hedlund picks up on that theme to show another way that virtualization—or, as he calls it, “inverse virtualization”—can be applied to today’s data centers and today’s applications.
  • My discussion on the end of the infrastructure engineer generated some conversations, which is good. One of the responses was by Aaron Sweemer in which he discusses the new (but not new) “data layer” and expresses a need for infrastructure engineers to be aware of this data layer. I’d agree with a general need for all infrastructure engineers to be aware of the layers above them in the stack; I’m just not convinced that we all need to become application developers.
  • Here’s a great post by William Lam on the missing piece to creating your own vSEL cloud. I’ll tell you, William blogs some of the coolest stuff…I wish I could dig in as deep as he does in some of this stuff.
  • Here’s a nice look at the use of PowerCLI to help with the automation of DRS rules.
  • One of my projects for the upcoming year is becoming more knowledgeable and conversant with the open source Xen hypervisor and Citrix XenServer. I think that the XenServer Design Handbook is going to be a useful resource for that project.
  • Interested in more information on deploying Oracle databases on vSphere? Michael Webster, aka @vcdxnz001 on Twitter, has a lengthy article with lots of information regarding Oracle on vSphere.
  • This VMware KB article describes how to enable centralized logging for vCloud Director cells. This is particularly important for HA environments, where VMware’s recommended HA strategy involves the use of multiple vCD cells.

I guess I should wrap it up here, before this post gets any longer. Thanks for reading this far, and feel free to speak up in the comments!

Tags: , , , , , , , , , , , , , ,

Technology Short Take #15

Monday, October 3, 2011 in Networking, Storage, Virtualization by slowe | 4 comments

Welcome to Technology Short Take #15, the latest in my irregular series of posts on various articles and links on networking, servers, storage, and virtualization—everything a growing data center engineer needs!

Networking

My thoughts this time around are pretty heavily focused on VXLAN, which continues to get lots of attention. I talked about posting a dissection of VXLAN, but I have failed miserably; fortunately, other people smarter than me have stepped up to the plate. Here are a few VXLAN-related posts and articles I’ve found over the last couple of weeks:

  • There is a three-part series over at Coding Relic that does a great job of explaining VXLAN, the components of VXLAN, and how it works. Here are the links to the series: part 1, part 2, and part 3. One note of clarification: in part 3 of the series, Denny talks about a VTEP gateway. Right now, the VTEP gateway is the server itself; anytime a packet on a VXLAN-enabled network leaves the physical server to go to a different physical server, it will be VXLAN-encapsulated. It won’t be decapsulated until it hits the destination VTEP (the ESXi server hosting the destination VM). If (when?) VXLAN awareness hits physical switches, then the possibility of a VTEP gateway existing outside the server exists. Personally, it kind of makes sense—to me, at least—to build VTEP gateway functionality into vShield Edge.
  • Some people aren’t quite so enamored with VXLAN; one such individual is Greg Ferro. I respect Greg a great deal, so it was interesting to me to read his article on why VXLAN is “full of fail”. Some of his comments are only slightly related to VXLAN (the rant over IEEE vs. IETF, for example), but Greg’s comment about VMware building a new standard instead of “leveraging the value of networking infrastructure” echoes some of my own thoughts. I understand that VXLAN accomplishes things that existing standards apparently do not, but was a new standard really necessary?
  • Omar Sultan of Cisco took the time to compile some questions and answers about VXLAN. One thing that is made more clear—for me, at least—in Omar’s post is the fact that VXLAN doesn’t address connectivity to the vApps from the “outside” world. While VXLAN provides a logical isolated network segment that can span multiple Layer 3 networks and allow applications to communicate with each other, VXLAN doesn’t address the Layer 3 addressing that must exist outside the VXLAN tunnel. In fact, in my discussions with some of the IETF draft authors at VMworld, they indicated that VXLAN would require a NAT device or a DNS update in order to address changes in externally-accessible applications. This, by the way, is why you’ll still need technologies like OTV and LISP (or their equivalents); see this post for more information on how VXLAN, OTV, and LISP are complementary. If I’m wrong, please feel free to correct me.
  • In case you’re still unclear about the key problem that VXLAN attempts to address, this quote from Ivan Pepelnjak might help (the full article is here):

    VXLAN tries to solve a very specific IaaS infrastructure problem: replace VLANs with something that might scale better. In a massive multi-tenant data center having thousands of customers, each one asking for multiple isolated IP subnets, you quickly run out of VLANs.

  • Finally, you might find this PDF helpful. Ignore the first 13 slides or so; they’re marketing fluff, to be honest. However, the remainder of the slides have some useful information on VXLAN and how it’s expected to be implemented.

Servers

I didn’t really stumble across anything strictly server hardware-related; either I’m just not plugged into the right resources (anyone want to make some recommendations?) or it was just a quiet period. I’ll assume it was the former.

Storage

Virtualization

  • Did you see this post about new network simulation functionality in VMware Workstation 8?
  • Here’s a good walk-through on setting up vMotion across multiple network interfaces.
  • VMware vSphere Design co-author Maish Saidel-Keesing has a post here on how to approximate the functionality of netstat on ESXi.
  • William Lam has a “how to” on installing the VMware VSA with running VMs.
  • Fellow vSpecialist Andre Leibovici did a write-up on a proof of concept that the vSpecialists did for a customer involving Vblock, VPLEX, and VDI. This was a pretty cool use case, in my opinion, and worth having a look if you need to design a highly available environment.
  • Thinking about playing with vShield 5? That’s a good idea, but check here to learn from the mistakes of others first. You’ll thank me later.
  • The question of defragmenting guest OS disks has come up again and again; here’s the latest take from Cormac Hogan of VMware. He makes some great points, but I suspect that this question is still far from settled.

It’s time to wrap up now; I hope that you found something useful. As always, thanks for reading! Feel free to share your views or thoughts in the comments below.

Tags: , , , , , , , , ,

Technology Short Take #14

Monday, September 12, 2011 in Networking, Storage, Virtualization by slowe | No comments

Welcome to Technology Short Take #14, another collection of links and tidbits of information I’ve gathered over the last few weeks. Let’s dive right in!

Networking

Much of my focus in the networking space recently has been around virtualization-centric initiatives, so the items on this list might seem a bit skewed in that direction. Sorry!

  • I’ve been doing some reading on 802.1Qbg (Edge Virtual Bridging). I still have a long way to go, but I think that I’m starting to better understand this draft and the problem(s) it’s attempting to address. As usual when I’m dealing with networking-related technologies, especially data center-focused networking technologies, I’m finding some of Ivan Pepelnjak’s articles useful. For example, he wrote an article on how EVB should ease VLAN configuration pains; this article is helpful in translating the terms the IEEE uses (like “virtual station interface” and “EVB station”) into terms virtualization-friendly folks like me can understand (like “vNIC” and “Hypervisor supporting EVB”, respectively). Ivan also provides a rough comparison of 802.1Qbh/FEX and 802.1Qbg, which I also found helpful in better understanding both technologies. There is still much that I want/need to understand, such as how 802.1Qbg affects or is affected by VXLAN, the recent darling of the cloud networking space.
  • Speaking of VXLAN, a number of articles have emerged since the announcement of VXLAN last week at VMworld 2011 in Las Vegas. Jon Oltsik of Network World called it “Cloud Network Segmentation at Layer 2.5″, which is catchy but doesn’t really delve into the details of VXLAN and how it plays into/with related data center protocols. Of course, there’s also the obligatory VMware post on the technology, talking about how great it is—naturally—but failing to again provide substantive information on the relationships between VXLAN and other, related data center technologies. If anything, Allwyn’s post made VXLAN seem even more proprietary and linked to vCloud Director and vShield Edge than I’d understood it to be. Fortunately, Ivan weighed in on the proposed new standard and also provided some information on the relationship between VXLAN, OTV, and LISP. I’m still digging into VXLAN myself and I plan to post an article within the next week or so (I’ve been a bit busy with moving halfway across the United States).
  • Ivan also has a post with more details on the Brocade VCS fabric load-balancing behaviors that’s worth having a look.

Servers

  • This article on AES-NI in the newer Intel CPUs is a great look at the benefits of adding symmetric encryption support at the CPU level. Almost makes me want to go out and buy a new MacBook Pro so that I could use File Vault 2 with hardware encryption support…

Storage

  • One cool find recently is this series of “hands-on” posts by Henri Hamalainen (aka @henriwithani) on the EMC VNXe 3300. I had the pleasure of meeting Henri in person at VMworld this year, and he mentioned that he’d started a series of posts on the VNXe 3300. His posts are here: part 1, part 2, part 3, part 4, part 5, and part 6. (Part 7 hasn’t yet been written.)
  • There’s been quite a hubbub going on in the FCoE space, with so many articles flashing back and forth from various contributors I’m still having a hard time deciphering all of it. From what I can tell, it all started with an article by a VP at Juniper about FCoE over TRILL. That sparked Ivan Pepelnjak to coin some new terms: “dense-mode FCoE” (in which FCFs exist at every hop) and “sparse-mode FCoE” (in which LAN switches may forward FCoE frames without any FCoE awareness). That, in turn, sparked an article by Tony Bourke in which he creates more new modes of operation: single hop FCoE (SHFCoE), dense-mode FCoE (DMFCoE), and sparse-mode FCoE (SMFCoE). A fantastic (and very informative) discussion ensued in the comments to that article and the follow-up article. Ivan also responded to Tony’s post as well with a post on FCoE network elements classification. I’m not sure that all the contributors ever came to a consensus, but you’ll learn a lot about FCoE and related technologies just following along, that’s for sure.
  • By the way, this transcript of questions and answers from a live FCoE webcast has some great information buried in it as well.
  • This is an older article, but Stephen Foskett does a good job with discussing FCoE vs. iSCSI. Like so many other IT-related decisions, it’s not an “either-or” discussion—it’s about finding the right tool to do the job.
  • This article provides one suggestion for handling zoning with multiple storage arrays, and provides some good information on EMC CLARiiON/VNX arrays in the process.

Virtualization

  • The idea of stretched clusters and interconnecting data centers continues to be an idea many people are interested in exploring. Rawley Burbridge, of IBM, discusses how this might be done using IBM SVC and VMware vSphere in this three-part series (part 1, part 2, and part 3).
  • Kendrick Coleman, in conjunction with a collection of folks from both VMware and VCE, recently published an article on design considerations for vCloud Director on a Vblock. I haven’t yet read the full document, primarily because it appears to require a Facebook login in order to download. (I don’t use Facebook.)
  • Andre Leibovici—who I had the pleasure of meeting in person at VMworld—has an article on how to modify the Windows Registry settings (or apply Group Policy) for the VMware View Client in order to integrate self-service password reset.
  • The VMware vCloud Architecture Toolkit (vCAT) version 2.0 is now available; get it here (via Beaker).
  • Forbes Guthrie—the lead author with whom I worked on VMware vSphere Design, published earlier this year—posted some great 10Gb Ethernet-related information from VMworld session SPO3040.
  • This is a slightly older post from Hany Michael, but a good one nevertheless; he posts information on how to publish the vCloud Director portal on the Internet.

I guess that will wrap things up for this time around. Thanks for reading, and I hope that you found something useful in this varied mix of links. Feel free to share more in the comments below!

Tags: , , , , ,

Technology Short Take #12: Networking Edition

Monday, May 16, 2011 in Networking by slowe | 4 comments

Now that I’ve published the Storage Edition of Technology Short Take #12, it’s time for the Networking Edition. Enjoy, and I hope you find something useful!

  • Ron Fuller’s ongoing deep dive series on OTV (Overlay Transport Virtualization) has been great for me. I knew about the basics of OTV, but Ron’s articles really gave me a better understanding of the technology. Check out the first three articles here: part 1, part 2, and part 3.
  • Similarly, Joe Onisick’s two-part (so far) series on inter-fabric traffic on Cisco UCS is very helpful and informative as well. There are definitely some design considerations that come about from deploying VMware vSphere on Cisco UCS. Have a look at Joe’s articles on his site (Part 1 and Part 2).
  • Kurt Bales’ article on innovation vs. standardization is a great read. The key, in my mind, is innovating (releasing “non-standard” stuff) while also working with the broader community to help encourage standardization around that innovation.
  • Here’s another great multi-part series, this time from Brian Feeny on NX-OS (part 1 here, and part 2 here). Brian exposes some pretty interesting stuff in the NX-OS kickstart and system image.
  • I’ve discussed LISP a little bit here and there, but Greg Ferro reminds us that LISP isn’t a “done deal.”
  • J Metz wrote a good article on the interaction (or lack thereof, depending on how you look at it) between FCoE and TRILL.
  • For a non-networking geek like me, some great resources to become more familiar with TRILL might include this comparison of 802.1aq and TRILL, this explanation from RFC 5556, this discussion of TRILL-STP integration, or this explanation using north-south/east-west terminology. Brad Hedlund’s TRILL write-up from a year ago is also helpful, in my opinion. All of these are great resources, in my mind.
  • And as if understanding TRILL, or the differences between TRILL and FabricPath weren’t enough (see this discussion by Ron Fuller on the topic), then we have 802.1aq Shortest Path Bridging (SPB) thrown in for good measure, too. If it’s hard for networking experts to keep up with all these developments, think about the non-networking folks like me!
  • Ivan Pepelnjak’s examination of vCDNI-based private networks via Wireshark traces exposes some notable scalability limitations. It makes me wonder, as Ivan does, why VMware chose to use this method versus something more widely used and well-proven, like MPLS? And isn’t there an existing standard for MAC-in-MAC encapsulation? Why didn’t VMware use that existing standard? Perhaps it goes back to innovation vs. standardization again?
  • If you’re interested in more details on vCDNI networks, check out this post by Kamau Wanguhu.
  • Omar Sultan of Cisco has a quick post on OpenFlow and Cisco’s participation here.
  • Jake Howering of Cisco (nice guy, met him a few times) has a write-up on an interesting combination of technologies: ACE (load balancing) plus OTV (data center interconnect), with a small dash of VMware vCenter API integration.

I think that’s going to do it for this Networking Edition of Technology Short Take #12. I’d love to hear your thoughts, suggestions, or corrections about anything I’ve mentioned here, so feel free to join the discussion in the comments. Thanks for reading!

Tags: , , , ,

Technology Short Take #12: Storage Edition

Sunday, May 8, 2011 in Storage by slowe | 4 comments

I’m so far behind in my technology reading that I have this massive list of blog posts and links that I would normally put into an issue of Technology Short Takes. However, people are already “complaining” that my Short Takes aren’t all that short. To keep from overwhelming people, I’m breaking Technology Short Take #12 into three editions: Virtualization, Storage, and Networking.

Here’s the “Storage Edition” of Technology Short Take #12!

  • When planning storage details for your vSphere implementation, be sure to keep block size in mind. Duncan Epping’s post on the performance impact of the different datamovers in a Storage vMotion operation should bring to light why this is an important storage detail to remember. (And read this post if you need more info on the different datamovers.)
  • Richard Anderson of EMC (aka @storagesavvy) posted a “what if” about using cloud storage as a buffer with thin provisioning and FAST VP. It’s an interesting idea, and one that will probably see greater attention moving forward.
  • Richard also shared some real-world results on the benefits of using FAST Cache and FAST VP on a NS-480 array.
  • Interested in using OpenFiler as an FC target? Have a look here.
  • Nigel Poulton posted an analysis of EMC’s recent entry in the SPC benchmarketing wars in which he compares storage benchmarking to Formula 1 racing. I can see and understand his analogy, and to a certain extent he has a valid point. On the other hand, it doesn’t make sense to submit a more “mainstream” configuration if it’s a performance benchmark; to use Nigel’s analogy, that would be like driving your mini-van in a Formula 1 race. Yes, the mini-van is probably more applicable and useful to a wider audience, but a Formula 1 race is a “performance benchmark,” is it not? Anyway, I don’t know why certain configurations were or were not submitted; that’s for far more important people than me to determine.
  • Vijay (aka @veverything on Twitter) has a good deep dive on EMC storage pools as implemented on the CLARiiON and VNX arrays.
  • Erik Smith has a couple of great FCoE-focused blog posts, first on directly connecting to an FCoE target and then on VE_Ports and multihop FCoE. Both of these posts are in-depth technical articles that are, in my opinion, well worth reading.
  • Brian Norris posted about some limitations with certain FLARE 30 features when used in conjunction with Celerra (DART 6.0). I know that at least one of these limitations—the support for FAST VP on LUNs used by Celerra—are addressed in the VNX arrays.
  • Brian also recently posted some good information on a potential login issue with Unisphere; this is caused by SSL certificates that are generated with future dates.
  • J Metz of Cisco also has a couple of great FCoE-focused posts. In To Tell the Truth: Multihop FCoE, J covers in great detail the various topology options and the differences in each topology. Then, in his post on director-class multihop FCoE, J discusses the products that implement multihop FCoE for Cisco.
  • If you’ve never used EMC’s VSI (Virtual Storage Integrator) plug-in for vCenter Server, have a look at Mike Laverick’s write-up.

OK, that does it for the Storage Edition of Technology Short Take #12. Check back in a couple of days for the Networking Edition of Technology Short Take 12.

Tags: , , , ,

Cisco’s Data Center Launch

Wednesday, March 30, 2011 in Networking, Virtualization by slowe | 19 comments

Cisco had a pretty major product/technology launch today. I was briefed on the product launch some time ago—one of the perks of being a “leading blogger”—and so it was great to see this stuff finally come to light. I haven’t had the chance to read some of the other blog posts on the topic (I know there have been several), so I apologize for any duplication.

First, let’s take a quick look at the “new product” news included in the launch:

  • Cisco is launching the Nexus 3000, a new member of the Nexus family. It’s a high density, ultra-low latency 1RU switch that supports both 1GbE as well as 10GbE. All ports are wire rate and non-blocking. At first glance, this seemed to be a bit of a conflict with some of the Catalyst switches (like the 4948, perhaps?), but I think the distinction here is the low latency target (approximately 1 usec port to port). It also makes sense if customers are interested in standardizing on switches running NX-OS instead of a mix of NX-OS and IOS.
  • Cisco is also launching new Nexus 5500 series switches, the Nexus 5548UP and Nexus 5596UP. These are very much like the other members of the Nexus 5500 family, except that the “UP” designation indicates that they will support “Unified Ports”. Unified Ports are ports that can support either Fibre Channel (supporting up to 8Gbps FC), Fibre Channel over Ethernet (at 10Gbps), or Ethernet at 1Gbps. You will need to swap SFPs (for now, at least) and reconfigure the port, and you must cycle/reset the module in which the ports reside in order to switch personality. (It’s my understanding that future improvements will require only cycling the specific port being switched.) Personally, I think the Unified Port functionality is extremely useful and I’m glad to see Cisco deliver it.
  • Cisco is also announced a new C-series rack mount server, the C260 M2. This is a dual-socket server running Intel Westmere CPUs, supporting up to 64 DIMM slots (using Cisco’s Memory Expansion technology) and supporting up to 16 HDDs or SSDs. It’s a pretty beefy server. From a virtualization perspective, the expanded memory footprint is pretty useful, but I’m not so sure that the 16 drive bays is much of a selling point. I’d love to hear others’ thoughts on that matter.
  • Also in the new product lineup are some new Catalyst 6500 modules, an ACE-30 module, an ASA service module, and a ES-40 40Gb DCI module for high-speed data center interconnects.
  • Finally, Cisco is launching an FCoE module for the MDS, which will allow customers to integrate FCoE into their networks while preserving their investment in the MDS infrastructure. On its own, this is interesting, but when coupled with other announcements…well, read on for more details.

However, the launch isn’t just about new products; if it were, it would be pretty boring, to be honest. The news included in the launch about the expansion of existing technologies is, in my mind, far more significant:

  • First, Cisco is stepping away from the “VN-Tag” moniker and focusing instead of the IEEE standardization of that functionality, a la 802.1Qbh. That’s a good move, in my opinion; they’re also going to be adding 802.1Qbg support in the future. Not such a good move (again, in my opinion) is the rebranding effort around VN-Tag and VN-Link, which are now going to be called Adapter FEX and VM-FEX. I don’t know that this new naming convention is going to be any less confusing or any more clear than the previous strategy. It is a bit more technically accurate, at least, since—if you read my post on network interface virtualization—you know that the Cisco Virtual Interface Controller (VIC), aka “Palo”, is really just a fabric extender (FEX) in a mezzanine card form factor. So, calling this Adapter FEX is a bit more accurate. VM-FEX….well, this still isn’t so clear. If any Cisco folks want to help clear things up, feel free to jump in by posting a comment to this post.
  • Cisco is adding FCoE support to the Nexus 7000 via the F1 line cards. This brings FCoE support directly to the Nexus 7000, which is a move that many expected. Along with the FCoE support, Cisco is also introducing the idea of a new type of virtual device context (VDC), the storage VDC. A storage VDC will process all FCoE traffic passing through the switch, in a completely separate fashion from LAN traffic (which would run in a separate LAN VDC). At least initially, Cisco will require the use of a storage VDC to use FCoE with the Nexus 7000; that might change over time. As with the introduction of the FCoE module for the MDS 9500, this news is interesting, but is really only useful in conjunction with other announcements. Specifically…(drum roll please)
  • In perhaps the biggest announcement of the event, Cisco is now supporting full multi-hop FCoE topologies. As I understand it, there will be a new release of NX-OS that will support the creation of VE_ports and enable multi-hop FCoE topologies (up to 7 hops supported). This functionality will exist not only on the Nexus 5000/5500, but also on the Nexus 7000 and on the FCoE module of the MDS 9500. As far as I know, this also means that any Nexus 2000 series fabric extenders that support FCoE will also now support multi-hop FCoE via their upstream Nexus switch. Cisco is going to require dedicated FCoE links between switches, at least at first, and as I mentioned earlier will require the use of a storage VDC on the Nexus 7000. I believe that this is probably the most significant announcement being made, and when taken together with other FCoE-related announcements like the FCoE module on the MDS 9500 and FCoE support on the Nexus 7000, opens up lots of new possibilities for FCoE and Unified Fabric in the data center. I, for one, am really excited to dive even deeper into the design considerations around multi-hop FCoE and Unified Fabric. Any network gearheads interested in doing a brain dump on multi-hop FCoE for me?
  • Equally important in the long-term but (apparently) not making a big impact immediately is LISP (Location ID/Separation Protocol). I’ve been talking LISP with Cisco for a while now (as I suspect many of you have), so it’s good to see the official announcement. Lots of people confuse the purpose/role of LISP when compared to OTV; they are both equally important but in very different ways. Further, LISP does not replace OTV (or vice versa). I’ll probably try my hand at a separate blog post to specifically discuss these two technologies and how the plan is for them to work hand-in-hand with each other for workload mobility. For now, it should suffice to say that OTV addresses Layer 2 connectivity between data centers while LISP helps the rest of the network more efficiently understand and adapt to the Layer 2 connectivity between data centers. Both are necessary.

There were a few other tidbits—like the ability to run up to 6 Nexus 1000V VSMs on the Nexus 1010, or support for the Virtual Security Gateway (VSG) on the Nexus 1010—but this covers the bulk of the information.

All in all, it’s exciting to see some of these technologies coming to light, and I’m really excited to see how the data center is going to evolve over the next couple of years. It’s a great time to be in this industry, especially if you’re a glutton for learning new technologies like me!

As always, I invite and encourage discussion, so if I’ve left something important out or if I’ve misrepresented something, please speak up in the comments (courteously and with full disclosure, where applicable). Thanks!

UPDATE: In response to information gathered via Twitter (I’m @scott_lowe), I’ve updated the information on the 5584UP/5596UP and Unified Ports functionality. Thanks for the great discussion!

Tags: , , , ,

Technology Short Take #9

Wednesday, December 29, 2010 in Networking, Storage, Virtualization by slowe | 17 comments

Welcome to Technology Short Take #9, the last Technology Short Take for 2010. In this Short Take, I have a collection of links and articles about networking, servers, storage, and virtualization. Of note this time around: some great DCI links, multi-hop FCoE finally arrives (sort of), a few XenServer/XenDesktop/XenApp links, and NTFS defragmentation in the virtualized data center. Here you go—enjoy!

Networking

  • Brad Hedlund has a great post discussing Nexus 7000 connectivity options for Cisco UCS. I’ll include it in this section since it focuses more on the networking aspect rather than UCS. I haven’t had the time to read the full PDF linked in Brad’s article, but the other topics he discusses in the post—FabricPath networks, F1 vs. M1 linecards, and FCoE connectivity—are great discussions. I’m confident the PDF is equally informative and useful.
  • This UCS-specific post describes how northbound Ethernet frame flows work. Very useful information, especially if you are new to Cisco UCS.
  • Data Center Interconnect (DCI) is a hot topic these days considering that it is a key component of long-distance vMotion (aka vMotion at distance). Ron Fuller (who I had the pleasure of meeting in person a few weeks ago, great guy), aka @ccie5851 on Twitter and one of the authors of NX-OS and Cisco Nexus Switching: Next-Generation Data Center Architectures (available from Amazon), wrote a series on the various available DCI options such as EoMPLS, VPLS, A-VPLS, and OTV. If you’re considering DCI—especially if you’re a non-networking guy and need to understand the impact of DCI on the networking team—this series of articles is worth reading. Part 1 is here and part 2 is here.
  • And while we are discussing DCI, here’s a brief post by Ivan Pepelnjak about DCI encryption.
  • This post was a bit deep for me (I’m still getting up to speed on the more advanced networking topics), but it seemed interesting nevertheless. It’s a how-to on redistributing routes between VRFs.
  • Optical or twinax? That’s the question discussed by Erik Smith in this post.
  • Greg Ferro also discusses cabling in this post on cabling for 40 Gigabit and 100 Gigabit Ethernet.

Servers

  • As you probably already know, Cisco released version 1.4 of the UCS firmware. This version incorporates a number of significant new features: support for direct-connected storage, support for incorporating C-Series rack-mount servers into UCS Manager (via a Nexus 2000 series fabric extender connected to the UCS 61×0 fabric interconnects), and more. Jeremy Waldrop has a brief write-up that lists a few of his favorite new features.
  • This next post might only be of interest to partners and resellers, but having been in that space before joining EMC I fully understand the usefulness of having a list of references and case studies. In this case, it’s a list of case studies and references for Cisco UCS, courtesy of M. Sean McGee (who I hope to meet in person in St. Louis in just a couple of weeks).

Storage

Virtualization

  • Using XenServer and need to support multicast? Look to this article for the information on how to enable multicast with XenServer.
  • A couple of colleagues over at Intel (I worked with Brian on one of his earlier white papers) forwarded me the link to their latest Ethernet virtualization white paper, which discusses the use of 10 Gigabit Ethernet with VMware vSphere. You can find the link to the latest paper in this blog entry.
  • Bhumik Patel has a good write-up on the “behind-the-scenes” technical details that went into the Cisco-Citrix design guides around XenDesktop/XenApp on Cisco UCS. Bhumik provides the details on things like how many blades were using in the testing, what the configuration of the blades was, and what sort of testing was performed.
  • Thinking of carving your storage up into guest OS datastores for VMware? You might want to read this first for some additional considerations.
  • I know that this has seen some traffic already, but I did want to point out Eric Sloof’s post on the Xenoss XenPack for ESXTOP. I haven’t had the opportunity to use it yet, but would certainly love to hear from anyone who has. Feel free to share your experiences in the comments.
  • As is usually the case, Duncan Epping has had some great posts over the last few weeks. His post on shares set on resource pools highlights the need to adjust the shares value (and other resource constraints) based on the contents of the pool, something that many people forget to do. He also provides a breakdown of the various vCenter memory statistics, and discusses an issue with binding a Provider vDC directly to an ESX/ESXi host.
  • PowerCLI 4.1.1 has some improvements for VMware HA clusters which are detailed in this VMware vSphere PowerCLI Blog entry.
  • Frank Denneman has three articles which have caught my attention over the last few weeks. (All his stuff is good, by the way.) First is his two-part series on the impact of oversized virtual machines (part 1 and part 2). Some of the impacts Frank discusses include memory overhead, NUMA architectures, shares values, HA slot size, and DRS initial placement. Apparently a part 3 is planned but hasn’t been published yet (see some of the comments in part 2). Also worth a read is Frank’s recent post on node interleaving.
  • Here’s yet another tool in your toolkit to help with the transition to ESXi: a post by Gabe on setting logfile location, swap file, SNMP, and vmkcore partition in ESXi.
  • Here’s another guide to creating a bootable ESXi USB stick (on Windows). Here’s my guide to doing it on Mac OS X.
  • Jon Owings had an idea about dynamic cluster pooling. This is a pretty cool idea—perhaps we can get VMware to include it in the next major release of vSphere?
  • Irritated that VMware disabled copy-and-paste between the VM and the vSphere Client in vSphere 4.1? Fix it with these instructions.
  • This white paper on configuration examples and troubleshooting for VMDirectPath was recently released by VMware. I haven’t had the chance to read it yet, but it’s on my “to read” list. I’ll just have a look at that in my copious free time…
  • David Marshall has posted on VMblog.com a two-part series on how NTFS causes I/O bottlenecks on virtual machines (part 1 and part 2). It’s a great review of NTFS and how Microsoft’s file system works. Ultimately, the author of the posts (Robert Nolan) sets the readers up for the need for NTFS defragmentation in order to reduce the I/O load on virtualized infrastructures. While I do agree with Mr. Nolan’s findings in that regard, there are other considerations that you’ll also want to include. What impact will defragmentation have on your storage array? For example, I think that NetApp doesn’t recommend using defragmentation in conjunction with their storage arrays (I could be wrong; can anyone confirm?). So, I guess my advice would be to do your homework, see how defragmentation is going to affect the rest of your environment, and then proceed from there.
  • Microsoft thinks that App-V should be the most important tool in your virtualization tool belt. Do you agree or disagree?
  • William Lam has instructions for how to identify the origin of a vSphere login. This might not be something you need to do on a regular basis, but when you do need to do it you’ll be thankful you have the instructions how.

I guess it’s time to wrap up now, since I have likely overwhelmed you with a panoply of data center-related tidbits. As always, I encourage your feedback, so please feel free to speak up in the comments. Thanks for reading!

Tags: , , , , , , , , , , ,

FCoE through a Nexus to a MDS-Attached Storage Array

Wednesday, December 15, 2010 in Interoperability, Storage by slowe | 11 comments

In this post, I want to pull together all the steps necessary to take a Converged Network Adapter (CNA)-equipped server and connect it, using FCoE, to a Fibre Channel-attached storage array. There isn’t a whole lot of “net new” information in this post, but rather I’m summarizing previous posts, organizing the information, and showing how these steps relate to each other. I hope that this helps someone understand the “big picture” of how FCoE and Fibre Channel relate to each other and how they interoperate (which, quite frankly, is one of the key factors for the adoption of FCoE).

The steps involved come from an environment with the following components:

  • A Dell PowerEdge R610 server running VMware ESXi and containing an Emulex CNA
  • A Cisco Nexus 5010 switch running NX-OS 4.2(1)N1(1).
  • A Cisco MDS 9134 Fibre Channel switch running NX-OS 5.0(1a).
  • An older EMC CX3-based array with Fibre Channel ports in the storage processors.

We’ll start at the edge (the host) and work our way to the storage. All these steps assume that you’ve already taken care of the physical cabling.

  1. Depending upon how old the software is on your hosts, you might need to install updated drivers for your CNA, as I described here. If you’re using newer versions of software, the drivers will most likely work just fine out of the box.
  2. The closest piece to the edge is the FCoE configuration on the Nexus 5010 switch. Here’s how to setup FCoE on a Nexus 5000. Be sure that you map VLANs correctly to VSANs; for every VSAN that needs to be accessible from the FCoE-attached hosts, you’ll need a matching VLAN. Further, as pointed out here, the VLAN and VLAN trunking configuration is critical to making FCoE work properly anyway.
  3. The next step is connecting the Nexus 5010 to the MDS 9134 Fibre Channel switch. Read this to see how to configure NPV on a Nexus 5000 if you are going to use NPV mode (and read this for more information on NPV and NPIV). Using NPV or not, you’ll also need to setup connections between the Nexus and the MDS; here’s how to setup SAN port channels between a Cisco Nexus and a Cisco MDS.
  4. Once the Nexus and the MDS are connected, then you’ll need to perform the necessary zoning so that the hosts can see the storage. Before starting the zoning, you might find it helpful to set up device aliases. After your device aliases are defined, you can create the zones and zonesets. This post has information on how to create zones via CLI; this post has information on how to manage zones via CLI.

At this point—if everything is working correctly—then you are done and you should be ready to present storage to the end hosts.

I hope this helps put the steps involved (all of which I’ve already written about) in the right order and in the right relationship to each other. If there are any questions, clarifications, or suggestions, please feel free to speak up in the comments.

Tags: , , , , , , ,

« Older entries