VMware

You are currently browsing articles tagged VMware.

Multiport Uplink Driver

Friday, December 4, 2009 in Networking, Virtualization by slowe | 2 comments

Virtual I/O is getting more attention. This press release from Chelsio crossed my desk this morning:

Virtual Multi-port Software allows for consolidation of switch ports and cabling by using 10Gb infrastructure while maintaining the existing Gigabit-based ESX setup. The software enables the consolidation by keeping the infrastructure update completely transparent to the ESX hypervisor, enabling a 10Gb adapter to appear to the hypervisor as eight virtual Gigabit adapters. By offloading the tasks performed by the hypervisor, the Chelsio adapters can deliver the best I/O performance for virtualized applications.

There’s no mention of SR-IOV (more information on SR-IOV is available in this post), so I’m guessing that this is a proprietary technology similar to what HP is using in Virtual Connect Flex-10. The key difference with HP Virtual Connect Flex-10 and the Chelsio solution is that Flex-10 doesn’t require any software support in the OS or hypervisor, whereas Chelsio’s solution does require software support (as does SR-IOV). Nevertheless, it’s clear that I/O virtualization—even relatively simple forms of I/O virtualization such as this—is gaining more and more attention.

Tags: , , ,

What is SR-IOV?

Wednesday, December 2, 2009 in Virtualization by slowe | 4 comments

I/O virtualization is a topic that has received a fair amount of attention recently, due in no small part to the attention given to Xsigo Systems after their participation in the Gestalt IT Tech Field Day. While Xsigo uses InfiniBand as their I/O virtualization mechanism, there are other I/O virtualization technologies out there as well. One of these technologies is Single Root I/O Virtualization (SR-IOV).

So what is SR-IOV? The short answer is that SR-IOV is a specification that allows a PCIe device to appear to be multiple separate physical PCIe devices. The SR-IOV specification was created and is maintained by the PCI SIG, with the idea that a standard specification will help promote interoperability.

SR-IOV works by introducing the idea of physical functions (PFs) and virtual functions (VFs). Physical functions (PFs) are full-featured PCIe functions; virtual functions (VFs) are “lightweight” functions that lack configuration resources. (I’ll explain why VFs lack these configuration resources shortly.)

SR-IOV requires support in the BIOS as well as in the operating system instance or hypervisor that is running on the hardware. Until very recently, I had been under the impression that SR-IOV was handled solely in hardware and did not require any software support; unfortunately, I was mistaken. Software support in the operating system instance or hypervisor is definitely required. To understand why, I must talk a bit more about PFs and VFs.

I mentioned earlier that PFs are full-featured PCIe functions; they are discovered, managed, and manipulated like any other PCIe device. PFs have full configuration resources, meaning that it’s possible to configure or control the PCIe device via the PF, and (of course) the PF has full ability to move data in and out of the device. VFs are similar to PFs but lack configuration resources; basically, they only have the ability to move data in and out. VFs can’t be configured, because that would change the underlying PF and thus all other VFs; configuration can only be done against the PF. Because VFs can’t be treated like a full PCIe device, then the OS or hypervisor instance must be aware that they are not full PCIe devices. Hence, OS or hypervisor support is required for SR-IOV so that the OS instance or hypervisor can properly detect and initialize PFs and VFs correctly and appropriately. At this time, SR-IOV support is only found in some of the open source Linux kernels; this means it will find its way into KVM and Xen first. I do not have a timeframe for SR-IOV support in VMware vSphere or Microsoft Hyper-V.

So, putting this all together: what do you get when you have an SR-IOV-enabled PCIe device in a system with the appropriate BIOS and hardware support and you’re running an OS instance or hypervisor with SR-IOV support? Basically, you get the ability for that PCIe device to present multiple instances of itself up to the OS instance or hypervisor. The number of virtual instances that can be presented depends upon the device.

The PCI SIG SR-IOV specification indicates that each device can have up to 256 VFs. Depending on the SR-IOV device in question and how it is made, it might present itself in a variety of ways. Consider these exampes:

  • A quad-port SR-IOV network interface card (NIC) presents itself as four devices, each with a single port. Each of these devices could have up to 256 VFs (single port NICs) for a theoretical total of 1,024 VFs. In this case, each VF would essentially represent a single NIC port.
  • A dual-port SR-IOV host bus adapter (HBA) presents itself as one device with two ports. With 256 VFs, this would result in 512 HBA ports spread across 256 dual-port virtual HBAs.

These are, of course, theoretical maximums. Because each VF requires actual hardware resources, practical limits are much lower. Currently, 64 VFs seems to be the upper limit for most devices.

In situations where VFs represent additional NIC ports or HBA ports, other technologies must also come into play. For example, suppose that you had an SR-IOV-enabled Fibre Channel HBA in a system; that HBA could present itself as multiple, separate HBAs. Of course, because these logical HBAs would still share a single physical HBA port, you’d need NPIV (more information here) to support running multiple WWNs and N_Port_IDs on a single physical HBA port.

Similarly, you might have a Gigabit Ethernet NIC with SR-IOV support. That NIC could theoretically (according to the PCI SIG SR-IOV specification) present itself as up to 256 virtual NICs. Each of these NICs would be discrete and separate to the OS instance or hypervisor, but the physical Ethernet switch wouldn’t be aware of the VFs. Switches wouldn’t, by default, reflect some types of traffic arriving inbound on a port (from one VF) back out on the same port (to another VF). This could create some unexpected results.

SR-IOV does have its limitations. The VFs have to be the same type of device as the PF; you couldn’t, for example, have VFs that presented themselves as one type of device when the PF presented itself as a different type of device. Also, recall from earlier that VFs generally can’t be used to configure the actual physical device, although the extent to which this is true depends upon the implementation. The SR-IOV specification allows some leeway in the actual implementation; this leeway means that some SR-IOV-enabled NICs may also have VF switching functionality present (where the NIC could switch traffic between VFs without the assistance of a physical switch) while other NICs may not have VF switching functionality present (in which case VFs would not be able to communicate with each other without the presence of a physical switch).

I do want to point out that SR-IOV is related to, but not the same as, hypervisor bypass (think VMDirectPath in VMware vSphere). SR-IOV enables hypervisor bypass by providing the ability for VMs to attach to a VF and share a single physical NIC. However, the use of SR-IOV does not automatically indicate the hypervisor bypass will also be involved. Hypervisor bypass is a topic that I’m sure I will discuss in more detail in the near future.

Finally, it’s worth noting that the PCI SIG is also working on a separate IOV specification that allows multiple systems to share PCIe devices. This specification, known as Multi-Root IOV (MR-IOV), would enable multiple systems to share PCIe VFs. I hope to have more information on MR-IOV in the near future as well.

You now should have a basic understanding of SR-IOV, what it does, what is necessary to support it, and some of the benefits and drawbacks that SR-IOV creates. Feel free to post any questions you have about SR-IOV in the comments and I’ll do my best to get answers for you.

Tags: , , , ,

vSphere 4.0 Quick Start Guide

Wednesday, December 2, 2009 in Virtualization by slowe | 6 comments

It’s funny how ideas evolve. Months ago, the idea of a “super blog” came up in a series of Twitter conversations. Stephen Foskett (@sfoskett on Twitter) ran with that idea to create Gestalt IT; Duncan Epping (@DuncanYB on Twitter) ran with that idea to create vSphere 4.0 Quick Start Guide: Shortcuts down the path of Virtualization.

I didn’t, unfortunately, get a chance to actually have a look at the QSG (easier to type than “Quick Start Guide”) until this past week. Some preview copies were given away at VMworld 2009 in San Francisco, but I didn’t manage to get a copy. After having some time to review it briefly this past week—I still haven’t read the entire book cover-to-cover—I must say that I am impressed.

What I’ve seen of the book is thorough, detailed, and takes a “common sense” approach to providing how-to information on VMware vSphere. In this respect, it’s a lot like my second book, VMware vSphere 4 Administration Instant Reference, which I co-wrote with Jase McCarty and Matthew Johnson and which will be officially released later this month (pre-orders available on Amazon right now). One thing that I do like about the QSG is the integration of PowerCLI at every stage. This makes it super-easy for aspiring PowerCLI gurus to learn their way around the cmdlets and how to put them together. It’s an idea I wish I’d thought of doing! Overall, given the breadth of coverage and the detail provided, the QSG does a great job.

If you’re looking for a how-to guide on VMware vSphere, buy both my book and the QSG for the most extensive coverage. Both books—mine or the QSG—provide excellent coverage on their own; the QSG certainly excels in the practical application of PowerCLI to the administration tasks. Either would be a great choice. To Duncan, Thomas, Dave, Stuart, Alan, and Bernie: great work, guys!

Here are links to both books on Amazon.com:

vSphere 4.0 Quick Start Guide: Shortcuts down the path of Virtualization

VMware vSphere 4 Administration Instant Reference

Tags: , , ,

Triangle Area VMUG Meeting Next Week

Monday, November 30, 2009 in Virtualization by slowe | No comments

The next meeting of the Triangle Area VMware User Group will be next Thursday, December 10, 2009, from 11:30AM to 2PM. The meeting will be held at RTP Organization in Building 18/1807 at 104 TW Alexander Drive in Research Triangle Park. A map of the location (courtesy of Google Maps) is available here, and a schedule/agenda of the meeting is available here.

I will be presenting at the meeting during a “Stump the Expert” (or maybe it should be “Stump the vExpert”?) session at the end of the meeting. I insisted that it wouldn’t be very hard to stump this vExpert; if you don’t believe me, go back and watch the video from the Ask The Experts VMworld sessions. Nevertheless, I appreciate the opportunity to join VMware, Veeam, and local users during the meeting and I hope that I’ll be able to provide some useful information.

I hope to see you there!

Tags: ,

Virtualization Short Take #31

Wednesday, November 18, 2009 in Virtualization by slowe | 12 comments

Welcome back to yet another Virtualization Short Take! Here is a collection of virtualization-related items—some recent, some not, but hopefully all interesting and/or useful.

  • Matt Hensley posted a link to this VIOPS document on how to setup VMware SRM 4.0 with an EMC Celerra storage array. I haven’t had the chance to read through it yet.
  • Jason Boche informs us that both Lab Manager 3 and Lab Manager 4 have problems with the VMXNET3 virtual NIC. In this blog post, Jason describes how his attempts to install Lab Manager server into a VM with the VMXNET3 NIC was failing. Fortunately, Jason provides a workaround as well, but you’ll have to read his article to get that information.
  • Bruce Hoard over at Virtualization Review (disclaimer: I write a regular column for the print edition of Virtualization Review) stirred up a bit of controversy with his post about Hyper-V’s three problems. The first problem is indeed a problem, but not an architectural or technological problem; VMware is indeed the market leader and has a quite solid user base. The second two “problems” stem from Microsoft’s architectural decision to embed the hypervisor into Windows Server. Like any other technology decision, this decisions has its advantages and disadvantages (these technology decisions are a real double-edged sword). Based on historical data, it would seem that the need to patch Windows Server will impact the uptime of the Windows virtualization solution; however, this is not to say that VMware ESX/ESXi are not without their patches and associated downtime as well. I guess the key takeaway here is that VMware seems to be doing a much better job of lessening (or even removing) the impact of the downtime through things like VMotion, DRS, HA, maintenance mode, and the like.
  • Apparently there is a problem with the GA release of the Host Update utility that is installed along with the vSphere Client, as outlined here by Barry Coombs. Downloading the latest version and reinstalling seems to fix the issue.
  • And while we are on the subject of ESX upgrades, here’s another one: if the /boot partition is too small, the upgrade to ESX 4.0.0 will fail. This isn’t really anything too new and, as Joep points out, is documented in the vSphere Upgrade Guide. I prefer clean installations of VMware ESX/ESXi anyway.
  • Dave Mishchenko details his adventures (part 1, part 2, and part 3) in managing ESXi without the VI Client or the vCLI. While it’s interesting and contains some useful information, I’m not so sure that the exercise is useful in any way other than academically. First of all, Dave enables SSH access to ESXi, which is unsupported. Second, while he shows that it’s possible to manage ESXi without the VI Client or the vCLI, it don’t seem to be very efficient. Still, there is some useful information to be gleaned for those who want to know more about ESXi and its inner workings.
  • I think Simon Seagrave and Jason Boche were collaborating in secret, since they both wrote posts about using vSphere’s power savings/frequency scaling functionality. Simon’s post is dated October 27; Jason’s post is dated November 11. Coincidence? I don’t think so. C’mon, guys, go ahead and admit it.
  • Thinking of using the Shared Recovery Site feature in VMware SRM 4.0? This VMware KB article might come in handy.
  • I’m of the opinion that every blogger has a few “masterpiece” posts. These are posts that are just so good, so relevant, so useful, that they almost transcend the other content on the blogger’s site. Based on traffic patterns, one of my “masterpiece” posts is the one on ESX Server, NIC teaming, and VLAN trunking. It’s not the most well-written post I’ve ever published, but it seems to have a lasting impact. Why do I mention this? Because I believe that Chad Sakac’s post on VMware I/O queues, microbursting, and multipathing is one of his “masterpiece” posts. Like Scott Drummonds, I’ve read that post multiple times, and every time I read it I get something else out of it, and I’m reminded of just how much I have yet to learn. Time to get back out of that comfort zone!
  • Oh, and speaking of Chad’s blog…this post is handy, too.

That’s all for now, folks. Stay tuned for the next installation, where I’ll once again share a collection of links about virtualization. Until then, feel free to share your own links in the comments.

Tags: , , , , , , ,

Thinking Out Loud: Is VMware Legitimizing the Competition?

Monday, November 16, 2009 in Microsoft, Virtualization by slowe | 14 comments

I was reading a completely unrelated post on Alessandro’s site this morning about how VKernel is reacting to VMware’s release of CapacityIQ when a thought occurred to me: is VMware legitimizing the competition?

Here’s the excerpt from Alessandro’s post that started me thinking:

And of course VKernel now is also in hurry to clarify that support for Microsoft Hyper-V and Citrix XenServer is coming.

Now, let me ask you this question: what is one of the largest complaints about products like Microsoft Hyper-V and Citrix XenServer? It’s the size of the partner ecosystem. Customers are a bit more hesitant to deploy these other solutions in part because there aren’t as many partner solutions out there to complement the virtualization solutions.

So, as VMware expands into new markets like capacity management and monitoring, backups, etc., former VMware-only partners are forced to adapt their products to work with Hyper-V and XenServer in order to protect themselves. This causes the size of the partner ecosystem for VMware’s competitors to grow, eliminating that complaint and removing one of VMware’s competitive advantages. In effect, VMware’s own actions are building out the partner ecosystem for their competitors and thus legitimizing the competition.

Am I crazy? Am I wrong? What is a company like VMware to do, if anything? I’d love to hear your thoughts.

UPDATE: Some readers have pointed out, rightfully so, that “legitimizing” isn’t really the best word to use here. Perhaps “assisting” or “helping” is a better word?

Tags: , , , , , ,

VCDX Design Exam Post-Mortem

Thursday, November 5, 2009 in Virtualization by slowe | 3 comments

It’s a bit later than I would have liked (sorry Jason!), but I wanted to write up a brief post-mortem on the VCDX Design Exam, which I took—and passed with a score of 408—this past Tuesday.

The exam wasn’t particularly difficult in the sense of needing to know specific details. For example, I saw very few questions asking about a specific command-line parameter, the output of a specific command, or how many X that feature Y supports. There were a few, but not many. That doesn’t mean you don’t need to know that sort of stuff, because you do—but in a very different way.

The exam is moderately difficult in the sense that you are required to take the product knowledge you have and put it together to solve a scenario. The real kicker, in my mind, is the “Select the best answer” prompt—implying that there was more than one technically correct answer. But which answer was best? Yes, you could do it via method A, but is method B better? Which method best satisfies the requirements described in the question? In this aspect, I did find the test challenging. Not incredibly difficult, but challenging.

Unfortunately, the test suffered from a few issues. There were several questions that had missing components necessary for a correct answer. I commented on those, but what else can you do? The design portion of the exam—by now you probably know there is a multiple choice section and a graphical design section—had a terrible interface. I struggled more with the interface than I did the question. Looking at it from VMware’s perspective, I’m sure that it was incredibly difficult to come up with something like that, and I give them credit for actually trying (and not totally failing).

All in all, I think that VMware did a reasonably good job with the exam. It was very different from the Enterprise Admin exam but equally challenging in a very different way.

Now comes the real question: what did I use to study? Here’s the resources that I used:

  • The VI3 connections and ports diagram, downloaded here from Forbes’ site
  • iSCSI Design Considerations and Deployment Guide from VMware
  • Configuring and Troubleshooting N_Port ID Virtualization document from VMware
  • Setup for Microsoft Cluster Service document from VMware
  • Fibre Channel SAN Configuration Guide from VMware
  • Virtual Machine Backup Guide from VMware
  • Resource Management Guide from VMware
  • Duncan’s VMware HA deepdive
  • VI3 security hardening white paper from VMware

I used all this in conjunction with the Design Exam blueprint, of course.

So, there you go: there’s my brief analysis of my VCDX Design Exam experience. Your mileage may vary, of course, since there are numerous versions of the exam.

Tags: ,

A Few Quick Thoughts on the VCE Coalition Announcement

Tuesday, November 3, 2009 in Gestalt, Networking, Storage, Virtualization by slowe | 21 comments

VMware, Cisco, and EMC made their official announcement of the VCE Coalition and the joint venture Acadia this morning. You can read one of the press releases here via MarketWire.

Acadia is interesting, but it really isn’t the meat of the announcement, in my opinion. The real substance of the matter is the nature of the coalition. There are many interesting questions/thoughts circling in my head right at the moment:

  • What impact will this have on VMware’s relationship(s) with HP, IBM, and Dell? “Throwing their hat in the ring” with Cisco’s UCS, so to speak, may greatly endanger VMware’s much larger (with respect to revenue) relationships with other OEMs. What will happen to VMware if those OEMs “throw their hat in the ring” with Microsoft and Hyper-V? This is not a good place to be.
  • The acrimonious Cisco-HP relationship adds further fuel to the concerns over VMware’s close alliance with Cisco’s computing platform.
  • Does this new coalition signal a move away from the “arms-length” relationship between EMC and VMware, a move that some (competitors, notably) have been talking about for some time? If so, what danger does that put VMware in with regards to storage relationships?
  • It seems to me that VMware has the most to lose here. What does EMC lose if this doesn’t go well? Nothing, really. What about Cisco? Nothing, really. VMware, on the other hand…well, it could be ugly.
  • What does this coalition offer that the three companies couldn’t deliver without the coalition? Why risk important relationships? This is a big question in my mind. Lots of technology companies have delivered validated designs without any sort of formal coalition. Why is one necessary in this case?
  • On the other end of the spectrum—keeping Acadia out of the picture for the moment—is this “new coalition” really anything more than what the three companies have already been doing? Is this really anything more than each of the companies dedicating resources to this effort? I know from my own direct interaction with at least one of these vendors that resources had already been dedicated to the VCE technology intersection before any sort of formal announcement. So, does this formal announcement really mean anything at all?

I don’t have any answers (yet), but you can at least read my thoughts—and contribute back to them via the comments—without having to pay $499 to some analyst firm.

By the way, if you’d like some other viewpoints on this matter, here are a couple from opposing viewpoints:

NetApp - Jay’s Blog: The Importance of Being Open
Chuck’s Blog: Announcing the VCE Coalition

Feel free to speak up in the comments below (courteous comments only, please, and be sure to include full vendor disclosure where appropriate). Thanks!

Tags: , , , , , , , ,

VM Hardware Version 7 and Windows Server 2008

Saturday, October 31, 2009 in Microsoft, Virtualization by slowe | 2 comments

Stu over at vInternals posted an article a couple of days ago about a problem he encountered with VMware vSphere and Windows Server 2008. Apparently, there is an unexpected behavior with Windows Server 2008 and VM hardware version 7 that is described in this VMware KB article. Stu, however, was seeing the behavior not on upgrading VMs from VM hardware version 4 to VM hardware version 7, but on new virtual machines created from the beginning with VM hardware version 7.

According to an update on Stu’s article, VMware has acknowledged this as a bug and will be investigating a fix to the problem. Until then, follow Stu’s advice and speak to your VMware account team if you are experiencing this problem. If you are getting ready to proceed with a VMware vSphere upgrade and have Windows Server 2008 Enterprise Edition VMs in place, keep this behavior in mind and plan accordingly.

Thanks to Stu for bringing this matter to light!

UPDATE: Stu posted an update with more information and an explanation for the unexpected behavior, so be sure to check it out.

Tags: , , , ,

VMware HA, VMware FT, and OS Clustering

Friday, October 30, 2009 in Microsoft, Virtualization by slowe | 15 comments

With the release of VMware vSphere 4 earlier this year, VMware officially introduced VMware Fault Tolerance (VMware FT), a new mechanism for providing extremely high levels of availability to virtual machine workloads. As I’ve talked with customers, I’ve noticed a growing number of customers who are unaware of the differences between the types of high availability that VMware provides (in the form of VMware HA and VMware FT) and operating system-level clustering (such as Microsoft Windows Failover Clustering). Although both types of technology are intended to increase availability and reduce downtime, they are very different and offer different types of functionality.

Consider these points:

  • While using VMware HA will protect you against the failure of an ESX/ESXi host, VMware HA won’t—by default—protect you against the failure of the guest operating system. An OS-level cluster, on the other hand, does protect against the failure of the guest operating system. +1 for OS-level clustering.
  • VMware clusters that are using VMware HA can choose to use VM Failure Monitoring and gain some level of protection against the failure of the guest operating system, but you still won’t get protection of the specific application within the guest operating system, unlike an OS-level cluster. +1 for OS-level clustering.
  • These same arguments also apply to VMware FT. VMware FT won’t protect you against guest operating system failure—a crash of the OS in the primary VM generally means a crash of the OS in the secondary VM at the same time—and it won’t protect you against application failure. +1 for OS-level clustering.
  • You can’t failover between systems using VMware HA or VMware FT in order to perform OS upgrades or apply OS patches. +1 for OS-level clustering.
  • Similarly, you can’t failover between systems using VMware HA or VMware FT in order to do a rolling upgrade of the application itself. +1 for OS-level clustering.
  • Of course, the VMware technologies do have some advantages. Both VMware HA and VMware FT are far, far simpler to enable and configure than an OS-level cluster. +1 for VMware.
  • Both VMware HA and VMware FT don’t require any application support in order to protect the VM and its workloads. +1 for VMware.
  • Neither VMware HA nor VMware FT require that you license specific editions of the guest operating system or application in order to be able to use their benefits. +1 for VMware.
  • VMware HA can produce higher levels of utilization within a host cluster than using OS-level clustering. +1 for VMware.
  • VMware FT can provide higher levels of availability than what is available in most OS-level clustering solutions today. +1 for VMware.

This is not a knock against any of technologies listed—VMware HA, VMware FT, or OS-level clustering—but rather an exploration of their advantages, disadvantages, similarities, and differences. Hopefully, this will help readers who might not be as familiar with these products make a more informed decision about which technologies to deploy in their data center. (Hint: You’ll probably need all of them.)

Tags: , , , , ,

« Older entries