Comments on: TA2384 – Deploying the Nexus 1000V

By: Nico

Nico — Thu, 21 Jan 2010 12:41:10 +0000

Hi there,
After a lot of weird behaviours, i have tested the nexus 1000V in deep and my conclusion is that you should NEVER use cdp to create your sub-group in a critical environment.
The reason is that cdp can take at least 5s to advertise a link coming up and my tests show that you can loose about 7-8 pings when a link come up ine a vPC-HM config.
Use the manual or mac-pinning config. (cisco community thread https://www.myciscocommunity.com/thread/8023;jsessionid=BB9B9668B537623B904EB675C62B3523.node0)
By the way i try the same thing with vsphere standard vSwitches and the fact is that when you link a standard virtual switch to 2 cards on 2 physical switches (virtual port id LB), nothing is lost when failing-over, but if you choose reverting back to the link when it comes up, you loose 6-7 pings.
That was not the case with 3.5.
After some tests, it does not happen with beacon probing activated, of course.
So i would recommand to activate beacon probing by default when configuring the type of LB!! (or do not choose to revert back to the original link)

By: Chris Smith

Chris Smith — Mon, 05 Oct 2009 14:36:23 +0000

After several weeks of testing with “self-hosted” VSMs (ie: VSMs running on top of VEMs), I’ve pretty much given up on the idea. The setup “works” – most of the time – but is far too fragile for production, in my opinion. I found probably half the time during my failure and disaster testing (ie: rebooting random things, shutting down the VSMs and trying to bring them back up, etc), that the whole shebang would eventually, and inevitably, get twisted up so badly it was necessary to essentially start over from scratch – unlink all the vmnics from the Nexus 1000V, set them up on regular vSwitches, reconfigure the VSMs to use those vSwitches, then migrate everything back to the 1000V DVS again. My feeling from that was that if you have a disaster that shuts down both VSMs, there’s about a 20% chance you’ll need to do some _serious_ recovery work to get them (and hence the entire cluster) running properly again.

For the moment, I’ve moved the VSMs out onto a couple of independent, low-end ESXi hosts, which has at least improved things dramatically from a disaster recovery perspective (haven’t been able to break it yet). However, I can only hope the rumblings I’ve heard about integrating the VSM functionality into the Nexus 5000 are true, and planned for the near future, because at the moment the 1000V doesn’t really deliver what it promises, and that is in no small part because you effectively “need” separate server(s) for the VSM(s).

I’ve also had trouble with vPC-HM in my UCS chassis, with the hardware and configuration described in this “Best Practices” document: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/white_paper_c11-558242.html (M71KR-Q CNAs). In particular, whenever I link the second vmnic to the DVS, connectivity becomes extremely unpredictable and unreliable (sometiems it breaks immediatley, sometimes it takes an hour – but it always breaks eventually). I think this might be something to do with the pinning within the UCS, and I’m about to open up a TAC with Cisco to try and figure out what’s wrong.

By: ProInteg

ProInteg — Thu, 01 Oct 2009 16:32:59 +0000

Will the Cisco 1000V allow me to use multiple 1Gbps NIC’s to create more bandwidth to my iSCSI Target?

If not then I guess I’m stuck with moving to 10GbE.

By: jason

jason — Mon, 28 Sep 2009 19:00:24 +0000

So.. can anybody tell me if there is a technical design that drives the use of 1000V over the Vsphere VDS or is it strictly to give the network folkes visibility into the virtual networking layer?

By: Omar Sultan

Omar Sultan — Tue, 08 Sep 2009 05:25:05 +0000

Scott/Chris:

Just checked with the product team to make sure, but you can currently put a VEM and VSM on the same host. What we don’t currently recommend is VMotioning your VSM. An upcoming update will offer more robust support for VSM VMotion.

Omar
Cisco

By: Chris Smith

Chris Smith — Sat, 05 Sep 2009 11:08:01 +0000

Scott:
That requirement was essentially from the horse’s mouth – the Cisco-recommended people who have just finished installing our new Nexus(incl. 1000v)+UCS+VMWare infrastructure.

I haven’t actually had time to make it to the 1000v docs myself, so if you have information otherwise, I’d love to know – since we’ve currently got an 8-core/48GB RAM blade that doesn’t do anything more than vCenter and the 1000v VSM.

By: slowe

slowe — Sat, 05 Sep 2009 05:09:21 +0000

Steve,

Please be sure to provide full vendor disclosure. Since your message originated from an IP address block that belongs to HP, I can only assume that you work for HP. If that is not the case, please let me know.

By: slowe

slowe — Sat, 05 Sep 2009 05:06:03 +0000

Chris Smith,

I think you’re incorrect about not being able to run the VSM on a host with the VEM–I’ve seen a couple of references to the necessary configuration although I’ve not yet done it myself.

By: Larry Passo

Larry Passo — Fri, 04 Sep 2009 16:04:33 +0000

I usually refer to Layers 8-11 (Financial, Legal, Political, Religious). Since they all cause different problems.

By: Steve Nicholson

Steve Nicholson — Fri, 04 Sep 2009 13:13:55 +0000

Here is a great interview that was released this week. The article reviews the emergence of VEPA as a base standard for exposing VM traffic beyond the NIC and it specifically points out the proprietary nature of Cisco’s plans with VNtag. I found this article extremely interesting. Any of your customers contemplating Cisco UCS (and VNTag) will definitely benefit from reading this article. It also does a great job outlining the need for this technology.

http://www.eweekeurope.co.uk/interview/hp-claims-victory-in-virtualisation-standards–1734?page=1

Interesting sections are captured below:

“Cisco has a proprietary packet format, that modifies the Ethernet frame to integrate UCS servers with switches,” said Congdon. “That was more intrusive than the Industry wanted to see. Modifying the frame is usually a bad thing, because it can make everyone’s silicon obsolete, and customers have to buy new equipment, to address an incremental problem. If we can avoid that, it is a good thing.”

Hardware NICs effectively now include an Ethernet switch, and Congdon wants to see a “small evolutionary step” that exposes the network traffic to the outside world. Cisco wanted a new tag format, new addresses, and new hardware structures, he said.

The end result wasn’t a battle to the death (unlike all too many IEEE standards spats) says Congdon: “Cisco and HP started out at odds with each other, but finally came to a compromise.”

That compromise is for VEPA to form a base standard, while Cisco addresses extensions that it thinks are necessary.