TA2384 - Deploying the Nexus 1000V

“One interesting statement from the presenter was that Layer 8 (the Human layer) can create more problems than Layers 1 through 7.”

… I see it every day unfortunately not only on Nexus 1KV

Scott:

Have you seen the N1KV deployment guide–I just put it up last week: http://bit.ly/eg95 There is also the guide for setting up a DMZ with vSphere and N1KV, but that has been set up for a while: http://bit.ly/2zMHEf.

Regards,

Omar

Josh:

Cool–glad you found it helpful. One other site folks might find helpful is the N1KV Community http://www.cisco.com/go/1000vcommunity. Outside of docs and the like, there is a forum that is staffed by the product managers and engineers, so if you have questions, its a good place to get answers from the source.

Regards,

Omar
Cisco

Here is a great interview that was released this week. The article reviews the emergence of VEPA as a base standard for exposing VM traffic beyond the NIC and it specifically points out the proprietary nature of Cisco’s plans with VNtag. I found this article extremely interesting. Any of your customers contemplating Cisco UCS (and VNTag) will definitely benefit from reading this article. It also does a great job outlining the need for this technology.

http://www.eweekeurope.co.uk/interview/hp-claims-victory-in-virtualisation-standards–1734?page=1

Interesting sections are captured below:

“Cisco has a proprietary packet format, that modifies the Ethernet frame to integrate UCS servers with switches,” said Congdon. “That was more intrusive than the Industry wanted to see. Modifying the frame is usually a bad thing, because it can make everyone’s silicon obsolete, and customers have to buy new equipment, to address an incremental problem. If we can avoid that, it is a good thing.”

Hardware NICs effectively now include an Ethernet switch, and Congdon wants to see a “small evolutionary step” that exposes the network traffic to the outside world. Cisco wanted a new tag format, new addresses, and new hardware structures, he said.

The end result wasn’t a battle to the death (unlike all too many IEEE standards spats) says Congdon: “Cisco and HP started out at odds with each other, but finally came to a compromise.”

That compromise is for VEPA to form a base standard, while Cisco addresses extensions that it thinks are necessary.

Scott:
That requirement was essentially from the horse’s mouth - the Cisco-recommended people who have just finished installing our new Nexus(incl. 1000v)+UCS+VMWare infrastructure.

I haven’t actually had time to make it to the 1000v docs myself, so if you have information otherwise, I’d love to know - since we’ve currently got an 8-core/48GB RAM blade that doesn’t do anything more than vCenter and the 1000v VSM.

So.. can anybody tell me if there is a technical design that drives the use of 1000V over the Vsphere VDS or is it strictly to give the network folkes visibility into the virtual networking layer?

After several weeks of testing with “self-hosted” VSMs (ie: VSMs running on top of VEMs), I’ve pretty much given up on the idea. The setup “works” - most of the time - but is far too fragile for production, in my opinion. I found probably half the time during my failure and disaster testing (ie: rebooting random things, shutting down the VSMs and trying to bring them back up, etc), that the whole shebang would eventually, and inevitably, get twisted up so badly it was necessary to essentially start over from scratch - unlink all the vmnics from the Nexus 1000V, set them up on regular vSwitches, reconfigure the VSMs to use those vSwitches, then migrate everything back to the 1000V DVS again. My feeling from that was that if you have a disaster that shuts down both VSMs, there’s about a 20% chance you’ll need to do some _serious_ recovery work to get them (and hence the entire cluster) running properly again.

For the moment, I’ve moved the VSMs out onto a couple of independent, low-end ESXi hosts, which has at least improved things dramatically from a disaster recovery perspective (haven’t been able to break it yet). However, I can only hope the rumblings I’ve heard about integrating the VSM functionality into the Nexus 5000 are true, and planned for the near future, because at the moment the 1000V doesn’t really deliver what it promises, and that is in no small part because you effectively “need” separate server(s) for the VSM(s).

I’ve also had trouble with vPC-HM in my UCS chassis, with the hardware and configuration described in this “Best Practices” document: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/white_paper_c11-558242.html (M71KR-Q CNAs). In particular, whenever I link the second vmnic to the DVS, connectivity becomes extremely unpredictable and unreliable (sometiems it breaks immediatley, sometimes it takes an hour - but it always breaks eventually). I think this might be something to do with the pinning within the UCS, and I’m about to open up a TAC with Cisco to try and figure out what’s wrong.

CS