The one way around the political issue for the larger folks is still to run a seperate physical network. Cost effective? No. Most efficient? No. Like you said in a well run shop the two teams working together should actually be better, but we all know in some cases they’ll still want to run their own gear. Basically at that point iSCSI and FCoE just become enablers of 10GB rather than convergance. That’s sort of ok though as I see it. When I first built out my iSCSI SAN I did so on the same standard Cisco switches I was using in the data network, but kept it physically seperate. I didn’t have political reason of course unless I wanted to be in a political battle with myself since I also work on the data network. I just knew the data network was peaked out and not ideal to handle the load. Now we are upgrading the infrastructure and bringing the SAN back into the mix. That’s the kind of flexibility I like about iSCSI.

While the technology debates are fun, other aspects must be considered for this technology to be deployed and accepted. At most large organizations politics drives technology decisions at least as much as the merits of the technologies being considered. This is sad, but mostly true. The technologies being debated here actually intensify the political debate.

Fibre Channel over Fibre Channel (FCoFC?) solves a political problem.
FCoE creates a political problem.

FC switches and infrastructure are popular not only because of many of the benefits and technical superiority in some cases over legacy Ethernet, but because the storage group often got to own the switches and infrastructure rather than the network group. One group owning the infrastructure from end-to-end had the benefit of own group being able to manage all aspects of storage without dependence on another group. Service levels could theoretically improve and political empire builders were happy because they owned more stuff and possibly more people. I’ve seen many 4Gbps FC deployments where iSCSI was more than adequate technically and the financial benefits were simply not debatable because the storage groups did not trust/like the network operations groups.

FCoE throws a kink in things because the network operations groups are more likely to own the switches rather than the storage groups. This breaks the end-to-end model and theoretically would drive down service levels because of the interfaces required between the 2 operations groups (I actually believe service levels would increase in well run shops, but that is another debate).

The problem is that while 10GB and DCE benefit both iSCSI and FCoE, they have the same political problems that have slowed the adoption of iSCSI in large enterprises. If the storage group doesn’t get to own the infrastructure from end-to-end, they are going to stick to FC regardless of the benefits of doing something else. And no, Role Based Access Controls for management doesn’t cut it in terms of the political problem.

Is this view cynical? Probably, however it was developed from not just my own experience and those of many people I’ve talked to at many customers, various manufacturers, and resellers.

Again, I say all this despite living clearly in the Ethernet camp.

For q2: FCoE without CEE is a non-thing. Practically, just consider FCoE short for FCoCEE. Getting FC into the E frame and getting lossless, nonblocking, PFC, etc., capabilities into E were just steps in making FCo(C)E(E) a viable technology.

And q3: As things stand today with the standards in progress, iSCSI would ride on the lower order priority queue in CEE and not get the same nonblocking, etc., that FCoE will. A software hack or specialized CNAs could change that, but none are being publicly discussed afaik.

1. FCoE is always mentioned hand-in-hand with 10 Gigabit Ethernet. Can’t iSCSI take advantage of 10 Gigabit Ethernet too?
A->>In short… Yes. iSCSI will function on both non-DCB enabled as well as DCB enabled 10Gb Ethernet. For those that don’t need DCB or don’t want to invest/replace their infrastructure with DCB enabled switching, iSCSI will run just fine on “standard” 10Gb (or 1Gbps Ethernet for that matter unlike FCoE which requires 10Gbps Ethernet). For those that desire the DCB functionality, iSCSI will sit on top of a DCB enabled network and take full advantage of what DCB provides. (side note.. DCB-Data Center Bridging = CEE).

2. FCoE is almost always mentioned in the same breath as “low latency” and “lossless operation”. Truth be told, it’s not FCoE that’s providing that functionality, it’s CEE (Converged Enhanced Ethernet). Does that mean that FCoE without CEE would suffer from the same “problems” as iSCSI?
A->> DCB enabled networking (NICs, switches, and storage arrays) is required for FCoE. FCoE will not work without it. The reason for this is the fact that FCoE itself does not include a mechanism for ensuring reliable delivery. It therefore requires that functionality to exist in the network (ie flow control for Ethernet), which is what a DCB enabled network infrastructure is targeted to provide.
iSCSI, on the other hand, has its own method for ensuring reliable transfer in the protocol layer (ie TCP). This enables iSCSI to run reliably on standard non-DCB enabled Ethernet switches (or remotely for that matter)

3. If iSCSI was running on a CEE network, wouldn’t it exhibit predictable latencies and lossless operation like FCoE?
A->>Yes

Catching up on some of the interesting points/statements in the comments:
– Justin mentioned some additional work required for iSCSI. This additional work(ie TCP) is what ensures reliable delivery in non-DCB enabled networks. FCoE pushes this work into the network and is why it requires DCB enabled NICs, switches, and storage devices. I would argue that for many typical workloads this additional processing is not noticeable. But in either case, if it is a pain point, iSCSI HBAs are available that offload this additional work. With the iSCSI HBA, the host side processing is equivalent to FC or FCoE (all enter under a similar storage stack).
I guess one way of looking at it is as follows: Both FCoE and iSCSI can leverage optimized HBAs(DCB enabled FCoE CNAs or iSCSI offloaded HBAs) and DCB enabled switches to achieve similar performance… but iSCSI also has the flexibility to use standard NICs with standard non-DCB networks.

– As far as Rodos’ point for fitting into existing FC frameworks. One question that comes to mind would be if those frameworks are integrating manageability for the Ethernet switches/networks? I would guess that both FCoE and iSCSI are in the same boat here.

– Justin also brought up an interesting point that iSCSI is routable where FCoE won’t be. This does have some interesting implications today with routing’s ability to enable remote mirroring/DR. I would also suspect that it may become an even more interesting differentiator with the growth of cloud computing.
I guess I’ll wind down with a tie to Nate’s point. FCoE might be appealing as a bridge back into existing Fibre Channel, but if the storage guys already have to swap out their network infrastructure toward ethernet, iSCSI’s flexibility to solve both ends of the cost/performance question and the fact that it is already here would seem to give it a leg up.

-Dan

It’s always exciting to me to see the chatter around a new standard or new “way” of doing storage. Fibre Channel over Ethernet (FCoE) has certainly had it’s share of chatter as well as the inherent criticisms from detractors. …

I agree with you: I cant see any reason to substitute iSCSI by FCoE. I think FCoE is another strategy to assure new bussiness to storage & networking vendors. Personally, I was using iSCSI for years in ESX environment without any special knowlegde of Pure Storage networking.. and it’s works good for me!. FCoE hide the manifest incapacity (or desire) of networking manufactures to improve ethernet with QoS capabilities (as an storage network requires). I’m sure iSCSI over serious datacenter ethernet can provide the same solutions of FCoE… without the expesive knowlegde & management of a FCxx network.

Correct, and I think that really the largest bottle neck becomes the San and or Server / Servers.

But it think that iSCSI is very flexible today, and will be more so in the future.

Thanks for your comment. Not all iSCSI arrays scale in exactly the same fashion, so some of what you are discussing may be specific to Dell/EQL. In addition, not all iSCSI initiators will scale overall throughput linearly with more links as well (think VMware ESX’s software initiator, for example). In this regard, I will say that FC (and presumably FCoE) have the advantage.

Stu,

“Easy interoperability” between FC and FCoE I will grant. As you describe, the ability to manage the FC and FCoE fabrics in much the same fashion, perhaps from the same tools, is quite compelling for large FC shops. But interoperability is not the same as compatibility, and to say that FC is compatible with FCoE is, in my opinion, incorrect. Perhaps I’m being a stickler, but if I can’t plug an FC HBA into an FCoE fabric then they’re not compatible. Interoperable, yes, but not compatible.

Thanks to both of you for your comments!