IDF 2014: Data Center Mega-Session

This is a liveblog of the Data Center Mega-Session from day 2 of Intel Developer Forum (IDF) 2014 in San Francisco.

Diane Bryant, SVP and GM of the Data Center Group takes the stage promptly at 9:30am to kick off the data center mega-session. Bryant starts the discussion by setting out the key drivers affecting the data center: new devices (and new volumes of devices) and new services (AWS, Netflix, Twitter, etc.). This is the “digital service economy,” and Bryant insists that today’s data centers aren’t prepared to handle the digital service economy.

Bryant posits that in the future (not-so-distant future):

  • Systems will be workload optimized
  • Infrastructure will be software defined
  • Analytics will be pervasive

Per Bryant, when you’re operating at scale then efficiency matters, and that will lead organizations to choose platforms selected specifically for the workload. This leads to a discussion of customized offerings, and Bryant talks about an announcement earlier in the summer that combined a Xeon processor and a FPGA (field-programmable gate array) on the same die.

Bryant then introduces Karl Triebes, EVP and CTO of F5 Networks, who takes the stage to talk about FPGAs in F5 and how the joint Xeon/FPGA integrated solution from Intel plays into that role. F5′s products use Intel CPUs, but they also leverage FPGAs to selectively enable certain functions in hardware for improved performance. Triebes talks about how F5 and Intel have been working together for about 10 years, and discusses how F5 uses instruction set changes (they write their own microkernel—is that really sustainable moving forward?), new features, etc., and that includes leveraging the integrated FPGA in Intel’s new product.

The discussion now shifts to low-power system-on-chips (SoCs), such as the 64-bit Intel Atom. Bryant announces the third-generation SoC, named Xeon D and based on the Xeon platform. The Xeon D is sampling now. Bryant brings on stage Patty Kummrow, who is Director of Server SoC Development. Bryant and Kummrow talk about how Intel is addressing the need to customize the platform to address critical workloads: software (storage acceleration library, for example); in-package accelerator (FPGA, for example); SoC (potentially incorporating customer IP); and instruction set architectures (like the AES-NI instructions to enhance cryptographic functions). Kummrow shows off a Xeon D SoC and board.

Bryant shifts the discussion to software-defined infrastructure (SDI). The first area of SDI that Bryant focuses upon is storage, where growth is happening rapidly but storage is still siloed. Per Bryant, Intel believes that software-defined storage will address these concerns, and doing so in three ways:

  • Intel Storage Acceleration Libraries (ISA-L)
  • Open source investments in Ceph and OpenStack Swift
  • Prototype SDS controller providing separate of control plane and data plane

Bryant now turns to software-defined networking (SDN) and network functions virtualization (NFV), and—quite naturally—points to the telcos as the prime example of why SDN/NFV are so important. According to Bryant, NFV originated in October 2011, and now (just three years later) there will be commercial deployments by companies like AT&T, Verizon Wireless, SK telecom, and China Mobile. Bryant also talks about Intel’s Network Builders program, and talks about Nokia’s recent announcement (which is based on Intel Xeon).

Shifting now to the compute side, Bryant talks about Intel’s rack-scale architecture (RSA) efforts. RSA attempts to provide disaggregated pools of resources, a standard method of exposing hardware to software, and a composable infrastructure that can be assembled based on application resources.

Core to Intel’s RSA efforts is silicon photonics, which is a key point to allowing high-speed, low-latency connections between the disaggregated resources within an RSA approach. Silicon photonics will enable 100Gbps at greater than 300 meters, at a low cost and with high reliability. Also important, but sometimes overlooked, is that the silicon photonics cabling will be smaller and thinner.

Bryant introduces Andy Bechtolsheim, Founder and Chief Development Officer and Chairman of Arista Networks. Bryant gives Bechtolsheim the opportunity to talk about Arista’s recent launch of 100Gb networking and why 100Gb networking is important and necessary in modern data centers. Bryant states that she believes silicon photonics will be essential in delivering cost-effective 100Gb solutions, and that leads to a discussion of the CLR4 alliance. CLR4 is focused on delivering 100Gb over even greater distances.

Next, Bryant introduces Das Kamhout to talk about the need for an orchestration system in the data center. Kamhout talks about how advanced telemetry can be exposed to the orchestration system, which can make decisions based on that advanced telemetry. This will eventually lead to predictive actions. It boils down to a “watch, act, learn” feedback loop. The foundation is built on Intel technologies like Cache Acceleration, ISA-L, DPDK, QuickAssist, Cache QoS, and power and thermal awareness.

This “finally” leads into a discussion of pervasive analytics, which is one of the three key attributes of future data centers. Bryant states that pervasive analytics will help improve cities, discover treatments, reduce costs, and improve products—obviously all through data centers powered by Intel products. Intel’s focus is to enable analytics, and is working closely with the Hadoop community (specifically Cloudera).

According to Bryant, the new Intel E5–2600 v3 more than doubles the performance of Cloudera’s Hadoop distribution. Bryant brings out Mike Olson, Co-Founder and Chief Strategy Officer for Cloudera. Olson states that the consumer Internet “discovered” the idea of big data, but this is now taking off in all kinds of industries. Olson gives examples of hospitals instrumenting neonatal care units and cities gathering data on air quality more frequently and more comprehensively. Both Olson and Bryant reinforce the value of open source to “amplify” the effect of certain efforts. Olson again conflates big data and the Internet of Things (IoT), indicating that he believes that the two efforts are naturally coupled and will drive each other. Bryant next gives Olson the opportunity to talk about Cloudera Hadoop 5.2, which is optimized for Intel architectures to provide more performance and more security, which in turn will lead to accelerated adoption of Hadoop. Bryant reinforces the link between IoT/wearables and big data, mentioning again the “A-wear” program discussed yesterday in the keynote.

At this point Bryant wraps up the keynote and the session ends.

Tags: , ,

IDF 2014 Day 1 Recap

In case you hadn’t noticed, I’m at Intel Developer Forum (IDF) 2014 this week in San Francisco. Here’s a quick recap of day 1 (I should have published this last night—sorry for not getting it out sooner).

Day 1 Keynote

Here’s a liveblog of the IDF 2014 day 1 keynote.

The IDF keynotes are always a bit interesting for me. Intel has a very large consumer presence: PCs, ultrabooks, tablets, phones, 2-in–1/convertibles, all-in–1 devices. Naturally, this is a big part of the keynote. I don’t track or get involved in the consumer space; my focus is on the data center. It is kind of fun to see all the stuff going on in the consumer space, though. There were no major data center-centric announcements yesterday (day 1), but I suspect there will be some today (day 2) in a mega-session with Diane Bryant (SVP and GM of the Data Center Group at Intel). I’ll be liveblogging that mega-session, so stay tuned for details.

Technical Sessions

I was able to hit two technical sessions yesterday and liveblogged both of them:

Both were good sessions. The first one, on virtualizing the network, did highlight an important development regarding hardware offloads for Geneve, the next-generation network overlay encapsulation protocol. Intel announced yesterday that the new XL710 network adapters (which are 40Gbps adapters) will support Geneve hardware offloads. This is the first hardware offload for Geneve of which I am aware, and it signals increased support for Geneve. (The XL710 also supports offloads for VXLAN and NVGRE.) That’s cool.

The second session was more of an introductory session than anything else, but was useful nevertheless. I was already familiar with all the concepts discussed regarding Docker and containers and virtualization, but I did pick up a few useful analogies from the speaker, Nick Weaver. Nick didn’t share anything specific to containers with regard to work Intel might be doing, but as I was thinking about this after the session I wondered if Intel might do some work around enabling containers to use the x86 privilege rings/protection rings. This would improve container security and move Linux containers closer to the Bromium “microvisor” architecture. Nick was also bullish on Intel SGX, something I’m going to have to explore in a bit more detail (I don’t know anything about SGX yet).

Coffee Chats

One of the nice things about attending IDF is that the Intel folks do a great job of connecting influencers (bloggers, press, analysts) with key folks within Intel to discuss announcements, trends, etc. This year, this took the form of “coffee chats”—informal discussions that were, sadly, lacking coffee.

In any case, the discussions wandered around a bit (as these sorts of things are wont to do). Here are a few thoughts that I gleaned from the discussions or that resulted from the discussions:

  • Intel does have/is working with very large customers on customized silicon, typically these are tweaks to create a custom SKU (like more cores, higher frequencies, different power envelope, etc.). This is interesting, but obviously applicable only to the largest of customers given the cost involved.
  • Intel is working with a few other companies (Dell, Emerson, and HP) on a hardware API specification; early work on the API can be found here.
  • Intel is pushing forward with the idea of rack-scale architecture (RSA); this is something I blogged about last year (see this post). There’s another RSA-related session on Thursday that I’m hoping to be able to attend so I can provide more information. I’m on the fence about RSA; I still don’t see a compelling reason why users/consumers/operators should switch to RSA instead of buying servers. I may publish something else specific about RSA later; I still need to have some discussions with the Intel engineers on the floor and see if I’m missing something.
  • The networking-focused Fulcrum assets that Intel purchased a few years ago are continuing to be leveraged in a variety of ways, some of which are also related to the rack-scale architecture efforts. Personally, I’m less interested in how Intel is using the Fulcrum stuff in RSA, and more interested in work Intel might be doing around making it easier for Linux vendors to “hook into” Intel-based hardware platforms for the purpose of building disaggregated network operating systems. You may already know that I’m pretty bullish on Cumulus Linux, but Cumulus right now is heavily tied to the Broadcom chipsets, and—according to discussions I’ve had with Cumulus—the effort to port over to Intel’s Fulcrum chips is not insignificant. Any work that Intel can do to make that easier/faster/cheaper is all positive in my book. It would be great to see Intel release a DPDK equivalent that is focused on integration into the switching chipsets in their Open Networking Platform (ONP) switch reference architecture (see this post from last year).

Closing Thoughts

Clearly, there’s a lot going on within Intel, as the company works hard—and is being reasonably successful—to differentiate hardware in an environment where abstraction layers like hypervisors and cloud management platforms are trying to homogenize everything. The work that Intel has done (in conjunction with HyTrust) on geofencing is nice and is, I think, an indicator of ways that Intel can continue to innovate beyond just more cores, more efficiency, and faster clock speeds (not that there’s anything wrong with those!).

Stay tuned for more liveblogs from IDF 2014, and I’ll post a day 2 recap as well. Thanks for reading!

Tags: , ,

This is a live blog of session DATS004, titled “Bare-Metal, Docker Containers, and Virtualization: The Growing Choices for Cloud Applications.” The speaker is Nicholas Weaver (yes, that Nick Weaver, who now works at Intel).

Weaver starts his presentation by talking about “how we got here”, discussing the various technological shifts that have affected the computing landscape over the years. Weaver includes a discussion of the drivers behind virtualization as well as the pros and cons of virtualization.

That, naturally, leads to a discussion of containers. Containers are not all that new—Solaris Zones is a form of containers that existed back in 2004. Naturally, the recent hype associated with Docker has, according to Weaver, rejuvenated interest in the concept of containers.

Before Weaver gets too far into containers, he first provides a background of some of the core containerization pieces. This includes cgroups (the ability to control resource allocation/utilization), which is built into the Linux kernel. Namespace isolation is also important, which provides full process isolation (so that one process can’t see processes in another namespace). Namespace isolation isn’t just for processes; there’s also isolation for network entities, mounts, and users. LXC is a set of user-space tools that attempted to make using these constructs easier, but it hasn’t (until recently) been easy to really leverage these constructs.

Weaver next takes this relatively abstract discussion and makes it a bit more concrete with a specific example of how a microservice architecture would look under virtualization (OS instance, microservice libraries, and microservice itself) and well as under containers (shared OS instance and shared libraries plus microservice itself). Weaver talks about the “instant start” attribute of a container, but puts that in the context of the lifetime of the workload that’s running in the container. Start-up times don’t really matter for long-lived workloads, but for temporary, ephemeral workloads start-up times do matter. The pattern of “container on VM” is also mentioned by Weaver as another design pattern that some people use.

Next Weaver provides a quick list of pros and cons of containers:

  • Pros: faster lifecycle vs. virtual machines; containers what is running within the OS; ideal for homogenous application stacks on Linux; almost non-existent overhead
  • Cons: very complex to configure (by itself, absent some sort of orchestration system or operating at scale); currently much weaker security isolation than VMs; applications must run on Linux (because Windows doesn’t have the same container technologies)

Next, Weaver transitions the discussion to focus on Docker specifically. Weaver describes Docker as “an easy button for containers,” making the underlying containerization constructs (cgroups, namespaces, etc.) easier to use. Docker is simpler and easier than LXC (where multiple binaries were involved). Weaver believes that Docker images—which he describes as an ordered set of actions to build a container—are the real game-changer. Weaver’s discussion of Docker images leads to a review of a Dockerfile, which is a DSL (domain specific language) for creating Docker images. Docker images are built on a series of layers; underlying layers could be “just” OS images (like Ubuntu or CentOS), but they could also be customized builds that contain applications and/or data.

Image registries are how users can create images and share images with other users. The public Docker Hub is an example of an image registry.

The discussion now transitions into a quick review of the underlying Docker architecture. There is a Docker daemon that runs on Linux; the Docker client can be run elsewhere. The Docker client communicates with the Docker daemon (although you should note that in many cases the daemon listens on a local socket, which means using a Docker client remotely over the network won’t work).

The innovations that Weaver attributes to Docker include: images (like templates for VMs, and the use of copy-on-write makes them behave like code); API and CLI tools for managing container deployments; reduced complexity around deploying and managing containers; and support for namespaces and resource limits.

Weaver provides a more concrete example of how Docker can change a developer’s process for creating code. Here Weaver’s DevOps background really starts to show, as he discusses how Docker and containers would help streamline CI/CD operations.

Next up are the gotchas with containers. Trust is one gotcha; can we trust that one container won’t affect other containers? The answer, according to Weaver, is “it depends.” You still need to follow current recommended practices, such as no root access, host-level patches, auditing, and being aware of the default settings (which might be dangerous, if you aren’t aware). One way to address some of these concerns is to use VMs to provide strong security isolation between containers that need a stronger level of isolation than the standard container mechanisms can provide.

Intel, of course, is working on making containers better:

  • Security (Intel AES-NI, INtel TXT/TCP, Intel SGX)
  • Performance/flexibility (Intel VT-x/VT-d/VT-c)

Weaver wraps up the session with a quick summary of the key points from the session and some Q&A.

Tags: , , , ,

This is a liveblog of IDF 2014 session DATS002, titled “Virtualizing the Network to Enable a Software-Defined Infrastructure (SDI)”. The speakers are Brian Johnson (Solutions Architect, Intel) and Jim Pinkerton (Windows Server Architect, Microsoft). I attended a similar session last year; I’m hoping for some new information this year.

Pinkerton starts the session with a discussion of why Microsoft is able to speak to network virtualization via their experience with large-scale web properties (Bing, XBox Live, Outlook.com, Office, etc.). To that point, Microsoft has over 100K servers across their cloud properties, with >200K diverse services, first-party applications, and third-party applications. This amounts to $15 billion in data center investments. Naturally, all of this runs on Windows Server and Windows Azure.

So why does networking need to be transformed for the cloud? According to Pinkerton, the goal is to drive agility and flexibility for your business. This is accomplished by pooling and automating network resources, ensuring tenant isolation, maximizing scale/performance, enabling seamless capacity expansion and workload mobility, and minimizing operational complexity.

Johnson takes over here to talk about how Intel is working to address the challenges and needs that Pinkerton just outlined. This breaks down into three core areas that have unique requirements and capabilities: network functions virtualization (NFV), network virtualization overlays (NVO), and software-defined networking (SDN).

Johnson points out that workload optimization is more than just networking; it also involves CPU (E5–2600 v3 CPU family), network connectivity (Intel XL710, now offering support for next-generation Geneve encapsulation), and storage (Intel SSDs). Johnson dives deep on the XL710, which was specifically designed to address some of the needs of cloud networking. Particularly, support for a variety of encapsulation protocols (NVGRE, IPinGRE, MACinUDP, VXLAN, Geneve), support for 40Gbps or 4x10Gbps connectivity in the same card, support for up to 8000 perfect match flow filters stored on die (this is Intel Ethernet Flow Director), and support for SR-IOV and VMDq are all areas where this card helps with NVO and SDN applications.

Next up Johnson walks through some behaviors in traditional networking as compared to network virtualization using an encapsulation protocol. Johnson uses two examples, one with VXLAN and one with NVGRE, but the basics between the two examples are very similar. Johnson also talks about why the stateless offloads in the XL710 (now supporting stateless offloads for both VXLAN and NVGRE, as well as next-generation Geneve) is important; this offloads some amount of work from the host CPU. The impact of network overlays on NIC bonding and link aggregation is another consideration; adapters and switches may not be aware of the encapsulation headers and therefore may not fully utilize all the links in a link aggregation group. The Intel X520/X540 had some offloads; the XL710 increases this support.

That wraps up the NVO portion, and now Johnson switches gears to talk about NFV. According to Johnson, service function chaining (SFC) is a key component of NFV. There are two options for SFC: Network Services Header (NSH), or Geneve. Johnson points out that Geneve was co-authored by Intel, MIcrosoft, VMware, and Red Hat, and is considered to be the next-generation encapsulation protocol. This leads Johnson into a live demo of Geneve and the importance of RSS. (Without RSS, bandwidth is constrained on the receiving system.)

One other key area for support of NFV is being able to transmit large numbers of small packets. This is enabled by Intel’s work on the Data Plane Development Kit (DPDK).

Johnson points out that 40Gbps Ethernet will not offer a BASE-T option; to help address 40Gbps connectivity, Intel is introducing new, low-cost optics (both transceivers and cables). Estimated cost for Intel Ethernet MOC (Modular Optical Connectors) is around $400—well down from costs like $1300 today.

Pinkerton now takes over again, talking about VM density and the changes that have to take place to support higher VM density in private cloud environments (although I would contend that highly virtualized data centers are not private clouds). In particular, Pinkerton feels that SMB3 and SMB Direct (RDMA support) are important developments. According to Pinkerton, these protocols address the need for lower network and storage CPU overhead, higher throughput requirements, lower variances in latency and throughput, better fault tolerance, and VM workload isolation.

Pinkerton insists that using file sharing semantics is actually a much better approach for cloud-scale properties than using block-level semantics (basically, SMB3 is better than iSCSI/FC/FCoE). That leads to a discussion of RDMA (Remote Direct Memory Access), and how that helps improve performance. Standardized implementations of RDMA include iWARP (RDMA over TCP/IP) and RoCE (RDMA over Converged Ethernet). InfiniBand also typically leverages RDMA. In the context of private cloud, having the ability to route traffic is important; that’s why Pinkerton believes that iWARP and RoCE v2 (not mentioned on the slide) are important.

That leads to a discussion of some performance results, and Pinkerton calls out incast performance (many nodes sending data to a single node) as an important metric in private cloud environments. In reviewing some performance metric for using RDMA, Pinkerton states that average latency is no longer satisfactory as a metric—instead, organizations should focus on 95th percentile and 99th percentile measurements instead of average. The metrics Pinkerton is using (based on tests with a Chelsio T580) show latency with SMB3 and RDMA to be very stable up to 90% load, and throughput is near line-rate.

Johnson takes back over now to announce that iWARP support will be built into the next generation of Intel NIC chipsets as a default for server environments.

At this point the session wraps up.

Tags: , , ,

IDF 2014 Day 1 Keynote

This is a liveblog for the day 1 keynote at Intel Developer Forum (IDF) 2014. The keynote starts with an interesting musical piece that shows how technology can be used to allow a single performer to emulate the sound of a full band, and then kicks off with a “pocket avatar” presentation by Brian Krzanich, CEO of Intel Corporation. Krzanich takes the stage in person a few minutes later.

Krzanich starts with a recap of some of the discussions from last year’s IDF, and he points out some of the results over the last year. Among the accomplishments Krzanich lists, he mentions that Intel was the #2 shipper of tablets last year. (One would assume that Apple is #1.) Krzanich clearly believes that Intel has a bright future; he points out that projections show as many as 50 billion x86-based devices by 2020 (just 6 years away). That’s pretty massive growth; there are only an estimated 2.2 billion x86-based devices today.

The line-up today includes talks from Diane Bryant (data center), Kirk Skaugen (clients), Doug Fisher (software and services), and a live Q&A by Krzanich.

Krzanich starts a discussion of wearables and related devices with a mention of the SMS Audio headphones that also provide heart rate monitoring and other fitness data while listening to music. Next up, Krzanich talks about a fashion bracelet that can retrieve textual information from your cell phone and display it. The bracelet was displayed at the opening ceremony of the NYC fashion show last week.

Greg McKelvey of Fossil takes the stage with Krzanich to discuss wearables and Fossil’s experience as a fashion brand and previous wearable technology efforts. McKelvey talks at length about Fossil, but there is very little discussion of specific technology or technology trends.

Krzanich now switches gears to discuss the Internet of Things (IoT), which he believes to be connected to wearables in some ways (“wearables for things”, he calls them). Krzanich believes that IoT will only be successful if you have full intelligence edge-to-edge. Krzanich points out a couple of IoT partnering efforts; one of which involves attaching sensors to HVAC systems to allow for more efficient servicing of the units and another that involves sensors to allow cities to monitor air quality. However, standards are needed, according to Krzanich; he points to two industry consortia involved in standards for IoT. Those consortia are the Open Interconnect Consortium (OIC) and the Industrial Internet Consortium (IIC). OIC is more consumer-focused; IIC is more industrial-focused (as you can guess by the name). The goal of these consortia is to drive standards and interoperability.

At this point Krzanich hands it off to Diane Bryant, SVP and GM of the Data Center Group at Intel. Bryant is here to talk about the data center side of things, and to help set the scale she talks about smartphones. There are 1.9 billion smartphones with an average of 26 apps. If each app does 20 transactions daily, that totals up to a trillion transactions. That scale is massive, but it will be eclipsed by wearables, which by 2020 will include 50 billion devices generation 35 zettabytes of data. Bryant seems to focus on “big data” as the key data center driver, from wearables to health records and health research. Bryant mentions work being done with the Michael J. Fox Foundation (for Parkinson’s research), the Broad Institute (for cancer genomics research), the Francis Crick Institute (for bioinformatics training targeting oncologists), and the Knight Cancer Institute (to help create a 1.2PB cloud for genomics research). Bryant lays out a very ambitious goal to target cancer treatment via genomics by 2020.

Bryant next announces an analytics program for developers called “A-wear” (Analytics for Wearables).

Bryant hands it off to Kirk Skaugen, who will talk about client devices and personal computing. Skaugen believes that wires will be eliminated, passwords will be replaced by bioinformatics, and the personal computing journey will be revolutionized. Skaugen reinforces that Intel is committed to all operating systems, all form factors, and all use cases. Skaugen announces that the Intel Core M, now running on 14nm process, is in full production and will be available on shelves very quickly. This translates into lighter and thinner tablets with greater processing power and lower power utilization. Skaugen shows off a few platforms built on Core M.

Next Skaugen announces Skylake, the code name for the next-generation processor architecture that is anticipated for arrival in the second half of next year. (Will we see it at next IDF?) He shows off a demo of Skylake as well as a hardware reference platform for Skylake playing full 4K video. The roadmap behind Skylake includes continued shrinking from 14nm to 10nm.

Skaugen announces that Samsung is shipping Intel LTE Advanded (Intel XMM 7260) in the Samsung Galaxy Alpha premium smartphone.

After discussion Intel’s work in communications, Skaugen shifts gears to talk about improving the user experience. This involves getting rid of wires, getting rid of passwords, and improving the natural user experiences. Intel has been talking about 3-D interfaces, touch, and voice for the last few years (check the live blogs from previous years), but we have yet to see this vision be realized. They show a demo of setting a laptop (2-in–1 convertible) on a desk and having it automatically connect to peripherals and displays. The demo also includes wireless charging.

Skaugen wraps up his portion of the session with a review of the announcements before handling the baton to Doug Fisher to talk about software and services. Fisher talks about how Intel is partnering with a number of other companies to help developers deliver software faster. Fisher announces a new reference platform for Android based on Intel technology.

Next Fisher shows off a demo of 3-D camera technology and the associated software that provide additional information and context about data captured in photos (like taking a photo of a crate, capturing the location and dimensions of the crate, and linking to shipping systems to ship the crate). Fisher reinforces Krzanich’s mention of OIC and Intel’s participation in creating open, royalty-free standards for compatibility and interoperability among the billions of devices in the IoT. Intel is, in fact, delivering open source code (under the Apache 2.0 license) and contributing relevant patents to the OIC.

Fisher now turns it back over to Krzanich, who shows off the first production model of Intel RealSense 3-D camera/imaging technology. This technology captures not only visual data, but also distance, depth, motion effects, etc. Krzanich shows a live demo of RealSense depth awareness and how that information can be used to enable new filtering effects in software (like removing the color from everything except the closest layer). Krzanich brings Michael Dell up on the stage to talk about the device he used during the demo, which is a Dell Venue 8 7000 series tablet—supposedly the world’s thinnest tablet at just 6mm. Dell shows off a few more examples of using Intel’s RealSense technology to measure distance and change focal points of photos.

Dell also makes mention of new PowerEdge servers, including the R720XD, a 2U box with 36 cores, up to 1.5TB of RAM, and up to 100TB of storage.

At this point Krzanich transitions to a Q&A session with Renee James, President of Intel. This is a live Q&A session, which is kind of nice (and not very common for a conference keynote like this). At this point I’m wrapping up the liveblog instead of trying to capture the live Q&A (which can be difficult at times).

Tags: , ,

Welcome to Technology Short Take #44, the latest in my irregularly-published series of articles, links, ideas, and thoughts about various data center-related technologies. Enjoy!

Networking

  • One of the original problems with the VXLAN IETF specification was that it (deliberately) didn’t include any control plane information; as a result, the process of mapping MAC addresses to VTEPs (VXLAN Tunnel Endpoints) wasn’t defined, and the early implementations relied on multicast to handle this issue. To help resolve this issue, Cumulus Networks (and possibly Metacloud, I’m not sure of their involvement yet) has release an open source project called vxfld. As described in this Metacloud blog post, vxfld is designed to “handle VXLAN traffic from any operationg system or hardware platform that adheres to the IETF Internet-Draft for VXLAN”.
  • Nir Yechiel recently posted part 1 of a discussion on the need for network overlays. This first post is more of a discussion of why VLANs and VLAN-based derivatives aren’t sufficient, and why we should be looking to routing (layer 3) constructs instead. I’m looking forward to part 2 of the series.
  • One ongoing discussion in the network industry these days—or so it seems—is the discussion about the interaction between network overlays and the underlying transport network. Some argue that tight integration is required; others point to streaming video services and VoIP running across the Internet and insist that no integration or interaction is needed. In this post, Scott Jensen argues in favor of the former—that SDN solutions shouldn’t just manage network overlays, but should also manage the configuration of the physical transport network as well. I’d love to hear from more networking pros (please disclose company affiliations) about their thoughts on this matter.
  • I like the distinction made here between network automation and SDN.
  • Need to get a better grasp on OpenFlow? Check out OpenFlow basics and OpenFlow deep-dive.
  • Here’s a write-up on connecting Docker containers using VXLAN. I think there’s a great deal of promise for OVS in containerized environments, but what’s needed is better/tighter integration between OVS and container solutions like Docker.

Servers/Hardware

  • Is Intel having second thoughts about software-defined infrastructure? That’s the core question in this blog post, which explores the future of Intel in a software-defined world and the increasing interest in non-x86 platforms like ARM.
  • On the flip side, proponents who claim that platforms like ARM and others are necessary in order to move forward with SDN and NFV initiatives should probably read this article on 80 Gbps performance from an off-the-shelf x86 server. Impressive.

Security

  • It’s nice to see that work on OpenStack Barbican is progressing nicely; see this article for a quick overview of the project and an update on the status.

Cloud Computing/Cloud Management

  • SDN Central has a nice write-up on the need for open efforts in the policy space, which includes the Congress project.
  • The use of public cloud offerings as disaster recovery targets is on the rise; note this article from Microsoft on how to migrate on-premises workloads to Azure using Azure Site Recovery. VMware has a similar offering via the VMware vCloud Hybrid Service recovery-as-a-service offering.
  • The folks at eNovance have a write-up on multi-tenant Docker with OpenStack Heat. It’s an interesting write-up, but not for the faint of heart—to make their example work, you’ll need the latest builds of Heat and the Docker plugin (it doesn’t work with the stable branch of Heat).
  • Preston Bannister took a look at cloud application backup in OpenStack. His observations are, I think, rational and fair, and I’m glad to see someone paying attention to this topic (which, thus far, I think has been somewhat ignored).
  • Interested in Docker and Kubernetes on Azure? See here and here for more details.
  • This article takes a look at Heat-Translator, an effort designed to provide some interoperability between TOSCA and OpenStack HOT documents for application deployment and orchestration. The portability of orchestration resources is one of several aspects you’ll want to examine as you progress down the route of fully embracing a cloud computing operational model.

Operating Systems/Applications

  • Looks like we have another convert to Markdown—Anthony Burke recently talked about how he uses Markdown. Regular readers of this site know that I do almost all of my content generation using MultiMarkdown (a variation of Markdown with some expanded syntax options). Here’s a post I recently published on some useful Markdown tools for OS X.
  • Good to see that Ivan Pepelnjak thinks infrastructure as code makes sense. I guess that means the time I’ve spent with Puppet (you can browse Puppet-related posts here) wasn’t a waste.
  • I don’t know if I’ve mentioned this before (sorry if that’s the case), but I’m liking this “NIX4NetEng” series going on over at Nick Buraglio’s site (part 1, part 2, and part 3).
  • Mike Foley has a blog post on how to go from zero to Windows domain controller in only 4 reboots. Handy.

Storage

Virtualization

  • Running Hyper-V with Linux VMs? Ben Armstrong details what versions of Linux support the various Hyper-V features in this post.
  • Here’s a quick write-up on running VMs with VirtualBox 4.3 on a headless Ubuntu 14.04 LTS server.
  • Nested OS X guest on top of nested ESXi on top of VMware Fusion? Must be something William Lam’s tried. Go have a look at his write-up.
  • Here’s a quick update on Nova-Docker, the effort in OpenStack to allow users to deploy Docker containers via Nova. I’m not yet convinced that treating Docker as a hypervisor in Nova is the right path, but we’ll see how things develop.
  • This post is a nice write-up on the different ways to connect a Docker container to a local network.
  • Weren’t able to attend VMworld US in San Francisco last week? No worries. If you have access to the recorded VMworld sessions, check out Jason Boche’s list of the top 10 sessions for a priority list of what recordings to check out. Or need a recap of the week? See here (one of many recap posts, I’m sure).

That’s it this time around; hopefully I was able to include something useful for you. As always, all courteous comments are welcome, so feel free to speak up in the comments. In particular, if there is a technology area that I’m not covering (or not covering well), please let me know—and suggestions for more content sources are certainly welcome!

Tags: , , , , , , , , , , , , ,

This is a semi-liveblog of the day 2 keynote at VMworld 2014 in San Francisco.

Ben Fathi, CTO of VMware, takes the stage to start the keynote, and goes through a recap of yesterday’s announcements: EVO and EVO:RAIL, vCloud Air rebranding and the availability of new services, VMware Integrated OpenStack (VIO), and new product versions (vCloud Suite 5.8, betas of vSphere and VSAN).

Fathi re-emphasizes that VMware’s SDDC vision is an architecture, one built on the “power of AND.” vCloud Suite (and vCloud Air) is a manifestation of the SDDC vision/architecture. VMware’s SDDC vision/architecture delivers the power of “AND.”

Fathi transfers the baton to Sanjay Poonen, EVP for End-User Computing at VMware. Poonen is a high-energy speaker, and his enthusiasm for end-user computing is clear (and contagious). The world of end-user computing is changing; not only users are changing, but also apps and data are changing. Poonen talks about VMware’s EUC vision: enabling users to work “at the speed of life.” Today’s environments are far more heterogenous than in the past. Poonen uses an example of a rooftop infinity pool in Singapore as a metaphor for the “three foundations” of EUC: SDDC, workspace services (authentication and content collaboration, for example), and then the desktop/mobile/cloud experience.

Poonen goes on to talk about VMware’s desktop leadership. Horizon delivers a unified application publishing and VDI solution. Next is Desktop-as-a-Service (DaaS), via Desktone, and Poonen mentions DaaS via vCloud Air. Next Poonen talks about real-time application delivery via the CloudVolumes acquisition. Finally, Poonen discusses the rich user experience, and he announces a partnership between Google, NVIDIA, and VMware, and rolls into a video about the results of the partnership. The partnership enables users of VMware Horizon View on VMware vSphere on next-generation Google Chromebooks to utilize massively graphics-intensive applications with outstanding performance. (This is a pretty cool announcement, IMHO.)

The second major pillar that Poonen discusses is enterprise mobility management. It’s not just about application management, but also about content management, email management, and device management. Poonen brings onto stage Kevin Ichhpurani from SAP, to discuss a partnership between VMware and SAP. The partnership brings together AirWatch and SAP Secure Mobile for the secure delivery of applications to mobile devices.

The third major pillar that Poonen discusses is content collaboration. This means “anytime/anywhere” access, hybrid deployment, enterprise-grade functionality/support. Poonen talks about how United Airlines is transforming their operations uses mobile devices (supported by Apple devices and AirWatch by VMware). Poonen talks about all the integrations that are present: integrations between SDDC and workspace services, integration between desktop services and content collaboration. Poonen revisits the VMware Workspace Suite announcement from yesterday, drawing a comparison between vCloud Suite for SDDC and the Workspace Suite for EUC.

Poonen mentions that VMware is being designated as the leader by GigaOm and Radicati Group.

At this point, Poonen brings out Kit Colbert, CTO for EUC at VMware. Colbert is well-known to the VMworld crowd and is a veteran at VMware. Colbert starts his discussion using the idea of a “mobile cloud architecture” as the basis for the discussion. Colbert starts the first demo of the day by framing the demo around the concept of a doctor moving through various devices and applications over the course of a day. The demo spans a Mac in the doctor’s office, an iPad that he carries with him, and a thin client leveraging rich 3D graphics functionality. AirWatch Secure Content Locker facilitates some fictional collaboration between Colbert and Poonen (who is also a doctor in this example) across different security zones. The summary of the demo is that all the applications were in one place, data followed the user, and 3D images were available on any device.

Next Colbert takes the audience into a demo of CloudVolumes, and how CloudVolumes makes application delivery fast and seamless (like provisioning applications via AirWatch to a mobile device). In the demo, Colbert provisions Office to a VDI desktop almost instantaneously.

Application delivery is important, but desktop delivery and desktop provisioning is also important. Colbert next dives into more detail on Project Fargo, which enables vastly faster provisioning of virtual desktops by via cloning process. The clone is very lightweight because it shares not only disk but also memory with the original. Colbert begins to tie things today: VMware uses Project Fargo to quickly clone the desktop, and then use CloudVolumes to attach applications to that desktop. When the desktop is no longer needed, the desktop is discarded. This radically simplifies the administration and management of desktops and applications. Colbert name-drops a new project name—Project Meteor—but doesn’t go into great level of detail on exactly what Meteor encompasses.

Poonen and Colbert turn the stage over to Raghu Raghuram, who heads up VMware’s SDDC efforts. Raghuram reviews the achievements of the core components of the SDDC architecture—compute (via vSphere, including the vSphere 6.0 beta announced yesterday), storage (via VSAN), networking (via NSX—go VCDX-NV!), and management (via the newly-rebranded vRealize suite). Once again, Raghuram talks about SDDC as “the power of AND”.

Raghuram next transitions into a review of the “three paths” to SDDC that were introduced by Gelsinger yesterday: build your own, converged infrastructure, and hyper-converged infrastructure. Naturally, this leads to a discussion of the EVO family, especially EVO:RAIL. Fathi takes the stage again and prompts Raghuram into discussing how quickly EVO:RAIL can be turned up (up and running in 15 minutes, scales up to 100 VMs or 250 desktop VMs, supports non-disruptive upgrades), and that leads into a demo of EVO:RAIL. The hardware specification for EVO:RAIL is a 2U, four-node enclosure (think Dell C6100 or similar hardware form factor). EVO:RAIL building blocks will automatically discover each other, allowing users to easily build 16 node clusters (four EVO:RAIL blocks). EVO:RAIL also offers a greatly simplified web-based UI, although the vSphere Web Client is still available for use. EVO:RAIL includes vSphere and VSAN, and includes Log Insight as well.

EVO:RACK is a larger-scale implementation that builds on EVO:RAIL and includes all of the vCloud Suite. It starts at half a rack, and scales to multiple racks (no mention I caught of exactly how many racks). VMware is shooting for being up and running deploying applications in less than 2 hours for EVO:RACK. EVO:RACK, of course, is still in technical preview.

Next Raghuram moves onto talking about VMware Integrated OpenStack (VIO), which was announced yesterday as being available in beta. Over the next few minutes he discusses VMware’s involvement in OpenStack, and reviews why running OpenStack on VMware is the best choice for deploying OpenStack today. It doesn’t hurt that VMware offers a single point of contact for deploying OpenStack on VMware using VIO, although VMware continues to partner with companies like Canonical, HP, SuSE, and Mirantis for OpenStack deployments as well. (HP announced yesterday that HP Helion will support vSphere and NSX.)

From there, Raghuram transitions to a discussion of traditional applications and next-generation applications. This leads to mention of support for 4 vCPUs with VMware Fault Tolerance in the vSphere 6 beta. (This expansion of support is a long time coming.) Cross-vCenter vMotion is also coming in vSphere 6, as well as long-distance vMotion (coast-to-coast vMotion mentioned as an example). Raghuram transitions to Fathi for a discussion of next-generation applications.

Fathi talks about the use of microservices as the basis for next-generation applications, which clearly indicates he will be talking about containers. Fathi paints containers as “having been around for years,” including pointing to a feature in Windows Data Center Server 15 years ago. However, Docker has clearly captured the attention of developers, and so Fathi talks about where Docker fits. This leads into the “containers without compromise” message, which means that running containers on VMs on VMware’s SDDC architecture is the best way to deliver containerized applications and microservices today. Fathi mentions VMware working closely with Docker and Kubernetes, and explains the concept of a Kubernetes pod as a collection of containers. The value that VMware offers in a Docker/Kubernetes environment includes persistent storage via VSAN, pod mobility via vMotion/DRS, enhanced security via the VM operational model, and the rich ecosystem of tools supporting VMware virtualized infrastructure. Fathi re-iterates that SDDC is a single platform for running traditional applications as well as next-generation applications, and reminds attendees that CloudFoundry has supported containers (via Warden) for a number of years. (In other words, containers are not new to VMware.) VMware announces today that VMware is working with Docker, Google, and Pivotal to make containers a “first-class” citizen in the VMware SDDC architecture.

This leads into a demo by Fathi showing the use of vCAC (vRealize Automation?) deploying a blueprint. The demo shows CoreOS and Kubernetes deploying Vulcan, which is a service registration/service discovery tool. Raghuram asks about Project Fargo, and Fathi talks about how Fargo can help make containers even better than on bare metal through the use of technologies like those in Project Fargo.

Next up, Raghuram talks about the vRealize suite of management applications and tools. vRealize will be available as an on-premises implementation as well as available via vCloud Air in a SaaS model. I think that the value of this second approach is possibly being overlooked by a good number of conference attendees, but it is quite a significant announcement in my humble opinion.

Fathi takes over again to discuss the need for policy in the SDDC architecture. Policy can (and should) be applied not only to networking, but also to storage, compute, and other areas. This leads into a demo, hosted within vCAC (vRealize Automation?), and shows how various policies like storage policies are implemented to affect how applications are deployed within an SDDC architecture. (There is, of course, another side of policy that isn’t being mentioned here, and that’s what Congress attempts to address. But that’s another story for another day.) There is a nice demo of integrations between vCAC and NSX, where vCAC creates NSX security groups (with associated firewall rules) on-demand as part of a blueprint. That’s nice, and will be quite useful I think.

Speaking of NSX, Fathi jumps into a discussion of microsegmentation as a key value-add for VMware NSX in data centers. While microsegmentation was possible before VMware NSX, the value that NSX offers is that microsegmentation now becomes more manageable and more operationally feasible than it’s been in the past. Fathi reminds the audience that this functionality can be driven by policy.

Raghuram summarizes the various demonstrations and announcements, and leaves the stage. Fathi then invites Simone Brunozzi, VP and Chief Technologist for Hybrid Cloud. Brunozzi uses his Google Glass as an excuse to show off a mobile application he uses to “check” on his data center. This leads into a demo that shows vCenter Operations Manager to show the health of vCloud Air resources as well as VMware NSX resources—this shows the expanding reach of vCenter Operations Manager and the tight integration between the various product lines that VMware is advancing. This leads into a demonstration of extending a network from your own data center into vCloud Air, and then migrating workloads (via a copy operation) to vCloud Air directly from within the vSphere Web Client. That’s nothing new, according to Fathi, but what is new is the coordination between firewall rules in NSX and firewall rules in vCloud Air to enable full connectivity between on-premises workloads and workloads in vCloud Air.

Brunozzi next demos using vSphere Replication to replicate to a cloud provider for cloud-based recovery. There was no mention I caught of which providers support this, though I’m sure that vCloud Air via Recovery-as-a-Service does.

At this point I needed to leave to get ready for a session, so I wasn’t able to capture the rest of the keynote. I hope that what I was able to capture was useful—feel free to share any corrections or additions in the comments below. Thanks!

Tags: , , , ,

This is a liveblog of the day 1 keynote at VMworld 2014 in San Francisco, CA. (Well, it’s a “semi-liveblog”; I’ll post this immediately after the keynote session wraps up.) As in previous years, the keynote is being held in the massive space in Moscone North, and this year—like previous years—it looks like there is quite a crowd gathered here this morning. I have the privilege of sitting with newly-minted VCDX-NVs and other VCDXes.

The keynote starts with a catchy, jazzy dance/music piece, and then Robin Matlock, Chief Marketing Officer for VMware, takes the stage. Matlock shares that over 22,000 people are gathered here for VMworld, which (if I recall correctly) is down a little bit from last year. Matlock talks about change, and how change can be either a barrier or an opportunity. That leads into a discussion of the building of the Golden Gate Bridge as an allegory for pushing boundaries, exploring endless possibilities, and expanding your knowledge and expertise. This fits into the “No Limits” theme for the conference this week.

After discussing “Operation Giveback” (a charity fundraiser backed by the VMware Foundation), Matlock turns the stage over to VMware’s CEO, Pat Gelsinger, who comes onto the stage in a dramatic “breaking through” effect.

Gelsinger starts his portion of the keynote by introducing the “liquid world,” where change is paramount. Gelsinger provides some examples of various businesses that have embraced the “liquid business” model. The biggest risk to success, according to Gelsinger, is perpetuating the status quo. Change is inevitable. The best way to predict the future is to invent it, and Gelsinger says that’s what brave IT leaders need to do. This leads Gelsinger into a discussion of what the word “brave” means, and that in turn leads to a family-friendly discussion of the first day of school, a kid’s penalty shot in soccer, a driving test, or a public speaking event. Next up is a “who’s who” list of brave business leaders, like Elon Musk and the leader of 23andMe (I didn’t catch the spelling for her name). VMware is on a voyage of brave transformation itself, and Gelsinger provides examples of VMware’s BYOD efforts and running all of VMware’s SAP ERP efforts on vCloud Air.

Gelsinger brings the “fluid” and “brave” themes together in that brave IT involves fluidity and choice (and a third item I couldn’t catch). This leads into a discussion of the software-defined data center (SDDC). As a result of this discussion, Gelsinger goes into some announcements:

  • VMware is announcing VMware vCloud Suite 5.8, which includes new suite “fit and finish” changes
  • VMware is announcing vSphere 6.0 beta
  • VMware is announcing VSAN 2.0 beta
  • As part of the vSphere 6.0 beta, VMware is also announcing Virtual Volumes (VVols)
  • VMware is rebranding the management suite as VMware vRealize Suite

The path to SDDC has three branches: “build your own”, converged infrastructure, and hyper-converged infrastructure. This leads Gelsinger to announce VMware EVO, which is VMware’s family of hyper-converged infrastructure offerings. EVO is the SDDC packaged with hardware to provide the fastest way to realize the benefits of the SDDC. The first member of the EVO family is EVO:RAIL, which is a “building” block designed at around 100 VMs of capacity and is designed to be up and running in 15 minutes. Hardware partners for EVO:RAIL include Dell, EMC, Fujitsu, Inspur, Net One, and Supermicro.

The EVO Zone, found in the Solutions Exchange, has some examples of EVO products being brought to market.

The second member of the EVO family is EVO:RACK, which is being announced as a technical preview. EVO:RACK builds on EVO:RAIL for greater scale—basically building an “entire cloud” in 2 hours or less. There will be a range of certified hardware solutions available as EVO:RACK solutions. VMware is also joining the OpenCompute Project, and anticipates that EVO:RAIL and EVO:RACK will be available on OCP hardware.

Gelsinger also announces VMware Integrated OpenStack (VIO), VMware’s own OpenStack distribution that will incorporate key VMware technologies like vSphere, VSAN, and NSX. VIO is available in beta.

Continuing the discussion around next-generation platforms, Gelsinger announces partnerships with Docker, Google, and Pivotal around building open container APIs and making containers run better on VMware’s SDDC platform—”containers without compromise.”

That leads Gelsinger into a discussion of security, and that naturally involves VMware NSX. Microsegmentation is becoming a key use case for customers adopting VMware NSX.

After a brief discussion of end-user computing—admittedly brief because Sanjay Poonen will be discussing this in more detail in tomorrow’s keynote—Gelsinger moves on to a discussion of hybrid cloud. VMware vCloud Hybrid Service has been rebranded to vCloud Air, and all VMware products that are offered “as a service” will be rebranded as part of the Air family. Gelsinger brings Bill Fathers, EVP & GM for Hybrid Cloud at VMware, onto the stage.

Fathers recaps what’s happened with vCloud Air over the last 12 months; the service was officially launched last year in the day 1 keynote at VMworld 2013. vCloud Air has been successful in meeting all of the goals originally announced. Fathers reviews the key benefits of using public cloud, especially VMware’s public cloud: agility, efficiency, and cost savings. Fathers discusses the various phases of cloud adoption, and states that he believes the market is currently transitioning from the experimental phase to the professional era.

For companies that are being successful transitioning into the professional era—which involves much greater use of public cloud resources—the two things these companies are doing is 1) thinking of the apps first; and 2) putting the IT professionals back in charge. According to Fathers, vCloud Air provides the perfect platform for accomplishing both of these goals, due in no small part because vCloud Air uses the same hypervisor, same VM formats, and same management tools as companies are using today in their private clouds.

Fathers announces a government-focused FedRAMP-compliant vCloud Air offering launching soon (September); this will be very useful for US government-focused partners, service providers, and systems integrators. VMware is also expanding the number of vCloud Air partners in the vCloud Air Network, with 3900 partners available around the world in over 100 countries. vCloud Air Network is, according to Fathers, the world’s largest hybrid cloud network.

Next Fathers moves into a discussion of new services and offerings available with vCloud Air. These new services include continuous integration as a service offering (CIaaS?), MySQL and MS SQL database-as-a-service offering (will later expand to all popular RDBMS, and will offer DR services for on-premises databases), object storage service based on EMC ViPR, AirWatch and Pivotal CloudFoundry mobile services, and vRealize Air Automation (which is hosted cloud management tools available as a SaaS). Fathers also announcing a beta of on-demand, PAYG (pay as you go) access to vCloud Air.

Fathers transitions back to Gelsinger, who wraps up the themes and announcements from the keynote, and introduces Carl Eschenbach, President and COO of VMware.

Eschenbach reinforces that the “engine of innovation” is alive and well at VMware, but it is the brave who drive change (referring to the VMworld attendees in the audience). That leads into a customer discussion between Eschenbach with three key customers: MedTronic, MIT IST (Information Systems and Technology), and Ford. Each of the customers comes out on the stage and has a brief conversation with Eschenbach.

At this point I had to leave to prepare for a session, so I wasn’t able to capture any of the rest of the keynote.

Tags: , ,

In this post, I’ll share a simple template for deploying Docker containers in an OpenStack environment using Heat. Given that Docker is targeted at application deployment, then I felt that using Heat was a more appropriate way of leveraging Docker in an OpenStack environment as opposed to treating Docker as a form of a hypervisor. Later in this post, I’ll compare this approach to using a more container-aware solution such as fleet.

I assume you’re already familiar with OpenStack Heat and Docker. If you aren’t, take a look at these articles first:

Prerequisites

Before you can actually use Heat to orchestrate Docker containers, there are some prerequisites you’ll need to have done first:

  1. You’ll need to have the Docker plugin for Heat installed. This can be tricky; see here for some instructions that worked for me. To verify that the Docker plugin is working as expected, run heat resource-type-list and check the output for “DockerInc::Docker::Container”. If that resource type is included in the output, then the Docker plugin is working as expected.
  2. Any Docker hosts you’re running must have Docker configured to listen on a network-accessible socket. I was running CoreOS in my environment, so I followed the instructions to make Docker on CoreOS listen on a TCP socket. (In case the link doesn’t take you to the right section, see the section titled “Enable the Remote API on a New Socket.”) In my case, I selected TCP port 2345. Make note of whatever port you select, as you’ll need it in your template.
  3. Any Docker hosts that will be orchestrated by Heat must have an IP address assigned that is reachable from the server where Heat is running (typically the cloud controller). In my case, I use Neutron with NSX, so I had to assign floating IPs to the instances with which Heat would be communicating.
  4. You’ll need to be sure that the TCP port you select for Docker to use (I used TCP port 2345) is accessible, so modify any security groups assigned to the instances to allow inbound TCP traffic on that port from the appropriate sources.

Once these prerequisites are addressed—Docker plugin installed and working, Docker listening on a TCP port, instance reachable from cloud controller on selected TCP port—then you’re ready to go.

Template for Docker Orchestration

Here is a sample template that will create a Docker container on an existing instance:

(Click here if you don’t see the code block above.)

As I said, this is pretty simple. The image property is the name of the Docker image you want to use; in this case, I’m using an image containing the popular Nginx web server. The docker_endpoint property should be a URL that specifies the protocol (TCP), IP address (in my case, a floating IP address assigned to the instance), and the port number on which the Docker daemon is listening. Note that the format for this property isn’t documented anywhere I’ve found.

In the “stable/icehouse” branch of the Docker plugin (required if you’re using distro packages for your OpenStack installation, as I am), there are some additional properties available as well. Unfortunately, without any documentation on what these properties should look like, I was unable to make it work with any of those properties included. In particular, the port_specs property, which controls how ports in a Docker container are exposed to the outside world, would have been very useful and applicable. However, I was unable to make it work with the port_specs attribute included. If anyone has information on the exact syntax and format for the port_specs property in the “stable/icehouse” branch of the plugin, please speak up in the comments.

Naturally, you could embed this portion of YAML code into a larger HOT-formatted template that also launched instances, created Neutron networks, attached the instances to Neutron networks, created a logical router, and mapped a floating IP address to the instance. I leave the creation of such a template as an exercise for the reader, but I will point out that I’ve already shared with you almost all the pieces necessary to do exactly that. (See the blog posts I provided earlier.)

Summary

I mentioned at the start of this post that I’d provide some comparison to other methods for deploying containers in an automated fashion. With that in mind, here are a few points you’ll want to consider:

  • There is no container scheduling in this solution. Containers are statically mapped to a container host (the VM instance, in this case, although this could be a bare metal host running Docker as well). Other solutions, like fleet, at least let you just point to a cluster of systems instead of a specific system. (See this write-up on fleet for more information.)
  • Docker must be listening on a TCP socket. This isn’t Docker’s default configuration, so this is an additional change that must be incorporated into the environment. Fleet doesn’t have this requirement, although other solutions such as Mesos might (I haven’t tested any other solutions—yet.)
  • There is very little documentation available right now. Note that this may be true for other solutions as well (this entire space is relatively new and growing/evolving rapidly). Regardless, until someone can at least figure out how to expose Docker containers to the network via a Heat template, this isn’t very useful.

My initial assessment is that OpenStack needs container scheduling, not static assignment, in order for Docker integration into OpenStack to be truly useful. Proponents of the Nova-Docker approach (treating Docker as a hypervisor and Docker images as Glance images) point to their approach as superior because of the integration of Nova’s scheduling functionality. It will be interesting to see how things develop on this front.

If you have any questions, have more information to share, or have corrections or clarifications to any of the information presented here, please speak up in the comments.

Tags: , , ,

CoreOS Continued: Fleet and Docker

This post is the third in a series of posts on CoreOS, this time focusing on the use of fleet and Docker to deploy containers across a cluster of systems. This post builds on my earlier introduction to CoreOS and the subsequent more in-depth look at etcd.

I’m assuming that you’re already reasonably familiar with CoreOS, etcd, and Docker. If you aren’t familiar with CoreOS or etcd, have a look at the links in the previous paragraph. If you need a quick introduction to Docker, check out this quick introduction to Docker. While the example I’m going to provide here is fairly simple, it should serve as a reasonable basis upon which to build later.

An Overview of Fleet

The GitHub page for fleet describes it as a “distributed init system” that operates across a cluster of machines instead of on a single machine. It leverages etcd, the distributed key-value store that ships with CoreOS, as well as systemd. Fleet combines etcd and systemd to allow users to deploy containers (configured as systemd units) across a cluster of CoreOS systems.

Using fleet, users can deploy a single container anywhere on the cluster, deploy multiple copies of the same container, ensure that containers run on the same machine (or different machines), or maintain a certain number of instances of a service (thus protecting against failure).

Note that even though fleet helps with scheduling containers across a cluster of systems, fleet doesn’t address some of the other significant challenges that arise from an architecture based on distributed micro-services in containers. Namely, fleet does not address inter-container communications, service registration, service discovery, or any form of advanced utilization-based scheduling. These are topics I hope to be able to explore here in the near future.

Now that you have an idea of what fleet does, let’s take a closer look at actually using fleet.

Interacting with Fleet

By default, the fleetctl command-line client that is provided to interact with fleet assumes it will be interacting with a local etcd endpoint on the loopback address. So, if you want to run fleetctl on an instance of CoreOS in your cluster, no further configuration is needed.

However, it may be easier/more effective to use fleetctl from outside the cluster. There are a couple of different ways to do this: you can tell fleetctl to use a specific endpoint, or you can tunnel the traffic through SSH. Each approach has advantages and disadvantages; I’ll leave it to the readers to determine which approach is the best approach for their specific configurations/situations.

Using a Custom Endpoint

This method is pretty straightforward and simple. Just set an environment variable named FLEETCTL_ENDPOINT, like this:

export FLEETCTL_ENDPOINT=http://10.1.1.7:4001

Obviously, you’d want to make sure that you have the correct IP address (can be any node in the etcd cluster) and port (4001 is the default, I believe). With this environment variable set, now anytime you use fleetctl it will direct traffic to the endpoint you specified. If that specific node in the etcd cluster becomes unavailable, then fleetctl will stop working, and you’ll need to point it to a different node in the cluster.

Tunneling Through SSH

The second way of using fleetctl remotely is to tunnel the traffic through SSH. This method may be a bit more complicated, but naturally offers a bit more security.

To make fleetctl tunnel its communications with etcd through SSH, set an environment variable called FLEETCTL_TUNNEL to the IP address of any node in the etcd cluster, like this:

export FLEETCTL_TUNNEL=10.1.1.7

However, the configuration involves more than just setting the environment variable. The fleetctl doesn’t expose any options to configure the SSH connection, and it assumes you’ll be using public key authentication. This means you’ll need access to a public key that will work against the nodes in your etcd cluster. If you followed my instructions on deploying CoreOS on OpenStack via Heat, then you can review the Heat template to see which key was specified to be injected when the instances were spawned. Once you know which key was used, then you’ll need to either:

  • place that key on the system where fleetctl is installed, or
  • install fleetctl on a system that already has that key present.

There’s still at least one more step required (possibly two). Because fleetctl doesn’t expose any SSH options, you’re going to need to run an SSH agent on the system you’re using. OS X provides an SSH agent by default, but on Linux systems you will probably have to manually run an SSH agent and add the appropriate SSH key:

eval `ssh-agent -s`
ssh-add ~/.ssh/keyfile.pem

Once the SSH agent is running and the appropriate key is loaded (you’d clearly need to make sure the path and filename are correct in the command listed above), then the last step is to configure your ~/.ssh/config file with options for the CoreOS instances. It’s possible you might be able to get by without this step; I haven’t conducted enough testing to say with absolute certainty one way or another. I suspect it will be needed.

In the ~/.ssh/config file, add a stanza for the system through which you’ll be tunneling the fleetctl traffic. The stanza will need to look something like this:

Host node-01
  User core
  Hostname 10.1.1.7
  IdentityFile ~/.ssh/keyfile.pem

This configuration stanza ensures that when the system you’re using attempts to communicate with the IP address listed above, it will use the specified username and public key. Since the SSH agent is loaded, it won’t prompt for any password for the public key (even if the public key doesn’t have a password associated, you’ll still need the SSH agent), and the SSH connection will be successful without any user interaction. That last point is important—fleetctl doesn’t expose any SSH options, so the connection needs to be completely automatic.

Once you have all these pieces in place, then you can simply run fleetctl with the appropriate commands (described in the next section), and the connection to the etcd cluster will happen over SSH via the specified host. Naturally, if that node in the cluster goes away or is unavailable, you’ll need to point your connection to a different node in the etcd cluster.

Using Fleet

Once you have access to the etcd cluster via fleetctl using one of the three methods described above (direct access via a CoreOS instance, setting a custom endpoint, or tunneling over SSH), then you’re ready to start exploring how fleet works.

First, you can list all the machines in the cluster with this command:

fleetctl list-machines

Note the “METADATA” column; this allows you to do some custom scheduling by associating systemd units with specific metadata parameters. Metadata can be assigned either via cloud-config parameters passed when the instance is spawned, or via modifications to the fleet config files.

To see the units about which the cluster knows, use this command:

fleetctl list-units

If you’re just getting your etcd cluster up and running, the output of this command is probably empty. Let’s deploy a unit that spawns a Docker container running the popular Nginx web server. Here’s a (very) simple unit file that will spin up an Nginx container via Docker:

(If you can’t see the code block above, click here.)

With this file in place on the system where you are running fleetctl, you can submit this to the etcd cluster with this command:

fleetctl submit nginx.service

Then, when you run fleetctl list-units, you’ll see the new unit submitted (but not started). Start it with fleetctl start nginx.service.

Where fleet becomes really useful (in my opinion) is when you want to run multiple units across the cluster. If you take the simple Nginx unit I showed you earlier and extend it slightly, you get this:

(Click here if you can’t see the code block above.)

Note the difference here: the Docker container name is changed (to nginx-01) and the filename is different (now nginx.1.service). If you make multiple copies of this file, changing the Docker container name and the unit filename, you can submit all of the units to the etcd cluster at the same time. For example, let’s say you wanted to run 3 Nginx containers on the cluster. Make three copies of the file (nginx.1.service, nginx.2.service, and nginx.3.service), modifying the container name in each copy. Make sure that you have the “X-Conflicts” line in there; that tells fleet not to place two Nginx containers on the same system in the cluster. Then submit them with this command:

fleetctl submit nginx.*.service

And start (launch) them with this command:

fleetctl start nginx.*.service

Give it a few minutes to download the latest Nginx Docker image (assuming it isn’t already downloaded), then run fleetctl list-units and you should see three Nginx containers distributed across three different CoreOS instances in the etcd cluster, all listed as “loaded” and “active”. (You can then test connectivity to those Nginx instances using something like curl.) Congratulations—you’ve just deployed multiple containers automatically across a cluster of systems!

(Want to see some of the magic behind fleet? Run etcdctl --peers <IP address of cluster node>:4001 ls /_coreos.com --recursive and see what’s displayed. You’re welcome.)

Admittedly, this is a very simple example. However, the basic architecture I’ve shown you here can be extended. For example, by using additional fleet-specific properties like “X-ConditionMachineOf” in your unit file(s), you can run what is known as a “sidekick container.” These containers do things like update an external load balancer, or register the presence of the “primary” container in some sort of service discovery mechanism. (In fact, as I alluded to in my etcd post, you could use etcd as that service discovery mechanism.)

Naturally, fleetctl includes commands for stopping units, destroying units, etc., as well as submitting and starting units. You can use fleetctl help to get more information, or visit the fleet GitHub page.

I hope you’ve found this post to be helpful. Feel free to post any questions, corrections, clarifications, or thoughts in the comments below. Courteous comments are always welcome.

Tags: , , ,

« Older entries § Newer entries »