Scott's Weblog The weblog of an IT pro focusing on cloud computing, Kubernetes, Linux, containers, and networking

Technology Short Take 121

Welcome to Technology Short Take #121! This may possibly be the last Tech Short Take of 2019 (not sure if I’ll be able to squeeze in another one), so here’s hoping that you find something useful, helpful, or informative in the links that I’ve collected. Enjoy some light reading over your festive holiday season!

Networking

  • Brian Linkletter has a post on how to run the Antidote network emulator on KVM for better performance. Antidote, as I understand it, is the network emulator that runs the labs on the NRELabs web site (which is another cool thing in and of itself). In perusing some of Brian’s content, I see lots of stuff at the intersection of networking and virtualization. That’s a good place to be right now, in my opinion.
  • Anders Olsson explains how to map NSX-T UUIDs to the actual logical objects.
  • Julien Demierre has three-part series on interfaces and uplinks for ESXi/Edge in NSX-T deployments (part 1, part 2, and part 3).
  • This is an old(er) post that I’ve probably mentioned here before, but I think it still has relevance. Brent Salisbury’s post on Golang for Network Ops not only points out why networking pros should learn Golang, but also provides some resources on how to learn it.
  • Philippe Bogaerts has an article on how to use tcpdump effectively in Kubernetes. The one caveat to his approach that may be worth mentioning is that some policies (a PodSecurityPolicy or the use of Open Policy Agent) may prevent the user from launching a Pod connected to the host’s network namespace.

Servers/Hardware

Nothing this time around, although there may have been some hardware-related news coming out of re:Invent that I missed (such as new generations of AWS-designed chips being announced).

Security

Cloud Computing/Cloud Management

Operating Systems/Applications

Storage

Nothing this time; sorry! I’ll be on the lookout for stuff I can include next time around.

Virtualization

Career/Soft Skills

I haven’t included anything this time, as this Short Take has been pretty heavy on technical content. I’ll stay alert for content to include here next time.

OK, that’s it for now! I hope this was helpful and informative. Feedback is always welcome; it’s probably easiest to contact me via Twitter, but I’m around a lot of different places if you look. Have a great weekend!

Technology Short Take 120

Welcome to Technology Short Take #120! Wow…hard to believe it’s been almost two months since the last Tech Short Take. Sorry about that! Hopefully something I share here in this Tech Short Take is useful or helpful to readers. On to the content!

Networking

Servers/Hardware

I don’t have anything to share this time around, but I’ll stay alert for content to include future Tech Short Takes.

Security

Cloud Computing/Cloud Management

  • Marc Boorshtein has a write-up on authentication in Kubernetes identity management. I’m glad the author explicitly called out that Kubernetes doesn’t directly connect to any kind of user store; this seems to be a point of confusion for new Kubernetes users with whom I’ve spoken.
  • Yan Cui with Lumigo tackles the concept of serverless vendor lock-in. I have to say, I agree with a lot of the statements Cui makes regarding lock-in, coupling, risk, and the role of data in lock-in. Much of what’s said in this article applies to all forms of “lock-in,” to be honest.
  • I’m glad I read this article from Michael Gasch. I understand why it’s titled the way it is, but the title does not do the excellent content justice. If you’re seeking to deepen your understanding of the Kubernetes architecture, I can heartily recommend this article. (Not to mention there’s a great set of links at the end with even more information.)
  • Jack Lindamood writes about how he regrets switching from CloudFormation to Terraform, and shares lessons learned—both good and bad—about each of these two options for infrastructure as code.
  • I love this article by Liz Fong-Jones. The sentence from which the title is taken captures it all: “Even the experts on the team were afraid to touch our Terraform configs, treating them like a haunted graveyard in which to seldom tread.” The idea of using CI for infrastructure as code is something that’s really been on my mind for the last few weeks, and so I’m glad I came across this article with lessons learned from the Honeycomb team.
  • Ryan Matteson provides some very useful information on creating Kubernetes manifests. The “TL;DR” is that kubectl explain is one of your best friends.
  • This article by Alibaba Cloud on how they scaled Kubernetes to 10,000 nodes has some useful information in it. It’s extremely unlikely that any of the steps this team took would be needed by other organizations, but the information shared does help illuminate some of the inner workings of Kubernetes—and knowing more about Kubernetes can be helpful to anyone supporting or implementing it.
  • I enjoyed this article on describing fault domains by Will Larson. In fact, I spent several hours a couple months ago playing around with dot to visually describe fault domains.

Operating Systems/Applications

Storage

Nothing this time! If you happen to find something you think other readers would find useful, send it my way and I’ll see about including it in a future Tech Short Take.

Virtualization

Career/Soft Skills

  • I’ll put this here, since it’s most closely aligned to career: Gustavo Franco and Matt Brown, both Customer Reliability Engineers with Google, discuss potential SRE team organization. I could see this article being helpful for organizations—or individuals—who are starting down the SRE path.
  • So long, Datanauts! It turns out the show is ending, and Nick Korte takes some time to write a tribute to Datanauts.

That’s all for now; I’ll have more links and articles in the next Tech Short Take. Feel free to contact me on Twitter to share any feedback you may have on this or other content here on the site. Thanks for reading!

KubeCon 2019 Day 3 and Event Summary

Keynotes

Bryan Liles kicked off the day 3 morning keynotes with a discussion of “finding Kubernetes’ Rails moment”—basically focusing on how Kubernetes enables folks to work on/solve higher-level problems. Key phrase from Bryan’s discussion (which, as usual, incorporated the humor I love to see from Bryan): “Kubernetes isn’t the destination. Kubernetes is the vehicle that takes us to the destination.” Ian Coldwater delivered a talk on looking at Kubernetes from the attacker’s point of view, and using that perspective to secure and harden Kubernetes. Two folks from Walmart also discussed their use case, which involves running Kubernetes clusters in retail locations to support a point-of-sale (POS) application at the check-out register. Finally, there was a discussion of chaos engineering from folks at Gremlin and Target.

No Breakout Sessions

Due to booth duty and my flight home, I wasn’t able to attend any breakout sessions today.

Event Summary

If I’m completely honest, I didn’t get as much out of the event as I’d hoped. I’m not yet sure if that is because I didn’t get to attend as many sessions as I’d hoped/planned (due to problems with sessions being moved/rescheduled or whatever), if my choice of sessions was just poor, if it was just me/my personality, if I’m not the target audience for this event, or if it is some combination of these factors. I did enjoy seeing other folks in the community, meeting some folks that I hadn’t met before, and being able to help answer some questions. From that perspective, the event was a success.

Feel free to contact me on Twitter if you’d like to discuss anything in more detail.

KubeCon 2019 Day 2 Summary

Keynotes

This morning’s keynotes were, in my opinion, better than yesterday’s morning keynotes. (I missed the closing keynotes yesterday due to customer meetings and calls.) Only a couple of keynotes really stuck out. Vicki Cheung provided some useful suggestions for tools that are helping to “close the gap” on user experience, and there was an interesting (but a bit overly long) session with a live demo on running a 5G mobile core on Kubernetes.

Running Large-Scale Stateful Workloads

Due to some power outages at the conference venue resulting from rain in San Diego, the Prometheus session I had planned to attend got moved to a different time. As a result, I sat in this session by Lyft instead. The topic was about running large-scale stateful workloads, but the content was really about a custom solution Lyft built (called Flyte) that leveraged CRDs and custom controllers to help manage stateful workloads. While it’s awesome that companies like Lyft can extend Kubernetes to address their specific needs, this session isn’t helpful to more “ordinary” companies that are trying to figure out how to run their stateful workloads on Kubernetes. I’d really like the CNCF and the conference committee to try to promote talks that are more applicable to the general audience instead of these “Look what we built!” sessions.

Leveling Up your CD: Unlocking Progressive Delivery on Kubernetes

This was a useful session covering Argo and some functionality being added to Argo to support what’s being called progressive delivery. Progressive delivery is defined by them as continuous delivery with some fine-grained controls over the blast radius, and the presenters demonstrated new features in Argo that provide fine-grained control over rollouts and the use of canary deployments (they showed a feature whereby Prometheus metrics can be used to determine the validity of a canary deployment and roll it back if unsuccessful). Argo was already on my “list of things to dig into,” but now I think it’s moved up a few spots on that list.

I had on my schedule to attend a talk on Krane, but the room moved from what was on my schedule (I don’t know if this is related to the power outages in other areas of the conference venue or what). After realizing this and wandering around trying to find the room, I realized I’d missed too much of the session so gave up.

That’s it for day 2. Tune in tomorrow for a recap on day 3, the final day of the event.

KubeCon 2019 Day 1 Summary

This week I’m in San Diego for KubeCon + CloudNativeCon. Instead of liveblogging each session individually, I thought I might instead attempt a “daily summary” post that captures highlights from all the sessions each day. Here’s my recap of day 1 at KubeCon + CloudNativeCon.

Keynotes

KubeCon + CloudNativeCon doesn’t have “one” keynote; it uses a series of shorter keynotes by various speakers. This has advantages and disadvantages; one key advantage is that there is more variety, and the attendees are more likely to stay engaged. I particularly enjoyed Bryan Liles’ CNCF project updates; I like Bryan’s sense of humor, and getting updates on some of the CNCF projects is always useful. As for some of the other keynotes, those that were thinly-disguised vendor sales pitches were generally pretty poor.

Introduction to the Virtual Kubelet

I was running late for the start of this session due to booth duty, and I guess the stuff I needed most was presented in that portion I missed. Most of what I saw was about Netflix Titus, and how the Netflix team ported Titus from Mesos to Virtual Kubelet. However, that information was so specific to Netflix’s particular use of Virtual Kubelet that it wasn’t all that helpful (for me, anyway).

TGIK Live

Joe Beda did a “TGIK Live” session on the show floor, discussing CUE. CUE’s been on my list to evaluate more closely (and possibly blog about). Joe’s demonstration of CUE helped provide a bit of a foundation for CUE, but I still have a lot of learning to do before I’ll be ready to write about it.

Russian Doll: Extending Containers with Nested Processes

Due to customer meetings and such, this was the only afternoon session I was able to attend in its entirety (I did manage to catch part of the session on Vitess). This session was presented by a couple of Google employees, and focused around efforts to run containers within a Pod in a sequential fashion (in a more complex fashion than initContainers support). The use case was to support tasks in Tekton, and the solution (or hack?) was to use a Go binary that “waits” on previous containers before running the original container ENTRYPOINT. I understand the use case, but I also have to wonder if a CustomResourceDefinition wouldn’t have been more appropriate for this.

Vitess: Stateless Storage in the Cloud

I was only able to capture the first portion of this session, which was about the Vitess project (a MySQL-compatible, scale-out, cloud-native database). The Vitess project, which is hosted by CNCF, announced today that the project has graduated (putting it on par with Kubernetes and Prometheus and others). The presenter shared the story—often with humorous anecdotes—of how Vitess came to be. I was hoping for some technical depth, but didn’t get any before I had to step out due to a customer meeting.

That’s it for day 1. Tune in tomorrow for a day 2 recap!

Recent Posts

Using Kustomize with Cluster API Manifests

A topic that’s been in the back of my mind since writing the Cluster API introduction post is how someone could use kustomize to modify the Cluster API manifests. Fortunately, this is reasonably straightforward. It doesn’t require any “hacks” like those needed to use kustomize with kubeadm configuration files, but similar to modifying kubeadm configuration files you’ll generally need to use the patching functionality of kustomize when working with Cluster API manifests. In this post, I’d like to take a fairly detailed look at how someone might go about using kustomize with Cluster API.

Read more...

Programmatically Creating Kubernetes Manifests

A while ago I came across a utility named jk, which purported to be able to create structured text files—in JSON, YAML, or HCL—using JavaScript (or TypeScript that has been transpiled into JavaScript). One of the use cases was creating Kubernetes manifests. The GitHub repository for jk describes it as “a data templating tool”, and that’s accurate for simple use cases. In more complex use cases, the use of a general-purpose programming language like JavaScript in jk reveals that the tool has the potential to be much more than just a data templating tool—if you have the JavaScript expertise to unlock that potential.

Read more...

Spousetivities in Barcelona at VMworld EMEA 2019

Barcelona is probably my favorite city in Europe—which works out well, since VMware seems to have settled on Barcelona at the destination for VMworld EMEA. VMworld is back in Barcelona again this year, and I’m fortunate enough to be able to attend. VMworld in Barcelona wouldn’t be the same without Spousetivities, though, and I’m happy to report that Spousetivities will be in Barcelona. In fact, registration is already open!

Read more...

Using Kustomize with Kubeadm Configuration Files

Last week I had a crazy idea: if kustomize can be used to modify YAML files like Kubernetes manifests, then could one use kustomize to modify a kubeadm configuration file, which is also a YAML manifest? So I asked about it in one of the Kubernetes-related channels in Slack at work, and as it turns out it’s not such a crazy idea after all! So, in this post, I’ll show you how to use kustomize to modify kubeadm configuration files.

Read more...

Technology Short Take 119

Welcome to Technology Short Take #119! As usual, I’ve collected some articles and links from around the Internet pertaining to various data center- and cloud-related topics. This installation in the Tech Short Takes series is much shorter than usual, but hopefully I’ve managed to find something that proves to be helpful or informative! Now, on to the content!

Read more...

Exploring Cluster API v1alpha2 Manifests

The Kubernetes community recently released v1alpha2 of Cluster API (a monumental effort, congrats to everyone involved!), and with it comes a number of fairly significant changes. Aside from the new Quick Start, there isn’t (yet) a great deal of documentation on Cluster API (hereafter just called CAPI) v1alpha2, so in this post I’d like to explore the structure of the CAPI v1alpha2 YAML manifests, along with links back to the files that define the fields for the manifests. I’ll focus on the CAPI provider for AWS (affectionately known as CAPA).

Read more...

An Introduction to Kustomize

kustomize is a tool designed to let users “customize raw, template-free YAML files for multiple purposes, leaving the original YAML untouched and usable as is” (wording taken directly from the kustomize GitHub repository). Users can run kustomize directly, or—starting with Kubernetes 1.14—use kubectl -k to access the functionality (although the standalone binary is newer than the functionality built into kubectl as of the Kubernetes 1.15 release). In this post, I’d like to provide an introduction to kustomize.

Read more...

Consuming Pre-Existing AWS Infrastructure with Cluster API

All the posts I’ve published so far about Kubernetes Cluster API (CAPI) assume that the underlying infrastructure needs to be created. This is fine, because generally speaking that’s part of the value of CAPI—it will create new cloud infrastructure for every Kubernetes cluster it instantiates. In the case of AWS, this includes VPCs, subnets, route tables, Internet gateways, NAT gateways, Elastic IPs, security groups, load balancers, and (of course) EC2 instances. But what if you didn’t want CAPA to create AWS infrastructure? In this post, I’ll show you how to consume pre-existing AWS infrastructure with Cluster API for AWS (CAPA).

Read more...

Highly Available Kubernetes Clusters on AWS with Cluster API

In my previous post on Kubernetes Cluster API, I showed readers how to use the Cluster API provider for AWS (referred to as CAPA) to instantiate a Kubernetes cluster on AWS. Readers who followed through the instructions in that post may note CAPA places all the nodes for a given cluster in a single AWS availability zone (AZ) by default. While multi-AZ Kubernetes deployments are not without their own considerations, it’s generally considered beneficial to deploy across multiple AZs for higher availability. In this post, I’ll share how to deploy highly-available Kubernetes clusters—defined as having multiple control plane nodes distributed across multiple AZs—using Cluster API for AWS (CAPA).

Read more...

VMworld 2019 Vendor Meeting: Lightbits Labs

Last week at VMworld, I had the opportunity to meet with Lightbits Labs, a relatively new startup working on what they called “disaggregated storage.” As it turns out, their product is actually quite interesting, and has relevance not only in “traditional” VMware vSphere environments but also in environments more focused on cloud-native technologies like Kubernetes.

Read more...

Bootstrapping a Kubernetes Cluster on AWS with Cluster API

Yesterday I published a high-level overview of Cluster API (CAPI) that provides an introduction to some of the concepts and terminology in CAPI. In this post, I’d like to walk readers through actually using CAPI to bootstrap a Kubernetes cluster on AWS. This walkthrough is for the v1alpha1 release of CAPI (a walk through for CAPI v1alpha2 is coming).

Read more...

An Introduction to Kubernetes Cluster API

In this post, I’d like to provide a high-level introduction to the Kubernetes Cluster API. The aim of Cluster API (CAPI, for short) is, as outlined in the project’s GitHub repository, “a Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management”. This high-level introduction serves to establish some core terminology and concepts upon which I’ll build in future posts about CAPI.

Read more...

Liveblog: VMworld 2019 Day 1 General Session

This is the liveblog from the day 1 general session at VMworld 2019. This year the event is back at Moscone Center in San Francisco, and VMware has already released some juicy news (see here, here, and here) in advance of the keynote this morning, foreshadowing what Pat is expected to talk about.

Read more...

Technology Short Take 118

Welcome to Technology Short Take #118! Next week is VMworld US in San Francisco, CA, and I’ll be there live-blogging and meeting up with folks to discuss all things Kubernetes. If you’re going to be there, look me up! Otherwise, I leave you with this list of links and articles from around the Internet to keep you busy. Enjoy!

Read more...

Creating Tagged Subnets Across AWS AZs Using Pulumi

As I mentioned back in May in this post on creating a sandbox for learning Pulumi, I’ve started using Pulumi more and more of my infrastructure-as-code needs. I did switch from JavaScript to TypeScript (which I know compiles to JavaScript on the back-end, but the strong typing helps a new programmer like me). Recently I had a need to create some resources in AWS using Pulumi, and—for reasons I’ll explain shortly—many of the “canned” Pulumi examples didn’t cut it for my use case. In this post, I’ll share how I created tagged subnets across AWS availability zones (AZs) using Pulumi.

Read more...

Older Posts

Find more posts by browsing the post categories, content tags, or site archives pages. Thanks for visiting!