12 May 2023
Welcome to Technology Short Take #168! Although this weekend is (in the US, at least) celebrated as Mother’s Day weekend—don’t forget to call or visit your mom!—I thought you all might want some light weekend reading. I’m here to help, after all. To that end, here’s the latest Technology Short Take, with links to a variety of articles in various disciplines. Enjoy!
Networking
Security
- Via Russ White, I came across an article on Fully Homomorphic Encryption (FHE) as a privacy-enhancing technology (PET). Unfortunately, the article didn’t even bother to define terms like FHE and PET, which made it quite difficult to parse. Let this be a reminder to you (and me!): not everyone immediately understands the acronyms you’re using, so define them on first use, please.
- In Technology Short Take 166 I linked to an article about BlackLotus, an exploit capable of bypassing UEFI Secure Boot. Microsoft responded to the BlackLotus threat with this guidance related to Secure Boot Manager changes which address the bypass vulnerability used by BlackLotus.
- Again I ask: why Cedar instead of Open Policy Agent (OPA)? What problems does Cedar solve that OPA doesn’t? Is it performance? Scale? Flexibility? Does AWS feel that Cedar is “more secure” or “more correct” than OPA?
Cloud Computing/Cloud Management
- I shared this on Twitter recently, but I wanted to share it here as well. David Egbert wrote up a tutorial on using Pulumi to deploy a production-grade static site on AWS. What I liked about David’s tutorial—in contrast to so many others—is that David didn’t stop at the standard S3/CloudFront component. He went on to add Route53 entries and TLS certificates via ACM. Additionally, David showed one way to break up your Pulumi code (he used Python) into multiple files. Well worth the read, in my opinion.
- Via a colleague at work, I was pointed to Jim Ferrari’s article on integrating Azure Key Vault with Azure Kubernetes Service. (Completely unrelated side note: What a cool last name! Thankfully he didn’t have the last name Lamborghini, can you imagine trying to learn to spell that as a kid in school?)
- I appreciate Nigel Poulton’s clear writing on WebAssembly, containerd, and WASM. First, he has a two-part series discussing WebAssembly on Kubernetes; part 1 provides a high-level overview of the various components, and part 2 is more of a hands-on tutorial. Also, Nigel has another article on understanding containerd shims as they relate to WebAssembly.
- Yan Cui addresses concerns about complex serverless application diagrams.
- I also found an article about a really neat Cilium feature involving integration with AWS security groups. I’m not providing a link here because it was behind a “regwall” (you had to create a free account in order to read it). I’m not a fan of such mechanisms, so…
- Eric Pauley’s analysis of rising EC2 spot instance pricing prompts the question of whether we should bid farewell to the era of cheap EC2 spot instances.
Operating Systems/Applications
Storage
Virtualization
Career/Soft Skills
- I appreciated Carly Richmond’s piece on respecting the social battery. There are times when my social battery is depleted and I need a bit of quiet time to recharge. I also appreciated Carly’s description of extroversion and introversion as a spectrum; so many discussions of this topic tend to classify folks as either one or the other, instead of recognizing that people can have tendencies of both.
- Chasing the productivity Holy Grail can be draining…and even counter-productive. Josh Mitrani’s review of the book titled How to Calm Your Mind (written by former “productivity guru” Chris Bailey) talks a bit about that. I can definitely identify with some of the things mentioned here.
That’s all I have for you this time around; hopefully you’ve found something helpful, informative, educational, or at least interesting. If you have feedback for me on this post (or any post on my site), I always love hearing from readers. The best way to interact with me is via Twitter, Mastodon, or Slack (you can find me in the Kubernetes and Pulumi communities pretty much all the time). Feel free to reach out!
7 Apr 2023
Welcome to Technology Short Take #167! This Technology Short Take is a tad shorter than the typical one; I’ve been busy recently and my intake volume of content has gone down, thus resulting in fewer links to share with all of you! I opted to go ahead and publish a shorter Technology Short Take instead of making everyone wait around for a longer one. In any case, here’s hoping that I’ve included something useful for you!
Networking
Servers/Hardware
Security
Cloud Computing/Cloud Management
Operating Systems/Applications
Programming/Development
Storage
Virtualization
That’s all for now! As always, feel free to contact me if you have any questions, feedback, or suggestions for how I can improve this post or the site in general. Find me on Mastodon, find me on Twitter, or find me in one of several different Slack communities. Have a great weekend!
31 Mar 2023
A little over a month ago I published a post on creating a Talos Linux cluster on AWS with Pulumi. Talos Linux is a re-thinking of your typical Linux distribution, custom-built for running Kubernetes. Talos Linux has no SSH access, no shell, and no console; instead, everything is managed via a gRPC API. This post is something of a “companion post” to the earlier AWS post; in this post, I’ll show you how to create a Talos Linux cluster on Azure with Pulumi.
The program I’ll share with you in this post is written in Go, but the process outlined in this post and the accompanying code is equally applicable in other languages supported by Pulumi. (TypeScript is a popular choice for lots of folks.) The code is available in this GitHub repository. It’s based on this documentation from Sidero Labs, and I also found this blog post to be helpful as well.
The Pulumi program follows this overall flow:
- First, the program creates the base infrastructure objects that are required—a resource group, a virtual network, some subnets, and a network security group.
- Next, it creates a load balancer, gets a public IP address for the load balancer, and creates the associated backend address pool, health probe, and load balancing rule. (This load balancer is used only for Kubernetes API traffic.)
- The program uses the Talos provider to generate configurations for the control plane VMs and the worker VMs.
- Next up are the public IP addresses, network interfaces, and the control plane VMs (along with the necessary associations to put the interfaces into the backend address pool for the load balancer and the network security group). The Talos machine configuration is passed to the VMs at startup.
- Last, but certainly not least, the program creates the worker VMs and their network interfaces, along with the association required to make the interfaces part of the network security group. As with the control plane VMs, the Talos machine configuration is passed to the VMs at startup.
- The last step is to bootstrap the cluster.
Let’s take a bit of a closer look at each of these steps.
Creating the Azure infrastructure
The first part of the Pulumi program creates the requisite base Azure infrastructure: a resource group to contain all the resources, a virtual network, three subnets, and a network security group to control the traffic allowed to the VMs.
There’s one very important task that the Pulumi program does not resolve for the user: the creation of a Talos Linux virtual machine image. You’ll want to follow the documentation for the steps to upload a Talos Linux VHD file and register an image, and then use that image’s ID with pulumi config set talosImageId <id-of-image-you-created>
before running pulumi up
.
Creating the Kubernetes API Load Balancer
Because the program sets out to create three control plane nodes for an HA control plane, a load balancer to handle the Kubernetes API traffic is needed. On Azure, this means creating a number of different things:
- The load balancer itself
- A public IP address to be assigned to the “front end” of the load balancer
- A backend address pool (where the control plane VMs will join to be directed traffic)
- A health probe
- A load balancing rule that instructs the load balancer on how and where to direct traffic
All of this takes you up through line 184 of the Pulumi Go program.
Creating the Talos Configuration
Next up is creating the machine configurations that will be applied to the control plane and worker VMs. First, though, the program has to create some resources first: public IP addresses and network interfaces for the control plane VMs, along with associating these interfaces with the backend address pool and the network security group.
Once those addresses and interfaces are created, they’re used—along with the public IP address created for the load balancer—to generate the machine configurations for the VMs that will actually run Talos Linux.
There are four parts to the configuration generated:
- Some PKI assets, referred to here as machine secrets
- A client configuration file, which the
talosctl
command-line utility uses to connect to the Talos Linux nodes
- Machine configuration for the control plane nodes
- Machine configuration for the worker nodes
All of these are generated using the prerelease Talos Linux provider for Pulumi. To install the prerelease provider, you can follow these instructions.
Launching the Nodes
Once the Talos configurations have been created, then the program moves on to create the control plane VMs. These are linked to the network interfaces (and public IP addresses) used earlier. Since those interfaces were associated with the network security group, the correct traffic is permitted; and since the interfaces were associated with the backend address pool, the load balancer will send Kubernetes API traffic to them.
The worker VMs are handled similarly, except they aren’t given public IP addresses, and aren’t registered to the load balancer’s backend address pool.
In both cases—for both the control plane and the worker VMs—the appropriate machine configuration is passed to the VM at startup.
Bootstrapping the Cluster
Finally, it comes down to bootstrapping the cluster, which—again using the Talos provider for Pulumi—is accomplished with calling talos.NewTalosMachineBootstrap
against the first control plane VM to be created. The Pulumi program will complete after this; it doesn’t wait for the bootstrap process to complete (which will take a few more minutes).
At this point, you can use pulumi stack output
to retrieve a configuration file for the talosctl
command-line utility, and then run talosctl health
to watch the cluster bootstrap. Once the cluster is up and running, talosctl kubeconfig
will get you a Kubeconfig file you can use to access the Kubernetes API via kubectl
. Boom—just like that you’ve got a Kubernetes cluster.
Using this Pulumi Program
As mentioned earlier in this post, all of the Pulumi code is available on GitHub in this repository. Please note that I tested the code and it works for me, but it is supplied “as-is”. Don’t use this for your production environment without performing your own testing and validation!
If you do want to give it a spin, you’ll need to (these steps assume you’ve already installed Pulumi, the Azure CLI, and Go):
- Create your own Talos Linux image (by uploading a VHD and registering it as an image)
- Clone the
talos-azure-pulumi
GitHub repository to your machine and switch into that directory.
- Run
pulumi stack init
to create a new stack.
- Run
pulumi config set talosImageId <image-id>
to set the ID of the image you created in step 1.
- Run
pulumi config set azure:location <region-name>
to set the Azure region where you want your resources created.
- Run
pulumi up
.
That’s it!
Additional Resources
In addition to this blog post, the code and instructions are found in the associated GitHub repository. The code contains links to the API documentation for both the Pulumi SDK and the Talos provider; check the comments in the code.
I hope this is helpful to you! If you have questions, you’re welcome to open an issue in the GitHub repository, or you can contact me directly. It’s not hard to get in touch with me—you can find me on Twitter, on Mastodon, and on various Slack communities (including the Kubernetes and Pulumi Slack communities).
10 Mar 2023
Welcome to Technology Short Take #166! I’ve been collecting links for the last few weeks, and now it’s time to share them with all of you. There are some familiar names in the links below, but also some newcomers—and I’m really excited to see that! I’m constantly on the lookout for new sources (if you have a site you think I should check out, hit me up—my contact info is at the bottom of this post!). But enough of that, let’s get on with the content. Enjoy!
Networking
Servers/Hardware
Security
- Jeff Warren discusses a potential way for malicious players to bypass multi-factor authentication, aka the “Pass the Cookie” attack.
- Aditya Patel takes a closer look at AWS’ recent announcement to enable server-side encryption (SSE) on S3 by default, and whether this new default setting offers any real improvement in security posture. I won’t spoil you by sharing his conclusion; go read the article (which is really well-written, in my opinion) to find out for yourself.
- Alberto Pellitteri with Sysdig discusses SCARLETEEL, an operation conducted by an attacker that leveraged many of the tools found in modern cloud environments: Kubernetes, Terraform, and AWS. I highly recommend reviewing the article and considering what takeaways apply to your environment, if any.
- Martin Smolár of ESET provides the first public analysis of a UEFI bootkit that is capable of bypassing UEFI Secure Boot, a bootkit known as BlackLotus.
Cloud Computing/Cloud Management
- This article on using Open Policy Agent (OPA) as a custom Lambda authorizer for the AWS API Gateway was informative and helpful. It did underscore something for me, though: I need to improve my coding skills.
- Do you need Argo CD? This article by Kirill Shirinkin provides, in the author’s words, “some guidelines that will help you to assess if Argo CD makes sense for your setup”.
Operating Systems/Applications
Storage
Virtualization
I don’t have any career/soft skills links for you this time, so that’s all for now! I hope that I’ve included something that you’ll find useful. As always, I invite your feedback on this post or any post on my site; feel free to reach out to me on Twitter or find me on Mastodon. I’m also present in a number of Slack communities, and you’re welcome to contact me directly there as well. Thank you for reading!
24 Feb 2023
Talos Linux is a Linux distribution purpose-built for running Kubernetes. The Talos web site describes Talos Linux as “secure, immutable, and minimal.” All system management is done via an API; there is no SSH access, no shell, and no console. In this post, I’ll share how to use Pulumi to automate the creation of a Talos Linux cluster on AWS.
I chose to write my Pulumi program in Go, but you could—of course—choose to write it in any language that Pulumi supports (JavaScript/TypeScript, Python, one of the .NET languages, Java, or even YAML). I’ve made the Pulumi program available via this GitHub repository. It’s based on these instructions for standing up Talos Linux on AWS.
The Pulumi program has four major sections:
- First, it creates the underlying base infrastructure needed for a Talos Linux cluster to run. This includes a VPC (and all the assorted other pieces, like subnets, gateways, routes, and route tables) and a load balancer. The load balancer is needed for the Kubernetes control plane, which we will bootstrap later in the program. This portion also creates the EC2 instances for the control plane.
- Next, it uses the Talos Pulumi provider to generate the Talos configuration that will be applied to the instances after they are created. This configuration needs information from step 1; specifically, it needs the DNS name of the load balancer and the IP addresses of the control plane nodes.
- Third, it applies the correct configuration to the control plane nodes, and launches some EC2 instances to serve as worker nodes in the Kubernetes cluster. The program then takes the Talos configuration from step 2 and applies it to the worker nodes.
- The fourth and final step is to bootstrap the cluster.
Let’s examine each of these sections in a bit more detail.
Creating the AWS Infrastructure
The first part of the Pulumi program (running through about line 197) creates the necessary AWS infrastructure. The program leverages Crosswalk for AWS to simplify the creation of the underlying VPC and associated components. Most of the code involves creating security groups and security group rules.
This part of the program also creates the load balancer that will be used for the Kubernetes API.
Finally, the program launches three EC2 instances using one of the official Talos AMIs (you have to supply the AMI ID as a configuration value). The IDs and the private IP addresses for these EC2 instances are stored in two different arrays; we’ll need this information later.
Note that you could extend the Pulumi Go program to look up the correct AMI for you so that this information doesn’t need to be passed in as a configuration value via pulumi config
. I’ll leave this as an exercise for the reader.
Creating the Talos Configuration
The next part of the Pulumi program builds the Talos configuration, using the prerelease Pulumi provider and information (outputs) from the resources created earlier. Specifically, the Talos configuration uses the DNS name of the load balancer created earlier, and the IP addresses of the EC2 instances (which will become the control plane nodes for the Kubernetes cluster).
There are three parts to the configuration:
- A client configuration file, which the
talosctl
command-line utility uses to connect to the Talos Linux nodes
- Machine configuration for the control plane nodes
- Machine configuration for the worker nodes
To install the prerelease provider, you can follow these instructions.
Configuring the Nodes
Once the Talos provider builds the correct machine configurations, the program then applies the configuration to each of the control plane nodes (worker nodes are handled later). It does this by looping through the array with the instance IP addresses and applying the configuration to each instance.
Once the machine configuration has been applied to the control plane nodes, three more EC2 instances are launched for worker nodes, and then the correct machine configuration is applied to those nodes (using the mechanism of looping over an array).
Bootstrapping the Cluster
Finally, it comes down to bootstrapping the cluster, which—again using the Talos provider for Pulumi—is accomplished with calling talos.NewTalosMachineBootstrap
against the first control plane node to be created. The Pulumi program will complete after this; it doesn’t wait for the bootstrap process to complete (which will take a few more minutes).
At this point, you can use pulumi stack output
to retrieve a configuration file for the talosctl
command-line utility, and then run talosctl health
to watch the cluster bootstrap. Once the cluster is up and running, talosctl kubeconfig
will get you a Kubeconfig file you can use to access the Kubernetes API via kubectl
. Nice!
Additional Resources
All of the Pulumi code is available on GitHub in this repository. Please note that I tested the code and it works for me, but it is supplied “as-is”. Don’t use this for your production environment without performing your own testing and validation!
I hope you’ve found this post helpful. If you have questions, you’re welcome to open an issue in the GitHub repository, or you can contact me directly. It’s not hard to get in touch with me—you can find me on Twitter, on Mastodon, and on various Slack communities (including the Kubernetes and Pulumi Slack communities).
Recent Posts
17 Feb 2023
Welcome to Technology Short Take #165! Over the last few weeks, I’ve been collecting articles I wanted to share with readers on major areas in technology: networking, security, storage, virtualization, cloud computing, and OSes/applications. This particular Technology Short Take is a tad heavy on cloud computing, but there’s a decent mix of other articles as well. Enjoy!
Read more...
9 Feb 2023
I’ve been using macOS Stage Manager off and on for a little while now. In Stage Manager, I can see the beginnings of what might be a very useful paradigm for desktop computing. Unfortunately, in its current incarnation, I believe Stage Manager is incomplete.
Read more...
8 Feb 2023
Normally, installing a Pulumi provider is pretty easy; you run pulumi up
and the provider gets installed automatically. Worst case scenario, you can install the provider using pulumi plugin install
. However, sometimes things have to be done manually. In this post, I’ll walk you through installing a prerelease provider—in this case, the prerelease Pulumi provider for Talos Linux—using both pulumi plugin install
as well as using a manual process.
Read more...
6 Feb 2023
Since I switched my primary workstation to an M1-based MacBook Pro (see my review here), I’ve starting using temporary AWS EC2 instances for compiling code, building Docker images, etc., instead of using laptop-local VMs. I had an older Mac Pro (running Fedora) here in my home office that formerly filled that role, but I’ve since given that to my son (he’s a young developer who wanted a development workstation). Besides, using EC2 instances has the benefit of access when I’m away from my home office. I use Pulumi to manage these instances, and I extended my Pulumi code to also include managing local Docker contexts for me as well. In this post, I’ll share the solution I’m using.
Read more...
27 Jan 2023
Welcome to Technology Short Take #164! I’ve got another collection of links to articles on networking, security, cloud, programming, and career development—hopefully you find something useful!
Read more...
6 Jan 2023
Welcome to Technology Short Take #163, the first of 2023! If you’re new to this site, the Technology Short Takes are essentially “link lists”—I collect links and articles about various technologies and I share them about every 3-4 weeks (sometimes more frequently). I’ll often add a bit of commentary here and there, but the real focus is the information in the linked articles. But enough of this, let’s get on with it! Here’s hoping you find something useful here.
Read more...
5 Jan 2023
Off and on for a number of years, I published a “projects for the coming year” post and a “report card for last year’s projects” post (you can find links to all of these here). Typically, the project list was composed of new things I would learn and/or new things I would create or do. While there’s nothing wrong with this sort of thing—not at all!—I came across an idea while reading that I’ve decided I’ll adopt for 2023: a depth year.
Read more...
9 Dec 2022
Welcome to Technology Short Take #162! It’s taken me a bit longer than I would have liked to get this post assembled, but it’s finally here. Hopefully I’ve managed to find something you’ll find useful! As usual, the links below are organized by technology area/discipline, and I’ve added a little bit of commentary to some of the links where it felt necessary. Enjoy!
Read more...
28 Oct 2022
Welcome to Technology Short Take #161! It’s been a little over a month since the last Technology Short Take, although the Full Stack Journey recently did an “Audio Edition” of a Technology Short Take that you should probably check out. In any case, I’ve spent the last month collecting links to articles and tutorials from around the web on all the various technologies that us IT folk are likely to encounter in our day-to-day adventures. I hope there’s something here that you find useful!
Read more...
6 Oct 2022
Lately I’ve been spending a little bit of time building Pulumi programs to assist with standing up Azure Kubernetes Service (AKS) clusters. I’ve learned a pretty fair amount about Azure and AKS along the way, as expected, but I was taken aback by the poor user experience (in my opinion) when it came to accessing the AKS clusters once they’d been established. In this post, I’ll share a small tweak you can make that will, in most cases, make accessing your AKS clusters a great deal smoother.
Read more...
23 Sep 2022
Welcome to Technology Short Take #160! This time around, my list of links and articles is a tad skewed toward cloud computing/cloud management, but I’ve still managed to pull together some links on other topics that readers will hopefully find useful. For example, did you know about the secret macOS network quality tool? You didn’t? Lucky for you there’s a link to an article about it below. Read on to get all the details!
Read more...
21 Sep 2022
Lately I’ve been doing a fair amount of work with Pulumi’s YAML support (see this blog post announcing it), and I recently ran into a situation where I wanted to read in and use a configuration value (set via pulumi config
). When using one of Pulumi’s supported programming languages, like TypeScript or Python or Go, this is pretty easy. It’s also easy in YAML, but not as intuitive as I originally expected. In this post, I’ll share how to read in and use a configuration value when using Pulumi YAML.
Read more...
6 Sep 2022
As I was winding down things at Kong and getting ready to transition to Pulumi (more information on why I moved to Pulumi here), I casually made the comment on Twitter that I needed to start managing my AWS key pairs using Pulumi. When the opportunity arose last week, I started doing exactly that! In this post, I’ll show you a quick example of how to use Pulumi and Go to declaratively manage AWS key pairs.
Read more...
2 Sep 2022
Welcome to Technology Short Take #159! If you’re interested in finding some links to articles around the web on topics like WASM, Git, Sigstore, or EKS—among other things—then you’ve come to the right place. I’ve spent the last few weeks collecting articles I think you’ll find useful, gleaning them from the depths of Twitter, RSS feeds, Reddit, and Slack. Enjoy, and never stop learning!
Read more...
15 Aug 2022
For quite a few years, I’ve had this desktop wallpaper that I really love. I don’t even remember where I got it or where it came from, so I can’t properly attribute it to anyone. I use this wallpaper from time to time when I want to be reminded to challenge myself, to learn new things, and to step outside of what is comfortable in order to explore the as-yet-unknown. Looking at this wallpaper on my desktop a little while ago, I realized that I may have started taking the inspirational phrase on this wallpaper for granted, instead of truly applying it to my life.
Read more...
Older Posts
Find more posts by browsing the
post categories,
content tags, or
site archives pages. Thanks for visiting!