17 Apr 2024
As a sort of follow-up to my previous post on using the AWS CLI to track the specific Elastic Network Interfaces (ENIs) used by Amazon Elastic Kubernetes Service (EKS) cluster nodes, this post focuses on the EC2 instances themselves. I feel this is less of a “problem” than tracking ENIs, but I wanted to share this information nevertheless. In this post, I’ll show you which AWS CLI command to use to list all the EC2 instances associated with a particular EKS cluster.
If you read the previous post on tracking ENIs used by EKS, you might think that you could use a very similar AWS CLI command (aws ec2 describe-instances instead of aws ec2 describe-network-interfaces) to track the EC2 instances in a cluster—and you’d be mostly correct. Like the ENIs, EKS does add a cluster-specific tag to all EC2 instances in the cluster. However, just to make life interesting, the tag used for EC2 instances is not the same as the tag used for ENIs. (If someone at AWS knows of a technical reason why these tags are different, I’d love to hear it.)
Instead of using the cluster.k8s.amazonaws.com/name tag that is used on the ENIs, you’ll need to use the aws:eks:cluster-name tag instead, like this:
aws ec2 describe-instances --filters Name=tag:aws:eks:cluster-name,\
Values=<name-of-cluster>
Just replace <name-of-cluster> in the above command with the name of your EKS cluster, and you’re good to go. As I mentioned in the previous post, if you’re using an automation tool such as Pulumi or Terraform, you may need to explicitly specify the name of the cluster in your code (or look it up after the cluster is created).
I hope this information is useful to folks. If you have questions (or corrections, in the event I have something incorrect here!), please feel free to reach out. You can find me on Twitter, on the Fediverse, or in a number of different Slack communities. Thanks for reading!
15 Apr 2024
I’ve recently been spinning up lots of Amazon Elastic Kubernetes Service (EKS) clusters (using Pulumi, of course) in order to test various Cilium configurations. Along the way, I’ve wanted to verify the association and configuration of Elastic Network Interfaces (ENIs) being used by the EKS cluster. In this post, I’ll share a couple of AWS CLI commands that will help you track the ENIs used by an EKS cluster.
When I first set out to find the easiest way to track the ENIs used by the nodes in an EKS cluster, I thought that AWS resource tags might be the key. I was right—but not in the way I expected. In the Pulumi program (written in Go) that I use to create EKS clusters, I made sure to tag all the resources.
For example, when defining the EKS cluster itself I assigned tags:
eksCluster, err := eks.NewCluster(ctx, "eks-cluster", &eks.ClusterArgs{
Name: pulumi.Sprintf("%s-test", regionNames[awsRegion]),
// Some code omitted here for brevity
Tags: pulumi.StringMap{
"Name": pulumi.Sprintf("%s-test", regionNames[awsRegion]),
"owner": pulumi.String(ownerTag),
"team": pulumi.String(teamTag),
"usage": pulumi.String(usageTag),
"expiry": pulumi.String("2025-01-01"),
},
})
And I assigned tags again when defining the node group for the EKS cluster:
_, err = eks.NewNodeGroup(ctx, "node-group", &eks.NodeGroupArgs{
ClusterName: eksCluster.Name,
// Some code omitted here for brevity
Tags: pulumi.StringMap{
"Name": pulumi.Sprintf("%s-nodegroup-01", regionNames[awsRegion]),
"owner": pulumi.String(ownerTag),
"team": pulumi.String(teamTag),
"usage": pulumi.String(usageTag),
"expiry": pulumi.String("2025-01-01"),
},
})
I thought that these tags would carry over to the ENIs attached to the EC2 instances in the node group. Assuming the value of ownerTag was set to “slowe”, it would be possible to see all the ENIs with this command:
aws ec2 describe-network-interfaces --filters Name=tag:owner,Values=slowe
Alas, these tags don’t carry over (not that I’ve observed, anyway). However, all is not lost! EKS creates its own tag you can use with the describe-network-interfaces command:
aws ec2 describe-network-interfaces \
--filters Name=tag:cluster.k8s.amazonaws.com/name,Values=cluster-name
The cluster.k8s.amazonaws.com/name tag is automatically added to ENIs created for use by EKS; you just need to supply the correct value (to replace cluster-name in the above command). If you’re using an automation tool like Pulumi or Terraform, you’ll want to be sure you know what the EKS cluster name is; you can assign it, as I did in the code above, or you can look it up.
While I didn’t share anything amazingly unique or earth-shattering here, I do hope that this post is helpful to folks. Feel free to find me on various social media platforms—such as on Twitter or on the Fediverse—if you have questions or comments about this post. Constructive feedback is always welcome!
15 Mar 2024
Welcome to Technology Short Take #176! This Tech Short Take is a bit heavy on security-related links, but there’s still some additional content in a number of other areas, so you should be able to find something useful—or at least interesting—in here. Thanks for reading!
Networking
Servers/Hardware
Security
- In early February a vulnerability was uncovered in a key component of the Linux boot process. The vulnerability affects virtually all Linux distributions and allows attackers to bypass the secure boot protections and insert a low-level bootkit. While the requirements for exploiting the vulnerability are not insurmountable, they do require a certain level of effort. More details available via Ars Technica and via ZDnet.
- Nick Frichette shares how to bypass GuardDuty Tor client findings (basically, how to connect to Tor without GuardDuty detecting it).
- The Sysdig Threat Research Team uncovered the malicious use of a network mapping tool called SSH-Snake. Read more about it in this post.
- VMware is patching a set of severe “sandbox escape” bugs. Two of the vulnerabilities are rated a 9.3 out of 10, and even VMware’s flagship ESXi hypervisor is affected. More details are available from Ars Technica.
- Think Linux doesn’t have malware? A new Bifrost remote access trojan (RAT) for Linux employs a number of techniques to remain hidden, including using a “VMware-esque” domain name for command and control servers.
- And here’s another example of malware that is targeting Linux (along with Windows).
- This would be why I hate it when companies force me to use SMS for two-factor authentication—at least let me use a one-time passcode or something.
Cloud Computing/Cloud Management
Operating Systems/Applications
Storage
Virtualization
- In the wake of Broadcom discontinuing VMware ESXi Free, Nutanix is hoping to fill the gap with Nutanix Community Edition. Vladan Seget provides some additional details in his blog post. Given that Nutanix Community Edition is based on the open source KVM hypervisor, this could lead to greater KVM adoption among small businesses and virtualization hobbyists who formerly would have used VMware’s solution.
- Staf Wagemakers (I think I have the name right) describes running OpenBSD as a UEFI virtual machine on a Raspberry Pi.
- I stumbled across a pair of articles by Greg Gant on the use of QEMU to run older versions of Mac OS (including pre-Mac OS X versions): there’s the original piece, and then an updated piece.
Career/Soft Skills
That’s all for now! I always love hearing from readers, so if you found something useful in this post—or in any post—don’t hesitate to reach out! You can reach me on Twitter, on the Fediverse, or in a number of different Slack communities. You’re also welcome to drop me an e-mail; my address is here on the site (it’s not hard to find). Enjoy!
1 Mar 2024
It’s no secret I’m a fan of Markdown. The earliest mention of Markdown on this site is all the way back in 2011, and it was only a couple years after that when I migrated this site from WordPress to Markdown. Back then, the site was generated from Markdown using Jekyll (via GitHub Pages); today it is generated from Markdown sources using Hugo. One thing I’ve not done, though, is perform linting (checking for errors or potential errors) of the Markdown source files. That’s all about to change! In this post, I’ll share with you how I started linting my Markdown files.
To handle the linting, there are (at least) a couple different options:
- markdownlint-cli (GitHub repository)
- markdownlint-cli2 (GitHub repository)
Both of these use the same markdownlint library under the hood. They’re both available as both a CLI tool or as a Docker container; markdownlint-cli2 is also available as a GitHub Action. In both cases, the CLI tool is installed via npm install (typically globally with --global or -g). The key difference between the two is that markdownlint-cli2 is configuration-driven, whereas markdownlint-cli offers the ability to use either a configuration file or command-line flags. I decided to use markdownlint-cli, as the ability to use command-line flags makes it a tad easier to get started.
I performed initial testing with the Docker container, which you would tend to invoke like this:
docker container run --rm -v "$PWD:/workdir" ghcr.io/igorshubovych/markdownlint-cli:latest "path/to/*.md"
However, I later switched to the CLI tool for better cross-platform portability (yes, I know that macOS can run Docker containers via Docker Desktop, but you still have to pay the tax of running a Linux VM in the background). The CLI tool is invoked in much the same way:
markdownlint "path/to/*.md"
In the default configuration, markdownlint-cli flagged a lot of violations in the over 2,200 blog posts on the site. After fine-tuning the configuration by disabling a few rules (more details on the rules is found here), there were still a lot of violations—but not nearly as many. Notably, I disabled MD013 (“line-length”) and MD052 (“reference-links-images”); the former because I use soft line-wraps in my Markdown paragraphs and the latter because I use Hugo’s relref shortcode for cross-referencing other posts.
Initially it was a bit unclear to me how to use the .markdownlint.jsonc configuration file to disable some of the rules. (This was probably just me being dense, if I’m honest.) For example, a configuration for MD052 might look like this:
// Rule details : https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md052.md
"reference-links-images": {
"shortcut_syntax": false
},
To disable this rule, it needs to look like this:
// Rule details : https://github.com/DavidAnson/markdownlint/blob/v0.33.0/doc/md052.md
"reference-links-images": false,
In retrospect, setting the top-level entry to false is obvious now, but when I first started looking at the configuration file I was expecting a property like disabled: true or similar.
Even with a few rules disabled, there were still quite a few violations, which I fixed manually over the course of a couple weeks, until I was finally able to run markdownlint over the entire list of ~2,230 Markdown posts without any violations. Yay!
The next step was to automate the process of running the Markdown lint checks—but that’s a topic for a separate post!
Additional Resources
While researching what was involved in linting Markdown files, I found this post to be helpful in getting started with markdownlint. The GitHub repositories (here, here, and here) were, of course, also very helpful (especially the rule descriptions).
I hope this post is useful to some folks out there. Please feel free to reach out to me on Twitter or on the Fediverse if you have comments, questions, or feedback (on this post or any post on my site). Thanks for reading!
23 Feb 2024
Welcome to Technology Short Take #175! Here’s your weekend reading—a collection of links and articles from around the internet on a variety of data center- and cloud-related topics. I hope you find something useful here!
Networking
Security
- I attended a local meetup here in the Denver metro area a short while ago and was introduced to
sops.
- AMD processors have been discovered to have multiple security flaws; more details available here.
- The Linux kernel project has become a CVE Numbering Authority (CNA); Greg Kroah-Hartman wrote a blog post that discusses this in more depth.
Cloud Computing/Cloud Management
- Josh Biggley shows how to deploy Tetragon with Cribl Edge. The blog post is a bit heavy on the Cribl marketing, but I suppose that is to be expected (it’s extremely common with most vendor blogs).
- Jack Lindamood’s list of infrastructure decisions he endorses or regrets provides some valuable insight into his personal experience with a variety of technologies and processes. Well worth reading, in my opinion. (Hat tip to Simon Wardley for sharing this on Twitter.)
- Ivan Yurochko of PerfectScale discusses how to manage S3 throttling.
- This post is an interesting look “inside” the CNCF Technical Oversight Committee (TOC), with a view on some of the challenges facing the CNCF and its related projects.
- Tyler Treat argues that it’s possible—preferable, perhaps—to do cloud without Kubernetes.
- Rory McCune reviews his final Kubernetes census.
- The Open Constructs Foundation recently launched a “community-driven CDK construct library initiative,” which seeks to provide a way for the CDK community to build and share CDK constructs.
- Michael Levan insists that cloud-native is in shambles. I think the article title is a bit click-baity, but the key point in the article—focusing on the expected outcome—is spot on.
- Tony Norlin discusses running Kubernetes with Cilium on FreeBSD.
- This is an older post (but still useful, I think, given the review of the code that implements the functionality) on Kubernetes leader election.
Operating Systems/Applications
Programming/Development
- Although it gets a bit deep into Rego, this article by Jasper Van der Jeugt of Snyk explains how automatic source code location for violations—pinpointing the file, line, and column where policy violations occur.
- Josh Collinsworth weighs in regarding LLMs and generative AI in his essay regarding GitHub Copilot. The experiences Josh describes with Copilot are not unique to Copilot; I’ve experienced the same with other LLM-based generative AI tools. The key takeaway (for me) is that generative AI doesn’t make things more accessible; it’s actually the opposite, because you need to know enough to know whether or not the generative AI tool is actually accurate or not.
Virtualization
- While certainly not unique to virtualization, I think it’s fair to say that virtualization has had a pretty significant impact on home labs. Sean Massey takes a moment to provide an update on his latest home lab update.
That’s all I have for you this time around. I love to hear from readers, so if you have feedback on this post (or any post!) on my site, please feel free to reach out. You can find me on Twitter, on the Fediverse, or in a number of different Slack communities. My e-mail address is also on this site and isn’t too hard to find…feel free to drop me a line!
Recent Posts
9 Feb 2024
Welcome to Technology Short Take #174! For your reading pleasure, I’ve collected links on topics ranging from Kubernetes Gateway API to recent AWS attack techniques to some geeky Linux and Git topics. There’s something here for most everyone, I’d say! But enough of my rambling, let’s get on to the good stuff. Enjoy!
Read more...
5 Feb 2024
For folks using AWS in their day-to-day jobs, it comes as no secret that AWS’ Managed NAT Gateway—responsible for providing outbound Internet connectivity to otherwise private subnets—is an expensive proposition. While the primary concern for large organizations is the data processing fee, the concern for smaller organizations or folks like me who run a cloud-based lab instead of a hardware-based home lab is the per-hour cost. In this post, I’ll show you how to use Pulumi to use a NAT instance for outbound Internet connectivity instead of a Managed NAT Gateway.
Read more...
22 Jan 2024
In August 2023, Pulumi released a version of the Docker provider that supported SSH-based connections to a Docker daemon. I’ve written about using SSH with Docker before (see here), and I sometimes use AWS-based “Docker build hosts” with my M-series Macs to make it easier/simpler (and sometimes faster) to build x86_64-based Docker images. Naturally, I’m using an SSH connection in those cases. Until this past weekend, however, I hadn’t really made the time to look deeper into how to use SSH with the Pulumi Docker provider. In this post, I’ll share some details that (unfortunately) haven’t yet made it into the documentation about using SSH with the Pulumi Docker provider.
Read more...
19 Jan 2024
Welcome to Technology Short Take #173! After a lull in links to share last time around, it looks like things have rebounded and folks are in full swing writing new content for me to share with you. I think I have a decent round-up of links for you; hopefully you can find something useful here. Enjoy!
Read more...
5 Jan 2024
Welcome to Technology Short Take #172, the first Technology Short Take of 2024! This one is really short, which I’m assuming reflects a lack of blogging activity over the 2023 holiday season. Nevertheless, I have managed to scrape together a few links to share with readers. As usual, I hope you find something useful. Enjoy!
Read more...
3 Jan 2024
Because Pulumi operates declaratively, you can write a Pulumi program that you can safely run (via pulumi up) multiple times. If no changes are needed—meaning that the current state of the infrastructure matches what you’ve defined in your Pulumi program—then nothing happens. If only one resource needs to be updated, then it will update only that one resource (and any dependencies, if there are any). There may be times, however, when you want to force the replacement of specific resources. In this post, I’ll show you how to target specific resources for replacement when using Pulumi.
Read more...
18 Dec 2023
I’m a big fan of direnv, the tool that lets you load and unload environment variables depending on the current directory. It’s so very useful! Not too terribly long ago, I wanted to find a way to “dynamically activate” the Azure CLI using direnv. Basically, I wanted to be able to have the Azure CLI disabled (no configuration information) unless I was in a directory where I needed or wanted it to be active, and be able to make it active using direnv. I finally found a way to make it work, and in this blog post I’ll share how you can do this, too.
Read more...
15 Dec 2023
Building on the earlier article on automatically transforming Git URLs, I’m back with another article on a (potentially powerful) feature of Git—the ability to conditionally include Git configuration files. This means you can configure Git to be configured (and behave) differently based on certain conditions, simply by including or not including Git configuration files. Let’s look at a pretty straightforward example taken from my own workflow.
Read more...
11 Dec 2023
Git is one of those tools that lots of people use, but few people truly master. I’m still on my own journey of Git mastery, and still have so very far to go. However, I did take one small step forward recently with the discovery of the ability for Git to automatically rewrite remote URLs. In this post, I’ll show you how to configure Git to automatically transform the URLs of Git remotes.
Read more...
8 Dec 2023
Welcome to Technology Short Take #171! This is the next installation in my semi-regular series that shares links and articles from around the interwebs on various technology areas of interest. Let the linking begin!
Read more...
17 Nov 2023
In January 2016, I published the first-ever episode of the Full Stack Journey podcast. In October 2023, the last-ever episode of the Full Stack Journey podcast was published. After almost seven years and 83 episodes, it was time to end my quirky, eclectic, and unusual podcast that explored career journeys alongside various technologies, products, and open source projects. In this post, I wanted to share a few thoughts about saying goodbye to the Full Stack Journey.
Read more...
12 Oct 2023
(This is a guest post by Simen A.W. Olsen.)
Pulumi recently shipped Pulumi ESC, which adds the “Environment” tab to Pulumi Cloud. For us at Bjerk, this means we can move secrets into a secrets manager like Google Secrets Manager. Let me show you how we did it!
Read more...
11 Sep 2023
Appropriately tagging resources on AWS is an important part of effectively managing infrastructure resources for many organizations. As such, an infrastructure as code (IaC) solution for AWS must have the ability to ensure that resources are always created with the appropriate tags. (Note that this is subtly different from a policy mechanism that prevents resources from being created without the appropriate tags.) In this post, I’ll show you a couple of ways to assign tags by default when creating AWS resources with Pulumi. Code examples are provided in Golang.
Read more...
8 Sep 2023
Welcome to Technology Short Take #170! I had originally intended to get this published before the long Labor Day weekend, but didn’t quite have it ready. So, here you go—here’s your latest collection of links from around the internet focused on data center and cloud-related technologies. I hope that you find something useful here.
Read more...
28 Aug 2023
Both Jason Snell and John Gruber, both stalwarts in the Apple journalism world, have recently weighed in on this topic. Jason says he’s given up on the iPad-only travel dream; John says he keeps throwing his iPad in his bag when he travels, even if he never uses it. I have thoughts on this topic—as you might think, considering I decided to write about it! (Ah, but what device did I use to write?)
Read more...
Older Posts
Find more posts by browsing the
post categories,
content tags, or
site archives pages. Thanks for visiting!
