Tag: english

Upgrading my cluster

My cluster is now running on k3s 1.20.6 and Argo CD 2.0.0 with its Helm chart at 3.2.2. Actually, upgrading Argo itself wasn’t much of a problem. I just changed the targetRevision of the Application and it was up and running in a few minutes. Then a few days later things got interesting.

There were no downtimes, but I noticed that Argo started failing to sync itself. Apparently a new minor version of the Helm chart came out (though it was still the same application version) that added support for the networking.k8s.io/v1 version of Ingress. However, it also accidentally broke clusters running Kubernetes before 1.19. And mine was one such.

While the Argo people are figuring out how to fix this (if), I decided to go and take this opportunity to upgrade my cluster. This wasn’t as painless as it should’ve been though.


Gear I want and gear I can’t seem to find

The mountain gear I look for the most is: very light but durable protection for my limbs. That means gloves and boots that are as light and breathable, quick-drying as possible so my hands and feet don’t rot in sweat in the summer heat, while being durable enough not to fall apart scrambling over rocks in the Japanese alps.

Rocky route in the Japanese alps (at Nishi-Hotaka)

Dealing with DiskPressure

My 4-node k3s cluster (where this blog is hosted too) kept dying every now and then. Looking at kubectl describe nodes it quickly became evident that this was caused by the nodes running out of disk space. Once a node gets tainted with HasDiskPressure, pods might get evicted and the kubelet will be using (quite a lot of) CPU trying to free disk space by garbage collecting container images and freeing ephemeral storage.

My setup by default uses local storage (the local-path provider) where volumes are actually local folders on the node. This means that pods that use persistence are stuck with the same node forever and can’t just move around. This makes eviction a problem, since they have nowhere else to go. It also means that disk usage is actually disk usage on the node, and not on some block volume over the network.


Down the __extmap rabbit hole

Was trying to figure out what defrecord does in the impl of ILookup it generates and I absolutely do not understand that magic. Essentially it amounts to (get __extmap key)  and I can’t even figure out how this magical __extmap gets into scope (a global or some special runtime thing?) and this isn’t even used. Confused

zerusski on Clojurians Slack

This post on Clojurians sent me down the rabbit hole looking for where this “mysterious” __extmap comes from. It really isn’t obvious at first, but when I found it, it was “wow” times “duh”.


Beyond burger

Pretty much starved and three more hours to go until my flight out of Dubai, I was looking for food. Something filling. Tokyo destroyed my plan of eating something before taking off there since there was literally not a single shop open in Narita airport. Luckily Dubai wasn’t dead like that so I had plenty to choose from.


Mount Eggplant!

One of the 100 famous Japanese mountains in reach for a day hike from Tokyo is Mt Nasu (which in Japanese is a homophone with “eggplant”) in Tochigi. It’s easy (though not particularly cheap) to get there by (bullet) train and bus. This time I went for a quick hike to the Chausu peak (which is a much shorter climb than the highest Sanbon-yari). The weather wasn’t exactly great, cloudy and extremely windy, but at least it wasn’t dumping on me.


QUIC and quicker

The past week or so I’ve been working on implementing the QUIC protocol in Clojure. Currently there is no Java implementation to use either (that I know of), and I just found out the other day that netty decided to use the Cloudflare’s Rust library quiche under the hood instead of rolling their own. The protocol is currently a IETF draft at version 32, expected to turn into an RFC soon.


GitOps with Argo

It’s been a year since I wrote about bootstrapping a cluster with Argo and using Argo Rollouts for canary deploys based on Prometheus metrics. Since then many things have changed. I moved from Digital Ocean to Linode (mostly because Linode has a Tokyo region) and from a single-node k3s “cluster” to a 4-node one. But most of how I use Argo CD for GitOps hasn’t changed.

orange and brown tree branch

Clojure and Java functional interfaces

Java 8 came out in 2014 and brought along functional interfaces. Functional in general just means that you can treat functions (or methods) as “things” instead of having no proper way to talk about them. In this sense Javascript for example is functional: you can pass around functions all you want. Java’s had Runnable and Callable that are pretty similar in concept.

Then came Java 8 and with it the “mighty arrows.” For some reason Ruby, Javascript and Java all opted to use the same bit of syntax to talk about lambdas (anonymous functions): ->. In Ruby it’s ->(foo) { foo }, in Javascript it’s (foo) -> foo, and surprisingly in Java it’s the same. Run a few rounds with futures and/or streaming stuff and you’ll definitely want to pass such a lambda to forEach for example.


Clojure proxy

Next up in the series complaining about Clojure’s Java interop is proxy. While vararg method calls are inconvenient at worst, there are some (I’d say common) things that simply cannot be achieved with proxy.

Once again this is something I ran into while working with Netty. In one of the HTTP/2 examples, they have one implementation extending AbstractHttp2ConnectionHandlerBuilder<T, B> (have I mentioned I find these extremely long Java class names just hilarious?). The Java implementation is pretty straightforward: implement the abstract method of the class and be done with it.