> There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.
Why does AWS insist on building this way? Has anyone tried to get the teams together and say, "stop, let's evaluate and simplify where we can?"
As an AWS customer, I find AWS a huge pile of overcomplicated offerings that is dense with its own jargon and way of doing things. IAM is a trash fire. Everything built on IAM like IRSA is also a trash fire. Why are there managed worker nodes, spot managed worker nodes, Fargate worker nodes, self-managed worker nodes, and spot self-managed worker nodes? Why are there circular dependencies for every piece of infra I stand up making it almost impossible to delete any resources? Why can't I click on a service and see how much it costs me in one or two clicks?
This is a completely insane way of building software to me and I have to eat it.
Amazon has "Invent and Simplify" as one of the leadership principles. However, each team has its own hiring bar and culture, so some will take it seriously and some will not.
> I find AWS a huge pile of overcomplicated offerings
I fully agree. I'm speaking exclusively about the internal infrastructure, not about AWS.
I can't even imagine, sometimes I feel like AWS offers this pretty facade of "scale it up easy" and in reality its like "wait, I can't even see why this thing is failing because its a layer below, and AWS support costs mega bucks"
Fascinating because I would have thought the whole idea is AWS is dog-fooded by retail. Yet in truth AWS is itself too expensive and / or complex for Amazon itself! (Or at least the original part of it.)
I'm curious what you mean here, since CAPI allows you to manage and scale clusters like any other Kubernetes object. You can literally "kubectl scale --replicas" a cluster control-plane and worker nodes.
You end up needing this anyway, because Kubernetes clusters are confined to single geographic regions and you need some way of managing the separate cluster(s) you have in each region
Not saying you should, but it's entirely possible to have nodes all over the world managed by a single set of master nodes. The system is not really designed with that use case in mind though, the added latencies will hurt a lot in getting etcd consensus.
While it is certainly possible to have mixed region clusters, from my experience it is better to have one cluster per region. Do your synchronous communication within the cluster / within the datacenter with sub millisecond latencies and do asynchronous communication across cluster boundaries with up to triple digit millisecond latencies.
In a way this reminds me of Kubernetes Virtual Clusters. Each virtual cluster has its own tenant control plane, namespaces. Multiple virtual clusters exist in a super cluster.
I think at a certain point up in the stack you don't want to have another orchestrator for the orchestrator. You would just tell each team to keep their cluster up-to-date without creating a (dangerous) master button to update them all at once.
That's what Federation in Kubernetes was for, sort of, but it has never really come to fruition. The closest I've seen is Admiralty, but I'm not sure how it actually works out in reality.
Kubernetes solves a problem. It isn't appropriate for all use cases, but the real problem isn't kubernetes but rather the knowledge gap around new technology.
