It was true in 2005, but still? As I described in another post here, a modern strategy is to get a beefy server than can run the same ABI, then start a docker container and assemble a system from Alpine's package repo, then compile a kernel and a few in-house things, then extract a subset of that into an image. No cross-compile, and most of the useful software is in pre-built binary packages with the compile-time options you would have selected anyway.
Even if you don't have a beefy server of the same architecture, you can probably run it in qemu instead of docker to the same effect. And even if qemu is slow, you can run a build of the kernel and your in-house stuff in parallel on 64 cores and not really be affected by the qemu slowdown.
What if this hegemony in ISAs is partially because of the high costs of porting software and toolchains?
Maybe the next generation of embedded devices will have ISAs that are, say, extremely optimized for real time industrial controllers, but terrible at compiling C++.
It seems a little silly to give up on cross-compilation just because _today_ everything runs on ARM.
Especially because cross-compilation is not a particularly hard problem, the problem is with decades of tooling assuming the build and target environments are the same.
Well, I'd say cross-compiling is at least a little problematic... if nothing else, it is a hurdle to running automated tests. And cross-compiling still needs to exist for the people building the package repositories. I'm just sort of advocating that if you have limited amounts of time but want to put together a system image to run in an embedded or embedded-adjacent manner, the least pain / most agile solution is by building on top of a popular binary distro and then using native compilation on appropriate hardware inside a chroot or container that resembles the target.
"Ampere processors natively support both 32 & 64 bit Android applications and require no binary translation for maximum instance density"
...
"Run up to 120+ 3D cloud game instances per socket"
...
"DRAM: 384GB - 512GB"
I've not personally tested this, but it very much appears that just like x86, you can use a 64-bit ARM kernel that has 32-bit support enabled to run a 32-bit userspace, likely compatible with docker, or even just in a chroot. Then with 120 cores, "make -j 120"
I'm not familiar with the problems that would cause "target build environment often differs significantly from your server environment". If you have an ARM server, then you should be able to download and run Alpine docker images, and then fetch most system utilities from the Alpine package repo and then compile a kernel and a few other binaries on top of that, then export an image.
The main reason to not run Debian is that Debian usually makes lots of compile-time decisions for you, and chooses to maximize the enabled options of the software. Alpine makes the opposite choices for you, creating very minimal feature sets.
It's been working well for me, but my use cases is a lot more like a custom in-house DDWRT firmware kind of a thing. If you've experienced additional constraints that make this pattern unworkable, I'd be interested to hear about them.
I share some of the same observations that seem to have motivated this project, but my solution was to just use Alpine on the same architecture as the target (possibly via qemu) and then export a subset of it to a filesystem image. I implemented it as a perl module collection with no dependencies other than core perl modules, and then run the export from within the image itself. Among other things, it lets me use strace to find the runtime library dependencies of the things that need to be in the image.
Implementing a WoW classic server is actually fairly easy. The game client comes with the entire engine, art, music, and quest content. The server is basically a fancy IRC server, taking client events and rebroadcasting them to other clients.
Even many of the events are implied, like how regular attacks continue at a fixed frequency once started, so other clients only need to know when the player started attacking and whether they are still in range, and player run speed is a constant so a player running in a straight line doesn't generate additional events.
I even suspect the dice rolls are coming from a shared RNG that each client maintains independently, but haven't researched it.
This is how WoW classic was playable over a 33K modem.
Offline, the client knows movement, graphics, statics (the world mesh, excluding the location and functionality of npcs, doors, plants, etc.) and some localized text. Almost all the gameplay logic is server-side.
We're talking about hit games created specifically as a sequel to a hit mod of another game, and communication to the community of the hit mod that this is where the developers are going, and that they should move to the standalone game if they want to thank the developers for all that unpaid work they did on the mod over the years.
It was a "hit game" while it was still a mod. They were able to find investment to graduate to a standalone game because they already had a player base in the tens of thousands.
Yeah but for $6/mo you can get a tiny linode or digital ocean droplet, and not worry about hardware failing. It's true that a laptop probably has more resources than the smallest VMs, but no remote management interface and can't scale if you suddenly had a surge of traffic.
> Yeah but for $6/mo you can get a tiny linode or digital ocean droplet
That gets you, what, 1 "vCPU" with maybe a gig of ram and a couple of dozen gig of disk.
If you (or a friend) work for a company of any size, there's probably a cupboard full of laptops that won't upgrade to Win11 sitting there doing nothing that you could get for free just by asking the right person. It'll have 4 or 8 cores, each of which is more powerful that the "vCPU" in that droplet. It'll have 8 or maybe 16gig of ram, and at least half a TB of disk and depending on that laptop quite likely to be able to be configured with half a TB of fast nVME storage and a few TB of slower spinning rust storage.
If you want 8vCPUs/cores, 16GB of ram, and 500GB of SSD, all of a sudden Digital Ocean looks more like $250/month.
If you are somewhere in that grey area where you need more than ivCPU and 1GB of memory, grabbing the laptop out of the cupboard that your PM or one of the admin staff upgraded from last year and shipping not off to a datacenter with your flavour of linux installed seems like it's worth considering.
Hell, get together with a friend and have two laptops hosted for 14Euro/month between you, and be each others "failing hardware" backup plan...
I bet colos will plug a KVM into your hardware and give you remote access to that KVM. I also bet rachelbythebay has at least one article that talks about the topic.
> ...can't scale if you suddenly had a surge of traffic.
1) If your public server serves entirely or nearly-entirely static data, you're going to saturate your network before you saturate the CPU resources on that laptop.
2) Even if it isn't, computers are way faster than folks give them credit for when you're not weighing them down with Kubernetes and/or running swarms of VMs. [0]
Yeah. I got bored a couple of hours after I posted that speculation and found several other colo facilities that mentioned that they'd do remote KVM. I'd figured that it was a common thing (a fair chunk of hardware you might want to colo either doesn't have IPMI or doesn't have IPMI that's worth a damn), but wasn't sure.
You (the person paying to co-locate hardware) don't buy the KVM that the colo facility uses. The colo facility hooks up the KVM that they own to your hardware and configures it so that you can access it. Once you stop paying to colo your hardware, you take your hardware back (or maybe pay them to dispose of it, I guess) and they keep the KVM, because it's theirs.
k8s doesn't really weigh you down, especially if tuned for the low end use case (k1s). It encourages some dumb decisions that do, such as using Prometheus stack with default settings, but by itself it just eats a lot of ram.
Now using CPU limits in k8s with cgroups v1 does hurt performance. But doing that would hurt performance without k8s too.
> k8s doesn't really weigh you down, especially if tuned for the low end use case (k1s).
Sorry, what "k1s" are you referring to? The only projects with that name that I see are either cut-down management CLIs and GUIs that only work with a preexisting Kubernetes cluster, or years-old abandoned proof-of-concept/alpha-grade Kubernetes reimplementations that are completely unsuitable as a Kubernetes replacement.
The only actually-functional cut-down Kubernetes I'm aware of is 'minikube'. Minikube requires -at minimum- 2 CPUs, 2GB of RAM and 20GB of disk space. [0] That doesn't fit on either of the machines 'nrdvana' was talking about... not the "tiny ... digital ocean droplet", with its 1CPU, 1GB of RAM, and 25GB of disk space [1], nor the "tiny linode" (which has roughly the same specs [2]).
Given that it's not at all uncommon for discarded laptops to have 4 CPUs, 8GB of RAM, and like 250GB of disk, eating 1/4th of the RAM, (intermittently) half of the CPU power, and roughly a tenth of the disk space just for Kubernetes housekeeping kinda sucks. That's pretty damn "heavy", in my judgment. So. Do you have a link to this 'k1s' thing you were talking about? Does it use less than 2 CPUs, 2GB of RAM, and 20GB of disk?
There are a couple of things here: first off, minikube isn't tuned for low resources, it's a full k8s packaged for developers. Second, it doesn't burn much CPU at all unless you blew it up and it's churning pods. Third, try to remember that you're running a tool that provides most of what you need to run a service, it's not a bare VM, it comes with reverse proxy, tons of tools, and cluster management.
Basically those minimal specs let you actually run quite a lot of stuff on that minikube, they're not just for the management system.
k0s needs roughly 500MB RAM and 1.5GB drive to run a controller+worker. Can probably pare k3s down to that as well.
And I repeat, this gets you single pane cluster management across all your laptops, reverse proxy, DNS, resource shaping, namespaces, etc. Only big problem with it is that adding distributed storage is quite heavy and unstable (longhorn) or really heavy (ceph), so storage management might need to be manual which is a pain in k8s.
You noticed that I said "(intermittently) half of the CPU power", yeah?
> Third, try to remember that you're running a tool that provides most of what you need to run a service...
I already get everything that I need to run a service with old-ass systems like OpenRC+netifrc or -hack, gag- systemd and its swarm of dependencies. "Run a service on a *nix box" is a thing that we've had pretty well nailed down for decades now. It is -after all- how the services that run Kubernetes get run. Do note that you're talking to someone who does Linux system administration both as a hobby and professionally.
> Sorry I meant k0s. Off by one error at 3 am.
Sure, no problem. Shit happens.
So, k0s? Compared to minikube, the official minimum spec tables [0] indicate that -if you colocate the controller and worker- it cuts the CPU and RAM needs in half, and cuts the disk space by a factor of ten. That's nice, but that's still an eighth of the RAM and (intermittently) a quarter of the CPU of our hypothetical-but-plausible castoff laptop. That's still a lot of resources. And compared to what it costs you, you don't get much. If we were talking about some big, bad, mutually-untrusted-multitenant situation, it could be worth the cost, but -despite what folks like the CNCF might like you to believe- that's not the only scenario out there.
Also Mirantis is responsible for k0s? [1][3] After their rugpull with their Openstack distro way back when, I don't trust that they'll keep maintaining and providing complex stuff that's free to use for long enough to make it worth making a critical part of one's business. (Yes, this has absolutely nothing to do with its resource usage. I'm absolutely not bringing it up to support that argument in any way. I "just" thought it important to mention that I don't trust that Mirantis's free stuff will continue to be free for long enough to safely build (more of?) your business on.)
[1] From [2] "Mirantis offers technical support, professional services and training for k0s. The support subscriptions include, for example, prioritized support (Phone, Web, Email) and access to verified extensions on top of your k0s cluster."
[3] As additional evidence, the only two humans on the first page of commits to the k0s repo are Tom Wieczorek, whose Github profile indicates his affiliation with Mirantis [4], and Jussi Nummelin who is very, very obviously a Mirantis employee. [5] I tried to look at the complete Github contributor list for the k0s repo, but it simply wouldn't load. [6] But, I'd be shocked if this wasn't Mirantis's totally-functional-but-still-pet project that intends to more than make up the cost of development and maintenance with support contracts.
Hey I'm not arguing that you in particular shouldn't use old tech for fun. Heck serve your emacs written internet website via nc from your Amiga for all I care.
I'm just pointing out that it's easy and sufficiently efficient to run k8s on old computers, which makes running homelabs quite easy and also allows projects such as the OP to really shine. You seem to really enjoy telling me how bad it is that k8s needs 0.05 CPU and 500MB of RAM to run but the thing is it will scale horizontally a lot, and also presents the APIs you'll be expected to know in a devops job in 2026.
Maybe at rendering menus and documents, but flash had graphic routines written in optimized assembly that simply weren't possible with JavaScript on that era of hardware.
I feel like people are talking past each other a bit here. FlashScript was never very fast, and rendering a document as a giant collection of bezier curves was not fast, but the people doing animations with it were getting the equivalent of modern day CSS3 animations + SVG, and it ran nicely on hardware two orders of magnitude slower than what we need for CSS3+SVG
Maybe the large number of standard library functions that operate on globals and require you to remember the "_r" variant of that function exists, or the mess with handling signals, or the fact that Win32 and Posix use significantly different primitives for synchronization? Or maybe just the fact that most libraries for C/++ won't have built-in threading support and you need to synchronize at each call site?
Unless I'm writing Java, I avoid multithreading whenever possible. I hear it's also nice in Go.
Even if you don't have a beefy server of the same architecture, you can probably run it in qemu instead of docker to the same effect. And even if qemu is slow, you can run a build of the kernel and your in-house stuff in parallel on 64 cores and not really be affected by the qemu slowdown.
I'm interested to hear counterexamples, though.
reply