2024–W42 #
As I've finished at archive.org I did some hardware maintenance: - Moved the NVMe SSD from the 1U HP box to my Mini-ITX machine - Moved the SATA SSD from the NUC into a second bay in the 1U HP server - Left both powered off
See, since I have extra compute capacity from Kimsufi I am leaning towards running less from my home. I still have feelings about self-hosting, like with ttw, but also about the amount of hardware complexity I have invited for no reason other than to have something to tinker with. When that tinkering brings joy: go for it, but when it feels like a time-sink or drudgery: not so much!
I'll still end up running something from my house, probably the Acer business computer with an NVMe and a single large HDD to serve as on-site backup -- but some of other things which are dynamic (like ts.binch.top, ironically) as well as my music collection and video library could actually be served from static file storage (or even object storage + a nice frontend) instead of requiring raidz volumes and filesystems -- as cool as those are.
This ties in with one of my "avoid mental pitfalls" values ("Virtual can be better than physical") and one of the areas I wanted to learn more about for my thoughts about my future in computing ("Serverless databases").
Virtual can be better than physical
This is a reminder that tinkering with lots of old, small or underpowered computing doesn't give me that much joy. I don't need a dedicated NetBSD or Haiku machine: I can use a VM, on my already very capable VM host. The same goes for trying to acquire or fix-up vintage hardware: I, personally, would be as happy or happier with an emulator, and sometimes need to remind myself of this.
Serverless databases
Having run various databases for organizations for almost my whole career, I think that many deployments of many database technologies are overkill, and we are doing them by rote or because we learned out-of-date lessons. I think other people feel this too, given the rise of SQLite to solve all sorts of problems people would have jumped straight to PostgreSQL or MySQL for in the past.
My basic theory is that, given good locking primitives and data-structures, many light uses of databases could be replaced with flat files or object storage. Cache is cheap, and loading entire pages from SSD is so much faster than it was from spinning-disk (and seeking is almost free). That's not a new observation, it underpins things like ScyllaDB's approach -- but they are in the business of selling a product and taking that observation and solving "big" production data problems with it. I'm much more interested in solving small problems:
- How many uses of Zookeeper or
etcdcould be an S3 bucket with CAS/PutIfAbsent semantics? - How do RocksDB/LevelDB's LSM architectures map on to object storage?
- Kafka KIP 405 lets Kafka use object storage for the cold segments which aren't being written to -- but all access is still intermediated through Kafka brokers, requiring servers. What would it look like to consume data straight from a bucket and not need a client library or a daemon when bulk-loading?