🚧 Oh, finally! An article that doesn't just analyze #Kafka but also gives us the complete #genealogy of #Jack Vanlightly's blog posts. 🙄 Who knew we needed a #diskless Kafka #roadmap that could double as a sleep aid? 💤
https://jack-vanlightly.com/blog/2025/10/22/a-fork-in-the-road-deciding-kafkas-diskless-future #Vanlightly #SleepAid #HackerNews #ngated
A Fork in the Road: Deciding Kafka’s Diskless Future — Jack Vanlightly

“ The Kafka community is currently seeing an unprecedented situation with three KIPs ( KIP-1150 , KIP-1176 , KIP-1183) simultaneously addressing the same challenge of high replication costs when running Kafka across multiple cloud availability zones. ” — Luke Chen, The Path Fo

Jack Vanlightly
A Fork in the Road: Deciding Kafka’s Diskless Future — Jack Vanlightly

“ The Kafka community is currently seeing an unprecedented situation with three KIPs ( KIP-1150 , KIP-1176 , KIP-1183) simultaneously addressing the same challenge of high replication costs when running Kafka across multiple cloud availability zones. ” — Luke Chen, The Path Fo

Jack Vanlightly

Outra coisa que eu estava pesquisando na madrugada: sistemas #diskless. Explico: com a profusão de pequenos computadores chineses, vários deles baseados em processadores Ryzen de bom desempenho, fiquei pensando no seguinte:

"Para fins de otimização tributária e maximização de utilidade, não faz mais sentido importar o sistema sem RAM e disco? Vários fabricantes oferecem a opção e assim é mais fácil configurar do jeito que eu quero..."

Na sequência, pensei que faria sentido importar sem RAM e sem disco e colocar apenas RAM. Com a economia, é possível ter mais RAM, mas isso exige boot pela rede. Eis que comecei a pesquisa.

Depois de me deparar com uma documentação desatualizada desde 2017 no #Debian, acabei descobrindo que o #LTSP segue vivo. Desde 2019, o projeto foi reorientado. Os clientes não podem ser leves agora, pois há todo um "encantamento" baseado em squashfs, o qual demanda mais RAM, porém, o desempenho tende a ser melhor e o servidor não precisa servir desktops inteiros pela rede (algo que, graças à ineficiência atual de toolkits, Wayland, etc, é pesado, já que virou streaming de vídeo).

Para clientes modestos, ainda existem outras opções, com ou sem LTSP envolvido.

So, #Kafka wants to go "diskless" now? 🤔 Because all that #data #storage is, like, soooo 2022. 🎉 Next up: a Kafka plugin that stores your data in the clouds... literally! ☁️✨
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1150%3A+Diskless+Topics #Diskless #Cloud #Innovation #CloudComputing #DataStorage #2023TechTrends #HackerNews #ngated
KIP-1150: Diskless Topics - Apache Kafka - Apache Software Foundation

KIP-1150: Diskless Topics - Apache Kafka - Apache Software Foundation

Warewulf is a stateless and diskless container OS provisioning system — https://github.com/warewulf/warewulf
#HackerNews #Warewulf #Diskless #Containers #OS #Provisioning #Hackers #News #OpenSource
GitHub - warewulf/warewulf: Warewulf is a stateless and diskless container operating system provisioning system for large clusters of bare metal and/or virtual systems.

Warewulf is a stateless and diskless container operating system provisioning system for large clusters of bare metal and/or virtual systems. - warewulf/warewulf

GitHub

@SweetAIBelle very nifty...

Ideally with some nifty tools (compareable to Tails' #PersistentSotrage) to allow mounting local/remote storage as $HOME and potentially leverage toram as boot option.

Certainly that "#RescueLinux" part is mid- to ongterm goal of OS/1337 and I am working on it (abeit at a pace slower than I want to cuz 'things happen in meatspace' and those sadly require my full attention...

  • Needless to say it's always good to have a nifty system at hand (worst-case I'd install @ubuntu / #UbuntuLTS #Desktop on an external SSD/HDD and have that at hand) if one doesn't have their Distro on a hot-swappable / toolless replaceable drive anyway...

Using #ArchLinux is quite good because it makes it easy to get started and adapt said system as need be.

#OS1337

@dalias @eniko Personally, I'll sooner or later have to engange the whole #ARMv5 / #ARM11r7 / #RaspberryPi architecture anyway with OS/1337.

  • Tho given it's a minimalist Linux merely booting an initramfs and never doing any writes, there should be no indurance issues.

But I know a #readonly - OS isn't practical and where it is people already use #iPXE & #iSCSI for #diskless setups!

  • THO AFAICT there should be an option to basically partition the system and only keep /boot on an SD card, and the rest could be on any USB mass storage device (i.e. SSD or even HDD)...

That being said Raspberry Pi do have the key advantage of being by far the best in terms of #documentation.

#OS1337 #Linux #distro #development

@uastronomer it's something I.did implement in the past (abeit #KVM + #Proxmox, but the steps are similar enough):

You can seperate #Storage and #Compute given you have a Storage-LAN that is fast enough (and does at least 9k if not 64k Jumbo Frames) and have the "Compute Nodes" entirely #diskless (booting via #iPXE from the #SAN) and then mount the storage via #iSCSI or #Ceph.

  • Basically it allows you to scale Compute and Storage independently from each other as they are transparent layers and not be confined to limits of a single chassis & it's I/O options...

Did a bigger project (easily 8-digits in hardware, as per MSRP) where a Employer/Client did #CloudExit amidst escalating costs and #ROI being within quarters (if not months at the predicted growth rate)...

Hold on... #AlpineLinux's update-kernel tool actually has an option for adding kernel modules and firmware!

```
mount /dev/mmcblk0p2 /mnt/
mkdir /mnt/tmp
mkdir /mnt/boot
TMPDIR=/mnt/tmp/ update-kernel -a aarch64 -f rpi -p zfs-rpi -M /mnt/boot
```

Copy the files in /mnt/boot over to the root of the SD card. Job done! I just need to pay attention at the next Alpine update to make sure I rebuild the kernel at the same time.

This should really be in the wiki.

#ZFS #diskless