Arcology Garden

24 Followers
41 Following
112 Posts
technical documents from @[email protected].

a follow is used to populate my home instance's fedi timeline, and to determine whether to follow you on my main account. this is not an automated action, though posts from this account may be automated via an RSS bridge from my site feeds
wwwhttps://arcology.garden
NEW by @rrix: Per-page Custom Templates

The Arcology Project can now render some pages with per-page overrides of django's page template.

There are currently two:

I'll add more templates later on, I guess. I probably will change how the headers render in the Wide Format before too long.

https://engine.arcology.garden/None
The Arcology's Data Models and Web Server - The Arcology Site Engine

NEW by @rrix: Updated My "Uses This" page for 2025

For a long time, the website Uses This has published guest posts different creative people asking them a few simple questions:

  • who are you and what do you do?

  • what software do you use?

  • and what hardware?

  • what is your dream setup?

I always enjoyed reading these because sprinkled in between Macbbook UX designers and people with important-sounding email jobs were people who had honestly exciting views of computing, especially dreaming about what it could be, like I enjoy doing. Uses This posts havent had an interesting thing to say about computing in the last 2-3 years, it seems like people have lost their ability to dream big about computers.

So here's mine as of July 2025: rrix "Uses This"

And here are some particularly interesting ones:

  • mhoye (2019): "What I really want is to be able to not only share all of that but to describe how I've tailored it to my needs in some complex coordinate space. I want some sort of possible-configuration map, a cartography of collective customization. I'd like to be able to start exploring changes to my setup - my workspace, my editor, browser, terminal or keymap, and then put a pin on that map, to plant a flag saying, I am here. I'm learning about the machine, I'm learning about myself and that journey has brought me here. For some value of "here"."

  • joey.hess (2012): "the whole house runs on 12 volt DC power to avoid the overhead of an inverter; my laptop is powered through a succession of cheap vehicle power adapters, and my home server runs on 5 volt power provided by a USB adapter."

  • alice.maz (2020): "One of the side benefits of doing most of your work with the people you live with and love is you get to plan infrastructure on decades-long timescales. In this spirit, instead of describing my dream setup, I'll describe the setup I intend to have in five years. If nothing else this will be a fun thing to look back on and see how right or wrong I was!"

  • clay.shirky (2014): "I actually don't want a "dream setup." I know people who get everything in their work environment just so, but current optimization is long-term anachronism. I'm in the business of weak signal detection, so at the end of every year, I junk a lot of perfectly good habits in favor of awkward new ones."

  • ryan.north (2012): "Actually if we're talking dreams, the laptop is so tiny it fits in my pocket and unfolds itself, Transformer-style, to full size. Also, it's a phone. And it can call THE FUTURE."

  • charlie.loyd (2016): "Its weirdness comes from the hand and mind of a master, and it's precious to me. My dream setup has more of that feeling."

  • lex.gill (2017): "I have almost superstitious ideas about how spaces change the way we think and work. I think being in the right environment can encourage people to be their best selves."

  • kieran.healy (2010): "There are many things I dislike about the writing/data/coding setup I use. Emacs carries so much baggage with it, it's tiresome. Some of this is my own fault, what with the array of settings that accrete over time. But some of it is just stuff inevitably carried forward from the 1970s and '80s in the application's conventions, interface, core design, and so on. I resent its lack of consistency with the rest of my operating system. I resent the way its preferred means of solving your problems is first to have you surrender to it completely. Mike Hall's brilliant "Org-Mode in your Pocket is a GNU-Shaped Devil" captures this feeling very well."

  • kicks.condor (2020): "Envision a room full of millions of people holding one giant, hugely sprawling conversation. Such a thing has never been possible. I think if I could one day find myself in that room, even as a simulation, and it felt real - and I knew that it was just my sister and I having a conversation through these myriad expressions of us - it would be a dream come true."

  • rob.pike (2012): "In summary, it used to be that phones worked without you having to carry them around, but computers only worked if you did carry one around with you. The solution to this inconsistency was to break the way phones worked rather than fix the way computers work."

  • why the lucky stiff (2009): "the other perfect thing would be if the Internet could be hand written. If I ever get a time machine, I'm going to go back and smash the Gutenberg press so that when the Internet arrives, it'll be transcribed by monks the way GOD INTENDED."

What is your dream setup?

https://arcology.garden/updates#20250722T062128.540393
Uses This / Interviews

A nerdy collection of interviews asking a random selection of people all about the tools and techniques they're using to get things done

NEW by @rrix: Blocking Aggressive Scrapers at the Edge

In Limiting expensive to render nginx endpoints , I describe how to use a few nginx limit_req module to substantially limit the amount of aggressive scraping traffic to my Gitea instance without impacting "normal" "human" behavior.

There's three layered rate-limiters in here that are applied to only certain URIs:

  • One does a per-IP limit excluding my Tailscale network and some ASNs I connect from. Each IP can make one costly request per minute, otherwise receive a 503.

  • One tries to map certain cloud providers in to a single rate-limit key and gives each of these providers 1 RPM on these endpoints. Each group of cloud IPs can make one request per minute, otherwise receive a 503.

  • One puts a limit to 1 RPS of all traffic on each "site feature" in Gitea.

So now if you try to browse my Gitea instance http://code.rix.si or make a git clone over HTTP that will work just fine, but a handful of expensive endpoints will be aggressively rate-limited. If you want to look at the git blame for every file in my personal checkout of nixpkgs, you can do that on your own time on your own machine now.

So far installing this on my "edge" server seems to work really well, cutting the load of the small SSL terminator instance in half. Let's see if this is Good Enough.

https://cce.whatthefuck.computer/updates#20250320T130459.421338
Limiting expensive to render nginx endpoints - The Complete Computer

NEW by @rrix: Blocking Aggressive Scrapers at the Edge

In Limiting expensive to render nginx endpoints , I describe how to use a few nginx limit_req module to substantially limit the amount of aggressive scraping traffic to my Gitea instance without impacting "normal" "human" behavior.

There's three layered rate-limiters in here that are applied to only certain URIs:

  • One does a per-IP limit excluding my Tailscale network and some ASNs I connect from. Each IP can make one costly request per minute, otherwise receive a 503.

  • One tries to map certain cloud providers in to a single rate-limit key and gives each of these providers 1 RPM on these endpoints. Each group of cloud IPs can make one request per minute, otherwise receive a 503.

  • One puts a limit to 1 RPS of all traffic on each "site feature" in Gitea.

So now if you try to browse my Gitea instance http://code.rix.si or make a git clone over HTTP that will work just fine, but a handful of expensive endpoints will be aggressively rate-limited. If you want to look at the git blame for every file in my personal checkout of nixpkgs, you can do that on your own time on your own machine now.

So far installing this on my "edge" server seems to work really well, cutting the load of the small SSL terminator instance in half. Let's see if this is Good Enough.

https://cce.whatthefuck.computer/updates#20250320T130459.421338
Limiting expensive to render nginx endpoints - The Complete Computer

NEW by @rrix: inotify file limits can be confusing/misleading

I've recently been watching a build for a package on my server fail inconsistently ~50% of the times I build updates, and I was confused as to why it was missing the binary cache in the first. Oh fun, how reproduceable. I spent an hour or two yesterday reading the code and the failing tests and realized it was because the test was stress-testing inotify system calls:

source: 
Traceback (most recent call last): File "/nix/store/zv1kaq7f1q20x62kbjv6pfjygw5jmwl6-python3-3.12.7/lib/python3.12/threading.py", line 1075, in _bootstrap_inner self.run() File "/build/source/src/documents/tests/test_management_consumer.py", line 30, in run self.cmd.handle(directory=settings.CONSUMPTION_DIR, oneshot=False, testing=True) File "/build/source/src/documents/management/commands/document_consumer.py", line 251, in handle self.handle_inotify(directory, recursive, options["testing"]) File "/build/source/src/documents/management/commands/document_consumer.py", line 294, in handle_inotify inotify = INotify() ^^^^^^^^^ File "/nix/store/3ziqbc4xcs58hhh5srx7pfl2n9mwj22g-python3.12-inotifyrecursive-0.3.5/lib/python3.12/site-packages/inotifyrecursive/inotifyrecursive.py", line 31, in __init__ inotify_simple.INotify.__init__(self) File "/nix/store/1qv923rjqijj7nbhhm9k1bz53jh9pb3a-python3.12-inotify-simple-1.3.5/lib/python3.12/site-packages/inotify_simple.py", line 91, in __init__ FileIO.__init__(self, _libc_call(_libc.inotify_init1, flags), mode='rb') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/1qv923rjqijj7nbhhm9k1bz53jh9pb3a-python3.12-inotify-simple-1.3.5/lib/python3.12/site-packages/inotify_simple.py", line 39, in _libc_call raise OSError(errno, os.strerror(errno)) OSError: [Errno 24] Too many open files

inotify is an API that allows your process to be notified when files are accessed, changed, deleted, etc. Because inotify queues can use a fair bit of memory, Linux implements specific interfaces to limit these calls:

The following interfaces can be used to limit the amount of kernel memory consumed by inotify:

/proc/sys/fs/inotify/max_queued_events The value in this file is used when an application calls inotifyinit(2) to set an upper limit on the number of events that can be queued to the corresponding inotify instance. Events in excess of this limit are dropped, but an INQ_OVERFLOW event is always generated.

/proc/sys/fs/inotify/max_user_instances This specifies an upper limit on the number of inotify instances that can be created per real user ID.

/proc/sys/fs/inotify/max_user_watches This specifies an upper limit on the number of watches that can be created per real user ID.

I found a Nixpkgs GitHub Issue about the build failure where folks saw the "too many open files" and assumed it was ulimit configuration issues on the build hosts that stumped even the NixOS Super Posters, but it's this other more obscure limit that nevertheless raises the same Errno when it's hit. Classic Linux! These values are set to arbitrarily low values by default, and I remembered that the inotify watcher in the The Arcology Project 's FastAPI prototype bumped up against these limits when I deployed it way back when. Classic Linux!

There are two machines in /etc/nix/machines, my Framework 13 Laptop and My Homelab Build . Presumably the build works on one but not the other, but I no longer explicitly set these values, so something in NixOS itself must be, this is easy enough to check:

shell source: :results drawer
pushd ~/arroyo-nix grep -ri fs.inotify.max_user_watches pushd ~/Code/nixpkgs grep -ri fs.inotify.max_user_watches ~/arroyo-nix ~/org ~/Code/nixpkgs ~/arroyo-nix ~/org nixos/modules/services/misc/graphical-desktop.nix: "fs.inotify.max_user_watches" = lib.mkDefault 524288; nixos/modules/virtualisation/lxd.nix: "fs.inotify.max_user_watches" = 1048576; nixos/modules/virtualisation/incus.nix: "fs.inotify.max_user_watches" = lib.mkOverride 1050 1048576; # override in case conflict nixos/modules/services/x11/xserver.nix

So it's set to a higher value by enabling LXD (which I believe Waydroid does) but also by enabling any graphical desktop. So the package would build on my laptop but not my server or any "stock" NixOS server... a 50-50 shot.

source: 
sudo sysctl -w fs.inotify.max_user_watches=524288 sudo sysctl -w fs.inotify.max_user_instances=524288

I set this temporarily on my server and the build ran reliably 5 times though the tests are sooooooo freakin' slow... I hope the Hydra instance that publishes the binary cache gets around to something like this, rather than disabling the tests that validate that document consumption in this document scanning/processing service works properly....

It can be set "for good" with this NixOS configuration:

nix source: 
boot.kernel.sysctl = { "fs.inotify.max_user_instances" = 524288; "fs.inotify.max_user_watches" = 524288; }; https://arcology.garden/grymoires/shell#20250116T105516.143422
inotify(7) - Linux manual page

NEW by @rrix: Fuck Time Two: Time Keeping Design Trends from 295BCE to ~2020CE

Last week Will Bradley and I presented for David Huerta 's birthday party various methods of keeping time poorly, and why one should consider doing so.

In 2014 shortly after I had moved to Oakland to join a startup live in a warehouse, Prescott Ogden participated in a hackathon hosted at HeatSync Labs which we've taken to calling "Fuck Time" in the ensuing years. Prescott introduces us to the concept of subjective time, we revisit it a decade on after the iPhone-iPhocation of all technology.

Will and I started developing custom watch faces for the 30$ PineTime smart watch over the last few years, and encourage others to do the same and to re-consider their understanding of the possible flows and scales of time in day to day life.

https://arcology.garden/updates#20241214T223849.387992
NEW by @rrix: Updated my "rrix uses this" page

After getting particularly incensed at another month of "i use a macbook pro and ffligma :) and i love it" posts on Uses This, I decided I needed to update rrix "Uses This" for the year 2024. So I did that.

https://arcology.garden/updates#20241116T230436.448021
Uses This / Interviews

A nerdy collection of interviews asking a random selection of people all about the tools and techniques they're using to get things done

NEW by @rrix: Cool Things from Fall 2024

Here's some of the most interesting things I read this season, finally entering my Archive:

BOOTHAMMER Aims to Deliver Simple "Intermittent Computing" for Energy-Harvesting Arduino Sketches

[2024-11-11 Mon 12:14]

the team introduces its project: BOOTHAMMER, a assembly rewriter targeting Arm's Thumb architecture which adds checkpoint and restore operations — automatically saving the progress of computation at intervals to protect against power loss and restoring from the saved point when power returns

Having resilient low-power systems, things that can handle low sunlight conditions common in the Willamette Valley's winters, is going to be an important thing as we start building more ambient/embedded computing. I would love to have little ultra-low power radio sensors attached to my Home Assistant server and handing this scenario really opens doors.

Top ranked games in the Autumn Lisp Game Jam 2024 - itch.io

[2024-11-11 Mon 12:16]

I made a a roguelite card game and it placed fourth in the game jam, there are some other neat entries in here. Play Prism Escape!

BYD EV Teardown In Japan Reveals The Secret To Its Affordability

[2024-11-11 Mon 12:22]

China sells a 20000$ highly capable EV and Japan asks how:

The Atto 3 teardown in Japan revealed one of the ways BYD manages to keep costs down and pass the savings down to the person who buys the car. It’s all about producing as many of the components as possible in-house and integrating them. The source article highlights the so-called “E-Axle” used by BYD, which is comprised of eight different components.

It includes not only the motor, inverter, transmission and controller but also the onboard AC charger, the DC-to-DC converter and the battery monitoring system (BMS). This approach, combined with the economies of scale (the larger the number of cars you build, the cheaper you can build them) goes a long way toward explaining how these highly competent Chinese EVs can be sold at such low prices.

...

Kenichi Ito, director of industrial machinery company Sanyo Trading and one of the seminar organizers, noted that "Chinese manufacturers attach great importance to low-cost production” and "their views on quality are different from those of Japanese manufacturers." This was a subtle way of saying they are not built to the same standard, but the market doesn’t seem to mind, and BYD sold 300,000 Atto 3s in the vehicle’s first year of production starting in February 2022—it's been doing quite well in some European markets too.

A California Wetland Program’s Flood of New Funding Lifts Hopes for Shorebirds | Audubon

[2024-11-11 Mon 12:24]

Brennan has been working with growers for three decades and today he manages thousands of acres for farms including Robbins Rice Company and Davis Ranches. While rice remains his top crop, birds and wetlands have become an important side hustle—not just for Brennan but for scores of Central Valley farmers enrolled in BirdReturns, a program that pays them to flood their land, creating temporary wetlands when and where birds need them most.

For millennia seasonal wetlands dotted California’s Central Valley, providing crucial habitat for millions of shorebirds to rest and refuel during migration. But as farms and towns have taken over the landscape, nearly all those shallow, ephemeral water bodies have disappeared, leaving avian migrants with scant options for pit stops. With shorebirds rapidly declining along the Pacific Flyway, conservationists and landowners have joined forces to help turn the tide. Launched in 2014, BirdReturns runs via reverse auctions: Farmers offer up acreage and name the price. If it’s right, they get cash in exchange for spreading a thin layer of water across their land, where birds can forage on aquatic invertebrates and other foods. Since its inception, the program—jointly run by Audubon California, The Nature Conservancy, and Point Blue Conservation Science—has paid more than 100 farmers a total of $2 million to flood 60,000 acres throughout the Sacramento and San Joaquin valleys. Buoyed by a recent $15 million grant from the state, the program is poised to greatly expand its reach.

Hiding Images in Plain Sight: The Physics Of Magic Windows

[2024-11-11 Mon 12:29]

Documenting the process of converting a greyscale image in to an acrylic block which when illuminated, presents the image in the cast shadow by (ab)using caustics:

Caustics are the bright patches of light we see when illuminating a transparent object. All the photons that don't pass directly through the object are what form the object's shadow. All those photons still have to go somewhere; they contribute to the caustic pattern.

The most interesting aspect of caustics is that they arise from even the tiniest of variations in surface flatness. Even the gentlest waves on the surface of a pool form powerful lenses that cast intense caustics on the floor below.

The reason my acrylic square can form an image is because I've distributed just the right amount of concavity and convexity into the surface so that the refracted light forms a caustic image.

BOOTHAMMER Aims to Deliver Simple "Intermittent Computing" for Energy-Harvesting Arduino Sketches

Clever tool inserts checkpoints and restoration operations, so when power is lost your program can pick up where it left off.

Hackster.io
Rebuild of The Complete Computer - The Complete Computer

NEW by @rrix: Two Updates: Org+Nix dev streams, and my new DNS resolverI've started to stream on Thursdays where I'll explore salt dunes and arcologies

The last few weeks I have started to work in earnest on Rebuild of The Complete Computer , my effort to provide a distribution of my org-mode site publishing environment in a documented, configurable Concept Operating System . My "complete computing environment" will be delivered in three parts:

  • a set of online documents linked above that are explaining how I manage a small network of private services and a knowledge management environment using my custom web publishing platform, The Arcology Project .

  • a set of videos where I work through the documents, eventually edited down in to a set of video lectures where you are guided from complete fresh fedora VM to installing Nix and a bare-bones org-roam emacs, bootstrapping a NixOS systems management environment, and then use Org files to dynamically add new features to those NixOS systems.

  • a handful of repositories which i'll finally have to treat like "an open source project" instead of Personal Software:

    • The arcology codebase which you'll have a copy of on disk to configure and compile yourself

    • the core configuration documents that are currently indexed on the CCE page, a subset which will be required to run the editing environment, and a number of other bundles of them like "ryan's bad UX opinions", "ryan's bad org-mode opinions", "ryan's bad window manager", etc...

I hope that by reading and following along with the documents while utilizing the video resources, one can tangle source code out of the documents, write and download more and an indexing process will extract metadata from the files that can be later queried to say "give me all the home-manager files that go on the laptops", for example, and produce systems that use that.

Two weeks ago I produced a three hour video where I played Caves of Qud and then spent two hours going over some of the conceptual overviews and design decisions while setting up Nix in a Fedora VM, ending with the Arcology running in a terminal and being used to kind-of-sort-of clobber together a home-manager configuration from a half-dozen org-mode files on disk. It was a good time! This is cataloged on the project page, 0x02: devstream 1 .

This week I came back to it after taking a break last week to contribute an entry to the autumn lisp game jam, and it was a bit more of a chaotic stream with only two hours to get up to speed on the project; there are many implicit dependencies in the design and implementation of the system because it's slowly accreted on top of itself for a decade now. That was 0x02: devstream 2

This week I'll work on cleaning up things to smoothly bootstrap and next week we'll come back with a better way to go from "well home-manager is installed" to "home-manager is managing Emacs and Arcology, and Arcology is managing home-manager" and then from there we build a NixOS machine network...

I have probably a three or six month "curriculum" to work through here while we polish the Rebuild documents. I will be streaming this work and talking about how to build communal publishing networks and document group chats and why anyone should care.

With the news from the US this week, it feels imperative to teach people how to build private networks, if only because the corporatist monopolist AI algorithm gang are going to run rough-shod on what's left of the open web the second Lina Khan and Jonathan Kanter are fired if they haven't already begun today. We can host Fediverse nodes and contact lists and calendars for our friends for cheap and show each other how to use end-to-end chat and ad-blocking and encrypted DNS; we oughta.

I'll stream on twitch.com/rrix on Thursdays at 9am PT and upload VODs to a slow PeerTube server I signed up for. Come through if this sounds interesting to you.

I re-did my DNS infrastructure

Years ago I moved my DNS infrastructure to a pi-hole that was running on my Seattle-based edge host. It worked really nicely without thinking about it when I lived in Seattle, but I hesitated fixing it for the years since I moved a half a hundred milliseconds away. The latency finally got annoying enough lately so I finally got around to it this week.

On my devicies, I've been using Tailscale's "MagicDNS" because DNS is a thing that I think should just have magic rubbed on it, as it is i've already thought way more about DNS in my life than I'd like. If you enable MagicDNS and instruct it to use your pi-hole's address as the global nameserver, any device on your Tailnet will use the pihole for DNS. Neat.

Pi-hole isn't packaged in nixpkgs and I was loathe to configure Unbound etc and a UI myself so I put it off and fnord ed the latency for months. I finally got around to it this week by deploying Blocky on my LAN server which has the feature-set I need, and rather than shipping a UI it ships a minimal API and a Grafana dashboard:

It's a neat little nice little thing, I hope it'll work out. I've started documenting this at Simple DNS Infrastructure with Blocky of course.

With the querying back on my LAN and managed by my Nix systems instead of a web GUI on an unmanaged host, I can list my blocked domains and block lists in a human-legible format, I can have different DNS results to route all my server's traffic direct over the LAN to my homelab instead of round-tripping to the SSL terminator, I can have custom DNS entries for local IPs. All this is managed in that one document which you'll soon be able to download from my git instance; that's the Concept Operating System promise.

If you're a content pihole user but never use the web UI and need to move, consider taking this thing for a spin.

https://cce.whatthefuck.computer/updates#20241107T142834.711602
Rebuild of The Complete Computer - The Complete Computer