Some working thoughts on Steve Yegge’s Welcome to Gas Town . I read the essay a few times, then dug into the implementation to better understand how the system is structured. What follows is a mix of appreciation, architectural curiosity, and open questions about where this kind of agent harness
Hey folks, I am switching instances to @dehora.
If you’re following, I gather you should be copied along and see no real change, but you might have to re-follow.
Where I’m following you, I'm using the export/import feature and I see that's already worked for most, and I'll refollow soon where it didn't.
See you there.
Watching The Peripheral and it's really, quite good [1], while wanting to say, Eli Goree, is a standout performer among standout performances.
----
[1] A strong adaptation of @GreatDismal. Aside from having that noir style dialog even in the sticks, it's visually very dense. From Scotland Yard rolling up in a sleek alpaca-like deerhunter-thingy, because of course the future Yard would appropriate irony, to the knife carves on Conner's porch.
Back to basics:
The Unified Logging Infrastructure for Data Analytics at Twitter, Lee, Lin, Liu, Lorek, Ryaboy, 2012
In recent years, there has been a substantial amount of work on large-scale data analytics using Hadoop-based platforms running on large clusters of commodity machines. A less-explored topic is how those data, dominated by application logs, are collected and structured to begin with. In this paper, we present Twitter's production logging infrastructure and its evolution from application-specific logging to a unified "client events" log format, where messages are captured in common, well-formatted, flexible Thrift messages. Since most analytics tasks consider the user session as the basic unit of analysis, we pre-materialize "session sequences", which are compact summaries that can answer a large class of common queries quickly. The development of this infrastructure has streamlined log collection and data analysis, thereby improving our ability to rapidly experiment and iterate on various aspects of the service.
