It's been 4.5 years since I last updated the https://thi.ng/hiccup-carbon-icons collection and synced it with the upstream repo, i.e. IBM's Carbon design system. Spent a few hours today updating the icons to the current version, filtering out a hundred unnecessary ones (e.g. obsolete brand logos, IBM product/service related icons etc.) and updated the converter & code generator[1] to produce more concise outputs, then manually cleaned up the structure for dozens of them (in addition to optimizing/minimizing the SVG sources via the `svgo` CLI).

The new set has exactly 2222 icons in https://thi.ng/hiccup format (SVG expressed as nested JS arrays). These icons can be used in any context where https://thi.ng/hiccup format is supported, i.e. both for static HTML/SVG generation and/or interactive scenarios.

A contact sheet of the full collection (the attached image only shows a tiny selection of this):
https://demo.thi.ng/umbrella/hiccup-carbon-icons/

For tree-shaking purposes each icon is defined in its own source file, e.g. the Mastodon logo can be then imported like so:

`import { MASTODON } from "@thi.ng/hiccup-carbon-icons"`

Example icon definition:
https://codeberg.org/thi.ng/umbrella/src/branch/develop/packages/hiccup-carbon-icons/src/logo-mastodon.ts

The new version is still unreleased, but the readme already contains up-to-date information and small usage examples (incl. links to live example projects to see usage in situ).

[1] Converter/codegen tool: https://codeberg.org/thi.ng/umbrella/src/branch/develop/packages/hiccup-carbon-icons/tools/convert-icons.ts

#ThingUmbrella #UI #Design #SVG #Icons #OpenSource

Added, updated & simplified the growing collection of darkroom-related calculators and super happy how elegant and concise the code has turned out, making it super easy to add more of them in the future.

I think it's also another great, if minimal, example to illustrate how otherwise completely separate https://thi.ng/umbrella packages can seamlessly compose/combine to enable a reactive dataflow UI, all without the need for any virtual DOMs and/or completely over-the-top frameworks like React & co. It's also doing so via mostly JS-native data structures for declaring the UI (plain objects/arrays/iterables) and various constructs directly managing the reactive value streams, thus providing a lot more finegrained control over UI updates/timing/throttling). Any value changes done by the user only trigger specific, pin-point calculations which then result in equally specific UI updates to show new results. Any user action only ever triggers the minimum amount of work needed to reflect the new state.

Calculators:
https://demo.thi.ng/umbrella/darkroom-calc/

Source code:
https://codeberg.org/thi.ng/umbrella/src/branch/develop/examples/darkroom-calc/src

The attached images show the source code of the entire main app (UI root) and one of the calculators...

Ps. Please let me know if you'd like to see more of these posts in the future. I'm tempted to launch season 2 of #HowToThing (see link below for 30 previous mini projects/tutorials) — but since this is very time consuming to produce & document these projects/examples, and because there has been _very little feedback_ to these previous projects/posts, I first need to gauge interest... Thank you! 🫶

https://codeberg.org/thi.ng/umbrella#howtothing

#ThingUmbrella #Darkroom #Calculator #Tool #Reactive #UI #WebDev #TypeScript #JavaScript #OpenSource

tl;dr Using https://thi.ng/column-store to accelerate tag intersection queries by a factor of 880x...

Working on the static website generator/export plugin for my personal knowledge tool has been one of the main projects this past month. A key part of this setup is tagging, not just simple flat keywords/categories, but actually treating tags as sets. The system doesn't just allow browsing content by a single tag, but also supports adding (or removing) tags to narrow or widen the current topic. E.g. The combination of "3d + geometry + typescript" would select only works which have all of these three tags...

In the local version of my tool there's no limit to the number of tags (and it also supports tag negation), but for the static site generation I have to limit the set size (due to combinatoric explosion) and pre-compute all possible permutations, then create HTML documents for each these individual combinations which actually produce results.

So far I'm having ~400 unique tags in use, meaning if I want to aim for a max set size of 3, there're theoretically ~64,000,000 possibilities to check[1]! For the roughly 3500 content items used for testing, a naive JS approach to filter the result array and only retain items matching the entire current permutation is so extremely slow, that I stopped the process after 3.5 minutes just for the first 250k (aka 0.4%) of the 64 million permutations, i.e. at that rate the full process would have taken ~15 hours, pretty slow for a SSG... :)

Naive approach 🫣:

```
permutation = ["3d", "geometry", "typescript"]
results.filter(item => permutation.every(tag => item.tags.includes(tag)))
```

But since I'm using https://thi.ng/column-store as my database, such queries can be optimized by a few magnitudes, since here these intersection queries are applied only to bitfields (explained in the pkg readme). This results in all 64+ million permutations being processed in just 62 seconds (1+ million per second). Quite the difference, i.e. ~880x faster than the above approach!

Also, of these 64 million initial possibilities, there're fewer unique ones (excluding duplicates and ignoring ordering), and currently only ~24,000 are actually producing a result. Still, that's 24,000 index pages to generate & host and it's, of course, far, far too much!

So I will have to also spend more effort curating and severely reducing the tag vocabulary, at least the subset used for the website. On the other hand, I think this system will really help with browsing this large body/archive of work much more meaningfully than the boring single-tag/category approach most websites are offering. And it will do so without any backend (other than file hosting)...

[1] Permutations = 400 + 400^2 + 400^3

#ThingUmbrella #Tagging #Intersection #Query #Bitfield #WebDev #JavaScript #TypeScript #Optimization

#ReleaseThursday A historic moment in the https://thi.ng/umbrella microverse!

I just re-published all 214 projects, the first release referencing their new home at @Codeberg with updated links in all packages, examples, readmes and other documentation... 🎉

(Also added a prominent note [hopefully prominent enough!] at the top of the main readme to inform about this project migration...)

Happy Coding! :) The detangling from US big tech continues...

#ThingUmbrella #OpenSource #Codeberg #TypeScript #JavaScript

PSA: Part of the ongoing #ThingUmbrella maintenance and migration to #Codeberg, I've written a script to batch delete 20600+ tags from the Git repo[1], basically any release tags older than 2023-01-01. There're still ~4600 tags remaining, covering all releases since that date. This includes all releases mentioned in various changelogs, which also have the same cut-off date...

https://codeberg.org/thi.ng/umbrella

[1] To be a good citizen, I deleted chunks of 200 tags and waited ~10 seconds between each chunk...

Started creating browser-based calculators for quickly computing a bunch of recurring things in darkroom processing, so far:

Scale amount by area:
Given reference dimensions and amount, compute scaled amount needed for new target dimensions.
Example: Compute drops of emulsion needed for a new print size

Two-part solution ratio:
For a total volume of a two-part solution with a A:B target mix ratio, compute how many units are needed of both A and B.
Example: For N drops in total, compute how many drops are needed for A & B using a 1.2:1 target ratio

Solution addition:
Compute the amount of solution A (with given concentration) to add to another volume to achieve a certain target concentration (for solution A)
Example: How many extra drops of a 25% contrast agent solution are needed to achieve a 5% solution with a certain base emulsion volume

More calculators forthcoming... Feel free to ping me with any other useful formulas you'd like to have added...

Bookmark this on your phone:
https://demo.thi.ng/umbrella/darkroom-calc/

(The UI is responsive to the system's dark mode setting)

Source code:
https://codeberg.org/thi.ng/umbrella/src/branch/develop/examples/darkroom-calc/

Btw. This is https://thi.ng/umbrella example project #186 and relies mostly on these packages:

- https://thi.ng/rdom & https://thi.ng/rstream for reactive UI & computation
- https://thi.ng/rdom-forms for reactive form element abstraction/creation

#ThingUmbrella #AltProcess #Darkroom #DarkroomPrint #AnalogPhotography #Calculator #Reactive #UI #TypeScript

One from the archives for #TextmodeTuesday. The post might be 3 years old, but I'm still using these snippets almost daily to visualize and debug data whilst I'm working in the Node REPL...

https://mastodon.thi.ng/@toxi/110942967462856117

#ThingUmbrella #DataViz #REPL #Terminal

PSA: Migrating the https://thi.ng/umbrella monorepo to Codeberg, including:

- updating thousands of links in ~970 files (readme's, media, API docs/snippets in source files, examples, wiki etc.)
- updated 215 package short links to point to new locations on Codeberg
- re-configured & re-uploaded hundreds of megabytes of package API docs to https://docs.thi.ng/ (with new backlinks to Codeberg)

Re: short links, for example http://thi.ng/wasm-api is an alias for the more unwieldly package homepage URL in the larger umbrella repo on Codeberg: https://codeberg.org/thi.ng/umbrella/src/branch/develop/packages/wasm-api

Along with these changes, Codeberg is also the main/default remote for this project now. I will keep the Github repo (https://github.com/thi-ng/umbrella) around for a bit longer, but will add a note in some places to say that this is only a passive mirror from now on...

This exercise has taken up most of my Sunday today, for a body of work which spans close to 10 years of my life... I had migrated this large repo already in 2024, but finally got around to "make the switch". Other thi.ng projects will be migrated over the coming weeks/months...

As always, a big thank you to all the people who've been supporting this work and its maintenance. Self-promotion is absolutely not my forte and I always have prioritized putting my energy into these projects instead. But if you in any way have benefited from these varied projects and/or want to support their ongoing development, I'd highly appreciate any donations/sponsoring via:

https://codeberg.org/thi.ng/umbrella/src/branch/develop/CONTRIBUTING.md#donations

#ThingUmbrella #Monorepo #OpenSource #Migration #Codeberg #GitHub

As is somewhat customary, spring time is maintenance time and so this past week was again "infrastructure week" in #ThingUmbrella land, updating various tool scripts, switching to Rclone for uploading/syncing API docs, updating/pinning Prettier version & config, reformatting thousands of source files, re-uploading hundreds of MB of documentation for all current 214 projects, updating package metadata etc.

Also lots of recent updates to several packages, including some larger ones like:

https://thi.ng/column-store

Re-indexing of dictionary-based columns now re-creates the dictionary using the histogram (usage frequency) of stored values. This can result in major file size savings (I do still care about this aspect.. 😅). For example, in my PKM system (also my main dev focus at the moment), this change alone has reduced the database file size by almost 30% (containing tens of thousands of tagged items and where tags are stored in such a dictionary-based column).

The query engine now also supports result paging and sorting by an arbitrary number of columns/criteria & custom comparators (each with ascending/descending order options).

#ThingUmbrella #OpenSource #Maintenance #TypeScript #JavaScript

Finally have been trying out https://github.com/boyter/scc for counting (unique) lines of code, comments, DRYness and complexity and getting a better and more aggregated overview of the scale of my https://thi.ng/umbrella monorepo. SCC is really much better and faster than what I've been using previously (cloc) and also provides more meaningful results, including COCOMO project stats.

So selected current data points for this mega repo (incl. 214 projects and 185 example projects):

Estimated Cost to Develop (organic) $10,032,312
Estimated Schedule Effort (organic) 33.03 months
Estimated People Required (organic) 26.98

(This is equivalent of 74 years for one person and an assumed salary of $56k/year... Considering the repo in this form is only 8+ years old, that literally makes me a 9.25x developer 🙃)

#TypeScript
Files: 5,439
Lines of code: 323,816 (unique 161,337)
Comment lines: 85,303

#Zig
Files: 46
Lines of code: 4,828
Comment lines: 2,332

Markdown (aka #Documentation)
Files: 423
Lines: 66,370 (unique 31,330)

#ThingUmbrella #OpenSource #CodeAnalysis