@mucio

9 Followers
32 Following
82 Posts
I write in Italian or English. I do data stuff. I read comic books. I like to play games. I like Lego. I don't sleep enough.
githubgithub.com/francescomucio
@Maverynthia @nash because in Japanese "untitled goose" sounds like the words for "immortal suffering of the bull and tiger hell after a sudden death caused by a dull sword", which was a common insult in the Sengoku Era and still used by the fishermen sailing off Kanagawa.
Welp.

@webology @adamchainz @simon anything else to add to the "stop what you are doing and learn it today" list?

Asking for a friend

OTD in 1990 Soviet Union admitted that Soviet Union carried out Katyn massacre. These are Soviet documents with Stalin personally approving the murder of 22,000 Poles.

We blame(d) it on the Nazis. We lie(d).

Imagine us committing attrocities today & blaming it on Nazis...

@simon I had a t68i with the attached external camera. It was possible to send images via MMS. Then I bought a P800 and the first Xperia. I just liked tech gadgets (I had also a cheap Bluetooth headset) and I had a job to buy them. Similarly a few yeara before I bought a laptop. I didn't really taught about what will be the future, I just assumed it will happen (like for personal computer, internet, or mobile phones). Looking back it happened, faster than home computers
How much material/issues do you need before starting a substack newsletter?
Back when I was working on the board of the CBLDF I got very used to the idea of material for adults being challenged by people who thought comics were for children. The idea of books aimed at children being challenged just because they had positive black characters was unimaginable. The world's changed.

https://www.washingtonpost.com/comics/2023/04/08/jerry-craft-new-kid-school-trip-book-ban/
Jerry Craft drew a positive Black story. Then the calls for a ban began.

The celebrated author of “New Kid,” a graphic novel aimed at young readers, was caught off guard when his books started showing up on lists of inappropriate material.

The Washington Post

The 10k dags remembered me of when a colleague tried to convert data from json to delta, at real time, for thousand of tables.

Airflow couldn't make it. Looking at the problem, they didn't need a scheduler, they needed to react when a new json file was written in S3.

Their solution? S3 notification to SQS and a streaming job processing the new data. It worked like a charm.

Now, it is even easier if you are on Databricks (that was our case) using Live Tables (and before the autoloader)

Il reading the Shopify article about Airflow. I know I'm slow (kids, life).

While I learned a few things, I am still wrapping my head about having 10K dags on the same instance: just loading the web UI should take forever.

Few suggestions from my side:
- run airflow on kubernetes (use the helm chart)
- data processing is done outside Airflow
- if you generate thousand of dags with the same code, probably there is a better solution
- read dags from git, actually put everything on git

I need an app to schedule lunches/coffees/calls with friends and family