430 Followers
110 Following
62 Posts
Using Rust to try to make the Python stack more energy efficient. Co-founder of Open Climate Fix. Father. Previously at DeepMind & consulted for National Grid Electricity System Operator (ESO). Currently focused on speeding up IO 🙂
Twitterhttps://twitter.com/jack_kelly
Homepagehttps://jack-kelly.com
CountryUnited Kingdom
I'm going to focus my micro-blogging efforts on Bluesky (at least for now): https://bsky.app/profile/jack-kelly.com
Jack Kelly (@jack-kelly.com)

Rust • Python • Climate change • Weather & energy forecasting • Open data • Earth observation • Co-founder @openclimatefix.org • Machine learning • TR-909s at 160

Bluesky Social
On my way from London to EuroRust in Vienna by train...

New blog post: "Lazy loading: Making it easier to access vast datasets of weather & satellite data"

https://openclimatefix.org/post/lazy-loading-making-it-easier-to-access-vast-datasets-of-weather-satellite-data

Lazy loading: Making it easier to access vast datasets of weather & satellite data | Open Climate Fix Blog

AI has developed at a stunning pace over the last few years in fields such as natural language processing and image recognition. Given all this development, it's surprising that we haven't seen similarly rapid advancement in the application of AI to electricity forecasting given its critical role in people's lives and the transition to net zero carbon emissions. We'd love to live in a world where it's super-easy to work with huge NWP and satellite datasets. We are already dedicating significant time and effort towards this, and would love to hear from people who support this vision and who can help!

I, for one, am not willing to live in a world where we tiptoe around the feelings of hateful dumbfucks who would deny the humanity of others.

It's time to grow a backbone, people.

It's time to call a Nazi a Nazi.

And then call those Nazis wankers.

https://joanwestenberg.com/nazis-arent-protestors-theyre-wankers

Nazis Aren’t Protestors. They’re Wankers.

There's a dangerous game of verbal gymnastics being played out in the public sphere, and it's high time we called it what it is: a steaming pile of i...

The lesson here is to never, ever do anything that gives money to or otherwise interacts with an Elon Musk company because if then you try to stop he might literally sue you https://journa.host/@chrisgeidner/112916187069167821
Chris Geidner (@[email protected])

Attached: 2 images NEWS: Twitter/X wants to force people to advertise there, and they're going to court. Elon and Linda went forum-shopping, filing this case in Wichita Falls Division so that Judge Reed O'Connor was guaranteed to be assigned the case. Here's the complaint: https://storage.courtlistener.com/recap/gov.uscourts.txnd.393003/gov.uscourts.txnd.393003.1.0_1.pdf

Journa.host

Introducing dynamical.org!

⚡ fast, optimized, and usable weather and climate data
🏛 stewardship organization designed for long-term stability
🍦 all open-source and open-access

Ooh, exciting that 128TB SSDs are due to launch in a matter of months. And these Phison Pascari X200 SSDs are fast: 14 GB/s & 3 million IOPs (random 4kB reads).

In general, I'm excited that we seem to be entering a world where most storage will be SSDs. It feels like it's only a matter of time before even cloud object storage offers high performance random reads (which is good for training ML models from multi-dimensional datasets like weather predictions and satellite)

https://www.techradar.com/pro/exclusive-first-128tb-ssds-will-launch-in-the-coming-months-and-yes-this-little-known-rival-will-challenge-samsung-solidigm-kioxia-for-the-storage-capacity-crown

Exclusive: first 128TB SSDs will launch in the coming months — and yes, this little-known rival will challenge Samsung, Solidigm, Kioxia for the storage capacity crown

Phison's Pascari brand is super-sizing its X200 Series SSDs

TechRadar pro
I've just added two new sections to the blog :)
My family are _finally_ getting over a horrible bout of whooping cough. Here's a long blog post about our experience with whooping cough: https://jack-kelly.com/blog/2024-06-11-our-experience-of-whooping-cough
Our experience of whooping cough (spoiler: It was horrible!)

My family is finally getting over whooping cough. It has been a very bumpy ride. The aim of this blog post is to try to explain the ways in which our experience of whooping cough differed from the “textbook” description of whooping cough. And I’ll very cautiously offer some advice for folks who think they may have whooping cough. But I must emphasise that I’m not a medical doctor! Please take my advice with a massive pinch of salt, and make sure you see a real doctor! What is whooping cough Whooping cough (sometimes called the “100-day cough”) is caused by a bacterial infection in the lungs. The whooping cough bacterium is called Bordetella pertussis. It is highly contagious (until the patient has cleared the bacterial infection). In England and Wales, whooping cough is a “notifiable disease”. Luckily, Bordetella pertussis can be killed with macrolide antibiotics. (Well, if we’re being pedantic: macrolides are bacterio_static_, so macrolides don’t actually kill bacteria, instead macrolides stop bacteria from reproducing.) Macrolide antibiotics include azithromycin, erythromycin and clarithromycin (note that the names of all macrolide antibiotics end in “mycin”). Note that penicillins don’t have much (if any) effect against whooping cough. This is a really important point. If you go to a UK GP complaining of a chest infection then the first antibiotics the GP will reach for will probably be amoxicillin (a penicillin) which won’t help with whooping cough. If you go back to the GP complaining that your symptoms haven’t improved (despite finishing your course of amoxicillin) then the GP may infer that you have a viral infection, or nasal drip (this happened to us). For more info, see the wikipedia page on whooping cough. Why does Bordetella pertussis make you cough? Whooping cough usually has three distinct phases: 1) the catarrhal phase (where the symptoms are very similar to having a “normal” cough), 2) the paroxysmal phase (where you get intense coughing fits), and 3) the convalescent phase (where you have cleared the bacteria from your body, but your cough lingers for weeks or months. Here’s an interesting 2022 paper discussing some evidence that sheds light on why the bacteria causes a cough. (And, interestingly, this paper also makes the point that one reason that we don’t know much about coughs is that, until very recently, we thought there were no good animal models for coughs. In particular, it was thought that mice don’t cough. But, recently, it’s been found that mice do cough, just very quietly!) The short answer is that evolution is very clever: The Bordetella perussis bacteria excretes toxins into your lungs which sensitise your lungs to coughing (so you start coughing more frequently), and also disrupt the signalling pathway that would usually stop a cough (so you get coughing fits). Supposedly, this helps the bacteria to spread to other hosts! And, supposedly, the effects of these toxins linger after the bacteria have gone. What was whooping cough like for our family? In short: This has been the worst “cold” I’ve ever experienced. This has been way worse for our family than covid. (We’ve had covid at least twice; and we’ve had all our covid jabs. And I thought covid was bad!). (Although, of course, I appreciate that covid can be far worse for other people!) My 12 yearold daughter was the first in our family to start coughing. I started coughing about two weeks after my daughter. For my daughter and I, our symptoms started as a fairly mild cough, with no other symptoms (the “catarrhal” phase). But, about a week after starting coughing, we both started having severe coughing fits several times per day (the “paroxysmal” phase), and generally feeling really ill. These fits are no joke. These coughing fits completely take over your body for about a minute. It can feel impossible to breath. The only thing that stops you panicking that you’ll never breath again is knowing that you’ve had many of these fits before, and they eventually subside. You wake several times a night, already in a coughing fit, fighting for breath. When fighting for breath between coughs, both my daughter and I made “whooping” sounds. There were some differences between my daughter’s symptoms and my symptoms. My daughter’s coughing fits would make her vomit about once a day. So much so that she lost 3 kg of weight over the 8 week course of her cough. My daughter also had a temperature, which would come and go. At worst, her temperature got to about 38.4 ℃. I didn’t vomit, nor did I have a temperature. But I did pass out about 4 times during a coughing fit. And I pulled a muscle in my back from coughing so hard. And, towards the end of my whooping cough, my lungs made a worrying “crackling” sound on most breaths (as if there was sticky mucus throughout my lungs). After a few weeks of coughing, your ribs really ache when you cough. In general, whooping cough is a very “physical” illness. The coughing fits are very violent. For me, whooping cough was like my worst bout of covid (where I was exhausted for 3 weeks) PLUS really violent coughing fits PLUS the fact that whooping cough can last for months. Whooping cough isn’t like any other cough or cold I’ve ever had because whooping cough really hurts. You don’t just feel horribly ill. You also spend a lot of time in physical pain. My daughter and I also had periods of several weeks of just feeling really ill: very low energy, just wanting to stay in bed all day. Even a 15 minute walk felt like a huge challenge. My daughter and I also experienced a weird effect where, about half way through the course of the infection, we had a few days where it really felt like we were getting better (possibly due to antibiotics). Only for it suddenly to get even worse. In total, my daughter was ill for over 8 weeks. I was ill for about 4 weeks. My kids were both vaccinated against whooping cough when they were younger. But immunity wanes after several years. My 10 yearold son and my wife both also caught coughs (a few weeks after my cough started). My son had the least symptoms (which I’d guess is because he’s most recently had his whooping cough vaccine). My son just had a slightly annoying cough for about 10 days. No other symptoms. My wife had an annoying cough for a few weeks, with no other symptoms. Neither my son nor my wife got full-blown coughing fits. Neither my wife nor my son needed to see a doctor (so they didn’t get any meds). What did the doctor say? Which medications did we get? I have a huge amount of respect for doctors. They work under enormous stress. All the doctors we saw were lovely, and attentive, and we always got a GP appointment within a day or two of requesting one. My criticisms below are entirely aimed at the health system, not at any individual doctors! I won’t go into every detail of our interactions with the health service. When my daughter started getting serious coughing fits (including “whooping” during coughing fits), my wife and I suspected whooping cough. Whooping cough had been in the news. And, as far as we could tell, our daughter’s symptoms exactly matched the “textbook” symptoms for whooping cough. But, for at least the first 3 weeks of our daughter’s cough, no doctor would entertain the idea that she had whooping cough. Despite my wife and I asking politely but increasingly firmly about whooping cough. In the first 6 weeks of her cough, our daughter was given 4 different courses of antibiotics (3 penicillins and 1 macrolide. In order, she had: amoxicillin, clarithromycin, co-amoxiclav, and back to amoxicillin. In retrospect, she did get quite a bit better after the clarithromycin. But I guess the clarithromycin didn’t completely kill the bacteria.). To be fair to the doctors, the doctors said that they were seeing a lot of coughs at the time. One doctor said that every second patient was complaining of a cough. And, in the first week or two of whooping cough, whooping cough looks very similar to other coughs. When I started getting coughing fits, I saw a doctor 3 times, over several weeks. I wasn’t given a macrolide antibiotic. The first doctor gave me amoxiccilin. The second doctor said I had nasal drip and didn’t give me any antibiotics, but did give me a steroid nasal spray (which made things worse). The third doctor thought I probably had a viral infection (because the amoxicilin hadn’t had any effect on my symptoms), but still gave me co-amoxiclav (which didn’t fix the whooping cough but did upset my stomach a lot). (I did eventually obtain azithromycin, which seemed to work very well… more on that later…) In general I was disappointed that for the first 7 weeks of my daughter’s cough she wasn’t offered a single lab test. Neither was I. Despite us asking for lab tests. (Again, to be fair to the doctors, I assume this is because lab tests cost money, and lots of people have coughs, so it would cost a lot of money to test every person who complains of a cough. But, still, in 2024, I’m amazed that we’re not testing every serious cough. Although my expectations are way too high… When I first started researching tests for whooping cough, I actually expected that, in 2024, it would be routine to sequence the DNA of every organism found in your spit or blood! (Spoiler: that’s definitely not what happens!)) Eventually, after 8 weeks of coughing, and still having indications of an active infection (raised pulse rate, tired, temperature) my daughter was sent to A+E. The A+E doctor strongly suspected whooping cough. Although, very worryingly for us, it was also possible that our daughter had TB or sepsis! So we finally got a bunch of lab tests for whooping cough. Her blood tests showed a strong positive result for whooping cough antibodies, and her blood tests showed multiple markers of an active bacterial infection (and this is 8 weeks after she started coughing!). Her tests were negative for all the respiratory viruses they tested for, and negative for TB (phew!). The A+E doctor prescribed azithromycin. Which appeared to clear up the bulk of her symptoms within a few days. To be honest, we don’t know for sure that I had whooping cough (because I didn’t get any lab tests). But I had very similar symptoms to my daughter, and I almost certainly caught my cough from my daughter, and it appears that azithromycin finally cleared my infection. How our experience of whooping cough differs from the “textbook” The NHS web page on whooping cough says that “If you’ve had whooping cough for more than 3 weeks, you’re no longer contagious and do not need antibiotics.”. The experience of my daughter and I suggests that both of these statements are wrong. After 8 weeks of coughing, my daughter was getting worse, and still had an active bacterial infection. Then she was given azithromycin. And the bulk of her symptoms went away within a few days. Azithromycin helped her after 8 weeks. I had a horrible cough for 4 weeks. I still felt really ill after 4 weeks (which is evidence - albeit weak evidence - that I still had an active infection after 4 weeks). And then I started on azithromycin. And I felt better within a few days. Perhaps this is just coincidence. Perhaps we were already getting better (without the azithromycin). And, of course, this is all anecdotal, and not done under “lab conditions”. The “textbook” says that serum (antibody) testing isn’t appropriate for ages 17 or younger. But a whooping cough serum test came back strongly positive for my 12 yearold daughter. Advice for people who think they may have whooping cough Again, to repeat what I said at the beginning: I’m not a medical doctor! Please see a real doctor! As far as I can tell, one of the big challenges with whooping cough is that the only way to know for sure that you have whooping cough is to get a lab test. Whooping cough symptoms (especially in the first week or two) are too similar to other coughs to identify whooping cough from symptoms alone. Chest x-ray might show a non-specific lung infection but, as far as I know, x-ray can’t definitively tell you that you have whooping cough (unlike, for example, advanced TB, which can have very distinctive appearance on chest x-ray). If you suspect that you have whooping cough then ask politely but firmly for a lab test for whooping cough. See the NICE web page on whooping cough for details of the available tests. Note in particular that you don’t need to cough up phlegm to be tested for whooping cough (one doctor we saw said that we couldn’t be tested for whooping cough because neither I nor my daughter could cough up phlegm from our lungs on demand). Some whooping cough tests can be done on swabs or blood samples. Also note that the cheapest test - growing bacteria in a culture - is not very sensitive to whooping cough, and takes 3 to 7 days to get a result. Lee et al. 2018 say that cultures are 64% sensitive for whooping cough. Lauria & Zabbo 2022 say that cultures are only 20 - 40% sensitive for whooping cough! And that sensitivity is highly dependent on the quality of the sample. Which makes sense: for a culture test to be positive, you have to extract sufficient living bacteria from the patient, and those bacteria have to survive being transported. In contrast, PCR is 90% sensitive and doesn’t require living bacteria (Lee et al. 2018). If you think you have whooping cough, but your GP prescribes a penicillin antibiotic, then maybe gently ask whether it’s possible to swap to a macrolide antibiotic. (Even if the doctor suspects some other bacterial infection, macrolides are still effective against community acquired pneumonia (Kyprianou et al. 2023). My daughter and I had no side effects from azithromycin.) If you start on a macrolide antibiotic as early into whooping cough as possible (ideally within a few weeks of starting to cough), then it may be possible to avoid at least some of the pain, and you can limit the number of people you transmit whooping cough onto. If your GP refuses to test you, and if you’re lucky enough to be able to afford it, then you can pay for whooping cough tests (e.g. Blood London can perform a whooping cough PCR test (which tests for an active infection) and a whooping cough antibody test. But note that a “standard” (not specific to whooping cough) sputum culture test isn’t at all sensitive to whooping cough, because the whooping cough bacteria are very fussy about the medium that they grow on! If you’re comfortable reading slightly technical medical literature then definitely read the Pertussis page on the NIH website. It’s the best single article I’ve found on whooping cough. Random trivia (not at all related to the rest of this blog post. Honest.) Some UK pharmacists sell chlamydia treatment online. In the UK, azithromycin is used as a second line treatment for chlamydia. Please please please don’t abuse this! Antibiotic-resistance in bacteria is a very serious problem (go and read about the horrible effects of drug resistant TB). And you can cause yourself serious damage by taking too many antibiotics and/or the wrong antibiotics. And you should only take antibiotics prescribed by your doctor. The official figures for UK whooping cough infections are almost certainly an under-estimate In England and Wales, whooping cough is a “notifiable disease”. Which means that, in theory, the UK Health Security Agency should be informed of every whooping cough infection. In our household, it’s quite likely (although not certain) that all four of us had whooping cough. Yet, as far as the UK HSA are concerned, only one of us has had whooping cough, because only my daughter has had a positive lab test result. And, if my daughter had gotten better within 6 weeks then no one in our household would have appeared in UK HSA’s statistics. My concern is that there’s a feedback loop here: Whooping cough is perceived as being rare. So - being good Bayesians - doctors are reluctant diagnose whooping cough, and reluctant to order whooping cough tests. And, if the doctor does order a test, then they might order a culture, which is shocking insensitive to whooping cough (see above). So lots of whooping cough is very likely being misdiagnosed (e.g. as a viral cough, or nasal drip, or pneumonia). So the UK HSA is only being informed of a tiny fraction of the UK’s whooping cough cases. So the UK HSA statistics show that whooping cough is rare. So doctors continue to perceive whooping cough as rare, and are reluctant to diagnose whooping cough, and reluctant to order tests. And so the cycle goes on… Why are cases of whooping cough increasing? Anti-vaxxers carry a lot of the responsibility. See: https://www.bbc.co.uk/news/health-68990222 (It makes me feel physically sick that misinformation in the virtual world has caused real, material harm to my family, and to my friends).

Jack Kelly

I'm working on a Rust library crate for efficient & fast I/O & processing of large, multi-dimensional arrays.

I'm about to change the name of this project!

Up until now, the project has been called `light-speed-io`. But, for various reasons, it's clear that this name isn't quite right. (The reasons are listed here: https://github.com/JackKelly/light-speed-io/issues/110 )

I'm considering changing the name to `farr` (short for "Fast ARRay"). Please shout now if you know of any reasons why `farr` wouldn't work!

Maybe change name (away from LSIO) ¡ Issue #110 ¡ JackKelly/light-speed-io

Problems with the name "light speed IO": This project will ultimately be at least as much about compute as about IO! (This is the main problem!) "Light Speed IO" has always felt just a bit too cock...

GitHub