Wrote a custom kubernetes operator in C#. I can now quickly deploy a prefect environment on my cluster: https://github.com/wmeints/cartographer
| Blog | https://fizzylogic.nl/ |
| Github | https://github.com/wmeints/ |
Willem Meints
- 20 Followers
- 25 Following
- 24 Posts
Chief AI architect for Aigency. Microsoft AI MVP. I love good BBQ and building cool stuff.
PyTorchIf you installed PyTorch-nightly on Linux between Dec. 25 and Dec. 30, uninstall it and torchtriton immediately and use the latest nightly binaries.
Read the security advisory here: https://pytorch.org/blog/compromised-nightly-dependency/
It's been a while since I last wrote a single line of go. I'm writing an operator to automatically maintain our mlops resources on kubernetes that we currently manually deploy to a vm. I tried building one in C#, but the SDK is really meh. The Go SDK is way better in that area. I guess because people wrote kubernetes in Go.
Hopefully my muscle memory builds fast, because right now, it's garbage.
Today's fun data science job: Create a workflow to generate synthetic data for a computer vision model.
Step 1: Use a photoshop script to extract backgrounds from existing sample images.
Step 2: Use prefect and opencv to build a workflow to generate new samples
I love photoshop!
Been having a ton of fun with Prefect and Ray. The perfect combo for scalable machine learning workflows.
Wrote a post about it on my website: https://fizzylogic.nl/2022/12/23/how-to-build-scalable-workflows-with-prefect-and-ray
How to configure memory limits in WSL2
One of the downsides to building machine learning models is that you need quite a bit of power in your machine. I've just upgraded my home desktop that I use for side projects and experiments to 64GB of memory because it was clearly no longer up to the task. However, when I booted up my WSL linux distro, it couldn't get more than 32GB of memory. Time to figure out how to fix that! In this quick post I'm showing you how you can limit or increase the memory available to your WSL2 distributions. Let's go!
I experimented with Prefect today. It's quite a big step up from airflow and works nicely with Ray and Dask. It looks like Ray combined with things like Dask and Prefect are going to be very powerful.
Friendly reminder, don't add datasets and models to the git repo for your machine-learning project. It gets out of hand real quick.
Manu Cornet
Got the keychron Q3 keyboard. It supports QMK, so I'm going to mod the crap out of this keyboard. Hope I can use all four layers instead of just the two. But who knows. I guess I'll find out some time in the next weeks.