New blog post: making a transformer by hand, without training! Want to understand transformers and attention better? This post goes through assigning each weight for a GPT-2-like transformer to understand how they work.

Goes thru designing a simple tokenization scheme, embeddings, the qkv weights and attention head, and projecting that back to get 100% accuracy at predicting a simple sequence! It even has explainers for matrices and softmax if you're a little rusty :-)

https://vgel.me/posts/handmade-transformer

float z=-9.;vec2 F=FC.xy/r.xy-.5;for(int i=0;i++<30;){float X=F.x*cos(t)-z*sin(t),Z=F.x*sin(t)+z*cos(t),h=sqrt(X*X+F.y*F.y)-.2,d=sqrt(h*h+Z*Z)-.05;if(d<.01){vec3 n=normalize(vec3(X*h,F.y*h,Z*(h+.2)));o=vec4(.95,.5,mix(.8,.3,sign(n.z+.3)),1.)*vec4(.3+max(0.,n.y));break;}z+=d;}

// #sdf #rendering #glsl #donut

game inspiration brewing...

(from http://blog.presentandcorrect.com/27986-2)

New blog post! Signed distance functions in 46 lines of Python, where I demonstrate how simple SDFs can be, by iteratively building up to a program that renders this ASCII donut. https://vgel.me/posts/donut/

#python #sdf #graphics #blog

me and my wife made a new game in 3 hours! Blackout in Apopol: The power has been out for a generation in Apopol, can you turn it back on? A texty, post-apocalyptic map exploration game with a roguelite twist!

https://vgel.itch.io/blackout-in-apopol

#indiedev #gamedev #indiegame #gamejam