Steve Troughton-SmithBack at it
Operating well outside of my level of expertise here with Codex, but it's doing a pretty credible job at a lot of complex Metal rendering code I would never be able to wrap my head around
(Ignore the UV issues)
You can just do stuff, now. It's wild.
I'm not yet sure what the end goal is here, but I'm making a lot of progress regardless
I started with a blank project template 5 hours ago. Now I've got a little 3D scene graph editor with gizmos, wireframe and shaded view modes, texturing, drag and drop OBJ file importing, and tile-based raytraced rendering, that runs on Mac and iPad.
Thanks Codex!
OBJ drag and drop
Multi-select and grouping
You know what I really could use at WWDC?
Teach a generative model to build high-quality vector SF Symbols so we can make custom ones, like say a full set for 3D modeling suites or PencilKit drawing apps, on demand 👀
…asking for a friend…
Upgraded my raytraced renderer with bounce lighting and true reflections. Now it looks pretty legit!
'Could somebody with no programming experience recreate Photoshop with an LLM?'
I have absolutely zero Metal and near-zero 3D modeling experience. I know the basics of how to use a scene editor, and the names of rendering terms.
And I effectively vibecoded all of this in less than half a day with Codex 5.3 Medium, based on screenshots of a cool-looking app I've never used (Valence3D) and just a general sense of what a 3D app should do
Raytracing uses every last ounce of my poor M1 Mac mini's GPU. It has never seen such a workout
Since I am doing this on the GPU, not CPU, finding a way to render complex meshes without blowing the budget is a headache. There's a hard cap on how much work you can do before the Metal driver crashes
Added some targeted calls to the hardware raytracing APIs, and a toggle switch so I can flip between non-accelerated RT and accelerated RT. There are plenty of bottlenecks in my renderer to stop it taking /full/ advantage of acceleration, but holy heck it’s much faster regardless. This is on the M5 iPad Pro
Turns out the M1 supports enough of the hardware accelerated raytracing APIs that I get a massive speedup here, too?
Trying to parse Apple's Metal feature support tables is hurting my brain, so I'll just accept it and move on
Not me offloading raytracing workloads to my iPad because it's so much faster than my Mac…
I spent untold hours in Bryce3D as a kid making silly scenes just like this. It did this stuff with machines a thousand times less powerful than my Mac mini. Certainly puts things into perspective…
It's real nice having a complex app like this on iPad, that's for sure. This is by far the most complex and impressive thing I've vibecoded with Codex 5.3. I haven't seen or touched a single line of code, I didn't have a detailed plan to work from — this isn't at all like the structured ports I detailed previously
I thought my water should have some translucency and falloff, so I can put things underneath it
Recap: I vibecoded (code unseen, no plan) a 3D editor/renderer that has a scene graph, editing controls, primitives and gizmos, materials, procedural terrain and water, and hardware-accelerated Metal raytracing with soft shadows, clouds and bounce lighting, that runs on Mac and iPad.
Tool: Codex 5.3 Medium
Time: About a day's worth of work has gone into it
The thing about material shaders is I don't quite know the right questions to ask — the unknown unknowns. Am I taking into account the right refraction, internal reflection, attenuated shadows, etc. Why is this too bright, why is this too dark. You can tell from these examples where things are obviously wrong, and it takes quite a bit of iteration
Maybe I should have bought a faster Mac before trying to write a raytracer…
Trying not to fry my GPU with caustics, but Metal isn't happy
I was sitting through hour-long renders (!) on my iPad yesterday, so I did an optimization pass on the hardware acceleration and it's much, much improved for simpler scenes, even on an M1
While this raytracer may never become a finished app, there are certainly elements from it I intend to yoink for future projects — like the really neat toolbars that go around all the screen edges, they would fit into a complex pro app very nicely
I made my control groups collapsible, with a priority system. Honestly they're my favorite part of this prototype
TheEJJ 🇨🇦@stroughtonsmith post it on GitHub.
Jonathan Grynspan@stroughtonsmith This is just Bryce!
Kirk Reynolds@stroughtonsmith at the start I thought you were going for something like a desktop Tinkercad or Shapr 3D, I would use that 😊
Kevin Conner@stroughtonsmith You might have to get used to the idea of making stuff just for fun again :)
Cian Walsh@stroughtonsmith It could be a scene blocking app. Attaching a material tag to each color which can be passed along to an AI generator with adherence instructions.
Currently > Reference Photograph > FSpy > Blender > Workbench Render Image with separated colours > Gemini is a bit fiddly. But an AI friendly simple modeller could make this a lot easier.
Jacob Wolman@stroughtonsmith you have the makings of a fine screensaver.
Ravishanker@stroughtonsmith make it a polished Storyboarder. It was great tool but hasn’t been supported in a while https://wonderunit.com/storyboarder
Ryan Goodlett 
@stroughtonsmith put it in the about screen for all your apps of course! Hi @jamesthomson.
Enrico@stroughtonsmith release it!
@stroughtonsmith can one vibecode Cyberpunk 2077 with pathtracing? i wonder
MatchaMakes@stroughtonsmith Thank goodness this project has been years in the making or else i would be mad at the efficiency and rapid development.
Finn Voorhees@stroughtonsmith have you considered rendering at a smaller scale and using MetalFX to upscale? they even have a denoising upscaler now, seems like it was pretty much designed for exactly this
@finnvoorhees I'm already using the metalfx denoiser after every pass, but I want to render out high res images, not just upsampled ones
D. Griffin Jones@stroughtonsmith Say, could you make a benchmarking mode in it? I’d love to be able to speed test ML and graphics performance between my M2 Pro Mac mini against the MacBook Neo next week 👀
@dgriffinjones I don't think it's stable enough for me to be able to do that. I have to baby the renderer a bit to get good results. Also I don't know yet if my performance is gated on me rather than the hardware!
@stroughtonsmith Fair enough!
Jimmy Callin@stroughtonsmith Did you see Glaze from Raycast? https://www.glazeapp.com/
Bernd Goldschmidt@stroughtonsmith Add a feature to use all the macs in the network as a render farm 😁
@BerndGoldschmidt XSan support? ;p
Alpacheez@stroughtonsmith a few things to look into that have to due with light interacting with objects: Rayleigh Scattering, Mie scattering, Kubelka-Munk scattering and absorption model, Saunderson correction to said K-M model. I have no clue if these are ever used in a typical ray tracing algorithm, but they come up when discussing the color development of coatings. My understanding of typical ray tracing is that it follows Snell's Law but I don't know beyond that.
Justin Scholz@stroughtonsmith I guess everything is ultimately converging towards a Salvador Dali painting? I want to see liquid clocks now! 😂
Andy Wilkinson@stroughtonsmith I wrote a vastly less capable 3D editor/renderer for my final year project at university. Year being the operative word there as that’s roughly how long it took me.
Thomas Sturm@ankinson @stroughtonsmith And you learned something doing it.
@tsturm @stroughtonsmith Yes, of course. We had radically different goals though. Producing what Steve has in a day, even if it’s just a throw-away bit of fun, remains striking to me.
𝑀𝒶𝓀𝑒𝔸𝕧𝕠𝕪🦀
Arcticulate@stroughtonsmith With your long-time experience and know-how about Apple’s SDKs, do you think the code it generated was high quality or did you need to make a lot of manual adjustments?
@Arcticulate the code is effectively indistinguishable from my own. I have made zero adjustments
@stroughtonsmith Ah! Thanks, that’s genuinely interesting.
Sim@stroughtonsmith is this ever coming to TestFlight? It's making me miss Bryce so much
Sunsets next?
@simsaens this is still a ‘throwaway prototype’ 😅 I wasn’t planning on productizing it
@stroughtonsmith @simsaens sure, but give it another week. 😉
@TheEjj @stroughtonsmith in another week it will be Blender, I want Bryce
Rafi Saar@stroughtonsmith Looks like the excitement made you stay up all night …
@stroughtonsmith One area that 3D software is only touching the edges of is using primitives to block out the scene and creating adhered AI renders using reference images. Currently a multi step process or a kludgy Blender add-on but so useful.
Dave Klein@stroughtonsmith loved loved loved Bryce and Strata. was so fortunate to have access to Power Macs in summer school. @ljharb
Edward Munn@stroughtonsmith That sounds like a handy handoff feature for those with older Macs but newer iPad/iPhone.
@stroughtonsmith Watching the progress of this app over the past day has been incredible.
@edmn I'm just as much along for the ride as everybody else
Jonathan Heron@stroughtonsmith love the Amiga reference :)