The new year got me started on a small creative-coding project, which is a multitrack A/V compositor. It's not meant to be scalable or high performance, rather the idea is that the type of video I'd want to make with it is something that Love2D could easily do with a fast turnaround - karaoke text, looped animations, some procedural stuff - so why not make an editor out of that, made for that, instead of trying to cobble together other tools.
My inspiration for this is "Cartoon Cat Karaoke" - so simple, so effective, I think of it a few times a year
https://www.lexaloffle.com/bbs/?pid=31029
In three hours I bashed out 320 lines of code which do all of this:
Play a WAV music track, displaying what frame it corresponds to
Move a playhead through the track and loop or not loop it
Mark events on each frame on one of eight tracks, one event per frame per track, each event is "just a string"
Save and load the marks to a text file("lousy CSV" since it uses the first two commas semantically and consumes the rest of the line as an event string)
Select mark data from a palette and auto-advance through the palette so that you can time all the lyrics in order.
Cache and compute the spans between each mark so that you can do bouncing-ball text, etc.
The secret sauce of the output, of course, is that the mark data and spans are just going to go straight into hardcoded rendering that triggers effects and animations. Getting rid of formal structure and letting it be very messy and global allows everything as a possibility - wacky state-variable physics stuff, triggering samples over the main track, custom parse strings for one-off effects, and so forth. I dub this approach "A/V jamming" since it's meant to proceed roughly at the pace of a game jam - still edited and curated, but also performed, improvised and bespoken in many respects.
#gamedev #musicvideo #videosynthesis