hesselschutBrain Activity
March 2026 - A video (https://doi.org/10.1101/649822) of a brain MRI is slit-scanned with color dispersion and fed to the YOLO object recognition model. Activations in YOLO's 7th backbone layer are used to modulate alpha transparency and luminance in a 3D render of the slit-scanned video as a volume (of width x height x time). The accompanying music is made by injecting embeddings from the CLIP image description model running on the resulting video into the conditioning pathway of Facebook's MusicGen generative music model.
#slitscan #compuationalart #brain #generativeart #deeplearning #abstract #videoart #musicgen #yolo #clip #MRI
March 2026 - A video (https://doi.org/10.1101/649822) of a brain MRI is slit-scanned with color dispersion and fed to the YOLO object recognition model. Activations in YOLO's 7th backbone layer are used to modulate alpha transparency and luminance in a 3D render of the slit-scanned video as a volume (of width x height x time). The accompanying music is made by injecting embeddings from the CLIP image description model running on the resulting video into the conditioning pathway of Facebook's MusicGen generative music model.
#slitscan #compuationalart #brain #generativeart #deeplearning #abstract #videoart #musicgen #yolo #clip #MRI