Testing out Riffusion with the prompts from my blog post about prompt-based music generation: https://thisisimportant.net/posts/prompt-based-music-generation/
Testing out Riffusion with the prompts from my blog post about prompt-based music generation: https://thisisimportant.net/posts/prompt-based-music-generation/
Then I tried the most detailed prompt I considered in my post:
Melancholy breakup song with acoustic guitars in the style of Taylor Swift’s folklore album with lyrics about a queer couple
https://soundcloud.com/denyinghipster/prompt-melancholy-breakup-song
My guess is that the taylor swift comparison really helped out the model here, but WOW does it fall apart in terms of rhythm and meter.
Testing out Riffusion with the prompts from my blog post about prompt-based music generation: https://thisisimportant.net/posts/prompt-based-music-generation/
The next prompt also uses an artist comparison.
"Short vocal track with flute that sounds like Godford"
the model was able to produce something vaguely high pitched and with a higher bpm, but no discernible flute and the backbeat hiccups.
https://soundcloud.com/denyinghipster/riffusion-prompt-short-vocal
Testing out Riffusion with the prompts from my blog post about prompt-based music generation: https://thisisimportant.net/posts/prompt-based-music-generation/
the most detailed and perhaps highbrow prompt is also pretty nonsensical in terms of what I might expect to hear:
"High instrumentality and energy with bright timbre 120 BPM in the key of C minor"
https://soundcloud.com/denyinghipster/riffusion-prompt-high
turns out this results in something that sounds like an organ just pressing and holding a C-minor chord. At least I'm pretty sure it's a C-minor chord.
Testing out Riffusion with the prompts from my blog post about prompt-based music generation: https://thisisimportant.net/posts/prompt-based-music-generation/
Finally, in honor of the Switched On Pop book I just started reading, I decided to revisit the folk song prompt with some helpful meter guidance:
"melancholy folk song with meter in 3"
https://soundcloud.com/denyinghipster/riffusion-prompt-melancholy
It sounds kind of like an incoherent circus noise that you'd get from playing a music box.
Testing out Riffusion with the prompts from my blog post about prompt-based music generation: https://thisisimportant.net/posts/prompt-based-music-generation/
Seems like as predicted, we have a long way to go before we can produce coherent music from prompts. Rules-based systems will probably continue to... rule... for quite some time.
Try out Riffusion for yourself on HuggingFace:
https://huggingface.co/spaces/fffiloni/spectrogram-to-music
And check out my thoughts on my blog:
https://thisisimportant.net/posts/prompt-based-music-generation/
Sarah Moir
