This analysis is part of our weekly DAOnload research pulse, which we send to paying Water & Music members as part of our Season 3 research on creative AI for music.
Yes, it IS about the music
As a part of our recently released research report on the state of music AI, we surveyed over 150 music-industry professionals on their familiarity, sentiment, desires, and fears regarding how AI is currently impacting the music industry, and how it might evolve in the future. Producers (84 respondents) and recording artists (58) were represented heavily in the dataset, alongside many other professionals including developers, sound engineers, session musicians, A&Rs, founders, and artist team members.
The #1 area in which respondents were excited about developments in AI was in promotional tools — with 30% of respondents mentioning that AI tools for visual asset creation, social media, website design, copywriting, and marketing would be most useful for their careers.
This checks out with what many of us know about artists’ love-hate relationship with social media. There has never been a tool so powerful and accessible as social media to take an artist from anonymity to relevance. At the same time, the price to play this game is having to constantly feed the content machine.
Time — and creative energy — is finite. More time creating content and managing social media means less time making music. So when it comes to areas where artists want the most help from AI, it really is about freeing up time to make more music.
Dive deeper into our music AI survey results.
The next AI tooling frontier: Video generation
Image generators are solidly here (lawsuits pending). Namely, it is fairly easy to find one’s way to an easy-to-use image generation interface, either through more developer-facing tools like Hugging Face or through more wide-ranging creative platforms like Adobe, Canva, and Runway.
Aside from audio, one of the hottest next frontiers is video — a high-engagement media format that requires a whole palette of skills to master, many of which do not overlap with music production or performance. Squint and you might be able to see some editing overlap, but by and large being a great music producer, writer, or singer does not mean creating compelling visual content is within the same technical wheelhouse.
I have been looking into video generation ever since image generation broke onto the scene. In many ways, I am a great barometer for video generation tools: Highly interested and in the trenches of making music and developing artists, but with little to no technical or visual art skills.
My lack of technical knowledge led me to many dead ends in terms of finding a video AI interface that was easy to use. This highlights one of our major takeaways from the business model deep-dive that we did for Season 3: UX is both one of the biggest challenges and greatest opportunities for founders and builders working on AI tooling for the creative industries.
Tweets like the below arguably miss the point:
Video AI breakdown: Kaiber
This video generator is founded by Water & Music member @stokebuilder. I found the interface to be quite friendly. Users upload a visual reference and an audio excerpt (I suggest starting shorter, since files longer than a minute go into a separate cue which can take several hours to run….as I found out). The pro account is $10/month and provides 1,000 credits, which equate to approximately 100 audio-reactive videos/month and can roll over.
… and voilà! The final video generation from Kaiber:
Video AI breakdown: Pollinations
Pollinations was featured in our Season 3 report, and even dropped in for a private workshop for our community. They focus on chaining together AI models for functionality that isn’t readily possible through one model alone, and creating friendlier interdisciplinary AI interfaces for users. Founder @thomash can be found in our server at large as well in the Pollinations workshop channel in our Season 3 hub.
Their interface for making AI music videos was also a two-prompt process — first writing out multiple scenes the user would like to interpolate between, then writing in the “style” of some desired aesthetic references, then finally uploading the audio file.
The first creation was free. After that, I had to sign up to GitHub and connect that to Replicate, where the model is hosted and costs approximately $1.38 per run.
I want to make a note that I got the style prompt above from a website called Lexica.art, which provides prompt guides for AI art tools. I chose a photo of which I liked the aesthetic, and Lexica provided text which would help direct a model to reproduce that aesthetic. I copied it over to Pollinations and ran the model, later realizing it had the name of a living comic artist, Geof Darrow. I ran the same prompt replacing Geof Darrow with “comic, hand drawn,” and it was not nearly as good.
I want highlight how mindless and easy it was to fall into a use for this AI tool that I might otherwise be ethically opposed to. The urge to use video of this quality out in social media or for real commercial purposes is very high. Opt-in models, attribution, and compensation have been covered in previous DAOnload issues and we will continue to cover it in the future.
Below is the initial video I generated on Pollinations (with Geof Darrow’s name in the prompt):
Below is the final video I generated on Pollinations (without the artist name):
Even more resources
Didn’t have time to drop into our Discord server this week? No worries. Stay up to date right here in your inbox with the best creative AI links and resources that our community members are sharing each week.
Shout-out to @cheriehu, @aflores, @moises.tech, @yung spielburg, and @Gareth_Simpson for curating this week’s featured links:
Music-industry case studies
- Spotify debuts a new AI DJ, right in your pocket
- Audiobook narrators fear Apple used their voices to train AI
- Roblox’s vision for on-platform generative AI
- Endel becomes the first generative AI company to produce a playlist with Amazon Music
AI tools, models, and datasets
- Aimi Live (24/7 AI-generated music livestreams on YouTube)
- A quick and sobering guide to cloning yourself
- Spawning’s tool for opting out of training for Stable Diffusion v3 (deadline March 3)
Other resources
- Neutone’s Neural Audio Plug-in competition
- UK Government study on how algorithmically driven recommendation systems impact music consumption and production
- Actors say they’re being asked to sign away their voice to AI
To follow more real-time updates on the music AI market, learn more about our membership here.
