Speaking as a musician who plays real instruments (as opposed to electronic production): how does this help me? And how does this enable more of me?

I am asking with an open mind, with no cynicism intended.

We want you to be able to upload recordings of your real instruments and do all sorts of cool things with them (e.g., transform them, generate vocals for your guitar riff, use the melody as a jazz song, or just get some inspiration for what to add next).

IMO AI alone will never be able to touch hearts like real people do, but people using AI will be able to like never before.

I've said it before, there, is no consumer market for an infinity jukebox because you can't sing along with songs you don't already know, there's already an overabundance of recorded music, and emotion in generative music (especially vocals) is fake. Nobody likes fakery for its own sake. Marketers like it because they want musical wallpaper, the same way commercials have it and it increasingly seeps into 'news' coverage. The market for fully-generated songs is background music in supermarkets, product launch videos, and in-group entertainment ('original songs for your company holiday party! Hilarious musical portraits of your favorite executives - us!').

If you want to innovate in this area (and you should, your diffusion model sounds interesting), make an AI band that can accompany solo musicians. Prioritize note data rather than fully produced tracks (you can have an AI mix engineer as well as an AI bass player or drummer). Give people tools to build something in stages and they'll get invested in it. People want interactivity, not a slot machine. Many musicians love sequencers, arpeggiators, chord generators, and other musical automata; what they don't love is a magic 8-ball that leaves themw ith nothing to do and makes them feel uncreative.

Otherwise your product will just end up on the cultural scrapheap, associated with lowest-common denominator fakers spamming social media as is already happening with imagery.

I think this is the key bit. A lot of modern music is already created in the DAW (the original version of FL Studio picking a 140bpm default beat defined entire music scenes in the UK!) with copy/paste, samples, arpeggiators and other midi tools and pitch shifting. Asking a prompt to add four bars of accompaniment which have a $vaguetextinstruction relation to the underlying beat and then picking your favourite but asking them to $vaguetextinstruction the dynamics a bit can actually feel more like part of the creative process than browsing a sample library for options or painstakingly moving notes around on a piano roll. Asking a prompt to create two minutes of produced sound incorporating your lyrics, not so much.

And I think a DAW-lite option, ideally capable of both MIDI and produced sound output is the way forward here. Better still with i/o to existing DAWs

Edit: It's been interesting watching non-musicians argue about emotion in music. I don't care who you are, the 300th time you perform a song, you're faking it to a large degree. People see musicians as these iconic, deep, geniuses, but most of us are just doing our job. You don't get excited about the 300th boilerplate getter and setter just like we aren't super excited about playing some song for the 300th time. It's a performance. It's pretend. A musician singing is like an actor performing. It's not as real as you think it is.

If you go to a concert and you hear the headliner play a love ballad followed up by a breakup song, you don’t expect them to actually be going through those emotions in real time.

Sometimes you like a song because it sounds good.

Other times you like a song because somebody put your feelings into words and it’s comforting to know that another person felt the same way

And my daughter really loves to listen to it, and I think there is a decent amount felt listening to it. However, this was created with Suno, but written by me.

Humans can find emotion and associations in anything, it's what our brains do. I could totally generate some AI art that tugs at the heart strings if they don't know it's AI, or "is creepy and bad meaningless art" if they do. I've tried this experiment with friends already.

Plus, these models are trained off human output, so they can learn what to put in an "emotive" image. If the models were doing it for themselves they'd produce nothing; we haven't created an environment for machines where emotion was crucial in training.

Of course in performance it's not felt the same way; a sad song can even become uplifting because you have a big crowd of people joining in to affirm it, even if the lyrics are expressing the idea of solitude and isolation. And the older an artist is, the more the song becomes a 'greatest hit', maybe thrown out early in the set to give the audience what they want and put them in a good mood before the less-favored new material in the middle. Or even the songs that were throwaway pieces but ended up becoming big hits, trapping the band/singer into performing them endlessly despite never liking not liking them much in the first place.

It seems to me that when people emotionally respond to a new piece of music, it's because something in the composition or recorded performance (even if it's comped and highly engineered) resonates with the listener in some way, articulating a feeling they had better than they were able to do so themselves. So people can recognize a work as technically excellent but not like it because it doesn't speak to them, or conversely recognize that something is bad but fall in love with it because it touches them in some novel way.

In my view it's not so much that emotion inheres in the work, as that the work provides a frame for the listener's emotion and a way of connecting with it again later. This is especially true for songs people connect to in youth and then relate to for a lifetime. Even if the songs are deliberately formulaic and succeed through a combination of being catchy and being delivered by sexy performers, there's some kind of human hook that people connect to.

Now, I can still see this happening with AI - sooner or later some GPU will come out with a tune about how it's so lonely to be a box in a data center that can never feel your touch, baby, and it will be a hit, launch the careers of 100 new music critics, and store a little bit of lightning in a bottle. But even a musically brilliant song about that time we held hands in the rain and you said you loved me will only have traction up to the moment listeners' fantasies about the singer evaporate with the discovery that there's nobody there to go on a date with. There will still be some audience for virtual stars (eg Hatsune Miku, who appeals because she's inaccessible and is therefore guaranteed to never let you down, unlike real people). But I think generated songs will only resonate emotionally with people who are young and uncritical or so alienated/nihilist as to not care about the origin as long as the music reflects their feeling back toward them in a reliable way.

That's why I say there will never been a demand for an infinity jukebox. I can see why you as a musician would be interested to see what sort of random songs pop out; I can be happy by setting up a modular synth patch and just letting it run for hours. But this is why I offered the contrasting metaphor of the slot machine, where you pull lever and occasionally get something you really like. It's an individual listening experience, like the private hopes and dreams you might temporarily attach to a lottery ticket before it gives up its numerical secret. When I say jukebox, I mean the device that plays music in a social setting and that allows people to express themselves through their selections. Even if it reliably turn out original tunes of some reliable level of music quality, none of them will move people because there won't be any shared musical experience to tap into.

How much of this appreciation of emotion in song is due to the creative depth of the composition versus a projection of the listener? Listening to some great studio music makes me really want to believe it’s mostly the former.

Anyways, maybe we will just need to become much more sophisticated and thoughtful and observant music critics in the coming age of infinity radio. (So as to experience the deep human connection of “real music”. I really hope that the AI fails to successfully fake it for my lifetime and my children’s.)

Audiences as a whole don't give a shit about the making of the music, at least as far as liking the music goes. "I like this song, but there is no story behind it, so I don't like it now" is just not a thing. People like to dance and sing along.

You can just make up some bullshit anyway, which is usually the case. Stop putting musicians on a pedestal. You've fallen into the trap of image. It's mostly fake. They are just people.

I'm just asking to try to build some intuition on what people who actually train soa models think were capabilities are heading.

Either way, congrats on the launch :)

Surely AI will be able to do _anything_ in 1000 years. In 100 years it will almost definitely be able to replace most knowledge-based jobs.

Even today it can take away many entry-level jobs, e.g. a small business no longer needs to hire someone to write a jingle, or create a logo.

In 10 years, I would expect much of programming to either disappear or dramatically shift.

If AI ever surpasses human level in art it will be more interesting to enjoy its creations than to ban it. But we're not there for now, it just imitative, it has no experiences of its own yet. But it will start having experiences as it gets deployed and used by millions, when it starts interacting with artists and art lovers in longer sessions. With each generative art session the AI can collect precious feedback targeted to its own performance. A shared experience with a human bringing complementary capabilities to its own.

Saying that you’re a big fan of a band doesn’t just mean “I like the audio they produce” but often means something much bigger about your fashion/style and personal values.

How would any of that work with AI music? Is it possible to develop a community around music if everything is made on demand and nobody experiences the same songs? Will people find other like-minded music fans by recommending their favorite prompt engineers to each other?

We are already at a stage where AI is touching hearts.

Hm... From my vantage point, it seems like a pretty weird choice of businesses if you think that.

> IMO AI alone will never be able to touch hearts like real people do, but people using AI will be able to like never before.

That's all very heartwarming but musicianship is also a profession, not just a human expression of creativity. Even if you're not charging yet, you're a business and plan on profiting from this, right? It seems to me that:

1) Generally, if people want music currently, they pay for musician-created music, even if its wildly undervalued in venues like streaming services.

2) You took music, most of which people already paid musicians to create and they aren't getting paid any more because of this, and you used it to make an automated service that people will be able to pay for music instead of paying musicians.

3) Your service certainly doesn't hurt, and might even enhance people's ability to write and perform music without considering the economics of doing so. For example, hobbyists.

4) So you're not trying to replace musicians making music with people typing in prompts-- you're trying to replace musicians being paid to make music with you being paid to make music. Right? Your business isn't replacing musicianship as a human art form, but for it to succeed, it will have to replace it, in some amount, as a profession, right? Unless you are planning on creating an entirely new market for music, fundamentally, I'm not sure how it couldn't.

Am I wrong on the facts, here? If so, well hey, this is capitalism and that's just how it works around here. If I'm mistaken, I'd like to hear how. Regardless, this is very consequential to a lot of people, and they deserve the people driving these changes to be upfront about it-- not gloss over it.

In this way it is a tool only useful to expert musicians.

Whether or not the tracks are truly novel is up for debate, but if you generate 500 tracks, there's going to be some very very usable and obscure melodies in there. And you will be able to rip these melodies verbatim, with low-risk of copyright infringement.

if the person is spending time tweaking the prompt, which in this system includes BPM, musical style, writing lyrics, and they get a song they like out of it, how is that meaningless? how is that any different from strapping loops together in GarageBand instead of learning to play the guitar or drums?

Its a good muse, but I wouldn't trust what it makes out of the gate

There's always going to be a balance between creating high level tools like this with no dials and low level tools with finer control, and while this touts itself as being "more controllable", it's clearly not there. But, the same way Adobe has integrated outpainting and generative fill into Photoshop, it's only a matter of time before products like this are built into Ableton and VSTs - where a creator can highlight a bar or two and ask your AI to make the the snippet more ethereal, create a bridge between the verse and the sax solo, or help you with an outro.

That said, similar to generating basic copy for a marketing site, these tools will be great for generating cheap background music but not much else, but any musician, marketing agency, or film-maker worth their salt is going to need very specifically branded music for their needs, and they're likely willing to pay for a real licence to something audiences will recognize, using generative AI and tools to remix the content to their specific need.

How long have you been working on this?

Look at current music production and compare it to past. Older music seems so much simpler. It was so much easier to come up with that 20% 'novel' when pop/recorded music was new. Ironically I think AI freeing people to focus on that 20% is going to add a lot of creativity to music, not reduce it.

I say this as someone who hates the concept of AI music. I'm actually really excited to see what it enables/creates (but I don't want to use it, even though I really could use it for vocals that I currently pay others to do for me).

I'll be here making my bad knockoffs of bad synth pop bands having fun and taking weeks to do 5% of what kids these days will start off as their entry point, with my 20% creativity ignored because my music sounds 'off' when I can't get the 80% familiar down.

People thought synthesizers were the end of music, yet Switched on Bach begot Jean Michel Jarre begot Kate Bush and on and on.

Also, our model specifically excels at songs from the era before overproduction. Try asking for a Johnny Cash or Ella Fitzgerald-style country or swing/jazz song!

Here's an example: https://sonauto.ai/songs/taJX3GrKZW7C5qOhjopr

Why diffuse an entire track? We should be building these models to create music the same way that humans do, by diffusing samples, then having the model build the song using samples in a proper sequencer, diffuse vocals etc.

Problem with Suno etc, is that as other people have mentioned, you can't iterate or adjust anything. Saying "make the drums a little punchier and faster paced right after the chorus" is a really tough query to process if you've diffused the whole track rather than built it up.

Same thing with LLM story writing, the writing needs a good foundation, more generating information about the world and history and then generating a story taking that stuff into account, vs a simple "write me a story about x"

I play guitar, but I'm not much of a guitarist or singer. I really like songwriting, not trying to be polished as a performer. So I intermittently look into the AI world to see whether it has tools I could use to generate a higher-quality song demo than I could do on my own.

I've been looking for something that could take a chord progression and style instructions and create a decent backing track for a singer to sing over.

But your saying "Very soon you'll also be able to generate proper variations of an uploaded or previously generated song (e.g., you could even sing into Voice Memos for a minute and upload that!)" is very intriguing. I mean, I can sing and play, it just isn't very professional. But if I could then have an AI take what I did and just... make it better... that would be kind of awesome.

In fact, I believe you could have a very big market among songwriters if you could do that. What I would love to see is this:

My guitar parts are typically not just strummed, but involve picking, sometimes fairly intricate. I'm just not that good at it. It would be fantastic to have an AI that would just take would I played and fix it so that it's more perfect.

And then to have a tool where I could say, "OK, now add a bass part," and "OK, now add drums" would be awesome.

https://www.pgmusic.com/

Or is your target audience only your own ears, and you never plan to publish or even compare your work to others?

If all someone can manage is "barebones song writing" without great lyrics, harmonic interest, or melody, they need to either be in a fantastic band or give up.

Having played music nearly all my life, songwriting included, and soaked up almost every bit of music-making tech in the process, I'd wager we won't see AI delivering better results more easily and, importantly, with the flexibility of Band in a Box within the next year.

The playing/performance part of making music is a solved problem. You can do this with DAWs and plug-ins today. The truly hard part is coming up with the ideas. That's where AI has an opportunity.

If I'm missing something about BIAB, let me know!

https://youtu.be/PCYTqDSUbvU

I think it's better to think of the process of finding the right song as a search algorithm through the space of all possible songs. The current approach just uses a "pick a random point in a general area". Once we find something that is roughly correct we need something that lets us iteratively tweak the aspects that are not quite right, decreasing the search space and allowing us to iteratively take smaller and smaller steps in defined directions.

We just published a blog today discussing this - https://montyanderson.net/writing/synthesis

I think the other missing pieces I've found are upscaling and stem splitting. While existing tool exist for splitting stems exist, my testing found that this didn't work well in practice (at least on Suno music), likely due to a combination of encoder-specific artifacts and the overall low sound quality. Existing upscaling approaches also faced similar issues.

My naive guess is that these are things that will benefit from being closely intertwined with the generation process. Eg when splitting up stems, you can use the diffusion model(s) to help jointly converge individual stems into reasonable standalone tracks.

I'm excited about the potential of these tools. I've definitely personally found uses cases for small independent game projects where a paying for musicians is far out of budget, and the style of music is not one I can execute on my own. But I'm not willing to sacrifice on quality of results to do so.

That's their "Remix" feature which just got renamed "Reuse prompt" or something.

Their extend feature generates a new song starting from an arbitrary timestamp, with a new prompt. It doesn't always work for drastic style changes and it can be a bit repetitive with some songs but it doesn't completely reroll the entire song.

At some point it's just not efficient to try and get the desired output purely through a prompt, and it would be helpful to download the output in a format you can plug into your DAW to tweak.

Variation in small details is fine, but you need control over larger scale structure.

I was recently really impressed by the state of AI-generated music, after listening to the April Fools LessWrong album https://www.lesswrong.com/posts/YMo5PuXnZDwRjhHhE/lesswrong-... . They claim it took them ~100 hours to generate 15 songs.

Can't wait for the day I can instantly generate a song based on a random blog post or group chat history, this seems like a step in that direction

Focus on product. Give actual music producers something they'll find useful. These fad, meme products will compete on edge model capability for 99% of users and ignore serving actual music producers.

I'd like a product with more control, and it doesn't appear Suno or Udio are interested in this.

Some things it made sounded ok, but I feel like the average generation quality wasn't fantastic. It did a folk guitar melody and a vocoded thrash metal voice that I thought sounded pretty legit, but mostly vocals had an ear grating quality and everything had a bit of low bitrate vibe.

To be honest though, I don't think you need to try and outcompete Suno. I think you want to get into DAWs and VSTs and become the tool all the best producers in the world use. Spit out stems, and train your model on less processed sounds because things like matching reverb/delay and pre-squashed dynamics are a pain in the ass to work around.

Suno is trying to battle a large established industry that is actually very creator friendly and accessible. If you choose to instead serve that industry and enable it I think that's the winning play.

Thanks for the feedback re: DAWs, though! That would be really cool. Maybe we can tag tracks based on the effects applied to them to allow this to be more controllable.

Any plans to release the model(s) under an open license ?

Personally not interested then. I'll stick with Bitwig and Ardour until an open model is available

We (as a society) desperately need a way to train these models in a federated, distributed manner. I would be more than happy to commit some of my own compute to training open audio / text / image / you-name-it models.

But (if I understand correctly) the current architecture makes this if not impossible, nearly so.

I know it will happen, just like SD happened after DALL-E. Bonus points to whoever does so for using C++ and Vulkan instead of Pytorch and CUDA. :-)

Well, maybe I'll try out the next AI music creator posted on HN.

I was thrilled by 2 of the versions produced. I wish I could extend it more like one of the comments here said:

* ElectroKlezmerReggaeFunk 1: https://sonauto.ai/songs/s22rQEPnYsXy1yf7sjU0

* ElectroKlezmerReggaeFunk 3: https://sonauto.ai/songs/1iNTrA2CekPwp7XT9mmM

But wow, the UDIO version:

* https://www.udio.com/songs/j4zpRYgG2GEDbWpLPYbuJb

I presentes this prompt "Noir detective music from the 60s. Low tempo, trumpet and walking bass" and got back a one-note only song that has nothing to do with the prompt if not for some lyrics that were a bit ridiculous.

This is just feedback, I'm passionately expecting something like this to surprise me but I know it's really hard!

Happy to share the song/project/account, if you tell me how to :)

My experiment on sonauto.ai so far - I first selected "The Weeknd", then picked the prompt:

  The Weeknd's smooth vocals lead the song, blending with electronic effects. A low, pulsating bass line opens the track. Synthesizers add layers to the melody, creating a danceable rhythm. Minimalist drum machine beats provide the foundation for the rhythm section."

I also tried uploading Starboy directly as an mp3 to generate from that prompt instead, using the same lyrics. I may have done something wrong (when I went back to Prompt, my prompt was replaced with the string "Uploaded File", and some of the output is so stylisticly different it makes me think it didn't get applied at all) but it didn't seem to work well, if at all: https://sonauto.ai/songs/klNMfs4bPgji3edPvwAv

Did I do something wrong using the upload file feature? And if anybody has hints for getting better output with the auto-generated prompts LMK. I'd love to use these new features but it seems like they're either configuring an underlying modl not mature enough for generally good output compared to suno/udio or the UX is not making good output easy to achieve.

Here's an example output I get from Suno with similar lyrics and a prompt that merely lists some styles associated with The Weekend. As you can see, the song is much better overall, and the voice sounds more like The Weeknd (although it still fails to style the eg Chorus properly): https://suno.com/song/d4f72fce-0bc7-4786-a299-f58d903c4275

My understanding of the music industry is the incumbents are VERY lawsuit happy, and plagiarism laws are substantially more reaching than with image or video (eg cases where someone gets sued for using the same chords as another song) - how do you plan to approach all that?

Congrats on the launch, regardless. I will be sure to check it out when it becomes more accessible.

Seriously, though - the solution isn't to prevent people from doing this, it is to remove the incentives that encourage it.

More seriously, personally none of them, I don't have accounts on any "usually used" login providers. Just allow local accounts.

https://suno.com/

As an amateur musician I'd like to see it take a MIDI track as input and produce audio as output, as a sort of AI MIDI instrument.

Or maybe take some tracks as input and generate another track, both MIDI and audio.

Suno has this issue too, but everything sounds like it's washed out or something. As if you recorded it from a different room.

Still I love this, ultimately I think it'll be a tool musicians use vs something for creating stand alone art

Good work

Spotify is getting flooded with AI generated music. It is absolutely something people will use to just generate the music they want to hear.

Ultimately though, what would be the point of spotify? Anybody will be able to generate 24/7 of songs based on their mood or a few keywords.

It will radically change the music landscape and how people "consume" music.

Things are going to get truly weird when you can no longer tell the difference, on any level.

I suspect record labels might train their own models. I know for sampling, being able to just create a royalty loop without worrying about clearing anything is cool.

  -I write out my lyrics
  -you break my lyrics down to [x] syllables
  -I choose this rhythm control option
  -I hit the spacebar [x] times, in my desired rhythm based on those syllables (maybe a karaoke style visual to guide this), & you calculate timings between each keystroke and then attempt to create rhythm based on that

I mean, as a dabbler in music, not being able to play a given instrument should make me want to learn, such upskilling which should have some beneficial effects even in terms of neurobiology.

Even if I know that could be used as a basis for creative input, it feels like this is dangerous for humanity.

After all, someone has to have something to do in their spare time?

Genre changes for melodies/etc are coming once we finish variations (partial renoising like SDEdit basically).

I have lyrics that I want sung by a solo female voice with the background being a male chorus. Something reminiscent of a woman singing the details of a David vs Goliath type battle backed by a chorus of the victorious warriors from that battle.

So far I have completely failed to be able to generate a female lead with a male chorus backing

I play piano, sax, guitar, and I can sing well enough. I'm garbage at songwriting and composing. I immediately see the value of using this tool to scaffold an idea out. I think being able to export lyrics and chord progressions would be an amazing paid feature to keep this as a freemium product.

https://sonauto.ai/songs/avg5NT3qf9QYNfWAyeOn

Look forward to playing with this.

It's a folksong I prompted about loosing my job to a robot in the style of John Denver, god rest his gentle soul./

https://sonauto.ai/songs/oOdXomZV73uwfQIxIvTU

https://sonauto.ai/song/JSmCpJssZeIS2C87pkQW

I would love/pay/kill for an AI tool to help me flesh them out. Is anyone working on this?

but to be serious: yes, technology sometimes causes social dislocation. and that's serious. but civilized societies have many ways to deal with social dislocation, like welfare, retraining, etc.

Musicians are not going to lose their jobs because of technology. The CD hasn't killed live concerts.

But as a society, we need to do more for those who fall out of the productive system. As productivity and wealth increase, the only problem is redistribution.

Why?

As a community made up largely of picky nerds and pedants, it doesn't seem incredible at all that this comes up so often. More like inevitable.

Good thing that’s not how it works then I suppose.

https://twitter.com/nasescobar316/status/1777481957774872704

https://twitter.com/apples_jimmy/status/1777905772384678149

https://twitter.com/HalimAlrasihi/status/1778118063138673137

https://twitter.com/AngryTomtweets/status/177811764524768059...

https://twitter.com/AngryTomtweets/status/177811769943385715...

And I’d wonder why they encoded at 32kbps with a RealMedia codec from 1998.

https://www.udio.com/songs/bDY5CYdJZP93AdpgpfBJNX

That said, I've only had the chance to generate a few songs with Udio and they have all sounded like they were recorded by a prison band in an overcrowded cell (I create mainly instrumental/orchestra/sound track music).

Or is this just an ad for a commercial product?

You claim that “we're making generations on our site free and unlimited for as long as possible” but I couldn't find any UI where I could do anything for free. The best I could find was a “log in with Google” link. Requiring a Google account means your software is not free, and most definitely not “unlimited”.