Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Mofi – Content-aware fill for audio to change a song to any duration (loud.red)
708 points by jaflo on June 26, 2023 | hide | past | favorite | 161 comments
I worked on a web service that allows you to import a song and define a target length that the song will be shortened or lengthened to. It does this by analyzing the song and finding repeating audio patterns. This is helpful for making any song match a video or performance with a set duration. You can also specify areas of the song to prefer or avoid. An example is available here: https://mofi.loud.red/edit/8bd3fdf780f8c3927e41029f3b957f8a7...

The cool thing is that after the song is analyzed on the server, the client can recompute and preview the results completely client-side through an implementation that uses Web Workers and WebAssembly. The audio previewing uses Tone.js. I am thinking of writing up some more details about the implementation in the future.

I'm still working on a way to explain this easily, but I like the idea of carrying over the concept of content-aware fill from images to audio.

Please let me know if you have any comments or questions!




There used to be something called the eternal jukebox I think. But you could give it a song and it would endlessly loop the audio like you have here. But it would also switch it up and change the parts it looped. And it would have a nice visualisation showing how the parts of the song connect and the different paths it takes.

I think it was https://eternalbox.dev/ since that's all I can find on Google. But that site is down.


Yes! It was "Infinite Jukebox," created by Paul Lamere … it was awesome because it would analyse a track, then visualize its "components" and you could watch as the new "infinite" track looped back on itself and jumped from point-to-point in the original track to create an everlasting one.

He created some excellent products from the Rdio API, and later Spotify … and I believe his analysis engine ended up being the foundation upon which Spotify's _play more tracks like these_ capability is based.

Looks like he's moved over to publish on Substack—there's a recent(ish) post reflecting on 10 years of Infinite Jukebox: https://musicmachinery.substack.com/p/the-infinite-jukebox-1...


  However, that wasn’t the end of the Infinite Jukebox. An enterprising developer: Izzy Dahanela made her own hack on top of mine. To make it work without using uploaded content, she matches up the Echo Nest / Spotify music analysis with the corresponding song on YouTube. She hosts this at eternalbox.dev. It runs just as well as it ever did, 10 years later.


unfortunately at some point between November 2022 and today it stopped doing that, currently eternalbox.dev returns a gateway timeout. The source is still available at https://github.com/UnderMybrella/EternalJukebox, and it sounds like you can get it to run locally relatively easily (?)


The Echo Nest had lots of cool toys: https://en.wikipedia.org/wiki/The_Echo_Nest

Edit: I remember a multitrack file format from the past that allowed following both a composer-defined path as well as random/infinite ordering of sequences, it was called digimpro: https://www.kvraudio.com/forum/viewtopic.php?t=102403


The Echo Nest was really cool... they got bought my Spotify and some of the Echo Nest APIs made it into Spotify I believe: https://developer.spotify.com/documentation/web-api/referenc...


Still have my echo nest tshirt. Pull it out on special occasions :)


Echo Nest! That's right.

Did I over-attribute the above to Paul? I didn't realize he wasn't a founder.


Probably not, in the 10 years retrospect post he writes that he built it :) In an Echo Nest company blog post he is referred to as "The Echo Nest director of developer platform". I have no first-hand information however.


I was working on some music retrieval stuff in 2010, so I joined the EchoNest developer program and played around with their web apis that let you upload music and download an analysis that you could use in all kinds of cool ways. They had an SDK with some great demos and example code. I discussed it with Eric Swenson and Paul Lamere, and had the chance to hang out with Paul Lamere and Ben Fields at ISMIR 2010 (the International Society for Music Information Retrieval conference) in Utrecht, where they gave a tutorial about playlisting:

https://ismir2010.ismir.net/program/tutorials/index.html#tut...

https://musicmachinery.com/2010/08/06/finding-a-path-through...

>Tutorial 4: Finding A Path Through The Jukebox – The Playlist Tutorial The simple playlist, in its many forms – from the radio show, to the album, to the mixtape has long been a part of how people discover, listen to and share music. As the world of online music grows, the playlist is once again becoming a central tool to help listeners successfully experience music. Further, the playlist is increasingly a vehicle for recommendation and discovery of new or unknown music. More and more, commercial music services such as Pandora, Last.fm, iTunes and Spotify rely on the playlist to improve the listening experience. In this tutorial we look at the state of the art in playlisting. We present a brief history of the playlist, provide an overview of the different types of playlists and take an in-depth look at the state-of-the-art in automatic playlist generation including commercial and academic systems. We explore methods of evaluating playlists and ways that MIR techniques can be used to improve playlists. Our tutorial concludes with a discussion of what the future may hold for playlists and playlist generation/construction.

The EchoNest SDK was hosted on google code but this seems to be where it lives now:

https://github.com/echonest/remix

https://echonest.github.io/remix/

Examples:

https://github.com/echonest/remix/tree/master/examples

More Cowbell, just 113 lines of Python code:

https://github.com/echonest/remix/blob/master/examples/cowbe...

EchoNest's technology is fantastic, and it's no surprise Spotify bought them and put it to good use.

Here's an email I wrote about it when I discovered it in June 2010 (linkes updated to archive.org):

Subject: Just found and assimilated a Python music analysis library!

This is great stuff!

https://web.archive.org/web/20100623053057/http://the.echone...

Perhaps you have heard of some of the cool recent demos by EchoNest, which have been covered by boingboing/reddit/etc, that use their music analysis library (and are implemented in just a few lines of Python on top of their modules):

More Cowbell -- Adds cowbell or Christopher Walkin to music, on the beat:

https://web.archive.org/web/20100606100215/http://morecowbel...

Swinger: The Swinger is a bit of python code that takes any song and makes it swing. It does this be taking each beat and time-stretching the first half of each beat while time-shrinking the second half.

http://musicmachinery.com/2010/05/21/the-swinger/

Lots of other demos of apps developed with their technology, on their web site:

https://web.archive.org/web/20101101000000*/http://the.echon...

They have an online music analysis and fingerprinting server, to which you can upload an audio file, and download an analysis, look up an identifier (for various services including lastfm, MusicBrainz, etc), and download metadata about the song.

It's all packed up in a nice Python library, which includes a module for uploading and analyzing songs, audio processing, pitch shifting, time warping, etc.

https://web.archive.org/web/20100201151826/http://lindsay.at...

SongBird music visualizer that uses the EchoNest API:

https://web.archive.org/web/20101023215452/http://runningwit...

I applied for an account, got an API key, installed the software, read over it all to learn how it works, and ran some of the demos. It works great, as advertised!

This will be really useful for generating dynamic race tracks, synchronized music videos, and stuff like that!

-Don


Songbird is another cool piece of software that didn't survive the test of time (also the slow disappearance of MP3 blogs). One of the sites that was bookmarked by default IIRC was the Hype Machine, which surprisingly is still up: https://hypem.com/


Yeah, Infinite Jukebox works surprisingly well, and it's fun to manually tweak some of the probabilistic looping points to create something fun.


This project actually builds on the same analysis techniques as the Infinite/Eternal Jukebox! I really liked it when the original was up. I think they also had a version just for Gangnam Style along with a matching infinite video!

There is this version now, but it does not allow custom uploads anymore: https://eternalboxmirror.xyz/jukebox_index.html


> There is this version now, but it does not allow custom uploads anymore: https://eternalboxmirror.xyz/jukebox_index.html

Muchos Gratias to whoever keeps that spinning. I've been itching for infinite Jepsen for a while - but discovered the original site was very dead.


is there any detailed write-up on how this works? just curious to know.


It was Infinite Jukebox. The subreddit is still there: https://www.reddit.com/r/infinitejukebox/


Neat. Went down a rabbit hole and ended up finding this out. Quite interesting

https://publicinfrastructure.org/podcast/80-nick-seaver/


This is neat, I have wondered if anything of this nature existed, in the past, as a child of the 80s/90s attempting to master the art of the perfect mixtape… 30 minutes a side down to the second the tape runs out, would be a win.

But for today’s music, shortening the 2010s/2020 already shorter lengths would mean a song might not be more than a minute in length. On average, full unedited tracks today end up being a bit shorter than they used to be, solely due to the economics of streaming. Rather than paying for the content second by second, it is done by paying per track play. The result is a lot of 2 minute tracks, which were produced with the “verse” parts getting jammed together into the “chorus” with no break in vocals, which also uses pitch adjustments, “the “bridge” is an afterthought that is terrible, or more recently, nonexistent……… Instrumental solo? Anyone? Bueller?

Music is no longer anticipated, budgeted for, and purchased on launch day with great fanfare. We have grown accustomed to the idea that we should have everything available at our fingertips, and as a consequence of this we get exactly what we pay for.


You're not wrong about modern songs - but that's not really the use case for this tool:

A few days ago I was given the task of creating a corporate video - just a rolling slideshow for a shop-window display. Then suddenly it was going to go on YouTube as well - so needed some music! I found a suitable track but needed to edit it for length so its closing chord coincided with the credits card at the end of the video.

This tool might have saved me the bother of splicing the music in Audacity.


I just did the same thing. I made a promo video for an art project which is a youtube short that is 60 seconds long. I wanted both the opening of the song, where things start slow and pick up, and the ending of the song where things build to a crescendo and then nicely close, so I had to find some place in the middle to blend the two cuts of the track together. I just tried this tool and it appears to work well. I have not A/B tested it against my manual cut yet, but I might swap out my cut for this one. Nice!


I mean, I’ll easily concede there’s been a consistent trend for shorter songs, more singles and more solo artists since the… 60s (?), but it is also heavily dependent on your taste.

https://youtu.be/OvC-4BixxkY

and that’s just the first random 2021 song I can come up with within a second.


Hell yeah, King Buffalo has been consistently great for the last few years.


I imagine it'd be a particular boon for dance choreographers and intructors too


Couldn't you also just manipulate the slide duration?


Yes, if the discrepancy is a few seconds - in this case I needed an extra minute or so of music, and the customer had already requested a specific slide duration so I had limited leeway there.


Music is a self eating beast these days. Any new artist is competing with not only all contemporary artists but all past artists too, since its so trivial to just go on spotify and find a playlist with the "best alternative albums of all time" and find all the usual suspects of the last 25 years or so. It's really unprecedented for old material to be at an even pedestal as new material. These older already monied acts tend to get the bigger venues in town and tour all the time too, so it extends beyond the digital realm. Not to mention how many small act sized venues have closed and no longer exist so there are even fewer potential opportunities for such bands compared to the arena filling acts.


My local music venue has ‘indie’ size of couple hundred max. But they mostly program ‘tribute’ bands or ‘classic’ acts because apparently that is the most safe to sell.

Tonight I was at a Melvins show in Amsterdam. It is their 40th anniversary your, amazing! I’ve also seen Einstürzende Neubauten a few months back. Back in early 90s everybody was talking how the Stones were still playing for so long. But nowadays so many bands are still going.


I've rewritten this comment ten times already and I guess I don't really know what I _want_ to say, but this really irks me the wrong way. What you are describing is not Music (with a capital M) but rather the very similar product that is available in high abundance and nearly worthless. Real artists still have a considerable following that will absolutely anticipate and purchase physical media on launch day.

I'm sorry, that's the best I could do.


> Real artists still have a considerable following that will absolutely anticipate and purchase physical media on launch day.

I adore music, always have and follow what I hope are "real artists".

I haven't bought physical media for several decades and often buy albums weeks or months after release.

I think you've sliced this wrong.


She has a cult following, but Taylor Swift's Midnights broke over 1 million vinyl sales (from late October -> late January).

But I mostly agree with you - I get very excited for artists that I really enjoy releasing new work, and "line up" so to speak by listening to it the day it releases on Spotify.


I did not mean to say that "real" fans must buy physical media. Just that there are plenty of them out there who do.


I think you're describing the difference between art and entertainment. Both of these efforts use "music" as their medium, and of course there's a lot of gray area between. But it's not necessarily a bad thing that there are many different goals when producing music.


Totally. I should have posted what I had written earlier, had a section on this in my comment, and even spent a half hour trying to explain how this doesn’t apply to all artists and genres. I wrote it about 10 times and I do agree…

Identifying the good from the rest in such a place as Apple Music is a miserable experience. To Apple, they also make more revenue from their prioritization of marketing similar sounding two minute tracks in this “nouveau pop” format, backed up by a small amount of older superstar artist anniversary editions. Good original new music never makes it to the featured content sections.

And Genres and algorithms are a mess. It applies across the board to all music and is really a problem for the flavors of House, Techno, and other genres that are simply labeled “Electronic” or “Dance.” I’m getting Avalon Emerson one minute, Bicep the next, to be ruined by corporate mass marketed deadmau5, follow by a Bad Bunny remix….worse is the algorithm which thought I might like corporate edc-esque superstars and poorly autotuned remixes- though I have not added a single song to my collection in 20 years of digital music consumption…

Can we do better than “Dance” and “Electronic?” Of course they could, they haven’t. One must go elsewhere.

For House and Techno, this dearth of music discovery and search-ability methods by streaming companies makes room for independent music station alternatives, like Fault radio, or gives a reason for one to seek out artists via other means, like going to independent music festivals like Sunset Campout, Honcho campout, and other events highlighted on Resident Advisor, a poster in a nightclub, a text message listing the warehouse location.

(Apple Music did at least bring back Beats in Space.)

I think about the proliferation of streaming, and how it actually makes finding new content difficult for people who are not familiar with those other means of distribution.

- We have deprioritized the concept of the local radio station, now what was alternative rock is a rebroadcast of an AM Sports broadcast (i.e KFOG)

- the death of the sale of “singles,” made it a cheap entry point for people to experience a new artist. Releasing a track on Spotify doesn’t feel as substantial.

- and exclusively agreements contract provisions with corporate entities that engage in predatory practices that force up and coming artists to choose between performing for their fans at local venues, or extending their potential reach by putting their name on the bill for Coachella, sacrificing potential shows for a few hundred of miles away and several months on either side of the festival.

- Additionally entertainment conglomerates like AEM and LiveNation are increasingly becoming the owners, or managers of, the entertainment venues in cities around the world. Similar exclusivity agreements can have a significant negative impact on unaffiliated independent venues ability to compete.

I can’t speak for anyone in particular from the Gen Z or Alpha generation. I think for them, it’s all Apple or Spotify, music festivals if they can afford them, sharp discerning music choices on TikTok if they feel the need to branch out… and the question is - do they?

The norm is now a fully mass market formula that is almost impossible to break through… and the effect of this puts the chill on the ability to link good music with committed audiences.

I think I mostly explained how I feel there…


> I can’t speak for anyone in particular from the Gen Z or Alpha generation.

I'm just barely young enough to be a zoomer, so maybe my view will be interesting. From my perspective my ability to find "good" music is better than it ever would have been in the past.

I find music suggestions from forums, review sites, subreddits, friends, online people I follow, etc. Then I can immediately listen to it with no effort or expense. I found my favorite album of all time from a random comment someone left on an internet thread.

To me the idea of having to wait until a local radio station played a song, and then make a leap of faith on purchasing the album seems like such a worse experience.

Similarly, in terms of creating music, it has never been easier to learn, create and distribute your own music. The rise of the internet has made it so much easier to find a niche communities of people making incredibly diverse and experimental music together.

In my mind, streaming sites have two roles and do a very good job at both:

- make all music as accessible as possible

- suggest music for people who want to "passively" listen to music, which is how the average person has always listened to music (and that's a completely legitimate thing to want and enjoy)

I see a lot of "the death of cinema", "the death of music", "the death of video games" takes around, and I can't help but feel like these views come from people who have lost track of where the "niche" communities has moved on to and then feel despair when to them it looks like the "mainstream" is all that exists.


> as a child of the 80s/90s attempting to master the art of the perfect mixtape

I hit upon the plan of taping stuff off the radio onto 1/4", and then I could splice a not-talked-over beginning onto a not-talked-over ending.

Later, I worked out that I could extend or shorten tracks, particularly if I could get a tape of the instrumental version, using the same trick.

No-one uses tape and razors these days, but it was good fun.


1/4” is a much better “master” of capturing higher fidelity than my method, a compact cassette using two different cassette tape decks. I remember when I swapped it out and got a dual Pioneer compact cassette deck, which could also be controlled by a remote control. It was fantastic, until the home Compact Disc recorder drive came on to the scene. Alas, compact cassettes are not as effective to splice, and I miss the 90’s.


You can do the same thing with cassettes but they're so fiddly to splice.


I agree, I think another part of it is accessibility to tools and content. Same thing happens with gaming to an extent, when tools became more accessible, so did the content, but the quality dropped significantly, even among AAA games.

I think music suffers even more so because we're all so tuned into having the best at our fingertips that if a single moment in a song isn't to our liking we can skip and forget about it completely - i think this fuels the fast-short song market, easier to saturate with many short songs and get listens rather than to work/slave on a longer more intricate piece.

Back in the day, mixtapes with songs were slaved on and cherished, today slaving over something is seen as a negative quality.


> if a single moment in a song isn't to our liking we can skip and forget about it completely

I do sometimes worry about this. Some of my favorite songs growing up were ones that I didn't care for initially but which grew on me only after listening to it repeatedly. Now it's easy to dismiss anything that doesn't hook me right away. Thankfully albums are still being released, and you can force yourself to listen though them, or keep more of the "meh" songs in your playlists for longer periods of time, even adding them back into rotation after a while.

I don't think I'd give up the variety we have now though and go back to only having what songs are pushed at us through top 40 radio or the limited selection found in local stores. I get my music now from countries all over the globe. Finding old stuff all the time I'd never heard and new things just released.

We can still invest in the music we listen to and be rewarded. We just aren't forced to, so it needs to be deliberate.


In this world of algorithmic content distribution, we have more genres of music than any one person can enumerate.

I don't see any of the effects you describe on my feeds (tidal; previously apple music). Perhaps you need to switch to a service with a better recommendation algorithm, or nuke your personalization profile and start over.


Meanwhile the ghost of the bridge lives on as a sample in contemporary songs. Future generations will watch these ruins in awe of what the masters in the past were able to construct. Will we ever be able to surpass the classics?


I think we peaked with music already, maybe in the early 2000s. Popular music was so simple before the 1970s. There were a few chord structures and you just had some generic verses and put it on a record. Take someone like leadbelly or bb king, they were just playing folk music that had been played for at least 100 years before either of them were born, reusing old verses or adding their own, adding a short guitar lick between verses if so inclined (bb admits this was even because he could not sing and play at the same time). A lot of their fame as a result was happenstance of being at the right place at the right time (leadbelly in particular) versus them being a head and shoulders better folk musician than the other musicians of their time. A lot of the Beatles and other such acts in the early 60s were like this too, just basic formulaic rock n roll songs with the artists marketed as heartthrobs more than musicians even. They all even had the same haircuts.

Fast forward to something like Radiohead's Kid A and you have orders of magnitude more complexity going on the track. So many different sounds layered in very complicated manners. Its almost like a classical composition how there are motifs, movements, different emotions being evoked, but with any sound imaginable compared to what few are possible from the orchestra ensemble. A song from Kid A is just a part of the greater album itself. Nothing was made to stand on its own between radio advertisements.

It seems these days we are reverting to how commercial music was always made. Very commercial studio focused with the artists removed from production. Generic lyrics written by low paid songwriting staff and same old tried and true chord progressions we've heard forever. The artist is a brand and a sexy person meant to sell products versus someone particularly talented with an instrument or with songwriting skills.


And yet the Beatles stood the test of time much better than the alternative hits of the ‘90s (Radiohead included). (IMO of course, but I think its supported by alternative still being an alternative, while Beatles are still THE Beatles after all this year. BTW I’m not a great fan of either, but appreciate both)

It is easy to create a supercomlex composition, that’s what classical music have been doing way before 1970s. But to capture the minds and hearts of many people with simple things, now that’s the real mastery.


What does stand the test of time even mean in this case? People still listen to both Radiohead and the Beatles and lesser known "alternative" acts from the 1960s like the velvet underground today. All went on to influence others. I'd say they all stood the test of time. Stuff like record sales depends a lot more on how well your label commercialized you versus your musicianship. The mid century media era was also much smaller in terms of competing artists that were actually put in front of listeners. 3 stations on TV, a few radio stations playing music from the same record labels, a shop in town selling records from a few major labels, and that's all the discovery you have. popular acts were far more popular proportionally than popular acts in later years, just because you didn't have much option or choice otherwise back then.


It means that the Beatles (who the parent mentioned explicitly as a basic formulaic rock in contrast to the "peak" from the '90s) is much more well known now than Radiohead, and could even have been more popular than Radiohead even during the peak of Radiohead's popularity (with the general audience, of course). I doubt that the problem of alternative bands was media exposure, because they had plenty in their heyday. The issue is that their music appeal only to a small(ish) subset of people, and that most people would like something else. I don't mean this as a bad thing. It is what it is, but I find amusing how cultish their following is :)


This is another sentiment I could not put into words. I agree 100%. Thank you.


The way this works out in practice is a DJ makes their mix by blending these songs together at the start/end via remixes with similar beats.


My word is this impressive.

I uploaded Wuthering Heights[1], selected just the trailing guitar solo (3:09 to just before the fade-out starts), and got a bunch of seemless sounding 3m tracks of just that guitar shredding away.

All the ones it generated were perfectly cut and pasted, with one exception in track 2 where I could tell where it was cut/pasted.

Can't wait to try it with the guitar at the end of Brothers in Arms (Dire Straits), or the flute solo in Locomotive Breath.

If anyone is looking for a good use of AI, I wouldn't mind a webpage that lets me say "complete this solo that was abruptly cut short" and get something great.

[1] This one is especially sad because the sound engineer/producer later lamented that he faded the solo out so quickly because the player was still improving like mad and what he had left after the fadeout was apparently better than what he had done while recording.


Thanks! I took up guitar because recorded solos were never long enough.


There's a specific sect of deadmau5 fans that would love to use this to generate multi-hour continuous mixes of a song called Strobe.


I feel attacked



So the difference there is that's just the song on repeat. OP's tool here will try to fill the song in with content-aware filler similar to how Photoshop uses a content-aware fill feature to try to guess what could be in the gaps.

For people who know every beat of Strobe, it sounds odd when things don't hit the same way, but at the same time it's kind of like a unique live mix.


I hate that the frequency of the bass kicks doesn't match the frequency of the eyes flashing, so they go out of sync. The eye flashing feels about 1 bpm slower than the beat.


This is the coolest thing ever!

I like to have multiple hour continuations of songs which I use to help me fall asleep. In the past I've made my own, but I'm terrible at audio editing. It is a ton of work and, for me, really hard to get right.

After a little bit of experimenting with Mofi, it seems to do a very good job and selecting when to repeat a sections.

[edit]

Here's my first attempt for a 30 minute song:

https://mofi.loud.red/edit/ebbf4b410181aa767152945cbb6a2d679...


I wish this worked. I wanted to listen to an example. But all I got was “You haven't imported this file yet. Import it first to use it.”


Fearing copyright lawyers, I implemented this message so you cannot share audio by just sending a link anymore, but have to import the song yourself to be able to use it :(

You can use this example though if you just want to try: https://mofi.loud.red/edit/8bd3fdf780f8c3927e41029f3b957f8a7...


Local storage can be easily bypassed if you don't encrypt the values in that array...


This gives me

"Not Imported You haven't imported this file yet. Import it first to use it."


After spending all afternoon playing with this. I have two inputs.

1] A lot of the heavy lifting is done client side, which is excellent; I have a super fast processor! But, watching the CPU utilization, it seems to spike the CPU for a few milliseconds, then waits for a second or two and, spikes the CPU again, then repeats. I suspect there may be some significant performance improvements to be had on client recomputes.

2] While it is doing recomputes, it would be great to get a progress bar or, at least, a message saying, "This will take a long time, be patient." It took me multiple attempts to determine that the service was indeed working, just that my client was taking its sweet time recomputing.


Can’t wait to use this to stretch my 19 minute prog rock tracks into 190 minutes :)


I was fortunate to help bring similar tech to life as a PM for Adobe Audition and Premiere Pro in a feature dubbed Remix. Since we were designing to help music fit the duration of a scene or video project, our goals resulted in a slightly different experience from those of music fans wanting to recut a song for general purpose listening, but it worked like magic for most tracks. Still one of my favorite projects that I worked on.


And I just realized you acknowledged it at the bottom of the page! ;)


The tech behind the Remix feature looks really impressive! I haven't used it myself, but while working on this project I had some people mention it to me and I looked at some videos showing how it's used. Very cool feature!


How did you make the animation in the homepage? It's definitely not a screen recording...it feels like something animated in After Effects at 60fps. Is that true? Or did you use something else?



Correct, this is it! It's basically a blend of screen recording and AE. The recording is made and then the software smoothes cursor movements and creates the zoom effects.


As a fan of tools like this and Eternal Jukebox, I cannot wait for the ones that have an option to request a cut specifically, like "from approximately this point, jump back to around this other point", while using the 'smart' features to line up the audio cuts (I'm sure stuff like that is built into the CC suite but I'm not springing that kind of money to recut a few of my tunes for personal listening ).

With Eternal Jukebox I was able to emulate it by setting the jump percent very high manually at the right time (but had to be paying attention + had to get lucky with which branch it took), whereas here I'll see if I can get it going by calculating what the cut I want would set the song length to, and requesting that length and fewest cuts. (Edit: didn't work)

Sometimes the simplest features are the easiest to overlook but most useful


Surprising that the only mention of intellectual property in the FAQ and the legal terms page is about copyright that the service claims in the outputs, not restrictions on the inputs.


The ToS for this claims that you have to pay for it to use the results commercially, but I couldn't find the pricing for it anywhere or any way to sign up.

Interesting, as the primary use case for this seems to be something along the lines of editing music to fit into video projects.


I plan to implement paid features later with a basic free tier. I am still trying to figure that out. I've updated the terms to be more clear about that!


Ah, I see. Nitpick: It currently says "Do not upload rights that you do not have the rights to".


Whoops, thank you, will update!


since there is no login mechanism, I used the anonymous feedback section to request their agreements with BMI and ASCAP for creating liscenced mechanical renditions of copyrighted materials.


This is a good point, I have updated the site to only allow you to view content you submitted and also made it clear that you need to have the rights to the audio to edit it.


Thank you, now I can remove Dababy from Dua Lipa - Levitating. https://www.youtube.com/watch?v=TUVcZfQe-Kw&pp=ygUTZHVhIGxpc...

He just butchers an otherwise perfect song!

Will report back with results.

Edit: Results are great! https://voca.ro/13ar1g88LSKK


Can't you just listen to the version he's not featured on?


I'm not sure there's an "official" version? I've listened to a few on youtube and it seems like they are all fan-made.


Track 5 on Future Nostalgia is Levitating without Dababy's verse.


The first thing that comes up on Apple Music when you search for Levitating is the version without DaBaby.


Seeing the domain has me curious, is it a reference to a Pokemon[0] or is it something else? (Sorry for off-topic!)

> On repeat: Make an extended version of your song's favorite part by choosing the catchy part and seamlessly repeating part of it!

John Mulaney's "Best Meal I Ever Had" story[1] had me wanting to do this for Tom Jones' What's New Pussycat so I could play that as an inside joke with some friends. This would definitely have made that easier. (Ultimately, I had a version of the song which started in the middle and "ended" right before it "started" so the effect could be mimicked if the file was played on loop. This had a short silence manifest as the media player "loaded a new song" from its perspective; didn't have that issue in Audacity.)

[0] https://www.pokemon.com/us/pokedex/loudred

[1] https://www.youtube.com/watch?v=Tv1l1eUhN-E (Usually called "The Salt and Pepper Diner")


Yes, it is! I wanted a short domain for side projects, and then this one came across and I registered it. I tried to give an homage to the Pokemon in the favicon in the root domain :) https://loud.red/favicon-32x32.png

And yes, I had this issue while testing also! A lot of players added a small gap when looping which was a bit annoying to test with...



> This is helpful for making any song match a video or performance with a set duration.

This is perfect for TikTok videos!

Will definitely be using your tool. Very cool!

> I am thinking of writing up some more details about the implementation in the future.

Please do :D


I'm enjoying the results, I can make my favourite tunes 10 minutes long and often it's pretty seamless! I can't wait to get sick of all my songs :)

Is the project open-source?


Glad you like it! Sadly this version is not, but this project is a rewrite from an old version I created that is open-source! https://github.com/jaflo/snipsnip

If you are interested in audio analysis in general, I would also recommend checking out librosa: https://librosa.org/doc/latest/index.html


What's the maximum length? I can't find it documented anywhere and keep running into it when trying to try this with Tool tracks.


There is no enforced maximum length for either input or output right now. In practice, the remote machine times out trying to analyze a song after 10 or so minutes. For output, your machine will take forever to generate really long results.


I see an error saying "Remote media exceeds maximum duration" for this track that is 9:24 long: https://www.youtube.com/watch?v=Y7JG63IuaWs

I'm guessing that's the timeout?


I had forgotten about this check actually! I just removed it and deployed the changes. You can try again but am unsure if it will break something (it will spin indefinitely, but I would give it up to 10 minutes). Good luck!


The way this works is interesting. I asked it to cut 30 seconds from the middle of a song (a spoken word portion without music) and then shorten the track by 30 seconds. I figured it would just cut that out and do some fancy blending of the music before and after the cut to make it sound good. Instead I got:

    at 00:00 a 00:16 segment (00:00–00:16)
    at 00:16 a 00:51 segment (02:05–02:57)
    at 01:07 a 02:31 segment (01:34–04:05)
    at 03:39 a 00:11 segment (04:03–04:14)
It's adding and repeating chunks of the song and cutting other parts and I'm not sure why.


I can finally see and hear what Bohemian Rapsody would have sounded like had they caved in to make a radio-friendly cut.


Does this create "new" music for the filled parts, like some AIs (MusicLM, MusicGen, etc) do? Or just repeat in a more seamless way?


It does not create new music from scratch using one of the techniques you mentioned, but just reuses parts from the song, blending the transitions between them. That is a cool idea though!


I'm a tad disappointed. Cool idea, but I haven't gotten anything to really sound seamless when shortening or lengthening yet.


It definitely works better on pop songs or music with repeating choruses or patterns as those can be matched to each other more easily. What kind of music did you try it with?


This will be great for my programming music.

Though it's just anecdotal but I have seen that the quickest way to enter into the zone is to play repeating loops at 140 bpm.

Unfortunately I haven't really found many of these as most of the tracks I love are non looping. I think this can take care of it. Kudos on making this!


Perfect! You automated my hobby of extending music videos to last up to the max video length available on YouTube in 2008

https://youtu.be/XCJs_eAkNS8


This is pretty good! However, there are some unusual splices. One I found was with a recording of Cumberland Gap[0]. If you run a 10 minute loop, you'll find that the first splice in multiple tracks (happens at about the 2:30 mark), the song suddenly cuts to the middle of a mandolin solo that clearly doesn't belong there. With that being said, I'm still overall impressed by this.

[0]: https://youtu.be/VAk4_CMn7To


I tried lengthening Another Day by Dream Theater from 4:26 to 8:00.

I set it to avoid the segment that the key changes and to favor the 2 solo segments.

It doesn't seem like its doing a good job. Only the first 5 seconds of the intro made it and then is just repeating over and over the big solo. In the end, a very small part of the short solo makes it in as an outro.

Maybe it works better at shortening songs or maybe changing duration closer to the original duration.


This is exciting. It adds to the tools creators have to remix songs. A middle ground between full AI and full human-made songs is a mix, like what Bronze AI has done with one of my favorite songs. (https://bronze.ai/listen/jai-paul/)

A less rosy scenario is like Spotify generating music to get around having to pay artists.


If you're stuck on Loading (0 of 19), make sure you have a new enough Firefox version to support dynamic import for workers (https://bugzilla.mozilla.org/show_bug.cgi?id=1540913, i.e. 113 or newer).

For example, ESR 102 does not have the feature and it breaks the site.


I created a 10 minute version of The Less I know The Better (https://mofi.loud.red/edit/dde278b8c131c1a6e47117ceb29d1ceb6...) but the export doesn't seem to work, it just errors out with "Invalid Crossfade" multiple times


Whoops, looks like there was a bug, just fixed it, could you try again?


Looks like it works now, thanks!


is it just me or does exporting not work to share it with people, it says i need the original song to view that link above ?


Are you worried about trademark infringement?

https://mofi.com/


Totally.. Mobile Fidelity is also in the music space, I'd expect a lawsuit or the name of this software to have to change.


Can we use something like this to remove the laugh track, please? PLEASE?

It needs to be generalized to video content, but the task is a bit easier.


Very cool! It seems to struggle with mixed meter music, though (understandably). I've uploaded a song of mine that's mostly in 6/8, but has some bars of 7/8. It still makes smooth transitions, but on the wrong beat, which is very noticeable. I will try it again with a song in only one odd measure, maybe that works.


Interesting how the youtube videos are pulled down in a cloud function and stored on a file host for caching. I know there is a disclaimer about making sure you have the applicable rights, and IANAL, but this is for sure violating some terms and potentially copyright when users are using videos they don't have rights to.


Really nice! Do you use a library or service to find the self similar points, or is it your work?

Found a bunch of cool related links here [1].

1: https://en.wikipedia.org/wiki/Music_information_retrieval


It would be cool if you could link to a particular cut (i.e. a five star 10 minute version I really like)


Perhaps tangential to this project, but I have always wanted something that can detect different instruments/sounds and allow me to adjust the volume on them/remix them.

There are a few songs I love, but some of the best parts seem to be "buried" under other instruments.


This can be done with quite aged AI actually. You can split a song into vocals+instrumental, or even vocals+drums+bass+other

https://github.com/Anjok07/ultimatevocalremovergui


I'd like to simply filter vocals (for listening while work/coding/...)... any tools for that? Preferably local without requirement for AI/nVidia card...


First thought was that we finally have an answer for Symphony No. 8 (https://en.wikipedia.org/wiki/Symphony_No._8_(Schubert))


What's the difference between "Search" and "Edit song"? It seems like Search is just about length, and Edit also considers sections you marked? Or does Edit do the same thing but with many edits, rather than a few?


Seems kinda cool but I'm stuck on "Loading results... (0%, 0 of 19 done)"


Same here, you have any script blocking etc?


I like this but there are a lot of studios in the world where I will never be able to access it, so I'd really like to have a standalone/non-web version if its at all possible. I'd pay for that.


My big question: what have you learned while working on this (very cool) project?


This project brought together a lot of different pieces of engineering and was a larger personal project for me, so there were a lot of parts and making those work together was a challenge at times.

For the backend, I found it helpful to implement monitoring proactively to be able to detect and debug issues without having to bug users which has been really helpful so far and allowed me to make some performance improvements too!

For the frontend it was interesting to optimize the result searching using the Web Worker + WebAssembly approach. I found some interesting ways to speed up things by parallelizing and, interestingly, manually collecting garbage in the WASM runtime.

And of course learning about the entire audio analysis portion and reading on how to find similar sections of audio!


Thank you for making this tool. So impressive how many features work well in this initial version.


Please add the option for a longer fade out, plenty of songs finish too early and it would be great if they could just fade out rather than stop abruptly.


This works great with the clips made from MusicGen. The loops are seamless. Thank you for making it free. I would love to know more about how you are analyzing the audio.


Seems really nice, though for some reason after pasting a YouTube link it's stuck when reaching 100%, would have liked to give it a try.


Looks like the machine ran out of disk space, I bumped it and it should work. Could you try again?


Works well now, thanks! Overhaul quite impressive even though sometimes the cut is still noticeable.


Pretty soon I can see how people will make a content-aware fill for relationships, heartfelt messages etc


[offtopic] Could you share what tool you used to generate the demo gif/video? Very nicely done!


Interesting. Is there any chance you would release this to run natively rather than as a web service?


I think it could be possible (backend is in Python), but for most users a web service has a lower barrier of entry. Could you share your use case? I am curious.


There's nothing special about my use case. It's just that I avoid relying on anything that is only available as SaaS, because it can vanish or change at any time without notice.


Super cool, not what you would typically think of when it comes to gen music!


Getting an SSL error - invalid HTTPS certificate. Site is being tagged as malware.


Cool! It seems to work well with slow songs, the transition is very smooth


It's possible to add multiple files and added them at once?


Would love to sort by most cuts just to see what happens


This is pretty cool! Will you consider open-sourcing it?


I know I’m in the minority here, but I hate it. The artist created created a song of a particular length for a very specific reason. I prefer to listen to a song the way it was meant to be listened too.


They created it that length for their reasons. You might want it a different length for your reasons. It's not clear that their reasons trump your reasons, given that your modification has no effect on them whatsoever.


I choose to listen to songs by an artist because I enjoy the choices that artist made.


That's fine. Other people make other choices.


That’s fine, as long as you have obtained permission from the copyright holder to create a derivative work. Otherwise, their reasons very much do trump your reasons.


Sharing a derivative work could breach copyright, but if you are doing it for yourself you don't need permission.


Sometimes they create music at a particular length because that's what worked well on radio, or TikTok, or for a club set. I would go so far as to say that song length is probably chosen for some specific non-artistic constraint more often than it is chosen for some reason intrinsic to the art. So changing the music to accommodate your own constraint, i.e. "this is the part I like when I'm working out and wish it was longer" seems completely reasonable, and isn't even justified in being considered some sacrilegious act.


adding to this, how streaming made rap songs shorter: https://pitchfork.com/thepitch/considering-the-rise-of-the-s...

I don't remember exactly but back in the radio days, songs had to be a very specific length to be a pop single.


Do you ever decide to stop playing a song before it is over, or do you refuse to because that would be a violation of the artists intent? Likewise do you ever listen to only some songs on an album, or do you always play all the songs on an album from beginning to end?

Where do you draw the line between acceptable "edits" vs unacceptable?


This is thinking from a bygone passive-media era--the one of television and radio--where you were forced to be the passive consumer and had no choice but to obey your mass media masters with respect to what they delivered.

There are so many tools to customize and make your audio experience your own. Ultimately my ears are my own and I love living in a world where I can precisely control my audio experiences.


Baffling comment. Practically every trailer has music cut to length.


this seems to assume artists are infallible and in total control of the creative process. I would never assume either to be true.


You can still do that.


Do these get tagged by contentID on Youtube?


How do you get the audio from Youtube?


I use https://github.com/yt-dlp/yt-dlp

You can use -F to get the available formats and find out an audio-only stream then -f FORMAT to download the specified stream.


I wish google ranked services like these right at the top when users search for these tools, currently they are filled with ad filled, paywalled, slow & tedious media editing services

I wouldn't mind giving money to services like this


super fun to use. very useful right away


very cool. is this purely DSP based?


Cool!


Cool!




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: