You're making the problem a lot harder than it needs, or ought, to be. The most telling example, IMO, is the Aphex Twin one...
> Oh, and his real name is Richard David James. What are you
> supposed to use for the file system directories and files
> name? His name? The most common nickname? Both? One file
> system solution is to have symbolic links (do you link
> Richard David James to Aphex Twin, or vice versa?). For
> tags, if you don't want to lose information, this is
> another story...
You should—obviously—not attempt to "link" AFX to Aphex Twin to GAK to Blue Calx to etc. etc. The artist behind all of the monikers has made a deliberate decision to release work under different names. Organize accordingly.
Many of your nightmare scenarios appear to be a result of the same kind of over-thinking, or invention of nonsensical requirements. How are you supposed to deal with Japanese artist names? It literally doesn't matter—pick a scheme you can understand, and be consistent. How are you supposed to deal with multiple artists? List them, separated by commas, in the artist field. If they appear on an album released by a different group or person, use the "album artist" ID3 field. And since (if?) you use the ID3 fields to store your metadata, and presumably navigate your collection through an interface over that metadata, all of your questions regarding how to store files physically on disk are totally irrelevant, as long as you pick some scheme which doesn't generate conflicts. The default iTunes structure (Artist/Album/01 - Song Title.mp3) seems to work fine, for example.
Well, it's the same person behind. Even if the artist tried to give a different "personality" to his music over the time, following all his creations is actually a good listening experience. The "hey I just want to listen to the musical evolution of this guy" isn't a rare feeling IMO. Grabbing all his nicknames is kind of a problem.
Of course, you have a point, I'm making the problem harder than it needs, and you can just don't care about most of these issues. Hey, that's actually to the conclusion I reached. But still, I believe the current solutions are not optimal if you want to match N songs, store all the related "context" information, or just keep a consistent way of storing them.
But that's not a problem, either. In my music collection I have, e.g. tags ARTIST=AFX and FILED_UNDER=Aphex Twin. Then when you make a playlist with your favorite music player, you can just sort things by FILED_UNDER, if you prefer. What's so hard about that?
Interesting, so you have a way of making the relationship at the end. Now I could start nitpicking about how you decide to make the link to Aphex Twin and not his name (or another nick), and how would you make that decision for any similar cases.
Note that I don't consider this issue the main problem, it's just one I hit a few times, and I wasn't able to select a correct solution.
Also, discogs will track which albums were released under different artist tags. Foobar+discogs tagger combination lets you pick, if you want to use most common general name (so all tracks are Aphex twin), or the alternative name that songs were released under (aka: AFX songs stay as AFX).
---
This is same issue as a book ''Running Man'' (yes, followed by a movie with Arnold). Was it written by Richard Bachman, or Steven King? How does your public library list book?
Sure, but the answer is you just make an arbitrary decision that feels right. Same thing with name ordering and the other issues you mention. There isn't much benefit to coming up with some elaborate system of rules for these things -- after all, the goal is to create a categorization that is easy for you to find things with, not to generate the timeless perfect metadata which all humans agree is correct.
> The default iTunes structure (Artist/Album/01 - Song Title.mp3) seems to work fine, for example.
It only works fine for basic rock and pop (because even if there is a composer different from the interpreter, that's optional). It starts breaking for remix albums released under a given artist (the original composer) where one song may have been remixed with three different people, now you have to start infecting the track names with the remixer.
And then you've got classical music, where each of the composer, orchestra, conductor and soloist may be considered the artist, but none really is because the authorship of this precise interpretation is a combination of all of them, and any missing will lead to a collision. TFA also touches upon original sountracks (of movies, animes or games) which regularly open whole new cans of worm, and special compilations such as Dark was the Night (40-odd artists — excluding composer for covers — for 31 tracks in a single album, good luck handling that one without relying on the "various artists" crutch)
> where one song may have been remixed with three different
> people, now you have to start infecting the track names
> with the remixer.
For what it's worth, I don't consider that "infecting". It's really the only sane way to handle huge collaborative works.
There are always exceptions which break any categorization scheme. The point is to treat them as exceptions, ie. fold them into the 90% model as well as you can, rather than restructuring your model to accomodate for 100% of every conceivable artistic license.
> And then you've got classical music,
This one _is_ interesting, but I solved it (personally) when I realized I only cared about the original composer (eg. Mozart) in terms of "artist", and the minimum nomenclature to disambiguate movements, etc. in terms of "title". Trivia like the performing orchestra is perfectly well homed in the album title, or ignored entirely. I admit I'm only interested in listening to classical music, not cataloging it to some deeper academic purpose.
It comes with a Musicbrainz plugin which allows you to select an album in your library and search for it on musicbrainz (e.g. by artist name and year or number of tracks-- so you really dont need to have much info) and then you can select a match (there are usually many releases of a given album and it will actually differentiate between them) and then tag your album with the musicbrainz tags.
In fact you don't even need to use quodlibet to get this feature. It has a separate tagging component called "ex falso" which you can run standalone and then use the player of your choice.
But I would strongly recommend Quodlibet for its organizational capabilities as well. For example it uses special internal tags (not id3 but stored in a separate db) that allow you to associate "people" and "performers" to a track so that the track will appear when you search for any of those people.
Also there are sort tags that allow you to customize where stuff shows up, e.g. I can have a track with Artist tag "London Symphony Orchestra", composer tag "Ludwig von Beethoven" (the proper ID3 tags) BUT I give it the artistsort tag "Beethoven" so it shows up under B. Perfect!
Lastly it has regex search! And you can make "saved searhes" e.g. playlists. And it's lightweight and has an uncluttered (but highly customizable) interface. And it's very easy to write plugins for it.
Nice, I'm sure a lot of people would be interested in trying this.
But talking to myself, I'm disgusted with all of this, so I'm just maintaining my mess in its current state for now.
Still, I like to see such solution, and I'd be really interested in a counter article to what I wrote dealing with each issue. Even if at the end, I will likely not use the given solution.
About the regex search, I'm not sure that's really the solution to the "textual problem". As mentioned in the article, the music content retrieval system is in my opinion the future. Echonest and similar services are trying to achieve something like this. Looking for one artist isn't really what you actually want most of the time. It's likely you are looking for good music, and just want to listen to things who sound "like this".
Sorry-- I was unclear. The regex is not for organization. It's just an easy and flexible way to browse subsets of your library, like smart playlists in iTunes, but more powerful.
I only go so far as sanitizing and standardizing my music collection through Tag&Rename (and I haven't found a good OSX equivalent to this yet sadly). It gets the data from Amazon in 98% of cases, adds the album art (which I like having on my player), etc. Then I store the files in:
Artist/Album/Track# - Song name [- Artist name]
The last is only there for soundtracks and other "Various Artists" type collections.
This is Good Enough [tm] for me. I can sync this across hard drives (backup), minimize duplicates (although I end up with these through compilations of various sorts), etc.
Unfortunately the ID3 tag system is All Wrong [tm] for this in many ways stated (in this post and elsewhere). For example:
- Albums don't really have an artist; songs do;
- Programs for automating this that get info from Amazon and elsewhere tend to use what year the particular CD was released rather than when it was originally released, which is far more useful and relevant (eg if you want the Beatles White album you don't care the CD was released in 1998, it should come up under 1968;
- Albums don't really have years either. Or at least they have publication years. The songs have years. Normal studio albums have a common year. Compilations and soundtracks do not;
- Genres are coarse-grained, arbitrary and (IMHO) mostly useless;
- What I like is greatly influenced by the circumstances around the song, sometimes more than the song itself. I might like a song because it reminds me of a particular person, place or event. Or even mood. Sometimes its the lyrics. Sometimes its the sound. No recommendation engine is going to capture this sort of angle.
This goes beyond music: people just aren't interested in classifying, well, pretty much anything. Playlists seem to be about as far as most people are willing to go. Playlists are a fairly convenient way of coming up with s event-specific music eg for working out, for relaxing, for dancing, for a party, etc.
Efforts at far strong and more accurate metadata, classification and organization speak more about one's festidious--even anal-retentiveness--more than any real need or better outcome (IMHO). It's just rabbit-holing really.
Unfortunately ID3 tags are a terrible failure for classical music: do I put Chopin's Nocturnes by Martha Argerich under Chopin or under Martha Argerich? I actually have a random mix of both : a Chopin folder with stuff played by Argerich, Pollini, etc. and some Argerich, Ashkhenazy, etc. folders.
Standard genres are terrible: you have like 20 different sub-genres of electronic music, but Machaut, Bach, Verdi and Boulez should all go under "Classical", are you friggin' kidding me?
There's a much worse problem which isn't related to classification by the way: it's almost impossible with any compressed audio format to chain parts without an audible cut. It's a big, big problem for classical music too.
Yeah. Music is classifiable in some sense, but its a hugely complex and multi-directional thing and most 'classifications' really just scratch the surface. And while algorithmic recommendation services (lastfm, spotify, etc.) can be nice for a while, actually people are complex and always change their views and interests, so they are never always going to give you what you want.
Personally for my own collection - I have a loosely interpreted directory structure that I've evolved that is basically 'continent and/or genre'/'artist'/'album (with year recorded at the start if I have lots of them)' that works well for me and I don't have to constantly rename everything I download (theres some less-sorted directories of things I don't listen to much within this).
For searching, tagging and playing I use foobar2000 (unfortunately still windows-only) because it has plenty plugins which handle everything from tagging (eg. from discogs and freedb), through to searching (keeps a constantly updated and strongly index media library, which can be searched from a range of front-end UIs) and information finding (getting album covers, lyrics, artist biographies, etc. and displaying as you like). I don't do lots of retagging, but I do find it useful sometimes - for instance I have tagged some of my more listened classical music by composer and performer separately, so I can look at it from either direction when I want to.
Albums often have a person that mixes songs together. In ID3v2 series we use a tag "TPE2". Early ID3v2 adopters used "TXXX ALBUM ARTIST" and "TXXX ALBUMARTIST" as originally TPE2 was reserved for Band definition. iTunes and WMP both standardized on TPE2 for album artist.
Armin Van Buuren - State of Trance v561- #2 - Artsy - Open Space
TPE1 (song author) - Arsty
TPE2 (album author) - Armin Van Buuren
TALB (album title) - State of Trance v561
TRCK (track number) - 2
TPE2 typically comes in use with TCMP designating that a song is a compilation. (Before TCMP became standard, people also used "TXXX COMPILATION").
--
[[ You need multiple versions of songs ! ]]
Most programs automating software will ask which version of album it is.
The last two songs differ. You need to pick right year to ensure you tag them properly.
Alternatively, the year could foretell how remastered the song is. Is this original compression, or the new louder version? See Loudness war article. http://en.wikipedia.org/wiki/Loudness_war
--
[[ date/year is screwed up -- agreed ]]
We have to differentiate three types of dates:
When song was recorded -- aka: This song was released in 1994
When album compilation was released -- aka: the best of 90s was released in 2005
What decade the song is trying to recapture -- aka: the song feels like music from the 60s
--
[[ Genres / taggins is a solution -- although at time blunt ]]
Genres were terrible in old ID3v1, where we were locked to 127 genres.
In ID3v2 we can enter whatever we want into genre, and store multiple of them. Good taggers read data off Last.fm / discogs / music brainz enter complete information about songs. This includes tags detailing genre, instruments, style, moods, decades. Certain players, like XBMC can use style and moods. Some systems like discogs even support hierarchal tags (see their style vs genre).
I'm not sure how much of this is conjecture. My thinking differs from yours with respect to the following -
"Albums don't really have years either. Or at least they have publication years. The songs have years. Normal studio albums have a common year. Compilations and soundtracks do not."
An album that is released by a record label does have a year, as upon release it becomes a publication. By this train of thinking i would also say a soundtrack does have a year, too, as it is published as a collection timely to the context of its release. The same can be said of compilations, as they are a contemporary release.
Yes, but you're usually more interested in when the song came out than when the album came out.
If you sort your playlist by date so you can pick songs from the 90's, you probably wanted to hear something like Nirvana instead of The Best of The Who: Volume 3.
I agree that year is screwed up, but some taggers (like the picard-lastfm tagger) reads also decade feel tags. So music that sounds like 90s is tagged as 90s music, even if it isn't recorded in 90s.
For example, ""La Roux -- In It For The Kill"" will get the song tagged as sounding 80s like:
Genres are coarse-grained, arbitrary and (IMHO) mostly useless
They're as useful as you want them to be. They're just strings, strings which might be a way of grouping works just as ALBUMARTIST or YEAR might be.
If you want highly specific genres, use them. If you want very general genres, use them. If you want some general genres for areas of your music collection that is less populated, have a mixture.
Somewhat related but as I've been building a rails project I've been meaning to open source the song parser I've been building alongside it. It scans an mp3 and pulls out the artists along with the type of role they played on the song. Here's a quick gist I pulled from my model:
If there's any interest, I'd love to turn it into a proper github repo and accept some pull requests.. it's far from perfect (both code-wise and generally) but works well for most cases.
This looks awesome. You should consider sticking it somewhere and giving it a name, so that I might find it 6 months from now when I'm motivated to fix my shitty Python script [1].
Also, a side question: is it normal/considered a best practice in ruby to monkey-patch built-in libraries like that? I know ruby has open classes, but I'm just curious how using them in this way is regarded.
I put the link to the gist with the code for now, when I get a couple hours free sometime (soon I promise) I will do just that and put it on my github, so feel free to watch my account there.
As for patching string.rb I don't know 100% but I believe thats the point of being full OO is that I can make a patch like that where it makes sense to do it. I was using those functions in multiple places, and given they are meant for strings it made sense. I can see the potential pitfalls when it comes to sharing things like this, but again, Ruby makes it easy for anyone to make the same patch for a reason.
> Also, a side question: is it normal/considered a best practice in ruby to monkey-patch built-in libraries like that?
No, it's not considered to be good practice. I think it's generally frowned on by more experienced folk.
However, ActiveSupport (which is part of Rails) does a lot of monkey-patching of the basic Ruby classes in this way, and that probably helps to perpetuate monkey-patching among newer developers.
Let's say we have a compilation made by a group called 'Air'. This compilation is called 'Deck Safari'. Do you use Album Artist instead of artist if it is a compilations (aka: if TCMP flag exists, use TPE2 instead of TPE1?)
And then, how do you combine it with the fact that a year later was a different mix released by Avex Asia Ltd. (AVTCDS-235) with different set of songs?
1) How do you deal with songs that have multiple genres?
In "Singles" I sometimes copy individual MP3's into both Rock and Metal. It comes up less often than you'd think.
2) how do you deal with album that has songs of varying styles?
I just pick the one genre that makes the most sense. As long as I know, or can quickly find which genre an artist is in, that's what matters. How long does it really take to check if I put an AC/DC album in Rock, or Metal?
3) Compilations
I either place it in /Albums/Soundtracks (if it's a soundtrack), or make a "Various Artists" directory in /Albums/Electronic or whichever genre makes sense. Almost all of my compilations are dance or electronic music.
4) I don't understand how you handle singles.... To me it looks like your singles folder is intended for random songs, not to handle actual CDS/CDM style singles
I actually removed most of the EP's and CDS rips from my MP3 collection because I found I never listened to them. I would take the individual MP3's that I really liked and placed those into my /Singles/ directories. Otherwise I could treat them just like I do my Albums.
I guess I enjoy putting more tags into collection. I try to fit few from last.fm , and few that came with the tagging. So a single track can have multiple tags (I think I average 10 tags per track). It is normal to have things like like 'Symphonic' and 'Rock' and '80s' and 'Krautrock' as separate tags on one song. Or in case of some electronica (like ones found at http://ektoplasm.com ): 'IDM', 'downtempo', 'goa', 'psy-dub'.
For compilations I try to keep track of who made compilation, especially if it is a mix by a DJ. I do have many that are just VAs too.
In the end, I have three types of music plays:
- everything by an artist -- this is why I keep CDS / CDM / EP to have remixes, and to track them with covers, etc
- specific album -- this works great for movie scores, or albums that have songs tailored to go one to another (Jean Michelle Jarre work from 70s and 80s is awesome in this)
- party mode -- something like pandora -- last.fm is queried about tracks similar to current one (and past one), and sets up playlist of similar songs to play. See http://forum.xbmc.org/showthread.php?tid=83915
> I can see few issues with this that wouldn't work with me:
And then, good luck with classical pieces, who's your artist of the composer, the orchestra (or group), the conductor (if any) and the soloist (if there's one)?
Yeah this is essentially what I did (before giving up and starting my own music database model and management software), except I have a few more collections (Albums, EPs, Singles, Live, Mixes, Best Of), no genre folder, and years/labels in the folder names..
What I lack (and I doubt I'm the only one) is well-thought-out tag system and more advanced players.
All textual entries (title, album, author, ...) should be stored in original language using original alphabet. Player could transliterate them if user doesn't know the alphabet (e.g. doing romanization of hiragana, katakana and kanji using Hepburn system for japanese music). Such entry should be able to store also a translated text, usually at least English one for non-English stuff.
That would solve also another problem the blog post author mentions, first name and last name ordering issue. Quoting Wikipedia:
"In Hungary, along with China, Korea, Japan and in many other East Asian countries, the family name is placed before a person's given name."
Thus in original language the order would be original, but in English one, Western-style, i.e. placing last name after the first name (and of course transliterated already).
It would be up to a user to choose what player should show her or him: original names, transliterated names, translated names.
But AFAIK ID3v2 and Vorbis don't support such stuff (well, you can try going with custom keys, but non-standard means mess) and I haven't heard about player that would do any transliteration either.
---
As for filenames I think that the best is Latin alphabet, with the most-widely used romanizations of non-Latin alphabets and simplifications of extended Latin alphabets (like removing diactric marks etc.). Clean visible ASCII!
I know that Asian people would mostly disagree on such file naming rule. :)
It would be interesting to make an analysis on how MusicBrainz deals with all these problems, I admit I didn't look much into this. But AFAICT it wouldn't really solve the file system problem, except if you decide to name your files with a hash. Also, you might still want at some point to keep extra information MusicBrainz wouldn't handle, even if you have a MusicBrainz ID stored in the file to identify the music.
About the ASCII, my point was just all about the fact that you can't actually keep only the international name and you need to store the name in different language version. Obviously, I don't have any problem with using UTF-8.
PS: note that UTF-8 won't be able to represent properly mathematical formula... :)
yeah, sorry, was in a rush to go eat. didn't really mean that your worries would all go away, only that musicbrainz might be a good place to look for more info.
The single most annoying thing I encountered was the inability of ID3 to handle multiple albums. Every artist sooner or later releases the exact same piece of music on another album. WHY DIDN'T THEY THINK OF THAT?
That's irrelevant, it's either ripped from one or the other, and that's the information that should be there. Lookinng up on which albums a song has appeared should be a database / wikipedia lookup. Think about it, is it relevant on which christmas compilations "so this is christmas" has appeared?
It's not irrelevant. If the exact same recording of a song appears on two albums, an ideal categorization system would allow it to appear as such without storing multiple (redundant) copies of the song. An easy way to think of it is a normalized relational database. If the song has an id of 5 and is on two albums with id 2 and 3, then there would be an AlbumSong join table with schema (album_id, song_id) and two rows: (2, 5) and (3, 5). It should be irrelevant which album you ripped or downloaded first.
> If the exact same recording of a song appears on two
> albums, an ideal categorization system would allow it to
> appear as such without storing multiple (redundant) copies
> of the song.
I don't agree. The song was released twice; if you have both albums, you ought to have two copies of it.
Blue Oyster Cult are one of my favorite bands. I have many of their "greatest hits" albums because they release one every few years, and frequently there will be a few previously unreleased tracks that are worth having.
This means that in my iTunes library I currently have seven copies of Don't Fear the Reaper. Three of these are copies of the studio version. Incidentally, I hate this song, and though I think you're right and I should have two copies of the song, I can't help but react with mild revulsion whenever I realize I'm going to be hosting yet another copy of it.
Again, I think you're right, and it's not particularly meaningful to worry about the extra ~100 MB of space they're taking up (ALAC format), but it's an annoyance.
Yeah, it's not going to be a big storage hit in reality (and even if it were, you could just rely on file system level compression/deduplication). I, like the author, just enjoy musing about the "perfect" way to organize a music library.
I wonder how often a bit exact copy of one song appears... Sure the songs might have the same title and duration, but some very trivial edits might be done. You need to be extra careful when identifying these songs as identical if you don't want to lose any information.
Of course, from a listener point of view it might not mean much, but how would you decide at which level of non-similarity a track is the same as another one?
It's a bit like choosing a lossy codec quality, it's very subjective, and for indexation this might not be the best way.
You bring up an interesting point: the identity function for a song is not self-evident. Even if two versions of a song are from the exact same recording and mastering, one could have a bit of extra silence at the end, or if they're lossy encodings, different encoders or bitrates would yield different files. What I said would only work with songs that are declared to be identical, however you choose to do that.
Each album is going to be mastered separately, and thus include slightly different tweaks (to volume, compression, whatever). You'll almost never see the exact same audio file stored twice.
Why do you care about this? When you listen to the album, you obviously want to hear every song on the album, and hard drives now are so big (and this occurence rare enough) that the extra space can hardly matter. So why do you care?
Obviously, it's not a huge problem. Like the author of this article, I am just annoyed at how difficult it is to conceive of and maintain a perfectly organized music collection. As for your specific question, however, it would be trivial to query your collection like SELECT * FROM songs WHERE song.album_id = 2, which would get all the songs on that album even if one of those songs is also part of album 3.
There seems to be confusion between the filesystem and the metadata.
The filesystem is for storage... it's only important to be able to group tracks together in small batches (releases - albums/singles/EPs) to be able to manage the files
The metadata is for searching, grouping and locating in your music player.
With that in mind, a lot of the problems he's cited vanish.
I have ~84,000 tracks from over 6,000 albums. The result of a 5 year ripping spree after a decade worked in the music industry.
Every track, with no exception, has been scanned by MusicBrainz Picard, had it's PUID generated and meta data normalised.
I've allowed my definition of genre be influenced by general opinion... I simply learned how the mass tag things.
The result is that I can find everything in my interface (Squeezebox) within seconds.
The file storage I only need to care about for management of the files, and copying to my portable player or deleting old releases (which I do rarely, but it does happen).
The file storage starts with file type:
MP3|FLAC|FLAC_9624
Below those are folders for high level type of content:
I have no problem at all storing and finding tracks, and I've no problem at all searching for tracks.
One of the good things about Squeezebox is that when you search it searches all metadata and the full file path. So a search for "quadro poly" would turn up the Polydor version of Quadrophenia.
Agreed. Although I appreciate the 'hacker sensibility' of using a hierarchical filesystem, such a structure is simply never going to cut it for music. There are too many opportunities for multiple tagging and too many areas that don't fit into the hierarchical approach.
In addition, depending on your chosen music player, practicalities dictate that changing file and folder paths is invasive and error prone - the file location is often a music player's 'primary key' for a given piece of music. Moving it around will lose associated data. It's best just to go with a real simple filesystem hierarchy of data that won't change, artist/album. I've written about this here http://www.blisshq.com/music-library-management-blog/2010/07...
What is Squeezebox exactly? I found many programs named like that and the one I think is you are talking about seems to have been discontinued now. Do you have a link or something?
I use Squeezebox server on a QNAP NAS storage thing.
Then I use a Squeezebox Transporter for the superlative DAC that it has.
Squeezebox server allows me to listen remotely too, so for more than a decade I've had the ability to listen to my home music collection from wherever I am in the world (assuming I have an internet connection). I use DNSMadeEasy for their dyndns stuff.
Having worked with it for a few years, modelling music metadata is indeed an absolute nightmare.
There are some efforts to standardise this stuff though, see http://www.ddex.net which a lot of the digital supply chain is starting to adopt. It's something of a set of scary great kitchen-sink XML schemas (schemata?) but might be of interest to those who get massively nerdy about this sort of thing.
This sounds like exactly the kind of problem that relational databases were designed to solve. You can organize it with an entity-relationship model fairly easily. Once you have stored your information in a database, the filename doesn't really matter as long as each mp3, ogg, etc. gets a unique name; you can look up the file by querying the database for files that have the properties you care about.
This isn't something that relational database fixes. It is about schema and data tagging. It can all still sit in ID3v2 tags, as long as people use all tags properly.
The problem is that most tags are badly written, and most software that displays songs only displays bare minimum. Typically tags I miss seeing: Style / Moods / Genres / Composers / Performers. Even Album Artist or Compilation tags are bad. Same with some misspelled tags, or inconsistent (is it 'D&B' or 'DnB' or 'Drum-n-Base', etc).
I think Figs is onto something though: the OP is clearly worrying about how to stuff all this information into the filesystem, and the plain fact is that while all of the information needs to be accessible beyond the database, either in the hierarchy or in the tag embedded in the file itself, a lot of that information is not structured in a manner conducive to access via those two rather limited methods.
For the sake of argument, let's be platonic idealists for a second and say that a song is a concept and the files are simply recordings of performances—they're not "songs" at all, per se. Say I play "Black Betty" by Lead Belly. I get curious about the song, so I want to hear other versions of it. There's no way to express this data in ID3 or through a naming convention—it's just a reference to a concept that has no existence in a pile of MP3s—and it's hard to imagine a linking regime that would be maintainable. To do this, you need a place to put metadata that isn't attached to the files themselves, which can point to them. Note that this kind of scenario is not outside the OP's scope: they say in the article they are interested in providing all music to everyone, so these kinds of relationships would need to be documented and searchable.
I personally think a relational database is a fine tool for this problem, but lately I've been looking at RDF for things and I think RDF might be an even better choice, since it's really designed for metadata description, ontologies (like music categorization) and graphs. I think it would turn out to be a ton of work, but I think it would satisfy many of the requirements.
Oh god, I tried renaming files on my own (and I'm kind of a beginner programmer). Then I found out about unicode problems and then came 2 problems.
problem #1 was apparently I set the encoding wrong while renaming the ID3 for the music files so the foreign languages turned into question marks. I thought scratch that, time to use someone else's tool. foobar2000 solved everything I had a problem with except problem #2.
for problem #2, I have no idea if it'll ever be solved. The question marks or blank boxes ending up on my computer deal with the encoding for the OS itself (or something). And even with multi-language support, windows doesn't let me fix that. There's obviously something wrong when my blank box issue disappears when I restart my computer from time to time...
This is the shitty python script I use to organize music, which seems to work pretty well except in one case[1]: https://gist.github.com/3681165.
For the love of god, try it on a small sample of your music or it will wreak all kinds of havoc and you will yell and curse my name.
What it does: takes a directory of folders with MP3s, and makes them into the a directory of folders like yourdir/[ARTIST]/[ALBUM]/[TRACK No.] - [Track name].mp3. More importantly though, I believe it handles unicode correctly, and it's just over 100 lines of code, so you could adapt it to whatever system you prefer.
Find the python library called Mutagen that deals with parsing ID3 tags and put it in the same directory as this horrible script.
[1]: The one case it handles spectacularly poorly is when you have an album with multiple artists. This could be a soundtrack, a compilation, or even that thing that happens especially often in rap where the artist is like "[Some dude] ft. [Some other dude]". Honestly, I just handle those cases manually, but it might not be that hard to make the script do it right.
By the way, I've really only run this script on Windows. There are undoubtedly some tweaks that should be made to make it work correctly on Unix-y systems.
That's nice, thanks for sharing. I used exfalso, for which mutagen was written, to build a similar directory structure for my music. Mine was "Audio/[ALBUM_ARTIST]else[ARTIST]/[ALBUM]/[TRACK 00]. - [TITLE].mp3" The one thing it did that your script doesn't is it will enter TPE2 if present and backtrack to TPE1. And similar considerations can be entered for adding a disc prefix for multi disc albums. (Since that anachronism is still relevant.) The only problem was I needed to go back through and delete a bunch of empty directories.
My late friend Alain Fournier once told
me that he considered the lowest form of
academic work to be taxonomy.
And you know what? Type hierarchies are
just taxonomy. You need to decide what
piece goes in what box, every type's parent,
whether A inherits from B or B from A.
Is a sortable array an array that sorts or a sorter
represented by an array?
If you believe that types address
all design issues you must make that decision.
It looks like the author has come to an understanding with his/her disorganized music collection, but beets makes it pretty easy to bring sanity back to a messy music library: http://beets.radbox.org
I second beets, it handles big libraries better than any tool I've come across. My once messy library is now pretty much perfectly tagged (thanks to MusicBrainz) and organised.
I feel your pain. I have also stopped caring about music classifying, for much the same reasons you listed. Nowadays I use only Spotify, and I trust in their tagging abilities, and in Last.FM's auto-correcting ability.
My main concern when I used foobar used to be how to handle multiple artists on the same album, which always resulted in the album being split up when displayed in a list. I was unbelievably happy when I discovered the "Album artist"-tag, which unites all songs in an album under the same banner, while still preserving (and scrobbling) the original artist name.
Last.fm and Spotify generally give me terrible recommendations and no discovery at all. I think this is because they use machine learning on the person <-> track graph to determine which tracks are related, so they're very good at finding you DragonForce or IOSYS, but very bad at finding you Hammers of Misfortune, Gallowbraid, Umbrella, or ｍ’ｓ. Of course, if you listen to music that Last.fm thinks is adjacent to IOSYS or DragonForce, you've probably already heard of IOSYS or DragonForce.
Pandora doesn't have this issue because they use human analysts to determine the properties of the tracks and give you similar tracks rather than tracks that are listened to by similar people. Unfortunately their music library is pretty limited. Google Music's automatic playlist feature does something similar without the human analysts, but this only works if you already have all the music you want and it fits onto your Google Music account. They don't seem to provide an option to pay for extra space.
I don't think there is a perfect way to organize a music collection in a hierarchical manner, so I don't even bother. Good tag metadata and foobar's search does all the work for me.
To me, the purpose of a filesystem is not to implement fine-grained categorization, but to provide basic grouping of related files so that I can easily operate on them all at once. To this end, my music collection mostly consists of one folder per album in a root music directory. Folders are usually named "Artist/Group Name - Album Title". That naming scheme doesn't always fit (albums featuring various artists, soundtracks in which I'm more likely to care about the title of the work rather than the artist that composed it, etc), but I don't try to separate soundtracks from regular albums or anything like that, I just throw them in the same root directory. With this scheme, it's easy to delete/share/transcode an album when it's contained within a single directory, convenient for people I share with, and I don't waste any time obsessing over something that I rarely need to see.
Some people have advocated a more database or metadata-oriented approach where you strip all metadata from the filename and folder hierarchy and stuff all your files in one directory. It's an interesting idea, for sure, more closely resembling the way web services like Youtube store their content. It makes one begin to imagine a desktop operating system that featured a metadata database as the primary filesystem organization scheme in place of the traditional hierarchical filesystem.
With our currently available tools, however, having some kind of useful metadata in the filename and/or filesystem hierarchy, even if it is redundant, is incredibly useful when performing manual file manipulation, especially the aforementioned sharing of files. You'd need ubiquitous categorization metadata in files (that is, not just ID3 and company for music files) and ubiquitous support for parsing this metadata in everyday applications (that is to say, when beginning a download of a song or a document, your web browser would show you the relevant metadata and hide the filename, if it exists. when opening a file, one would have to be greeted with a search box instead of a traditional hierarchy dialog) before we could ever entirely transition from having meaningful filenames to having meaningless hashes, timestamps, or garbage as the primary identifiers of files.
No, there is a whole field that deals with issues like this. It's called library science. Specifically, dealing with this problem is called [authority control](http://en.wikipedia.org/wiki/Authority_control).
Some of the other problems are also solved by cataloging experts.
I have two classifications for my digital collection: Old Music and New Music. Old Music is everything recorded before I was born. New Music is everything recorded after I was born. It cuts down on putting too much time into it all.
Another annoyance the author didn't mention is that unzippers frequently mangle the filename and/or metadata if it uses Unicode. Archive Manager is the worst at this.
Many of your nightmare scenarios appear to be a result of the same kind of over-thinking, or invention of nonsensical requirements. How are you supposed to deal with Japanese artist names? It literally doesn't matter—pick a scheme you can understand, and be consistent. How are you supposed to deal with multiple artists? List them, separated by commas, in the artist field. If they appear on an album released by a different group or person, use the "album artist" ID3 field. And since (if?) you use the ID3 fields to store your metadata, and presumably navigate your collection through an interface over that metadata, all of your questions regarding how to store files physically on disk are totally irrelevant, as long as you pick some scheme which doesn't generate conflicts. The default iTunes structure (Artist/Album/01 - Song Title.mp3) seems to work fine, for example.