This is great! I've always wanted to get into computational music.
I spent five years working with audio/video codecs and APIs on the Xbox 360. Had to support various DSPs and help games improve their performance (curse you, denormals!). But I never got a chance to go full synthesis.
Thank you for this! Look forward to digging into it when I have more time.
Just curious: on your list of publications[1], music shows up only once. How does this course relate to your other work, if at all? Or is this more of a side interest for you?
Thank you for writing this book! I'm currently working on a music creation program based on modular synthesis, and I expect the filters chapter of your book will really come in handy.
Hi, I'm a computer musician. I just had a quick look at the book, but this looks like a really nice resource! I'll put this on my reading list for summer.
This seems nice, introductory and very up-to-date. If I may offer some superficial criticism, I found the covered material was at odds with the title. The title evokes themes of algorithmic composition and experimental whole-music generation, while the book is actually about digital synthesizers and effects on a digital signal processing level. Of course I'm having a hard time coming up with anything better, but maybe in the lines of "Audio Programming for Music", "Music DSP for Programmers", "Digital Synthesizers and Effects"...
My first thought was the same. Swapping "music" for "audio" would probably work. On a side note is generative music even music? (I think it can be but) According to my undergrad music prof the answer would be no, it would be classified as "not music", same as bird song. I've since seen research that refutes her birdsong claim (she wrote extensively on it) but I'm curious what the general consensus is?
There is no general consensus. It's a work in progress.
But given that a fair amount of post-war contemporary classical was written using aleatoric techniques - basically constrained randomisation - and no one had a problem thinking of it as music, and modern generative music can be far more sophisticated both culturally and structurally, I think your prof was just plain biased and wrong.
See also: digital photography isn't real photography, vinyl is better than FLAC, computers will never play chess/go, etc.
A lot of modern electronic music actually has generative elements in it. Granular synthesis is very often random-based, though it's more in between an audio effect and generative. Especially in modular synthesis there exist very advanced and interesting arpeggiator modules. I know for a fact that these techniques are used during certain psy-trance livesets (forest psy, among other things).
I mean you still could argue it's not music, but I bet the audience on the dance floor disagrees.
On that note, what some favorite (ideally freely available) lectures/notes/books on the subjects of algorithmic composition and generative music? Ready, go:
This format is amazing. I love watching these types of videos on YouTube, but the random video-hopping the algorithm gives me via "recommendations" never feel productive. Now when I procrastinate by watching internet videos I can point to this book and say, "See, this isn't procrastination, I'm actually working towards something". Many thanks to the author.
Thanks for submitting, I'm currently working through Stanford Audio Processing for Music Applications but this books looks like a nice companion (and I tend to prefer written content to videos).
I studies some signal processing stuff in college but didn't really connect it to music unfortunately so struggled. Since then I've gotten very interested in electronic music production so it's a 2020 goal to learn more about the underlying software and math techniques, so I really appreciate this resource.
I downloaded it first "without honor", puzzled why he wanted my email. Then I tried entering my details sans email, and was rewarded with compliments to my character – it seems email is not required.
It would be better for the author's information collection, and friendlier to readers, to indicate that email is not a required field – otherwise people will assume giving email is necessary, as in a zillion similar forms online. And/or explain why email is wanted.
I'm the author. In all my free books I have collected email addresses only for very rare events. How rare? In 15 years of publishing free books I've only used the books' email list once: I used a previous book's list to let people know that Edition 2 was available.
If you put in fake information, my little collector is none the wiser. Or just click on the "without honor" link, which is really there so Google can grab it.
Thanks for submitting this! Sounds really great and I’m excited to read through this. The theory of FFTs, equalizers, etc is sometimes really unapproachable because of the dense mathematics, it’s much easier for me to comprehend algorithms and concepts as code.
I'd strongly advise against trying to write music software for a living though. The market is just too competitive now and DAWs are increasingly bundling everything a user might need to make music.
Why discourage people from entering music tech R&D? I’ve worked in this space for 7 years and we’re constantly on the lookout for talent! DAW programming is just one option for people wanting to develop musical applications, e.g. game industry (procedural audio, adaptive synthesis, etc), generative/adaptive music services (generative composition/arrangement for media or context dependent listening), mobile app development (on device synthesis, voice processing, etc). Furthermore, thanks to online learning resources (e.g. this author, JUCE ADC, etc) and online communities (e.g. KVR), it’s never been easier to learn how to build music DSP applications. However, we’re increasingly looking for people who have a strong background in both musical DSP and ML, so if you want to pursue a career in this space I suggest brushing up on traditional DSP and modern DSP/ML combined approaches (e.g. DDSP https://magenta.tensorflow.org/ddsp, QMUL research topics http://c4dm.eecs.qmul.ac.uk/audioengineering.html)
I hope the current giants in DAWs are doing much experimentation with VR tools, with the addition of multi-modal interfaces (like the mimu gloves and others), which could open up some marvelous ways of re-thinking music creation as a whole-body experience.
The tech for VR might still be a bit limited but the potential for the intersection of it and multi-modal interfaces is, in my opinion, very interesting.
Almost every DAW fails to handle spatial audio rendering properly, so I doubt this is the case. Spatial computing that relies on spatial audio rendering is basically impossible to predict in current DAW offerings - you have to rely on game engine technology for the most part.
The most accessible DAW that can handle spatial audio is Reaper, which doesn't even allow an odd number of channels.
Could you expand on what you mean by not handling properly? I don't work with non stereo audio and mostly use Logic but I assumed something like Nuendo would have excellent multi channel support as it's targeted at audio for film.
The problem is deeper than multichannel support, which is poorly supported everywhere except Pro Tools. Even that is fairly deep, Nuendo relies on VST3's model of channel set configuration which is incapable of representing the kinds of multichannel configurations that are in vogue for spatial audio rendering, like you'd find in VR.
But even without leaving stereo, DAWs are already really bad. Think about how a pan or balance control works, instead of representing the perceptual localization of a sound they ask you to control the gain feeding two channels, left/right. The semantics of those two channels change after rendering, which makes it difficult to convey the same intent of a mixer/producer to the audience using that interface. Compare a stereo mix on near field monitors positioned perfectly, vs reference headphones, vs a car, vs a club system... the pan/balance control you set in your DAW changes the localization of the sound depending on ultimate render target, whereas what you need for spatial audio is for the localization to be independent from those targets.
The solution is object based audio, which is not perfect and a misnomer 90% of the time, but even that is because of how terribly DAWs handle spatialization. And it's not something you can really control with a plugin without hacking the mixbus configurations inside a DAW, and you can't propagate position down to other plugins.
Waving your arms around explicitly conflicts with the very fine motor control required to play an instrument accurately and expressively.
You need between two and 12 or so (including feet) channels of very precise movement data to make a useful musical instrument. VR hardware isn't designed to have that level of precision.
Of course you can have virtual pads/buttons triggering samples, with a few faders for control of effects. But that's more like sample-based DJing than instrumental performance. It might well have commercial value, but it's a very limited subset of music production as a whole.
No doubt it's hard and technically limited too, with the state of customer-grade sensors (like the mimu, or eye-tracking devices) and VR. There are still many hurdles to simply running VR devices, so we're far from the kind of adoption that would lead to meaningful inventive approaches to something as complex as what DAWs and their physical extensions offer.
However it's more about thinking of musical creation very laterally. It could mean leveraging tilts of any place where one can put an accelerometer (heck, the OP1 does it in a gimmicky way to change any specific setting you want for a given track for example), your finger movements position in 3d space (mimu does some of the finger movement stuff), heck even smoothed-out eye "gestures" could be explored (google-glass-y devices would need to succeed since you're not going anywhere with current tech without invasive things like the EyeLink II). Pivoting around 360 degrees in a "musical cockpit" could also be a thing.
Voice too, and NLP could be leveraged in creative ways (to bring forward different aspects of the tool or to give instructions at a high-level - "take the first four measures of the first guitar track and apply melodic inversion, shift down one octave and paste at the 2min mark on the bass track").
Different parts of the music-making process could take on different shapes in spaces that add multiple dimensions. As the mimu shows, a lot of what those dimensions do could be entirely configurable and fitting the creator.
Conductors are a good example of someone leveraging rotations and depth (over half-sphere surfaces) to create expressions and rhythm and so on.
Plus nobody has to throw away the current tools either, once could think of the advantages of AR (Augmented Reality) which could leave perfectly clear the areas of your synths (or under a certain line of vision or even physically marked delimitations) during a performance, and all of the glory of ye olde knobs and faders can remain intact.
Current VR is out of reach for too many people for much experimentation (basically forces you to be on Windows, requires either heavy-duty hardware or has very little room for fine details), which means all the exploratory stuff to lead to even a subset of what I mentioned is only accessible to a handful of people so far. Same for multi-modal interfaces (the mimu is still a toy for rich artsy-techy people, the Tobiis are either too basic or too expensive, etc).
But we're still in the infancy of much of these kinds of tools. And while the usual engineer will talk about why everything's impossible, a designer or an artist will always be curious to see what just could be eventually.
Mostly I just read a lot of DSP articles and picked up random techniques from forums. There are a few good books now that would save you a lot of time if you wanted to do this today though.
