My last company (Zip Phone, YC S14) was a direct result of the fantastic work that the Opus team has done in the last few years. I remember researching audio codecs around the end of 2013 and stumbling across Opus, and being amazed at what it could do at extremely low bitrates, and everything was available completely for free! Spent my fair share of time on the Opus IRC channel on freenode (shout out to derf, gmaxwell, jmspeex, mark4o) bugging them with basic queries, and getting excellent support.
If you listen to older versions of Opus @ 32kbps, you'll hear the distortion more pronounced and then when you slowly upgrade versions you can hear it go away and then when comparing to uncompressed, you'll notice just because you know what you're looking for.
Xiph is amazing; I hope Daala actually becomes commonplace, but with the recent Apple announcements it's looking like a chance to usurp h.265 is basically gone.
Many pieces of Daala are being folded into AV1, which is an industry effort supported by most large software and hardware companies to produce an HEVC replacement. This post summarizes the results of the different experiments: https://people.xiph.org/~jm/daala/revisiting/
Unfortunately some of the most innovative techniques around frequency-domain prediction ended up not panning out. :(
The AOM patent license covers any implementation of the format. It additionally covers the reference implementation, because the encoder may use techniques that aren't part of the bitstream format, and it's important to grant rights to those too.
Indeed, the whole point of Daala and AV1 is to do a really good job on the IP front so as to make it much harder for patent trolls to hold everyone hostage [0].
Yes. They have applied for a patent on ANS and video compression - ANS is one of the biggest advances in lossless encoding in the last 10 years. The inventor wants to keep it freely available, has helped Google for three years to incorporate ANS in AV1 - their reward is to try to patent his invention. More details here:
Anyway, as TD-Linux said above, any implementation is allowed.
> 1.1. Patent License. Subject to the terms and conditions of this License, each Licensor, on behalf of itself and successors in interest and assigns, grants Licensee a non-sublicensable, perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as expressly stated in this License) patent license to its Necessary Claims to make, use, sell, offer for sale, import or distribute any Implementation.
> 2.6. Implementation. “Implementation” means any implementation, including the Reference Implementation, that is an Encoder and/or a Decoder. An Implementation also includes components of an Implementation only to the extent they are used as part of an Implementation.
> I really have no idea what "to the extent they are used as part of an Implementation" means though.
Say, for example, you make a chip that does motion search acceleration and you want to sell it to people who do both AV1 and H.264 encoding. You'd be covered for any patents included under this license to the extent the chip was used for AV1 encoding, even though you didn't make a complete encoder. To the extent someone wanted to use it for H.264 encoding, your or they would still have to negotiate a separate license (just as you would have if AV1 didn't exist).
That's intentional. We want to cover components that are themselves used in other components, recursively up the whole supply chain, as long as they're ultimately used for AV1.
The two big Internet streaming companies have chosen VP9 over h.265 and are both developing AV1 to use instead of h.265's successor. Apple's announcement is unlikely to change anything as far as what gets used on the Internet.
I predict a lot of angry smart TV owners - I have a 4K TV that can't decode certain VP9 profiles in hardware, so YouTube on the TV itself is very limited. Not sure what's available in S90x chipsets, though. Does anything do VP9.2 + Opus in hardware?
I think the solution there is to get a streaming stick or a streaming set top box that supports VP9 to plug into the TV. Or alternatively a small form factor PC like a Zotac mini pc, or palmtop if it's strong enough:
Such people can either use lower quality H.264 versions which are commonly provided, or as others said, get an external device to decode modern free codecs.
That was expected, since Apple are part of MPEGLA cartel and they strongly oppose free codecs. But it doesn't really mean anything, since everyone else would have to pay arm and leg to use it, so the pressure to adopt free codec is still here. I hope AV1 will come out soon enough. Daala itself isn't really ready as is.
Apple held off on adoption of HEVC and they certainly don't make as much money as they pay. As much as I hate Apple's love for proprietary technology, had AV1 been released on schedule Apple might have adopted it.
More to the point, VP9 is available today and is a perfectly good substitute for HEVC, with the benefit that you don't have to pay for a license from three different patent pools and an as-yet unknown (but certainly non-zero) number of third parties on undisclosed terms in order to use it.
Apple probably held off adoption so that HEVC patent license situation improves a bit (at least most companies with patents join in some pool or start to offer licenses).
If AV1 would be released sooner it wouldn't be even as good as HEVC, which would be bad. VP9 is already that free-but-not-as-good-as-HEVC codec. Also they would need to release AV2 in 1 or 2 years (as the better-than-HEVC codec), which is just too soon after AV1 and would put a big question on why AV1 was released at all...
I'm actually unsure what's meant by reference here. But VP9/10 is the base codec on which AV1 is being built, but many of the most significant innovations from Daala are being folded into AV1.
Daala isn't dead and will continue to have value, at the very least as an experimental/development codec, but it may also form the base of a future static image format (it compares very well against JPG and HEVC/BPG.
This is comparison is over a year old so it's even better now: https://people.xiph.org/~jm/daala/revisiting/compare.html
I don't think it can move fully, since Daala is by design a different type of codec (using lapped transforms), while AV1 is more "classic" / block-DCT type of codec.
I.e. it makes sense to develop Daala further. Not sure what's going on in practice though. I guess it's better to ask Xiph / Mozilla.
We may return to Daala in the long term: it has competitive performance with HEVC on perceptual metrics despite a vastly simpler design than AV1 or even VP9, and despite being less mature than the classic block-based approaches and missing many tools that we simply didn't have a chance to implement (e.g., Daala has only basic MPEG2-style B-frames with no bi-prediction, as just one example).
Like the old adage says, if you have two baseball players who can run to first base in the same time, and one has perfect form while the other one looks lousy, which one do you pick? The guy with lousy form, because teach him the right form...
However, a lot of Daala's design is predicated on having a very constrained legal budget and not being able to rely on anyone else's patents. With the Alliance for Open Media, both of those constraints are relaxed. So even in the best case the result is likely to look pretty different from the way Daala looks today. Ultimately we're interested in making a codec people will actually use, and that means working with our partners.
The threshold for "transparency" varies a great deal across users. Some people I know cannot ABX 64 kb/s Opus from the uncompressed original, yet once in a while we hear of someone being able to ABX one particular sample all the way up 192 kb/s. There's also the issue of whether you have the reference to compare. Personally, I stop being able to ABX Opus somewhere around 128 kb/s, but I won't be able to tell that a song has been compressed with Opus at 96 kb/s unless I have the original to compare it to.
Listening through all of them, 32 and 48 do have noticeable artifacting if you are using a good sound setup. On my phone though, even 48 was "as good as it gets" for its speaker.
It is still pretty crazy that I find a sub 100 kbps sound sample to be indistinguishable from lossless. Definitely considering transcoding my flac library to 96 vbr opus for backup purposes.
I have a local backup of native files, but I also have some external 1TB drives I usually get access to once a year and store at relatives places in case my house burns down. They aren't big enough to store my entire media library so I usually end up downsampling to "good enough" copies in case of a disaster.
It's really amazingly impressive that it also holds well across the different musical pieces and different frequencies / timbres. Is this yet another order of magnitude improvement in audio file size?
I wonder why the MP3 at low bitrate sounds a bit like a lowpass filter.
At low bit-rate, lame applies a low-pass and resamples to a lower sampling rate. If it didn't to that, MP3 would end up completely starved of bits (trying to code all frequencies) and have really atrocious artefacts. Based on personal experience and the results from a few listening tests I saw, I think you can assume that Opus can now give you about the same quality as MP3 for about 60% of the bits. For example, 76 kb/s Opus should be around the same quality as 128 kb/s MP3.
I wasn't able to hear a difference on that website, but I tried some songs on my computer and there was audible hissing in some parts (it mostly sounded great to me, though). All that was gone at 48kbps, though. It's kind of crazy to see a full-length album sounding amazing at 20MB. I think if I were to convert everything I'd go a little higher (64?) to be safe, but wow.
> Opus defaults to variable bitrate (VBR), but for a variety of reasons some people prefer to use constant bitrate (CBR), which it has always supported.
For instance, if you want to encrypt speech, and don't want to leak information about its contents.
I'm having all my music stored as Flac and automatically converted as Opus to my phone. I couldn't be happier to the quality and encoding speed. The only thing I think I should've done different is to use a lower bitrate than 192 kbps to store more music to the memory card.
I reasoned the additional lossy encoding of Bluetooth would be too much with lower bitrates, but now I'm not so sure...
You can play them in VLC, but a few years ago I remember that Opus wasn't recognised by Android's "media service" (meaning the native players couldn't even detect the existence of Opus files). It's probably changed since then though.
EDIT: It looks like TFA says that Android does use Opus:
> Opus is now widely deployed on a large range of platforms and devices (including Android, iOS, and all major browsers) and is exposed to untrusted data.
Android supports the codec but there are issues with the MediaStore and how it deals with containers and file extensions.
My tests (on a Pixel running 7.0 and the O beta) have determined that Opus is usually fine as long as the container and file extensions are both Ogg. I have had a few issues with certain files but I'm not sure why yet, I suspect it's album art or something.
I don't believe Android <= 6.0 supports Opus as audio on its own. The CDD initially only required support for Opus in a Matroska container and I attempted to put an Opus-only Matroska file on my phone and none of the players were able to play it. As of either 6.0 or 7.0 (can't remember which), Ogg support is required though.
It's really hard to exaggerate the quality of the opus codec.
It's just amazing.
You can throw anything at it. No matter the sample rate and the bitrate and the output has good quality without any of that narrowband, wideband, ultra-wideband speex nonsense.
All that is missing is an "hybrid" mode just like wavpack that produces combined output of roughly the same size as flac.
I don't know what you're looking for with the wavpack comparison, but 510kbps approaches the noise floor for most audio that isn't white noise or black metal. 96kbps is widely considered transparent on speakers (you have to try quite hard to notice 48kbps), 112-128kbps on headphones (and 96kbps at most for people older than 15, lol).
High-quality lossy files are fine for listening, but lossless is required if you want to sample the audio to create a remix or DJ set from it, because you're inevitably going to have to encode the output artifact of that—and you don't want to be chaining lossy encoding steps, as the loss-function of most psychoacoustic encodings tends to be exponential when applied repeatedly.
re: wavpack, from wikipedia:
> WavPack also incorporates a "hybrid" mode which still provides the features of lossless compression, but it creates two files: a relatively small, high-quality, lossy file (.wv) that can be used by itself; and a "correction" file (.wvc) that, when combined with the lossy file, provides full lossless restoration. This allows the use of lossy and lossless codecs together.
This hybrid encoding format lets you keep files for both purposes (lossless for audio production and archiving, pre-encoded to HQ lossy for audio consumption) in less space than would be required to keep e.g. an ALAC file + a cached lossy M4A encoding of it, the way iTunes does when you specify "optimize tracks' space-usage for portable devices."
Such a hybrid design is ideal for e.g. the Internet Archive, where they want to keep lossless archival representations of files for historians to study, and to create encodings from; but also want to serve usable representations of those artifacts to users who care more about transfer-speed and bandwidth cost than perfect fidelity.
This has been tried - the problem is that FLAC works by removing redundancies and correlation in the audio signal. When you subtract an Opus encoding from the original, the resulting difference file is the difference, which is much less correlated noise - often it actually compresses worse than the original FLAC. WavPack gets around this by making its lossy format a subset of the lossless data, but as a downside the lossy compression is far worse - about 3x worse than Opus.
Other attempts include Dolby TrueHD and DTS HD - but both of those ended up larger than just separate FLAC and Opus files.
Keep in mind that Opus does not define a bit-exact decoder. That allows it to have both floating-point and fixed-point implementations, as well as the flexibility to use different internal resamplers, MDCT implementations, etc. That flexibility helps to allow efficient implementations across a broad variety of devices.
However, as a result, the difference between an Opus decode and the original will only get you lossless reconstruction if you decode with the same build of Opus on the same machine. Also, since Opus uses IIR filters that ring to infinity, even on the same machine you only get bit-exact results if you decode each file from the beginning. It won't work if you seek.
Really, if you're concerned about the disk space required for separate FLAC and Opus encodes, delete the Opus version. FLAC decoding is extremely fast (5x faster than Wavpack), and Opus encoding is pretty efficient, too (on the same order as Wavpack decoding).
It's not published yet, though I might. It's pretty simple, just grabs a static JSON manifest (which is generated by the shell script I run when I add music to the collection, which adds replaygain tags and transcodes) which includes basic metadata and the ReplayGain tags so that I can set that manually in the browser. Uses the standard rule for continuous play and replaygain: play whatever was in the view when you played the first song then shuffle play, when the song you're playing is followed by the next song or follows the previous song on the same album, apply album gain, else apply track gain.
I have yet to add dynamic features like cross-player play count tracking, and server-side play counts for the webapp. I think these would be a lot more work than I put in.
I just use separate files, honestly. Hybrid could save me a small amount of disk space, which is itself quite cheap. Allows me to have a central repository for files of every bitrate for each device class. Smartphone gets the 128k, I stream the 96k, 128k, or FLAC (depending on connection and browser constraints) from my private listening webapp, and I keep the 128k Opus on my laptop.
FLAC and Opus are each more widely supported than Wavpack, Chrome can play both as of the last stable release. My loudness scanner, cmus, MPD, and others all recognize each fine, including ReplayGain and album art, but I'm not sure I'd have the same experience with Wavpack.
I have about 600GiB FLAC music from bandcamp, random publishers, and CDs. I then have, separately, the same directory structure but encoded as 96k and 128k Opus. Still all fits just fine on a mere TiB disk. In a couple years the difference between 96k and 128 in terms of space won't matter and I'll drop 96.
And yeah, it's very nice to have the FLACs when I want to timewarp, transcribe, analyze, or remix. I usually do that at home though, so I can splurge on the bandwidth.
WRT the Internet Archive, Wavpack hybrid decoding is probably too slow and too rare to bother including in any major browser. Even if they did, if lossy listening is a priority, a dedicated lossy codec will save them money in short order on bandwidth. Indeed, this is exactly how IA does things, they even have wav in addition to flac and two or three lossy versions of (almost?) every recording.
Not a user of Wavpack myself, but I think the idea would be to have a sort of "pressure-based fidelity retention" setup: where you'd put the lossy-encoded versions of files onto your portable player device, but as long as your portable device has the space, it would also retain the lossless encodings as supplemental files. As soon as your portable starts running out of space, it treats the lossless encodings as a cache folder, evicting things from it to free up space.
Then, you could actually do mixing in apps like DJay straight on your iPad, on a whim, sometimes—without having planned for it in advance—and make it straight through to mastering and uploading the result... unless you filled up your whole iPad with random videos/apps/etc. in the meantime, in which case the lossless versions would be gone, you'd have to work with the lossy copies, and then, when you get home, restore the lossless copies (or transfer the project to your PC) to master the encoding.
So, not so much when you want the lossless version as canonical (most times when you have a lot of space), but rather when you want the lossy version to be canonical, and the lossless version to be there as a sort of "optional file to enable mastering using this track" file, that's sometimes available and sometimes not.
If you are sensitive to the quality of encrypted audio calls, a co-inventor of Opus (Koen Vos of Skype/SILK) now works at http://wire.com, which is taking an open approach to messaging.
Given that Opus is built into the browser, is the format backwards compatible? Or will browsers need to update their Opus Libraries to get these changes?
All Opus releases since 1.0 are fully compatible with the specification (RFC 6716) and all future releases will also be. That's why we have standards. If we ever decided to make something that's not compatible, it would be called a different name.
That's so awesome. I guess the webrtc audio encoding libraries would need to be updated in the browsers to take advantage of these improvements though?
Even that should happen relatively quickly since so far we've also preserved API and ABI compatibility (since 1.0). Also, we're working closely with the Firefox and Chrome teams so they know what's going on with Opus.
My ears still feel that uncompressed is king for several of the test passages but Opus 1.2 was quite good. The problems with the compressed versions is hard to describe because I don't have the vocabulary but it was something that was only discernable on headphones and I would say had to do with phasing between left and right, a stereo 'presence' that was missing in all the compressed variations.
I really hope every Linux distro and all these other OSs will support it from now on, right from the start. Including recognicing .opus files. It's the perfect format for podcasts, because it's great with voices even if the bitrate is low.
My last company (Zip Phone, YC S14) was a direct result of the fantastic work that the Opus team has done in the last few years. I remember researching audio codecs around the end of 2013 and stumbling across Opus, and being amazed at what it could do at extremely low bitrates, and everything was available completely for free! Spent my fair share of time on the Opus IRC channel on freenode (shout out to derf, gmaxwell, jmspeex, mark4o) bugging them with basic queries, and getting excellent support.
Opus rocks.