To me, this bug is most similar to the Timsort bug back in 2015 [1].
Timsort is an ingenious hybrid sorting algorithm originated from CPython and many implementations including OpenJDK adopted it mostly via a source-by-source translation. Timsort particularly maintains a stack of sorted runs, and due to the construction there is a small enough finite limit in the maximum possible stack size. However the original CPython implementation didn't exactly match what was proven, so there were rare cases where stack overflow could happen. So this was a serious security bug in CPython, but wasn't in OpenJDK because Java instead threw an exception in that case.
Similarly, this WebP bug occurred because the largest table size was formally proven but it didn't match what was fed to the source code. This kind of bugs is not only hard to verify but also hard to review, because of course there is a proof and the source code seems to match the proof, so it should be okay! This bug suggests strong needs for approachable formal verification, including the use of memory-safe languages (type systems can be regarded as a weak form of formal verification), not human reviews.
Specifically, since performance is crucial for this type of work, it should be written in WUFFS. WUFFS doesn't emit bounds checks (as Java does and as Rust would where it's unclear why something should be in bounds at runtime) it just rejects programs where it can't see why the indexes are in-bounds.
You can explicitly write the same checks and meet this requirement, but chances are since you believe you're producing a high performance piece of software which doesn't need checks you'll instead be pulled up by the fact the WUFFS tooling won't accept your code and discover you got it wrong.
This is weaker than full blown formal verification, but not for the purpose we care about in program safety, thus a big improvement on humans writing LGTM.
It's not full-blown formal verification but it doesn't have to be. Unlike most formal verification projects that I've seen, the proof part of Wuffs isn't about correctness (proving that "this implementation satisfies the PNG specification"), it's about safety (proving that "this implementation won't read/write out of bounds"), which is much more tractable. Especially as the PNG spec (or WebP spec) doesn't mandate how to treat malicious input (like unbalanced Huffman tables).
Wuffs doesn't have a WebP decoder yet but it's literally in the Roadmap and I've previously written the golang.org/x/image/webp package in the Go programming language.
Wuffs does have a PNG decoder, PNG uses Deflate compression and Deflate and WebP's "decode a Huffman tree" data tables are very similar. Wuffs also has an equivalent of zlib's enough.c program mentioned in the original blog post to calculate worst-case memory requirements. Wuffs' version is called script/print-deflate-huff-table-size.go and it says that, for a 9-bit primary table, we need 852 table entries. Round that up to 1024, for nearest power-of-two.
If you look for HUFFS_TABLE_SIZE = 1024 and HUFFS_TABLE_MASK = 1023 in Wuffs' std/deflate source code, you'll notice that Wuffs' Deflate decoder isn't susceptible to the same problem as the C/C++ WebP decoder the original blog post discussed. This is because, unless the Wuffs compiler can prove otherwise, the code doesn't just look up the table like `this.huffs[0][i]`, it's like `this.huffs[0][i&mask]` and `i&mask` is always in bounds. The mask is either HUFFS_TABLE_MASK or it's of a refinement type guaranteed <= 511. The array is always (statically) allocated with 1024 entries even though enough.c says that, with dynamic allocation, we could possibly get away with a smaller table.
As you already know, for Wuffs (and unlike C/C++), taking out the `&mask` will lead to a compile-time error. Wuffs code won't compile unless the compiler can prove that array indexes are always within bounds.
From the grandparent:
> this WebP bug occurred because the largest table size was formally proven but it didn't match what was fed to the source code
With WebP+enough.c this 'largest table size' was calculated by exhaustive, brute-force search (not really a formal proof) but it was based on assumptions (balanced codes) that didn't match actual (malicious) input. Or, in zlib-the-library, there's other C code (https://github.com/madler/zlib/blob/ac8f12c97d1afd9bafa9c710...) that rejects unbalanced Huffman codes, but IIUC similar enforcement was (until very recently) missing in libwebp.
With Wuffs, even if the worst-case calculation was based on an incorrect model (or forgetting to separately reject unbalanced codes), the end result (on malicious input) might be "the wrong pixels" but it shouldn't be "buffer overflow".
Hi Nigel, Thanks for interjecting (and of course Thanks for WUFFS). I'm glad to hear that WebP is "on the Roadmap" but I think this sort of work ought to have been considered by vendors as a must-have, high priority project, rather than being sort of vaguely welcomed sideshow that's not on their critical path.
Yes, I actually think that the choice to unequivocally prove safety rather than to attempt to prove that you implemented something specific (but with the risk that our specification is wrong), is almost invariably the right choice for this problem space. It would not have been appropriate for the TLS 1.3 protocol (which has a machine proof that it satisfies our intended criteria stated in the RFC, modulo the Selfie attack and assuming our cryptographic primitives all do what they said they do), but it's exactly the right thing for say a DOCX parser, or ZIP parser, or a JPEG compressor or similar for which WUFFS is the right choice.
> With WebP+enough.c this 'largest table size' was calculated by exhaustive, brute-force search (not really a formal proof) but it was based on assumptions (balanced codes) that didn't match actual (malicious) input.
Maybe I'm stretching the definition, but the exhaustive search is also a proof, especially when it can be done quickly or an efficient proof certificate can be generated (enough.c is the former). I even think that enough.c can be modified to generate a reasonably sized trace to aid verification.
As an intellectual 'puzzle', I'd be curious to see what that sort of trace-guided verification could look like, if that's even feasible. My recollection of enough.c is that, algorithmically, it's sufficiently complicated that I need to really think hard when I'm reading the code. But also, it uses clever data structures specifically to reduce the memory requirements at runtime.
In comparison, "array size is always 1024, array index is always bitwise-anded with 1023, therefore always in-bounds" is undeniably simple and, per "The Fastest, Safest PNG Decoder in the World", practical and fast.
While I agree that you don't need the tight bound (especially because I think the tight bound for incomplete trees is not much higher than one for complete trees), you do need to be confident that 1024 is either a loose but correct bound or highly unlikely to be reached in practice. So you need to calculate some kind of bounds anyway... And this is another reason I thought something more than Wuffs is desirable [1]; a dynamic memory allocation, when allowed, makes many things easier to implement and prove.
1024 is a loose but correct bound. That's what enough.c (or script/print-deflate-huff-table-size.go) says.
Stepping back, I'm not sure if I understand what you're saying. Perhaps we're just disagreeing on how "formal" (as in, part of a "formal verification" process) the enough.c program (or equivalent) needs to be?
---
As for dynamic memory allocation, Wuffs has a mechanism for the callee (Wuffs code) to tell the caller (C/C++ glue code) that it needs N bytes of "scratch space" or "work buffer", where N is dynamically dependent (e.g. dependent on the image dimensions in pixels).
It's not "dynamic memory allocation" in that Wuffs still doesn't malloc/new and free/delete per se. But it is dynamically (run time, not compile time) sized.
Grepping for "work buffer" or "workbuf" in Wuffs' example/ or std/ code (or SkWuffsCodec.cpp) should give some leads, if you want to study some code. The workbuf_len is often zero but, for Wuffs' std/jpeg and std/png, it is positive. Manage the work buffer in the same way as the pixel buffer: it's always caller-owned and callee-borrowed.
> Stepping back, I'm not sure if I understand what you're saying. Perhaps we're just disagreeing on how "formal" (as in, part of a "formal verification" process) the enough.c program (or equivalent) needs to be?
Ah, I think I misremembered that bit of Wuffs code, specifically around `HUFFS_TABLE_SIZE`. The comment does mention enough.c but I thought `HUFFS_TABLE_SIZE` was also set to the tight bound found by it, which was not the case, and wondered why you seemed to downplay the importance of enough.c verification. I agree that you don't need any additional verification in the current code.
> As for dynamic memory allocation, Wuffs has a mechanism for the callee (Wuffs code) to tell the caller (C/C++ glue code) that it needs N bytes of "scratch space" or "work buffer", where N is dynamically dependent (e.g. dependent on the image dimensions in pixels).
Isn't it fixed during the decoding process though? If my understanding is correct `decode_frame` is not supposed to change what `workbuf_len` will return later (or examples don't account for them yet). Technically speaking you can already use the suspend mechanism to request a memory allocation of any size, so it is more about a matter of public API, but such code would be very regular and repetitive. I thought a limited mechanism to enable an in-situ dynamic allocation---preferably desugared---might improve a usability a lot.
> Isn't it fixed during the decoding process though?
For Wuffs' image decoding, it's flexible up until decode_image_config returns. It's fixed afterwards.
In practice, that works fine for BMP, GIF, JPEG and PNG (and I'm confident will work for WebP too). No in-situ dynamic allocation (with or without sugar) needed. Even though, for the libjpeg-turbo C code, `grep mem..alloc_ jd*.c | wc -l` shows more than 50 "allocation" call sites, Wuffs' JPEG decoder does just fine with just the one work buffer.
Yeah, Wuffs can be classified as memory-safe languages in this case and should be endorsed as much as possible. I mean, I said a lot about the limitation of Wuffs in the last conversation, but that's because I believe something like Wuffs should be used more widely, and the current limitation of Wuffs can hinder the progress. We need several angles of attack to make it real, and Wuffs is only one of them.
> type systems can be regarded as a weak form of formal verification
Depends on the type system, right? I was recently looking at Idris which is a general purpose programming language but apparently can be used for proofs.
Not entirely - I wrote a lot of numerical backend code intended for 24/7/365 continuous use in the face of potentially any input .. aquisition | instrumentation equipment failures can produce glitch inputs just as damaging as delibrately crafted malicious inputs from a hacker reverse engineering existing code.
The common ground being code that has to be robust in the face of any input has to be (correctly!) verified, constrained within hard resource limits, have graceful fail over behaviour, some measure of sanity checking, etc.
Otherwise, if an e\/il hax0r doesn't get you .. mother nature and sheer bloody chance will.
I would really recommend trying to write a little C program that deliberately causes a stack overflow. It is completely harmless and is good fun if you've never encountered this kind of low-level systems code before.
It's also pretty fun to try asking for weird amounts of memory with 'malloc' or seeing how many times you can ask the OS for no memory before something fails. You might need to force-reboot your computer after that though!
The answer to this stuff isn't to fuzz, it's to cut this code out like it's a tumor. Then if it breaks stuff throughout the OS/browser, write one heavily sandboxed and memory-safe format converter that can handle the problem. I'd rather have an iPhone or browser that is annoying in a few edge cases than have code like this where vulnerabilities are almost guaranteed (irrespective of fuzzing.) I know I'm being optimistic here, but I'm positive this won't the last chapter in this story.
I'm positive this won't the last chapter in this story
I'm sure you're right about that.'Fuzzing resistant, takes human-directed fuzzing to recreate a PoC' seemed fun, but, as you say, that's the magic of memory unsafety.
Good questions -- yes other Chromium-based browsers would likely be affected by this bug. Many of these do a commendable job of following security updates in Chromium (like Brave), but others tend to fall quite far behind (like Samsung's SBrowser).
Chrome desktop was affected as well, both on Linux and Windows. Chrome bundles its own version of libwebp, so even if your Linux distribution hasn't patched yet, as long as Chrome is up-to-date you should be OK (in terms of browser attacks at least).
There's lots of wonderfully obscure image file formats that are supported by the major browsers and operating systems. For example you can load a KTX2 file (Khronos Texture Container) on MacOS, or a DNG file (Adobe Digital Negative) on Android. Lots of interesting and highly exposed attack surface for attackers to explore.
Chrome on MacOS was affected as well, yeah. Note that we don't know if attackers exploited the bug on platforms other than iOS, but its certainly possible that they did (I'd argue even probable).
1. Everything that supports WebP is affected. Not just Chrome and Electron, but all browsers, desktop and mobile, and non-browser software too. All kinds of image viewers, graphics programs, email clients, even your file manager that shows thumbnails.
The bug is in the codec library, and WebP has implementation monoculture, so everyone uses the same library, and everyone needs to patch.
3. Google tried to make WebP a thing 10 years ago, but it didn't get much traction, since it was Chrome-only for a long time. It never got properly standardized (it is open source tho). It compresses low-quality images better than JPEG, but tends to blur and smear colors in higher-quality images.
Ironically WebP became widely supported at the same time when it became technically obsoleted by AVIF and JPEG XL.
Desktop Linux is on the relatively safer side, just because so much of the open source ecosystem still uses dynamic linking so it just needs your distro to package a new version of libwebp.
All the proprietary software with their own bundled versions of electron or vendored libraries, etc. on the other hand...
> so it just needs your distro to package a new version of libwebp.
That and every snap/flatpack/etc. package, every container image you are using and possibly pip packages that can come with and compile all kinds of dependencies and haven't been maintained for ten years...
The security benefit a well maintained Linux distro provides has been eroding for years now.
However, you can choose to largely avoid these. Yes, people are pushing the other way, but you can not use snap and flatpack of you use a distro with large repos. You can use Python virtualenvs with --system-site packages and put just pure python packages in your requirements.txt. You can run things in containers for security without using images.
I think there are two problems:
1. people running single/small numbers of servers copying practices that are used by people running fleets of containers who can have someone promptly updating everything has needed.
2. As always, convenience. The easiest and best supported way to pip install things is without --system-site-packages.
I have always felt we were going the wrong way with this. I thought I was the only one!
What do you mean? Dolphin displays thumbnails perfectly fine, and I love how easy it is to change the thumbnail size (ctrl-mousewheel or the slider at the bottom).
I don't think I've ever seen that in the past several years of using Dolphin. The context here being webp, I checked the many webp files I have, and Dolphin thumbnails all of them without any issue.
Yeah, but once you fix the library the system should be safe. Well, except for all of the snaps and docker containers and whatnot. Those will need to be updated as well.
I ignore them without problem, I do most of my web reading on my iPad that does not support webp and most sites display images without issues. Usually the ones that only offer images in webp are low quality sites and it’s usually a good sign I should just bounce.
You've almost certainly downloaded WebP images, but you didn't realize it because many websites serve multiple formats at the same URL. This is often done with a HTTP reverse proxy that automatically converts between formats so if you have a modern (last decade) browser, you'll get a WebP even if the download's file extension seems to be ".png". All modern image editors support WebP so you'll never notice the difference
On The Guardian for example it looks like I'm getting WebP for the photo collections and AVIF for the story images, both from .jpg URLs (in Firefox hit Ctl-I to see image info). Their photo galleries have been WebP for years. Firefox is clever enough to correct the suffix if you try to save it.
>All modern image editors support WebP so you'll never notice the difference
This support has always been overstated. Photoshop still doesn’t support it, you encounter other weirdness like Apple Preview.app can view them but not edit them.
Preview.app is the exception to the rule, but it still has acceptable UX. After you finish drawing the first brush stroke on a WebP file, it will ask you to reopen the edited file as a tiff before you continue editing. No loss of data or functionality
> 3. Why haven't I heard of WebP before? Am I living under a rock, or is this a mobile-first technology?
It's been supported in desktop chrome for a long while. There's dozens of JPEG replacements that have come and gone, and WebP is primarily notable for having Google's clout behind it. When Google bought Duck/On2 they got a lot of video compression technology, some of which went into WebP.
Oh, it's definitely something that makes/made sense for them to do; just pointing out that not knowing one of many JPEG replacements doesn't mean you live under a rock.
s/JPEG/PNG/ and my comment still works. In any event it's clearly designed for both, since the lossy and lossless modes are relatively independent of each other.
Every browser is affected, Firefox issued security update as well.
WebP is gaining popularity as replacement for JPEG, I'm surprised you haven't stumbled it on yet, cause more and more images I download from the web turn out to be WebP.
Many CDNs use it automatically. They detect you're on a modern browser and transparently compress sub-optimal images like PNGs without loss of quality.
I did find that `ffmpeg -i file.png -lossless 1 test.webp` could produce a full-res chroma image, recognized as WebP by file and opening successfully in Chrome and Firefox. I was under the impression this was not possible, but I suppose it is (today, not sure in the past).
Why do I see WebP as an image format used to sneakily degrade PNG files? I've seen gaming wikis and CDNs serve PNG URLs as lossy WebP, ruining pixel art and degrading color detail in 2D art. And Discord CDN's "file.webp?size=1024&quality=lossless" serves icons/emotes with chroma subsampling (and ffprobe doesn't say the file is lossless, unlike test.webp above).
Dunno about ffmpeg, but the official library supported lossless encoding a decade ago
> Why do I see WebP as an image format used to sneakily degrade PNG files?
Why does Twitter reencode PNG to JPEG even when this results in larger file sizes and terrible quality for 2D art and technical drawings? All the services you've listed are free and they all cater to the lowest common denominator. They'll never use lossless WebP by default for the same reason they won't use PNG. Lossless media is virtually unheard-of outside our bubbles. If you're lucky there will be a "download original" button
Well, we don't know for sure that an exploit exists for this bug on Android yet (the original exploit was for iOS via iMessage), but there's a reasonably high chance that one has been developed already. These types of exploits are in very high demand for Android right now, I've heard some eye-watering prices being mentioned recently.
Updating Chrome on an unsupported device would fix the issue, but you would still need an Android OS upgrade to fix the issue for apps like Signal and WhatsApp. Chrome bundles its own version of libwebp, but messaging apps and other highly exposed stuff like Gmail all use the OS provided interfaces for displaying images. Hopefully we'll start getting updates for security-supported Android devices in early October.
This is honestly pretty bad.
You either wait a month (or more) for the OS security update, if you are lucky enough that your device is supported.
Or the app could bundle the library, they can push updates faster but then again they could just use an old version and never update like most 3rd party dependencies these days.
This is the primary reason why I push the the non technically inclined (grandparents, older friends, etc) in my life to use iPhones. The latest Pixel I believe is now offering 6 years security patched (which makes it an Android leader), but 7+ years is already the standard on the cheapest iPhone option.
The argument of a $1000 Androd phone vs a $1500 iPhone falls over if you have to replace that Android phone 2 more times in the same period.
I have iOS devices from 2015 that still get security updates and already patched for this issue. It's really just straight up irresponsible at this point that Android can't actually do this.
You’re saying that Google knew about the bug and purposely dropped security support for the pixel 4a right before? Support didn’t age out, like it typically does (age of device)? Google just pulled security support for this one device?
No, I’m saying it’s extremely frustrating that Google just drops security support entirely for devices like this, when they could continue to at least patch egregious platform bugs like this one, even if they can’t fix everything.
It’s just icing on the proverbial cake that a month after they drop security support for the 4a a 0-click remote exploit is found.
The only publicly posted price list that I know of is zerodium’s (evil people). http://zerodium.com/program.html They currently offer 2.5 million for an android zero click with persistence. This doesn’t give you the persistence piece without another bug so maybe 2m. Of course, they are only willing to offer that price if they could sell it for much more.
The platform image decoding libraries really ought to be one of the system modules that are updated through the Play Store, can anyone confirm whether they are or not?
I certainly hope it is, but even still Project Mainline only came out with Android 10, and as of May 30, 2023, 27.8% of Android users are on Android 9 or lower, according to Google's stats listed in Android Studio. [1]
Considering there are over 3 billion active Android users according to Google [2], that's at least 834 million people who are potentially vulnerable to this exploit that will likely never receive a patch. That's awful enough by itself, but particularly terrible since the exploit could be as simple as having someone view an image. Chrome may be fixed, but there are over 2 billion WhatsApp users. [3]
“Native image decoder - New NDK APIs let apps decode and encode images (such as JPEG, PNG, WebP) from native code for graphics or post processing, while retaining a smaller APK size since you don’t need to bundle an external library. The native decoder also takes advantage of Android’s process for ongoing platform security updates. See the NDK sample code for examples.”
However, poking about in the Android mainline module docs I can find no evidence that the image decoder libraries are included in the system updates that get pushed through Google Play. I’ve unpacked the obvious modules & found the video codecs, but no sign whatsoever of the image decoder libraries,
It would be helpful to be able to tell whether a given phone has received such an update, especially in the case of 0-click exploits like this one! Are all Android 11+ phones guaranteed to receive it? (it would seem so, if this is a core security component) Is there anyway to tell whether you’ve received it? This aspect of Android updates seems to be entirely opaque to the end user - updates get pushed through Google Play services & that’s it.
which gives some details of Android system updates pushed through Google Play. This changelog does not list CVEs or specific bugfixes, so it’s impossible to tell which bugs have been closed via this route, but it is at least slightly better than nothing at all.
It is possible to map those updates to actual CVEs mentioned in the AOSP sources via a circuitous route involving adb & running a bunch of commands on your phone, according to this blog post: https://www.esper.io/blog/building-a-google-play-system-upda...
Looking at the current AOSP wepb sources, no module tags have been attached to the current release, so it doesn’t look like it’s been pushed out yet, if it’s possible for Google to do so at all outside the monthly security updates for in-support phones.
I'd really rather it was possible for me to update vulnerable image decoding libraries than having to install a Play service and give it root access to do that for me
Some messaging apps have options to not automatically download/preview images that are sent, so that might be a good step to take. I know that Google Messages has this feature.
If your android is out of security updates, this won’t be the only thing to worry about, I would probably worry about the easier to exploit ones first.
Agreed, I found infuriating that after 2ish years your android is no longer secure and you are on your own, and you have two options here, throw it away and get a new one that will be thrown away soon, or go in the efforts to find a hacky way to install a custom ROM that hopefully the maintainers will keep it secure. It’s that reason why I switched to iOS years ago.
They're currently bugging me about updating a 9 year old ipad I just use as a kindle. They flatly blow Google away on device updates, and will continue to do so even if Google isn't lying about their new / supposed 5 year update plan (and NB: that's 5 years from the release date, not 5 years from the last sale date, which could be reasonable.)
iMessage is sandboxed, and does attachment processing in yet another sandbox (the exploit is named BLASTpass as it circumvents this). These days iMessage itself is probably more tightly sandboxed than other apps.
It sounds like the exploit was triggered in another process by iMessage forwarding the post-parse attachment content to that process - the blog post says this is passkit related so I assume any app that has passkit interaction could do this. iMessage is simply universally available so why use a different medium.
>In practice there are many such inputs that will overflow huffman_tables
This looks like the generalized version of the problem...
In other words, you have Software A, it generates a lookup table B which is then used to process an input stream of data.
Now the responsibility shifts to you as a software developer (if you care even a little bit about security/correctness of your code) -- to either assert that:
A) The software is written in such a way that there are NO possible cases of input data misusing/failing the lookup table, or
B) The software will only be used in a controlled environment (i.e., point to point communication where both communicants are trusted) such that the stream is guaranteed never to contain data that misuses/fails/causes anomalies with the lookup table.
Since B is all-but-impossible for anything other than a small group or office, that is, truly impossible on the Internet scale, that leaves only A).
Thus, the generalized "best practice" for present or future Software Engineering, can be summarized as follows:
If a lookup table is used in someone's software for whatever reasons -- then then the responsibility goes to the software developer(s) to assert that that lookup table functions correctly and for all types or data, OR that the software detects and appropriately handles erroneous data BEFORE it gets to the lookup table...
In fact, if I were a serious security researcher and had the time -- I'd collect a list of ALL reported security vulnerabilities in the past that had to do, one way or another with lookup tables...
Then I'd read through them, one by one, and compare them for generalities.
I'm guessing (but not knowing) -- that there is a pattern there...
Then I'd go through all software that used lookup tables on streams of data in one way or another -- and audit ALL of them for security vulnerabilities.
Now, clearly this is not a task for one man in one lifetime...
This is a "team sport"...
But if I were a serious security vulnerability researcher -- that is the generalized path that I would take...
To what extent did the source code enable NSO to find this bug? Had the code been blob-only, would modern decompilers have worked well enough to help them understand the code and find this obscure bug?
Of course having source is more convenient, but not as much as it may seem. The attacker is looking for something the source did not intend to do, so they're already "reading between the lines".
Many memory safety bugs can be found by fuzzing code as a black box. Fuzzing is used by code authors too, because even people who wrote the source code don't fully understand what edge cases exist based on the source code! If the code has Undefined Behavior bugs, then the source code may not even match what the actual program does.
There are good decompilers, and as I've mentioned, writing an exploit will necessary depend on working with compiled code — you must know what's in memory, on the stack, and where "gadgets" are the exploit can jump to. This information is not present in the source code. Deep understanding of compiled code is a prerequisite for writing an exploit.
Bugs have been found in closed-source Windows for as long as it existed. Even the recent attack on Apple Messages combined this bug with a bug in Apple's closed-source sandbox. Security by obscurity has always been tempting, and never worked as well as hoped.
It's surprising that this isn't in a "newer" part of the image format; Huffman compression has been around for over 70 years, Canonical Huffman for a few decades less, but even JPEG uses Huffman. This is a decades-old technology that should've had the bugs worked out of its many implementations by now and there are also countless articles about how to implement it.
The important information is in section 6. The first thing I notice is that it's not very clear how the codes are constructed, unlike the JPEG one (which actually has a ton of very readable flowcharts on the process), but it appears to be similar to LZH/deflate(zlib). The "specification" looks more like a selected set of source code fragments with accompanying descriptions.
Perhaps I should try writing a WebP decoder too, having already done GIF, JPEG, and PNG, but based on the above "specification", it's almost as if they don't want you to.
> it's not very clear how the codes are constructed
I agree the specification really lacks examples, but it explicitly states that it uses canonical Huffman trees so that only code lengths have to be transmitted. I think this is clear enough to pinpoint the actual tree. (I don't think there is any canonical Huffman tree implementation that uses the inverse lexicographical order.)
> This is a decades-old technology that should've had the bugs worked out of its many implementations by now and there are also countless articles about how to implement it.
Because there are many implementations of Huffman trees with different trade-offs? Charles Bloom once said that the definitive 1997 paper on Huffman optimizations [1] is still not well known at that point (2010) and many optimizations were rediscovered and then forgotten, so there should be many inefficient implementations out there.
One can avoid the most egregious security concerns (rce) if software vendors use slightly slower libraries to render their images. Avoid libraries written in C. It'll almost eliminate all rce and your users will be safer because of it.
Can't see that happening any time soon, browsers/users love render speed.
If one is concerned about this as an end user, I've seen some extensions that block webp and try to request a png/jpg/etc. version from the host.
I can't attest to how effective it is as I didn't use it long. But it worked with some of the big image hosting sites like imgur.
For me, this was just so I was able to download images in a usable format. Most OSs can't treat webp like normal images, like generating thumbnails or opening a preview app.
That was a few years ago though so maybe things have changed.
I might be making stuff up here but I do remember the same happening to I think png and jpg at least once, to some degree.
It just sounds like typical growing pains from using non-safe languages (C).
I get the appeal for browser speeds, but I really wish we as an industry could move away from methods that encourage the same mistakes we've been making since we started writing in C.
It feels like we're using self tapping screws to build a bridge instead of rivets because it's faster. And we can just keep adding more screws if the bridge starts to sag.
> The good news is that Apple and Chrome did an amazing job at responding to this issue with the urgency that it deserves
Excuse me? It is Google that assigned this as Chrome only. Over the last 7 days alone every single major Linux distribution has had to push an update (including Red Hat which assigned this a 9.6 score), and Docker images like Python which has over 1 billion pulls, not to mention Puppeteer(hello?), WordPress, Node.js, etc. and CRBug is still private to this day.
I am not being condescending but sites like BleepingComputer reported this as they saw it rather than doing any investigation. And the same goes for a lot of security companies that reported on this issue in third person. It’s really difficult to foster trust when you know that the person on the other side hasn’t bothered to do any due diligence.
Adam Caudill (1Password was one of the first to patch it) did a nice blog post, “Whose CVE is it anyway?”[0] highlighting the issue I am talking about in my comment.
Citizen Lab has refused to comment on whether both are related, but it doesn’t take a genius does it…
Apple and Chrome specifically matter here because those where the targets being exploited in the wild, and have the most direct attack surface with the largest number of users.
The author mentions that many other systems need to patch as well. However, wow many of those billion Python docker pulls are rendering untrusted WebP images? Same for Node, etc. These should also be promptly patched, but they're not in the same ballpark here as iOS/Android/Chrome.
Millions of people in the world are affected by this library, 10 times over for every devices and apps they use.
I'm sorry but I call for libraries used by millions around the world to NOT use C. And I love C... But this risk ratio is off the charts and they ought to not use C for such critical libraries.
Even as a C guru, you are going to make a mistake, at some point.
Same here. Rust should be the "new C". Because you can code in C doesn't mean you should produce large quantities of complex code in it for "performance", "portability", or "legacy compatibility" "reasons".
C/C++ as well as dynamic languages create huge surfaces of undefined behavior and subtle bugs that are too difficult to lint and too burdensome for even the most astute coders.
Fundamental libraries should also be formally verified in a manner similar to seL4.
Also, another problem is a pervasive attitude of unprofessionalism and dismissiveness of rigor, quality, correctness, and security in FOSS. The current approach of building empires on quicksand is foolish.
I'm surprised the summary of the article only talked about over-reliance on fuzzing and then suggested 1) more thorough code reviews and 2) sandboxing as solutions?! To me, the solution lies in using memory-safe languages.
I think sandboxing is the more powerful solution. You think in terms of "What privileges can the attacker gain if this code blows up?" and limit the code's privileges to the minimum.
Problem is, sandboxing is harder to implement so it's often done suboptimally or not at all.
Yes and Google Chrome has invested heavily in sandboxing and still had to ship this as a high-priority fix. I'd say sandboxing in conjunction with memory-safe languages is the future.
The problem is that rewriting existing code into a memory-safe language is a huge investment -- and realistically the world depends on a lot of code built over many decades that cannot be rewritten overnight. Consider that Mozilla created Rust specifically so they could rewrite their browser in it, yet still only a small fraction of Firefox code is Rust today -- much more is still C/C++. Realistically we're going to have a lot of heavily used C/C++ code forever. The singularity will come before we can replace it all.
The nice thing about sandboxing and fuzzing can be applied to existing code.
Yes, but sandboxing and fuzzing are insufficient. As pointed out in the article Google had been fuzzing this library and it didn't find the issue. They even tweaked the fuzzing after this issue was found to specifically target the area of the vulnerability and it apparently still didn't trigger the issue.
Google Chrome also implements sandboxing and many areas. It's not feasible everywhere. So for new code / libraries we should default to a memory-safe language.
I think almost everyone agrees new greenfield projects should not choose C/C++, now that Rust has matured enough and covers essentially the same use cases.
But realistically that only solves a tiny fraction of the problem, since realistically new greenfield projects started today will likely take 10+ years to become widely used, if they do at all.
WebP was written at a time when C/C++ was still the only viable language in which to write an image compression library. Saying "things like this should be written in Rust!" just doesn't actually do anything to make software like WebP secure. Improving fuzzers and sandboxing might.
> realistically new greenfield projects started today will likely take 10+ years to become widely used, if they do at all.
I'm pretty sure that even now a lot speaks against using rust for greenfield projects precisely for that reason: few people want to integrate Rust into their build chain. You basically always have to have a compiler that's 1 release old or otherwise you cannot compile new Rust software like the very widely used time crate.
If you are a new and unproven format, do you really want to bear that hit? You could make two implementations, one in C and one in Rust, but that will mean you spread your probably quite scarce engineers over two projects.
Heh. Well, speaking as the lead engineer of a large C++ systems project that sadly started a liiiiitle bit to early to use Rust (the Cloudflare Workers runtime), I'd say our attitude is the other way around: We basically refuse to bring in any kind of C/C++ dependency unless it's something used in Chrome (implying they've vetted it). We are much more willing to bring in Rust dependencies, even though they're harder for us to invoke due to the language barrier.
That's a really good policy, and what I said above does not reflect my private opinion but the opinions of a lot of people that decoder projects target. You are in a way more controlled environment.
For dav1d I heard that one of the reasons why C was chosen was precisely this portability concern. format decoders are usually quite low level components and can get integrated into all sorts of environments. I'm sure there is someone out there who made a visual studio project with a 3 year old VS version with dav1d inside for some embedded project. For such devs, Rust is a way harder sell.
That's great, but the fact that the author of this article didn't mention memory safe languages is disturbing and an indication that not everyone is aware!
You can write your implementation in Rust and expose a C ABI, and anyone using your library doesn't have to know or care that the internals use Rust. That's pretty much the whole point of Rust, otherwise you'd just use OCaml.
You basically always have to have a compiler that's 1 release
old or otherwise you cannot compile new Rust software like the
very widely used time crate.
That's an interesting example as the time crate stagnated for quite a while and even the current version only requires Rust 1.67 or newer. The current Rust version is 1.72.
A rewrite of a webp decoder into Rust already exists (image-rs has a decoder for webp). I'm sure there is some polishing needed but it's nothing a company the size of Google can't afford.
That's for the encoding part. The 0day we are talking about was in the webp decoder, and decoders generally are the riskier component compared to encoders. I'm not sure if Chrome ships with the webp encoder at all.
Mozilla has an interesting sandboxing strategy: They compile some of the C/C++ parts of Firefox (e.g. the ogg parser) to Wasm and then back to C. Because Wasm is memory safe, the resulting C is too.
Second problem: sandbox aren't perfect either. It's indeed useful, as part of a defense-in-depth approach, but it's far from sufficient.
Memory safety could solve the problem altogether, but then again no program is 100% memory safe, there's always some kind of primitive that uses memory-unsafe code under the hood, so it's not perfect either.
The “perfect” solution would probably be:
- use memory safe languages
- all primitives using memory unsafe stuff should get formally verified
Rust is kind of aiming at this (with things like [1] and [2]), but it's not there yet.
Strong type systems can give provably correct code. For trusted code (e.g. not third party code), sandboxing is a post-exploit mitigation. And such a post-exploit mitigation cannot necessarily guard against any class of bugs that (at least in some aspect) provably correct code can.
Yes, of course privilege separation as much as possible is still extremely valuable, but to say that sandboxing is a "better" solution, implying that one should not pursue provable correct code in favor of post-exploit mitigation, is a harsh liability. It's the same as the "oh, we don't need to use a type safe language, we have unit tests"-crowd, only worse.
> Strong type systems can give provably correct code.
"Sound" [1] type systems only guarantee the absence of some class of bugs as well. There are a lot of bug classes that remain exploitable. Memory safety happens to be a low-hanging fruit because many existing softwares are not even written in such languages.
[1] "Strong" type systems generally refer to the intolerance towards implicit type conversions or memory unsafety, and that alone doesn't make type systems provably safe in some sense.
I did not claim that everything about the code was proven correct, but a subset of properties expressable by the type system.
I agree that using the word “strong” was wrong. I basically meant it in the sense of “good”/“elaborate”, mistakenly ignoring that “strong” already has a very specific meaning in type systems. Thanks for correcting.
Yeah, anything can be implemented incorrectly. My point was specifically that ignoring memory safe languages in favor of post-exploit mitigation is foolish, not that the latter is useless.
But in practice, type systems can be proven to be sound, implementations of type checkers can be proven correct, and while it’s still possible to make mistakes there, that isn’t so much an issue in mathematics, simply because of how rigorous it is. This does not just include type systems, there’s even an effort to rebase the foundation of mathematics on a type system instead of set theory!
In other words, we likely agree that steel vaults can have holes. But we probably also agree that an average steel vault is better in keeping things in or out of it than a velvet curtain.
> In theory, perhaps. In practice, the compiler / runtime will have bugs.
They probably won't. The trusted kernel for these systems is tiny; a sandbox is orders of magnitude more complex with orders of magnitude more chances for bugs to creep in.
What does provably correct mean here? I think you mean that the code doesn’t have any memory corruption vulnerabilities. However, that is only one class of vulnerability, so more techniques then just relying on a memory safe language are required for secure software.
It means that the type system can prove certain properties for you. For example, in languages with dependent type systems like Agda, you can construct a sorted list type that the compiler will prove it sorted at all times, otherwise it won’t compile. Or a complex tree type that is always balanced. Or a set of only even numbers, and again it won’t compile otherwise…
(Sadly, if you go that far, it isn’t generally Turing complete anymore. Though in some cases that’s a good thing.)
Using memory safe languages may help, but the core issue lies on the lack of understanding of the program as a whole. Hence, the over reliance on fuzzing.
Debating what the "core issue" is may be an interesting philosophical debate, but at the end of the day, using a memory-safe language would have prevented this issue from being exploitable.
I understand, really.
But much of the discussion is already on that topic. It's not that I don't consider it unimportant.
I'm just trying to point out a bigger issue with software, in my opinion. Even more so when talking about the standard implementation of an image format as big as this. It really puts a lot into perspective on how bad software is commonly designed.
Luckily type safe languages, which includes memory safe languages, help you in understanding the program as a whole better, and strictly prevent you from doing things against that understanding, because the types literally encode automatically proven properties of your code.
Not all memory safe languages are type safe. I'd argue that expressive ML-style typing is if anything more important than memory safety, although it's hard to tell since a type-safe language will almost always have to be memory-safe.
I never claimed all memory safe languages are type safe. But memory safety is a subset of type safety. Whether dynamic or static (though I’m much in the latter camp), memory safety is achieved through information associated with the types. Whether that’s exposed to the surface or not: Rust exposes it and is static, Haskell normally does not expose it and is also static, most modern dynamic languages are memory safe but don’t expose the associated information.
> But memory safety is a subset of type safety. Whether dynamic or static (though I’m much in the latter camp), memory safety is achieved through information associated with the types.
Nah. Dynamic languages don't actually have types (even if they have something that they call types), and even typed-but-GCed languages often don't really use the types for memory safety.
They have value types. The expressions don’t have types (or rather very general types), but the values do, and it’s still possible to have a dynamic language with very rigorous value types that have a well-defined set of inhabitants.
Any dynamic language that has a string type has something similar to (for example) a buffer and a length associated with it internally. You can formalize that from the compiler’s perspective, even if you don’t expose it to the outside.
You could argue that the language doesn’t necessarily have memory safety associated with its types, because a compliant compiler or interpreter could represent strings in any way it chooses, and on some academic level there’s merit to that, but in practice you’d be rather stupid to implement the string value type in an interpreter or compiler for a language with a common string type in a memory unsafe way.
At least theoretically one could have a dynamic language with e.g. C-style malloc/free manual memory management, although I haven't actually seen such a thing in practice.
And yet another argument for "You ought to be using Qubes." Random web access needs to be treated as "Genuinely high risk" anymore. A disposable VM with nothing of value in it for "casual web use" seems the right option for exploring the security hostile environment of "the internet."
Sure, that might help you on your desktop. But this exploit was only discovered because of a zero-click iOS exploit, where the payload was encoded in an iMessage attachment that opened in Passkit, which rendered the WebP file. (In fact the user didn't even need to "open" the attachment - just receiving the message was sufficient to trigger the exploit.)
So Qubes won't help you everywhere, and the WebP decoder is everywhere.
Unless you are making a vm per page refresh, I can't really see how a browser in the VM is any safer than a browser outside the VM.
My most valuable stuff (passwords, bank accounts, logins) is accessible from the browser. You'd need to somehow sandbox and frequently destroy/restore the rendering and javascript engine to avoid leaking this information cross site while having a fairly strict firewall between those and the external browser. (IE: cookie/session/password storage).
It's "A range of VMs, with different browsers in them, for different purposes."
My "random web use" browser VMs don't have anything in them - they're ephemeral. If I need a password, I copy it from another VM over. If you escape into that VM, you might be able to grab a password being pasted, but I don't access anything I consider sensitive in them - just random forum accounts, etc. And it's easy enough to spin up other disposable VMs for stuff in Qubes (I actually mostly browse through the Tor network, to add traffic to it).
So, for your use case, you'd have one VM with your "core" stuff - passwords, logged in to webmail, banking. And then you do everything else web related in a different VM.
I mean your mother is successfully using Apple's highly sophisticated sandboxes and things like authenticated pointers. (sHe's On AnDrOiD)
For Qubes specifically she can understand theres a bank Qube and a Facebook Qube and a nothing-special Qubes and its all color coordinated. She doesn't even need to know they are VMs, they're just color coded windows
As a qubes user myself, I don't think my mother would be able to perform the initial setup and segregation, not to mention endure the hurdles of copy/paste rules and filesharing across VMs. Oh sure, she could learn another set of copy paste keyboard shortcuts that will end up as another handwritten note on her list of keyboard shortcuts. Not to mention how frequently I'd get called due to slowness as she neglects to close the VMs. And say goodbye to remote support options unless she gives me SSH to dom0 but then why do any of this anyway at that point?
My bet is that over time most people would slip on the separation and end up with tons of tabs in the wrong containers. The only way it'd work in the long term is if there were some way of automatically taking link clicks in the "wrong" container, thoroughly sanitizing them, and then tossing them over the "right" container but then you're back to risking cross-contamination.
This is a fun idea, are there any browsers that hide the VM from end users so it looks and feels like a browser instance but is actually a tunnel into a sandboxed VM that’s being painted to?
That's kinda-sorta what they all do already. Not full OS-level VM abstraction, but surprisingly close to it. Exploits like this need to be paired with sandbox-escaping in order to do damage beyond the current browsing session (which VMs wouldn't help with in the first place). And the distinction between sandbox-escaping and VM-escaping is rather thin.
Yes, I should have added that I'm referring specifically to the scenario OP is suggesting, which would require a host <-> client IPC channel, opening up the VM to similar attack vectors to a sandbox.
There's a huge difference. Browser sandboxes are not "real" VMs and share a kernel. And in case of Chromium it's enough to read a few bytes from another process (token) to escape.
The problem is that browsers are the biggest attack vectors but also the most valuable targets. Getting a user's email password or cookie is probably the most damaging thing they could get unless you're the type to buy cryptocurrencies.
Not your bank? Email and bank login should be sufficient to change mfa and other settings and lock you out long enough to have forged checks drawn and cashed against your account.
Browsers already have sandboxes, they're not perfect but neither are VMs, and if you're going for a layer of security through obscurity by having an unusual setup (which is bad on its own but when layered with real security can be fine) then just running Linux is probably enough already
Tools should work for people, not the other way around. The cognitive load of manually managing sandboxes is huge. Instead we could build tools that aren’t fundamentally broken. Memory safety should be formally guaranteed before shipping.
Funny, because for ages I was running macOS on a 2011 MacBook Pro which had a version of Safari that was so old it didn't support webp. At the same time, my Amiga 3000 running AmigaDOS 3.2.2 could support webp by virtue of an operating system-wide datatype plugin for webp.
Updating the OS-wide datatype means all apps are updated, not just the browser. Why is this STILL not the case on supposedly modern OSes these days?
The reason is most likely security and stability. OS vendors don't want every application to be potentially vulnerable or unstable, because user installed some dubious codec pack.
One place where macOS allows arbitrary codecs is Quicklook plugins, but these are designed to run in a separate process. It'd be wise to implement image codecs the same way, but so far they're typically a library linked in the same process.
I don't really want this to be the case. Remember when Firefox accidentally exposed all gstreamer plugins to the web?
Having some insecure unaudited badly written codec is fine for a lot of use cases because it operates on trusted data. But this is of course quite a different situation than being exposed to all webpages you visit.
The Amiga 3000 lived in a different world with a different internet.
Especially infuriating because MacOS has the brilliant generic file viewing framework that they brand as Quick Look Plugins (QLPlugin). The name comes from where the feature started (pressing spacebar in the file viewer to ostensibly view literally any file) but it is used all over the place in Mac and iOS software.
IIRC, in the early days Safari would pass off any image type to the OS, but after some TIFF vulnerability they whitelisted it to supported "web" formats.
They also used to let you just embed a Quartz Composer file which let you do some "fun" stuff with the webcam that would terrify users.
The gigantic “Subscribe” button on this page is infuriating and highly distracting. It effectively covers 1/3 to 1/2 of the page if you scroll once a paragraph like me and many others.
It feels like this bug has only really flown down under the radar because the discoverers are not in the habit of giving bugs names and landing pages. Both because of level of access (remote code execution), the vector (image rendering, often done with untrusted data) and the widespread nature of the affected library.
I would argue that it has flown under the radar because it has only been contextualized with respect to Chrome and iOS. The issue has and continues to affect many other critical places, including server-side image processing services.
Compared to the likes of log4shell, shellshock or heartbleed, yes. It feels like the immediately exploit possibility of it is arguably more than heartbleed, but I don't see every security person chasing after it in the same way.
I've been following the progress of some of the fixes in apps I use and it's meandering through intermediates at an urgency that is more akin to the ssh 9.1p1 vulnerability which required peopel to ssh into an affected server.
It's nothing close to heartbleed which was 'extract key material from every TLS-serving endpoint in the universe'. There are almost certainly exploitable buffer overflows in whatever device you're using right now.
BLASTPASS is an ok exploit name, but it is kinda specific. People might think it was only about bypassing BlastDoor on iPhones. A better name might have been something like "WebPwn", which would have made it much more clear that it was a vulnerability in the image format.
Ubuntu and Fedora have put out security updates for libwebp, so any program that uses the shared library to access WEBP images is safe, once security updates are applied. Not sure about other Linux distros, but I expect that almost everyone has dealt with this. Unfortunately, snap and flatpak applications are a problem; it's hard to tell which ones might have a bundled, vulnerable WEBP decoder linked in.
Not really “any” program but those that don’t ship their own version and didn’t statically link against libwebp (if that’s an option?). Notably, browsers often don’t use the system version of the library. That’s not the end of the world since browser vendors are on top of the security game for the most part - but the million Electron apps you might be running also each need to be updated individually!
Basically, don’t rest easy and get complacent. Updating the OS distros is not enough.
I've just had a look into my flatpak installation, where my default "Platform" (runtime) is still vulnerable. That didn't even seem too much of a worry though since as I found out all of the flatpak apps I use that have a dependency ship their own libwebp. And sure enough, except for Firefox which is patched, none of them is.
$ flatpak update --no-related --force remove
Checking for updates...
Nothing to do.
My base system (Debian) got the patch almost two weeks ago. Might as well trash flatpak for good.
Yes, the main benefit of WEBP is for lossless compression, it's reported to be about 25% better, though this would depend on the image. For lossy, there's no reason I can see to prefer it over JPEG.
But you don't have to add extensions to convert this crap. You do that to yourself. If you quit making your life hard it would be so much easier, but probably less complaining- so i guess that's the trade off.
You do have to handle it though, and rarely when you want to. You click on a jpg link in your browser and save it, but the CDN invisibly converted it to webp. Which the program you want to edit the image in doesn't support. I didn't ask for webp. I didn't try to make this difficult.
Timsort is an ingenious hybrid sorting algorithm originated from CPython and many implementations including OpenJDK adopted it mostly via a source-by-source translation. Timsort particularly maintains a stack of sorted runs, and due to the construction there is a small enough finite limit in the maximum possible stack size. However the original CPython implementation didn't exactly match what was proven, so there were rare cases where stack overflow could happen. So this was a serious security bug in CPython, but wasn't in OpenJDK because Java instead threw an exception in that case.
Similarly, this WebP bug occurred because the largest table size was formally proven but it didn't match what was fed to the source code. This kind of bugs is not only hard to verify but also hard to review, because of course there is a proof and the source code seems to match the proof, so it should be okay! This bug suggests strong needs for approachable formal verification, including the use of memory-safe languages (type systems can be regarded as a weak form of formal verification), not human reviews.
[1] http://envisage-project.eu/wp-content/uploads/2015/02/sortin...