Yeah, I don't think anyone has grasped the extent of how dangerous this vuln is -- was it released a little prematurely? is it still in "embargo"?
This is hundreds of times worse than heartbleed in terms of scope/attack surface for modern servers... (I say hundreds of times worse because heartbleed was scrape-some-data-till-you-get-private-keys-and-watch-communication, where this is just get-yourself-a-shell-and-pwn-the-machine)
While this is generally true of newer Debian installations, older machines that have been been regularly updated over the years to stable may still have /bin/sh pointing to bash. I fixed a few of my machines yesterday to use dash.
dash changelog.Debian.gz entry from July 22nd, 2009:
* Change the default for the system shell to dash.
* Ship /bin/sh in the package and fix the diversion handling
for it to make sure /bin/sh is always present.
* Set debconf priority to high when upgrading from an existing
system.
Unless the admin has--for some bizarre reason--set debconf to critical; during one of those "regular updates" the admin would have been presented with the following question:
The system shell is the default command interpreter for
shell scripts.
Using dash as the system shell will improve the
system's overall performance. It does not alter the shell
presented to interactive users.
Use dash as the default system shell (/bin/sh)?
<Yes> <No>
Does this change default to "yes" on unattended installs? I'd assume it doesn't, because that would be dangerous.
EDIT: Indeed, from the same changelog: debian/dash.NEWS.Debian: when upgrading existing installations, the system shell will not be changed automatically (closes:#539363).
Just so we are clear you would be talking about an unattended dist-upgrade from lenny to squeeze? Have you done this?
Please explain how your automated Debian administration system that has been in place since Lenny[^1] handles unattended dist-upgrades without the use of preseeding the debconf database for questions with a priority of high or critical?
[^1]: Squeeze was February 2011, any new stable installation since squeeze defaulted to dash as /bin/sh
EDIT: Nevermind, I answered my own question, unattended dist-upgrades from Lenny to squeeze is not something you have any experience with: "I'm now using Debian Wheezy, after primarily using Windows environments my entire life." https://news.ycombinator.com/item?id=6564610#up_6565766
While they sit there, oblivious to the fact that that router thing we use for the interwebs and all those IP cameras, NAS devices, and switches in their office are running linux, and bash, and are about to be used for industrial-grade extortion.
It's unlikely many consumer routers would be running Bash. They'd more likely have Busybox with the bog standard Bourne Shell. And same goes for most other embedded Linux devices too.
Though I'm not suggesting that one shouldn't check their own devices to be safe rather than sure.
Some Netgears do offer telnet, though you often need to enable it via a "magic packet" hack. If you Google your router model number and "telnet" then there should be more information on whether your product supports this and how to go about enabling.
Depends on the router. Cisco is famously runs IOS, which I believe is based on BSD (FreeBSD 2.2 specifically). Barracuda gear, however, is Linux based.
However in all cases, I think you'd still be right about Bash not being present.
IOS-XE and XR are linux boxes, as are the nexus. Arista is linux based, Juniper is FreeBSD and Force10 is NetBSD (at least the older one were). Many devices in the class of enterprise or service provider grade are not going to be obtaining their addresses via DHCP and have a heavily modified version of the OS, anyway. In addition, best practices should be used in armoring the devices in this class, meaning filtering of the control plane, etc.
IOS XR used to be QNX though, if I'm not mistaken?
I hadn't heard of these other Cisco products until yesterday, but from what I read IOS XE and NX OS also run on Linux.
Anyhow, getting back to my original point about FreeBSD, there's a few sources that have linked IOS to FreeBSD. But after doing some digging of my own on this topic, I've come to the conclusion that this is one of those urban legends that's proliferated for whatever reason. So it would seem that you're right about IOS being home rolled. Interestingly though, they do have other products which are FreeBSD powered (eg AsyncOS http://www.cisco.com/c/en/us/products/security/email-securit...) but, as you said, not IOS
While not exactly thrilled, as a Linux user I have to admit it's only fair. That's exactly what some of us did all those years when Windows used to have more holes than Linux. Ah well, back to BSD I guess...
Hmnn well I wonder just how much safer BSD is compared to Linux...
I would consider linux to have more eyes on it, and if I remember correctly, LibreSSL was not infallible (despite all the shaming of OpenSSL folks that went on)
At least many of the things you find on OpenBSD really are much smaller and simpler than commonly used alternatives in the Linux land. Order-of-magnitude differences are common, but even 20k lines vs 60k lines means a lot if you're actually going to dive into the code.
So when I poke around under /usr/src, I find some utility or daemon or whatever else I haven't looked into before. And I think, oh, that's only a couple k lines of code? I wonder how it works... It just invites me to read.
I get the exact opposite reaction when faced with some system that's 200k lines of code. That looks important, maybe I should audit it.. nah, I don't have the time now. Maybe I'll start tomorrow. Tomorrow comes. Maybe I'll start in the weekend. Weekend comes. Maybe I'll start in two weeks because now I'm busy and next week I'm busy too. Two weeks later, chances are I don't even remember. If I do, I might end up promising myself to take a look at it around next Christmas...
Another thing is that OpenBSD moves slower, and instead of constantly adopting another cool new thing as the new replacement for the old thing that kinda worked but nobody wanted to improve, they seem to put more effort into extending and polishing the old thing that has served well. So there's less code churn, i.e. less new code with new bugs.
OpenBSD is actually formally reviewed/audited, not just relying on "many eyes." Of course you have to remember that only the base install is covered; software in packages/ports is not included in that.
It bugs me a bit that "many eyes" took on a security connotation to begin with. That doesn't seem to have been the original claim - it didn't strike me as such when I originally read CatB, and revisiting I can see a way to give it a strong reading but it still doesn't seem to be the sense intended. "Many eyes" in the sense I read it starts once someone notices that there is a bug - for which audit is tremendously better suited than use or casual perusal when it comes to security issues.
My point was that some OpenBSD guys rewrote OpenSSL in an attempt to fix it, yet also introduced some bugs (I read one article about a bug that was introduced by them, though I don't have a link, nor want to search the internet for it)...
And if you pay attention, you'll notice that this overblown bug was only relevant to Linux. Linux simply lacked some functionality OpenBSD has so they tried to hack a solution into the compatibility layer. So the first one or two preview releases turned out not to be perfect.
OpenBSD was never affected.
Though there were a few other unintended OS-agnostic changes that did actually slip in and were subsequently noticed and corrected.
Apple is just about the only company that really benefits from that misconception (possibly by virtue of the fact that there's not really a "linux" company, though Canonical is trying)
I wonder if it would be a reach to say that the average mac user (though when I think about it, a disproportionate amount of those users are probably devs) will hear about this on the news, and assume Apple has their back (which they do, I'm sure they'll force a patch soon if they haven't already) and not even know there's a UNIX system down there
"At present, public disclosure is scheduled for Wednesday, 2014-09-24 14:00 UTC. We do not expect the schedule to change, but we may be forced to revise it."
http://seclists.org/oss-sec/2014/q3/650
As someone who has survived the 1990's scare-a-palooza of Sendmail exploits, and ssh before separation of privileges, and on and on - no, it's not. Not even close.
Is it just me or would it be a good time to learn a bigger lesson from Heartbleed and Shellshock:
Minimalism is seriously a good idea. "Features" are not harmless and cost way more than you think. Providing more flexibility or functionality than absolutely necessary should really be considered and called out as defective, smelly and a bad practice.
Arguably an insistence on stitching together systems out of 'do one thing and one thing only' minimalist components is precisely why Shellshock is so bad, though.
That so many things delegate setting environment variables to the system shell is what allows a vulnerability like this to be so pervasive. Why does a DHCP client pass server-originated data to a full shell? In some ways, it's a form of minimalism.
> That so many things delegate setting environment variables to the system shell
I think you've completely misunderstood the problem. The environment variable isn't and doesn't need to be set by a shell for shellshock to happen.
Why shouldn't a DHCP client pass server-originated data to a shell? Or any other program? There is nothing wrong with passing data. The only problem here is that a specific shell had a bug.
Send me sales people (or anyone for that matter) who want to market and sell "minimalism".
All the ones I know only want to sell "features".
Your words are absolutely 100% true, 1wd.
But I have become cynical that the world of software developers and their best customers (e.g., the ones who love "features") will never, ever follow your advice.
Personally, I prefer minimalism irrespective of the security benefits. But "engineering", a term many HN readers might attribute to their own work, like to speak of "trade-offs". All engineering involves trade-offs.
When you go without "features", sometimes you actually get something in return. What you "get" might not always be obvious. This week, it should be obvious. At least to everyone who chooses a more minimal shell without surplus features.
So... not really any difference, in any sense that is remotely relevant. My point was that the shell interface is a much better UI than it is programming language.
Apparently we disagree. My interaction with a readline interface in a context I understand is neither unplanned nor ill disciplined. You can throw around pejoratives, but clearly you cannot support them.
One thing to recall is that much of the complexity in systemd comes from replacing a whole bunch of dodgy, special purpose code written in shell scripts (init scripts) and in the daemons it manages (daemonization code, various kinds of racy startup dependencies, etc).
Just saying "systemd is complex" is fairly sloppy thinking; the question is, is it more or less complex than re-implementing that functionality poorly and incompatibly several dozen other times in various other daemons and startup scripts?
There's significantly less complex ways to do this sort of thing, though. One that I've been looking at implementing into a distro lately is nosh[0] (and execline[1]). Every bit of code is implemented as a separate, independent utility that can be chained together through the nosh/execlineb "shell", which does no parsing and only a tiny bit of lexing.
This way, it's easy to pick and choose what features and complexity each script needs, and the features that you don't use can't affect you. Nothing is running on your system that you don't know about, and there's no complex "magic" anywhere unless you explicitly ran a command which does magic things.
You seem to be committing Systemd Logical Fallacy #12: "The Linux kernel is complex, and you use the Linux kernel, therefore you are okay with systemd being complex too"
I believe there will be plenty of linux NASs that will be vulnerable for the forseeable future. NASs are usually bigger and more functional than routers, they tend to run a more full system. Many of these for exampe run bash as far as I remember: http://www.amazon.com/s/field-keywords=QNAP
Given the rate at which QNAP issues updates I'm not expecting it to be fixed for at least another month, and that will most likely be a beta release. I like my QNAP NAS but I don't think I'd buy another QNAP product. They are just too unresponsive to these sorts of things.
Is it possible to update bash on the QNAP? I know you can install an SVN server using Optware IPKG but would this work with the updated Bash? These QNAPs do way to many things with barely any software updates. Its a disaster waiting to happen.
There are a bunch of daemons running on a fully configured My Cloud. I haven't had much success in finding anything yet, but yeah, it has the bug.
nas:~# bash --version | head -1
GNU bash, version 4.2.37(1)-release (arm-unknown-linux-gnueabihf)
nas:~# uname -a
Linux nas 3.2.26 #1 SMP Tue Jun 17 15:53:22 PDT 2014 wd-2.2-rel armv7l GNU/Linux
nas:~# env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
vulnerable
this is a test
It would be even better if we could create a community driven script hosted at some trusted location that would basically download info how to upgrade specific distribution, and execute that script on the vulnerable system..
so something like
wget fix && chmod +x fix && ./fix :)
Has anyone tried this with an OS X client to see if it behaves like the Linux system in the article? I know OS X bash is vulnerable to the exploit, but I don't know if their DHCP system handles environment variables in an exploitable way.
I think one of the things Heartbleed PoCs did really well was show the result for the end user. Most people aren't going to know what any of this stuff means. Can we come up with a really straight forward explanation in laymans terms as to what this means for the average Internet user?
Um, "haven't bothered"? Think about all the Linux/Unix-based devices that could be affected here, it's in the millions. Do you run a local server on your box? How about your Linux-based router? Has it been patched yet? Why not, it's been more than 24 hours now.
In the embedded world, people rarely run full GNU userland utils. They're extremely big and bloated, and embedded devicse needs maximum bang for buck. Therefore most of them comes with busybox, which besides being incredibly compact also is 100% unaffected.
Same goes for Android/Linux-based phones. Most don't come with a proper shell at all, and those who do, usually have busybox.
How many of them run bash? I'm about to check my router (I didn't think of it until today), but I read that Tomato runs busybox which I understand is not affected.
Note that this will affect devices without listening services. Embedded devices are very likely to be affected for a long time.
A note for those trying to reproduce the PoC (as I was yesterday) - ISC's DHCP server only sends client-requested options by default, though this can be overridden [1]. tftpd [2], the software used in the PoC, is likely the easiest way to demo the vulnerability.
I reproduced it with dnsmasq as well; just set up a server with the following options. eth2 is the network adapter that I was using for the test network, and I picked the 10.0.10.0/24 prefix for this particular network.
Then on the target system (an Ubuntu system using ifupdown for configuration), I just configured eth0 for dhcp:
iface eth0 inet dhcp
and ran:
sudo ifdown eth0 && sudo ifup eth0
Unlike some of the other exploits, this one affects Debian and Ubuntu. Debian and Ubuntu use dash as /bin/sh, so many things that just use /bin/sh (such as the system() function, lots of shell scripts, etc) aren't affected like they are on other Linux distros.
But dhclient calls dhclient-script to execute various hook scripts, and dhclient-script uses #!/bin/bash and is thus vulnerable.
On my particular Debian system, it doesn't look like NetworkManager based DHCP is affected, as it doesn't call the dhclient-script. It does call various scripts in /etc/network/if-*.d/, but none of the ones that I have installed use bash, they all use sh. However, if I deliberately put a bash script in there, and attach it to the network with the rogue DHCP server, it does get passed the bad environment variable (though on my Debian system I've already updated Bash so I just get the error message rather than the exploit).
So yeah, there are a a lot of ways for a stray Bash script to cause problems here.
Did you try if you can do something else beside the echo on Ubuntu? dhclient runs under an AppArmor profile which tries to keep it in a pretty short leash.
No, I didn't try that, but we happen to be running a custom kernel that doesn't include AppArmor support, as we needed to pull in a newer upstream kernel version. Given this issue, we should probably revisit that decision.
Very true. To be clear to other readers, busybox is more common in smaller embedded devices, though today's AlienVault blog post [1] showed that Mitel VoIP systems (which use Debian/ARM, if I recall) use bash and are vulnerable.
Note that if you try to do the exploit via the hostname or another common option, the CVE-2011-0997 fix will stop you. However, the fix is incomplete - it does not check all options, only the ones the old CVE's fixers thought of.
You could probably exploit it via e.g. USB too. A lot of shell scripts get run when you plug in an USB device - And I'm sure you can cram the exploit into one of the device strings that the host queries for.
Well, yes. If you were going to exploit CGI scripts you'd likely use wget or curl instead of coding an HTTP client from scratch so why is this suprising?
So is the default dhclient conf vulnerable in any of the Linux distributions using bash for /bin/sh (eg CentOS/Fedora)?
edit: According to one poster on Stackexchange, Debian/Ubuntu may be vulerable despite not having bash for /bin/sh: "What's more, on Debian (and possibly many of its myriads of offsprings like Ubuntu), which uses dash as /bin/sh, dhclient-script is explicitely shebanged to /bin/bash, and it does seem to contain a bashism, too" (https://security.stackexchange.com/questions/68156/is-connec...)
What were you using to configure networking on the target machine? NetworkManager, ifupdown, something else?
I found that with ifupdown, you see the issue, as it uses dhclient which in turn calls out to dhclient-script, which is written in bash. If you use NetworkManager, it doesn't call dhclient-script; it does however call the scripts in /etc/network/if-.d, and if any of those are written in bash, then you see the issue. None of the scripts in /etc/network/if-.d on my system used bash, they all used /bin/sh, so on a Debian system where that points to dash instead of bash you're OK.
Of course I've used dhclient for this (also with -r option) with ifconfig up/down. Not working. I've also tried to re-simlink sh to bash. Not working.
I'd like to check py script by mschwager (https://github.com/mschwager/shellshock_poc), may be it will be OK.
dhcpcd-6.4.7 is hot off the press, the main improvement being mitigating the bash "ShellShock" exploit by escaping all characters as noted in IEEE Std 1003.1, 2004 Edition, 2. Shell Command Language, 2.2 Quoting except for the space character.
Needless to say, the entire BSD family is not affected by this bug as bash is not the default shell and to be fair a lot of Linux distributions are not affected either. If bash is your Linux distributions /bin/sh, OR you have applications directly calling bash, you should be telling them to get with the times as most people have since moved on to ash, dash or busybox for more efficient processing.
Regardless, shell is such an important in part of the system - it allows non programmers to "do things". Thanks to the dhcpcd hook system, a user was able to start tcpdump on hotplugged interface before dhcpcd actually started using it during the boot process. Why he wanted to do this, I don't know, probably for some debugging. But the point is, how would he have done this without shell hooks?
The important thing to take away from this is don't lock yourself into one technology - strive to be portable. dhcpcd works on many OS's, libcs, shells and userland tools. If any of them prove faulty, swap them out - including dhcpcd itself! But please at least tell me why you're swapping dhcpcd out so I can improve it :)
This means that DHCP clients that use bash and have DHCP server-controlled environment variables can have commands injected (as root) by a malicious DHCP server.
Notably, attackers in an unhardened network can reply to DHCP clients themselves, even if there's already a DHCP server on the network. So it's not just the sysadmin who can exploit this, but anyone on the same network (broadcast domain) as the vulnerable DHCP client.
why is a DHCP client calling bash for anything ? that's just a terrible way to program. and doubly so for passing unscrubbed data to it as environment variables
The question is, does it call bash in its dhclient-script? I took a look through the FreeBSD source tree, and their version of dhclient-script uses /bin/sh. As long as /bin/sh is not bash (and that script doesn't in turn call any other bash scripts) it should be OK.
The best way to find out if you're vulnerable is by testing. It takes just a few minutes to set up dnsmasq to serve up an exploit. Here were my settings:
Of course, replace that 'echo "hi"' with the exploit of your choice. In my case, the output from dhclient would be printed on screen when restarting networking, so 'echo "hi"' was sufficient to verify that it was being executed.
If any bash scripts are called, with the environment variables that are set by dhclient, then that snippet should be run. If bash is not invoked, then that snippet won't ever run.
NetworkManager's had vulnerabilities over the years ( http://www.cvedetails.com/product/5634/Gnome-Networkmanager.... ), as has systemd. Stating this as a reason to switch is ridiculous. One can easily switch to a much more audited, secure scripting environment, such as ksh, and still have all the power scripting brings.
Not saying I agree with GP, but "the power scripting brings" can be part of the problem. Scripting is used as a means to execute arbitrary commands on a single level privilege (that of the user hooking the scripts, very often root or some high-privilege user) as opposed to a limited set of application-specific functionality. This adds a whole level of complexity to the system, which in security terms is typically the kind of thing that will give you trouble.
Agreed. When in security classes we were taught to use e.g. execve() instead of system(), it wasn't because shells were thought of as particularly vulnerable. You just want to use a tool that has the minimum possible feature set, so you can be sure that no one malicious will be able to trick you into using even correctly-functioning features (e.g. through shell injection).
Sort of a special case of the principle of minimum privilege, when applied to the feature set of your tools.
pretty much. scripting envs are extremely flexible and powerful, thus very prone to such issues. thats why you dont give user input to shells unless you want the user to have the full shell access. environment included.
NetworkManager still mucks around in /etc/sysconfig/network-scripts and is probably a lot more complicated than it needs to be. I'd like to think that networkd is a better approach with a smaller attack surface but also right now networkd doesn't have the maturity or usage of NetworkManager or crappy network scripts so it would be unwise to suggest that networkd is more secure because of that. Especially when you lump in NetworkManager as another alternative that's done better, if anything I'd rather have no NetworkManager or networkd than have to use NetworkManager on something other than a laptop as random bugs in NetworkManager have personally bit me before.
That being said the design of networkd looks very nice indeed and against my better judgement I'm even running it in something that you could almost consider production.
Does NetworkManager completely obviate the need for the DHCP binary to call shell scripts?
Those scripts often exist because if the sysadmin needs something special to happen on DHCP, this is where he sets it. It's not "DHCP scripts get run by/are shell scripts," it's "DHCP binaries are prepared to call out to external scripts."
I've had to write these shell scripts (using ksh, since OpenBSD, so those are safe).
> A variety of other system services are used by NetworkManager
> to provide network functionality wpasupplicant for wireless
> connections and 8021x wired connections pppd for PPP and
> mobile broadband connections DHCP clients for dynamic IP
> addressing
It calls shell scripts, because, like I said in the comment you replied to, sometimes sysadmins need very specific things to happen when the machine gets a DHCP lease.
This is hundreds of times worse than heartbleed in terms of scope/attack surface for modern servers... (I say hundreds of times worse because heartbleed was scrape-some-data-till-you-get-private-keys-and-watch-communication, where this is just get-yourself-a-shell-and-pwn-the-machine)