> However, please consider adding our scanner (71.6.151.167) to your "whitelist". We are well-known cyber-sec researchers, we aren't trying anything nefarious or evil, and we are being as transparent as possible about our scans.
Sure and while we are at it I'll fetch the lube.
I don't complain about port 22 connections because none of my machines run anything on port 22 (I move SSH to a random port mostly to deter whatever is coming out of China this week).
Even if you do find the port you still have to get around the ssh key's (so unless you are the NSA (j/k)), you could try an exploit against ssh but as it doesn't report its version good look with that and do try to avoid triggering fail2ban.
I'm not a systems administrator (I guess I'd be DevOps) but I know enough to know I don't know enough to disregard best practice.
They aren't doing anything funny, nor are they asking you for permission to do anything funny.
They scan your ports. To the unobservant, that can look like they are trying to crack your systems. What is wrong with recognizing their IP (adding it to your "whitelist"), and knowing that it probably isn't something worth investigating.
Who exactly are "they"? Other than someone with a blog post saying whitelist this ip and claiming "no monkey business, promise!" No time/interest to vet them.
Also, did their webpage get hacked and that ip changed?
Will someone with access to their machines network succumb to maliciously using or selling access to this widely whitelisted ip?
Seven years from now will that ip still be theirs?
I'm not saying that these aren't good points, just that gp's comment about lube was out of place. They aren't trying to fuck you.
And we're not talking about a literal software-implemented whitelist, just a "it probably isn't worth investigating or sending complaints" whitelist. That makes the seven-years point less important.
What's up with this belief that anybody you haven't heard of is a member of some shadowy organization? I'm a well known security researcher, I give several presentations a year at cybersec conferences, and my blog gets regularly link to from news.ycombinator.com.
Also, the post above excludes the first part of that: "We are happy to add your IP addresses to our blacklist so we won't ever scan you again".
Its safe to assume on the Internet that anybody that comes to you, is a bot or a scam of some kind. Not a surprise to you, a security consultant.
I understand frustration that you can't convince everyone that you are a good guy. All good guys these days have that problem. Even the good folks at the NSA ( you aren't one of them, are you? )
Call me a cynic (I don't mind) but they do the sweep of the whitelisted address and then two weeks later I get an attempted break-in from a completely different address....
I kind of like running SSH on the default port as it gives me a list of IPs to firewall against on other web facing services across my infrastructure.
If someone is trying to break into my server(s) via SSH, then I sure as hell don't want them to have access to any other services regardless of how secure they might be.
"Just"? That requires more work to set up than my method and offers no additional security benefits as you're still running an open SSH port.
You're falling into the common trap of security through obscurity. But in fact it's very easy to find which port SSH listening on even when you're using a non-standard on. However if you harden SSH properly then it doesnt matter what port it's listening on. So your method is more work for zero gain
I've also seen people run a fake SSH server on 22 and log attempts (if you know that you don't use 22 then any connection attempt on 22 is interesting).
People. Port knocking. Seriously. There are very few things that have come and gone in the almost 20 years that I have been securing systems and dealing with attacks, but port knocking is a substantive, Truly Good Thing.
Works for a lot of other ports too, but ssh is the obvious one.
I think port knocking is really silly, for what it's worth. Disable password authentication in SSH, minimize the number of systems you expose SSH to at all, and leave it at that.
I agree that disabling password auth--or even just using something like fail2ban to blacklist attackers--is just as effective (and significantly easier) than setting up port knocking.
The substantive difference, though, is that port knocking can be used to hide the service entirely, rendering you safe (mostly) from 0day attacks, DoS against a specific service, etc. It can also be use to make things that shouldn't be on the Internet (say, an RDP route to the CEO's computer) a little safer. Still, though, it's not like VPNs are very difficult to use...
But if you are using SSH for administrative purposes on anything important that you might be reomte from in a time of crisis you don't want to layer something else on top of SSH that might fail and not let you in, or even if the port-knocking arrnagement is OK you might not be able to access it from where you are (perhaps on a mobile/wireless network that is blocking all but the most common standard ports in an illconeived effort to block P2P traffic, which would affect people who put SSH on a non-standard port as well). OK so if it is that bad maybe everything is down/inaccessible (SSH and all), and anything that important has local hands-on support available, but it still strikes me as a bad idea to potentially lock yourself out.
The worrying thing in this report though is how many hosts seems to be running ancient (now unsupported) versions such as the stock variant from Debian 4. I bet most of those hosts have no key-based auth and easily guessed passwords too...
This program is called by /etc/hosts.deny whenever someone connects to port 22. Unless they type in a plaintext password or type the wrong password, they get an ssh-compatible error message, and a syslog message is generated. If they type in the right password, they are added to /etc/hosts.allow, and their next connection will reach the real sshd.
Any other all-around good practices that you can recommend? I'm not really sure how to go about locking down a production server to a sane degree, and there's a lot of bad information out there when it comes to security.
* Run Denyhosts, to block IP addresses that have too many repeated failed login attempts.
* Use Jumphosts (AKA Bastions). One (or a few) servers that listen to ssh from the open Internet and have no private keys stored on them (use key forwarding). All other servers only listen to ssh from an internal network interface the Jumphosts are also on, or from the Jumphosts IP address directly.
sshd is equally susceptible, where ever you run it, sure.
With jumphosts, though, you can run denyhosts, redirect to tarpits, or other clever things, and not effect performance on your other servers.
I think the motivation to use jumphosts is to minimize the surface area of your other servers as much as possible. Each daemon on each server listening for incoming connections is a liability. If an sshd exploit is found (unlikely as that is), bots that mass scan and find my ssh ports and connect won't be able to delete any production files (right away, at least) if all they gain access to is my jumphost and not one of the prod db servers.
Thanks. I didn't consider that the mere act of having sshd deny attempts is a performance cost. I guess one could potentially dos a server by flooding the sshd with bogus attempts?
well, sshd denying requests isn't a performance cost, in itself.
But having a script like denyhosts or fail2ban that analyzes auth logs for IPs to block requires memory for the interpreter running the script, as well as however much of the log gets read into memory.
Blocking external ssh connections can avoid that performance hit, however large or small it ends up being.
I've been using knockd[1] for the last 5 or so years. Also has prebuilt binaries (and in many repositories[2]) for almost every distro, including cygwin, native windows and android.
Not knowing much about this stuff, are there any clients that will 'automate' the knock for you? If so, can you make any suggestions? Or is this a matter of CLI SSH will do it, but I'm just not finding it?
It's a client/server thing, so you have to have the server running on the PC you intend to connect to (like when using ssh).
Never used it in any other way than with the CLI and not with every server I admin or use as there's other best practices also mentioned in the thread that are typically good enough without throwing in port knocking like:
Most things you can just use knocking and connect with ssh forwarding/proxy and tunnel everything through it like DB clients, IDE connections instead of having a bunch of ports open to internet. Assuming you don't kill the connection, then you can keep reusing it as a tunnel for anything else until rebooting or losing internet connection.
Why do people think it's worthwhile to file abuse complaints about port 22 connections? If I wanted to file a complaint about every random connection to port 22 on one of my machines, it would be a full time job...
Don't manually file them in that case. Install Fail2ban and automatically ban anyone with x failed attempts, and have it email fail2ban@blocklist.de. Their system will look up the correct abuse department, and forward it along on your behalf, along with thousands of others that report it. A large portion of them probably go ignored, but if you can get a fraction of them taken offline, then why not spend the five minutes to set it up?
Well, if you had a "class A" address space, you would have seen 16777216 connection attempts in your firewall logs. Yeah, I imagine that would be a full time job.
A lot of 'rouge' ssh traffic comes from hacked machines. If you contact the abuse department of a provider, they may get in touch with the owner of the machine to let them know that their box has been compromised.
Yes I know that, but I'm not convinced that the cost/benefit really makes sense for complaining. Back in my more idealistic days I did report a lot of abuse. But it's so rare for the recipient to actually do anything about it, and the volume of scans so high, it's just not worth my time.
I'd guess a closer interpretation might be "there are only ~2,900,000-ish machines publicly answering on port 22 on the internet".
(Most of my "important" servers have port 22 firewalled off an only open to a small set of external ip addresses. Some of them aren't running ssh at all.)
OpenSSH 4.3 is likely the most popular because it is the version that comes with (redhat|centos|scientific|oracle) linux 5. It's still widely in use. 4.3 had a lot of bugs, but redhat has been backporting fixes to it since it came out.
I used to run SSH over some port != 22, does the trick to some degree.
After I picked up a Cisco ASA, went back to standard port 22 but only allow access for connected VPN users.
Of course if the ASA goes down, so does the entire network, yelp. SmartNET contract/warranty comes in handy, and the data center having backup ASAs on site for quick swap is pretty useful as well.
When I set up my first server I was thoroughly surprised at the amount of login attempts. By now I've got most of Asia on a blacklist for port 22 (which doesn't matter, since I'm not going to connect to my server from an Asian IP in the foreseeable future). That throttled it down to about 30 attempts/week, which is manageable in terms of log reading.
Excuse me? How is VPN not secure? The ASA won't let any traffic through on port 22, period. You have to be connected over VPN in order to even attempt SSH access to servers behind the ASA.
I run a VPS that only is only accessible over IPv6. I wonder if they'll ever scan it. Is there a way to narrow down the whole IPv6 search space to the most populated subranges?
Well you could get hold of the routing tables and filter out unallocated space. But thats not going to help much as there are still 64 low end bits to guess, some people use xxx::1 and ::2 but others use mac address based IPs (and random ones). So we are probably at the best point to portscan before the possibility goes away again...
I'm on a phone with shitty wifi atm so looking at the code would be hard to say the least. However, I am curious, how do you deal with packet loss?
The reason I'm asking is because, most people who claim to "scan the Internet" assume that the network is reliable. And they don't follow up on potential false negatives. If you scan the IPv4 address-space sequentially while only limiting bandwidth or time, rest assured that packets will be dropped.
If you are using the default config the chances are you are using passwords to login? This is bad because unless you are using something like fail2ban you are leaving yourself open to brute force attacks.
Rough best practice is, 1) Use SSH keys to login 2) Enforce SSH to login (no password) 3) Disable Root Logins (login then elevate permissions not logging in as root) 4) Move from port 22 to something else (if you do 1-3 this shouldn't matter I do it mostly to keep the crap out my logs) 5) install something like fail2ban to automatically blacklist IP addresses.
As an alternative to ( 4 ), only expose your SSH servers over IPv6.
Initially I thought this worked because the overwhelmingly huge address space, but even when I published come-get-me DNS records ( ssh.example.com ) there was not a single IPv6 attempt versus hundreds of IPv4 attempts every day on the same host.
Edit: there is plenty ( ~ 8% ) of inbound IPv6 web traffic to my hosts, so it's not a case of IPv6 being obscure. It just seems that none of the bad guys are interested.
Nice idea until you're logging in from a slow/overloaded connection, or your ssh host is being flooded with traffic, or has a runaway process that's causing it to thrash, or lots of other reasons you might want to use ssh.
I can't think of a scenario where a really short login grace time would protect you from an attacker, either. What's the motivation?
There is no security advantage to #4 (security through obscurity is no security) other than it prevents anyone getting a ping by sweeping Port 22, they actively have to look for an SSH port which will trip fail2ban.
Not entirely true. Moving your SSH port will not prevent a targeted attacker from finding it, it'll barely slow them down.
What it will do however is prevent automated script-kiddy scanning tools from seeing port 22 open, scraping the banner, and adding your IP to a list of target to brute force/exploit. That's a good thing, and is absolutely, by any definition, a security benefit.
Automated tools already won't get into your machine if you disable passwords. And they can easily adapt to non standard ports if the authors see any worth in doing that.
Setting a non-standard port to ssh is akin to adding a wood plank to the door of a safe with state of the art locks and 20" steel walls. A very annoying wood plank, by the way.
Wrong. That's how NOT to set up SSH. Use keys and / or log scanning to auto firewall failed login attempts. I recommend fail2ban, but there's a few daemons that do the same thing.
That's fine because it's a not for security reasons. In fact I've been known to do the same myself (tunnel web traffic over an SSH server running on 443 to circumvent firewalls on public WiFi hotspots)
If they are only hitting a port 22 a few times, who's wasting the ISP's time with abuse requests... I suppose its probably those guys with Class-A blocks and too much time on their hands.
> We are going to be extending this to more ports, such as FTP and SMTP. Soon, we should have weekly scans going for about 10 ports. I'm moving slowly forward to resolve abuse complaints, like this one generated for port 22. We plan on publishing the results, such as the anonymous counts above, in a nice weekly report for the public.
Sure and while we are at it I'll fetch the lube.
I don't complain about port 22 connections because none of my machines run anything on port 22 (I move SSH to a random port mostly to deter whatever is coming out of China this week).
Even if you do find the port you still have to get around the ssh key's (so unless you are the NSA (j/k)), you could try an exploit against ssh but as it doesn't report its version good look with that and do try to avoid triggering fail2ban.
I'm not a systems administrator (I guess I'd be DevOps) but I know enough to know I don't know enough to disregard best practice.