Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Coping With 1 Million SSH Authentication Failures?

kdawson posted more than 4 years ago | from the some-definitions-of-managed dept.

Security 497

An anonymous reader writes "I own a small Web development studio that specializes in open source software, primarily Drupal, WordPress, and Joomla for small businesses. Our production servers, which host about 50 sites and generate ~20K hits/week, are managed by a 3rd party that I'm sure many on Slashdot would recognize. Earlier today I was researching some problems on one of our sites and found that there have been over 1 million SSH authentication failures from ~1200 IP addresses on one of our servers over the last year. I contacted the ISP, who had promised me that server security would be actively managed, and their recommendation was, 'change the SSH port!' Of course this makes sense and may help to an extent, but it still doesn't solve the problem I'm facing: how do you manage server security on a tight budget with literally no system admin (except for me and I know I'm a n00b)? User passwords are randomly generated, we use a non-standard SSH port, and do not use any unencrypted services such as FTP. Is there a server monitoring program you would recommend? Is there an ISP or Web-based service that specializes in this?"

cancel ×

497 comments

Sorry! There are no comments related to the filter you selected.

suggestion: (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31385090)

eat out my asshole. Won't help with that SSH thing, but damn I enjoy it.

HELP! SOMEONE DIAL 911 (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#31385260)

eat out my asshole. Won't help with that SSH thing, but damn I enjoy it.

Call up Rob Malda. Tell him someone finally needs him to do what he studied so hard for at STU. By the way, STU is the acronym for Salad Toss University. Cheers.

fail2ban (5, Informative)

Anonymous Coward | more than 4 years ago | (#31385092)

fail2ban, key-auth

Re:fail2ban (5, Informative)

jimpop (27817) | more than 4 years ago | (#31385130)

> fail2ban, key-auth

+1

Change port. Use iptables to only allow access from known subnets/hosts.

Re:fail2ban (2, Informative)

B'Trey (111263) | more than 4 years ago | (#31385494)

And use DenyHosts [sourceforge.net]

Re:fail2ban (0)

Anonymous Coward | more than 4 years ago | (#31385704)

+1
There is no one answer and not every solution will fit everyone requirements but remember, security in layers.
Limiting the source IP to a range of where your users would be coming from will make the biggest difference. If you have random people coming in from everywhere that obviously won't work.

Re:fail2ban (2, Informative)

sstern (56589) | more than 4 years ago | (#31385372)

Install OSSEC, too. There may be other stuff going on.

Re:fail2ban (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31385660)

These are the knee-jerk responses of people who are downright know-nothings.

Why the FUCK would i run some fucking shitty script, written by some unknown cunt with ABSOLUTELY NO CLUE ABOUT ANYTHING, on a production system?

Why, when iptables recent module does the same thing, and doesnt require lame hackery by some dumb faggot who can't read man pages?

Seriously, anyone using fail2ban has failed themselves. The person who wrote it must be holding their head in shame knowing they re-invented a wheel that had rough bumpy bits which sometimes failed.

Re:fail2ban (4, Funny)

Kugrian (886993) | more than 4 years ago | (#31385724)

Someone fucked up big time when they taught you html:
[url]http://www.fail2ban.org/[/url]

Re:fail2ban (0)

Anonymous Coward | more than 4 years ago | (#31385736)

lolol

whatcouldposiblygowrong (-1, Troll)

ionix5891 (1228718) | more than 4 years ago | (#31385098)

self-proclaimed "noobs" managing public facing servers

Re:whatcouldposiblygowrong (0)

Anonymous Coward | more than 4 years ago | (#31385124)

what could go wrong?
worst case scenario? Slashdot trolls.

Re:whatcouldposiblygowrong (1)

asdf7890 (1518587) | more than 4 years ago | (#31385538)

what could go wrong?

Lots.

worst case scenario?

Depends how realistic we are being with our range of possible scenarios.

Worst not-ridiculously-unlikely case: ill configured service, service minus important patches, or poorly chosen authentication/privilege setup, or something similar, allows an automated hack in and therefore the server becomes yet another bot in the net or possibly even a C&C box for the botnet.

A zombied server is usually more trouble for the rest of the 'net then a zombied home machine because they generally have access to better bandwidth resources and are more likely to be openly connectible from the outside world so can more easily be used to host services such as phishing websites or a botnet C&C relay.

There is also the potential loss/damage to the information in the server itself to consider. If a "noob" admin is running the server then it is not unlikely that the services and data held on the server are not adequately backed up.

Re:whatcouldposiblygowrong (4, Funny)

jo_ham (604554) | more than 4 years ago | (#31385276)

He could get trolled on slashdot by the very people he's coming to ask for help to become *less* of a noob.

I'll bet you teach your kid gun safety by shooting him in the neck.

Re:whatcouldposiblygowrong (2, Insightful)

NNKK (218503) | more than 4 years ago | (#31385332)

If somebody tries to perform surgery without a proper medical education, do you offer pointers, or do you tell them to drop the knife and find a real surgeon?

Companies running public-facing servers without good technical expertise are a menace.

Re:whatcouldposiblygowrong (1)

jo_ham (604554) | more than 4 years ago | (#31385520)

Then the answer to the question is the link to someone who can provide that.

I doubt the responses would have been quite so hostile if the topic had been car repair - poorly maintained vehicles are a menace on the roads!

The answer does not necessarily have to be "here is what you do" - it can be "find someone who knows what they are doing". Trolling him is not likely to create any positive effect, except among the already insular server admin clique. Save the "oh my god, so this noob totally connected a 54-50 to a 35-27 with no logging!" drinking stories for the bar.

Re:whatcouldposiblygowrong (4, Insightful)

dintech (998802) | more than 4 years ago | (#31385534)

That's a beautiful and witty analogy but to reverse it:

If he asked "How can I stop my house being burgled?" and you said, "Without a full alarm, CCTV and patrol system, don't even bother. Leave it to the professionals."

Some others might see some practical value in suggesting, "Maybe lock your door at night."

My opinion in this situation is first to take what advice you can and do something yourself as soon as possible (since any extra security is better than nothing). Next, as you suggested and when he has the resources to employ someone to do it properly, seek professional help. Simply doing nothing until he can do it properly sounds a bit dangerous.

Re:whatcouldposiblygowrong (1)

bigsteve@dstc (140392) | more than 4 years ago | (#31385718)

Some others might see some practical value in suggesting, "Maybe lock your door at night."

The problem is there is no simple laymans advice like that that would solve his problem. He needs someone to fit new locks to his doors, etc not just turn the key in the door.

Re:whatcouldposiblygowrong (2, Insightful)

obarthelemy (160321) | more than 4 years ago | (#31385570)

Since it seems to have escaped your attention, I'd like to point out the fact that unqualified surgeons can (will ?) kill people; unqualified admins won't.

Which seems to be a somewhat relevant matter.

Re:whatcouldposiblygowrong (0)

Anonymous Coward | more than 4 years ago | (#31385624)

I'd tell them to wait a minute, then run and grab a video camera. But that might not work with the analogy...

the web is new (1)

circletimessquare (444983) | more than 4 years ago | (#31385744)

when surgery was first performed, anyone could and did do it

to much suffering, yes

but you can't expect the regimented laws and well-established protocols of long experience with a brand new technology

and yes, the internet is STILL brand new technology

we are only beginning to see standards and laws and social changes from its appearance

as well as further technological change

you can't expect the standards you are asking for

Re:whatcouldposiblygowrong (0, Offtopic)

darkpixel2k (623900) | more than 4 years ago | (#31385690)

He could get trolled on slashdot by the very people he's coming to ask for help to become *less* of a noob.

I'll bet you teach your kid gun safety by shooting him in the neck.

LMAO! Where are my mod points?

Re:whatcouldposiblygowrong (0)

Anonymous Coward | more than 4 years ago | (#31385482)

That's a hell of a lot better than some "l33t h$ck3r" who does not know his ass from a whole in the ground. I will take the "noob" any day. They at least show an open mind and admit their weaknesses.

Ignore it? (1, Informative)

Anonymous Coward | more than 4 years ago | (#31385104)

If you have a public site, it'll get scanned night and day by APNIC. If they see a banner for SSH, they'll jump on that. As long as you are using patched software and have good user security, what's the real emergency?

Exactly (0, Insightful)

Anonymous Coward | more than 4 years ago | (#31385118)

People are just paranoid. "How can I stop people from driving past my house and seeing if the lights are on?" You can't, cheaply.

Re:Exactly (1)

Darkness404 (1287218) | more than 4 years ago | (#31385386)

There is a difference between someone driving past and seeing, and a million cars driving past.

Re:Exactly (2, Interesting)

mjschultz (819188) | more than 4 years ago | (#31385582)

But a million cars haven't driven past. Only ~1200 --- in the past year. This is no big deal. Hell, I start thinking something is wrong when I DON'T see many failed SSH attempts in the daily log reports.

Re:Exactly (4, Funny)

Minwee (522556) | more than 4 years ago | (#31385394)

"How can I stop people from driving past my house and seeing if the lights are on?"

That's easy, just move the front door to where one of your upstairs windows is and install tiny robots that will draw the curtains if the traffic noise gets too loud.

Re:Exactly (0)

Anonymous Coward | more than 4 years ago | (#31385458)

Analogy:
Live in a Gated community, or put up a wall that checks if they are valid and if they are not 10 times in a row then they are blocked for a month.

For a server:
Only allow verified IPs to SSH, or make an IPTables rule to block them for a month after 10 sequential failures.

Re:Ignore it? (1)

MichaelSmith (789609) | more than 4 years ago | (#31385436)

Yeah this is just normal for me. I have a script which hooks into syslog and blocks offending hosts in pf but frankly I don't think it is worth the risk of doing that. Make sure sshd has a secure configuration and don't worry about the brute forcing.

Re:Ignore it? (4, Insightful)

tomhudson (43916) | more than 4 years ago | (#31385580)

There's no "brute forcing" going on. Look at the numbers.

1 million per year over 50 sites == 20,000 per year per site, or 54 per day per site. Change the passwords every few months (and use different ones for each site).

Further - 1,200 different remote hosts means what, 17 attempts per remote host per site per year. Probably randomly p0wned PCs that hit addresses at random, make a few attempts using default or ocmmon passwords, then move on.

Hardware firewall or use bfd (3, Informative)

Golbez81 (1582163) | more than 4 years ago | (#31385106)

bfd (and apf if you like it) are probably the best software solutions your gonna find, but if you're facing 1 million+ auth failures, I would seriously consider a hardware firewall and VPN of some sort.

Tar Pitting (4, Informative)

spydum (828400) | more than 4 years ago | (#31385108)

I'm a big fan of tarpitting SSH connections myself. It dramatically cuts down on the repeated ssh attempts, and keeps my logs much cleaner.

Basically, an iptables rule:
-A INPUTCHAIN -m state --state NEW -m recent -p tcp --dport 22 --update --seconds 30 --hitcount 4 --rttl --name SSH -j DROP

Re:Tar Pitting (2, Informative)

Anonymous Coward | more than 4 years ago | (#31385132)

Fail2ban [fail2ban.org] can do stuff that automatically!

Re:Tar Pitting (4, Insightful)

Minwee (522556) | more than 4 years ago | (#31385434)

Fail2ban can do stuff that automatically!

Let me see if I have this right. If iptables can keep count of incoming connections within the kernel and drop incoming connections as they happen without any intervention from a userspace program, that's _not_ automatic.

But running a gigantic shell script to scrape text messages out of your system logs and then call randomly chosen commands which may or may not have any effect on the observed problem some time after it occurs, then that's "doing that stuff automatically"?

Maybe those words don't mean what one of us thinks they do.

Re:Tar Pitting (4, Informative)

jonesy2k (934862) | more than 4 years ago | (#31385254)

That's not tarpitting. Tarpitting involves keeping the connection open in order to tie up the resources of the attacking host.

Re:Tar Pitting (5, Interesting)

Anonymous Coward | more than 4 years ago | (#31385284)

I use a variation on tarpitting that has worked very well for me.
It cut the attempts down from 60,000/day to 20 or 30 per day.

I just add a small delay between the initial connection attempt
and when I send the username/password prompt. The delay (in
seconds) is the number of attempts in the last 30 minutes,
squared. This makes all but the most determined attacker
give up and go away very quickly.

I have been using this with both FTP and SSH for the last
year, and it works great.

Re:Tar Pitting (4, Interesting)

IQgryn (1081397) | more than 4 years ago | (#31385518)

How would one go about implementing this delay?

Re:Tar Pitting (0)

Anonymous Coward | more than 4 years ago | (#31385554)

you slick bastard

I love it! (1)

Bananatree3 (872975) | more than 4 years ago | (#31385650)

You are making them work for their attempt. Soon, you're servers are just too expensive time wise to make it a worthwhile investment.

Re:Tar Pitting (0)

Anonymous Coward | more than 4 years ago | (#31385396)

I had the same problem the OP describes and implemented fail2ban. Solved it quick. After a while, they gave up and now it's clean. They must watch for failures, too.

Re:Tar Pitting (2, Insightful)

luizd (716122) | more than 4 years ago | (#31385438)

Yes, I think that this is your best solution. Instead of using iptables directly, you should check your distribution software relative option.Generally is is easier (but anyway, it will ultimately result in the same iptables command)

Re:Tar Pitting (4, Insightful)

cptdondo (59460) | more than 4 years ago | (#31385484)

That, and create only a single user who can log in, that takes you to the real log in prompt. That way an attacker has to guess the one user+password, and have a legitimate userid+password to gain access.

It's not foolproof, but it foils the vast majority of script attackers.

And DISABLE ROOT LOGIN!

sshblack (2, Insightful)

Anonymous Coward | more than 4 years ago | (#31385114)

sshblack works very well for me. It monitors failed ssh connections and inserts entries into iptables to block the IPs.

use openvpn ? (1)

yorugua (697900) | more than 4 years ago | (#31385116)

if you can manage the set of users of your server, you can use OpenVPN and then SSH. OpenVPN has a "feature" that if each packet of the VPN is not digitally signed by a previously arranged (and distribuyed) key, then the packet is "ignored". After the VPN session is established/authenticated, your users can log in using ssh. There are even some virtual appliances and special distros (untangle.com) that have a "openvpn appliance" built in for this purpose. The how-to for openvpn is also easy to follow.

Passwords? (4, Informative)

fm6 (162816) | more than 4 years ago | (#31385142)

Here's my excuse to ride my usual hobbyhorse about passwords being obsolete. SSH supports certificate-based authentication, which is not only more secure, it's less of a hassle for the user. Don't know if it would be practical for your application (you might tell us more about that) but it's worth a look.

Re:Passwords? (1)

timmarhy (659436) | more than 4 years ago | (#31385504)

certs are harder for a user to setup, which means support time which cuts into a small time operation a lot.

Re:Passwords? (1)

david.emery (127135) | more than 4 years ago | (#31385596)

I agree with both parent posts. PKI Certs are certainly the way to go, but it's really hard to do this right.

This is a case where some consulting to (a) set up the PKI stuff; (b) train our (unfortunately anonymous) questioner on how to disseminate the certs; and (c) apply the appropriate tarpit/other firewall settings, would probably be money-well-spent.

Re:Passwords? (0)

Anonymous Coward | more than 4 years ago | (#31385692)

If they can't figure out SHH passwords themselves, WTF are they doing logging into a remote shell?????

fwknop (4, Informative)

c0d3g33k (102699) | more than 4 years ago | (#31385144)

They can't fail authentication if they don't know it's there. http://cipherdyne.org/fwknop/ [cipherdyne.org]

Re:fwknop (1)

mprinkey (1434) | more than 4 years ago | (#31385720)

I agree completely. We use fwknop as a part of a required two-factor authorization scheme for a US Government customer. It is a minor inconvenience to use it to open the ssh port before connecting, but it virtually eliminates attack attempts. How can you hack a closed port?

You are being brute-forced (4, Informative)

Anonymous Coward | more than 4 years ago | (#31385150)

Welcome to the internet -- this happens to every site with a public IP.

First off, do not change your SSH port. It won't do a whole lot for you, and it will be more hassle than it works.

There are a whole lot of programs available to deal with SSH brute forcing. sshguard is one of them, and it's not too bad (you can apt-get install it). It's a bit of a hack -- it just watches your logs and takes appropriate action -- but it does work.

The other thing you can do immediately is simply turn off password authentication in ssh. Anyone getting in will need a key. This is what sourceforge and github both do. This isn't always practical for every site, but it will damn sure keep your passwords from being brute-forced.

Never passwords! (1)

Just Some Guy (3352) | more than 4 years ago | (#31385156)

Use public key authentication. It's super easy and much more secure, unless your random passwords happen to be over 300 characters long (ssh-keygen makes 2048-bit keys by default; assume 6 bits per character in a random password if each character is in the base64 set). Also, sshguard is your friend. After a small number of invalid logins - 4 by default - that IP gets firewalled for a short period of time.

Re:Never passwords! (1)

GringoChapin (1663533) | more than 4 years ago | (#31385406)

Actually, 2048 bits on a symmetric key (which is what a password is) is insane overkill. You are making the classic mistake of comparing the number of bits in an asymmetric key to the number in a symmetric key. SSH, at best, uses 256 bit symmetric encryption for the session, so you would only need a (using your numbers for 6 bits per char) password that was 43 chars long. Anything longer is pointless, because your encryption/authentication scheme is only as good as the weakest link.

Re:Never passwords! (0)

Anonymous Coward | more than 4 years ago | (#31385636)

Symmetric keys are generated via Diffie-Hellman and not related to authentication. You are making the classic mistake of not understanding what you're talking about.

Re:Never passwords! (0)

Anonymous Coward | more than 4 years ago | (#31385488)

Just so you know, you’re comparing RSA public keys with entropy there. Apples and oranges. You would get equivalent security with somewhere around 20-25 random alphanumeric mixed-case characters.

Fail2ban or denyhosts (3, Informative)

mystik (38627) | more than 4 years ago | (#31385158)

Fail2ban [fail2ban.org] or denyhosts [sourceforge.net] activly target ssh. fail2ban includes rules for other services, but denyhosts includes a mechanism for sharing lists of your denied hosts w/ other admins, as well as using their ban lists to protect your ssh logins.

SSH public key authentication (4, Informative)

whamett (917546) | more than 4 years ago | (#31385166)

This kind of brute-forcing is common. The simplest way to deal with it is to set up SSH public key authentication, disable SSH password authentication, and forget about it.

Fail2Ban (1)

j_kenpo (571930) | more than 4 years ago | (#31385168)

If your not willing to invest money into intrusion detection, vulnerability scanning, or moving to a more responsive provider, accept that your server is going to constantly be scanned by the droves of script kiddies out there throwing everything but the kitchen sink at any server that responds to a ping and keep monitoring those logs, which hopefully are stored on a separate server in case you are ever actually compromised. In the mean time, try installing Fail2Ban on your server. It will block an IP address after X number of failed authentication attempts, which will alleviate the noise of the brute force password guessing attempts.

What could all those people want to badly? (0)

Anonymous Coward | more than 4 years ago | (#31385176)

Everybody wants free porn. You're just going to have to live with that. You sleep with dogs, you're bound to get flees (unless you have mandatory government filters).

Spend more than 10 minutes? (2, Informative)

Seakip18 (1106315) | more than 4 years ago | (#31385186)

Seriously. I'm in the same shoes and it's easy. I came across it pretty quick and all these SSH log in problems went away.

Check out...waaaaait...

What's most troubling is this:

I own a small Web development studio that specializes in open source software, primarily Drupal, WordPress, and Joomla for small businesses

You operate a business and you haven't figure out this chief problem yet...but you want us to help you out?

Well...Here's the solution: denyhosts [sourceforge.net]

Someone else'll chime in with it....Just hope you read up and configure it properly....or you'll find yourself locked out of your own servers.

Re:Spend more than 10 minutes? (4, Informative)

Gearoid_Murphy (976819) | more than 4 years ago | (#31385338)

add the ip address and/or hostname of all the hosts you use to access your servers into /etc/hosts.allow. If denyhosts picks up 3 failed logins from a single ip address, that address is added to /etc/hosts.deny, if this happens to be your machine (and you're having a bad day entering your password), you could get locked out of your system.

DenyHosts (1)

Utoxin (26011) | more than 4 years ago | (#31385194)

Great tool that pools the resources of multiple servers to proactively block IPs that are trying to brute-force the SSH port on any server in the pool:

http://denyhosts.sourceforge.net/ [sourceforge.net]

I use it, and it has Just Worked for years.

So? (4, Interesting)

victim (30647) | more than 4 years ago | (#31385204)

If you really have secure passwords, the random guessers aren't going to get them. Log it and move on.

I get thousands of Chinese hackers attempting to break into the battery monitor in my tool shed, big deal. I don't know why my battery voltage and solar current is so important to them, but they can knock themselves out all day.

If the logs bother you then install fail2ban and configure it to lock people out after a few bad guesses. (Then be ready to unlock yourself from an alternate IP when you inevitably lock yourself out.)

Re:So? (0)

Anonymous Coward | more than 4 years ago | (#31385472)

I get thousands of Chinese hackers attempting to break into the battery monitor in my tool shed, big deal. I don't know why my battery voltage and solar current is so important to them, but they can knock themselves out all day.

don't you mean MY battery monitor?

Move to a higher order port and use denyhosts (3, Informative)

deadmongrel (621467) | more than 4 years ago | (#31385214)

I use one or more of these on my public facing servers.
1. Move the default ssh port to a higher order port (5000+)
2. Use Denyhosts http://denyhosts.sourceforge.net/ [sourceforge.net] to block repeated attempts
3. use key exchange instead of username/password
4. use network based IPS.
Just moving the ssh port reduced the ssh brute force attack for me. Either stop being a noob or hire a sys admin.

Re:Move to a higher order port and use denyhosts (5, Funny)

sootman (158191) | more than 4 years ago | (#31385258)

1. Move the default ssh port to a higher order port (5000+)

Agreed. The higher the better. For the ultimate in security, I recommend 65536.

Re:Move to a higher order port and use denyhosts (0)

Anonymous Coward | more than 4 years ago | (#31385748)

Your ip-packets go all the way up to 65536? That's one more!

Keys... (0)

Anonymous Coward | more than 4 years ago | (#31385226)

Key-based authentication is really what you need. If your situation doesn't allow for key-based auth, Fail2Ban can help reduce brute-force attempts for password-based auth, and allowing connections from specific IPs, if possible, can heighten security quite nicely.

BlockHosts (2, Informative)

psychosis (2579) | more than 4 years ago | (#31385266)

We started using BlockHosts to feed iptables rules, and our failure logs went from 30-50k per day to 100. Basically, with more than 'x' failed logins within 'y' time frame, the source IP is blocked for 'z' time period. Since it uses iptables, you could block it from just the ssh port, or the entire system (we do the latter).
All three variables are configurable, and we also have whitelisted a few select standby IPs for contingency use. (As another poster said, you **will** lock yourself out eventually.)

Re:BlockHosts (0)

Anonymous Coward | more than 4 years ago | (#31385364)

you wont lock yourself out if you only allow key access.

Throttle Inbound Connections (5, Informative)

Padrino121 (320846) | more than 4 years ago | (#31385288)

I have a similar situation and cannot limit to very specific IP ranges. I have done the following with good success. I pulled some examples from my configuration that can be tweaked for yours if you like.

1. Limit incoming SSH attempts to a low number. In my case I limit to 2 connections in 60 seconds. I can tighten it even more but this did a lot to kill brute force attempts.
iptables -I INPUT -p tcp -i vlan1 --dport 2242 -j DROP
iptables -I INPUT -p tcp -i vlan1 --dport 2242 -m state --state NEW -m limit --limit 2/min -j ACCEPT
iptables -I INPUT -p tcp -i vlan1 --dport 2242 -m state --state RELATED,ESTABLISHED -j ACCEPT

2. Automatic blacklist via DenyHosts. This helps cut down attempts from known ranges without even giving them the chance even at a slow rate. http://denyhosts.sourceforge.net/

An iptables recipie (4, Insightful)

Simon Carr (1788) | more than 4 years ago | (#31385292)

-A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -m recent --set --name SSH --rsource -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -m recent --update --seconds 60 --hitcount 8 --rttl --name SSH --rsource -j LOG --log-prefix "SSH_brute_force "
-A INPUT -p tcp -m tcp --dport 22 -m recent --update --seconds 60 --hitcount 8 --rttl --name SSH --rsource -j DROP

Stops 'em *somewhat* dead. If you want to whitelist hosts so they are not impacted by this rule, just add;

-A INPUT -s your.ip.address -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT

Before the throttling ruleset.

fail2ban (1)

EvilIdler (21087) | more than 4 years ago | (#31385296)

http://www.fail2ban.org/wiki/index.php/Main_Page [fail2ban.org]

That should cover automatic banning of repeat offenders. It works by reading logs, so it can work with many services. You could for example make it block all those attempting to exploit your CMSes.

OpenBSD PF (3, Insightful)

roman_mir (125474) | more than 4 years ago | (#31385298)

#SSH
pass in inet proto tcp from any to $ext_if port ssh
pass out inet proto tcp from any to $ext_if port ssh
pass quick proto tcp from any to $ext_if port ssh \
        flags S/SA keep state \
        (max-src-conn 15, max-src-conn-rate 5/3, \
          overload <bruteforce> flush global)

you know what I am saying?

I don't understand the problem (2, Interesting)

dingen (958134) | more than 4 years ago | (#31385340)

So there have been a million failed attempts to log in over SSH in a year. That's about one failed attempt every 30 seconds.

What's wrong with that? It's not like they're getting in, right?

Denyhosts (0)

Anonymous Coward | more than 4 years ago | (#31385358)

We use denyhosts on our production servers. Works like a charm. Just make sure you have an alternate ip to log on from in case you lock yourself out.

OSSEC is what I use (1)

loftwyr (36717) | more than 4 years ago | (#31385378)

OSSEC [ossec.net] is what I've been using for years and it works well. IT's much more comprehensive a security package than fail2ban but uses log monitoring as it's basis.

It's worth a look.

Easy (1)

Beached (52204) | more than 4 years ago | (#31385400)

Change the port and use private certificates for authentication.

However, changing the port to something will get the bots off your back. They still don't try other ports.

Limiting SSH Access (1)

sabaisabai (943536) | more than 4 years ago | (#31385428)

Change the default port, disable SSH access for root, disable password access entirely (login with public keys), install fcheck to monitor changed files and hence intrusions. If you have the luxury, remove SSH access entirely from your web server and block everything but ports 80 and 443, and enter via another server behind the same firewall. As a nOOb, I gained quite a lot from following this Hardening Linux Web Servers guide: http://www.freesoftwaremagazine.com/articles/hardening_linux [freesoftwaremagazine.com]

Change ports (1)

PhunkySchtuff (208108) | more than 4 years ago | (#31385430)

Although the solutions like fail2ban and denyhosts are undeniably useful, you will cut down well over 90% of the drive-by ssh attempts by simply changing the port.

None of the scripts, that I'm aware of, actually portscan your machine looking for the SSH login headers, they all hit port 22 and try to log in. It takes too long to scan a machine so if nothing is listening on port 22, they will move on.

You will still get the odd attempt from someone who has portscanned you - these will be relatively rare. In these cases simply disable password logins altogether and use the public key authentication that is a LOT more secure (and convenient once it's all set up) anyway.

Enforce RSA keys and disable passwords (1)

pestilence669 (823950) | more than 4 years ago | (#31385432)

RSA keys are notoriously time intensive to brute-force. As far as I know, no one is crazy enough to automate an attack. If you disable password authentication altogether, you eliminate your risk of an automated dictionary attack. It might confuse the hell out of some users, which is a con.

solution (1)

unix_geek_512 (810627) | more than 4 years ago | (#31385448)

--dport 22 -i DROP

Use a high SSH port and allow logins from two static IPs -j DROP everything else.

Hire me to fix the rest ;)

Change port (0)

Anonymous Coward | more than 4 years ago | (#31385450)

This is pretty common stuff trying random SSH logins on public facing servers, at least for me - just change the SSH port and have strong SSH passwords - that's all I do and stops it dead.

If you are really worried, firewall/iptables to fixed IP addresses.

Okay (1)

mindstrm (20013) | more than 4 years ago | (#31385464)

First..... 1 million password *failures* is not a security problem by itself. They're supposed to fail if they don't have the right password.

So - moving this to a more obscure port will definitely cut down on the automated scanning.. but it's not really the heart of the issue.

The heart of the issue is that it took you so long to realize someone was even doing this. That probably boils down to your running a business requiring sysadmin work without a sysadmin.

There are lots and lots of ways to approach this, some of which people have touched on already.

1) You need awareness of what's going on - daily log analysis. OSSEC is neat, give it a shot.
2) Moving SSH to a weird port is something some people do - I don't prefer that - it buys some obscurity and cuts down on some scanning, but doesn't address any real security issues.
3) OSSEC in active mode with a whitelist so you don't block out legitimate users - works pretty well.
4) Hire someone to deal with security/deployment/backups/recovery/service providers (sysadmin?)

Security through Obscurity, sortof (1)

Seriman (775126) | more than 4 years ago | (#31385468)

Obviously the port change idea is unpopular because it's just obscurity, which != security. However, most, of not all, of those attempts are probably from scripts rather than actual human attackers actively attempting to compromise your servers. The script kiddies rely on automation to rapidly find vulnerable hosts using easily-scripted vulnerabilities or brute force. The simple act of changing your ssh port will render those kinds of attacks useless against you. I had a similar scenario where my logs were full of failed ssh attempts and by changing the ssh port, they all stopped. ALL OF THEM. I haven't had an unknown IP attempt to log in via SSH in around five years. fail2ban, strong passwords, or whatever additional security you want to apply is great, and recommended, but a port change will solve more of the problem than you might think.

VPN for SSH (0)

Anonymous Coward | more than 4 years ago | (#31385496)

I've used ISPs that provided separate VPN access for protected ports such as SSH, so you only had to open HTTP to the public. But seriously, there options such as port knockers and the like. Of course, if they are actively managing, they could try to have a firewall that automatically filtered obvious break in attempts.

Working as intended. (1)

Schraegstrichpunkt (931443) | more than 4 years ago | (#31385498)

  • Protocol 2
  • Set PermitRootLogin to "no" or "without-password"
  • ChallengeResponseAuthentication no
  • PasswordAuthentication no

And then just ignore the attempts. fail2ban can sometimes be ok, but be aware that it creates a denial-of-service vulnerability that is exploitable by attackers who can can spoof source addresses.

Re:Working as intended. (1)

MichaelSmith (789609) | more than 4 years ago | (#31385632)

fail2ban can sometimes be ok, but be aware that it creates a denial-of-service vulnerability that is exploitable by attackers who can can spoof source addresses.

Yeah my server went off line a week ago and while debugging the problem I noticed the kernel complaining at startup that a file included from my pf configuration was corrupt. Its a script I wrote which updates that file, not fail2ban. The file was my blocked hosts table.

So from now on I am just going to put up with the dictionary attacks.

Okay... seriously..... (0)

Anonymous Coward | more than 4 years ago | (#31385530)

"Earlier today I was researching some problems on one of our sites and found that there have been over 1 million SSH authentication failures from ~1200 IP addresses on one of our servers over the last year."

What's to cope with? This isn't a problem - this is the internet. Unless servers were actually breached, or it's actually costing you money - why do you care? If your password practices are solid as you say - you should be fine.

People attempting to log in is not a security issue - people logging in is.

How do you manage server security with no system admin? Either pay the extra money for managed hosting where they do this for you, or hire someone to do it for you on staff. If those aren't viable - change your business plan (I mean, your business plan involves administering servers - somewhere along the way you made a decision that administering them yourself was the way to go for your business..... so why would you not hire a sysadmin?)

It happens. Deal with it. (1)

IGnatius T Foobar (4328) | more than 4 years ago | (#31385536)

I hate to sound unsympathetic here, but the Internet is a hostile place. If you have port 22 open to the world, you should expect to get pummelled with password cracking attempts more or less constantly.

Either learn to live with it, or at least take some steps to slow them down. I find that throttling back the number of connections any one IP address can make in a short time pretty much slows it down to a reasonable level. Alternatively, you can also put some "trap" logins on your system. Usernames like "admin" or "oracle" with the password set to the same as the username are good. When the login succeeds, you immediately drop the connection and lock the source IP address out using iptables.

Unless it's really causing a lot of pain, though, please don't harass the ISP from which the connection is originating. You're just going to make some overworked sysadmin's day a little more unpleasant. Chances are, they're already doing everything they can to keep rogues off their network.

Port Knocking (1)

chriswaco (37809) | more than 4 years ago | (#31385576)

Re:Port Knocking (1)

zoid.com (311775) | more than 4 years ago | (#31385602)

Yep.. Port knocking is a great low budget solution to this.

Do nothing (1)

dmiller (581) | more than 4 years ago | (#31385682)

If you are randomly generating your passwords and they are of a decent length then you don't really need to do anything. If your passwords contain lower-case letters only (not recommended), but are eight characters long then your million authentication attempts would represent only a 0.0005% chance of success. If you passwords contain numbers and upper-case characters too, then the likelihood is 1000 times less.

Wrong security concern. (1)

Vellmont (569020) | more than 4 years ago | (#31385694)


I own a small Web development studio that specializes in open source software, primarily Drupal, WordPress, and Joomla

Right off the bat, you're far too focused on the security of least concern, ssh. Unless you set an idiotic password, it's not going to be guessed. It sounds like you have that covered.

But.. the part you've missed is your FAR more vulnerable applications. You say your ISP is "actively managing" your security, but how certain are you they're doing a decent job of it? Do they have experts in each of these different CMS systems?

If you're serious about security, find experts to manage the security settings and versions of each of your chosen applications. I know for a fact that Joomla has historically had some issues with it. The others you mentioned wouldn't be terribly surprising to find problems as well. We're not talking a lot of money here. A good price would be in the neighborhood of $200 per application/site for an assessment of your configuration. Look for people that are reputable, not some fly-by-night "I know all applications" joker who just runs a security scanner and charges you $200.

And no, scanners aren't going to replace expert human beings that know what to look for.

Ignore it (1)

lisany (700361) | more than 4 years ago | (#31385700)

Just ignore it.

Good methods (1)

hmmdar (1130219) | more than 4 years ago | (#31385702)

A good method would be to configure your iptables to drop all packets that come in from a new connection on all ports by default, and only allow through the specific ports you want the outside world to have access to. It is also an extremely good idea to move the ssh port higher than 5k. With the right iptable rules you can configure your system to ignore repeat attacks at the kernel level. And for the love of all that is good do not use passwords use key auth only, and disable root ssh login.

apply patches fastidiously (1)

sneakyimp (1161443) | more than 4 years ago | (#31385738)

Somebody might want to check my math here...my stats are a bit rusty

Ok let's assume a few things:
1) your passwords are 8 chars long and use upper and lowercase alpha and numbers only
2) they are essentially random passwords that are NOT in the dictionary.

This would mean that there are 2^62 possible passwords in your password space. That's 4,611,686,018,427,387,904 possible passwords. The chances of guessing one password in that space within a million random tries is 1,000,000/4,611,686,018,427,387,904. That's about one chance in 4,611,686,018,427. I believe that one in 4.5 trillion is about as remote a possibility as you getting struck by a meteor.

I think it's much more likely that someone will exploit a buffer overflow or sql injection vulnerability on your servers than someone guessing a password. You should make sure that you've applied whatever patches that you can and that any sensitive data traveling to or from your servers is encrypted.

I also like the iptables suggestion -- this means your machine will ignore entire regions of the internet so your server's exposure is at least limited to friendly subnets.

Another useful technique is to prohibit root access to SSH on initial login. This means that everyone has to login with some other username before they can acquire root privileges. For hackers, this means guessing not just a password but also a username that is allowed to login via SSH on the machine.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>