I recent­ly told you about the choic­es you have to make for your sysad­min stack. In fact, these choic­es impact your whole tech­ni­cal chain, from your OS and its set­up to the sys­tem archi­tec­ture. It means you have to adjust the cur­sor between main­tain­ing each ele­ment by your­self or rely­ing on a part­ner to man­age them for you.

Securing remote access goes from triv­ial things to less-under­stand­able points for non-experts. Consider this arti­cle as pri­ma­ry guid­ance, and feel free to browse some links to more in-depth arti­cles1)2).

Cat GIF @Giphy

TL;DR: secu­ri­ty is a vast and com­plex realm of knowl­edge, and may be com­plex. If you only want to know how to improve your remote access set­up to always­da­ta servers, read the sec­tion about using secured meth­ods.


Don’t let anybody enter your home

You run your appli­ca­tion on remote servers. It allows you to pro­vide access to your ser­vice from any­where. There’s usu­al­ly no phys­i­cal access for you to those machines3). So, you use remote access to those machines. Most of the time, it’s through a remote shell, some­thing like SSH. What SSH does is to pro­vide you access to both a shell and a low-lev­el files-man­ag­er on your remote machine. Consider it as a door to your liv­ing room. Not tak­ing care of it is a non­sense. Never do this at home, kids.

An impor­tant secu­ri­ty con­cern is the way you man­age remote access to your pro­duc­tion servers. Here too, you may want to do it, or you may pre­fer to use a ser­vice that embeds a secured set­up for you. At always­da­ta, we think that using a ser­vice that does it for you is often the best choice, but some parts remain for you to han­dle. Because this part is some­times dif­fi­cult to under­stand and to main­tain, we want to give you some advice on how to secure your access cor­rect­ly.

The right lev­el of secu­ri­ty is the one you’re able to under­stand and to man­age. So, to make sure you’ve got all the exper­tise need­ed to be safe at home, let’s talk a bit about what you have to know to pro­tect your remote access and to avoid giv­ing any­body access. My goal is not to turn you into a secu­ri­ty expert; it is mere­ly to ensure you know the basics (and prob­a­bly a bit more) about secu­ri­ty and why it mat­ters.

Fortunately, com­mon SSH servers come with a default con­fig­u­ra­tion most­ly secured. OpenSSH, which is the default SSH serv­er in most plat­forms, does it. But dis­trib­ut­ing a default con­fig­u­ra­tion means that main­tain­ers need to address the com­mon use-cas­es rather than the high-lev­el secured set­up. This set­up will prob­a­bly lock out 90% of the mali­cious users. Think of it as adding a stan­dard robust door to your man­sion; it is bet­ter than noth­ing, but it is still not an armored door. We want to con­sid­er how to strength­en this default set­up.

Be Safe, Be Smart

Securing its remote access implies to take care of a few dig­i­tal con­cerns, main­ly because you stay in charge of the client parts. Even if you have a prop­er­ly secured serv­er, you may get into trou­bles if your client-side pro­tec­tion is too weak. Let’s take a look at your respon­si­bil­i­ty.

Use SSH Keys to connect

We would rec­om­mend using SSH key rather than pass­word authen­ti­ca­tion for SSH remote access. This way, you can effi­cient­ly man­age who can access your remote serv­er, on which device, and revoke access in case of emergency4). It’s a good idea to have one pair of keys5) for each user on each device. It gives you more flex­i­bil­i­ty to revoke access when a device is lost or stolen, instead of revok­ing the com­plete user access.

To know how to gen­er­ate a secure, updat­ed key, take a look at the ed25519 key for­mat sec­tion.

Avoid password connection

Passwords can’t be secured enough to be the only pro­tec­tion against mali­cious access. You should use a key-based login as men­tioned above to make sure that your remote user is pro­tect­ed, e.g., from a brute-force attack that will try to crack your user’s pass­word.

In always­da­ta, the SSH con­nec­tion using pass­word is dis­abled by default, and only con­nec­tions using SSH keys is allowed. You must have to write your pub­lic key in the ~/.ssh/authorized_keys file manually6). If you need to enable pass­word-access nonethe­less (e.g., to trans­fer your pub­lic key first), check the box to autho­rize con­nec­tion using pass­word. Please keep in mind that it’s a less-secured method, so turn it off when you don’t need it.

If you run your per­son­al serv­er, after copy­ing your pub­lic key on the serv­er, dis­able pass­word-based login in sshd con­fig­u­ra­tion.

It also seems that some users/administrators think that not pro­tect­ing their accounts with a pass­word is a good idea. Personally, I think that even the weak­est pass­word is bet­ter than no pass­word at all. In any case, you must avoid allow­ing those no-pass­word users to log in remote­ly with­out any pro­tec­tion.

We for­bid emp­ty pass­words on always­da­ta servers.

Use strong passphrases

Using strong keys is essen­tial, but you can’t rely on them with­out secur­ing them. It means that each SSH key must be passphrase-pro­tect­ed with a unique, ded­i­cat­ed, reli­able pass­word. It also applies to your user’s login access, so feel free to use the same method to set up your user’s pass­word.

I have two meth­ods to gen­er­ate ran­dom passphras­es in CLI7):

You should have openssl rand or gpg (or both) installed on your sys­tem. Make sure to check your pass­word resilience against brute-force attacks with the How Secure Is My Password tool. You don’t want a weak pass­word to com­pro­mise your whole busi­ness secu­ri­ty.

Use cryptographic hardware

If you’re con­cerned about secure remote access, you should con­sid­er using cer­tifi­cate client authen­ti­ca­tion cou­pled with a hard­ware device, so you keep it safe from attack­ers and unac­ces­si­ble from your com­put­er. You can then use some cryp­to­graph­ic hard­ware device like Nitrokey Pro or YubiKey 4 to store your keys/certificates and man­age the client authen­ti­ca­tion for you.

@Giphy

Stay up-to-date

Keep your SSH server updated

SSH secu­ri­ty breach­es are reg­u­lar­ly dis­cov­ered, and patched quick­ly by the main­tain­ers and con­trib­u­tors of the projects. Keeping your serv­er updat­ed is nec­es­sary. It also implies you keep your SSH client up-to-date too. As I’m writ­ing this blog post, the cur­rent OpenSSH ver­sion is 7.6, released Oct. 3, 2017. Check your cur­rent ver­sion:

Use SSH v2 protocol

SSH comes in two ver­sions of its pro­to­col. The cur­rent, mod­ern one, is the v2. It is more secure, more flex­i­ble, and even more-encrypt­ed in all steps of the process. Almost every client has sup­port­ed the v2 for a long time, so feel free to dis­able v1 and only serve the lat­est if you run your serv­er.

For lega­cy rea­sons, the v1 was still avail­able on always­da­ta servers. Since the release of OpenSSH 7.6, the v1 pro­to­col is now removed from the code­base (unavail­able even in com­pi­la­tion flags), so stay up to date.

Rely on strong SSH Key with ED25519

You can use sev­er­al key for­mats for your SSH Keys. Preferably, you should use the ED25519 algo­rithm, as it is the best choice nowa­days regard­ing secu­ri­ty. Yes, it’s rel­a­tive­ly new, but it’s well sup­port­ed in pro­duc­tion right now, and as you man­age the serv­er side (or rely on a trust­ed, ded­i­cat­ed part­ner), com­pat­i­bil­i­ty may not be a big deal.

It will:
 — gen­er­ate a new key
 — -a set the num­ber of rounds for the func­tion deriva­tion to 1000, which increas­es secu­ri­ty but also the login time. If a quick login is crit­i­cal to you, con­sid­er decreas­ing this val­ue (the default is 16; 100 offers robust pro­tec­tion against brute-force attacks)
 — -t force the for­mat to use the ed25519 algo­rithm
 — -C iden­ti­fy your key with your username/device

Network protection

This sec­tion con­tains advanced tips for a sysad­min peo­ple that want to high­ly secure their envi­ron­ment. They may or may not apply, depend­ing on your con­text and your threat mod­els. In fact, the for­mer sec­tions cov­er most use-cas­es of remote access. But you may want to rein­force your secu­ri­ty.

Firewall the SSH TCP Port

If you always con­nect from a known set of ded­i­cat­ed IPs to your servers, so it’s use­less (and dan­ger­ous) to let any incom­ing con­nec­tion try to log in. You should restrict SSH login to known address­es in your fire­wall rules.

Here, we allow con­nec­tions from the IP 10.111.111.10 (you have to use your own, obvi­ous­ly) to the SSH port 22 in TCP only on the local machine address (192.168.1.12) using ipt­a­bles8). Firewalling can be hard, but it’s one of your stronger defens­es against attack­ers.

Some of our machines, which han­dle parts of always­da­ta archi­tec­ture not aimed to be exposed pub­licly are IP fil­tered like that. If you’ve got a VPS or Dedicated Server offer, you can acti­vate IP Filtering in your admin pan­el to ensure only your IPs are avail­able to con­nect to your servers.

Blacklist SSH crackers and brute-force attackers

One of the prob­lems with attack­ers try­ing to get access to your machine is that they will request access a huge num­ber of times. Even if they don’t get access, it will over­load your SSH serv­er, maybe slow down your sys­tem, and per­haps result in DDoS. To pre­vent that, you can use your fire­wall to ban the attack­ers’ IPs. Indeed, you won’t black­list them man­u­al­ly. Instead, you can rely on tools like fail2ban, DenyHosts, or a cus­tom rule in ipt­a­bles or ufw to tem­porar­i­ly ban them.

In always­da­ta archi­tec­ture, SSH servers are pro­tect­ed by a fail2ban fil­ter.

Change SSH port and limit port binding

OpenSSH lis­tens to port 22 by default. It makes it vul­ner­a­ble to remote machine attacks because attack­ers will first try this port to tar­get SSH breach­es. If you don’t care about com­pat­i­bil­i­ty and use machines you’re com­fort­able with, don’t hes­i­tate to change the default port and the bind­ing address­es to which the servers are lis­ten­ing to to decrease the attack sur­face.

Port knocking

This is one of the most fun tips.

It’s a method that forces remote users to try to con­nect to a series of ports to trig­ger a rule in your fire­wall that will open the access for its IP to the SSH port. By default, the fire­wall rejects con­nec­tions to the SSH port. Using a ded­i­cat­ed tool (knock), you will con­tact your serv­er to a sequence of ports. If the series is cor­rect, the fire­wall is updat­ed, and a new rule is added to allow only your client IP to con­nect through SSH. Simple, smart, efficient9)). It uses the knockd tool.

Misc

Here’s our last tips sec­tion. Congrats if you read the arti­cle so far! It presents some extra set­up avail­able to increase a lit­tle bit your secu­ri­ty. It’s often use­less, but it’s a good know-how.

Trust SSH Host Keys

To be trust­ed by clients, SSH serv­er presents a key that allows you to authen­ti­cate it. This key is gen­er­at­ed auto­mat­i­cal­ly and belongs to the serv­er only. There’s one key per sup­port­ed algo­rithm, and they are pre­sent­ed to the client depend­ing on the set­up. You can com­pare this key at your first con­nec­tion with the one pro­vid­ed in your SSH admin sec­tion.

As mod­ern clients sup­port the ED25519 algo­rithm, your serv­er can serve only this one and dis­able oth­ers to force its use.

In always­da­ta set­up, we still keep all for­mats avail­able for com­pat­i­bil­i­ty rea­sons.

Disable root access

Your root user is the super admin user in your *nix sys­tem. It is the one allowed to do every­thing on the machine, even wipe it entirely10). So, it should be used _carefully_. Most of the time, you do _not_ need to be root­ed to do what you want, and your sys­tem per­mis­sions must allow you to exe­cute your com­mands safe­ly. So, it is a good idea to dis­able the root user access remotely11). This way, even if a mali­cious user gets access to the serv­er, that user won’t be able to gain root priv­i­leges eas­i­ly.

Idle log out timeout

It can be a good idea to lim­it the time a user can be main­tained in your sys­tem with­out any activ­i­ty, to pre­vent lat­er mali­cious use. Decrease the num­ber of idle con­nec­tion allowed and the inac­tiv­i­ty time to make sure nobody can stay longer than need­ed.

Chroot users

Users logged in remote­ly can access the whole sys­tem, sand­boxed by the sys­tem and files per­mis­sions. Chroot users in their home direc­to­ry could be a good idea. Unfortunately, it’s not as triv­ial as it seems, and it may require rely­ing on tools like rssh. It may be a com­pli­cat­ed set­up, but if your threat mod­el implies a risk of data leak­age, it’s a good idea.

wonder woman shield GIF @Giphy

I hope this long blog post allowed you to under­stand a lit­tle bit more about what mat­ters regard­ing remote access secu­ri­ty. As it relies on cryp­to­graph­ic strengths, it may be hard to man­age the whole secu­ri­ty of your stack by your­self. Pieces of advice here are what they are: safe­ty tips to increase your secu­ri­ty. Please nev­er under­es­ti­mate how hard it is to keep a whole sys­tem safe, and nev­er begrudge work­ing with com­pe­tent part­ners who are there to help you. One of our job at always­da­ta is to do it for you. We ensure the cur­sor is cor­rect­ly adjust­ed to give you flex­i­bil­i­ty with­out com­pro­mis­ing secu­ri­ty.

If you want to talk a bit about secu­ri­ty, I’ll be at Rennes next Mar. 29, 2018 for the Breizhcamp con­fer­ence to talk about cryp­tog­ra­phy and devel­op­ment. See you there!

Notes   [ + ]

1. duck­duck­go­ing/qwant­i­ng the miss­ing resources is also a great way to get more information
2. feel free to ask ques­tions in the com­ments 🙂
3. and you gen­uine­ly don’t want to run across the world to con­nect and repair your sys­tems
4. or when your col­lab­o­ra­tors quit
5. public and pri­vate
6. we’ll soon pro­vide a solu­tion in your admin pan­el to fill your pub­lic key right there
7. store them secure­ly in a pass­word man­ag­er, of course
8. you can also take a look at ufw to write your fire­walling rules
9. because we can :
10. never, ever, rm -rf / blind­ly
11. it is also a good idea to dis­able it local­ly and always use sudo to exe­cute root com­mands