Access your Linux system using SSH

Working on a server without SSH is a nightmare, so let’s set it up properly.

You really want SSH access when you’re working on a remote headless Linux system (meaning it has no desktop environment). I’ll use KiTTY here, which is a fork of PuTTY, which is the quasi-standard for SSH clients. They look almost identical, so everything I describe here works for both. Since using shells and SSH is a native concept and thus quite easy on Linux and also Mac OS (because it has BSD roots), I’ll only mention them briefly here and focus on Windows.

I also wrote a separate guide on how to get started with a Hetzner VPS and Termius, because the latter is a good modern mobile alternative to the powerful but dated established desktop clients. It’s available for all major desktop and mobile platforms (with a clear focus on mobile) and free to use in the basic version.

If you’re using a leased server and your provider offers to generate authentication keys for you, or you don’t want to use usernames and passwords and create those keys yourself, you can skip this part of the guide.

The most basic way to use SSH is by entering a combination of username and password. Your provider should have sent you the IP address of your system, as well as the login information. Enter the IP address in the Host Name (or IP address) field:

That’s all if your username is root. If it’s something different, navigate to Connection > Data in the left menu and change the Auto-login username accordingly. If you leave it blank, you’ll be asked for it on connect.

That’s it. You will be asked for the password; there is no way to save it here, as the developer considered that too much of a risk. If you want to save the changes you made, head back to the Session main dialog and enter a name for your session in the Saved Sessions/New Folder field, then click Save. If you leave the field blank, instead select Default Settings from the list and click Save, these will become the default settings which are automatically loaded when the client starts.

You can connect by clicking Open now.

Before I proceed with the more advanced (and secure) concept of public key authentication, I’ll show you how to upload files to your server, because that’ll be one of the methods you can use to setup your keys then.

If you’re using WinSCP, choose the Commander user interface when asked for it during installation. Then start it and you should be presented with a login dialog. You can use SCP or SFTP as File protocol, I prefer SFTP but it doesn’t really matter here (see this comparison). Note that SFTP has nothing to do with the ancient insecure FTP protocol (and it’s more secure form FTPS) where you had to think about firewalls and passive/active modes.

Anyway, enter your IP address, username and password again and you’re good to go; it should be self-explanatory which goes where. You can even save your password here, as the developer decided that it’s your own choice if you want to do so.

If you’re using FileZilla, you can enter your login information in the Quickconnect bar:

Note that you have to explicitly set the Port to 22, or the program will try to use the legacy FTP protocol which won’t work (hopefully, better change your provider if it does).

You can also access the Site Manager from the File menu and save your data as a session for future use. Click New Site and enter a name for the session, then select the SFTP Protocol. There’s no need to specify a Port this time, as SFTP implies port 22 already. Choose Logon Type normal.

My session list is blank here as I already had some sessions in it, which I obliterated for obvious reasons.

You already know how to access the shell of your server using SSH now, as well as how to upload files using SFTP (or SCP). But you’re not doing it in a secure and comfortable way yet. So, let’s create a key pair!

I’ll skip instructions on how to do it using ssh-keygen on the console here, as that doesn’t require key conversion and is pretty straightforward. If you’re using Linux or a Mac and don’t know how to do it already, follow this guide. It also covers the ssh-copy-id command, which puts the public key on your server.

If you’re using a leased server, some providers will offer to generate a key pair for you. If this is the case, you can skip the following two sections. The public key will usually be automatically provisioned to the system and you can simply use the provided private key to access it. This is fine; if he wanted to, your provider could get full access to your server anyway, so don’t worry about him possibly keeping a copy of your private key.

If you’re using Windows, download PuTTYgen here (search for puttygen.exe), which is the PuTTY Key Generator tool, and start it. It’ll look like this:

You could increase the Number of bits in a generated key now (or use a different key type altogether, eventually you’ve heard about elliptic-curve cryptography already), but 2048 Bit RSA is really strong enough for the foreseeable future and the given use case.

After you click Generate, you’ll be asked to move around the mouse over the window for randomness (also called entropy). You need some chaos to generate strong cryptographic keys, or they’d be too easy to guess. Your computer isn’t really good at creating chaos on its own, since all it can do is flip 0s and 1s really fast, so it needs your help here. After the key generation algorithm is done, you’ll see something like this:

You can change the Key comment to something more recognizable if you want to, that makes it easier to know which key is for what later on. You can put your username or email address there, or something completely different.

Select everything in the upper field which is captioned Public key for pasting into OpenSSH authorized_keys file (right click > Select All), press Ctrl + c to copy it and then paste it to a file called authorized_keys, as the description says. Simply use a text editor of your choice (like notepad++). It should end up as a single line without breaks, starting with ssh-rsa and ending with your Key comment. Do not use the File > Save public key option, as that’ll save the key in a different format you won’t be able to use!

You have saved the public key now, but you still need the matching private key.

If you enter a Key passphrase, your private key will be encrypted. That’s an additional line of defense in case someone gets access to your key file. If you don’t set a passphrase, everyone who has the file can gain full access to your system. Not setting a passphrase is more comfortable, but very insecure of course. I’ll show you a way to have both security and comfort further down, so the preferable method is to set a passphrase here.

Click Save private key and choose a safe place, especially if you didn’t set a strong passphrase. So please don’t put it in your Dropbox or Evernote.

You created both a public and private key now, so you can close PuTTYgen.

The next step is to put the public key on your server. If you’re using a leased server, your provider should offer an option to save your public key in the management interface, which will then be provisioned to every new instance you create. Usually you have to paste it somewhere. This is what it looks like for Vultr:

If you already have a running instance, or your provider doesn’t offer that function, you have to copy the key manually.

When you login via SSH using public key authentication, the server checks if the following file exists and contains your public key:

~/.ssh/authorized_keys

~ is your home directory on Linux. If you login as root, it’s /root. If you login as some other user, it’s usually /home/<user>.
.ssh is a hidden directory because it starts with a dot, meaning it’s usually not shown in directory listings by default. This also works for files, it’s pretty much the same as the hidden files and folders function on Windows.
authorized_keys contains the public keys authorized to access the system for the given user, meaning the one that the home directory it’s located in belongs to, one per line.

You have two main options here, both of which involve logging into the system using one of the methods described in the previous sections.

a) upload the authorized_keys file you saved during the key generation process using one of the graphical tools you already used earlier. I’m using WinSCP in this example, it works almost identically with FileZilla though.

Make sure you’re in your home directory after logging in. If you don’t see a .ssh subdirectory there, create it via right click > New > Directory. Enable the Set permissions checkbox and make sure they’re 0700, meaning only the current user can access it. It should look like this:

Now enter the .ssh directory and upload your authorized_keys file to it. If you can’t figure out how to do that because you don’t see your local folder structure on the left side of the window, navigate to Options > Preferences > Environment > Interface and select the Commander user interface. After restarting WinSCP, it should be obvious.

Make sure the uploaded file has the correct permissions by right clicking it and viewing its Properties. The permissions should be 0600 in this case:

b) you can also create or modify the authorized_keys file from the shell. Login using KiTTY or PuTTY first, then execute the following command to create the .ssh directory with the right permissions, so only you have access to it. If it’s already there, nothing will happen, so it doesn’t do harm in any case:

mkdir -p -m 0700 ~/.ssh

There’s a plethora of ways to create or modify the authorized_keys file now, I’ll go with the most user friendly approach here and use a simple text editor.

nano ~/.ssh/authorized_keys

If you get a command not found error, install nano using the system package manager first. On Ubuntu or Debian, that’s as easy as apt-get install nano.

You should see an empty new file now, unless it already existed and contains some public key(s). Go to the beginning of the first blank line using the cursor keys if it’s not empty, then paste the public key from your local authorized_keys file. To do so, open it with a text editor like notepad++, select the single line it should contain, press Ctrl + c to copy it, and then paste it remotely by right clicking on the console window, as Ctrl + v won’t work here.

Now save the file by pressing Ctrl + x and then Enter. Make sure its permissions are correct:

chmod 0600 ~/.ssh/authorized_keys

If you’re using Linux or Mac OS, the process is simpler. Please take a look at the end of this guide for instructions on how to use the ssh-copy-id command.

That was the complicated part. You can now finally use your private key to access the system. I’m adding an intermediate step here, which you can skip if you want the real deal when it comes to authentication. This is only the simplest way to use your keys.

a) if you’re using KiTTY or PuTTY, navigate to Connection > SSH > Auth in the left menu and select your key file in the Private key file for authentication field.

b) if you’re using WinSCP, go to the Advanced menu in the login dialog, navigate to SSH > Authentication and select your key file in the Private key file field.

c) if you’re using FileZilla, go to the Site Manager and change Logon Type to Key file, then select your key file in the Key file field.

Alternatively you can open the Settings from the Edit menu, navigate to Connection > SFTP and add your key file there.

No matter which client you use, always remember to properly save your sessions, or you have to change that setting each time.

If you protected the private key with a passphrase, you’ll be asked for it when you establish a connection. Otherwise you’ll login instantly.

Further up I said you can have both security and comfort, so let’s go one step further and use an authentication agent. You can use either KiTTY agent (kageant.exe) or PuTTY agent (pageant.exe).

After you downloaded and started one of the agents, it’ll sit in your system tray (the area next to the clock in the taskbar). Watch out for this icon:

Right click it, select Add Key from the menu and load your private key file. If you protected the private key with a passphrase, you’ll now be asked for it.

That’s it, you have an authentication agent up and running!

…so what? Well, for one you don’t have to enter your passphrase on every consequent login attempt. Every client that supports agent authentication will use the already decrypted private key the agent provides, unless that feature isn’t enabled of course.

For KiTTY and PuTTY, make sure that Attempt authentication using Pageant and Allow agent forwarding (I’ll come to that later) are enabled where you selected your private key earlier.

You can remove the private key here, as that will be handled by the agent.

It’s almost the same for WinSCP:

Nothing has to be changed for FileZilla, because the feature is always enabled there.

You can add even more comfort by putting the agent in your Autostart folder, so that it loads each time you login. To do so, press Win + r to access the Run command dialog, then type shell:startup and hit Enter.

Your Autostart folder should open. Right click into it and select New > Shortcut. Insert the full path to your kageant.exe or pageant.exe, directly followed by the full path to your private key file. It should look something like this, but with your actual paths and private key’s file name of course:

C:\Users\Iruwen\kageant.exe C:\Users\Iruwen\privkey.ppk

Simply create the shortcut and you’re done, the agent will now start automatically on every login.

But wait, there’s more! You’re done with the configuration part here, but remember that Allow agent forwarding setting? That’s the real killer feature of using authentication agents, especially when you’re working on several different servers. I’ll use two of my test systems and MobaXterm here, which is the client I’m normally working with.

It might not be instantly obvious what’s happening here and what’s so great about it. I started two separate SSH sessions to different systems in split screen mode, so you can see both.

First I looked up and copied the external IP address of fusion-small on the right, because I wanted to copy a file over there from fusion-tiny on the left.

Without the agent running I’ve been asked for my password, because fusion-tiny doesn’t know anything about the private key required to access fusion-small. Then I started the agent on my local system, and suddenly I could copy the file over without being asked for a password.

That is because of agent forwarding. You’ll almost never have to think about authentication again when you’re working with multiple systems, because requests for your private key are passed on between systems, even across multiple jumps, so you can chain SSH connections from system to system.

Want to sync two directories between your systems? Just do it!

rsync -a --del -e ssh /somelocaldir/ 203.56.113.178:/someremotedir/

While this is all great, there’s also a downside: using agent forwarding comes with a security caveat, because it creates a socket which can be intercepted by anyone with sufficient permissions. In my opinion this is a minor issue and the alternative isn’t as comfortable as the author of this article makes it sound, but if you want to be super safe, only use the feature with hosts you trust, like a set of VPS where you’re the only user.

There’s one more SSH feature I’d like to show you: tunnels. A tunnel in general is simply a transmission channel where data enters on one side and exits somewhere else on the other; you probably have heard of or even used VPN tunnels already. You can use tunnels to access services you otherwise couldn’t access because of firewall limitations or geoblocking for example.

To create a tunnel, start KiTTY or PuTTY (it should work with every decent client, but I can’t explain it for all of them of course) like you’d normally do, but before you actually connect, navigate to Connection > SSH > Tunnels in the left menu. You’ll normally want to open an unused port locally on your machine and forward it to some remote port.

Imagine you want to access a service running on your Linux server. The service is listening on port 9001 on the server’s loopback interface (172.17.0.2) only, so it’s not reachable from the outside.

First enter 9001 as Source port, which is the port you’re gonna connect to on your local system. It doesn’t have to be the same as the remote port, but it has to be unused.
Then enter 172.17.0.2:9001 as Destination, which is the listening IP address and port on the remote system. If the service was listening on the loopback interface (localhost), this would be 127.0.0.1:9001. You could also enter an external IP address, then your server would be used as a gateway to jump to some completely different remote location.
Then click Add to add that to the list of forwarded ports:

If you enable Local ports accept connections from other hosts here, other systems in your local network can access the tunnel too.
If you change Local to Remote under the Destination field, you reverse the direction of the tunnel, so a remote system can access a service on your local system.

You can now Open the connection to the server. A normal SSH shell session appears, but there will also be a tunnel running in the background.

You can check that it worked by executing a command on the Windows command prompt. To do so, press Win + r to access the Run command dialog, then type cmd and hit Enter. A console window will appear, where you can use this command:

~# netstat -na | findstr 9001

You should get a result like this, meaning port 9001 is open and ready for connections on your local machine:

TCP    127.0.0.1:9001         0.0.0.0:0              LISTENING

You might see more lines, but one of them should say LISTENING. If it’s not there, something went wrong.

If you’re using Linux or Mac OS, the shell command achieving the same result would look like this:

~# ssh -L 127.0.0.1:9001:172.17.0.2:9001 -l <user> <host>

Where <user> is your login and <host> is the IP address of your server.

My Telegram username is @Iruwen, you can find me in CoinMarketCap’s great international community where we discuss general crypto topics. I’ll gladly try to help you out with technical questions of all sorts.

If you think this was helpful in any way, feel free to buy me a beer!

ETH: 0x0afAB9b6dA9FBb79f3260F71E4a17d4AF9AC1020
BTC: 16yAtsdjzEaQbH8ucK1nbtkqrpo791EZ7a