Almost every Linux server we look at has SSH open to the internet, and most of them still run with default settings. This guide is for IT staff and business owners who manage their own servers or VPS. It shows how to tighten OpenSSH in a small, reviewable drop-in file, how to test it without locking yourself out, and how to verify the result. It refers to Ubuntu 24.04 LTS with OpenSSH; Debian works much the same.
Why a drop-in file instead of editing sshd_config
Ubuntu's /etc/ssh/sshd_config starts with Include /etc/ssh/sshd_config.d/*.conf. Your own settings therefore live in a separate file that package upgrades do not overwrite, and you can see at a glance what you changed. Two rules from the sshd_config manual matter here:
- Included files are read in lexical order (alphabetical).
- For each keyword, the first value obtained is used. Later duplicates are ignored.
Together this means that a file named 01-hardening.conf wins over 50-cloud-init.conf, which many cloud images ship and which may set PasswordAuthentication yes. A file that sorts later would be silently ignored for that setting. That is the most common reason a "hardened" server still accepts passwords.
Step 0: Prepare so you cannot lock yourself out
- Confirm that key-based login already works for the account you will use (
ssh <user>@<your-server>without a password prompt for the server). - Make sure this account can use
sudo. - Keep your current SSH session open for the entire procedure. Only close it after you have logged in successfully in a second terminal.
- If available, know where your provider's web console or rescue mode is.
Note: A mistake in sshd settings can lock you out of a remote server. Ubuntu's own documentation warns about this, so test every change before restarting the service.
Step 1: Create the drop-in file
sudo nano /etc/ssh/sshd_config.d/01-hardening.confA solid baseline for a server that is administered by key-holders only:
# Authentication
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitRootLogin no
PermitEmptyPasswords no
# Limit brute-force noise
MaxAuthTries 3
LoginGraceTime 30
# Reduce attack surface
X11Forwarding noWhat each setting does
- PasswordAuthentication no turns off password logins (default:
yes). Attackers can then no longer guess passwords; they need a private key. - KbdInteractiveAuthentication no closes the second door: keyboard-interactive (default:
yes) can otherwise carry password prompts via PAM.ChallengeResponseAuthenticationis a deprecated alias of the same option. - PermitRootLogin no blocks root logins entirely. The upstream default is
prohibit-password, which already forbids root passwords but still allows root keys. Withnoyou log in as a named user and usesudo, which also gives you an audit trail per person. - MaxAuthTries 3 lowers the number of authentication attempts per connection (default: 6). LoginGraceTime 30 disconnects clients that have not authenticated after 30 seconds (default: 120). Both are minor hygiene, not a replacement for disabling passwords.
- X11Forwarding no is already the upstream default, but stating it explicitly documents your intent. Note that distribution defaults can differ from upstream, so being explicit is useful.
Optional: restrict who may log in
AllowUsers accepts only the listed accounts and rejects everybody else. It can also limit a user to source addresses (user@host, CIDR ranges allowed).
AllowUsers <admin-user> <deploy-user>Note: If you set AllowUsers and forget your own account, you lock yourself out. Add it only after the baseline works, and test with a second session.
Step 2: Validate the syntax
sudo sshd -tThe -t test mode checks the validity of the configuration file and the sanity of the keys. No output means no errors found. If it prints an error, fix the file before you go on.
Step 3: Check the effective configuration
Because the first value wins, do not trust the file alone. Ask sshd what it will actually use. The -T flag (extended test mode) prints the effective configuration and includes the checks of -t:
sudo sshd -T | grep -E '^(passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|permitrootlogin|maxauthtries|logingracetime|x11forwarding)'Keywords are printed in lower case. Expected output includes passwordauthentication no and permitrootlogin no. If a value is different, another file wins; list the files with ls /etc/ssh/sshd_config.d/ and rename yours so that it sorts first.
If you use Match blocks, -C lets you simulate a connection, for example sudo sshd -T -C user=<user>,host=<client-hostname>,addr=<client-ip>.
Step 4: Apply and test in a second session
sudo systemctl restart ssh.serviceOn Ubuntu the service is called ssh.service. Existing sessions stay connected. Now open a new terminal and log in. Then confirm that passwords really are refused, forcing password authentication from a client:
ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password <user>@<your-server>You should get Permission denied, without being asked for a password. If anything fails, use your still-open first session to correct the file, then run sudo sshd -t and restart again. Logs are available with sudo journalctl -fu ssh.service.
What this does not cover
- Changing the port only reduces log noise, does not stop targeted attacks, and on Ubuntu 24.04 interacts with socket activation. We deliberately leave it out of this baseline.
- Keys need care too: protect private keys with a passphrase, remove keys of former staff from
~/.ssh/authorized_keys, and prefer one key per person. - Two factors and firewalls: restricting SSH to known IP addresses or a VPN at the firewall level is a stronger measure than any single sshd option.
- Automatic ban tools such as fail2ban are an optional addition once passwords are off.
Checklist
- Key login works for a non-root user with sudo before you start.
- One session stays open until the test from a second session succeeds.
- Drop-in file named
01-hardening.confso that it is read first. - Password and keyboard-interactive authentication disabled; root login disabled.
sudo sshd -tclean,sudo sshd -Tshows the expected values.- Restart with
sudo systemctl restart ssh.serviceand re-test, including a failing password attempt. - Document the change and review authorized keys regularly.
Our take
Key-only SSH with named accounts is the cheapest security improvement for a small server, and it takes about ten minutes when done in a drop-in file with a second session as safety net. At JZ Webservices we treat it as the first step of every Linux server setup, followed by firewalling, updates and backups.