Wiki source code of SSH tunnel

Last modified by Jan Rhebergen on 2023/01/04 15:57
Show last authors
1 = Intro =
2
3 The remote access to 'Beethoven' is realised by means of (an) ssh tunnel(s). This means the traffic is encrypted and access is limited to personnel who's public keys have been copied to their respective accounts on 'Beethoven' (we'll refer to 'Beethoven' as beethoven from now on)
4
5
6 Below you will find a desciption how this was implemented. Basically it boils down to configuring the restricted host to create an ssh tunnel to a middle man, a.k.a. jump server. This tunnel is persistent en kept alive using the autossh tool. One can subsequently login from home on the jump server and connect through to the restricted host. If so configured, it is also possible to be directly forwarded to the restricted host. In our case the jump server is called dashy (117.204.107.243) and is a JISTARC controlled host that is specifically used for cases where we need full internet access. Users that want to make use of the facility described here naturally need an account on dashy.
7
8
9 **NB:** the numbers in the sections below denote the order in which the steps are to be taken!
10
11
12 == Restricted host ==
13
14 This is the host that we want to reach from the outside (e.g. from home). This host is properly firewalled but is able to ssh to the outside world. In our case this is beethoven.
15
16 {{{#JBR lines like this are comments
17 }}}
18
19
20 **step 1**
21 Execute the following sequence of commands:
22
23 (% class="box" %)
24 {{{apt install openssh-server
25 apt install autossh
26 useradd -s /bin/true -m autotunnel
27 su - autotunnel -s /bin/bash
28 ssh-keygen -t rsa
29 ssh-keygen -t dsa
30 ssh-keygen -t ed25519
31 #JBR first make sure account on dashy exists (see step 2)
32 ssh-copy-id autotunnel@dashy}}}
33
34 This installs the ##autossh## program, adds the ##autotunnel## user which will **not** have a default login shell. To still be able to execute commands as ##autotunnel## we have to specify a shell to execute, i.e. ##/bin/bash##. This enables us to generate the keys and copy them to the middleman server ##dashy##. The ##ssh-copy-id## command copies the most recent ##id_*.pub## file. As of recent the ##ed25519## is considered safest (and quick). **NB:** the ##autotunnel## account password needs to be known (temporarily set) see step 2. **NB:** The last command of step 2 can only be executed when step 2 below has been completed! Also temporarily enable logins using a password  by setting ##PasswordAuthentication yes## in the ##/etc/ssh/sshd_config## file and restart the daemon (don't forget to restore original setitngs and restart again).
35
36 **step 3**
37
38 This step is to test some aspects of the setup:
39
40 First try a regular ##ssh## session:
41
42 (% class="box" %)
43 {{{ssh autotunnel@dashy}}}
44
45 This should work without supplying a password and relies on the public keys we copied in ##step 1##. Exit this after determining that it works. Now start an ##autossh## one:
46
47 (% class="box" %)
48 {{{autossh -M0 autotunnel@dashy -N -R 8082:localhost:22 -vvv}}}
49
50 Due to the ##-vvv## option this will cause a lot of output on screen. Switch over to the Middleman server execute ##step 4## and try to connect.
51
52 == Middle man ==
53
54 This is a server somewhere on the internet that we control. In our case this is ##dashy##.
55
56 **step 2**
57
58 (% class="box" %)
59 {{{useradd -s /bin/bash -m autotunnel
60 #JBR generate a password
61 pw=`date|md5sum|cut -c1-10` && echo $pw
62 #JBR sets the password for the autotunnel user
63 echo -e "$pw\n$pw"| passwd autotunnel}}}
64
65 Next edit the## /etc/ssh/sshd_config## file to include the
66
67 (% class="box" %)
68 {{{GatewayPorts clientspecified
69 }}}
70
71 line. This enables the client to specify an IP address from which connections to the port are allowed (it proved crucial in getting the tunnel to work like I wanted). Next we need to restart the ssh service for this to take effect.
72
73 (% class="box" %)
74 {{{systemctl status sshd.service
75 }}}
76
77 Make sure the firewall allows connection on port 8082 (or whatever port you chose).
78
79 (% class="box" %)
80 {{{ufw allow 8082
81 }}}
82
83
84 **step 4**
85
86 There are two ways to test the connection. First is simple and aimed at finding out if it connects at all (if you have a telnet client installed).
87
88 (% class="box" %)
89 {{{telnet localhost 8082}}}
90
91 This should generate some output on the restricted client due to the ##-vvv## option we supplied. When this indeed is the case just quit and commence with the following actual test.
92
93 (% class="box" %)
94 {{{ssh -p 8082 localhost
95 #JBR or alternatively for instance:
96 ssh -p 8082 jan@localhost
97 }}}
98
99 This should log you in on the restricted host. If you need to supply a password it means you did not copy your own public keys or the permission of the ##authorized_keys## file in set incorrectly. These keys are different (personal) public keys that you should have on your home pc/laptop account. Again this should generate respective verbose output on the restricted host. When succesful you can logout on the middle man host and also stop the ##autossh## running on the restricted host. If you cannot login from an outside host (not localhost) it might be because the provider that hosts your middle man server has a firewall that need to be adjusted (i.e. amazon, oracle, etc).
100
101 == Automation and persistence ==
102
103 In step 1-4 we have demonstrated the sshtunnel. Now we need to have it start automatically and make it persistent. This can be accomplished using appropriate systemd scripts on the //**restricted host**//, as indicated below.
104
105 Put the script below in ##/etc/systemd/system## and name it ##autossh-dashy.service##
106
107 (% class="box" %)
108 {{{[Unit]
109 Description=AutoSSH tunnel service to dashy
110 After=network-online.target
111
112 [Service]
113 Restart=always
114 RuntimeMaxSec=86400
115 Environment="AUTOSSH_GATETIME=0"
116 ExecStart=/bin/su -s /bin/bash autotunnel -c 'autossh -M 0 -q -N -C -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -o "ConnectTimeout 10" -o "ExitOnForwardFailure yes" -R *:8082:localhost:22 autotunnel@117.204.107.243'
117
118 [Install]
119 WantedBy=multi-user.target
120 }}}
121
122 Then activate it and start it by using the following commands:
123
124 (% class="box" %)
125 {{{systemctl enable autossh-dashy.service
126 systemctl start autossh-dashy.service
127 }}}
128
129 We also want to do the same to the web related ##ports 80,443##. Put the script below in ##/etc/systemd/system## and name it ##autossh-dashy-web.service##
130
131 (% class="box" %)
132 {{{[Unit]
133 Description=AutoSSH tunnel for web-service to dashy a.k.a. zij.informeer.de
134 After=network-online.target
135
136 [Service]
137 Restart=always
138 RuntimeMaxSec=86400
139 Environment="AUTOSSH_GATETIME=0"
140 #JBRv this is different than the other autossh-dashy script because it is run as root!
141 #OFF ExecStart=autossh -M 0 -q -N -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -o "ConnectTimeout 10" -o "ExitOnForwardFailure yes" -R :80:localhost:80 -R *:81:localhost:81 -R *:443:localhost:443 root@zij.informeer.de
142 #JBRv the entry for port 81 is only needed once and can be disabled after the admin has secured and configured the access over https
143 ExecStart=autossh -M 0 -q -N -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -o "ConnectTimeout 10" -o "ExitOnForwardFailure yes" -R :80:localhost:80 -R *:443:localhost:443 root@117.204.107.243
144
145 [Install]
146 WantedBy=multi-user.target
147 }}}
148
149 Then activate it and start it by using the following commands:
150
151 (% class="box" %)
152 {{{systemctl enable autossh-dashy-web.service
153 systemctl start autossh-dashy-web.service
154 }}}
155
156 **Important!**
157
158 * For this to work ##ports 80,443## **cannot** be occupied by other processes.
159 * Because these are privileged ports it needs root access! (hence the ##root@zij.informeer.de## entry above)
160 * The root public keys need to be copied to the middle man server (i.e. ##ssh-copyid## or manually into ##/.ssh/authorized_keys##)
161 * The host fingerprint needs to be accepted. This can be achieved by fist executing a manual ##ssh login## from the restricted host to the middle man server and manually accept the fingerprint when prompted.
162 * We initially open ##port 81## because this is the ##admin port## of ##nginxproxymanager##. This can be disabled (commented out) after ##nginxproxymanager## has been set up (webinterface https)
163 * When you (temporariy) edit/change the script you need to restart thedaemon i.e.: ##systemctl daemon-reloa##d
164
165 == Troubleshooting ==
166
167 Usually the SSH tunnel works fine. Problems can potentially occur when disconnecting and reconnecting the hosts ##liszt## and ##beeethoven##. Typically issues consist of:
168
169 * authentication problems (i.e. wrong keys of missing keys)
170 * connection problems (i.e. ports that are occupied or misdirected, unintended fail2ban jails)
171
172 Sometimes problems can simply be solved by restarting the ssh daemon on ##dashy## and the ##autossh## daemon that maintain the ssh tunnel(s) in ##liszt## and ##beethoven##.
173
174 (% class="box" %)
175 {{{#JBR on beethoven and liszt
176 systemctl restart autossh-dashy.service
177 systemctl restart autossh-dashy-web.service
178 }}}
179
180 (% class="box" %)
181 {{{#JBR on dashy
182 systemctl restart sshd.service
183 }}}
184
185 When using ##ssh## and entering a wrong password multiple times in quick succession ##fail2ban## (temporarily) blacklists the originating host IP. This so called jailing can be undone if you can login by another means (i.e. from a different host). Alternately one can permanently **whitelist** a specific trusted host to prevent an unintended lock-out situation. **NB:** ##fail2ban## naturally runs on ##dashy## the middleman server.
186
187
188