My Issue: I have a Ubuntu VM running on my Windows 11 via HyperV virtualization. I cannot SSH into that Ubuntu VM because its using the default networking which creates a NAT. I have the option to create an external interface via HyperV to resolve that issue, however, it is having a hard time creating that interface. I currently use HyperV KVM console to access the Ubuntu VM, which is very limiting as I have to use the UI, I can’t copy-paste between and its a small resolution.
Goal: Setup Reverse SSH tunnel from my Ubuntu VM to a Linux host in my network (or any network). This linux host is called starlord. Then I will be able to SSH into my Ubuntu VM thru starlord. This by passes the tunnel and the need to create an external interface.
{internet}
|
[ My Win11 PC ] ------ [Switch] ----- [Starlord Linux NAS]
[has Ubuntu VM]
Note, with the current default HyperV settings the Ubuntu VM is unreachable by anything in the network but it can reach anything (my Windows PC, the Switch, Starlord, and the internet).
This article was written in mind for the topology & hostnames above, however it can be applied to any similar layout. The Ubuntu VM can be any server you are trying to reach that’s behind a NAT or firewall (that at least allows outbound connections). Starlord is reachable SSH host, which can be reached by Ubuntu and other locations.
Proof of Concept & Prepare Linux host
Before creating the scripts that kick-off the tunnel. Lets do a proof of concept and make sure we can connect up like this, with some simple linux command.
Step 1. Make sure that the linux host (starlord) has “GatewayPorts yes” in /etc/ssh/sshd_config. Then restart the sshd service:
vi /etc/ssh/sshd_config
Edit the file to make sure it has “GatewayPorts yes” anywhere in the file. Usually it goes above X11 Forwarding directives.
Now restart sshd. If you were sshed into the linux host, this restart is too quick for you to actually be disconnected from the ssh session, so do not worry.
systemctl restart sshd
Important: Completing this step is very important otherwise we will only be able to connect from the NAS (see Sidenote in step 3)
Step 2. Start reverse ssh tunnel from Ubuntu to Starlord: ssh -p password ssh -f -N -T -R :22022:localhost:22 root@starlord
The -R command :22022:localhost:22 means that when anyone connects to to 22022 on starlord they will be connected to port 22 localhost on Ubuntu. Now what is localhost:22 from Ubuntu’s perspective, well thats just its own ssh session. I picked 22022 as the magic number to use when we want to connect, but you can pick anything.
Step 3. Connect from Windows PC to Ubuntu via this command: ssh ubuntu-user@starlord -p 22022
Sidenote: if you are on the NAS you can connect via ssh ubuntu-user@localhost -p 22022
Making It Persistent
If you skipped to this section, make sure you complete Step 1 from the section above or else you will only be able to connect to Ubuntu from starlord.
Step 1. Create service file
We create a systemd service file that kicks off the tunnel2me script. tunnel2me script is a wrapper script for an ssh command that tries to auto-reconnect. Additionally, if the service file notices the ssh session dies, it will restart it. The service file reconnects every 5 seconds. Also the ssh session checks the connection every 10 seconds.
Create the service file in either location below. I picked the /etc/ location.
/etc/systemd/system/tunnel2me.service
/lib/systemd/system/tunnel2me.service
You can copy paste the bold text to create the file.
cat > /etc/systemd/system/tunnel2me.service << ‘EOF’
[Unit]
Description=Setup reverse SSH tunnel thru Starlord
# make sure we only start the service after network is up
Wants=network-online.target
After=network-online.target
# Below did not wait for the network to come online, only for interface to come up
# After=network.target
# below for legacy systems
# After=network.target network-online.target
[Service]
Type=forking
ExecStart=/root/scripts/tunnel2me.sh
StandardOutput=console
Restart=always
RestartSec=5s
[Install]
WantedBy=multi-user.target
EOF
Sidenote: I couldn’t use the systemd service type of “Type=simple” because tunnel2me.sh forks. With “Type=simple” when I started the service, the ssh session would come up for a second and die off. Now this happened before I implemented all of the keep alive and auto restarting, however, I didn’t want the basic functionally to rely on that feature.
Step 2. Create the tunnel2me script. Which starts the reverse ssh tunnel.
Again copy paste the bold commands below. The will create the directory where the script will go and also make it executable.
mkdir /root/scripts
cat > /root/scripts/tunnel2me.sh << ‘EOF’
#!/bin/bash
PW=’password’
TARGET=10.11.12.79 # this is starlords IP
REMOTEPORT=22022
sshpass -p “$PW” ssh -vvv -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ServerAliveInterval=10 -o ExitOnForwardFailure=yes -f -T -N -R “:$REMOTEPORT:localhost:22” root@$TARGET
EOF
chmod +x /root/scripts/tunnel2me.sh
Step3. Test and launch the service at boot.
Refresh your services. Start this service. Enable it to start on boot. Check that its running with ps.
Refresh the services (useful when editing service files):
systemctl daemon-reload
Start the service:
systemctl start tunnel2me
Check that you see the ssh with all of the options.
ps aux | grep ss[h]
At this point, you can now connect to the Ubuntu VM using the command from the previous section step 3: ssh ubuntu-user@starlord -p 22022
You can test reconnecting it by getting its PID and then killing it.
ps aux | grep ss[h]
kill PID
Your connection should have dropped, however, you can try reconnecting in a few seconds with: ssh ubuntu-user@starlord -p 22022
If satisfied with the service, enable it for boot.
systemctl enable tunnel2me
Check that you see it has started and you see that its enabled.
systemctl status tunnel2me
Note, you can fully stop the tunnel without it trying to reconnect by stopping the service with systemctl stop tunnel2me.
Restart the Ubuntu VM to make sure it works after reboot: init 6
Your tunnel should be up even before you login to the Ubuntu VM.
Connect to the Ubuntu VM with ssh ubuntu-user@starlord -p 22022
Resources
* https://unix.stackexchange.com/questions/166473/debian-how-to-run-a-script-on-startup-as-soon-as-there-is-an-internet-connecti
* https://blog.stigok.com/2018/04/22/self-healing-reverse-ssh-systemd-service.html
* https://www.freedesktop.org/software/systemd/man/systemd.service.html#Type=