GoodByte embedded software

Local testing of IoT device

February 22, 2024

While "internet" is not the first thing we think of when we hear about embedded, today, in the era of the Internet of Things, such an association is most appropriate. More and more devices are now connecting to the network, sending data to servers and exchanging it with other devices. Therefore, knowledge of how computer networks work is very useful in this industry.

Working with devices that connect to the network, often wirelessly via GSM, involves additional difficulties in testing such devices. In this article, we will focus on a specific problem we may encounter when working with such devices.

Problem

Suppose we are working on the software of device X. While working, our device connects to device Y over TCP/IP, for example, to send measurement data, or to receive remote commands.
The situation is shown in the figure below:

Server Y is publicly available at a.b.c.d:e, however, we would like to be able to test the communication on our computer. Under such conditions, we are able to easily preview the communication and fix any errors. If we have access to the Y server software we could start it locally on our computer, if not - write a simplified version of the Y server to test specific communication scenarios. In both cases, the same problem arises - how to connect our device X to our computer?

Proxy

Our X device can only connect to IP addresses available to the public. Our computer, however, (usually) does not have one - most likely it is behind one (or more) routers that change packet addresses (NAT - Network Address Translation). In such a situation, to be able to connect device X to our computer we have to use some server that has a public IP address. Perhaps we have such a server (we or our employer), if not then we can, for example, relatively cheaply buy a so-called Virtual Private Server (VPS).
In the rest of this article we assume that, we have SSH access to such a server running Linux.

Outline of the solution

Terminology:

DEVICE - our device that wants to connect to the server application, we can not connect to it (we do not know the IP address);
PC - our computer, we do not know its IP address, but we want DEVICE to connect to it;
PROXY - a server with a public IP to which we have access;

To enable the connection, we need to use PROXY as an intermediary between DEVICE and PC, as shown in the following figure:

Connection cascade: IoT device -> Proxy -> PC

On the PROXY device, we will listen for connections from the DEVICE side. When DEVICE connects, we will redirect all communication to our PC. To enable such redirection we will have to prepare a so-called "reverse proxy" from our PC. We can do the whole thing with the help of several standard programs available on Linux.

Before we get to the final solution, we need to present the available tools and outline some problems.

Netcat

Netcat(nc) is a simple tool for transferring data over TCP/UDP. It is often the first choice when testing/debugging connections. Netcat allows listening/connecting over the network and transmits data from STDIN/STDOUT streams, making it easy to blend in with other Unix tools.

Two common types of calls are:

A TCP server listening on a given port, e.g. nc -l -p 8000 listens on port 8000 of the localhost and when the client connects, it writes out all the data on STDOUT and sends to the client the data we type on STDIN.
A TCP client connecting to a given address, e.g. nc 42.42.42.42 8000 will connect to 42.42.42.42 on port 8000 and, as before, will send the data received on STDIN, displaying on STDOUT the data received from the server.

To see how this tool can be used we can do the following:

Terminal 1: call nc -l -p 8123
Terminal 2: call nc localhost 8123 < README.md

Assuming that the file README.md exists in the current directory of Terminal 2, it will be uploaded to the server and displayed in Terminal 1.

Reverse proxy

Going back to our original problem, we still lack a way to connect from PROXY to our PC - we can only make connections from the PC to PROXY (since only PROXY has a publicly visible IP address). To enable connections from PROXY to the PC, we need to create a so-called "reverse proxy." This involves connecting from the PC to PROXY (that we know how to do), then redirecting connections from PROXY to the PC "inside" the previously created connection (known as tunneling).

Creating a reverse proxy is possible with SSH, just use the -R argument.

Call example:




ssh -N -R 8200:42.42.42.42:8300 user@server

The syntax can be difficult to decipher at first, so let's examine the call arguments one by one:

ssh user@server - standard connection over SSH to the server at the server address/domain as user user (e.g. with ssh IP address myuser@42.42.42.42 or ssh domain myuser@my.server.com). Sometimes you may need to specify a port other than the default 22, in which case use -p (e.g. -p 2222 );
-N - do not start the terminal (we do not need it in this case);
-R - create a reverse proxy;
- 8200 - local port on the server server from which connections will be tunneled;
- 42.42.42.42:8300 - the address (seen from the perspective of the computer on which we invoke the command) to which connections will be tunneled;

‍‍This situation is illustrated in the figure below:

In our case, when our local server starts on our PC, we can perform tunneling to the localhost (127.0.0.1), that is, for example, when the local server starts on port 8300:




ssh -N -R 8200:127.0.0.1:8300 user@PROXY

Local redirection to PROXY

While the reverse proxy presented earlier allows us to redirect calls from PROXY on port 8200 to our PC, SSH will only listen for local calls to port 8200, so calls from our DEVICE to PROXY on port 8200 will not be accepted. This can be changed in the settings of the SSH daemon(sshd), but we usually don't want to do this. Instead, we can create an additional redirect with the help of Netcat.

Assuming that our reverse proxy is listening on PROXY for local connections on port 8200, we can create a local forwarding (already on the PROXY device) from port 8100 to port 8200, with port 8100 also listening for remote connections (from outside the PROXY device). To do this, we can use Netcat as follows:




nc -k -l 8100 -c 'nc 127.0.0.1 8200'

Where:

-k makes Netcat also wait for subsequent connections when the first one disconnects, instead of aborting after the first connection;
-l 8100 listens on all connections (including remote connections) on port 8100
-c when the Netcat client connects, it will call the specified command and redirect the connection to it (STDIN/STDOUT links);
nc 127.0.0.1 8200 connects to localhost on port 8200, which is where our reverse proxy is listening

Note the availability of the flag -c in our Netcat implementation, as it is not always supported (Ncat supports this flag). The use of Unix streams ("pipe"), e.g. nc -k -l 8100 | nc 127.0.0.1 8200, will not work here, because the connection to 127.0.0.1 8200 would be established only once, and that immediately when the command is called, and not only when someone connects on port 8100.

Socat

While convenient for quick tests, Netcat has its limitations, and the available features vary between implementations. A much more powerful alternative is the socat tool , which allows us to replace Netcat's functionality, as well as much more.To replace our previous call nc -k -l 8100 -c 'nc 127.0.0.1 8200' (and become independent of any particular Netcat implementation), we can use Socat as follows:




socat tcp-listen:8100,reuseaddr,fork tcp:localhost:8200

Where:

tcp-listen:8100 - listening for TCP on port 8100, like nc -l 8100
reuseaddr,fork - is responsible for continuous listening, similar to nc -k
tcp:localhost:8200 - connects to localhost (127.0.0.1) on port 8200, replaces -c 'nc 127.0.0.1 8200'

Firewall

Before we present the final solution, it is worth mentioning the settings of our PROXY. It is possible that by default our server has a "firewall" configured to prevent connections on most ports. If this is the case, we need to add the appropriate exceptions to the firewall configuration.

In Linux, this can be done using the iptables command to configure the firewall built into the Linux kernel. For example, to add an exception to allow connections using the TCP protocol on port 9000, we can use the following command:





sudo iptables -I INPUT -p tcp --dport 9000 -j ACCEPT

Firewall configuration can vary depending on the configuration of our PROXY server, and it is difficult to provide a specific solution here.

A thorough explanation of how iptables works would take too long to fit in this article, you can find more information about this command in the Article References.

Final command

We already have all the necessary pieces of the puzzle:

we start our PC test server on port 8300
on the PC we start a reverse proxy from PROXY to our local server using ssh -N -R 8200:127.0.0.1:8300 user@PROXY
we connect to PROXY and call there socat tcp-listen:8100,reuseaddr,fork tcp:localhost:8200

Now whenever DEVICE connects to PROXY on port 8100, the connection will be redirected to our local PC test server on port 8300, and we can test the communication. The following figure illustrates the final configuration:

We can include all of this in a single command to make it easier to start our DEVICE->PROXY->PC redirection:




ssh -t -R 8200:127.0.0.1:8300 user@PROXY 'socat tcp-listen:8100,reuseaddr,fork tcp:localhost:8200'

Where:

ssh user@PROXY - we connect SSH to PROXY
-R 8200:127.0.0.1:8300 - we create a reverse proxy
socat tcp-listen:8100,reuseaddr,fork tcp:localhost:8200 - after the SSH connection we start the socat command with local redirection (already called on the PROXY server)
-t - allocates PTY, so that when the SSH connection is interrupted (pressing Ctrl+D / closing the terminal), it also interrupts the call to the socat command

We can test the redirection thus established on the PC, without DEVICE and without starting our local test server, using Netcat:

Terminal 1: nc -l -p 8300 | hexdump -C - imitates our local test server listening on port 8300, will write out hexdump of received data
Terminal 2: nc PROXY 8100 < README.md, where PROXY is the address of our PROXY server - mimics the data sent by DEVICE to PROXY on port 8100

File data sent this way README.md will go all the way from Netcat on the PC to PROXY:8100, via local redirection to PROXY:8200, and via SSH reverse proxy on PC:8300 to Netcat listening there with hexdump.

References:

‍