Attention

You are viewing an older version of the documentation. The latest version is v3.3.

RTH BKMs

Collection of BKMs (Best Known Methods) for the RTS Hypervisor.

BKM #1: Run a RT workload on POS and non-RT workload on GPOS

This BKM assumes that your target is already running RTS with access to the POS (Privileged OS) and GPOS (General Purpose OS). Otherwise, please follow rts-buster, rts-poky, Installing ECI-B/R/X and Real-Time Systems Hypervisor (RTH).

Furthermore, this BKM will use Debian GNU/Linux 10 (buster) for both operating systems, where POS runs 5.4.115-rt57-rts2.2.02.15407-intel-pk-standard+ and GPOS runs 5.4.115-intel-pk-standard+. Please notice the difference for POS rt57, indicating a real-time patched Linux kernel.

The idea of this BKM is to run a real-time (RT) workload (or benchmark) on the POS, while running a non real-time (RT) workload (or noise) on the GPOS:

Basically, any benchmark as found under /opt/benchmarking/ can be used for testing the system. Please find details for the individual benchmarks under System Performance Characterization. However, for this BKM a CODESYS Software PLC application, namely CODESYS ST-Fragment is used. Alternatively CODESYS Benchmark Application can be used as well. Primarily for generating noise, stress-ng will be used.

Additionally, this BKM will use SSH connections, to both, the POS and GPOS via a single ethernet device. Either follow Step 4: Optional - Port forwarding SSH into the ECI RT-OS runtime, or use the systemd service approach as described next.

Establish a SSH connection to the GPOS and create the following service: vim /etc/systemd/system/pos_to_gpos_ipforward.service

[Unit]
Description=Forward SSH port from POS to GPOS
After=systemd-networkd.service

[Service]
ExecStart=/etc/systemd/network/pos-forward.sh

[Install]
WantedBy=multi-user.target

Next, figure out the MAC-Address of your SSH ethernet device via e.g. ifconfig or ip a command. I.e., match your IP address and the network interface device, providing the right MAC-Address. For example, see ether 00:e0:4c:3e:f2:19 if using ifconfig:

enx00e04c3ef219: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet 192.168.44.52  netmask 255.255.255.0  broadcast 192.168.44.255
inet6 fe80::2e0:4cff:fe3e:f219  prefixlen 64  scopeid 0x20<link>
ether 00:e0:4c:3e:f2:19  txqueuelen 1000  (Ethernet)
RX packets 27543  bytes 5054966 (4.8 MiB)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 15490  bytes 1442772 (1.3 MiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Now, let’s create the script calling iptables, as triggered by systemd service: vim /etc/systemd/network/pos-forward.sh

#!/bin/bash

OS=`uname -r`
GPOS="intel-pk-standard"
MAC="00:e0:4c:3e:f2:19"
INTF="" # empty string
BRIDGE="" # empty string

sleep 5 # give some time to get ready for the BRIDGE

if [ -n "$MAC" ]; then
    if [[ $OS == *"$GPOS"* ]]; then
        BRIDGE=$(ip a | grep -B2 "inet 192.168.2.1" | awk -F'[ :]+' '$1 && $1!="lo" {print $2}')
        if [ -n "$BRIDGE" ]
        then
            echo '1' > /proc/sys/net/ipv4/conf/$BRIDGE/forwarding
        else
            echo "Not able to detect BRIDGE for ip-forwarding!"
            exit 1
        fi
        INTF=$(ip a | grep -B1 "$MAC" | awk -F'[ :]+' '$1 && $1!="lo" {print $2}')
        if [ -n "$INTF" ]
        then
            echo '1' > /proc/sys/net/ipv4/conf/$INTF/forwarding
            # flush
            iptables -F
            iptables -t nat -F
            # port forwarding
            iptables -A PREROUTING -t nat -i $INTF -p tcp --dport 2222 -j DNAT --to 192.168.2.2:22
            # CODESYS
            iptables -A PREROUTING -t nat -i $INTF -p udp --dport 1740 -j DNAT --to 192.168.2.2:1740
            iptables -A PREROUTING -t nat -i $INTF -p udp --dport 1741 -j DNAT --to 192.168.2.2:1741
            iptables -A PREROUTING -t nat -i $INTF -p udp --dport 1742 -j DNAT --to 192.168.2.2:1742
            iptables -A PREROUTING -t nat -i $INTF -p udp --dport 1743 -j DNAT --to 192.168.2.2:1743
            iptables -A PREROUTING -t nat -i $INTF -p tcp --dport 11740 -j DNAT --to 192.168.2.2:11740
            iptables -A PREROUTING -t nat -i $INTF -p tcp --dport 1217 -j DNAT --to 192.168.2.2:1217
            iptables -A PREROUTING -t nat -i $INTF -p tcp --dport 1217 -j DNAT --to 192.168.2.2:4840
            # accept forwarded packets
            iptables -A FORWARD -p tcp -d 192.168.2.2 --dport 2222 -j ACCEPT
            # CODESYS
            iptables -A FORWARD -p udp -d 192.168.2.2 --dport 1740 -j ACCEPT
            iptables -A FORWARD -p udp -d 192.168.2.2 --dport 1741 -j ACCEPT
            iptables -A FORWARD -p udp -d 192.168.2.2 --dport 1742 -j ACCEPT
            iptables -A FORWARD -p udp -d 192.168.2.2 --dport 1743 -j ACCEPT
            iptables -A FORWARD -p tcp -d 192.168.2.2 --dport 11740 -j ACCEPT
            iptables -A FORWARD -p tcp -d 192.168.2.2 --dport 1217 -j ACCEPT
            iptables -A FORWARD -p tcp -d 192.168.2.2 --dport 4840 -j ACCEPT
            # masquerade
            iptables -t nat -A POSTROUTING ! -s 127.0.0.1 -j MASQUERADE
            iptables -t nat -A POSTROUTING -o $INTF -j MASQUERADE
            iptables -L
        else
            echo "Not able to get MAC from cmdline for INTF!"
            exit 1
        fi
    fi
fi

Note

Please make sure to use your MAC="..." address in the script, as explained before.

Finally, make the script executable: chmod +x /etc/systemd/network/pos-forward.sh. Test the service:

# start the service:
systemctl start pos_to_gpos_ipforward.service
# wait a couple of seconds and check the status:
systemctl status pos_to_gpos_ipforward.service
● pos_to_gpos_ipforward.service - Forward SSH port from POS to GPOS
    Loaded: loaded (/etc/systemd/system/pos_to_gpos_ipforward.service; enabled; vendor preset: enabled)
    Drop-In: /usr/lib/systemd/system/service.d
            └─10-override-protect-proc.conf
    Active: inactive (dead) since Thu 2021-09-09 18:00:11 UTC; 1h 25min ago
    Process: 813 ExecStart=/etc/systemd/network/pos-forward.sh (code=exited, status=0/SUCCESS)
Main PID: 813 (code=exited, status=0/SUCCESS)

Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: target     prot opt source               destination
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: Chain DOCKER (0 references)
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: target     prot opt source               destination
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: Chain DOCKER-ISOLATION-STAGE-1 (0 references)
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: target     prot opt source               destination
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: Chain DOCKER-ISOLATION-STAGE-2 (0 references)
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: target     prot opt source               destination
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: Chain DOCKER-USER (0 references)
Sep 09 18:00:11 eci-intel-07e7 pos-forward.sh[844]: target     prot opt source               destination
Sep 09 18:00:11 eci-intel-07e7 systemd[1]: pos_to_gpos_ipforward.service: Succeeded.
# (optional) enable the service to start automatically after a reboot
systemctl enable pos_to_gpos_ipforward.service

Now you should be able to SSH to the POS and GPOS from the same IP address. For example:

# GPOS:
ssh root@192.168.44.52
# POS:
ssh root@192.168.44.52 -p 2222

For POS the standard SSH port 22 is redirected to GPOS port 2222!

As mentioned at the beginning, our target benchmark (or RT workload) will be CODESYS ST-Fragment. Please get the main details about this benchmark from the provided link. Any other CODESYS details can also be obtained from section CODESYS Software PLC. However, important to mention is that in this BKM the CODESYS related ports to communicate with the IDE (download the application to the target) are as well ‘port forwarded’ from POS to GPOS (see above pos-forward.sh script). Using the regular Scan Network... might not work. Instead it’s required to enter the target IP address directly:

If you’re not able to connect, try to restart the CODESYS service: systemctl restart codesyscontrol.service.

Now continue with ‘Download (execute) directly to the target’ from CODESYS ST-Fragment. Once the application is downloaded and started, SSH to the POS ssh root@192.168.44.52 -p 2222 and check the generated results file under /var/opt/codesys/PlcLogic/.

Let’s now create some noise on the GPOS. SSH to the GPOS ssh root@192.168.44.52. Using stress-ng, noise (or non-RT workload) is simulated. For example, let’s put some load on the GPOS CPUs using command stress-ng --cpu 2. Now, if returning back to the POS, restart the ST-Fragment benchmark by simply restarting the CODESYS service: systemctl restart codesyscontrol.service. This will capture a RT workload results file, while a non-RT workload (stress-ng) is running on the GPOS:

On the left hand side of the picture (GPOS), stress-ng is executed on the top, whereas monitoring is done via htop on the bottom. On the right hand side of the picture (POS), CODESYS service is restarted to trigger a new benchmark file (top), while also monitoring on the bottom.

Now, compare both results files. Obviously, one file should be captured without any ‘noise’ first, the second while running stress-ng! Ideally, the values should not differ significantly as the noise running on the GPOS should be isolated from the POS via the hypervisor. For instance, doing the counter-check by running stress-ng --cpu 2 on POS as well, plus restarting the CODESYS service to generate a new results file, should show significant differences!

Another way creating some non-RT workload (or noise) is to run some graphics stress. Here it’s recommended to connect directly to the screen output and use a monitor to access the desktop. Then simply run e.g. a terminal and execute glxheads, or another glx workload:

You may also install any other workload using apt install, if your GPOS is Debian GNU/Linux 10 (buster)!