ACRN BKMs¶
Collection of BKMs (Best Known Methods) for the ACRN Hypervisor.
BKM #1: Alternative to the ACRN hypervisor shell¶
In case there is no access to the ACRN hypervisor shell,
alternatively a terminal multiplexer like screen can be used to start RT-VMs or User-VMs. Of course, provided that a terminal multiplexer
is installed/build for the ACRN-SOS! For this BKM screen will be used!
SSH into ACRN-SOS and check if any VMs are already running using command:
root@ecs-intel-7365:~$ acrnctl list /usr/share/acrn/conf doesn't exist! _scan_added_vm: Failed to check directory /usr/share/acrn/conf, err: -1 /run/acrn/mngr doesn't exist! _scan_alive_vm: Failed to check directory /run/acrn/mngr, err: -1 There are no VMs
Pay attention to the last line mentioning There are no VMs. Next, open a screen instance providing a ‘meaningful’ name, e.g.:
root@ecs-intel-7365:~$ screen -S rtos
You should now be running within the ‘rtos’ screen instance. Within this instance navigate to the launch
scripts cd /var/lib/machines/scripts/ and start the RT-VM ./launch-rtvm.sh. Once the VM booted, check that
you have booted the right VM, e.g. uname -r or cat /etc/os-release.
Now, detach from this screen instance using ctrl-a + d. You may check again if the VM is indeed running:
root@ecs-intel-7365:~$ acrnctl list /usr/share/acrn/conf doesn't exist! _scan_added_vm: Failed to check directory /usr/share/acrn/conf, err: -1 vm0_rt started
Again, pay attention to the last line, i.e. vm0_rt started. Now repeat the same for the User-VM.
I.e., start a new screen instance screen -S uos. Navigate to the scripts and execute
the ./launch-uos.sh. Once the VM booted, check that you have booted the right VM. Again,
detach from this screen instance using ctrl-a + d and check if VMs are running:
root@ecs-intel-7365:~$ acrnctl list /usr/share/acrn/conf doesn't exist! _scan_added_vm: Failed to check directory /usr/share/acrn/conf, err: -1 vm0_rt started vm1 started
In case you want to ‘reconnect’ to a screen instance, use screen -r <instance-name>.
For this BKM it would be screen -r rtos or screen -r uos.
BKM #2: Run an ACRN Real-Time (RT) VM alongside a non-RT VM¶
The setup requires installation of the ACRN hypervisor including the Service OS, a RT VM image (e.g. ECI base poky) and an UOS (in this case ECI preempt buster).
In order to embed other OS’s to the ACRN hypervisor image, such user OS’s have to be build beforehand. In this example a base poky and a preempt-buster image are used as the UOS. To build the two images, please follow the instructions rts-poky and rts-buster.
After the OS’s are build, start building the ACRN hypervisor image with the Service OS (ACRN-SOS). Please follow the instruction acrn-sos until you reach the section for the “Automated Build”. Instead of the automated build, select the “Manual build” tab.
In order to embed the 2 extra OS’s into the ACRN image, please execute the build with the following command:
UOS_TARGET="base-poky preempt-buster" BB_ENV_EXTRAWHITE="UOS_TARGET" bitbake acrn-image-minimal
Once the build finished, continue installing the image onto the target: Installing ECI-A
If you are using the ACRN shell, start a console to the service OS by typing:
ACRN:\>vm_list VM_UUID VM_ID VM_NAME VM_STATE ================================ ===== ================================ ======== dbbbd4347a574216a12c2201f1ab0240 0 ACRN SOS VM Running ACRN:\>vm_console 0 ----- Entering VM 0 Shell ----- root@sosvm:~#
Next, navigate to /var/lib/machines/scripts/ and execute the script to start e.g. a RT-VM and a non-RT VM (UOS’s):
# NOTE: Either use different SSH sessions to start each # script, or use something like a terminal multiplexer for # separate instances! # Otherwise the terminals will overlap each other! # rtvm: /var/lib/machines/scripts/launch-rtvm.sh # or cd /var/lib/machines/scripts/ ./launch-rtvm.sh # HINT: Double check the script so it points to the right # vm0.wic or vm1.wic file! # uos /var/lib/machines/scripts/launch-uos.sh # or cd /var/lib/machines/scripts/ ./launch-uos.sh
Now go back to the ACRN shell (ctrl + space) and type vm_list. You should be listed with the current running VM’s:
ACRN:\>vm_list VM_UUID VM_ID VM_NAME VM_STATE ================================ ===== ================================ ======== dbbbd4347a574216a12c2201f1ab0240 0 ACRN SOS VM Running 495ae2e526034d64af76d4bc5a8ec0e5 2 ACRN VM_2 Running 615db82ae1894b4f8dbbd321343e4ab3 3 ACRN VM_3 Running
Once you have access to the VM’s either via SSH or the ACRN shell (Using the ACRN hypervisor shell), on a Real-Time VM you
may navigate to /opt/benchmarking/ to start some workloads for System Performance Characterization. Additional workloads of
interest would be the CODESYS Software PLC related applications like the CODESYS Benchmark Application and CODESYS ST-Fragment.
On a non-RT VM you may use the stress-ng application to generate some noise (aka non-RT related workload). For example, if you would
like to put some noise to the CPUs, use command stress-ng --cpu 2 for setting both CPUs to full load while executing a benchmark or workload on the RT-VM.
BKM #3: ACRN multiple NIC Passthrough¶
This BKM assumes that the ACRN hypervisor is already running and access to the Service OS (ACRN-SOS) is available. For adding additional VMs, you may refer to BKM #2: Run an ACRN Real-Time (RT) VM alongside a non-RT VM or to the ACRN Hypervisor in general.
Example 1: Karbon 700 IPC¶
In this example BKM, 2 network interfaces are passed through to a Real-Time VM. The Industrial PC (IPC) of scope has 6 I210 NIC’s
and we’re using enp6s0 and enp7s0 for passthrough. Check the lspci command for both NIC’s:
06:00.0 Class 0200: 8086:1533 07:00.0 Class 0200: 8086:1533
It is highly recommended to start the user VM either via a separate ACRN-SOS ssh sessions or via a terminal multiplexer like screen
using multiple instances (the current running ACRN-SOS shell may be overlayed with the user VM shell!).
In case you plugged in a loopback cable to the 2 selected NIC’s, you need to shut down the NIC’s manually first. If you miss that step, the ssh connection might not work.
This example commands shows how the NIC’s can be shut down:
ip link set dev enp6s0 down ip link set dev enp7s0 down
For NIC passthrough, you need to adapt the /var/lib/machines/scripts/launch-rtvm.sh script,
where the NIC’s need to be unbind from the ACRN-SOS as well:
declare -A passthru_vpid declare -A passthru_bdf passthru_vpid=( ["Dev6"]="8086 1533" ["Dev7"]="8086 1533" ) passthru_bdf=( ["Dev6"]="0000:06:00.0" ["Dev7"]="0000:07:00.0" ) modprobe pci_stub echo ${passthru_vpid["Dev6"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["Dev6"]} > /sys/bus/pci/devices/${passthru_bdf["Dev6"]}/driver/unbind echo ${passthru_bdf["Dev6"]} > /sys/bus/pci/drivers/pci-stub/bind echo ${passthru_vpid["Dev7"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["Dev7"]} > /sys/bus/pci/devices/${passthru_bdf["Dev7"]}/driver/unbind echo ${passthru_bdf["Dev7"]} > /sys/bus/pci/drivers/pci-stub/bind
Additionally, you need to edit the acrn-dm command in the script too, adding passthru for above mentioned devices:
-s 6,passthru,06/0/0 \ -s 7,passthru,07/0/0 \
The final acrn-dm command may look like this:
acrn-dm -A -m $mem_size -s 0:0,hostbridge -s 1:0,lpc \ -U ${UUID_POST_RT[$vm_id]} \ -s 2,virtio-console,@stdio:stdio_port \ --cpu_affinity 2,3 \ -s 3,virtio-rnd \ -s 4,virtio-blk,/var/lib/machines/images/$vm_image \ -s 5,virtio-net,$tap_name \ -s 6,passthru,06/0/0 \ -s 7,passthru,07/0/0 \ --mac_seed $mac_seed \ --ovmf /usr/share/acrn/bios/OVMF.fd \ --rtvm \ --virtio_poll 100000 \ --lapic_pt \ $vm_name
Finally, execute the script to start e.g. a RT-VM and a non-RT VM (UOS’s):
# NOTE: Either use different SSH sessions to start each # script, or use something like a terminal multiplexer for # separate instances! # Otherwise the terminals will overlap each other! # rtvm: /var/lib/machines/scripts/launch-rtvm.sh # or cd /var/lib/machines/scripts/ ./launch-rtvm.sh # HINT: Double check the script so it points to the right # vm0.wic or vm1.wic file! # uos /var/lib/machines/scripts/launch-uos.sh # or cd /var/lib/machines/scripts/ ./launch-uos.sh
When prompted for Do you want to passthrough an Ethernet interface to the VM? y/[n] during script execution,
please select n.
Example 2: Vecow SPC-7100 IPC¶
For this BKM it’s assumed that the target device (Vecow IPC) is already running the ACRN hypervisor, with the user having access to the ACRN-SOS shell or console.
Typically the Vecow SPC-7100 industrial PC is equipped with two ethernet ports:
| LAN 1 | Intel® Ethernet Controller I219LM GigE LAN supports iAMT | |--------|-----------------------------------------------------------| | LAN 2 | Intel® Ethernet Controller I225-IT supports 2.5 GigE LAN |
If you would like to attach a loopback cable to the ethernet ports, the following advice might be helpful.
Advice 1: USB to ethernet converter dongle¶
If both ethernet ports are used for e.g. a loopback test, an USB to ethernet converter dongle can be used to still have a SSH connection to the device. However, a serial console via e.g. RS-232 COM ports will also be fine using the ACRN shell console.
Once the USB to ethernet converter dongle is connected to one of the free USB ports, checking the ACRN-SOS dmesg command
may look like this:
dmesg | grep ax88179 [ 12.570354] ax88179_178a 4-1:1.0 eth2: register 'ax88179_178a' at usb-0000:00:14.0-1, ASIX AX88179 USB 3.0 Gigabit Ethernet, 00:0e:c6:48:53:74 [ 12.575265] usbcore: registered new interface driver ax88179_178a [ 15.948362] ax88179_178a 4-1:1.0 enp0s20f0u1: renamed from eth2 [ 20.404825] ax88179_178a 4-1:1.0 enp0s20f0u1: ax88179 - Link status is: 1
Typically driver ax88179 will assign a ethernet NIC name to the device, in this case enp0s20f0u1. Next, lets change
DHCP network assignment to only trigger our USB to ethernet converter dongle, as the other ethernet ports are not used for SSH:
# open the network file with an editor vim /lib/systemd/network/80-wired.network # change the file to something like: [Match] MACAddress=00:0e:c6:48:53:74 KernelCommandLine=!nfsroot KernelCommandLine=!ip [Network] DHCP=yes [DHCP] RouteMetric=10 ClientIdentifier=mac
The MACAddress= can be obtained from previous dmesg command. Alternatively also Name=enp0s20f0u1 can be
used instead of ‘MACAddress=’.
Advice 2: Set Link DOWN on boot for ethernet ports¶
Connecting a loopback cable to the ethernet ports prior configuration or reboot might lead to unexpected device behavior. This can be avoided by setting link state of both ethernet ports to state DOWN. In this BKM an approach using systemd will be presented. However, there might also be other ways to solve this problem and it’s up to the user to decide.
First, let’s check the ethernet NIC names of both ports. This could be done for example using:
grep PCI_SLOT_NAME /sys/class/net/*/device/uevent /sys/class/net/eno1/device/uevent:PCI_SLOT_NAME=0000:00:1f.6 /sys/class/net/enp1s0/device/uevent:PCI_SLOT_NAME=0000:01:00.0
From there it’s known that our port names are eno1 & enp1s0. You may cross check with /usr/bin/lspci.
Next, let’s create a script executing the link state DOWN command opening it in an editor vim /home/root/linkstate_down.sh:
#!/bin/sh sleep 3 ifconfig enp1s0 down sleep 3 ifconfig eno1 down
Double check the script is executable (chmod +x /home/root/linkstate_down.sh) and create a systemd service
vim /etc/systemd/system/linkstate_down.service:
[Unit] Description=Execute linkstate_down.sh script After=systemd-networkd.service [Service] ExecStart=/home/root/linkstate_down.sh RemainAfterExit=no [Install] WantedBy=multi-user.target
Finally, test and enable the new service: systemctl status linkstate_down.service,
systemctl start linkstate_down.service, systemctl enable linkstate_down.service. Check link state with either
command ifconfig or ip a.
Advice 3: Martian source?¶
You may receive ‘martian source’ prints due to the loopback connection. This topic is highly network setup specific and can be solved in multiple ways. However, one way would be to do it as follows:
# print current value sysctl net.ipv4.conf.all.rp_filter # set value to zero sysctl -w net.ipv4.conf.all.rp_filter=0 # make it persisten on reboots echo "net.ipv4.conf.all.rp_filter=0" >> /etc/sysctl.conf # double check cat /etc/sysctl.conf # apply the changes sysctl -p
Consult the internet on rp_filter for more details!
Continue with the NIC passthrough to the VM¶
Let’s start spinning up the VMs. As within section BKM #2: Run an ACRN Real-Time (RT) VM alongside a non-RT VM,
additionally the /var/lib/machines/scripts/launch-rtvm.sh script has to modified (only snippets are shown):
... declare -A passthru_vpid declare -A passthru_bdf passthru_vpid=( ["I219LM"]="8086 15fb" ["I225"]="8086 0d9f" ) passthru_bdf=( ["I219LM"]="0000:00:1f.6" ["I225"]="0000:01:00.0" ) modprobe pci_stub echo ${passthru_vpid["I219LM"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["I219LM"]} > /sys/bus/pci/devices/${passthru_bdf["I219LM"]}/driver/unbind echo ${passthru_bdf["I219LM"]} > /sys/bus/pci/drivers/pci-stub/bind echo ${passthru_vpid["I225"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["I225"]} > /sys/bus/pci/devices/${passthru_bdf["I225"]}/driver/unbind echo ${passthru_bdf["I225"]} > /sys/bus/pci/drivers/pci-stub/bind ... acrn-dm -A -m $mem_size -s 0:0,hostbridge -s 1:0,lpc \ -U ${UUID_POST_RT[$vm_id]} \ -s 2,virtio-console,@stdio:stdio_port \ --cpu_affinity 2,3 \ -s 3,virtio-rnd \ -s 4,virtio-blk,/var/lib/machines/images/$vm_image \ -s 5,virtio-net,$tap_name \ -s 6,passthru,00/1f/6,vmsix_on_msi,3 \ -s 7,passthru,01/0/0,vmsix_on_msi,3 \ --mac_seed $mac_seed \ --ovmf /usr/share/acrn/bios/OVMF.fd \ --rtvm \ --virtio_poll 100000 \ --lapic_pt \ $vm_name ...
Finally, execute the /var/lib/machines/scripts/launch-rtvm.sh script to start the RT VM.
If prompted for Do you want to passthrough an Ethernet interface to the VM? y/[n] during script execution,
please select n.