ACRN Best Known Methods¶
Collection of Best Known Methods (BKMs) for the ACRN Hypervisor.
BKM #1: Alternative to ACRN Hypervisor Shell¶
If you do not have access to the ACRN hypervisor shell, you can alternatively use a terminal multiplexer like screen to start RT-VMs or User-VMs. Of course, provided that a terminal multiplexer is installed/build for the ACRN-SOS. For this BKM screen will be used!
SSH into ACRN-SOS and check if any VMs are already running using command:
root@eci-intel-7365:~$ acrnctl list /usr/share/acrn/conf doesn't exist! _scan_added_vm: Failed to check directory /usr/share/acrn/conf, err: -1 /run/acrn/mngr doesn't exist! _scan_alive_vm: Failed to check directory /run/acrn/mngr, err: -1 There are no VMs
Pay attention to the last line mentioning There are no VMs. Next, open a screen instance providing a ‘meaningful’ name, e.g.:
root@eci-intel-7365:~$ screen -S rtos
You should now be running within the rtos screen instance. Within this instance navigate to the launch
scripts cd /var/lib/machines/scripts/ and start the RT-VM ./launch-rtvm.sh. Once the VM booted, check that you have booted the right VM, e.g. uname -r or cat /etc/os-release.
Now, detach from this screen instance using ctrl-a + d. You may check again if the VM is indeed running:
root@eci-intel-7365:~$ acrnctl list /usr/share/acrn/conf doesn't exist! _scan_added_vm: Failed to check directory /usr/share/acrn/conf, err: -1 vm0_rt started
Again, pay attention to the last line, i.e. vm0_rt started. Now repeat the same for the User-VM. I.e., start a new screen instance screen -S uos. Navigate to the scripts and execute the ./launch-uos.sh. Once the VM booted, check that you have booted the right VM. Again, detach from this screen instance using ctrl-a + d and check if VMs are running:
root@eci-intel-7365:~$ acrnctl list /usr/share/acrn/conf doesn't exist! _scan_added_vm: Failed to check directory /usr/share/acrn/conf, err: -1 vm0_rt started vm1 started
In case you want to ‘reconnect’ to a screen instance, use screen -r <instance-name>.
For this BKM it would be screen -r rtos or screen -r uos.
BKM #2: Run an ACRN Real-Time (RT) VM alongside a Non-RT VM¶
The setup requires installation of the ACRN hypervisor including the Service VM, a RT VM image (for example, ECI core-bullseye) and an User VM (in this case ECI core-jammy).
If you are using the ACRN shell, start a console to the Service VM by typing:
ACRN:\>vm_list VM_UUID VM_ID VM_NAME VM_STATE ================================ ===== ================================ ======== dbbbd4347a574216a12c2201f1ab0240 0 ACRN SOS VM Running ACRN:\>vm_console 0 ----- Entering VM 0 Shell ----- root@sosvm:~#
Next, navigate to /var/lib/machines/scripts/ and run the script to start, for example, a RT-VM and a non-RT VM (UOSs):
# NOTE: Either use different SSH sessions to start each # script, or use something like a terminal multiplexer for # separate instances! # Otherwise the terminals will overlap each other! # rtvm: /var/lib/machines/scripts/launch-rtvm.sh # or cd /var/lib/machines/scripts/ ./launch-rtvm.sh # HINT: Double check the script so it points to the right # uos_vm1.img or uos_vm2.img file! # uos /var/lib/machines/scripts/launch-uos.sh # or cd /var/lib/machines/scripts/ ./launch-uos.sh
Now go back to the ACRN shell (ctrl + space) and type vm_list. You should be listed with the current running VMs:
ACRN:\>vm_list VM_UUID VM_ID VM_NAME VM_STATE ================================ ===== ================================ ======== dbbbd4347a574216a12c2201f1ab0240 0 ACRN SOS VM Running 495ae2e526034d64af76d4bc5a8ec0e5 2 ACRN VM_2 Running 615db82ae1894b4f8dbbd321343e4ab3 3 ACRN VM_3 Running
BKM #3: ACRN multiple NIC Passthrough¶
This BKM assumes that the ACRN hypervisor is already running and access to the Service VM (ACRN-SOS) is available. For adding additional VMs, refer to BKM #2: Run an ACRN Real-Time (RT) VM alongside a Non-RT VM or to the ACRN Hypervisor in general.
Example 1: Karbon 700 IPC¶
In this example BKM, two network interfaces are passed through to a real-time VM. The Industrial PC (IPC) of scope has six I210 NICs and enp6s0 and enp7s0 are used for passthrough. Check the lspci command for both NICs:
06:00.0 Class 0200: 8086:1533 07:00.0 Class 0200: 8086:1533
It is highly recommended to start the user VM either via a separate ACRN-SOS ssh sessions or via a terminal multiplexer like screen using multiple instances (the current running ACRN-SOS shell may be overlayed with the user VM shell!).
If you plugged in a loopback cable to the two selected NICs, you need to shut down the NICs manually first. If you miss that step, the ssh connection might not work.
The follow commands will shut down the NICs:
$ ip link set dev enp6s0 down $ ip link set dev enp7s0 down
For NIC passthrough, you need to adapt the /var/lib/machines/scripts/launch-rtvm.sh script, where the NICs need to be unbound from the ACRN-SOS as well:
declare -A passthru_vpid declare -A passthru_bdf passthru_vpid=( ["Dev6"]="8086 1533" ["Dev7"]="8086 1533" ) passthru_bdf=( ["Dev6"]="0000:06:00.0" ["Dev7"]="0000:07:00.0" ) modprobe pci_stub echo ${passthru_vpid["Dev6"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["Dev6"]} > /sys/bus/pci/devices/${passthru_bdf["Dev6"]}/driver/unbind echo ${passthru_bdf["Dev6"]} > /sys/bus/pci/drivers/pci-stub/bind echo ${passthru_vpid["Dev7"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["Dev7"]} > /sys/bus/pci/devices/${passthru_bdf["Dev7"]}/driver/unbind echo ${passthru_bdf["Dev7"]} > /sys/bus/pci/drivers/pci-stub/bind
Additionally, you need to edit the acrn-dm command in the script, adding passthru for the above-mentioned devices:
-s #,passthru,06/0/0 \ -s #,passthru,07/0/0 \
The final acrn-dm command may look like this:
acrn-dm "${dm_params[@]}" \ -s 10,passthru,06/0/0 \ -s 11,passthru,07/0/0
Finally, execute the script to start, for example, a RT-VM and a non-RT VM (UOSs):
# NOTE: Either use different SSH sessions to start each # script, or use something like a terminal multiplexer for # separate instances! # Otherwise the terminals will overlap each other! # rtvm: /var/lib/machines/scripts/launch-rtvm.sh # or cd /var/lib/machines/scripts/ ./launch-rtvm.sh # HINT: Double check the script so it points to the right # uos_vm1.img or uos_vm2.img file! # uos /var/lib/machines/scripts/launch-uos.sh # or cd /var/lib/machines/scripts/ ./launch-uos.sh
When prompted for Do you want to passthrough an Ethernet interface to the VM? [y/N] during script execution, select n.
Example 2: Vecow SPC-7100 IPC¶
For this BKM it is assumed that the target device (Vecow IPC) is already running the ACRN hypervisor, with the user having access to the ACRN-SOS shell or console.
Typically, the Vecow SPC-7100 industrial PC is equipped with two Ethernet ports:
| LAN 1 | Intel® Ethernet Controller I219LM GigE LAN supports iAMT | |--------|-----------------------------------------------------------| | LAN 2 | Intel® Ethernet Controller I225-IT supports 2.5 GigE LAN |
To attach a loopback cable to the Ethernet ports, the following advice might be helpful.
Advice 1: USB to Ethernet Converter Dongle¶
If both Ethernet ports are used (for a loopback test), use an USB to Ethernet converter dongle to establish a SSH connection to the device. However, a serial console via (RS-232) COM ports will also be fine using the ACRN shell console.
Once the USB to Ethernet converter dongle is connected to one of the free USB ports, checking the ACRN-SOS dmesg command may look like this:
dmesg | grep ax88179 [ 12.570354] ax88179_178a 4-1:1.0 eth2: register 'ax88179_178a' at usb-0000:00:14.0-1, ASIX AX88179 USB 3.0 Gigabit Ethernet, 00:0e:c6:48:53:74 [ 12.575265] usbcore: registered new interface driver ax88179_178a [ 15.948362] ax88179_178a 4-1:1.0 enp0s20f0u1: renamed from eth2 [ 20.404825] ax88179_178a 4-1:1.0 enp0s20f0u1: ax88179 - Link status is: 1
Typically, driver ax88179 will assign a Ethernet NIC name to the device, in this case enp0s20f0u1. Next, change the DHCP network assignment to only trigger the USB to Ethernet converter dongle, as the other Ethernet ports are not used for SSH:
# open the network file with an editor vim /lib/systemd/network/80-wired.network # change the file to something like: [Match] MACAddress=00:0e:c6:48:53:74 KernelCommandLine=!nfsroot KernelCommandLine=!ip [Network] DHCP=yes [DHCP] RouteMetric=10 ClientIdentifier=mac
Obtain the MACAddress= from the previous dmesg command. Alternatively, you also use Name=enp0s20f0u1 instead of ‘MACAddress=’.
Advice 2: Set Link DOWN on Boot for Ethernet Ports¶
Connecting a loopback cable to the Ethernet ports prior to configuration or reboot might lead to unexpected device behavior. You can avoid this by setting the link state of both Ethernet ports to state DOWN. This BKM presents an approach using systemd. However, there might also be other ways to solve this problem and it is up to you to decide.
First, let us check the Ethernet NIC names of both ports. This could be done, for example, using:
grep PCI_SLOT_NAME /sys/class/net/*/device/uevent /sys/class/net/eno1/device/uevent:PCI_SLOT_NAME=0000:00:1f.6 /sys/class/net/enp1s0/device/uevent:PCI_SLOT_NAME=0000:01:00.0
The port names are eno1 & enp1s0. You may cross check with /usr/bin/lspci.
Next, create a script executing the link state DOWN by command opening it in an editor vim /home/root/linkstate_down.sh:
#!/bin/sh sleep 3 ifconfig enp1s0 down sleep 3 ifconfig eno1 down
Double check whether the script is executable (chmod +x /home/root/linkstate_down.sh) and create a systemd service vim /etc/systemd/system/linkstate_down.service:
[Unit] Description=Execute linkstate_down.sh script After=systemd-networkd.service [Service] ExecStart=/home/root/linkstate_down.sh RemainAfterExit=no [Install] WantedBy=multi-user.target
Finally, test and enable the new services: systemctl status linkstate_down.service, systemctl start linkstate_down.service, systemctl enable linkstate_down.service. Check the link state with either command ifconfig or ip a.
Advice 3: Martian Source¶
You may receive ‘martian source’ prints due to the loopback connection. This topic is highly network setup specific and can be solved in multiple ways. The following code is a solution:
# print current value sysctl net.ipv4.conf.all.rp_filter # set value to zero sysctl -w net.ipv4.conf.all.rp_filter=0 # make it persisten on reboots echo "net.ipv4.conf.all.rp_filter=0" >> /etc/sysctl.conf # double check cat /etc/sysctl.conf # apply the changes sysctl -p
Consult the internet on rp_filter for more details.
Continue with NIC Passthrough to VM¶
Let us start spinning up the VMs. As mentioned BKM #2: Run an ACRN Real-Time (RT) VM alongside a Non-RT VM,
modify the /var/lib/machines/scripts/launch-rtvm.sh script (only snippets are shown):
... declare -A passthru_vpid declare -A passthru_bdf passthru_vpid=( ["I219LM"]="8086 15fb" ["I225"]="8086 0d9f" ) passthru_bdf=( ["I219LM"]="0000:00:1f.6" ["I225"]="0000:01:00.0" ) modprobe pci_stub echo ${passthru_vpid["I219LM"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["I219LM"]} > /sys/bus/pci/devices/${passthru_bdf["I219LM"]}/driver/unbind echo ${passthru_bdf["I219LM"]} > /sys/bus/pci/drivers/pci-stub/bind echo ${passthru_vpid["I225"]} > /sys/bus/pci/drivers/pci-stub/new_id echo ${passthru_bdf["I225"]} > /sys/bus/pci/devices/${passthru_bdf["I225"]}/driver/unbind echo ${passthru_bdf["I225"]} > /sys/bus/pci/drivers/pci-stub/bind ... acrn-dm "${dm_params[@]}" \ -s 10,passthru,00/1f/6,vmsix_on_msi,3 \ -s 11,passthru,01/0/0,vmsix_on_msi,3 \ ...
Finally, execute the /var/lib/machines/scripts/launch-rtvm.sh script to start the RT VM.
If prompted for Do you want to passthrough an Ethernet interface to the VM? [y/N] during script execution, select n.
BKM #4: Minimize data corruption on power loss¶
If the ACRN host loses power, it is possible for data corruption to occur in the action virtual machines. This typically happens because a file is only partially written before power is lost. To minimize data corruption, follow these best practices:
Use a copy-on-write filesystem such as ZFS or BTRFS in the virtual machine. These file systems are more resilient to corruption since they only update their metadata after the file has fully written.
Update the ACRN device model parameters so that the block device performs a write-thru:
virtio-blk,writethru. This will cause all writes to to be marked completed only after they are written to physical storage as opposed to page cache. For more information on ACRN device model parameters, see ACRN Device Model Example Parameters.