Attention

You are viewing an older version of the documentation. The latest version is v3.3.

CODESYS ST-Fragment

The following fragments are exercised:

• Fragments of complex “floating point”, “arithmetic”, or “boolean” operations

• Instances of PID filter implementation

• Instances of KALMAN filter implementation

Cyclic CPU execution times are measured from which minimum, maximum, and jitter measurements are derived. These measurements allow performance characterization of the CODESYS* SoftPLC.

Execute ST-Fragment standalone

The following section is applicable to:

1. Setup the ECI APT repository.

2. install ST-Fragment prebuilt PLC logic from the ECI repository :

   $ sudo apt install codesys-st-fragments
   

   

   The following is the expected output for the Debian* ECI APT repository:

   Reading package lists... Done
   Building dependency tree... Done
   Reading state information... Done
   The following NEW packages will be installed:
   codesys-st-fragments
   0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
   Need to get 0 B/368 kB of archives.
   After this operation, 2191 kB of additional disk space will be used.
   Get:1 /mnt/eci-bullseye/pool/main/c/codesys-benchmark/codesys-st-fragments_4.5.0.0-1.3-bullseye.20d6d6725e_amd64.deb codesys-st-fragments amd64 4.5.0.0-1.3-bullseye.20d6d6725e [368 kB]
   debconf: delaying package configuration, since apt-utils is not installed
   Selecting previously unselected package codesys-st-fragments.
   (Reading database ... 176880 files and directories currently installed.)
   Preparing to unpack .../codesys-st-fragments_4.5.0.0-1.3-bullseye.20d6d6725e_amd64.deb ...
   Unpacking codesys-st-fragments (4.5.0.0-1.3-bullseye.20d6d6725e) ...
   Setting up codesys-st-fragments (4.5.0.0-1.3-bullseye.20d6d6725e) ...
   stopping codesys control ...
   

   

3. Edit /opt/benchmarking/codesys/BenchmarkConfigFile.txt to configure ST-Fragment PLC logic parameters :

   numberofcycles=1000
   numberoftestiterations=1
   cycletime=1000
   
   test=1_PID
   test=5_PID
   test=10_PID
   test=1_KALMAN
   test=5_KALMAN
   test=10_KALMAN
   test=1_FRAGMENT
   test=5_FRAGMENT
   test=10_FRAGMENT
   

   

   For example, 1000 cycles of various IEC function block executed back-to-back within a 1 ms cycle time deadline.

4. Start codesyscontrol runtime to auto-load and execute ST-Fragment prebuilt PLC logic:

   $ systemctl restart codesyscontrol.service
   

   

5. When PLC logic has exhausted all numberofcycles, the execution will generate ST-Fragment PLC logic postmortem report:

   $ less /var/opt/codesys/PlcLogic/IEC-61131-3_process-automation_scorecard*
   

   

   The following is the expected output for this example with only 1 second execution time:

   --- PLCopen 61131-3 Process Automation testcase scorecard ---
   
   Target: Id 5 Type 4102 Version 4.5.0.0
   Test at:
   Platform: IEC 61131-3 workload -  Control Function Block
   Extern: Extern
   Code: Ram
   Measured timer resolution [ns]: 40
   Measured overhead of SysTimeGetNs [ns]: 43.5064
   
   Compute Latency                      Avg       Min           Max    StdDev        NumCalls
   ------------------------------------------------------------------------------------------
                                 88760.23   87973.0      111443.0  1110.826            1001
   
   Test (in us)                         Avg       Min           Max    StdDev        NumCalls
   ------------------------------------------------------------------------------------------
   
   PID 1x (REAL)                   88.78131   72.4936      697.4936  39.64781            1001
   PID 5x (REAL)                   196.8842  174.4936      964.4936  36.53517            1001
   PID 10x (REAL)                  402.0321  358.4936      1799.494  59.03344            1001
   KALMAN 1x filter (REAL)         190.7464  155.4936      18378.49  576.1357            1001
   KALMAN 5x filter (REAL)         736.5935  718.4936      1154.494  26.28798            1001
   KALMAN 10x filter (REAL)        1447.047  1424.494      2223.494   36.0689            1001
   Logic+Arith 1K fragment         5435.206  5208.494     10919.493  211.7485            1001
   Logic+Arith 5K fragment         26053.42   25772.5       26886.5 115.10668            1001
   Logic+Arith 10K fragment         52148.9  51803.49      59229.49  275.6929            1001
   
   Histogram (Num nSec Hits)
   1   200.0   0
   2   400.0   0
   3   600.0   0
   4   800.0   0
   5   1000.0   0
   6   1200.0   0
   7   1400.0   0
   

   

6. use /opt/benchmarking/codesys/plot-st-fragment-histo-txt.py python script to generate a plot out of the scorecard .txt file containing histogram data.

   usage: plot-st-fragment-histo-txt.py [-h] -l TXT [-t TITLE] [-p PERCENT] [-n PNG_FILE_NAME] [-g]
   
   optional arguments:
   -h, --help            show this help message and exit
   -l TXT, --txt TXT     Path to .txt file.
   -t TITLE, --title TITLE
                           Add title to the plot.
   -p PERCENT, --percent PERCENT
                           Set percentage of the X-Axis around the highest hit-count. Default 10, i.e. 10%
   -n PNG_FILE_NAME, --png_file_name PNG_FILE_NAME
                           Define a new name for the PNG file output. Default: .txt name.
   -g, --no_grid         Disable grid plot
   

   

   $ sudo apt install python3-pandas python3-matplotlib
   $ python3 plot-st-fragment-histo-txt.py -l $(ls /var/opt/codesys/PlcLogic/IEC-61131-3_process-automation_scorecard* | head -n 1) -n ./my-st-fragment-run
   

   

   The following is the expected output for this example only 1 second execution time :

            0          1  2
   0         1      200.0  0
   1         2      400.0  0
   2         3      600.0  0
   3         4      800.0  0
   4         5     1000.0  0
   ...     ...        ... ..
   9995   9996  1999200.0  0
   9996   9997  1999400.0  0
   9997   9998  1999600.0  0
   9998   9999  1999800.0  0
   9999  10000  2000000.0  0
   
   [10000 rows x 3 columns]
         nanoSeconds  hitCount
   0           200.0         0
   1           400.0         0
   2           600.0         0
   3           800.0         0
   4          1000.0         0
   ...           ...       ...
   9995    1999200.0         0
   9996    1999400.0         0
   9997    1999600.0         0
   9998    1999800.0         0
   9999    2000000.0         0
   
   [10000 rows x 2 columns]
   Max nSec at: {'loc': 556, 'max': 111400.0}
   Max HitC at: {'loc': 441, 'max': 238}
   Number of X-Axis labels: 1441
   

   

   The following is the expected output for another example running for several hours:

   

For more information on running ST-Fragment performance tuning and containerization, refer to /opt/benchmarking/codesys/README.md.

Execute ST-Fragment from CODESYS IDE Project Archive

The following section is applicable to:

See also

This section assumes you are already familiar with CODESYS* and how to deploy a workload to the CODESYS* Linux* runtime. For a detailed tutorial, see section: Application #2: CODESYS OPC UA Publish/Subscribe

Note: Make sure that you have installed the correct version of the CODESYS* IDE. Refer to CODESYS IDE for additional details.

1. Download the ECI release archive, if not done already.

2. Copy the CODESYS_Example_Applications.zip archive from the ECI release archive (release-eci_#.#.zip) to the Microsoft Windows* system. This archive is located in the ECI release archive within the Edge-Controls-for-Industrial directory as follows:

   └── Edge-Controls-for-Industrial
       ├── Codesys_Example_Applications.zip
       ├── Dockerfiles.tar.gz
       └── eci-release.tar.gz
   

   

3. Extract the “CODESYS_Example_Applications.zip” archive. Navigate to the Benchmark directory.

   

4. Double-click PLCopen_ecs-st-fragment_Linux_3.5.x.x.project to open the project with CODESYS*.

   This project provides a basic set of ST-Fragment algorithms (base_algo) and tests (Test):

   

Download (Execute) Directly to Target

1. To download the application directly to a target, double-click the ‘device’ (CODESYS_Control_for_Linux_SL). Go to the Communications Settings tab, select a gateway, and scan the network for active targets.

2. Once a target is detected, double-click the target to select and establish a connection.

   Note: Make sure that the CODESYS* Linux* runtime is up and running on your target.

   

3. When the connection is established, click the login button (or press Alt + F8). If prompted for Click Yes to download the latest code…, select Yes. Next, click start (or press F5) to start the execution of the application. Once the application is running, [stop] will change to [run].

   

4. SSH into the target and check the location /var/opt/codesys/PlcLogic/ for the results file (.txt):

   root@eci-intel-0332:~# ls /var/opt/codesys/PlcLogic/ | grep .txt
   IEC-61131-3_process-automation_scorecard2020-2-12-18_14.txt
   
   root@eci-intel-0332:~# cat /var/opt/codesys/PlcLogic/IEC-61131-3_process-automation_scorecard2020-2-12-18_14.txt
   --- PLCopen 61131-3 Process Automation testcase scorecard ---
   
   Target: Id 5 Type 4102 Version 3.5.x.x
   Test at: 2020-2-12-18_14
   Platform: IEC 61131-3 workload -  Control Function Block
   Extern: Extern
   Code: Ram
   Measured timer resolution [ns]: 61
   Measured overhead of SysTimeGetNs [ns]: 59.9632
   
   Test (all values in ns)              Avg       Min       Max    StdDev Num Calls
   --------------------------------------------------------------------------------
   
   PID 1x (REAL)                    121.082   106.038  1517.037    39.364     24023
   PID 5x (REAL)                    391.802   369.038  9065.037    81.344     23067
   PID 10x (REAL)                   804.014   770.038  2984.037    87.164     22027
   KALMAN 1x filter (REAL)          307.249   266.038  3761.037    73.689     23430
   KALMAN 5x filter (REAL)          1330.85  1270.037  5260.037   105.735     20571
   KALMAN 10x filter (REAL)        2587.099  2493.037  8200.037   138.303     18009
   Logic+Arith 1K fragment         9131.948  8837.037  28886.04   657.531     10949
   Logic+Arith 5K fragment         43352.83  42700.04  57913.04  1104.249      4000
   Logic+Arith 10K fragment        86019.04  85064.04 100248.04  1337.769      2497
   

   

5. After the .txt file is created, click Stop (or press Shift + F8) to stop execution or keep the application running and Logout (or press Ctrl + F8) from the target.

Create a Boot Application (.app/.crc Files)

1. Instead of connecting to the target via the CODESYS* IDE, you can create boot application files and deploy your application in other ways. You can later insert these files to the target, if running CODESYS* in a container.

2. To create the boot application files, click Online - Create Boot Application:

   

3. Save the application to a new folder. For example, save the application to a new folder named my_application.

   

4. For easier handling and distribution, zip these files. Select all files (press Ctrl + a), right-click, and select Send to - Compressed (zipped) folder. Rename the zip file.

   

5. Boot application files as a .zip file:

Deploy a Boot Application (.app/.crc files)

This section only explains the deployment via a Docker* container. For additional points and details, refer to Microservice Architecture.

1. To deploy a boot application to Docker*, make sure that the target contains the codesys-control-apploader Docker* image.

   Check for the existing Docker* images on the target:

   $ docker images
   

   

   Here is a sample output:

   root@eci-intel-0332:~# docker images
   REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
   codesys-control-apploader   v0.2                68303ee0e33b        9 hours ago         248MB
   codesys-control             v0.2                e33c68e9aa19        30 hours ago        242MB
   debian                      buster              1b686a95ddbf        3 weeks ago         114MB
   

   

2. Reconnect to the target (for example, via SSH) and create a folder structure where the boot application will be copied to. Also, transfer the boot application (for example, .zip) to the target (for example, via SCP). For example:

   scp my_st-fragment-application.zip root@192.168.1.10:/home/root
   ssh root@192.168.1.10
   
   root@eci-intel-0332:~# unzip my_st-fragment-application.zip
   root@eci-intel-0332:~# mkdir PlcLogic/Application
   root@eci-intel-0332:~# cp Application.app PlcLogic/Application/
   root@eci-intel-0332:~# cp Application.crc PlcLogic/Application/
   
   root@eci-intel-0332:~# docker run -d --rm --name my_container --privileged -v /home/root/PlcLogic:/var/opt/codesys/PlcLogic/ -e SOURCE_CODESYS_APP_NAME=Application codesys-control-apploader
   bdd40c147dbb9669efe5e0cc831f86d6860387bf00b05101c8b2ceef8366e524
   
   root@eci-intel-0332:~# docker ps -a
   CONTAINER ID        IMAGE                       COMMAND                CREATED             STATUS              PORTS               NAMES
   bdd40c147dbb        codesys-control-apploader   "/run-codesysapp.sh"   6 seconds ago       Up 3 seconds                            my_container
   
   root@eci-intel-0332:~# ls -la PlcLogic/
   total 36
   drwxr-xr-x 8 root root 4096 Jun 30 19:32 .
   drwx------ 9 root root 4096 Jun 30 19:30 ..
   drwxr-xr-x 2 root root 4096 Jun 30 19:30 Application
   -rw-r--r-- 1 root root 1292 Jun 30 19:32 IEC-61131-3_process-automation_scorecard2020-6-30-17_32.txt
   drwxr-xr-x 2 root root 4096 Jun 30 19:32 _cnc
   drwxr-xr-x 2 root root 4096 Jun 30 19:32 ac_persistence
   drwxr-xr-x 2 root root 4096 Jun 30 19:32 alarms
   drwxr-xr-x 2 root root 4096 Jun 30 19:32 trend
   drwxrwxrwx 2 root root 4096 Jun 30 19:30 visu
   
   root@eci-intel-0332:~# cat PlcLogic/IEC-61131-3_process-automation_scorecard2020-6-30-17_32.txt
   --- PLCopen 61131-3 Process Automation testcase scorecard ---
   
   Target: Id 5 Type 4102 Version 3.5.x.x
   Test at: 2020-6-30-17_32
   Platform: IEC 61131-3 workload -  Control Function Block
   Extern: Extern
   Code: Ram
   Measured timer resolution [ns]: 90
   Measured overhead of SysTimeGetNs [ns]: 86.8312
   
   Test (all values in ns)              Avg       Min       Max    StdDev Num Calls
   --------------------------------------------------------------------------------
   
   PID 1x (REAL)                     198.75   153.168  383141.2  2204.627    111725
   PID 5x (REAL)                    671.154   513.168 6538235.0  21624.23     92601
   PID 10x (REAL)                  1296.243  1113.169  186760.2  2986.557     70028
   KALMAN 1x filter (REAL)          463.961   385.168  149259.2  1822.708     98754
   KALMAN 5x filter (REAL)         2074.018  1767.169  866158.2   6091.34     54644
   KALMAN 10x filter (REAL)        4061.857  3481.169  595034.2  5853.398     35561
   Logic+Arith 1K fragment         25245.01 12310.169     1.318 1180123.0     12438
   Logic+Arith 5K fragment         71906.81  61137.17 2381378.0  48442.89      2913
   Logic+Arith 10K fragment        143135.7 122236.17  650948.2  35581.53      1504
   
   root@eci-intel-0332:~# docker stop my_container
   

   

3. Start a Docker* container docker run ... and mount the boot application to the container -v /home/root/PlcLogic:/var/opt/codesys/PlcLogic/.

4. Verify whether the container is running:

   $ docker ps -a
   

   

5. While the container is running, check for the .txt results file frequently:

   $ ls -la PlcLogic/
   

   

   Once the results file is generated, stop the execution of the container:

   $ docker stop my_container
   

   

   Check the results file cat IEC-61131-3*.txt. You can also use the SCP command to securely copy the file to a PC for additional evaluation, visualization, or both.

Extend/Modify ST-Fragment Tests

This section does not discuss, in detail, the procedure to extend ST-Fragment tests, that is, the procedure to create and add ‘new’ function blocks (FB) to the base_algo and Test folders, as you can find in ST-Fragment-Algorithms. However, this section will use an existing test and extend it to show the principle.

1. For instance, right-click TEST_10x_FRAGMENT (FB) and select Copy. Right-click the Test folder and select Paste. You should see a copy of the TEST_10x_FRAGMENT (FB) as TEST_10x_FRAGMENT_1 (FB):

   

2. Right-click TEST_10x_FRAGMENT_1 (FB), select Refactoring > Rename ‘Test_10x_FRAGMENT_1’….

   

3. Click OK:

   

4. Double-click the TEST_15x_FRAGMENT (FB) function block to open it in the editor as structured text (ST). Add five additional ifrag function calls:

   

   FUNCTION_BLOCK TEST_15x_FRAGMENT EXTENDS FB_Base
   VAR
       ifrag0: IEC_FRAGMENT;
       ifrag1: IEC_FRAGMENT;
       ifrag2: IEC_FRAGMENT;
       ifrag3: IEC_FRAGMENT;
       ifrag4: IEC_FRAGMENT;
       ifrag5: IEC_FRAGMENT;
       ifrag6: IEC_FRAGMENT;
       ifrag7: IEC_FRAGMENT;
       ifrag8: IEC_FRAGMENT;
       ifrag9: IEC_FRAGMENT;
       ifrag10: IEC_FRAGMENT;
       ifrag11: IEC_FRAGMENT;
       ifrag12: IEC_FRAGMENT;
       ifrag13: IEC_FRAGMENT;
       ifrag14: IEC_FRAGMENT;
   END_VAR
   

   

   Your editor window is split into two sections: Variables and Code. Edit the above code in the Variables section.

5. Double-click the DoRun method under TEST_15x_FRAGMENT (FB) and add the ifrag function calls, five times, in the Code section, as shown in the following code snippet:

   ...
   
   ifrag9(In_B1:=A, In_I1:=B,In_R1:=C,In_R2:=D);
   R1:= ifrag9.Out_R1;
   I1:= ifrag9.Out_I1;
   B1:= ifrag9.Out_B1;
   
   ifrag10(In_B1:=A, In_I1:=B,In_R1:=C,In_R2:=D);
   R1:= ifrag10.Out_R1;
   I1:= ifrag10.Out_I1;
   B1:= ifrag10.Out_B1;
   
   ifrag11(In_B1:=A, In_I1:=B,In_R1:=C,In_R2:=D);
   R1:= ifrag11.Out_R1;
   I1:= ifrag11.Out_I1;
   B1:= ifrag11.Out_B1;
   
   ifrag12(In_B1:=A, In_I1:=B,In_R1:=C,In_R2:=D);
   R1:= ifrag12.Out_R1;
   I1:= ifrag12.Out_I1;
   B1:= ifrag12.Out_B1;
   
   ifrag13(In_B1:=A, In_I1:=B,In_R1:=C,In_R2:=D);
   R1:= ifrag13.Out_R1;
   I1:= ifrag13.Out_I1;
   B1:= ifrag13.Out_B1;
   
   ifrag14(In_B1:=A, In_I1:=B,In_R1:=C,In_R2:=D);
   R1:= ifrag14.Out_R1;
   I1:= ifrag14.Out_I1;
   B1:= ifrag14.Out_B1;
   
   SysTimeGetNs(st2);
   udiTime_ns := ANY_TO_UDINT(st2 - st1) ;
   udiNumOps := 1 ;
   

   

6. Double-click the fb_init method under TEST_15x_FRAGMENT (FB) and change the tdata.txt from Logic+Arith 10K fragment to Logic+Arith 15K fragment:

   tdata.txt := 'Logic+Arith 15K fragment ' ;
   

   

7. Add the extended/modified test to the test structure under the GVL global variable:

   

   The code in the Variable section might look similar to the following:

   VAR_GLOBAL
       g_strPlatform : STRING := 'IEC 61131-3 workload -  Control Function Block ';
       g_strRAM: STRING := 'Extern';
       g_strCode: STRING := 'Ram';
   
       hFile : SysFile.RTS_IEC_HANDLE;
       tracepoint   : FB_tracepoin
   
       i_1_PID: TEST_1x_PID;
       i_5_PID: TEST_5x_PID;
       i_10_PID: TEST_10x_PID;
       i_1_KALMAN: TEST_1x_KALMAN;
       i_5_KALMAN: TEST_5x_KALMAN;
       i_10_KALMAN: TEST_10x_KALMAN;
       iFRAGMENT1K: TEST_1x_FRAGMENT;
       iFRAGMENT5K: TEST_5x_FRAGMENT;
       iFRAGMENT10K: TEST_10x_FRAGMENT;
       iFRAGMENT15K: TEST_15x_FRAGMENT;
   END_VAR
   

   

8. Build the project to verify the changes. Follow the menu path Build > Build or press F11.

   

   Check the Messages - error(s), warning(s) and message(s) window.

   

9. To test your application, follow Download (Execute) Directly to Target or Create a Boot Application (.app/.crc Files)/Deploy a Boot Application (.app/.crc files).

   The results file (IEC-61131-3_process-automation_scorecard*.txt) will have a additional entry (see the last entry).

   --- PLCopen 61131-3 Process Automation testcase scorecard ---
   
   Target: Id 5 Type 4102 Version 3.5.x.x
   Test at: 2020-7-1-22_21
   Platform: IEC 61131-3 workload -  Control Function Block
   Extern: Extern
   Code: Ram
   Measured timer resolution [ns]: 61
   Measured overhead of SysTimeGetNs [ns]: 60.0672
   
   Test (all values in ns)              Avg       Min       Max    StdDev Num Calls
   --------------------------------------------------------------------------------
   
   PID 1x (REAL)                    115.865   105.938  1781.933    30.424     24113
   PID 5x (REAL)                    388.961   366.938  16443.93   195.049     23051
   PID 10x (REAL)                   800.809   766.938  9499.933   124.999     22010
   KALMAN 1x filter (REAL)          301.763   268.938  8786.933   117.475     23431
   KALMAN 5x filter (REAL)         1320.059  1263.933  8268.933   119.977     20612
   KALMAN 10x filter (REAL)        2599.164  2471.933 13181.933   179.799     17916
   Logic+Arith 1K fragment         9244.016  8985.933  28530.93   616.327     10667
   Logic+Arith 5K fragment         43830.39  42844.94  63562.94  2369.021      4000
   Logic+Arith 10K fragment        86871.29  85159.93 114983.93  3323.186      2496
   Logic+Arith 15K fragment       129889.08 128254.93  148228.9  2826.522      1500
   

   

Enable Tracing

1. To enable the tracing functionality within the CODESYS* application, right-click Application and select Properties….

   

2. In the Properties dialog, go to the Build tab. Add enable_ftrace in the Compilerdefines text box. Click Apply and OK.

   

3. Rebuild the application. Once the trace enabled application is deployed and running you can check cat /sys/kernel/debug/tracing/trace for trace messages. For example:

   # tracer: nop
   #
   # nop latency trace v1.1.5 on 4.19.94-rt38-intel-pk-preempt-rt
   # --------------------------------------------------------------------
   # latency: 0 us, #152425/152425, CPU#0 | (M:preempt VP:0, KP:0, SP:0 HP:0 #P:4)
   #    -----------------
   #    | task: -0 (uid:0 nice:0 policy:0 rt_prio:0)
   #    -----------------
   #
   #                  _--------=> CPU#
   #                 / _-------=> irqs-off
   #                | / _------=> need-resched
   #                || / _-----=> need-resched_lazy
   #                ||| / _----=> hardirq/softirq
   #                |||| / _---=> preempt-depth
   #                ||||| / _--=> preempt-lazy-depth
   #                |||||| / _-=> migrate-disable
   #                ||||||| /     delay
   # cmd     pid    |||||||| time   |  caller
   #     \   /      ||||||||   \    |  /
      <...>-5179    0....... 5988902us : tracing_mark_write: E|3|measure-trace_marker-overhead
      <...>-5179    0....... 5988905us : tracing_mark_write: E|2|measure-trace_marker-overhead
      <...>-5179    0....... 5988908us!: tracing_mark_write: E|1|measure-trace_marker-overhead
      <...>-5179    0....... 5989156us+: tracing_mark_write: B|0|PID 1x (REAL)
      <...>-5179    0....... 5989215us : tracing_mark_write: X|0|1623806.4|1623806.4
      <...>-5179    0....... 5989225us : tracing_mark_write: X|1|165806.4|894806.4
      <...>-5179    0....... 5989233us : tracing_mark_write: X|2|160806.4|650139.733333333
      <...>-5179    0....... 5989242us : tracing_mark_write: X|3|157806.4|527056.4
      <...>-5179    0....... 5989250us : tracing_mark_write: X|4|166806.4|455006.4
      <...>-5179    0....... 5989258us : tracing_mark_write: X|5|160806.4|405973.066666667
      <...>-5179    0....... 5989266us : tracing_mark_write: X|6|160806.4|370949.257142857
      <...>-5179    0....... 5989274us : tracing_mark_write: X|7|161806.4|344806.4
      <...>-5179    0....... 5989282us : tracing_mark_write: X|8|160806.4|324361.955555555
      <...>-5179    0....... 5989291us : tracing_mark_write: X|9|159806.4|307906.4
      <...>-5179    0....... 5989299us : tracing_mark_write: X|10|161806.4|294624.581818182