Attention

You are viewing an older version of the documentation. The latest version is v3.3.

systemd Network Manager TSN Endstation Configuration for Intel® Ethernet Controllers

Intel® ECI TSN Endstation introduces IEEE 802.1AS and IEEE 802.1Q-2018-EST (for example, Virtual LANs (VLANs), Earliest TxTime First (NET_SCHED_ETF) QDisc and Time Aware Priority (NET_SCHED_TAPRIO) QDisc) as systemd runtime services.

../../../_images/8021q_seq_diag_01.png

Intel® ECI makes generic assumptions that the Linux TSN Endstation settings are managed using ONLY distribution standard systemd Network Manager configurations rules for achieving IEEE 802.1Q-EST interoperability:

Attention

Intel® ECI maintains both feature backports and new Earliest TxTime First (NET_SCHED_ETF) QDisc and Time Aware Priority (NET_SCHED_TAPRIO) QDisc contribution for major Linux* Debian* 11 (Bullseye) and Canonical Ubuntu* 22.04 (Jammy Jellyfish) mainlines, but all distribution support is on the way to upstream.

The following section is applicable to:

../../../_images/target_generic6.png

Setup the ECI APT repository, then run the following commands to install this component:

Install from individual Deb package

Install TSN Endstation boot services by systemd tsn-endstation-init and dependencies:

sudo apt install tsn-endstation-init

For example, on Debian* distribution set with ECI APT repository, the following should be displayed:

The following additional packages will be installed:
ethtool network-irq-affinity
The following NEW packages will be installed:
ethtool network-irq-affinity tsn-endstation-init
0 upgraded, 3 newly installed, 0 to remove and 26 not upgraded.
Need to get 188 kB/192 kB of archives.
After this operation, 662 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 tsn-endstation-init all 3.0-bullseye-2 [3488 B]
Get:2 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 isar/main amd64 ethtool amd64 1:5.9-1+igcfpe [182 kB]
Get:3 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 isar/main amd64 network-irq-affinity all 1.1 [5984 B]
Fetched 188 kB in 0s (7964 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package ethtool.
(Reading database ... 62295 files and directories currently installed.)
Preparing to unpack .../ethtool_1%3a5.9-1+igcfpe_amd64.deb ...
Unpacking ethtool (1:5.9-1+igcfpe) ...
Selecting previously unselected package network-irq-affinity.
Preparing to unpack .../network-irq-affinity_1.1_all.deb ...
Unpacking network-irq-affinity (1.1) ...
Selecting previously unselected package tsn-endstation-init.
Preparing to unpack .../tsn-endstation-init_3.0-bullseye-2_all.deb ...
Unpacking tsn-endstation-init (3.0-bullseye-2) ...
Setting up network-irq-affinity (1.1) ...
Setting up ethtool (1:5.9-1+igcfpe) ...
Setting up tsn-endstation-init (3.0-bullseye-2) ...
Processing triggers for man-db (2.9.4-2) ...

Install TSN Endstation IEEE 802.1AS by systemd tsn-endstation-timesync runtime service:

sudo apt install tsn-endstation-timesync iotg-gptp-configs

For example, on Debian* distribution set with ECI APT repository, the following should be displayed:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
linuxptp
The following NEW packages will be installed:
iotg-gptp-configs linuxptp tsn-endstation-timesync
0 upgraded, 3 newly installed, 0 to remove and 26 not upgraded.
Need to get 181 kB/185 kB of archives.
After this operation, 823 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64  tsn-endstation-timesync all 3.0-bullseye-2 [4252 B]
Get:2 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 linuxptp amd64 3.1 [177 kB]
Get:3 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 iotg-gptp-configs amd64 0.9.4-bullseye-1.21180fce7a [3820 B]
Fetched 181 kB in 0s (7568 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package linuxptp.
(Reading database ... 62196 files and directories currently installed.)
Preparing to unpack .../linuxptp_3.1_amd64.deb ...
Unpacking linuxptp (3.1) ...
Selecting previously unselected package iotg-gptp-configs.
Preparing to unpack .../iotg-gptp-configs_0.9.4-bullseye-1.21180fce7a_amd64.deb ...
Unpacking iotg-gptp-configs (0.9.4-bullseye-1.21180fce7a) ...
Selecting previously unselected package tsn-endstation-timesync.
Preparing to unpack .../tsn-endstation-timesync_3.0-bullseye-2_all.deb ...
Unpacking tsn-endstation-timesync (3.0-bullseye-2) ...
Setting up linuxptp (3.1) ...
Setting up iotg-gptp-configs (0.9.4-bullseye-1.21180fce7a) ...
Setting up tsn-endstation-timesync (3.0-bullseye-2) ...
Processing triggers for man-db (2.9.4-2) ...

Check the Intel ECI systemd packages upgrade with Earliest TxTime First (NET_SCHED_ETF) QDisc and Time Aware Priority (NET_SCHED_TAPRIO) QDisc features:

sudo apt-cache policy systemd

For example, on Debian* distribution set with ECI APT repository, the following should be displayed:

root@eci-intel-3d56:~#
systemd:
Installed: 247.3-7
Candidate: 247.3-7-bullseye+etf+taprio
Version table:
   247.3-7-bullseye+etf+taprio 1000
      1000 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye/dists/isar/InRelease isar/main amd64 Packages
*** 247.3-7 500
      500 http://lava.lavalab/mirrors/debian bullseye/main amd64 Packages
      100 /var/lib/dpkg/status

Install TSN Endstation IEEE 802.1Q-2018-EST runtime services tsn-endstation-netlink with systemd packages upgrade, enabling the Earliest TxTime First (NET_SCHED_ETF) QDisc and Time Aware Priority (NET_SCHED_TAPRIO) QDisc built-in features:

sudo apt install systemd tsn-endstation-netlink

For example, on Debian* distribution set with ECI APT repository, the following should be displayed:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libpam-systemd libsystemd0 checkclocks
Suggested packages:
systemd-container
Recommended packages:
systemd-timesyncd | time-daemon
The following NEW packages will be installed:
tsn-endstation-netlink checkclocks
The following packages will be upgraded:
libpam-systemd libsystemd0 systemd
3 upgraded, 1 newly installed, 0 to remove and 26 not upgraded.
Need to get 5168 kB/5171 kB of archives.
After this operation, 65.5 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 tsn-endstation-netlink all 3.0-bullseye-2 [3012 B]
Get:2 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 libpam-systemd amd64 247.3-7-bullseye+etf+taprio [283 kB]
Get:3 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 systemd amd64 247.3-7-bullseye+etf+taprio [4510 kB]
Get:4 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 libsystemd0 amd64 247.3-7-bullseye+etf+taprio [376 kB]
Get:5 http://lava.lavalab/wheeljack/apt-repos/ECI/3.0/bullseye/eci-bullseye isar/main amd64 checkclocks amd64 0.2 [14.2 kB]
Fetched 5168 kB in 0s (58.7 MB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 62337 files and directories currently installed.)
Preparing to unpack .../libpam-systemd_247.3-7-bullseye+etf+taprio_amd64.deb ...
Unpacking libpam-systemd:amd64 (247.3-7-bullseye+etf+taprio) over (247.3-7) ...
Preparing to unpack .../systemd_247.3-7-bullseye+etf+taprio_amd64.deb ...
Unpacking systemd (247.3-7-bullseye+etf+taprio) over (247.3-7) ...
Preparing to unpack .../libsystemd0_247.3-7-bullseye+etf+taprio_amd64.deb ...
Unpacking libsystemd0:amd64 (247.3-7-bullseye+etf+taprio) over (247.3-7) ...
Setting up libsystemd0:amd64 (247.3-7-bullseye+etf+taprio) ...
Selecting previously unselected package tsn-endstation-netlink.
(Reading database ... 62337 files and directories currently installed.)
Preparing to unpack .../checkclocks_0.2_amd64.deb ...
Unpacking checkclocks (0.2) ...
Selecting previously unselected package tsn-endstation-netlink.
Preparing to unpack .../tsn-endstation-netlink_3.0-bullseye-2_all.deb ...
Unpacking tsn-endstation-netlink (3.0-bullseye-2) ...
Setting up checkclocks (0.2) ...
Setting up systemd (247.3-7-bullseye+etf+taprio) ...
Setting up libpam-systemd:amd64 (247.3-7-bullseye+etf+taprio) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 78.)
debconf: falling back to frontend: Readline
Setting up tsn-endstation-netlink (3.0-bullseye-2) ...
Processing triggers for dbus (1.12.20-2) ...
Processing triggers for libc-bin (2.31-13+deb11u3) ...
Processing triggers for man-db (2.9.4-2) ...

eci-ethirqs@.service- Networking Interrupts CPU Affinity for Intel® Ethernet Controllers

Intel® Ethernet Controllers can provide several MSI interrupts to improve the network throughput.

Intel® ECI makes generic assumptions that traffic-class prioritization is achieved on cores (for example, real-time communication tasks) other than default core 0 (for example, house-keeping tasks) than Network Interrupts Affinity to CPU critical Ethernet device interrupts

This section describes how to selectively map MSI interrupts to particular CPU cores for each particular Intel® Ethernet Controller using systemctl.

The following section is applicable to:

../../../_images/target_generic6.png

  1. Identify the available Intel® Industrial Ethernet Controllers:

    $ lspci -nnn | grep "Ethernet controller"

    In this example, the output pattern is ${device_id} ${description}, where ${device_id} is 01:00.0.

    Important

    Identify the Intel® Industrial Ethernet Controller interface that is wired to the TSN switch device or to another TSN end-station:

  2. Confirm the Linux network interface name of the Intel® Industrial Ethernet Controllers:

    $ networkctl status enp1s0

    The expected output should be similar to the following, with the default 51-wired-dhcp.network and 99-default.link networkd and udev rules setting:

    root@eci-intel-23f0:~# networkctl status enp1s0
? 5: enp1s0
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: /usr/lib/systemd/network/51-wired-dhcp.network
                        Type: ether
                        State: routable (configured)
                        Path: pci-0000:01:00.0
                        Driver: igc
                        Vendor: Intel Corporation
                        Model: Ethernet Controller I225-LM
                  HW Address: 00:a0:c9:00:00:00 (Intel Corporation)
                           MTU: 1500 (min: 68, max: 9216)
                        QDisc: mq
IPv6 Address Generation Mode: eui64
         Queue Length (Tx/Rx): 4/4
            Auto negotiation: yes
                        Speed: 1Gbps
                        Duplex: full
                        Port: tp
                     Address: 10.10.10.19 (DHCP4 via 10.10.10.1)
                              fe80::2a0:c9ff:fe00:0
                     Gateway: 10.10.10.1 (ALFA, INC.)
                           DNS: 10.10.10.1
               DHCP4 Client ID: IAID:0x56504d98/DUID
            DHCP6 Client DUID: DUID-EN/Vendor:0000ab11983c97c412262c820000
                  Connected To: cyclone5 on port 1c:a0:d3:20:06:1a (CE03)

eci-intel-23f0 systemd-networkd[328]: enp1s0: Gained carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv4 address 10.10.10.19/24 via 10.10.10.1
eci-intel-23f0 systemd-networkd[328]: enp1s0: Lost carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCP lease lost
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv6 lease lost
eci-intel-23f0 systemd-networkd[328]: enp1s0: Gained carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: Lost carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv6 lease lost
eci-intel-23f0 systemd-networkd[328]: enp1s0: Gained carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv4 address 10.10.10.19/24 via 10.10.10.1

    The output text is the interface name of i225 NIC, marked as ${interface_name}.

  3. Enable, start, and report the Intel® Industrial Ethernet Controller MSI interrupts affinity CPU cores for the particular Linux network interface name:

    $ systemctl enable eci-ethirqs@enp1s0.service && systemctl start eci-ethirqs@enp1s0.service && systemctl status eci-ethirqs@enp1s0.service

    The expected output should be similar to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-ethirqs@enp1s0.service ? /lib/systemd/system/eci-ethirqs@.service.
? eci-ethirqs@enp1s0.service - ECI: set CPU core to IRQ Ethernet device affinity
   Loaded: loaded (/lib/systemd/system/eci-ethirqs@.service; enabled; vendor preset: enabled)
   Drop-In: /usr/lib/systemd/system/service.d
            └─10-override-protect-proc.conf
   Active: active (exited) since Fri 2022-10-07 10:09:00 UTC; 4 years 1 months left
   Main PID: 23690 (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 9164)
   Memory: 0B
      CPU: 0
   CGroup: /system.slice/system-eci\x2dethirqs.slice/eci-ethirqs@enp1s0.service

eci-intel-23f0 systemd[1]: Starting ECI: set CPU core to IRQ Ethernet device affinity...
eci-intel-23f0 network-irq-affinity[23690]: network-irq-affinity: Assigning enp1s0-TxRx-3 on IRQ 136 to CPU 1
eci-intel-23f0 network-irq-affinity[23690]: network-irq-affinity: Assigning enp1s0-TxRx-2 on IRQ 135 to CPU 1
eci-intel-23f0 network-irq-affinity[23690]: network-irq-affinity: Assigning enp1s0-TxRx-1 on IRQ 134 to CPU 1
eci-intel-23f0 network-irq-affinity[23690]: network-irq-affinity: Assigning enp1s0-TxRx-0 on IRQ 133 to CPU 0
eci-intel-23f0 network-irq-affinity[23690]: network-irq-affinity: Assigning enp1s0 on IRQ 132 to CPU 0
eci-intel-23f0 systemd[1]: Finished ECI: set CPU core to IRQ Ethernet device affinity.

  4. Confirm whether the interrupt mapping is effective:

    $ cat /proc/interrupts | grep enp1s0

    The expected output should be similar to the following:

    root@eci-intel-3f6c:~# cat /proc/interrupts | grep enp3s0
            CPU0       CPU1       CPU2       CPU3
132:       2508          0          0          0  IR-PCI-MSI 524288-edge      enp1s0
133:     876072          0          0          0  IR-PCI-MSI 524289-edge      enp1s0-TxRx-0
134:     165157       4380          0          0  IR-PCI-MSI 524290-edge      enp1s0-TxRx-1
135:     165136        626          0          0  IR-PCI-MSI 524291-edge      enp1s0-TxRx-2
136:     164913        617          0          0  IR-PCI-MSI 524292-edge      enp1s0-TxRx-3

eci-ptp4l-p2p@.service - gPTP Time Synchronization Service for Intel® Ethernet Controllers

This section describes how to selectively enable IEEE 802.1as gPTP Profile Essential for each particular Intel® Ethernet Controller type using systemctl.

The following section is applicable to:

../../../_images/target_generic6.png

  1. Set up Intel® Ethernet Controllers according to eci-ethirqs@.service- Networking Interrupts CPU Affinity for Intel® Ethernet Controllers.

  2. Enable, start, and report the Intel® Industrial Ethernet Controllers IEEE 802.1AS on the particular Linux network interface name:

    $ systemctl enable eci-ptp4l-p2p@enp1s0.service && systemctl start eci-ptp4l-p2p@enp1s0.service && systemctl status eci-ptp4l-p2p@enp1s0.service

    The expected output should be similar to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-ptp4l-p2p@enp1s0.service ? /lib/systemd/system/eci-ptp4l-p2p@.service.
? eci-ptp4l-p2p@enp1s0.service - ECI: Precision Time Protocol (PTP) service for enp1s0
   Loaded: loaded (/lib/systemd/system/eci-ptp4l-p2p@.service; enabled; vendor preset: enabled)
   Drop-In: /usr/lib/systemd/system/service.d
            └─10-override-protect-proc.conf
   Active: active (running) since Fri 2022-10-07 10:09:02 UTC; 4 years 1 months left
      Docs: man:ptp4l
   Main PID: 23783 (ptp4l)
      Tasks: 1 (limit: 9164)
   Memory: 572.0K
      CPU: 35ms
   CGroup: /system.slice/system-eci\x2dptp4l\x2dp2p.slice/eci-ptp4l-p2p@enp1s0.service
            └─23783 ptp4l -f /run/eci/ptp4l-enp1s0.conf -i enp1s0 -ml 6 --step_threshold=1 --socket_priority=2

eci-intel-23f0 ptp4l[23783]: ptp4l[329420.035]: rms    4 max    6 freq +20886 +/-   2 delay    12 +/-   0
eci-intel-23f0 ptp4l[23783]: [329420.035] rms    4 max    6 freq +20886 +/-   2 delay    12 +/-   0
eci-intel-23f0 ptp4l[23783]: ptp4l[329421.037]: rms    2 max    4 freq +20888 +/-   3 delay    12 +/-   0
eci-intel-23f0 ptp4l[23783]: [329421.037] rms    2 max    4 freq +20888 +/-   3 delay    12 +/-   0
eci-intel-23f0 ptp4l[23783]: ptp4l[329422.040]: rms    2 max    3 freq +20887 +/-   3 delay    12 +/-   0
eci-intel-23f0 ptp4l[23783]: [329422.040] rms    2 max    3 freq +20887 +/-   3 delay    12 +/-   0
eci-intel-23f0 ptp4l[23783]: ptp4l[329423.042]: rms    1 max    2 freq +20887 +/-   1 delay    13 +/-   0
eci-intel-23f0 ptp4l[23783]: [329423.042] rms    1 max    2 freq +20887 +/-   1 delay    13 +/-   0
eci-intel-23f0 ptp4l[23783]: ptp4l[329424.045]: rms    1 max    2 freq +20885 +/-   2 delay    13 +/-   0
eci-intel-23f0 ptp4l[23783]: [329424.045] rms    1 max    2 freq +20885 +/-   2 delay    13 +/-   0

    Attention

    You might need to edit Intel® Industrial Ethernet controllers IEEE 802.1AS to match a specific ODM Ethernet PHY configuration available on the the iotg-gptp-configs ECI package:

    $ ls -l /opt/intel/iotg_tsn_ref_sw/common/

    Select a gPTP template from the list:

    -rw-r--r-- 1 root root 526 Apr 21  2021 gPTP.cfg
-rw-r--r-- 1 root root 540 Apr 21  2021 gPTP_RGMII-MV1510-1G.cfg
-rw-r--r-- 1 root root 540 Apr 21  2021 gPTP_SGMII-MV2110-1G.cfg
-rw-r--r-- 1 root root 543 Apr 21  2021 gPTP_SGMII-MV2110-2_5G.cfg
-rw-r--r-- 1 root root 541 Apr 21  2021 gPTP_TI-1G.cfg
-rw-r--r-- 1 root root 552 Apr 21  2021 gPTP_i225-1G.cfg

    Edit gPTP settings required for certain Ethernet PHY and speed:

    $ vi /run/eci/ptp4l-enp1s0.conf
#
# 802.1AS example configuration containing those attributes which
# differ from the defaults.  See the file, default.cfg, for the
# complete list of available options.
#
[global]
gmCapable               1
priority1               248
priority2               248
logAnnounceInterval     0
logSyncInterval         -3
syncReceiptTimeout      3
neighborPropDelayThresh 800
min_neighbor_prop_delay -20000000
assume_two_step         1
path_trace_enabled      1
follow_up_info          1
transportSpecific       0x1
ptp_dst_mac             01:80:C2:00:00:0E
network_transport       L2
delay_mechanism         P2P
tx_timestamp_timeout    100
first_step_threshold    0.0

    Then, restart to test and review gPTP settings:

    $ systemctl restart eci-ptp4l-p2p@enp1s0.service && journalctl -xe -u eci-ptp4l-p2p@enp1s0.service

eci-phc2sys@.service - PHC Time Synchronization Service for Intel® Ethernet Controllers

This section describes how to selectively enable IEEE 802.1as gPTP Profile Essential with CPU time on a particular Intel® Ethernet Controller type using systemctl.

The following section is applicable to:

../../../_images/target_generic6.png

  1. Set up Intel® Ethernet controllers according to eci-ptp4l-p2p@.service - gPTP Time Synchronization Service for Intel® Ethernet Controllers.

  2. Enable, start, and report the Intel® Industrial Ethernet Controllers’ PHC time sync on the particular Linux network interface name:

    $ systemctl enable eci-phc2sys@enp1s0.service && systemctl start eci-phc2sys@enp1s0.service && systemctl status eci-phc2sys@enp1s0.service

    The expected output should be similar to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-phc2sys@enp1s0.service → /lib/systemd/system/eci-phc2sys@enp1s0.service.
? eci-phc2sys@enp1s0.service - ECI: Synchronize system clock or PTP hardware clock (PHC)
   Loaded: loaded (/lib/systemd/system/eci-phc2sys@.service; enabled; vendor preset: enabled)
   Drop-In: /usr/lib/systemd/system/service.d
            └─10-override-protect-proc.conf
   Active: active (running) since Fri 2022-10-07 10:09:02 UTC; 4 years 1 months left
      Docs: man:phc2sys
   Main PID: 23786 (phc2sys)
      Tasks: 1 (limit: 9164)
   Memory: 484.0K
      CPU: 48ms
   CGroup: /system.slice/system-eci\x2dphc2sys.slice/eci-phc2sys@enp1s0.service
            └─23786 phc2sys -s enp1s0 -c CLOCK_REALTIME --step_threshold=1 --transportSpecific=1 -w -ml 7

eci-intel-23f0 phc2sys[23786]: phc2sys[330852.178]: CLOCK_REALTIME phc offset         2 s2 freq  +17504 delay   1509
eci-intel-23f0 phc2sys[23786]: [330852.178] CLOCK_REALTIME phc offset         2 s2 freq  +17504 delay   1509
eci-intel-23f0 phc2sys[23786]: phc2sys[330853.178]: CLOCK_REALTIME phc offset         1 s2 freq  +17503 delay   1509
eci-intel-23f0 phc2sys[23786]: [330853.178] CLOCK_REALTIME phc offset         1 s2 freq  +17503 delay   1509
eci-intel-23f0 phc2sys[23786]: phc2sys[330854.178]: CLOCK_REALTIME phc offset         4 s2 freq  +17507 delay   1510
eci-intel-23f0 phc2sys[23786]: [330854.178] CLOCK_REALTIME phc offset         4 s2 freq  +17507 delay   1510
eci-intel-23f0 phc2sys[23786]: phc2sys[330855.178]: CLOCK_REALTIME phc offset         0 s2 freq  +17504 delay   1507
eci-intel-23f0 phc2sys[23786]: [330855.178] CLOCK_REALTIME phc offset         0 s2 freq  +17504 delay   1507
eci-intel-23f0 phc2sys[23786]: phc2sys[330856.178]: CLOCK_REALTIME phc offset        -2 s2 freq  +17502 delay   1512
eci-intel-23f0 phc2sys[23786]: [330856.178] CLOCK_REALTIME phc offset        -2 s2 freq  +17502 delay   1512

eci-tsn-wait-sync-8021as@.service - TSN 802.1AS Services for Intel® Ethernet Controllers

This section describes how to selectively wait for IEEE 802.1as gPTP Profile Essential time synchronization on a particular Intel® Ethernet Controller using systemctl.

The following section is applicable to:

../../../_images/target_generic6.png

  1. Set up Intel® Ethernet Controllers according to eci-phc2sys@.service - PHC Time Synchronization Service for Intel® Ethernet Controllers.

  2. Enable, start, and wait for SUCCESS status of check_clocks for Intel® Industrial Ethernet Controllers on a particular Linux network interface name:

    $ systemctl enable eci-tsn-wait-sync-8021as@enp1s0.service && systemctl start eci-tsn-wait-sync-8021as@enp1s0.service && systemctl status eci-tsn-wait-sync-8021as@enp1s0.service

    The expected output should be similar after < 15 seconds to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-tsn-wait-sync-8021as@enp1s0.service ? /lib/systemd/system/eci-tsn-wait-sync-8021as@.service.
? eci-tsn-wait-sync-8021as@enp1s0.service - ECI: Wait for clocks synchronization
   Loaded: loaded (/lib/systemd/system/eci-tsn-wait-sync-8021as@.service; enabled; vendor preset: enabled)
   Drop-In: /usr/lib/systemd/system/service.d
            └─10-override-protect-proc.conf
   Active: active (exited) since Tue 2018-08-28 08:59:30 UTC; 27ms ago
   Process: 24138 ExecStart=/usr/bin/bash /usr/libexec/eci-tsn-wait-timesync.sh enp1s0 15 (code=exited, status=0/SUCCESS)
   Main PID: 24138 (code=exited, status=0/SUCCESS)
      CPU: 3ms

eci-intel-23f0 bash[24139]: rt tstamp:          1535446769376683711
eci-intel-23f0 bash[24139]: tai tstamp:         1535446806376683783
eci-intel-23f0 bash[24139]: phc tstamp:         1535446806376685317
eci-intel-23f0 bash[24139]: rt latency:         30
eci-intel-23f0 bash[24139]: tai latency:        30
eci-intel-23f0 bash[24139]: phc latency:        2080
eci-intel-23f0 bash[24139]: phc-rt delta:       37000001606
eci-intel-23f0 bash[24139]: phc-tai delta:      1534
eci-intel-23f0 bash[24139]: Clocks on this system are synchronized :)
eci-intel-23f0 systemd[1]: Finished ECI: Wait for clocks synchronization.

  3. Optionally, enable, start, and monitor check_clocks for Intel® Industrial Ethernet Controllers on a particular Linux network interface name:

    Important

    gPTP monitoring and tracing is only available on Intel® Ethernet Controller supporting PPS_CAPTUREASSERT callback pps_source_info API:

    as well as Intel® Ethernet Controller supporting PTP_SYS_OFFSET_PRECISE callback ptp_clock_info API (non-exhaustive list) :

    $ systemctl enable eci-tsn-track-sync-8021as@enp1s0.service && systemctl start eci-tsn-track-sync-8021as@enp1s0.service && systemctl status eci-tsn-track-sync-8021as@enp1s0.service

    The expected output should be similar to the following:

    ? eci-tsn-track-sync-8021as@enp1s0.service - ECI: Check that time is synchronized
   Loaded: loaded (/lib/systemd/system/eci-tsn-track-sync-8021as@.service; enabled; vendor preset: enabled)
   Drop-In: /usr/lib/systemd/system/service.d
            └─10-override-protect-proc.conf
   Active: active (running) since Tue 2018-08-28 09:01:42 UTC; 22ms ago
   Main PID: 24159 (check_clocks)
      Tasks: 1 (limit: 9164)
   Memory: 284.0K
      CPU: 1ms
   CGroup: /system.slice/system-eci\x2dtsn\x2dtrack\x2dsync\x2d8021as.slice/eci-tsn-track-sync-8021as@enp1s0.service
            └─24159 check_clocks -d enp1s0 -va

eci-intel-23f0 systemd[1]: Starting ECI: Check that time is synchronized...
eci-intel-23f0 systemd[1]: Started ECI: Check that time is synchronized.
eci-intel-23f0 check_clocks[24159]: Dumping timestamps and deltas
eci-intel-23f0 check_clocks[24159]: /dev/pps0 can be enabled from /dev/ptp0 on device enp1s0
eci-intel-23f0 check_clocks[24159]: trying PPS source "/dev/pps0"
eci-intel-23f0 check_clocks[24159]: found PPS source "/dev/pps0"
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446903         sequence: 1928         offset: 3569         trt_lat=8184         tai_lat=37000008267
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446904         sequence: 1929         offset: 3688         trt_lat=4427         tai_lat=37000004486
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446905         sequence: 1930         offset: 4284         trt_lat=4721         tai_lat=37000004780
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446906         sequence: 1931         offset: 3651         trt_lat=4371         tai_lat=37000004415
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446907         sequence: 1932         offset: 4308         trt_lat=4446         tai_lat=37000004471
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446908         sequence: 1933         offset: 3663         trt_lat=4466         tai_lat=37000004493
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446909         sequence: 1934         offset: 3644         trt_lat=4313         tai_lat=37000004346
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446910         sequence: 1935         offset: 4233         trt_lat=4505         tai_lat=37000004555
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446911         sequence: 1936         offset: 3662         trt_lat=4363         tai_lat=37000004397

    For more logs:

    $ journalctl -xe -u eci-tsn-track-sync-8021as@enp1s0.service | more

  4. Report the check_clocks statistics to ftrace pulse-per-second sequence id-1075 to id-1084 interval for Intel® Industrial Ethernet controllers on the particular Linux network interface name:

    less /sys/kernel/tracing/per_cpu/cpu1/trace

    # tracer: nop
#
# entries-in-buffer/entries-written: 7/7   #P:4
#
#                                _-------=> irqs-off
#                               / _------=> need-resched
#                              | / _-----=> need-resched-lazy
#                              || / _----=> hardirq/softirq
#                              ||| / _---=> preempt-depth
#                              |||| / _--=> preempt-lazy-depth
#                              ||||| / _-=> migrate-disable
#                              |||||| /     delay
#           TASK-PID     CPU#  |||||||  TIMESTAMP  FUNCTION
#              | |         |   |||||||      |         |
         <...>-24159   [001] ....... 331549.637493: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447154.3793 seq=2179 tai_lat=37000004746 ep=enp1s0
         <...>-24159   [001] ....... 331550.637510: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447155.3693 seq=2180 tai_lat=37000004506 ep=enp1s0
         <...>-24159   [001] ....... 331568.637825: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447173.3772 seq=2198 tai_lat=37000004657 ep=enp1s0
         <...>-24159   [001] ....... 331569.637842: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447174.3710 seq=2199 tai_lat=37000004450 ep=enp1s0
         <...>-24159   [001] ....... 331570.637860: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447175.4131 seq=2200 tai_lat=37000004474 ep=enp1s0
         <...>-24159   [001] ....... 331571.637877: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447176.3723 seq=2201 tai_lat=37000004374 ep=enp1s0
         <...>-24159   [001] ....... 331572.637895: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447177.4119 seq=2202 tai_lat=37000004408 ep=enp1s0

  5. Visualize IEEE 802.1Q packet tracing in Timeview-UI for Intel® Industrial Ethernet controllers on the particular Linux network interface name:

    ../../../_images/ptpv2_capture_01.png

Note:

To override International Atomic Time (TAI) and UTC currentUtcOffset default 37-second value:

$ pmc -u -b 0 -t 1 \
      "SET GRANDMASTER_SETTINGS_NP clockClass 248 clockAccuracy 0xfe offsetScaledLogVariance 0xffff currentUtcOffset 37 leap61 0 leap59 0 currentUtcOffsetValid 1 ptpTimescale 1 timeTraceable 1 frequencyTraceable 0 timeSource 0xa0"

The expected output should be similar to the following:

sending: SET GRANDMASTER_SETTINGS_NP
   2046a1.fffe.056bdc-0 seq 0 RESPONSE MANAGEMENT GRANDMASTER_SETTINGS_NP
            clockClass              248
            clockAccuracy           0xfe
            offsetScaledLogVariance 0xffff
            currentUtcOffset        0
            leap61                  0
            leap59                  0
            currentUtcOffsetValid   1
            ptpTimescale            1
            timeTraceable           1
            frequencyTraceable      0
            timeSource              0xa0