Attention

You are viewing an older version of the documentation. The latest version is v3.3.

systemd Network Manager TSN Endstation Configuration for Intel® Ethernet Controllers

Intel® ECI TSN Endstation introduces IEEE 802.1AS and IEEE 802.1Q-2018-EST (for example, Virtual LANs (VLANs), Earliest TxTime First (NET_SCHED_ETF) QDisc and Time Aware Priority (NET_SCHED_TAPRIO) QDisc) as systemd runtime services.

../../../_images/8021q_seq_diag_01.png

Intel® ECI makes generic assumptions that all Linux TSN Endstation settings shall be managed using Linux distribution systemd Network Manager standard configurations rules for achieving IEEE 802.1Q-EST interoperability:

Attention

Intel® ECI maintains both feature backports and new Earliest TxTime First (NET_SCHED_ETF) QDisc and Time Aware Priority (NET_SCHED_TAPRIO) QDisc contribution for major Linux Debian 12 (Bookworm) , Canonical® Ubuntu® 22.04 (Jammy Jellyfish) and Red Hat Enterprise Linux 9 maintained packages, but all distribution support is on the way to upstream.

The following section is applicable to:

../../../_images/target_generic6.png

Setup the ECI repository, then run the following commands to install this component:

logo_debian logo_ubuntu

Install from individual Deb package

Install TSN Endstation IEEE 802.1AS and IEEE 802.1Q-2018-EST systemd unit runtime service tsn-endstation-timesync and tsn-endstation-netlink :

sudo apt install tsn-endstation-netlink

For example, on Debian distribution set with ECI repository, the following should be displayed:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
0 upgraded, 0 newly installed, 4 reinstalled, 0 to remove and 0 not upgraded.
Need to get 19.8 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://wheeljack.ch.intel.com/apt-repos/ECI/bookworm isar/main amd64 iotg-gptp-configs amd64 0.9.13-0eci4 [4848 B]
Get:2 http://wheeljack.ch.intel.com/apt-repos/ECI/bookworm isar/main amd64 tsn-endstation-init all 3.2-2 [3996 B]
Get:3 http://wheeljack.ch.intel.com/apt-repos/ECI/bookworm isar/main amd64 tsn-endstation-netlink all 3.2-2 [6368 B]
Get:4 http://wheeljack.ch.intel.com/apt-repos/ECI/bookworm isar/main amd64 tsn-endstation-timesync all 3.2-2 [4572 B]
Fetched 19.8 kB in 0s (1570 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 115190 files and directories currently installed.)
Preparing to unpack .../iotg-gptp-configs_0.9.13-0eci4_amd64.deb ...
Unpacking iotg-gptp-configs (0.9.13-0eci4) over (0.9.13-0eci4) ...
Preparing to unpack .../tsn-endstation-init_3.2-2_all.deb ...
Unpacking tsn-endstation-init (3.2-2) over (3.2-2) ...
Preparing to unpack .../tsn-endstation-netlink_3.2-2_all.deb ...
Unpacking tsn-endstation-netlink (3.2-2) over (3.2-2) ...
Preparing to unpack .../tsn-endstation-timesync_3.2-2_all.deb ...
Unpacking tsn-endstation-timesync (3.2-2) over (3.2-2) ...
Setting up iotg-gptp-configs (0.9.13-0eci4) ...
Setting up tsn-endstation-init (3.2-2) ...
Setting up tsn-endstation-timesync (3.2-2) ...
Setting up tsn-endstation-netlink (3.2-2) ...

Check the Intel ECI systemd-networkd packages upgrade with Earliest TxTime First (NET_SCHED_ETF) QDisc and Time Aware Priority (NET_SCHED_TAPRIO) QDisc features:

logo_debian logo_ubuntu

Install from meta-package

$ sudo apt-cache policy systemd

For example, on Debian distribution set with ECI repository, the following should be displayed:

systemd:
Installed: 252.22-1~deb12u1-bookworm+etf+taprio+mqprio+fpe+ffw-r5
Candidate: 252.22-1~deb12u1-bookworm+etf+taprio+mqprio+fpe+ffw-r5
Version table:
*** 252.22-1~deb12u1-bookworm+etf+taprio+mqprio+fpe+ffw-r5 100
      100 /var/lib/dpkg/status
   252.22-1~deb12u1-ecir5+etf+taprio+mqprio+fpe+ffw 500
      500 http://wheeljack.ch.intel.com/apt-repos/ECI/bookworm isar/main amd64 Packages
   252.22-1~deb12u1 500
      500 http://deb.debian.org/debian bookworm/main amd64 Packages

if Installed: is displayed differently you need to manually install :

$ sudo apt install systemd=252.22-1~deb12u1-bookworm+etf+taprio+mqprio+fpe+ffw-r5

systemd-networkd patchset for 252.22-1~deb12u1-bookworm+etf+taprio+mqprio+fpe+ffw-r5 are available from the APT repository deb-src :

$ sudo apt source systemd
$ ls -l systemd-252.22-1~deb12u1/debian/patches/00*.patch

Reading package lists... Done
Need to get 13.5 MB of source archives.
Get:1 http://wheeljack.ch.intel.com/apt-repos/ECI/bookworm isar/main systemd 252.22-1~deb12u1-ecir5+etf+taprio+mqprio+fpe+ffw (dsc) [5565 B]
Get:2 http://wheeljack.ch.intel.com/apt-repos/ECI/bookworm isar/main systemd 252.22-1~deb12u1-ecir5+etf+taprio+mqprio+fpe+ffw (tar) [13.5 MB]
10% [2 systemd 1202 B/13.5 MB 0%]
46% [2 systemd 6089 kB/13.5 MB 45%]
Fetched 13.5 MB in 1s (11.7 MB/s)
dpkg-source: info: extracting systemd in systemd-252.22-1~deb12u1
dpkg-source: info: unpacking systemd_252.22-1~deb12u1-ecir5+etf+taprio+mqprio+fpe+ffw.tar.gz

-rw-r--r-- 1 root root 23730 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0001-network-tc-introduce-ETF-Earliest-TxTime-First-Qdisc.patch
-rw-r--r-- 1 root root 63457 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0002-network-tc-introduce-Time-Aware-Priority-Shaper-TAPR.patch
-rw-r--r-- 1 root root 18986 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0003-network-tc-introduce-Multiqueue-Priority-Shaper-MQPR.patch
-rw-r--r-- 1 root root  3698 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0004-link-Add-tc-and-l2-fwd-offload-features.patch
-rw-r--r-- 1 root root  1021 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0005-vlan-remove-ingress-egress-qos-maps-exclusion.patch
-rw-r--r-- 1 root root  9256 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0006-network-taprio-support-for-preemption-parameter.patch
-rw-r--r-- 1 root root  7331 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0007-link-set-device-EEE-behaviour.patch
-rw-r--r-- 1 root root 60321 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0008-link-Add-ethtool-rx-nfc-flow-type-filters.patch
-rw-r--r-- 1 root root  8618 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0009-link-Add-ethtool-set-frame-preemption.patch
-rw-r--r-- 1 root root 86249 Jan 26 21:49 systemd-252.22-1~deb12u1/debian/patches/0010-network-add-support-for-configuring-TC-filters.patch

eci-ethirqs@.service- Networking Interrupts CPU Affinity for Intel® Ethernet Controllers

Intel® Ethernet Controllers can provide several MSI interrupts to improve the network throughput.

Intel® ECI makes generic assumptions that traffic-class prioritization is achieved on cores (for example, real-time communication tasks) other than default core 0 (for example, house-keeping tasks) than Network Interrupts Affinity to CPU critical Ethernet device interrupts

This section describes how to selectively map MSI interrupts to particular CPU cores for each particular Intel® Ethernet Controller using systemctl.

The following section is applicable to:

../../../_images/target_generic6.png

  1. Identify the available Intel® Industrial Ethernet Controllers:

    $ lspci -nnn | grep "Ethernet controller"

    In this example, the output pattern is ${device_id} ${description}, where ${device_id} is 01:00.0.

    Important

    Identify the Intel® Industrial Ethernet Controller interface that is wired to the TSN switch device or to another TSN end-station:

  2. Confirm the Linux network interface name of the Intel® Industrial Ethernet Controllers:

    $ networkctl status enp1s0

    The expected output should be similar to the following, with the default 51-wired-dhcp.network and 99-default.link networkd and udev rules setting:

    root@eci-intel-23f0:~# networkctl status enp1s0
? 5: enp1s0
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: /usr/lib/systemd/network/51-wired-dhcp.network
                        Type: ether
                        State: routable (configured)
                        Path: pci-0000:01:00.0
                        Driver: igc
                        Vendor: Intel Corporation
                        Model: Ethernet Controller I225-LM
                  HW Address: 00:a0:c9:00:00:00 (Intel Corporation)
                           MTU: 1500 (min: 68, max: 9216)
                        QDisc: mq
IPv6 Address Generation Mode: eui64
         Queue Length (Tx/Rx): 4/4
            Auto negotiation: yes
                        Speed: 1Gbps
                        Duplex: full
                        Port: tp
                     Address: 10.10.10.19 (DHCP4 via 10.10.10.1)
                              fe80::2a0:c9ff:fe00:0
                     Gateway: 10.10.10.1 (ALFA, INC.)
                           DNS: 10.10.10.1
               DHCP4 Client ID: IAID:0x56504d98/DUID
            DHCP6 Client DUID: DUID-EN/Vendor:0000ab11983c97c412262c820000
                  Connected To: cyclone5 on port 1c:a0:d3:20:06:1a (CE03)

eci-intel-23f0 systemd-networkd[328]: enp1s0: Gained carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv4 address 10.10.10.19/24 via 10.10.10.1
eci-intel-23f0 systemd-networkd[328]: enp1s0: Lost carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCP lease lost
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv6 lease lost
eci-intel-23f0 systemd-networkd[328]: enp1s0: Gained carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: Lost carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv6 lease lost
eci-intel-23f0 systemd-networkd[328]: enp1s0: Gained carrier
eci-intel-23f0 systemd-networkd[328]: enp1s0: DHCPv4 address 10.10.10.19/24 via 10.10.10.1

    The output text is the interface name of i225 NIC, marked as ${interface_name}.

  3. Enable, start, and report the Intel® Industrial Ethernet Controller MSI interrupts affinity CPU cores for the particular Linux network interface name:

    $ systemctl enable eci-ethirqs@enp1s0.service && systemctl start eci-ethirqs@enp1s0.service && systemctl status eci-ethirqs@enp1s0.service

    The expected output should be similar to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-ethirqs@enp1s0.service ? /lib/systemd/system/eci-ethirqs@.service.
● eci-ethirqs@enp1s0.service - ECI: set CPU core to IRQ Ethernet device affinity
   Loaded: loaded (/usr/lib/systemd/system/eci-ethirqs@.service; enabled; preset: disabled)
   Active: active (exited) since Thu 2024-04-04 10:15:47 EDT; 14ms ago
   Process: 911074 ExecStartPost=systemctl daemon-reload (code=exited, status=0/SUCCESS)
   Main PID: 910992 (code=exited, status=0/SUCCESS)
      CPU: 129ms

Apr 04 10:15:47 localhost.localdomain systemd[1]: Starting ECI: set CPU core to IRQ Ethernet device affinity...
Apr 04 10:15:47 localhost.localdomain bash[910963]: set isolcpu  to online cpu1
Apr 04 10:15:47 localhost.localdomain network-irq-affinity[910992]: network-irq-affinity: Assigning enp1s0-TxRx-3 on IRQ 134 to CPU 1
Apr 04 10:15:47 localhost.localdomain network-irq-affinity[910992]: network-irq-affinity: Assigning enp1s0-TxRx-2 on IRQ 133 to CPU 1
Apr 04 10:15:47 localhost.localdomain network-irq-affinity[910992]: network-irq-affinity: Assigning enp1s0-TxRx-1 on IRQ 131 to CPU 1
Apr 04 10:15:47 localhost.localdomain network-irq-affinity[910992]: network-irq-affinity: Assigning enp1s0-TxRx-0 on IRQ 130 to CPU 0
Apr 04 10:15:47 localhost.localdomain network-irq-affinity[910992]: network-irq-affinity: Assigning enp1s0 on IRQ 129 to CPU 0
Apr 04 10:15:47 localhost.localdomain systemd[1]: Finished ECI: set CPU core to IRQ Ethernet device affinity.

  4. Confirm whether the interrupt mapping is effective:

    $ cat /proc/interrupts | grep enp1s0

    The expected output should be similar to the following:

    root@eci-intel-3f6c:~# cat /proc/interrupts | grep enp3s0
            CPU0       CPU1       CPU2       CPU3
132:       2508          0          0          0  IR-PCI-MSI 524288-edge      enp1s0
133:     876072          0          0          0  IR-PCI-MSI 524289-edge      enp1s0-TxRx-0
134:     165157       4380          0          0  IR-PCI-MSI 524290-edge      enp1s0-TxRx-1
135:     165136        626          0          0  IR-PCI-MSI 524291-edge      enp1s0-TxRx-2
136:     164913        617          0          0  IR-PCI-MSI 524292-edge      enp1s0-TxRx-3

eci-ptp4l-p2p@.service - gPTP Time Synchronization Service for Intel® Ethernet Controllers

This section describes how to selectively enable IEEE 802.1as gPTP Profile Essential for each particular Intel® Ethernet Controller type using systemctl.

Generalized Precision Time Protocol (gPTP) is designed specifically for industrial control systems and is optimal for use in distributed systems because it requires minimal bandwidth and very little CPU processing overhead.

An IEEE 802.1AS capable TSN domain is made up of gPTP enabled Ethernet endpoints and switches.

The following animation illustrates the PTP clocks in a primary and secondary Ethernet port hierarchy within a TSN domain.

../../../_images/tsn-endstation-timesync_01.gif

The following section is applicable to:

../../../_images/target_generic6.png

  1. Set up Intel® Ethernet Controllers according to eci-ethirqs@.service- Networking Interrupts CPU Affinity for Intel® Ethernet Controllers.

  2. Enable, start, and report the Intel® Industrial Ethernet Controllers IEEE 802.1AS on the particular Linux network interface name:

    $ systemctl enable eci-ptp4l-p2p@enp1s0.service && systemctl start eci-ptp4l-p2p@enp1s0.service && systemctl status eci-ptp4l-p2p@enp1s0.service

    The expected output should be similar to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-ptp4l-p2p@enp1s0.service ? /lib/systemd/system/eci-ptp4l-p2p@.service.
● eci-ptp4l-p2p@enp1s0.service - ECI: Precision Time Protocol (PTP) service for enp1s0
   Loaded: loaded (/usr/lib/systemd/system/eci-ptp4l-p2p@.service; disabled; preset: disabled)
   Drop-In: /usr/lib/systemd/system/eci-ptp4l-p2p@.service.d
            └─10-isolcpu-rt.conf
   Active: active (running) since Thu 2024-04-04 10:15:47 EDT; 1min 7s ago
      Docs: man:ptp4l
   Main PID: 911115 (ptp4l)
      Tasks: 1 (limit: 195210)
   Memory: 372.0K
      CPU: 39ms
   CGroup: /system.slice/system-eci\x2dptp4l\x2dp2p.slice/eci-ptp4l-p2p@enp1s0.service
            └─911115 ptp4l -i enp1s0 -f /run/eci/ptp4l-enp1s0.conf -ml 6 --step_threshold=1 --socket_priority=2

Apr 04 10:16:50 localhost.localdomain ptp4l[911115]: ptp4l[5100134.055]: rms    2 max    3 freq  +1281 +/-   3 delay     8 +/-   0
Apr 04 10:16:50 localhost.localdomain ptp4l[911115]: [5100134.055] rms    2 max    3 freq  +1281 +/-   3 delay     8 +/-   0
Apr 04 10:16:51 localhost.localdomain ptp4l[911115]: ptp4l[5100135.055]: rms    2 max    5 freq  +1283 +/-   3 delay     8 +/-   0
Apr 04 10:16:51 localhost.localdomain ptp4l[911115]: [5100135.055] rms    2 max    5 freq  +1283 +/-   3 delay     8 +/-   0
Apr 04 10:16:52 localhost.localdomain ptp4l[911115]: ptp4l[5100136.055]: rms    2 max    5 freq  +1282 +/-   3 delay     8 +/-   0
Apr 04 10:16:52 localhost.localdomain ptp4l[911115]: [5100136.055] rms    2 max    5 freq  +1282 +/-   3 delay     8 +/-   0
Apr 04 10:16:53 localhost.localdomain ptp4l[911115]: ptp4l[5100137.055]: rms    2 max    5 freq  +1281 +/-   3 delay     8 +/-   0
Apr 04 10:16:53 localhost.localdomain ptp4l[911115]: [5100137.055] rms    2 max    5 freq  +1281 +/-   3 delay     8 +/-   0
Apr 04 10:16:54 localhost.localdomain ptp4l[911115]: ptp4l[5100138.055]: rms    2 max    3 freq  +1282 +/-   3 delay     9 +/-   0
Apr 04 10:16:54 localhost.localdomain ptp4l[911115]: [5100138.055] rms    2 max    3 freq  +1282 +/-   3 delay     9 +/-   0

    Attention

    You might need to edit Intel® Industrial Ethernet controllers IEEE 802.1AS to match a specific ODM Ethernet PHY configuration available on the the iotg-gptp-configs ECI package:

    $ ls -l /opt/intel/iotg_tsn_ref_sw/common/

    Select a gPTP template from the list:

    -rw-r--r-- 1 root root 526 Apr 21  2021 gPTP.cfg
-rw-r--r-- 1 root root 540 Apr 21  2021 gPTP_RGMII-MV1510-1G.cfg
-rw-r--r-- 1 root root 540 Apr 21  2021 gPTP_SGMII-MV2110-1G.cfg
-rw-r--r-- 1 root root 543 Apr 21  2021 gPTP_SGMII-MV2110-2_5G.cfg
-rw-r--r-- 1 root root 541 Apr 21  2021 gPTP_TI-1G.cfg
-rw-r--r-- 1 root root 552 Apr 21  2021 gPTP_i225-1G.cfg

    Edit gPTP settings required for certain Ethernet PHY and speed:

    $ vi /run/eci/ptp4l-enp1s0.conf
#
# 802.1AS example configuration containing those attributes which
# differ from the defaults.  See the file, default.cfg, for the
# complete list of available options.
#
[global]
gmCapable               1
priority1               248
priority2               248
logAnnounceInterval     0
logSyncInterval         -3
syncReceiptTimeout      3
neighborPropDelayThresh 800
min_neighbor_prop_delay -20000000
assume_two_step         1
path_trace_enabled      1
follow_up_info          1
transportSpecific       0x1
ptp_dst_mac             01:80:C2:00:00:0E
network_transport       L2
delay_mechanism         P2P
tx_timestamp_timeout    100

    Then, restart to test and review gPTP settings:

    $ systemctl restart eci-ptp4l-p2p@enp1s0.service && journalctl -xe -u eci-ptp4l-p2p@enp1s0.service

eci-phc2sys@.service - PHC Time Synchronization Service for Intel® Ethernet Controllers

This section describes how to selectively enable IEEE 802.1as gPTP Profile Essential with CPU time on a particular Intel® Ethernet Controller type using systemctl.

The following section is applicable to:

../../../_images/target_generic6.png

  1. Set up Intel® Ethernet controllers according to eci-ptp4l-p2p@.service - gPTP Time Synchronization Service for Intel® Ethernet Controllers.

  2. Enable, start, and report the Intel® Industrial Ethernet Controllers’ PHC time sync on the particular Linux network interface name:

    $ systemctl enable eci-phc2sys@enp1s0.service && systemctl start eci-phc2sys@enp1s0.service && systemctl status eci-phc2sys@enp1s0.service

    The expected output should be similar to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-phc2sys@enp1s0.service → /lib/systemd/system/eci-phc2sys@enp1s0.service.
● eci-phc2sys@enp1s0.service - ECI: Synchronize system clock or PTP hardware clock (PHC)
   Loaded: loaded (/usr/lib/systemd/system/eci-phc2sys@.service; disabled; preset: disabled)
   Drop-In: /usr/lib/systemd/system/eci-phc2sys@.service.d
            └─10-isolcpu-rt.conf
   Active: active (running) since Thu 2024-04-04 10:15:47 EDT; 2min 34s ago
      Docs: man:phc2sys
   Main PID: 911116 (phc2sys)
      Tasks: 1 (limit: 195210)
   Memory: 516.0K
      CPU: 7ms
   CGroup: /system.slice/system-eci\x2dphc2sys.slice/eci-phc2sys@enp1s0.service
            └─911116 phc2sys -s enp1s0 -c CLOCK_REALTIME --step_threshold=1 --transportSpecific=1 -w -ml 7

Apr 04 10:18:17 localhost.localdomain phc2sys[911116]: phc2sys[5100221.064]: CLOCK_REALTIME phc offset        -2 s2 freq   -3843 delay      0
Apr 04 10:18:17 localhost.localdomain phc2sys[911116]: [5100221.064] CLOCK_REALTIME phc offset        -2 s2 freq   -3843 delay      0
Apr 04 10:18:18 localhost.localdomain phc2sys[911116]: phc2sys[5100222.064]: CLOCK_REALTIME phc offset        -7 s2 freq   -3848 delay      0
Apr 04 10:18:18 localhost.localdomain phc2sys[911116]: [5100222.064] CLOCK_REALTIME phc offset        -7 s2 freq   -3848 delay      0
Apr 04 10:18:19 localhost.localdomain phc2sys[911116]: phc2sys[5100223.064]: CLOCK_REALTIME phc offset        -7 s2 freq   -3850 delay      0
Apr 04 10:18:19 localhost.localdomain phc2sys[911116]: [5100223.064] CLOCK_REALTIME phc offset        -7 s2 freq   -3850 delay      0
Apr 04 10:18:20 localhost.localdomain phc2sys[911116]: phc2sys[5100224.064]: CLOCK_REALTIME phc offset       -24 s2 freq   -3869 delay      0
Apr 04 10:18:20 localhost.localdomain phc2sys[911116]: [5100224.064] CLOCK_REALTIME phc offset       -24 s2 freq   -3869 delay      0
Apr 04 10:18:21 localhost.localdomain phc2sys[911116]: phc2sys[5100225.064]: CLOCK_REALTIME phc offset        -9 s2 freq   -3862 delay      0
Apr 04 10:18:21 localhost.localdomain phc2sys[911116]: [5100225.064] CLOCK_REALTIME phc offset        -9 s2 freq   -3862 delay      0

eci-tsn-wait-sync-8021as@.service - TSN 802.1AS Services for Intel® Ethernet Controllers

This section describes how to selectively wait for IEEE 802.1as gPTP Profile Essential time synchronization on a particular Intel® Ethernet Controller using systemctl.

The following section is applicable to:

../../../_images/target_generic6.png

  1. Set up Intel® Ethernet Controllers according to eci-phc2sys@.service - PHC Time Synchronization Service for Intel® Ethernet Controllers.

  2. Enable, start, and wait for SUCCESS status of check_clocks for Intel® Industrial Ethernet Controllers on a particular Linux network interface name:

    $ systemctl enable eci-tsn-wait-sync-8021as@enp1s0.service && systemctl start eci-tsn-wait-sync-8021as@enp1s0.service && systemctl status eci-tsn-wait-sync-8021as@enp1s0.service

    The expected output should be similar after < 15 seconds to the following:

    Created symlink /etc/systemd/system/multi-user.target.wants/eci-tsn-wait-sync-8021as@enp1s0.service ? /lib/systemd/system/eci-tsn-wait-sync-8021as@.service.
● eci-tsn-wait-sync-8021as@enp1s0.service - ECI: Wait for clocks synchronization
   Loaded: loaded (/usr/lib/systemd/system/eci-tsn-wait-sync-8021as@.service; enabled; preset: disabled)
   Drop-In: /usr/lib/systemd/system/eci-tsn-wait-sync-8021as@.service.d
            └─10-isolcpu-rt.conf
   Active: active (exited) since Thu 2024-04-04 10:15:52 EDT; 3min 29s ago
   Main PID: 911119 (code=exited, status=0/SUCCESS)
      CPU: 11ms

Apr 04 10:15:52 localhost.localdomain bash[911130]: rt tstamp:        1712240151894184064
Apr 04 10:15:52 localhost.localdomain bash[911130]: tai tstamp:        1712240188894184125
Apr 04 10:15:52 localhost.localdomain bash[911130]: phc tstamp:        1712240188894187060
Apr 04 10:15:52 localhost.localdomain bash[911130]: rt latency:        35
Apr 04 10:15:52 localhost.localdomain bash[911130]: tai latency:        43
Apr 04 10:15:52 localhost.localdomain bash[911130]: phc latency:        3404
Apr 04 10:15:52 localhost.localdomain bash[911130]: phc-rt delta:        37000002996
Apr 04 10:15:52 localhost.localdomain bash[911130]: phc-tai delta:        2935
Apr 04 10:15:52 localhost.localdomain bash[911130]: Clocks on this system are synchronized :)
Apr 04 10:15:52 localhost.localdomain systemd[1]: Finished ECI: Wait for clocks synchronization.

  3. Optionally, enable, start, and monitor check_clocks for Intel® Industrial Ethernet Controllers on a particular Linux network interface name:

    Important

    gPTP monitoring and tracing is only available on Intel® Ethernet Controller supporting PPS_CAPTUREASSERT callback pps_source_info API:

    as well as Intel® Ethernet Controller supporting PTP_SYS_OFFSET_PRECISE callback ptp_clock_info API (non-exhaustive list) :

    $ systemctl enable eci-tsn-track-sync-8021as@enp1s0.service && systemctl start eci-tsn-track-sync-8021as@enp1s0.service && systemctl status eci-tsn-track-sync-8021as@enp1s0.service

    The expected output should be similar to the following:

    ? eci-tsn-track-sync-8021as@enp1s0.service - ECI: Check that time is synchronized
   Loaded: loaded (/lib/systemd/system/eci-tsn-track-sync-8021as@.service; enabled; vendor preset: enabled)
   Drop-In: /usr/lib/systemd/system/service.d
            └─10-override-protect-proc.conf
   Active: active (running) since Tue 2018-08-28 09:01:42 UTC; 22ms ago
   Main PID: 24159 (check_clocks)
      Tasks: 1 (limit: 9164)
   Memory: 284.0K
      CPU: 1ms
   CGroup: /system.slice/system-eci\x2dtsn\x2dtrack\x2dsync\x2d8021as.slice/eci-tsn-track-sync-8021as@enp1s0.service
            └─24159 check_clocks -d enp1s0 -va

eci-intel-23f0 systemd[1]: Starting ECI: Check that time is synchronized...
eci-intel-23f0 systemd[1]: Started ECI: Check that time is synchronized.
eci-intel-23f0 check_clocks[24159]: Dumping timestamps and deltas
eci-intel-23f0 check_clocks[24159]: /dev/pps0 can be enabled from /dev/ptp0 on device enp1s0
eci-intel-23f0 check_clocks[24159]: trying PPS source "/dev/pps0"
eci-intel-23f0 check_clocks[24159]: found PPS source "/dev/pps0"
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446903         sequence: 1928         offset: 3569         trt_lat=8184         tai_lat=37000008267
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446904         sequence: 1929         offset: 3688         trt_lat=4427         tai_lat=37000004486
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446905         sequence: 1930         offset: 4284         trt_lat=4721         tai_lat=37000004780
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446906         sequence: 1931         offset: 3651         trt_lat=4371         tai_lat=37000004415
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446907         sequence: 1932         offset: 4308         trt_lat=4446         tai_lat=37000004471
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446908         sequence: 1933         offset: 3663         trt_lat=4466         tai_lat=37000004493
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446909         sequence: 1934         offset: 3644         trt_lat=4313         tai_lat=37000004346
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446910         sequence: 1935         offset: 4233         trt_lat=4505         tai_lat=37000004555
eci-intel-23f0 check_clocks[24159]: pps-phc-timestamp: 1535446911         sequence: 1936         offset: 3662         trt_lat=4363         tai_lat=37000004397

    For more logs:

    $ journalctl -xe -u eci-tsn-track-sync-8021as@enp1s0.service | more

  4. Report the check_clocks statistics to ftrace pulse-per-second sequence id-1075 to id-1084 interval for Intel® Industrial Ethernet controllers on the particular Linux network interface name:

    less /sys/kernel/tracing/per_cpu/cpu1/trace

    # tracer: nop
#
# entries-in-buffer/entries-written: 7/7   #P:4
#
#                                _-------=> irqs-off
#                               / _------=> need-resched
#                              | / _-----=> need-resched-lazy
#                              || / _----=> hardirq/softirq
#                              ||| / _---=> preempt-depth
#                              |||| / _--=> preempt-lazy-depth
#                              ||||| / _-=> migrate-disable
#                              |||||| /     delay
#           TASK-PID     CPU#  |||||||  TIMESTAMP  FUNCTION
#              | |         |   |||||||      |         |
         <...>-24159   [001] ....... 331549.637493: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447154.3793 seq=2179 tai_lat=37000004746 ep=enp1s0
         <...>-24159   [001] ....... 331550.637510: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447155.3693 seq=2180 tai_lat=37000004506 ep=enp1s0
         <...>-24159   [001] ....... 331568.637825: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447173.3772 seq=2198 tai_lat=37000004657 ep=enp1s0
         <...>-24159   [001] ....... 331569.637842: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447174.3710 seq=2199 tai_lat=37000004450 ep=enp1s0
         <...>-24159   [001] ....... 331570.637860: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447175.4131 seq=2200 tai_lat=37000004474 ep=enp1s0
         <...>-24159   [001] ....... 331571.637877: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447176.3723 seq=2201 tai_lat=37000004374 ep=enp1s0
         <...>-24159   [001] ....... 331572.637895: tracing_mark_write: 0: trace_event_clock_sync: parent_ts=1535447177.4119 seq=2202 tai_lat=37000004408 ep=enp1s0

  5. Visualize IEEE 802.1Q packet tracing in Timeview-UI for Intel® Industrial Ethernet controllers on the particular Linux network interface name:

    ../../../_images/ptpv2_capture_01.png

Note:

To override International Atomic Time (TAI) and UTC currentUtcOffset default 37-second value:

$ pmc -u -b 0 -t 1 \
      "SET GRANDMASTER_SETTINGS_NP clockClass 248 clockAccuracy 0xfe offsetScaledLogVariance 0xffff currentUtcOffset 37 leap61 0 leap59 0 currentUtcOffsetValid 1 ptpTimescale 1 timeTraceable 1 frequencyTraceable 0 timeSource 0xa0"

The expected output should be similar to the following:

sending: SET GRANDMASTER_SETTINGS_NP
   2046a1.fffe.056bdc-0 seq 0 RESPONSE MANAGEMENT GRANDMASTER_SETTINGS_NP
            clockClass              248
            clockAccuracy           0xfe
            offsetScaledLogVariance 0xffff
            currentUtcOffset        0
            leap61                  0
            leap59                  0
            currentUtcOffsetValid   1
            ptpTimescale            1
            timeTraceable           1
            frequencyTraceable      0
            timeSource              0xa0