This series tries to pick up the work on the virtio-net timestamping
feature from Willem de Bruijn.
Original series
Message-Id: [email protected]
Subject: [PATCH RFC v2 0/4] virtio-net: add tx-hash, rx-tstamp,
tx-tstamp and tx-time
From: Willem de Bruijn <[email protected]>
RFC for four new features to the virtio network device:
1. pass tx flow state to host, for routing + telemetry
2. pass rx tstamp to guest, for better RTT estimation
3. pass tx tstamp to guest, idem
3. pass tx delivery time to host, for accurate pacing
All would introduce an extension to the virtio spec.
The original series consisted of a hack around the DMA API, which should
be fixed in this series.
The changes in this series are to the driver side. For the changes to qemu see:
https://github.com/strumtrar/qemu/tree/v8.1.1/virtio-net-ptp
Currently only virtio-net is supported. The original series used
vhost-net as backend. However, the path through tun via sendmsg doesn't
allow us to write data back to the driver side without any hacks.
Therefore use the way via plain virtio-net without vhost albeit better
performance.
Signed-off-by: Steffen Trumtrar <[email protected]>
---
Willem de Bruijn (4):
virtio-net: support transmit hash report
virtio-net: support receive timestamp
virtio-net: support transmit timestamp
virtio-net: support future packet transmit time
drivers/net/virtio_net.c | 211 ++++++++++++++++++++++++++++++++++++++--
include/uapi/linux/virtio_net.h | 24 ++++-
2 files changed, 228 insertions(+), 7 deletions(-)
---
base-commit: b85ea95d086471afb4ad062012a4d73cd328fa86
change-id: 20231218-v6-7-topic-virtio-net-ptp-3df023bc4f4d
Best regards,
--
Steffen Trumtrar <[email protected]>
From: Willem de Bruijn <[email protected]>
Add optional transmit time (SO_TXTIME) offload for virtio-net.
The Linux TCP/IP stack tries to avoid bursty transmission and network
congestion through pacing: computing an skb delivery time based on
congestion information. Userspace protocol implementations can achieve
the same with SO_TXTIME. This may also reduce scheduling jitter and
improve RTT estimation.
Pacing can be implemented in ETF or FQ qdiscs or offloaded to NIC
hardware. Allow virtio-net driver to offload for the same reasons.
The timestamp straddles (virtual) hardware domains. Like PTP, use
international atomic time (CLOCK_TAI) as global clock base. The driver
must sync with the device, e.g., through kvm-clock.
Changes RFC - RFCv2
- rename from transmit timestamp to future packet transmit time
- convert cpu_to_virtioXX to cpu_to_leXX
Signed-off-by: Willem de Bruijn <[email protected]>
Signed-off-by: Steffen Trumtrar <[email protected]>
---
drivers/net/virtio_net.c | 13 ++++++++++++-
include/uapi/linux/virtio_net.h | 1 +
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 4065834957fbd..fb3572074e8b8 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -286,6 +286,9 @@ struct virtnet_info {
/* Device will pass tx timestamp. Requires has_tx_tstamp */
bool enable_tx_tstamp;
+ /* Driver will pass CLOCK_TAI delivery time to the device */
+ bool has_tx_time;
+
/* Has control virtqueue */
bool has_cvq;
@@ -2371,6 +2374,8 @@ static int xmit_skb(struct send_queue *sq, struct sk_buff *skb)
}
if (vi->enable_tx_tstamp && skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)
ht->hash.hdr.flags |= VIRTIO_NET_HDR_F_TSTAMP;
+ if (vi->has_tx_time && skb->tstamp)
+ ht->tstamp = cpu_to_le64(skb->tstamp);
sg_init_table(sq->sg, skb_shinfo(skb)->nr_frags + (can_push ? 1 : 2));
if (can_push) {
@@ -4698,6 +4703,11 @@ static int virtnet_probe(struct virtio_device *vdev)
vi->hdr_len = sizeof(struct virtio_net_hdr_hash_ts);
}
+ if (virtio_has_feature(vdev, VIRTIO_NET_F_TX_TIME)) {
+ vi->has_tx_time = true;
+ vi->hdr_len = sizeof(struct virtio_net_hdr_hash_ts);
+ }
+
if (virtio_has_feature(vdev, VIRTIO_F_ANY_LAYOUT) ||
virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
vi->any_header_sg = true;
@@ -4941,7 +4951,8 @@ static struct virtio_device_id id_table[] = {
VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT, VIRTIO_NET_F_NOTF_COAL, \
VIRTIO_NET_F_VQ_NOTF_COAL, \
VIRTIO_NET_F_GUEST_HDRLEN, \
- VIRTIO_NET_F_TX_HASH, VIRTIO_NET_F_RX_TSTAMP, VIRTIO_NET_F_TX_TSTAMP
+ VIRTIO_NET_F_TX_HASH, VIRTIO_NET_F_RX_TSTAMP, VIRTIO_NET_F_TX_TSTAMP, \
+ VIRTIO_NET_F_TX_TIME
static unsigned int features[] = {
VIRTNET_FEATURES,
diff --git a/include/uapi/linux/virtio_net.h b/include/uapi/linux/virtio_net.h
index 03a7ef45372d7..18db1e1130d92 100644
--- a/include/uapi/linux/virtio_net.h
+++ b/include/uapi/linux/virtio_net.h
@@ -56,6 +56,7 @@
#define VIRTIO_NET_F_MQ 22 /* Device supports Receive Flow
* Steering */
#define VIRTIO_NET_F_CTRL_MAC_ADDR 23 /* Set MAC address */
+#define VIRTIO_NET_F_TX_TIME 48 /* Driver sets TAI delivery time */
#define VIRTIO_NET_F_TX_TSTAMP 49 /* Device sends TAI transmit time */
#define VIRTIO_NET_F_RX_TSTAMP 50 /* Device sends TAI receive time */
#define VIRTIO_NET_F_TX_HASH 51 /* Driver sends hash report */
--
2.42.0
From: Willem de Bruijn <[email protected]>
Add optional PTP hardware tx timestamp offload for virtio-net.
Accurate RTT measurement requires timestamps close to the wire.
Introduce virtio feature VIRTIO_NET_F_TX_TSTAMP, the transmit
equivalent to VIRTIO_NET_F_RX_TSTAMP.
The driver sets VIRTIO_NET_HDR_F_TSTAMP to request a timestamp
returned on completion. If the feature is negotiated, the device
either places the timestamp or clears the feature bit.
The timestamp straddles (virtual) hardware domains. Like PTP, use
international atomic time (CLOCK_TAI) as global clock base. The driver
must sync with the device, e.g., through kvm-clock.
Modify can_push to ensure that on tx completion the header, and thus
timestamp, is in a predicatable location at skb_vnet_hdr.
Tested: modified txtimestamp.c to with h/w timestamping:
- sock_opt = SOF_TIMESTAMPING_SOFTWARE |
+ sock_opt = SOF_TIMESTAMPING_RAW_HARDWARE |
+ do_test(family, SOF_TIMESTAMPING_TX_HARDWARE);
Signed-off-by: Willem de Bruijn <[email protected]>
Co-developed-by: Steffen Trumtrar <[email protected]>
Signed-off-by: Steffen Trumtrar <[email protected]>
--
Changes to original RFC v2:
- append write-able descriptor for TX timestamp
Signed-off-by: Steffen Trumtrar <[email protected]>
---
drivers/net/virtio_net.c | 79 +++++++++++++++++++++++++++++++++++------
include/uapi/linux/virtio_net.h | 1 +
2 files changed, 70 insertions(+), 10 deletions(-)
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 9bb6cdaf49cc9..4065834957fbd 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -280,6 +280,12 @@ struct virtnet_info {
/* Device will pass rx timestamp. Requires has_rx_tstamp */
bool enable_rx_tstamp;
+ /* Device can pass CLOCK_TAI transmit time to the driver */
+ bool has_tx_tstamp;
+
+ /* Device will pass tx timestamp. Requires has_tx_tstamp */
+ bool enable_tx_tstamp;
+
/* Has control virtqueue */
bool has_cvq;
@@ -760,6 +766,21 @@ static void virtnet_rq_set_premapped(struct virtnet_info *vi)
}
}
+static void virtnet_record_tx_tstamp(const struct send_queue *sq,
+ struct sk_buff *skb)
+{
+ const struct virtio_net_hdr_hash_ts *h = skb_vnet_hdr_ht(skb);
+ const struct virtnet_info *vi = sq->vq->vdev->priv;
+ struct skb_shared_hwtstamps ts;
+
+ if (h->hash.hdr.flags & VIRTIO_NET_HDR_F_TSTAMP &&
+ vi->enable_tx_tstamp) {
+ /* Write the timestamp from the userspace to the skb */
+ ts.hwtstamp = ns_to_ktime(le64_to_cpu(h->tstamp));
+ skb_tstamp_tx(skb, &ts);
+ }
+}
+
static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi)
{
unsigned int len;
@@ -771,6 +792,7 @@ static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi)
if (likely(!is_xdp_frame(ptr))) {
struct sk_buff *skb = ptr;
+ virtnet_record_tx_tstamp(sq, skb);
pr_debug("Sent skb %p\n", skb);
bytes += skb->len;
@@ -2312,7 +2334,7 @@ static int xmit_skb(struct send_queue *sq, struct sk_buff *skb)
struct virtio_net_hdr_mrg_rxbuf *hdr;
const unsigned char *dest = ((struct ethhdr *)skb->data)->h_dest;
struct virtnet_info *vi = sq->vq->vdev->priv;
- struct virtio_net_hdr_v1_hash *ht;
+ struct virtio_net_hdr_hash_ts *ht;
int num_sg;
unsigned hdr_len = vi->hdr_len;
bool can_push;
@@ -2321,7 +2343,8 @@ static int xmit_skb(struct send_queue *sq, struct sk_buff *skb)
can_push = vi->any_header_sg &&
!((unsigned long)skb->data & (__alignof__(*hdr) - 1)) &&
- !skb_header_cloned(skb) && skb_headroom(skb) >= hdr_len;
+ !skb_header_cloned(skb) && skb_headroom(skb) >= hdr_len &&
+ !vi->enable_tx_tstamp;
/* Even if we can, don't push here yet as this would skew
* csum_start offset below. */
if (can_push)
@@ -2342,10 +2365,12 @@ static int xmit_skb(struct send_queue *sq, struct sk_buff *skb)
u16 report = skb->l4_hash ? VIRTIO_NET_HASH_REPORT_L4 :
VIRTIO_NET_HASH_REPORT_OTHER;
- ht->hash_value = cpu_to_le32(skb->hash);
- ht->hash_report = cpu_to_le16(report);
- ht->hash_state = cpu_to_le16(VIRTIO_NET_HASH_STATE_DEFAULT);
+ ht->hash.value = cpu_to_le32(skb->hash);
+ ht->hash.report = cpu_to_le16(report);
+ ht->hash.flow_state = cpu_to_le16(VIRTIO_NET_HASH_STATE_DEFAULT);
}
+ if (vi->enable_tx_tstamp && skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)
+ ht->hash.hdr.flags |= VIRTIO_NET_HDR_F_TSTAMP;
sg_init_table(sq->sg, skb_shinfo(skb)->nr_frags + (can_push ? 1 : 2));
if (can_push) {
@@ -2362,7 +2387,23 @@ static int xmit_skb(struct send_queue *sq, struct sk_buff *skb)
return num_sg;
num_sg++;
}
- return virtqueue_add_outbuf(sq->vq, sq->sg, num_sg, skb, GFP_ATOMIC);
+
+ /* Append a writeable descriptor after the RO ones. TX timestamp is
+ * received via this descriptor
+ */
+ if (ht->hash.hdr.flags & VIRTIO_NET_HDR_F_TSTAMP) {
+ struct virtio_net_hdr_hash_ts *h = skb_vnet_hdr_ht(skb);
+ struct scatterlist *sgs[2];
+ struct scatterlist sg_in;
+
+ sg_init_one(&sg_in, &h->tstamp, sizeof(h->tstamp));
+ sgs[0] = sq->sg;
+ sgs[1] = &sg_in;
+
+ return virtqueue_add_sgs(sq->vq, sgs, 1, 1, skb, GFP_ATOMIC);
+ } else {
+ return virtqueue_add_outbuf(sq->vq, sq->sg, num_sg, skb, GFP_ATOMIC);
+ }
}
static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
@@ -3556,7 +3597,13 @@ static int virtnet_get_ts_info(struct net_device *dev,
info->rx_filters = HWTSTAMP_FILTER_NONE;
}
- info->tx_types = HWTSTAMP_TX_OFF;
+ if (vi->has_tx_tstamp) {
+ info->so_timestamping |= SOF_TIMESTAMPING_TX_HARDWARE |
+ SOF_TIMESTAMPING_RAW_HARDWARE;
+ info->tx_types = HWTSTAMP_TX_ON;
+ } else {
+ info->tx_types = HWTSTAMP_TX_OFF;
+ }
return 0;
}
@@ -3984,7 +4031,8 @@ static int virtnet_ioctl_set_hwtstamp(struct net_device *dev, struct ifreq *ifr)
return -EFAULT;
if (tsconf.flags)
return -EINVAL;
- if (tsconf.tx_type != HWTSTAMP_TX_OFF)
+ if (tsconf.tx_type != HWTSTAMP_TX_OFF &&
+ tsconf.tx_type != HWTSTAMP_TX_ON)
return -ERANGE;
if (tsconf.rx_filter != HWTSTAMP_FILTER_NONE &&
tsconf.rx_filter != HWTSTAMP_FILTER_ALL)
@@ -3995,6 +4043,11 @@ static int virtnet_ioctl_set_hwtstamp(struct net_device *dev, struct ifreq *ifr)
else
vi->enable_rx_tstamp = tsconf.rx_filter == HWTSTAMP_FILTER_ALL;
+ if (!vi->has_tx_tstamp)
+ tsconf.tx_type = HWTSTAMP_TX_OFF;
+ else
+ vi->enable_tx_tstamp = tsconf.tx_type == HWTSTAMP_TX_ON;
+
if (copy_to_user(ifr->ifr_data, &tsconf, sizeof(tsconf)))
return -EFAULT;
@@ -4009,7 +4062,8 @@ static int virtnet_ioctl_get_hwtstamp(struct net_device *dev, struct ifreq *ifr)
tsconf.flags = 0;
tsconf.rx_filter = vi->enable_rx_tstamp ? HWTSTAMP_FILTER_ALL :
HWTSTAMP_FILTER_NONE;
- tsconf.tx_type = HWTSTAMP_TX_OFF;
+ tsconf.tx_type = vi->enable_tx_tstamp ? HWTSTAMP_TX_ON :
+ HWTSTAMP_TX_OFF;
if (copy_to_user(ifr->ifr_data, &tsconf, sizeof(tsconf)))
return -EFAULT;
@@ -4639,6 +4693,11 @@ static int virtnet_probe(struct virtio_device *vdev)
vi->hdr_len = sizeof(struct virtio_net_hdr_hash_ts);
}
+ if (virtio_has_feature(vdev, VIRTIO_NET_F_TX_TSTAMP)) {
+ vi->has_tx_tstamp = true;
+ vi->hdr_len = sizeof(struct virtio_net_hdr_hash_ts);
+ }
+
if (virtio_has_feature(vdev, VIRTIO_F_ANY_LAYOUT) ||
virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
vi->any_header_sg = true;
@@ -4882,7 +4941,7 @@ static struct virtio_device_id id_table[] = {
VIRTIO_NET_F_RSS, VIRTIO_NET_F_HASH_REPORT, VIRTIO_NET_F_NOTF_COAL, \
VIRTIO_NET_F_VQ_NOTF_COAL, \
VIRTIO_NET_F_GUEST_HDRLEN, \
- VIRTIO_NET_F_TX_HASH, VIRTIO_NET_F_RX_TSTAMP
+ VIRTIO_NET_F_TX_HASH, VIRTIO_NET_F_RX_TSTAMP, VIRTIO_NET_F_TX_TSTAMP
static unsigned int features[] = {
VIRTNET_FEATURES,
diff --git a/include/uapi/linux/virtio_net.h b/include/uapi/linux/virtio_net.h
index 6a0a6f299a13b..03a7ef45372d7 100644
--- a/include/uapi/linux/virtio_net.h
+++ b/include/uapi/linux/virtio_net.h
@@ -56,6 +56,7 @@
#define VIRTIO_NET_F_MQ 22 /* Device supports Receive Flow
* Steering */
#define VIRTIO_NET_F_CTRL_MAC_ADDR 23 /* Set MAC address */
+#define VIRTIO_NET_F_TX_TSTAMP 49 /* Device sends TAI transmit time */
#define VIRTIO_NET_F_RX_TSTAMP 50 /* Device sends TAI receive time */
#define VIRTIO_NET_F_TX_HASH 51 /* Driver sends hash report */
#define VIRTIO_NET_F_VQ_NOTF_COAL 52 /* Device supports virtqueue notification coalescing */
--
2.42.0
Steffen Trumtrar wrote:
> This series tries to pick up the work on the virtio-net timestamping
> feature from Willem de Bruijn.
>
> Original series
> Message-Id: [email protected]
> Subject: [PATCH RFC v2 0/4] virtio-net: add tx-hash, rx-tstamp,
> tx-tstamp and tx-time
> From: Willem de Bruijn <[email protected]>
>
> RFC for four new features to the virtio network device:
>
> 1. pass tx flow state to host, for routing + telemetry
> 2. pass rx tstamp to guest, for better RTT estimation
> 3. pass tx tstamp to guest, idem
> 3. pass tx delivery time to host, for accurate pacing
>
> All would introduce an extension to the virtio spec.
>
> The original series consisted of a hack around the DMA API, which should
> be fixed in this series.
>
> The changes in this series are to the driver side. For the changes to qemu see:
> https://github.com/strumtrar/qemu/tree/v8.1.1/virtio-net-ptp
>
> Currently only virtio-net is supported. The original series used
> vhost-net as backend. However, the path through tun via sendmsg doesn't
> allow us to write data back to the driver side without any hacks.
> Therefore use the way via plain virtio-net without vhost albeit better
> performance.
>
> Signed-off-by: Steffen Trumtrar <[email protected]>
Thanks for picking this back up, Steffen. Nice to see that the code still
applies mostly cleanly.
For context: I dropped the work only because I had no real device
implementation. The referenced patch series to qemu changes that.
I suppose the main issue is the virtio API changes that this introduces,
which will have to be accepted to the spec.
One small comment to patch 4: there I just assumed the virtual device
time is CLOCK_TAI. There is a concurrent feature under review for HW
pacing offload with AF_XDP sockets. The clock issue comes up a bit. In
general, for hardware we cannot assume a clock. For virtio, perhaps
assuming the same monotonic hardware clock in guest and host can be
assumed. But this clock alignment needs some thought.
On Tue, Dec 19, 2023 at 12:36 AM Willem de Bruijn
<[email protected]> wrote:
>
> Steffen Trumtrar wrote:
> > This series tries to pick up the work on the virtio-net timestamping
> > feature from Willem de Bruijn.
> >
> > Original series
> > Message-Id: [email protected]
> > Subject: [PATCH RFC v2 0/4] virtio-net: add tx-hash, rx-tstamp,
> > tx-tstamp and tx-time
> > From: Willem de Bruijn <[email protected]>
> >
> > RFC for four new features to the virtio network device:
> >
> > 1. pass tx flow state to host, for routing + telemetry
> > 2. pass rx tstamp to guest, for better RTT estimation
> > 3. pass tx tstamp to guest, idem
> > 3. pass tx delivery time to host, for accurate pacing
> >
> > All would introduce an extension to the virtio spec.
> >
> > The original series consisted of a hack around the DMA API, which should
> > be fixed in this series.
> >
> > The changes in this series are to the driver side. For the changes to qemu see:
> > https://github.com/strumtrar/qemu/tree/v8.1.1/virtio-net-ptp
> >
> > Currently only virtio-net is supported. The original series used
> > vhost-net as backend. However, the path through tun via sendmsg doesn't
> > allow us to write data back to the driver side without any hacks.
> > Therefore use the way via plain virtio-net without vhost albeit better
> > performance.
> >
> > Signed-off-by: Steffen Trumtrar <[email protected]>
>
> Thanks for picking this back up, Steffen. Nice to see that the code still
> applies mostly cleanly.
>
> For context: I dropped the work only because I had no real device
> implementation. The referenced patch series to qemu changes that.
>
> I suppose the main issue is the virtio API changes that this introduces,
> which will have to be accepted to the spec.
>
> One small comment to patch 4: there I just assumed the virtual device
> time is CLOCK_TAI. There is a concurrent feature under review for HW
> pacing offload with AF_XDP sockets. The clock issue comes up a bit. In
> general, for hardware we cannot assume a clock.
Any reason for this? E.g some modern NIC have PTP support.
> For virtio, perhaps
> assuming the same monotonic hardware clock in guest and host can be
> assumed.
Note that virtio can be implemented in hardware now. So we can assume
things like the kvm ptp clock.
> But this clock alignment needs some thought.
>
Thanks
Jason Wang wrote:
> On Tue, Dec 19, 2023 at 12:36 AM Willem de Bruijn
> <[email protected]> wrote:
> >
> > Steffen Trumtrar wrote:
> > > This series tries to pick up the work on the virtio-net timestamping
> > > feature from Willem de Bruijn.
> > >
> > > Original series
> > > Message-Id: [email protected]
> > > Subject: [PATCH RFC v2 0/4] virtio-net: add tx-hash, rx-tstamp,
> > > tx-tstamp and tx-time
> > > From: Willem de Bruijn <[email protected]>
> > >
> > > RFC for four new features to the virtio network device:
> > >
> > > 1. pass tx flow state to host, for routing + telemetry
> > > 2. pass rx tstamp to guest, for better RTT estimation
> > > 3. pass tx tstamp to guest, idem
> > > 3. pass tx delivery time to host, for accurate pacing
> > >
> > > All would introduce an extension to the virtio spec.
> > >
> > > The original series consisted of a hack around the DMA API, which should
> > > be fixed in this series.
> > >
> > > The changes in this series are to the driver side. For the changes to qemu see:
> > > https://github.com/strumtrar/qemu/tree/v8.1.1/virtio-net-ptp
> > >
> > > Currently only virtio-net is supported. The original series used
> > > vhost-net as backend. However, the path through tun via sendmsg doesn't
> > > allow us to write data back to the driver side without any hacks.
> > > Therefore use the way via plain virtio-net without vhost albeit better
> > > performance.
> > >
> > > Signed-off-by: Steffen Trumtrar <[email protected]>
> >
> > Thanks for picking this back up, Steffen. Nice to see that the code still
> > applies mostly cleanly.
> >
> > For context: I dropped the work only because I had no real device
> > implementation. The referenced patch series to qemu changes that.
> >
> > I suppose the main issue is the virtio API changes that this introduces,
> > which will have to be accepted to the spec.
> >
> > One small comment to patch 4: there I just assumed the virtual device
> > time is CLOCK_TAI. There is a concurrent feature under review for HW
> > pacing offload with AF_XDP sockets. The clock issue comes up a bit. In
> > general, for hardware we cannot assume a clock.
>
> Any reason for this? E.g some modern NIC have PTP support.
I meant that we cannot assume a specific clock, if aiming to offload
existing pacing (or "launch time") methods.
The issue discussed in the AF_XDP thread is whether to use CLOCK_TAI
or CLOCK_MONOTONIC. Both of which are already in use in software
pacing offload, in the ETF and FQ qdiscs, respectively.
But for virtio it may be acceptable to restrict to one clock, such as
CLOCK_REALTIME or CLOCK_TAI.
CLOCK_MONOTONIC being boottime is almost certain to have an offset.
Even if the clocks' rates are synchronized with phc2sys.
> > For virtio, perhaps
> > assuming the same monotonic hardware clock in guest and host can be
> > assumed.
>
> Note that virtio can be implemented in hardware now. So we can assume
> things like the kvm ptp clock.
>
> > But this clock alignment needs some thought.
> >
>
> Thanks
>