kvm ptp targets to provide high precision time sync between guest
and host in virtualization environment. Here, we enable kvm ptp
for arm64.
change log:
from v11 to v10:
(1) rebase code on 5.7_rc2.
(2) remove support for arm32, as kvm support for arm32 will be
removed [1]
(3) add error report in ptp_kvm initialization.
from v10 to v9:
(1) change code base to v5.5.
(2) enable ptp_kvm both for arm32 and arm64.
(3) let user choose which of virtual counter or physical counter
should return when using crosstimestamp mode of ptp_kvm for arm/arm64.
(4) extend input argument for getcrosstimestamp API.
from v8 to v9:
(1) move ptp_kvm.h to driver/ptp/
(2) replace license declaration of ptp_kvm.h the same with other
header files in the same directory.
from v7 to v8:
(1) separate adding clocksource id for arm_arch_counter as a
single patch.
(2) update commit message for patch 4/8.
(3) refine patch 7/8 and patch 8/8 to make them more independent.
from v6 to v7:
(1) include the omitted clocksource_id.h in last version.
(2) reorder the header file in patch.
(3) refine some words in commit message to make it more impersonal.
from v5 to v6:
(1) apply Mark's patch[4] to get SMCCC conduit.
(2) add mechanism to recognize current clocksource by add
clocksouce_id value into struct clocksource instead of method in patch-v5.
(3) rename kvm_arch_ptp_get_clock_fn into
kvm_arch_ptp_get_crosststamp.
from v4 to v5:
(1) remove hvc delay compensasion as it should leave to userspace.
(2) check current clocksource in hvc call service.
(3) expose current clocksource by adding it to
system_time_snapshot.
(4) add helper to check if clocksource is arm_arch_counter.
(5) rename kvm_ptp.c to ptp_kvm_common.c
from v3 to v4:
(1) fix clocksource of ptp_kvm to arch_sys_counter.
(2) move kvm_arch_ptp_get_clock_fn into arm_arch_timer.c
(3) subtract cntvoff before return cycles from host.
(4) use ktime_get_snapshot instead of getnstimeofday and
get_current_counterval to return time and counter value.
(5) split ktime and counter into two 32-bit block respectively
to avoid Y2038-safe issue.
(6) set time compensation to device time as half of the delay of
hvc call.
(7) add ARM_ARCH_TIMER as dependency of ptp_kvm for
arm64.
from v2 to v3:
(1) fix some issues in commit log.
(2) add some receivers in send list.
from v1 to v2:
(1) move arch-specific code from arch/ to driver/ptp/
(2) offer mechanism to inform userspace if ptp_kvm service is
available.
(3) separate ptp_kvm code for arm64 into hypervisor part and
guest part.
(4) add API to expose monotonic clock and counter value.
(5) refine code: remove no necessary part and reconsitution.
[1] https://patchwork.kernel.org/cover/11373351/
Jianyong Wu (8):
psci: export psci conduit get helper.
ptp: Reorganize ptp_kvm modules to make it arch-independent.
clocksource: Add clocksource id for arm arch counter
psci: Add hypercall service for ptp_kvm.
ptp: arm64: Enable ptp_kvm for arm/arm64
ptp: extend input argument for getcrosstimestamp API
arm64: add mechanism to let user choose which counter to return
arm64: Add kvm capability check extension for ptp_kvm
Thomas Gleixner (1):
time: Add mechanism to recognize clocksource in time_get_snapshot
drivers/clocksource/arm_arch_timer.c | 33 ++++++++
drivers/firmware/psci/psci.c | 1 +
drivers/net/ethernet/intel/e1000e/ptp.c | 3 +-
drivers/ptp/Kconfig | 2 +-
drivers/ptp/Makefile | 1 +
drivers/ptp/ptp_chardev.c | 8 +-
drivers/ptp/ptp_kvm.h | 11 +++
drivers/ptp/ptp_kvm_arm64.c | 53 ++++++++++++
drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 85 ++++++--------------
drivers/ptp/ptp_kvm_x86.c | 89 +++++++++++++++++++++
include/linux/arm-smccc.h | 21 +++++
include/linux/clocksource.h | 6 ++
include/linux/clocksource_ids.h | 12 +++
include/linux/ptp_clock_kernel.h | 3 +-
include/linux/timekeeping.h | 12 +--
include/uapi/linux/kvm.h | 1 +
include/uapi/linux/ptp_clock.h | 4 +-
kernel/time/clocksource.c | 3 +
kernel/time/timekeeping.c | 1 +
virt/kvm/arm/arm.c | 1 +
virt/kvm/arm/hypercalls.c | 44 +++++++++-
21 files changed, 322 insertions(+), 72 deletions(-)
create mode 100644 drivers/ptp/ptp_kvm.h
create mode 100644 drivers/ptp/ptp_kvm_arm64.c
rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%)
create mode 100644 drivers/ptp/ptp_kvm_x86.c
create mode 100644 include/linux/clocksource_ids.h
--
2.17.1
In general, vm inside will use virtual counter compered with host use
phyical counter. But in some special scenarios, like nested virtualization,
phyical counter maybe used by vm. A interface added in ptp_kvm driver to
offer a mechanism to let user choose which counter should be return from
host.
To use this feature, you should call PTP_EXTTS_REQUEST(2) ioctl with flag
set bit PTP_KVM_ARM_PHY_COUNTER in its argument then call
PTP_SYS_OFFSET_PRECISE(2) ioctl to get the cross timestamp and phyical
counter will return. If the bit not set or no call for PTP_EXTTS_REQUEST2,
virtual counter will return by default.
Signed-off-by: Jianyong Wu <[email protected]>
Suggested-by: Marc Zyngier <[email protected]>
---
drivers/clocksource/arm_arch_timer.c | 11 ++++++++++-
drivers/ptp/ptp_chardev.c | 8 +++++++-
drivers/ptp/ptp_kvm_common.c | 7 ++++---
include/uapi/linux/ptp_clock.h | 4 +++-
4 files changed, 24 insertions(+), 6 deletions(-)
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 47d69b3f2d9a..52f629d9c561 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -1650,7 +1650,16 @@ int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *ts,
struct arm_smccc_res hvc_res;
ktime_t ktime_overall;
- arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, &hvc_res);
+ /*
+ * an argument will be passed by a0 to determine weather virtual
+ * counter or phyical counter should be passed back.
+ */
+ if (*ctx)
+ arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID,
+ ARM_SMCCC_HYP_KVM_PTP_PHY, &hvc_res);
+ else
+ arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID,
+ ARM_SMCCC_HYP_KVM_PTP_VIRT, &hvc_res);
if ((int)(hvc_res.a0) < 0)
return -EOPNOTSUPP;
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index fef72f29f3c8..505ed7eb0ca5 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -122,6 +122,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg)
struct ptp_pin_desc pd;
struct timespec64 ts;
int enable, err = 0;
+ static long flag;
switch (cmd) {
@@ -149,6 +150,11 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg)
err = -EFAULT;
break;
}
+ /*
+ * Tell driver to call for physical counter.
+ * This is only for arm.
+ */
+ flag = req.extts.flags & PTP_KVM_ARM_PHY_COUNTER;
if (cmd == PTP_EXTTS_REQUEST2) {
/* Tell the drivers to check the flags carefully. */
req.extts.flags |= PTP_STRICT_FLAGS;
@@ -235,7 +241,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg)
err = -EOPNOTSUPP;
break;
}
- err = ptp->info->getcrosststamp(ptp->info, &xtstamp, NULL);
+ err = ptp->info->getcrosststamp(ptp->info, &xtstamp, &flag);
if (err)
break;
diff --git a/drivers/ptp/ptp_kvm_common.c b/drivers/ptp/ptp_kvm_common.c
index 4fdd8ab11a28..39367e230176 100644
--- a/drivers/ptp/ptp_kvm_common.c
+++ b/drivers/ptp/ptp_kvm_common.c
@@ -36,7 +36,7 @@ static int ptp_kvm_get_time_fn(ktime_t *device_time,
spin_lock(&kvm_ptp_lock);
preempt_disable_notrace();
- ret = kvm_arch_ptp_get_crosststamp(&cycle, &tspec, &cs);
+ ret = kvm_arch_ptp_get_crosststamp(&cycle, &tspec, &cs, ctx);
if (ret != 0) {
pr_err_ratelimited("clock pairing hypercall ret %lu\n", ret);
spin_unlock(&kvm_ptp_lock);
@@ -57,9 +57,10 @@ static int ptp_kvm_get_time_fn(ktime_t *device_time,
}
static int ptp_kvm_getcrosststamp(struct ptp_clock_info *ptp,
- struct system_device_crosststamp *xtstamp)
+ struct system_device_crosststamp *xtstamp,
+ long *flag)
{
- return get_device_system_crosststamp(ptp_kvm_get_time_fn, NULL,
+ return get_device_system_crosststamp(ptp_kvm_get_time_fn, flag,
NULL, xtstamp);
}
diff --git a/include/uapi/linux/ptp_clock.h b/include/uapi/linux/ptp_clock.h
index 9dc9d0079e98..71e388a82244 100644
--- a/include/uapi/linux/ptp_clock.h
+++ b/include/uapi/linux/ptp_clock.h
@@ -32,6 +32,7 @@
#define PTP_RISING_EDGE (1<<1)
#define PTP_FALLING_EDGE (1<<2)
#define PTP_STRICT_FLAGS (1<<3)
+#define PTP_KVM_ARM_PHY_COUNTER (1<<4)
#define PTP_EXTTS_EDGES (PTP_RISING_EDGE | PTP_FALLING_EDGE)
/*
@@ -40,7 +41,8 @@
#define PTP_EXTTS_VALID_FLAGS (PTP_ENABLE_FEATURE | \
PTP_RISING_EDGE | \
PTP_FALLING_EDGE | \
- PTP_STRICT_FLAGS)
+ PTP_STRICT_FLAGS | \
+ PTP_KVM_ARM_PHY_COUNTER)
/*
* flag fields valid for the original PTP_EXTTS_REQUEST ioctl.
--
2.17.1
sometimes we may need tell getcrosstimestamp call back how to perform
itself. Extending input arguments for getcrosstimestamp API to offer more
exquisite control for the operation.
Signed-off-by: Jianyong Wu <[email protected]>
---
drivers/clocksource/arm_arch_timer.c | 2 +-
drivers/net/ethernet/intel/e1000e/ptp.c | 3 ++-
drivers/ptp/ptp_chardev.c | 2 +-
drivers/ptp/ptp_kvm.h | 2 +-
drivers/ptp/ptp_kvm_x86.c | 2 +-
include/linux/ptp_clock_kernel.h | 3 ++-
6 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 7a33993c0d05..47d69b3f2d9a 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -1645,7 +1645,7 @@ TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init);
#if IS_ENABLED(CONFIG_PTP_1588_CLOCK_KVM)
#include <linux/arm-smccc.h>
int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *ts,
- struct clocksource **cs)
+ struct clocksource **cs, long *ctx)
{
struct arm_smccc_res hvc_res;
ktime_t ktime_overall;
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c b/drivers/net/ethernet/intel/e1000e/ptp.c
index 439fda2f5368..e9270d98a497 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -150,7 +150,8 @@ static int e1000e_phc_get_syncdevicetime(ktime_t *device,
* clock values in ns.
**/
static int e1000e_phc_getcrosststamp(struct ptp_clock_info *ptp,
- struct system_device_crosststamp *xtstamp)
+ struct system_device_crosststamp *xtstamp,
+ long *__unused)
{
struct e1000_adapter *adapter = container_of(ptp, struct e1000_adapter,
ptp_clock_info);
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index 93d574faf1fe..fef72f29f3c8 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -235,7 +235,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg)
err = -EOPNOTSUPP;
break;
}
- err = ptp->info->getcrosststamp(ptp->info, &xtstamp);
+ err = ptp->info->getcrosststamp(ptp->info, &xtstamp, NULL);
if (err)
break;
diff --git a/drivers/ptp/ptp_kvm.h b/drivers/ptp/ptp_kvm.h
index 4bf1802bbeb8..ccceacbe8398 100644
--- a/drivers/ptp/ptp_kvm.h
+++ b/drivers/ptp/ptp_kvm.h
@@ -8,4 +8,4 @@
int kvm_arch_ptp_init(void);
int kvm_arch_ptp_get_clock(struct timespec64 *ts);
int kvm_arch_ptp_get_crosststamp(unsigned long *cycle,
- struct timespec64 *tspec, void *cs);
+ struct timespec64 *tspec, struct clocksource **cs, long *ctx);
diff --git a/drivers/ptp/ptp_kvm_x86.c b/drivers/ptp/ptp_kvm_x86.c
index 55417b3b282d..f372555671eb 100644
--- a/drivers/ptp/ptp_kvm_x86.c
+++ b/drivers/ptp/ptp_kvm_x86.c
@@ -55,7 +55,7 @@ int kvm_arch_ptp_get_clock(struct timespec64 *ts)
}
int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *tspec,
- struct clocksource **cs)
+ struct clocksource **cs, void *ctx)
{
unsigned long ret;
unsigned int version;
diff --git a/include/linux/ptp_clock_kernel.h b/include/linux/ptp_clock_kernel.h
index 121a7eda4593..1638ad75b5e5 100644
--- a/include/linux/ptp_clock_kernel.h
+++ b/include/linux/ptp_clock_kernel.h
@@ -133,7 +133,8 @@ struct ptp_clock_info {
int (*gettimex64)(struct ptp_clock_info *ptp, struct timespec64 *ts,
struct ptp_system_timestamp *sts);
int (*getcrosststamp)(struct ptp_clock_info *ptp,
- struct system_device_crosststamp *cts);
+ struct system_device_crosststamp *cts,
+ long *flag);
int (*settime64)(struct ptp_clock_info *p, const struct timespec64 *ts);
int (*enable)(struct ptp_clock_info *ptp,
struct ptp_clock_request *request, int on);
--
2.17.1
ptp_kvm modules will get this service through smccc call.
The service offers real time and counter cycle of host for guest.
Also let caller determine which cycle of virtual counter or physical counter
to return.
Signed-off-by: Jianyong Wu <[email protected]>
---
include/linux/arm-smccc.h | 21 +++++++++++++++++++
virt/kvm/arm/hypercalls.c | 44 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 64 insertions(+), 1 deletion(-)
diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
index 59494df0f55b..747b7595d0c6 100644
--- a/include/linux/arm-smccc.h
+++ b/include/linux/arm-smccc.h
@@ -77,6 +77,27 @@
ARM_SMCCC_SMC_32, \
0, 0x7fff)
+/* PTP KVM call requests clock time from guest OS to host */
+#define ARM_SMCCC_HYP_KVM_PTP_FUNC_ID \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_SMC_32, \
+ ARM_SMCCC_OWNER_STANDARD_HYP, \
+ 0)
+
+/* request for virtual counter from ptp_kvm guest */
+#define ARM_SMCCC_HYP_KVM_PTP_VIRT \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_SMC_32, \
+ ARM_SMCCC_OWNER_STANDARD_HYP, \
+ 1)
+
+/* request for physical counter from ptp_kvm guest */
+#define ARM_SMCCC_HYP_KVM_PTP_PHY \
+ ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
+ ARM_SMCCC_SMC_32, \
+ ARM_SMCCC_OWNER_STANDARD_HYP, \
+ 2)
+
#ifndef __ASSEMBLY__
#include <linux/linkage.h>
diff --git a/virt/kvm/arm/hypercalls.c b/virt/kvm/arm/hypercalls.c
index 550dfa3e53cd..a5309c28d4dc 100644
--- a/virt/kvm/arm/hypercalls.c
+++ b/virt/kvm/arm/hypercalls.c
@@ -3,6 +3,7 @@
#include <linux/arm-smccc.h>
#include <linux/kvm_host.h>
+#include <linux/clocksource_ids.h>
#include <asm/kvm_emulate.h>
@@ -11,8 +12,11 @@
int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
{
- u32 func_id = smccc_get_function(vcpu);
+ struct system_time_snapshot systime_snapshot;
+ long arg[4];
+ u64 cycles;
long val = SMCCC_RET_NOT_SUPPORTED;
+ u32 func_id = smccc_get_function(vcpu);
u32 feature;
gpa_t gpa;
@@ -62,6 +66,44 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
if (gpa != GPA_INVALID)
val = gpa;
break;
+ /*
+ * This serves virtual kvm_ptp.
+ * Four values will be passed back.
+ * reg0 stores high 32-bit host ktime;
+ * reg1 stores low 32-bit host ktime;
+ * reg2 stores high 32-bit difference of host cycles and cntvoff;
+ * reg3 stores low 32-bit difference of host cycles and cntvoff.
+ */
+ case ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
+ /*
+ * system time and counter value must captured in the same
+ * time to keep consistency and precision.
+ */
+ ktime_get_snapshot(&systime_snapshot);
+ if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
+ break;
+ arg[0] = upper_32_bits(systime_snapshot.real);
+ arg[1] = lower_32_bits(systime_snapshot.real);
+ /*
+ * which of virtual counter or physical counter being
+ * asked for is decided by the first argument.
+ */
+ feature = smccc_get_arg1(vcpu);
+ switch (feature) {
+ case ARM_SMCCC_HYP_KVM_PTP_PHY:
+ cycles = systime_snapshot.cycles;
+ break;
+ case ARM_SMCCC_HYP_KVM_PTP_VIRT:
+ default:
+ cycles = systime_snapshot.cycles -
+ vcpu_vtimer(vcpu)->cntvoff;
+ }
+ arg[2] = upper_32_bits(cycles);
+ arg[3] = lower_32_bits(cycles);
+
+ smccc_set_retval(vcpu, arg[0], arg[1], arg[2], arg[3]);
+ return 1;
+
default:
return kvm_psci_call(vcpu);
}
--
2.17.1
Currently, there is no mechanism to keep time sync between guest and host
in arm64 virtualization environment. Time in guest will drift compared
with host after boot up as they may both use third party time sources
to correct their time respectively. The time deviation will be in order
of milliseconds. But in some scenarios,like in cloud envirenment, we ask
for higher time precision.
kvm ptp clock, which choose the host clock source clock as a reference
clock to sync time clock between guest and host has been adopted by x86
which makes the time sync order from milliseconds to nanoseconds.
This patch enables kvm ptp on arm64 and improve clock sync precison
significantly.
Test result comparison between with kvm ptp and without it in arm64 is
as follows. This test derived from the result of command 'chronyc
sources'. we should take more care of the last sample column which shows
the offset between the local clock and the source at the last measurement.
no kvm ptp in guest:
MS Name/IP address Stratum Poll Reach LastRx Last sample
========================================================================
^* dns1.synet.edu.cn 2 6 377 13 +1040us[+1581us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 21 +1040us[+1581us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 29 +1040us[+1581us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 37 +1040us[+1581us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 45 +1040us[+1581us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 53 +1040us[+1581us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 61 +1040us[+1581us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 4 -130us[ +796us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 12 -130us[ +796us] +/- 21ms
^* dns1.synet.edu.cn 2 6 377 20 -130us[ +796us] +/- 21ms
in host:
MS Name/IP address Stratum Poll Reach LastRx Last sample
========================================================================
^* 120.25.115.20 2 7 377 72 -470us[ -603us] +/- 18ms
^* 120.25.115.20 2 7 377 92 -470us[ -603us] +/- 18ms
^* 120.25.115.20 2 7 377 112 -470us[ -603us] +/- 18ms
^* 120.25.115.20 2 7 377 2 +872ns[-6808ns] +/- 17ms
^* 120.25.115.20 2 7 377 22 +872ns[-6808ns] +/- 17ms
^* 120.25.115.20 2 7 377 43 +872ns[-6808ns] +/- 17ms
^* 120.25.115.20 2 7 377 63 +872ns[-6808ns] +/- 17ms
^* 120.25.115.20 2 7 377 83 +872ns[-6808ns] +/- 17ms
^* 120.25.115.20 2 7 377 103 +872ns[-6808ns] +/- 17ms
^* 120.25.115.20 2 7 377 123 +872ns[-6808ns] +/- 17ms
The dns1.synet.edu.cn is the network reference clock for guest and
120.25.115.20 is the network reference clock for host. we can't get the
clock error between guest and host directly, but a roughly estimated value
will be in order of hundreds of us to ms.
with kvm ptp in guest:
chrony has been disabled in host to remove the disturb by network clock.
MS Name/IP address Stratum Poll Reach LastRx Last sample
========================================================================
* PHC0 0 3 377 8 -7ns[ +1ns] +/- 3ns
* PHC0 0 3 377 8 +1ns[ +16ns] +/- 3ns
* PHC0 0 3 377 6 -4ns[ -0ns] +/- 6ns
* PHC0 0 3 377 6 -8ns[ -12ns] +/- 5ns
* PHC0 0 3 377 5 +2ns[ +4ns] +/- 4ns
* PHC0 0 3 377 13 +2ns[ +4ns] +/- 4ns
* PHC0 0 3 377 12 -4ns[ -6ns] +/- 4ns
* PHC0 0 3 377 11 -8ns[ -11ns] +/- 6ns
* PHC0 0 3 377 10 -14ns[ -20ns] +/- 4ns
* PHC0 0 3 377 8 +4ns[ +5ns] +/- 4ns
The PHC0 is the ptp clock which choose the host clock as its source
clock. So we can be sure to say that the clock error between host and guest
is in order of ns.
Signed-off-by: Jianyong Wu <[email protected]>
---
drivers/clocksource/arm_arch_timer.c | 22 ++++++++++++
drivers/ptp/Kconfig | 2 +-
drivers/ptp/ptp_kvm_arm64.c | 53 ++++++++++++++++++++++++++++
3 files changed, 76 insertions(+), 1 deletion(-)
create mode 100644 drivers/ptp/ptp_kvm_arm64.c
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index 0f44f296ed17..7a33993c0d05 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -1641,3 +1641,25 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
}
TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init);
#endif
+
+#if IS_ENABLED(CONFIG_PTP_1588_CLOCK_KVM)
+#include <linux/arm-smccc.h>
+int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *ts,
+ struct clocksource **cs)
+{
+ struct arm_smccc_res hvc_res;
+ ktime_t ktime_overall;
+
+ arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, &hvc_res);
+ if ((int)(hvc_res.a0) < 0)
+ return -EOPNOTSUPP;
+
+ ktime_overall = (long long)hvc_res.a0 << 32 | hvc_res.a1;
+ *ts = ktime_to_timespec64(ktime_overall);
+ *cycle = (long long)hvc_res.a2 << 32 | hvc_res.a3;
+ *cs = &clocksource_counter;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(kvm_arch_ptp_get_crosststamp);
+#endif
diff --git a/drivers/ptp/Kconfig b/drivers/ptp/Kconfig
index 86400c708150..0733c8c61541 100644
--- a/drivers/ptp/Kconfig
+++ b/drivers/ptp/Kconfig
@@ -106,7 +106,7 @@ config PTP_1588_CLOCK_PCH
config PTP_1588_CLOCK_KVM
tristate "KVM virtual PTP clock"
depends on PTP_1588_CLOCK
- depends on KVM_GUEST && X86
+ depends on KVM_GUEST && X86 || ARM64 && ARM_ARCH_TIMER && ARM_PSCI_FW
default y
help
This driver adds support for using kvm infrastructure as a PTP
diff --git a/drivers/ptp/ptp_kvm_arm64.c b/drivers/ptp/ptp_kvm_arm64.c
new file mode 100644
index 000000000000..9752e849e0ae
--- /dev/null
+++ b/drivers/ptp/ptp_kvm_arm64.c
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Virtual PTP 1588 clock for use with KVM guests
+ * Copyright (C) 2019 ARM Ltd.
+ * All Rights Reserved
+ */
+
+#include <linux/kernel.h>
+#include <linux/err.h>
+#include <asm/hypervisor.h>
+#include <linux/module.h>
+#include <linux/psci.h>
+#include <linux/arm-smccc.h>
+#include <linux/timecounter.h>
+#include <linux/sched/clock.h>
+#include <asm/arch_timer.h>
+
+int kvm_arch_ptp_init(void)
+{
+ struct arm_smccc_res hvc_res;
+
+ arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID,
+ &hvc_res);
+ if ((int)(hvc_res.a0) < 0)
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+int kvm_arch_ptp_get_clock_generic(struct timespec64 *ts,
+ struct arm_smccc_res *hvc_res)
+{
+ ktime_t ktime_overall;
+
+ arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID,
+ hvc_res);
+ if ((int)(hvc_res->a0) < 0)
+ return -EOPNOTSUPP;
+
+ ktime_overall = (long long)hvc_res->a0 << 32 | hvc_res->a1;
+ *ts = ktime_to_timespec64(ktime_overall);
+
+ return 0;
+}
+
+int kvm_arch_ptp_get_clock(struct timespec64 *ts)
+{
+ struct arm_smccc_res hvc_res;
+
+ kvm_arch_ptp_get_clock_generic(ts, &hvc_res);
+
+ return 0;
+}
--
2.17.1
Let userspace check if there is kvm ptp service in host.
Before VMs migrate to another host, VMM may check if this
cap is available to determine the next behavior.
Signed-off-by: Jianyong Wu <[email protected]>
Suggested-by: Marc Zyngier <[email protected]>
---
include/uapi/linux/kvm.h | 1 +
virt/kvm/arm/arm.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 428c7dde6b4b..668049ad78e1 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1017,6 +1017,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_S390_VCPU_RESETS 179
#define KVM_CAP_S390_PROTECTED 180
#define KVM_CAP_PPC_SECURE_GUEST 181
+#define KVM_CAP_ARM_KVM_PTP 182
#ifdef KVM_CAP_IRQ_ROUTING
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index 48d0ec44ad77..4726a88949f5 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -195,6 +195,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_ARM_IRQ_LINE_LAYOUT_2:
case KVM_CAP_ARM_NISV_TO_USER:
case KVM_CAP_ARM_INJECT_EXT_DABT:
+ case KVM_CAP_ARM_KVM_PTP:
r = 1;
break;
case KVM_CAP_ARM_SET_DEVICE_ADDR:
--
2.17.1
On Tue, Apr 21, 2020 at 11:23:00AM +0800, Jianyong Wu wrote:
> ptp_kvm modules will get this service through smccc call.
> The service offers real time and counter cycle of host for guest.
> Also let caller determine which cycle of virtual counter or physical counter
> to return.
>
> Signed-off-by: Jianyong Wu <[email protected]>
> ---
> include/linux/arm-smccc.h | 21 +++++++++++++++++++
> virt/kvm/arm/hypercalls.c | 44 ++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 64 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
> index 59494df0f55b..747b7595d0c6 100644
> --- a/include/linux/arm-smccc.h
> +++ b/include/linux/arm-smccc.h
> @@ -77,6 +77,27 @@
> ARM_SMCCC_SMC_32, \
> 0, 0x7fff)
>
> +/* PTP KVM call requests clock time from guest OS to host */
> +#define ARM_SMCCC_HYP_KVM_PTP_FUNC_ID \
> + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
> + ARM_SMCCC_SMC_32, \
> + ARM_SMCCC_OWNER_STANDARD_HYP, \
> + 0)
> +
> +/* request for virtual counter from ptp_kvm guest */
> +#define ARM_SMCCC_HYP_KVM_PTP_VIRT \
> + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
> + ARM_SMCCC_SMC_32, \
> + ARM_SMCCC_OWNER_STANDARD_HYP, \
> + 1)
> +
> +/* request for physical counter from ptp_kvm guest */
> +#define ARM_SMCCC_HYP_KVM_PTP_PHY \
> + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
> + ARM_SMCCC_SMC_32, \
> + ARM_SMCCC_OWNER_STANDARD_HYP, \
> + 2)
ARM_SMCCC_OWNER_STANDARD_HYP is for standard calls as defined in SMCCC
and companion documents, so we should refer to the specific
documentation here. Where are these calls defined?
If these calls are Linux-specific then ARM_SMCCC_OWNER_STANDARD_HYP
isn't appropriate to use, as they are vendor-specific hypervisor service
call.
It looks like we don't currently have a ARM_SMCCC_OWNER_HYP for that
(which IIUC would be 6), but we can add one as necessary. I think that
Will might have added that as part of his SMCCC probing bits.
> +
> #ifndef __ASSEMBLY__
>
> #include <linux/linkage.h>
> diff --git a/virt/kvm/arm/hypercalls.c b/virt/kvm/arm/hypercalls.c
> index 550dfa3e53cd..a5309c28d4dc 100644
> --- a/virt/kvm/arm/hypercalls.c
> +++ b/virt/kvm/arm/hypercalls.c
> @@ -3,6 +3,7 @@
>
> #include <linux/arm-smccc.h>
> #include <linux/kvm_host.h>
> +#include <linux/clocksource_ids.h>
>
> #include <asm/kvm_emulate.h>
>
> @@ -11,8 +12,11 @@
>
> int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
> {
> - u32 func_id = smccc_get_function(vcpu);
> + struct system_time_snapshot systime_snapshot;
> + long arg[4];
> + u64 cycles;
> long val = SMCCC_RET_NOT_SUPPORTED;
> + u32 func_id = smccc_get_function(vcpu);
> u32 feature;
> gpa_t gpa;
>
> @@ -62,6 +66,44 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
> if (gpa != GPA_INVALID)
> val = gpa;
> break;
> + /*
> + * This serves virtual kvm_ptp.
> + * Four values will be passed back.
> + * reg0 stores high 32-bit host ktime;
> + * reg1 stores low 32-bit host ktime;
> + * reg2 stores high 32-bit difference of host cycles and cntvoff;
> + * reg3 stores low 32-bit difference of host cycles and cntvoff.
> + */
> + case ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
Shouldn't the host opt-in to providing this to the guest, as with other
features?
> + /*
> + * system time and counter value must captured in the same
> + * time to keep consistency and precision.
> + */
> + ktime_get_snapshot(&systime_snapshot);
> + if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
> + break;
> + arg[0] = upper_32_bits(systime_snapshot.real);
> + arg[1] = lower_32_bits(systime_snapshot.real);
Why exactly does the guest need the host's real time? Neither the cover
letter nor this commit message have explained that, and for those of us
unfamliar with PTP it would be very helpful to know that to understand
what's going on.
> + /*
> + * which of virtual counter or physical counter being
> + * asked for is decided by the first argument.
> + */
> + feature = smccc_get_arg1(vcpu);
> + switch (feature) {
> + case ARM_SMCCC_HYP_KVM_PTP_PHY:
> + cycles = systime_snapshot.cycles;
> + break;
> + case ARM_SMCCC_HYP_KVM_PTP_VIRT:
> + default:
> + cycles = systime_snapshot.cycles -
> + vcpu_vtimer(vcpu)->cntvoff;
> + }
> + arg[2] = upper_32_bits(cycles);
> + arg[3] = lower_32_bits(cycles);
> +
> + smccc_set_retval(vcpu, arg[0], arg[1], arg[2], arg[3]);
I think the 'arg' buffer is confusing here, and it'd be clearer to have:
u64 snaphot;
u64 cycles;
... and here do:
smccc_set_retval(vcpu,
upper_32_bits(snaphot),
lower_32_bits(snapshot),
upper_32_bits(cycles),
lower_32_bits(cycles));
Thanks,
Mark.
On 2020/4/21 5:57 PM, Mark Rutland wrote:
Hi Mark,
> On Tue, Apr 21, 2020 at 11:23:00AM +0800, Jianyong Wu wrote:
>> ptp_kvm modules will get this service through smccc call.
>> The service offers real time and counter cycle of host for guest.
>> Also let caller determine which cycle of virtual counter or physical counter
>> to return.
>>
>> Signed-off-by: Jianyong Wu <[email protected]>
>> ---
>> include/linux/arm-smccc.h | 21 +++++++++++++++++++
>> virt/kvm/arm/hypercalls.c | 44 ++++++++++++++++++++++++++++++++++++++-
>> 2 files changed, 64 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
>> index 59494df0f55b..747b7595d0c6 100644
>> --- a/include/linux/arm-smccc.h
>> +++ b/include/linux/arm-smccc.h
>> @@ -77,6 +77,27 @@
>> ARM_SMCCC_SMC_32, \
>> 0, 0x7fff)
>>
>> +/* PTP KVM call requests clock time from guest OS to host */
>> +#define ARM_SMCCC_HYP_KVM_PTP_FUNC_ID \
>> + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
>> + ARM_SMCCC_SMC_32, \
>> + ARM_SMCCC_OWNER_STANDARD_HYP, \
>> + 0)
>> +
>> +/* request for virtual counter from ptp_kvm guest */
>> +#define ARM_SMCCC_HYP_KVM_PTP_VIRT \
>> + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
>> + ARM_SMCCC_SMC_32, \
>> + ARM_SMCCC_OWNER_STANDARD_HYP, \
>> + 1)
>> +
>> +/* request for physical counter from ptp_kvm guest */
>> +#define ARM_SMCCC_HYP_KVM_PTP_PHY \
>> + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
>> + ARM_SMCCC_SMC_32, \
>> + ARM_SMCCC_OWNER_STANDARD_HYP, \
>> + 2)
> ARM_SMCCC_OWNER_STANDARD_HYP is for standard calls as defined in SMCCC
> and companion documents, so we should refer to the specific
> documentation here. Where are these calls defined?
yeah, should add reference docs of "SMC CALLING CONVENTION" here.
> If these calls are Linux-specific then ARM_SMCCC_OWNER_STANDARD_HYP
> isn't appropriate to use, as they are vendor-specific hypervisor service
> call.
yeah, vendor-specific service is more suitable for ptp_kvm.
>
> It looks like we don't currently have a ARM_SMCCC_OWNER_HYP for that
> (which IIUC would be 6), but we can add one as necessary. I think that
> Will might have added that as part of his SMCCC probing bits.
ok, I will add a new "ARM_SMCCC_OWNER_VENDOR_HYP" whose IIUC is 6
and create "ARM_SMCCC_HYP_KVM_PTP_ID" base on it.
>
>> +
>> #ifndef __ASSEMBLY__
>>
>> #include <linux/linkage.h>
>> diff --git a/virt/kvm/arm/hypercalls.c b/virt/kvm/arm/hypercalls.c
>> index 550dfa3e53cd..a5309c28d4dc 100644
>> --- a/virt/kvm/arm/hypercalls.c
>> +++ b/virt/kvm/arm/hypercalls.c
>> @@ -3,6 +3,7 @@
>>
>> #include <linux/arm-smccc.h>
>> #include <linux/kvm_host.h>
>> +#include <linux/clocksource_ids.h>
>>
>> #include <asm/kvm_emulate.h>
>>
>> @@ -11,8 +12,11 @@
>>
>> int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
>> {
>> - u32 func_id = smccc_get_function(vcpu);
>> + struct system_time_snapshot systime_snapshot;
>> + long arg[4];
>> + u64 cycles;
>> long val = SMCCC_RET_NOT_SUPPORTED;
>> + u32 func_id = smccc_get_function(vcpu);
>> u32 feature;
>> gpa_t gpa;
>>
>> @@ -62,6 +66,44 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
>> if (gpa != GPA_INVALID)
>> val = gpa;
>> break;
>> + /*
>> + * This serves virtual kvm_ptp.
>> + * Four values will be passed back.
>> + * reg0 stores high 32-bit host ktime;
>> + * reg1 stores low 32-bit host ktime;
>> + * reg2 stores high 32-bit difference of host cycles and cntvoff;
>> + * reg3 stores low 32-bit difference of host cycles and cntvoff.
>> + */
>> + case ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
> Shouldn't the host opt-in to providing this to the guest, as with other
> features?
er, do you mean that "ARM_SMCCC_HV_PV_TIME_XXX" as "opt-in"? if so, I
think this
kvm_ptp doesn't need a buddy. the driver in guest will call this service
in a definite way.
>> + /*
>> + * system time and counter value must captured in the same
>> + * time to keep consistency and precision.
>> + */
>> + ktime_get_snapshot(&systime_snapshot);
>> + if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
>> + break;
>> + arg[0] = upper_32_bits(systime_snapshot.real);
>> + arg[1] = lower_32_bits(systime_snapshot.real);
> Why exactly does the guest need the host's real time? Neither the cover
> letter nor this commit message have explained that, and for those of us
> unfamliar with PTP it would be very helpful to know that to understand
> what's going on.
oh, sorry, I should have added more background knowledge here.
just give some hints here:
the kvm_ptp targets to sync guest time with host. some services in user
space
like chrony can do time sync by inputing time(in kvm_ptp also clock
counter sometimes) from
remote clocksource(often network clocksource). This kvm_ptp driver can
offer a interface for
those user space service in guest to get the host time to do time sync
in guest.
>> + /*
>> + * which of virtual counter or physical counter being
>> + * asked for is decided by the first argument.
>> + */
>> + feature = smccc_get_arg1(vcpu);
>> + switch (feature) {
>> + case ARM_SMCCC_HYP_KVM_PTP_PHY:
>> + cycles = systime_snapshot.cycles;
>> + break;
>> + case ARM_SMCCC_HYP_KVM_PTP_VIRT:
>> + default:
>> + cycles = systime_snapshot.cycles -
>> + vcpu_vtimer(vcpu)->cntvoff;
>> + }
>> + arg[2] = upper_32_bits(cycles);
>> + arg[3] = lower_32_bits(cycles);
>> +
>> + smccc_set_retval(vcpu, arg[0], arg[1], arg[2], arg[3]);
> I think the 'arg' buffer is confusing here, and it'd be clearer to have:
>
> u64 snaphot;
> u64 cycles;
>
> ... and here do:
>
> smccc_set_retval(vcpu,
> upper_32_bits(snaphot),
> lower_32_bits(snapshot),
> upper_32_bits(cycles),
> lower_32_bits(cycles));
it's better to use a meaningful variant name. I will fix it.
thanks
Jianyong
>
> Thanks,
> Mark.
On Fri, Apr 24, 2020 at 03:50:22AM +0100, Jianyong Wu wrote:
> On 2020/4/21 5:57 PM, Mark Rutland wrote:
> > On Tue, Apr 21, 2020 at 11:23:00AM +0800, Jianyong Wu wrote:
> >> diff --git a/virt/kvm/arm/hypercalls.c b/virt/kvm/arm/hypercalls.c
> >> index 550dfa3e53cd..a5309c28d4dc 100644
> >> --- a/virt/kvm/arm/hypercalls.c
> >> +++ b/virt/kvm/arm/hypercalls.c
> >> @@ -62,6 +66,44 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
> >> if (gpa != GPA_INVALID)
> >> val = gpa;
> >> break;
> >> +/*
> >> + * This serves virtual kvm_ptp.
> >> + * Four values will be passed back.
> >> + * reg0 stores high 32-bit host ktime;
> >> + * reg1 stores low 32-bit host ktime;
> >> + * reg2 stores high 32-bit difference of host cycles and cntvoff;
> >> + * reg3 stores low 32-bit difference of host cycles and cntvoff.
> >> + */
> >> +case ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
> > Shouldn't the host opt-in to providing this to the guest, as with other
> > features?
>
> er, do you mean that "ARM_SMCCC_HV_PV_TIME_XXX" as "opt-in"? if so, I
> think this
>
> kvm_ptp doesn't need a buddy. the driver in guest will call this service
> in a definite way.
I mean that when creating the VM, userspace should be able to choose
whether the PTP service is provided to the guest. The host shouldn't
always provide it as there may be cases where doing so is undesireable.
> >> +/*
> >> + * system time and counter value must captured in the same
> >> + * time to keep consistency and precision.
> >> + */
> >> +ktime_get_snapshot(&systime_snapshot);
> >> +if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
> >> +break;
> >> +arg[0] = upper_32_bits(systime_snapshot.real);
> >> +arg[1] = lower_32_bits(systime_snapshot.real);
> > Why exactly does the guest need the host's real time? Neither the cover
> > letter nor this commit message have explained that, and for those of us
> > unfamliar with PTP it would be very helpful to know that to understand
> > what's going on.
>
> oh, sorry, I should have added more background knowledge here.
>
> just give some hints here:
>
> the kvm_ptp targets to sync guest time with host. some services in user
> space
>
> like chrony can do time sync by inputing time(in kvm_ptp also clock
> counter sometimes) from
>
> remote clocksource(often network clocksource). This kvm_ptp driver can
> offer a interface for
>
> those user space service in guest to get the host time to do time sync
> in guest.
I think it would be very helpful for the commit message and/or cover
letter to have a high-level desctiption of what PTP is meant to do, and
an outline of the algorithmm (clearly splitting the host and guest
bits).
It's also not clear to me what notion of host time is being exposed to
the guest (and consequently how this would interact with time changes on
the host, time namespaces, etc). Having some description of that would
be very helpful.
Thanks,
Mark.
Hi,
On Sun, Apr 26, 2020 at 06:02:23AM +0100, Jianyong Wu wrote:
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
> </head>
> <body>
Please fix your mail client to reply in plain text rather than HTML;
it's much more painful than necessary to review and have a conversation
when mails revert to HTML.
It would be very helpful if you could resend this mail as plaintext as
above.
Thanks,
Mark.
On 2020/4/24 6:39 PM, Mark Rutland wrote:
it's a resend of the last email, please ignore if you have received this.
Hi Mark,
thank you for remainder, I hope this email is in the correct format.
> On Fri, Apr 24, 2020 at 03:50:22AM +0100, Jianyong Wu wrote:
>> On 2020/4/21 5:57 PM, Mark Rutland wrote:
>>> On Tue, Apr 21, 2020 at 11:23:00AM +0800, Jianyong Wu wrote:
>>>> diff --git a/virt/kvm/arm/hypercalls.c b/virt/kvm/arm/hypercalls.c
>>>> index 550dfa3e53cd..a5309c28d4dc 100644
>>>> --- a/virt/kvm/arm/hypercalls.c
>>>> +++ b/virt/kvm/arm/hypercalls.c
>>>> @@ -62,6 +66,44 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
>>>> if (gpa != GPA_INVALID)
>>>> val = gpa;
>>>> break;
>>>> +/*
>>>> + * This serves virtual kvm_ptp.
>>>> + * Four values will be passed back.
>>>> + * reg0 stores high 32-bit host ktime;
>>>> + * reg1 stores low 32-bit host ktime;
>>>> + * reg2 stores high 32-bit difference of host cycles and cntvoff;
>>>> + * reg3 stores low 32-bit difference of host cycles and cntvoff.
>>>> + */
>>>> +case ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
>>> Shouldn't the host opt-in to providing this to the guest, as with other
>>> features?
>> er, do you mean that "ARM_SMCCC_HV_PV_TIME_XXX" as "opt-in"? if so, I
>> think this
>>
>> kvm_ptp doesn't need a buddy. the driver in guest will call this service
>> in a definite way.
> I mean that when creating the VM, userspace should be able to choose
> whether the PTP service is provided to the guest. The host shouldn't
> always provide it as there may be cases where doing so is undesireable.
>
I think I have implemented in patch 9/9 that userspace can get the info
that if the host offers the kvm_ptp service. But for now, the host
kernel will always offer the kvm_ptp capability in the current
implementation. I think x86 follow the same behavior (see [1]). so I
have not considered when and how to disable this kvm_ptp service in
host. Do you think we should offer this opt-in?
[1] kvm_pv_clock_pairing() in
https://github.com/torvalds/linux/blob/master/arch/x86/kvm/x86.c
>>>> +/*
>>>> + * system time and counter value must captured in the same
>>>> + * time to keep consistency and precision.
>>>> + */
>>>> +ktime_get_snapshot(&systime_snapshot);
>>>> +if (systime_snapshot.cs_id != CSID_ARM_ARCH_COUNTER)
>>>> +break;
>>>> +arg[0] = upper_32_bits(systime_snapshot.real);
>>>> +arg[1] = lower_32_bits(systime_snapshot.real);
>>> Why exactly does the guest need the host's real time? Neither the cover
>>> letter nor this commit message have explained that, and for those of us
>>> unfamliar with PTP it would be very helpful to know that to understand
>>> what's going on.
>> oh, sorry, I should have added more background knowledge here.
>>
>> just give some hints here:
>>
>> the kvm_ptp targets to sync guest time with host. some services in user
>> space
>>
>> like chrony can do time sync by inputing time(in kvm_ptp also clock
>> counter sometimes) from
>>
>> remote clocksource(often network clocksource). This kvm_ptp driver can
>> offer a interface for
>>
>> those user space service in guest to get the host time to do time sync
>> in guest.
> I think it would be very helpful for the commit message and/or cover
> letter to have a high-level desctiption of what PTP is meant to do, and
> an outline of the algorithmm (clearly splitting the host and guest
> bits).
ok, I will add high-level principle of kvm_ptp in commit message.
> It's also not clear to me what notion of host time is being exposed to
> the guest (and consequently how this would interact with time changes on
> the host, time namespaces, etc). Having some description of that would
> be very helpful.
sorry to have not made it clear.
Time will not change in host and only time in guest will change to sync
with host. host time is the target that time in guest want to adjust to.
guest need to get the host time then compute the different of the time
in guest and host, so the guest can adjust the time base on the difference.
I will add the base principle of time sync service in guest using
kvm_ptp in commit message.
Thanks
Jianyong
> Thanks,
> Mark.
-->
On Tue, Apr 28, 2020 at 07:14:52AM +0100, Jianyong Wu wrote:
> On 2020/4/24 6:39 PM, Mark Rutland wrote:
> > On Fri, Apr 24, 2020 at 03:50:22AM +0100, Jianyong Wu wrote:
> >> On 2020/4/21 5:57 PM, Mark Rutland wrote:
> >>> On Tue, Apr 21, 2020 at 11:23:00AM +0800, Jianyong Wu wrote:
> >>>> diff --git a/virt/kvm/arm/hypercalls.c b/virt/kvm/arm/hypercalls.c
> >>>> index 550dfa3e53cd..a5309c28d4dc 100644
> >>>> --- a/virt/kvm/arm/hypercalls.c
> >>>> +++ b/virt/kvm/arm/hypercalls.c
> >>>> @@ -62,6 +66,44 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
> >>>> if (gpa != GPA_INVALID)
> >>>> val = gpa;
> >>>> break;
> >>>> +/*
> >>>> + * This serves virtual kvm_ptp.
> >>>> + * Four values will be passed back.
> >>>> + * reg0 stores high 32-bit host ktime;
> >>>> + * reg1 stores low 32-bit host ktime;
> >>>> + * reg2 stores high 32-bit difference of host cycles and cntvoff;
> >>>> + * reg3 stores low 32-bit difference of host cycles and cntvoff.
> >>>> + */
> >>>> +case ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
> >>> Shouldn't the host opt-in to providing this to the guest, as with other
> >>> features?
> >> er, do you mean that "ARM_SMCCC_HV_PV_TIME_XXX" as "opt-in"? if so, I
> >> think this
> >>
> >> kvm_ptp doesn't need a buddy. the driver in guest will call this service
> >> in a definite way.
> > I mean that when creating the VM, userspace should be able to choose
> > whether the PTP service is provided to the guest. The host shouldn't
> > always provide it as there may be cases where doing so is undesireable.
> >
> I think I have implemented in patch 9/9 that userspace can get the info
> that if the host offers the kvm_ptp service. But for now, the host
> kernel will always offer the kvm_ptp capability in the current
> implementation. I think x86 follow the same behavior (see [1]). so I
> have not considered when and how to disable this kvm_ptp service in
> host. Do you think we should offer this opt-in?
I think taht should be opt-in, yes.
[...]
> > It's also not clear to me what notion of host time is being exposed to
> > the guest (and consequently how this would interact with time changes on
> > the host, time namespaces, etc). Having some description of that would
> > be very helpful.
>
> sorry to have not made it clear.
>
> Time will not change in host and only time in guest will change to sync
> with host. host time is the target that time in guest want to adjust to.
> guest need to get the host time then compute the different of the time
> in guest and host, so the guest can adjust the time base on the difference.
I understood that host time wouldn't change here, but what was not clear
is which notion of host time is being exposed to the guest.
e.g. is that a raw monotonic clock, or one subject to periodic adjument,
or wall time in the host? What is the epoch of the host time?
> I will add the base principle of time sync service in guest using
> kvm_ptp in commit message.
That would be great; thanks!
Mark.
On 2020/4/30 6:36 PM, Mark Rutland wrote:
> On Tue, Apr 28, 2020 at 07:14:52AM +0100, Jianyong Wu wrote:
>> On 2020/4/24 6:39 PM, Mark Rutland wrote:
>>> On Fri, Apr 24, 2020 at 03:50:22AM +0100, Jianyong Wu wrote:
>>>> On 2020/4/21 5:57 PM, Mark Rutland wrote:
>>>>> On Tue, Apr 21, 2020 at 11:23:00AM +0800, Jianyong Wu wrote:
>>>>>> diff --git a/virt/kvm/arm/hypercalls.c b/virt/kvm/arm/hypercalls.c
>>>>>> index 550dfa3e53cd..a5309c28d4dc 100644
>>>>>> --- a/virt/kvm/arm/hypercalls.c
>>>>>> +++ b/virt/kvm/arm/hypercalls.c
>>>>>> @@ -62,6 +66,44 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu)
>>>>>> if (gpa != GPA_INVALID)
>>>>>> val = gpa;
>>>>>> break;
>>>>>> +/*
>>>>>> + * This serves virtual kvm_ptp.
>>>>>> + * Four values will be passed back.
>>>>>> + * reg0 stores high 32-bit host ktime;
>>>>>> + * reg1 stores low 32-bit host ktime;
>>>>>> + * reg2 stores high 32-bit difference of host cycles and cntvoff;
>>>>>> + * reg3 stores low 32-bit difference of host cycles and cntvoff.
>>>>>> + */
>>>>>> +case ARM_SMCCC_HYP_KVM_PTP_FUNC_ID:
>>>>> Shouldn't the host opt-in to providing this to the guest, as with other
>>>>> features?
>>>> er, do you mean that "ARM_SMCCC_HV_PV_TIME_XXX" as "opt-in"? if so, I
>>>> think this
>>>>
>>>> kvm_ptp doesn't need a buddy. the driver in guest will call this service
>>>> in a definite way.
>>> I mean that when creating the VM, userspace should be able to choose
>>> whether the PTP service is provided to the guest. The host shouldn't
>>> always provide it as there may be cases where doing so is undesireable.
>>>
>> I think I have implemented in patch 9/9 that userspace can get the info
>> that if the host offers the kvm_ptp service. But for now, the host
>> kernel will always offer the kvm_ptp capability in the current
>> implementation. I think x86 follow the same behavior (see [1]). so I
>> have not considered when and how to disable this kvm_ptp service in
>> host. Do you think we should offer this opt-in?
>
> I think taht should be opt-in, yes.
ok, what about adding "CONFIG_ARM64_KVM_PTP_HOST" in kvm_ptp host
service to implement this "opt-in"?
>
> [...]
>
>>> It's also not clear to me what notion of host time is being exposed to
>>> the guest (and consequently how this would interact with time changes on
>>> the host, time namespaces, etc). Having some description of that would
>>> be very helpful.
>>
>> sorry to have not made it clear.
>>
>> Time will not change in host and only time in guest will change to sync
>> with host. host time is the target that time in guest want to adjust to.
>> guest need to get the host time then compute the different of the time
>> in guest and host, so the guest can adjust the time base on the difference.
>
> I understood that host time wouldn't change here, but what was not clear
> is which notion of host time is being exposed to the guest.
>
> e.g. is that a raw monotonic clock, or one subject to periodic adjument,
> or wall time in the host? What is the epoch of the host time?
sorry, I misunderstood your last comment.
I think it is one of the key part of kvm_ptp. I have confused with these
clock time and expect to hear the comments from experts of kernel time
subsystem like you.
IMO,kvm_ptp targets to sync time in VM with host and if all the VMs in
the same host do so, they can get time sync from each other. with no
kvm_ptp, both host and guest may affected by NTP, the time will diverge
between them. kvm_ptp can avoid this issue. if the host time vary with
something like NTP adjustment, guest will track this variation with the
help of kvm_ptp. I acquire these knowledge originally from [2]. also ptp
driver will compare the wall time(see [3]). so I think wall time clock
which subject to NTP adjustment is a good choice for kvm_ptp. in the
current implementation I get the wall time clock from ktime_get_snapshot.
I'm not sure if I give the correct knowledge of kvm_ptp and make a right
choice of host clock time. So WDYT?
[2]https://opensource.com/article/17/6/timekeeping-linux-vms
[3] see PTP_SYS_OFFSET2 in ptp_ioctl in
https://github.com/torvalds/linux/blob/master/drivers/ptp/ptp_chardev.c,
it uses ktime_get_real_ts64 as the host time to calculate the difference
between ptp time and host time.
Thanks
Jianyong
>
>> I will add the base principle of time sync service in guest using
>> kvm_ptp in commit message.
>
> That would be great; thanks!
>
> Mark.
>