Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754012AbbLKImL (ORCPT ); Fri, 11 Dec 2015 03:42:11 -0500 Received: from mx1.redhat.com ([209.132.183.28]:60316 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752579AbbLKImJ (ORCPT ); Fri, 11 Dec 2015 03:42:09 -0500 Subject: Re: [PATCH 2/5] x86, vdso, pvclock: Simplify and speed up the vdso pvclock reader To: Ingo Molnar References: <6b51dcc41f1b101f963945c5ec7093d72bdac429.1449702533.git.luto@kernel.org> <56694153.7020309@redhat.com> <20151211075249.GA12564@gmail.com> Cc: Andy Lutomirski , x86@kernel.org, Marcelo Tosatti , Radim Krcmar , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Alexander Graf , Andy Lutomirski From: Paolo Bonzini Message-ID: <566A8C5B.7090209@redhat.com> Date: Fri, 11 Dec 2015 09:42:03 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <20151211075249.GA12564@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5315 Lines: 144 On 11/12/2015 08:52, Ingo Molnar wrote: > > * Paolo Bonzini wrote: > >> >> >> On 10/12/2015 00:12, Andy Lutomirski wrote: >>> From: Andy Lutomirski >>> >>> The pvclock vdso code was too abstracted to understand easily and >>> excessively paranoid. Simplify it for a huge speedup. >>> >>> This opens the door for additional simplifications, as the vdso no >>> longer accesses the pvti for any vcpu other than vcpu 0. >>> >>> Before, vclock_gettime using kvm-clock took about 45ns on my machine. >>> With this change, it takes 29ns, which is almost as fast as the pure TSC >>> implementation. >>> >>> Signed-off-by: Andy Lutomirski >>> --- >>> arch/x86/entry/vdso/vclock_gettime.c | 81 ++++++++++++++++++++---------------- >>> 1 file changed, 46 insertions(+), 35 deletions(-) >>> >>> diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c >>> index ca94fa649251..c325ba1bdddf 100644 >>> --- a/arch/x86/entry/vdso/vclock_gettime.c >>> +++ b/arch/x86/entry/vdso/vclock_gettime.c >>> @@ -78,47 +78,58 @@ static notrace const struct pvclock_vsyscall_time_info *get_pvti(int cpu) >>> >>> static notrace cycle_t vread_pvclock(int *mode) >>> { >>> - const struct pvclock_vsyscall_time_info *pvti; >>> + const struct pvclock_vcpu_time_info *pvti = &get_pvti(0)->pvti; >>> cycle_t ret; >>> - u64 last; >>> - u32 version; >>> - u8 flags; >>> - unsigned cpu, cpu1; >>> - >>> + u64 tsc, pvti_tsc; >>> + u64 last, delta, pvti_system_time; >>> + u32 version, pvti_tsc_to_system_mul, pvti_tsc_shift; >>> >>> /* >>> - * Note: hypervisor must guarantee that: >>> - * 1. cpu ID number maps 1:1 to per-CPU pvclock time info. >>> - * 2. that per-CPU pvclock time info is updated if the >>> - * underlying CPU changes. >>> - * 3. that version is increased whenever underlying CPU >>> - * changes. >>> + * Note: The kernel and hypervisor must guarantee that cpu ID >>> + * number maps 1:1 to per-CPU pvclock time info. >>> + * >>> + * Because the hypervisor is entirely unaware of guest userspace >>> + * preemption, it cannot guarantee that per-CPU pvclock time >>> + * info is updated if the underlying CPU changes or that that >>> + * version is increased whenever underlying CPU changes. >>> * >>> + * On KVM, we are guaranteed that pvti updates for any vCPU are >>> + * atomic as seen by *all* vCPUs. This is an even stronger >>> + * guarantee than we get with a normal seqlock. >>> + * >>> + * On Xen, we don't appear to have that guarantee, but Xen still >>> + * supplies a valid seqlock using the version field. >>> + >>> + * We only do pvclock vdso timing at all if >>> + * PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to >>> + * mean that all vCPUs have matching pvti and that the TSC is >>> + * synced, so we can just look at vCPU 0's pvti. >>> */ >>> - do { >>> - cpu = __getcpu() & VGETCPU_CPU_MASK; >>> - /* TODO: We can put vcpu id into higher bits of pvti.version. >>> - * This will save a couple of cycles by getting rid of >>> - * __getcpu() calls (Gleb). >>> - */ >>> - >>> - pvti = get_pvti(cpu); >>> - >>> - version = __pvclock_read_cycles(&pvti->pvti, &ret, &flags); >>> - >>> - /* >>> - * Test we're still on the cpu as well as the version. >>> - * We could have been migrated just after the first >>> - * vgetcpu but before fetching the version, so we >>> - * wouldn't notice a version change. >>> - */ >>> - cpu1 = __getcpu() & VGETCPU_CPU_MASK; >>> - } while (unlikely(cpu != cpu1 || >>> - (pvti->pvti.version & 1) || >>> - pvti->pvti.version != version)); >>> - >>> - if (unlikely(!(flags & PVCLOCK_TSC_STABLE_BIT))) >>> + >>> + if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT))) { >>> *mode = VCLOCK_NONE; >>> + return 0; >>> + } >>> + >>> + do { >>> + version = pvti->version; >>> + >>> + /* This is also a read barrier, so we'll read version first. */ >>> + tsc = rdtsc_ordered(); >>> + >>> + pvti_tsc_to_system_mul = pvti->tsc_to_system_mul; >>> + pvti_tsc_shift = pvti->tsc_shift; >>> + pvti_system_time = pvti->system_time; >>> + pvti_tsc = pvti->tsc_timestamp; >>> + >>> + /* Make sure that the version double-check is last. */ >>> + smp_rmb(); >>> + } while (unlikely((version & 1) || version != pvti->version)); >>> + >>> + delta = tsc - pvti_tsc; >>> + ret = pvti_system_time + >>> + pvclock_scale_delta(delta, pvti_tsc_to_system_mul, >>> + pvti_tsc_shift); >>> >>> /* refer to tsc.c read_tsc() comment for rationale */ >>> last = gtod->cycle_last; >>> >> >> Reviewed-by: Paolo Bonzini > > Thanks. I've added your Reviewed-by to the 1/5 patch as well - to be able to put > the whole series into the tip:x86/entry tree. Let me know if you'd like it to be > done differently. The 1/5 patch is entirely in KVM and is not necessary for the rest of the series to work. I would like it to be separate, because Marcelo has not yet chimed in to say why it was necessary. Can you just apply patches 2-5? Paolo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/