Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <dd53c419-efc4-5b2f-9fdc-e23e7145dbd1@intel.com>
Date:   Mon, 16 May 2022 10:20:42 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.8.1
Subject: Re: [PATCH V2 03/11] perf/x86: Add support for TSC in nanoseconds as
 a perf event clock
Content-Language: en-US
To:     Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>
Cc:     Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Jiri Olsa <jolsa@redhat.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        "x86@kernel.org" <x86@kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        H Peter Anvin <hpa@zytor.com>,
        Mathieu Poirier <mathieu.poirier@linaro.org>,
        Suzuki K Poulose <suzuki.poulose@arm.com>,
        Leo Yan <leo.yan@linaro.org>,
        "jgross@suse.com" <jgross@suse.com>,
        "sdeep@vmware.com" <sdeep@vmware.com>,
        "pv-drivers@vmware.com" <pv-drivers@vmware.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "seanjc@google.com" <seanjc@google.com>,
        "kys@microsoft.com" <kys@microsoft.com>,
        "sthemmin@microsoft.com" <sthemmin@microsoft.com>,
        "virtualization@lists.linux-foundation.org" 
        <virtualization@lists.linux-foundation.org>,
        "Andrew.Cooper3@citrix.com" <Andrew.Cooper3@citrix.com>,
        "Hall, Christopher S" <christopher.s.hall@intel.com>
References: <20220214110914.268126-1-adrian.hunter@intel.com>
 <20220214110914.268126-4-adrian.hunter@intel.com>
 <YiIXFmA4vpcTSk2L@hirez.programming.kicks-ass.net>
 <853ce127-25f0-d0fe-1d8f-0b0dd4f3ce71@intel.com>
 <YiXVgEk/1UClkygX@hirez.programming.kicks-ass.net>
 <30383f92-59cb-2875-1e1b-ff1a0eacd235@intel.com>
 <YiYZv+LOmjzi5wcm@hirez.programming.kicks-ass.net>
 <013b5425-2a60-e4d4-b846-444a576f2b28@intel.com>
 <6f07a7d4e1ad4440bf6c502c8cb6c2ed@intel.com>
 <c3e1842b-79c3-634a-3121-938b5160ca4c@intel.com>
 <50fd2671-6070-0eba-ea68-9df9b79ccac3@intel.com> <87ilqx33vk.ffs@tglx>
 <ff1e190a-95e6-e2a6-dc01-a46f7ffd2162@intel.com> <87fsm114ax.ffs@tglx>
 <c8033229-97a0-3e4c-66d5-74c0d1d4e15c@intel.com> <87ee1iw2ao.ffs@tglx>
From:   Adrian Hunter <adrian.hunter@intel.com>
Organization: Intel Finland Oy, Registered Address: PL 281, 00181 Helsinki,
 Business Identity Code: 0357606 - 4, Domiciled in Helsinki
In-Reply-To: <87ee1iw2ao.ffs@tglx>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 28/04/22 02:10, Thomas Gleixner wrote:
> On Tue, Apr 26 2022 at 09:51, Adrian Hunter wrote:
>> On 25/04/22 20:05, Thomas Gleixner wrote:
>>> On Mon, Apr 25 2022 at 16:15, Adrian Hunter wrote:
>>>> On 25/04/22 12:32, Thomas Gleixner wrote:
>>>>> It's hillarious, that we still cling to this pvclock abomination, while
>>>>> we happily expose TSC deadline timer to the guest. TSC virt scaling was
>>>>> implemented in hardware for a reason.
>>>>
>>>> So you are talking about changing VMX TCS Offset on every VM-Entry to try to hide
>>>> the time jumps when the VM is scheduled out?  Or neglect that and just let the time
>>>> jumps happen?
>>>>
>>>> If changing VMX TCS Offset, how can TSC be kept consistent between each VCPU i.e.
>>>> wouldn't that mean each VCPU has to have the same VMX TSC Offset?
>>>
>>> Obviously so. That's the only thing which makes sense, no?
>>
>> [ Sending this again, because I notice I messed up the email "From" ]
>>
>> But wouldn't that mean changing all the VCPUs VMX TSC Offset at the same time,
>> which means when none are currently executing?  How could that be done?
> 
> Why would you change TSC offset after the point where a VM is started
> and why would it be different per vCPU?
> 
> Time is global and time moves on when a vCPU is scheduled out. Anything
> else is bonkers, really. If the hypervisor tries to screw with that then
> how does the guest do timekeeping in a consistent way?
> 
>     CLOCK_REALTIME = CLOCK_MONOTONIC + offset
> 
> That offset changes when something sets the clock, i.e. clock_settime(),
> settimeofday() or adjtimex() in case that NTP cannot compensate or for
> the beloved leap seconds adjustment. At any other time the offset is
> constant.
> 
> CLOCK_MONOTONIC is derived from the underlying clocksource which is
> expected to increment with constant frequency and that has to be
> consistent accross _all_ vCPUs of a particular VM.
> 
> So how would a hypervisor 'hide' scheduled out time w/o screwing up
> timekeeping completely?
> 
> The guest TSC which is based on the host TSC is:
> 
>     guestTSC = offset + hostTSC * factor;
> 
> If you make offset different between guest vCPUs then timekeeping in the
> guest is screwed.
> 
> The whole point of that paravirt clock was to handle migration between
> hosts which did not have the VMCS TSC scaling/offset mechanism. The CPUs
> which did not have that went EOL at least 10 years ago.
> 
> So what are you concerned about?

Thanks for the explanation.

Changing TSC offset / scaling makes it much harder for Intel PT on
the host to use, so there is no sense in my pushing for that at this
time when there is anyway kernel option no-kvmclock.