Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1;
Message-ID: <38722ba7-b7d9-368b-f946-b6c0c0a661b8@os.amperecomputing.com>
Date:   Tue, 19 Sep 2023 11:45:44 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: [PATCH v2 2/2] KVM: arm64: timers: Save restore CVAL of a ptimer
 across guest entry and exits
To:     Marc Zyngier <maz@kernel.org>
Cc:     linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev,
        linux-arm-kernel@lists.infradead.org, Christoffer.Dall@arm.com,
        eauger@redhat.com, miguel.luis@oracle.com,
        darren@os.amperecomputing.com, scott@os.amperecomputing.com
References: <20230904114218.590304-1-gankulkarni@os.amperecomputing.com>
 <20230904114218.590304-3-gankulkarni@os.amperecomputing.com>
 <86bkeadzf0.wl-maz@kernel.org>
 <ec322123-bfe7-6019-7f35-de326ee7b6c3@os.amperecomputing.com>
 <8634zben3d.wl-maz@kernel.org>
Content-Language: en-US
From:   Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>
In-Reply-To: <8634zben3d.wl-maz@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eGtaOTNMQk1hbHZySDAyc3NTQjFDbEs1YWVvMC9xRW02Y1k2VmVHVnk0dk9r?=
 =?utf-8?B?MS8wZVhaY0QxYms2cXlWUHFGSG8rVkswOWV5QlB4RGVyUDUvQ3RYNHNQVVhM?=
 =?utf-8?B?aE9rcWE0ZUl0TWRlZ3ViWDBwc2dxaTFOWkJoL1FmRGhDQzVTQzhaVnlSWFJp?=
 =?utf-8?B?d2lhVTdvTEF1UGJMWmxQZkNIYlk5eVQ1VjhvNERsV05KNmlWdlBlczk1ZGcv?=
 =?utf-8?B?U05Cc0lSdWxOOE1DNnVuR0hpY3NHdkNXSVQ4M1poL2huWE1iVENJbklFLzVK?=
 =?utf-8?B?MXdFR3Z0L2VtWnpVVUwvdEN6T1R4Q3BoYXh5azBoSnp1V1hJWGo1RXQxTmph?=
 =?utf-8?B?MjFycC8zT1R5bUgwUitUY1ZubVdGL1RHTlgyUmJsUUdKSXd0bGYwKzlsUEs0?=
 =?utf-8?B?OWZUZ3RiQ3hkOXZvZXlENnlnaEN4ZnljMDd0UnBtc3lYWWhjdHl5QTJ4QTRL?=
 =?utf-8?B?Tmx6MkQzWlhmQ2lsT1V3aElKOVhjbXJabWkvY1AyNTE1cy9GcmcvRFBjZXRZ?=
 =?utf-8?B?S2RBMGdRQlk2VWwvTmZtcjlQbEh6andndHFrZ2RXaXZhVnhCUXF0eWw5bi96?=
 =?utf-8?B?Zk50VFlFL1FBQXBabGxxWXF5NnoyMW1jMldIekppeHJ0TStoRk5MeGdCNjdF?=
 =?utf-8?B?WSs2eHl1cm9tQ2RVdXhidzNrYjM4c2FPZTR4Q1lPeHFmSGMzcDJveU45S21q?=
 =?utf-8?B?RHpoOUtZeGxTaUFTQVgybkRnTUF1bFFzcHBjaEIvOXp1cjZDYzVHYWlSL1lF?=
 =?utf-8?B?OW1aUnh6d1pyYlN2TEpTeHJwd3VhV1RXRFJ0UGlEV3YyT1pYMmc5NWxSZ0pm?=
 =?utf-8?B?QnVaZW5xNjZWOGF0dkJIMjAzN1JUYU5mTGF5d3N1RDJtQzMzNmh3SXRQclNN?=
 =?utf-8?B?RUZuSDAzVlJ3MEQvN1JzWGwwemMzL0JMcWU3eEwxRkt4TXlFS0YwdnVlMENu?=
 =?utf-8?B?QlIvVXNoOEd6WHNtaVFHYVl6Ryt5aVNhSlE2WXVMZXluOVBFZU1Jb1lUeURB?=
 =?utf-8?B?TTZTTVNhTlFmWVZXWHowZ1FFT3lYK3JVWjAvcXpyaDRQeHMrclRtVkNDaHFr?=
 =?utf-8?B?L1c4bUQ1R1V3eTlJZDhTYThLY2h2ZEd6QVE3RVdSMjBFeG5TeE44Z0FPT3l3?=
 =?utf-8?B?blVSNlphLzQ4bEMwZCt2akxRbjEvWDZOejhqcllMcFkwZXpVRWpzblpURHhm?=
 =?utf-8?B?NmJaTk5Ncnc0dG1QU3BydnhBOGZIcjhIZEpYR1FKcU84OVVMNFhSbm85djls?=
 =?utf-8?B?SkVJbG9TeDRxcHV1ZWxJblhBTDR4MXIzMlNsRzUrcFpZNjNZMGZxS2xEVlJw?=
 =?utf-8?B?TkMzdVpzNGtjenlVYzFyY2t5MDNTR0lsUFg1ZVdqYXZ4L1NKUTZ1THZiWU00?=
 =?utf-8?B?MnV0elY3cEJFLzg4QVJoZFc5UHdBV212amRkRmJJV1hYODBvWDVybkl5OUxx?=
 =?utf-8?B?d3pvcXpTVG1mMnUxWUpmU2xFSndUa2JOSlRjWkF3Z0VrRk8wdXYwb0RYNVpF?=
 =?utf-8?B?RzZNRGlSY3RJK20yU1ZVUXlzbnoyVVpYRDdBNjVhb0VFakNmUFppRmRHckkw?=
 =?utf-8?B?d1EvVXd0UGhxUFFEK0tOWVBwQWkzcGsxNm5qbHZDcFVKU2tYMUJOZHd4bWZz?=
 =?utf-8?B?RHRhck0vQkZKemw2YTN5R0xzT2xOV2I4M25OZEVSQ0pPNWl1TUxLSzdkVUM2?=
 =?utf-8?B?a0ZIb1l0dmEvN01xNTZkd0VLMnhTQzV5RE0zT2xidEk0bFRZU3ZoZWViRGpW?=
 =?utf-8?B?Z0dyM3dnT0RreVRya2lWT0wzVVdmQkFLSm9yVjd6YjFMY2xRWng0T2pOdDhI?=
 =?utf-8?B?bEpWQzArR0hJMkY5bmJWQ21pWktYVmRRdnVaTDVZRXNsUllKUFlIQm1KQzI5?=
 =?utf-8?B?bzVSejc5ekJaVEs0ZTk3Znd1OE9XN1FnWmVZYkNvbXd0cE4vRGo2QUMzZTl6?=
 =?utf-8?B?cVBVVHQxb05QeGFxOXF4S01WN3FXVXVpdjdNOGZwamV4LzFyeVVkMzZtTEpK?=
 =?utf-8?B?dXc2YXVXT24vTmJOZUF4eG9KUjVMVTYveit2YUlzSERQeGlCcmgxM05wUkUr?=
 =?utf-8?B?TmR6cTNncjY4V3pKaWh0RnIvNXFTRzJ0MzJuWkR3aU5MWVB4anp3TkFKZVFI?=
 =?utf-8?B?Mk5MeFRVYlIvVHEzam5EUU12TUc2bzR0Z0JvWVFMdWFOMnltdGlxeEN1NWtm?=
 =?utf-8?Q?thxQ4Z02p7z3o1mXJI07mZU=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c385e5b4-3116-4cc3-a38b-08dbb8d7e151
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR01MB8101.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2023 06:15:54.4732
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ERqoKR8TRaZYhNMgvLmWYruBaDUsHYUsumGgEfvYD7z7GiyvCbL0MWjH8SzN0ZJCRsC6YGb6jb3hLNptobF66leUMs417aeVCntsTX0ucXeSE9CbW6Z3/mMMp4dvOTjh
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR01MB6253
X-Spam-Status: No, score=-2.3 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Mon, 18 Sep 2023 23:16:21 -0700 (PDT)


On 18-09-2023 04:59 pm, Marc Zyngier wrote:
> On Fri, 15 Sep 2023 10:57:46 +0100,
> Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com> wrote:
>>
>> This patch did not work.
>> After adding changes as in below diff, it is started working.
> 
> Thanks for looking into this.
> 
>>
>> diff --git a/arch/arm64/kvm/hyp/vhe/switch.c
>> b/arch/arm64/kvm/hyp/vhe/switch.c
>> index b0b07658f77d..91d2cfb03e26 100644
>> --- a/arch/arm64/kvm/hyp/vhe/switch.c
>> +++ b/arch/arm64/kvm/hyp/vhe/switch.c
>> @@ -117,7 +117,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
>>                          val = __vcpu_sys_reg(vcpu, CNTHP_CVAL_EL2);
>>
>>                  if (map.direct_ptimer) {
>> -                       write_sysreg_s(val, SYS_CNTP_CVAL_EL0);
>> +                       write_sysreg_el0(val, SYS_CNTP_CVAL);
> 
> Duh, of course. Silly me.
> 
>>                          isb();
>>                  }
>>          }
>> @@ -161,8 +161,6 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)
>>
>>          ___deactivate_traps(vcpu);
>>
>> -       write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
>> -
>>          if (has_cntpoff()) {
>>                  struct timer_map map;
>>                  u64 val, offset;
>> @@ -173,7 +171,7 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)
>>                   * We're exiting the guest. Save the latest CVAL value
>>                   * to memory and apply the offset now that TGE is set.
>>                   */
>> -               val = read_sysreg_s(SYS_CNTP_CVAL_EL0);
>> +               val = read_sysreg_el0(SYS_CNTP_CVAL);
>>                  if (map.direct_ptimer == vcpu_ptimer(vcpu))
>>                          __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) = val;
>>                  if (map.direct_ptimer == vcpu_hptimer(vcpu))
>> @@ -182,12 +180,13 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu)
>>                  offset = read_sysreg_s(SYS_CNTPOFF_EL2);
>>
>>                  if (map.direct_ptimer && offset) {
>> -                       offset = read_sysreg_s(SYS_CNTPOFF_EL2);
>> -                       write_sysreg_s(val + offset, SYS_CNTP_CVAL_EL0);
>> +                       write_sysreg_el0(val + offset, SYS_CNTP_CVAL);
>>                          isb();
>>                  }
>>          }
>>
>> +       write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
> 
> Why moving the HCR_EL2 update? I don't grok what it changes. Or is it

This the line of code which flips the TGE and making timer cval ready to 
handle the TGE flip is more safe way(avoids even corner case of false 
interrupt triggers) than changing after the flipping?

> that you end-up with spurious interrupts because your GIC is slow to
> retire interrupts that are transiently pending?

IIUC, If there are any transient interrupts or asserted already, anyway 
they will be handled when irq is unmasked.

> 
> Thanks,
> 
> 	M.
> 

Thanks,
Ganapat