Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH] KVM: arm/arm64: vgic: Handle GICR_PENDBASER.PTZ filed as
 RAZ
To:     Zenghui Yu <yuzenghui@huawei.com>, Marc Zyngier <maz@kernel.org>
Cc:     andre.przywara@arm.com, linux-kernel@vger.kernel.org,
        wanghaibin.wang@huawei.com, kvmarm@lists.cs.columbia.edu,
        linux-arm-kernel@lists.infradead.org
References: <20191220111833.1422-1-yuzenghui@huawei.com>
 <3a729559-d0eb-e042-d6bd-b69bacb0dd23@huawei.com>
 <c084aa29c029f97cdfb1b5dc9e6b29ac@www.loen.fr>
 <1225d839-3cf7-d513-778e-698e12e94875@huawei.com>
From:   Auger Eric <eric.auger@redhat.com>
Message-ID: <12a1e25b-617d-6b04-6a5a-4c67a39565a5@redhat.com>
Date:   Tue, 24 Dec 2019 05:45:19 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <1225d839-3cf7-d513-778e-698e12e94875@huawei.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Zenghui,

On 12/24/19 3:52 AM, Zenghui Yu wrote:
> Hi Marc, Eric,
>=20
> On 2019/12/23 22:07, Marc Zyngier wrote:
>> Hi Zenghui,
>>
>> On 2019-12-23 13:43, Zenghui Yu wrote:
>>> On 2019/12/20 19:18, Zenghui Yu wrote:
>>>> Although guest will hardly read and use the PTZ (Pending Table Zero)
>>>> bit in GICR_PENDBASER, let us emulate the architecture strictly.
>>>> As per IHI 0069E 9.11.30, PTZ field is WO, and reads as 0.
>>>> Signed-off-by: Zenghui Yu <yuzenghui@huawei.com>
>>>> ---
>>>> Noticed when checking all fields of GICR_PENDBASER register.
>>>> But _not_ sure whether it's worth a fix, as Linux never sets
>>>> the PTZ bit before enabling LPI (set GICR_CTLR_ENABLE_LPIS).
>>>> And I wonder under which scenarios can this bit be written as 1.
>>>> It seems difficult for software to determine whether the pending
>>>> table contains all zeros when writing this bit.
>>>> virt/kvm/arm/vgic/vgic-mmio-v3.c | 5 ++++-
>>>> =C2=A0 1 file changed, 4 insertions(+), 1 deletion(-)
>>>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> index 7dfd15dbb308..ebc218840fc2 100644
>>>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>>>> @@ -414,8 +414,11 @@ static unsigned long
>>>> vgic_mmio_read_pendbase(struct kvm_vcpu *vcpu,
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 gpa_t addr, unsigned int len)
>>>> =C2=A0 {
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct vgic_cpu *vgic_cpu =3D &vcpu->=
arch.vgic_cpu;
>>>> +=C2=A0=C2=A0=C2=A0 u64 value =3D vgic_cpu->pendbaser;
>>>> =C2=A0 -=C2=A0=C2=A0=C2=A0 return extract_bytes(vgic_cpu->pendbaser,=
 addr & 7, len);
>>>> +=C2=A0=C2=A0=C2=A0 value &=3D ~GICR_PENDBASER_PTZ;
>>>> +
>>>> +=C2=A0=C2=A0=C2=A0 return extract_bytes(value, addr & 7, len);
>>>> =C2=A0 }
>>>> =C2=A0 static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
>>>>
>>>
>>> I noticed there is no userspace access callbacks for GICR_PENDBASER,
>>> so this patch will make the PTZ field also 'Read As Zero' by userspac=
e.
>>> Should we consider adding a uaccess_read callback for GICR_PENDBASER
>>> which just returns the unchanged vgic_cpu->pendbaser to userspace?
>>> (Though this is really not a big deal. We now always emulate the PTZ
>>> field to guest as RAZ. And 'vgic_cpu->pendbaser & GICR_PENDBASER_PTZ'
>>> only indicates whether KVM will optimize the LPI enabling process,
>>> where Read As Zero indicates never optimize..)
>>
>> I don't think adding a userspace accessor would help much. All this
>> bit tells userspace is that the guest has programmed a zero filled
>> table. On restore, we'd avoid a rescan of the table if there was
>> no LPI mapped.
>=20
> Yes, I agree.
>=20
>> And thinking of it, this fixes a bug for non-Linux guests: If you writ=
e
>> PTZ=3D1, we never clear it. Which means that if userspace saves and
>> restores
>> PENDBASER with PTZ set, we'll never restore the pending bits, which is
>> pretty bad (see vgic_enable_lpis()).
>=20
> But I'm afraid I can't follow this point. After reading the code (with
> Qemu) a bit further, the Redistributors are restored before the ITS.

This is also part of the kernel documentation:
Documentation/virt/kvm/devices/arm-vgic-its.txt (ITS restore sequence)
 So
> there should be _no_ LPI has been mapped when we're restoring GICR_CTLR
> and enabling LPI, which says we will not scan the whole pending table
> and restore pending by vgic_enable_lpis()/its_sync_lpi_pending_table(),
> regardless of what the PTZ is.
>=20
> Instead, vgic_its_restore_ite()/vgic_v3_lpi_sync_pending_status() is
> where we actually read the guest RAM and restore the LPI pending state.
yes the pending state is restored from
vgic_its_restore_ite/vgic_add_lpi/vgic_v3_lpi_sync_pending_status and
this path ignores the PTZ.

Thanks

Eric
> Which means we will still do the right thing even for non-Linux guests.
> Not sure if I've got things correctly here.
>=20
> In the end, let's keep the patch as it is.
>=20
>>
>> This patch on its own fixes more than one bug!
>>
>=20
> If so, just by luck ;-)
>=20
>=20
> Thanks,
> Zenghui
>=20
>=20
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel