Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
From:   Jianyong Wu <Jianyong.Wu@arm.com>
To:     Marc Zyngier <maz@kernel.org>
CC:     James Morse <James.Morse@arm.com>,
        Andre Przywara <Andre.Przywara@arm.com>,
        "lushenming@huawei.com" <lushenming@huawei.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "kvmarm@lists.cs.columbia.edu" <kvmarm@lists.cs.columbia.edu>,
        "linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Justin He <Justin.He@arm.com>
Subject: RE: [PATCH] doc/arm: take care restore order of GICR_* in ITS restore
Thread-Topic: [PATCH] doc/arm: take care restore order of GICR_* in ITS
 restore
Thread-Index: AQHXfhHId+GcJ7DQt0CbtBH7VL2wwqtNMGmAgAETNNCAAGJ4AIABY9DA
Date:   Fri, 23 Jul 2021 05:27:13 +0000
Message-ID: <AM9PR08MB7276B650897536DF9511E357F4E59@AM9PR08MB7276.eurprd08.prod.outlook.com>
References: <20210721092019.144088-1-jianyong.wu@arm.com>
        <87czrc2dsu.wl-maz@kernel.org>
        <AM9PR08MB727690189F03ED71450B8EB3F4E49@AM9PR08MB7276.eurprd08.prod.outlook.com>
 <875yx23h0p.wl-maz@kernel.org>
In-Reply-To: <875yx23h0p.wl-maz@kernel.org>
Accept-Language: en-US
Content-Language: en-US
Authentication-Results-Original: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
nodisclaimer: true
X-Microsoft-Antispam-Message-Info-Original: Qqh028PQOzY8bwxbLec4NcFjT7lsfW8+0tC7ZahsWeAjEsq0hQ0lD9jWVeV31TLszHDD9QFIbE7+wymCLCo5SBL/KrAGhD5WIZ5lmoLjxjbZiddI+v0JblHPi6mZ/W4FElnxnrA+V2VtAsVXVgFcc0DIk1hjMNZYlgJ391TJeEci/suRw+xYLl0tY+2/rUXxC9bbnuLPnV+3zrNIM6gcniFoC5d4zYPztgkQGkkzXiM5OTJ9ZPYdB95zpMkm/3HEuxvNGF61O1YG9oFfQ0+CxT81WK50bpsbVfRM511K0XMSPqBwVN6nujtmEH0Gpyv5SdpIcASsT3R/sKWTAPgZu229h6Lsyko7my1OJOlvrQPbG7VPAOMbbPwtUdZ+uWOD9uxRx6x5/XyCx6tihf+1qOnCaSm3eEIxwxFrhtIPbXtY6v90bF8hLtRBi2a1l0YL1oC4H+IYNAI3QhJGX/378V4bq7byUbImQO6DIVISm+zQ14WSy+wtUtLaUMlHzJdGcGPzBGk8P/g1ICsn/tmscPtBbKNMioiSmj0xWDq5SAwosnJ1D/+3bNK4NpnCRQIN3BNmXbuEds3rOy7kjzf5WBiOBfbGeJy6sRU679J3YnFBRdFJGwhPKbC1JkJ+RsdGiH9Mgvy/T3vfw/iNHeiDToc3QubFmXYMx5GEQhyGENktVPw+5mccgwGda/8mOYnb6H4ofA4tFcpgUaj98eM147BfF3xfp7o6vYDg4/35wxk=
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM9PR08MB7276.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(346002)(39850400004)(136003)(376002)(366004)(76116006)(316002)(7696005)(5660300002)(66556008)(64756008)(66476007)(53546011)(52536014)(9686003)(66446008)(54906003)(2906002)(186003)(71200400001)(33656002)(8936002)(6916009)(83380400001)(8676002)(55016002)(6506007)(66946007)(478600001)(122000001)(4326008)(38100700002)(86362001)(90184004)(38070700004);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?l1PYbqMpifL2B6+qQkmckpoK0CXNIdJq7v6PV4bRTXWMemSS7M+T3d8hlse6?=
 =?us-ascii?Q?jgookrTYsU601j2ZbPJ800CTTfeYvtlZV5uZ3ymkuHs116S3OvXJB4CorShI?=
 =?us-ascii?Q?acDhggPPBVM/QMX24qDWyzfX5qvgSrJr16Bvocm4OdmXVfMGqYYHAYZ+hMfN?=
 =?us-ascii?Q?3uhYQO9eKv/fWd5X9ktq0IDFbj/nNn/fESIr+8A/VmFGNuox+kCR3W7WaPf1?=
 =?us-ascii?Q?eFX6WxoQeD9Tec3ksw7i6WVCzcvBiX4+ADacEjhVM5+VePz6GDz3G/d5aPCs?=
 =?us-ascii?Q?+ffJkF1KTtT46XAWheEMquyYSISdkn4yO73in1m88miTyVTaNNRFy5Sf7YMV?=
 =?us-ascii?Q?dN/Ccrg9cCAVi0ZXMDYzHT4lVb0zJKZi1DWOExJL4EVkQ1PyqbPrAiDIjDLR?=
 =?us-ascii?Q?bgEIIoAEzBuKdjp4+LkfLkRTXUtrpBWHvSuGAoqLu+oSPI76oZgDKXxI+6Ww?=
 =?us-ascii?Q?QtUZ28UYjh6RrKfFmkkln8fALqCIIJuxvEx82m/Gzml6tOQDe9ZRG+UGpPB/?=
 =?us-ascii?Q?58o0XcGhL2LYlEUuj/ZBc9xHy/S74I+jR0gA3QRkU3oSZpe3uT6qsbuOjsNv?=
 =?us-ascii?Q?R5XP2gfSl6nWj2PFexWi6RM+bJgmMg0fMgTbmMgLgiLBKienpSdYpVfUInvV?=
 =?us-ascii?Q?MMzr4BZO608LIrHnJ3wRVZBRMhRPcPqTQzP4hbz2uYY1ttYdICheMwDygKnZ?=
 =?us-ascii?Q?u+nrq9++YerwRZ9q8Nc0Do4RmpdRM1ueJVMF7FNL5gKDi136CtLrrH8zBUdf?=
 =?us-ascii?Q?vDH0YBqpdfUVbX+fquqkEgFPQR31M8hqh3+mbgp3uO/2uhTftbrQp8Wjy8ON?=
 =?us-ascii?Q?6Dv/5YlJElJjkOyKWrFADrmCrmbnF62b8bMD+vIy6WbD7KE+pEEP1d40s9CK?=
 =?us-ascii?Q?xkxiYUr17cFzookcFwzV3ZJtvwp6x2JcI+zb9sNuW+kqdTZULwDDQseDG8vN?=
 =?us-ascii?Q?wl5syZ2lreP8NirKV7zZygmFUxhNBULP5pyHbtOoFr7Jx1Nv2Mb2+QOwOXnY?=
 =?us-ascii?Q?RZ0YKCrN3GIEtFDxMh5qkH7QhvuIOhERJyll7K0zelEf14aBqz5mO7AO3YG/?=
 =?us-ascii?Q?XBdmFzY7BCBI29QT9mOxYbiF+JxMIcAfzep5R/dQNIeLJsmeLpBXg2iFJqCC?=
 =?us-ascii?Q?30E/lxEDESPpGTobhoLi59s3tQ2b/0d4osYprs9CqP8CM7l2hH5usIlThT4D?=
 =?us-ascii?Q?dokpWCJixeQqIHpMRZE8k2gheiufV7IlJE1h+keYp+sXqCJAilxkV3fAbqMy?=
 =?us-ascii?Q?14T130amJb5C224RlAGzQkv4/Egz7SdF7Byn4MMTGY6nfPwBPZSaxJYZ26KO?=
 =?us-ascii?Q?Wr3mnCZksqIX3hX7/FwCBali4HYjoszbczbK5/ZRPDxRs4GDmyhh1RPYX+cn?=
 =?us-ascii?Q?H6vMQqsDMaIsQ4/K7zwvvEvS6Yyc?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB7276
Original-Authentication-Results: kernel.org; dkim=none (message not signed)
 header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs: ab70076a-2b33-445c-f0f4-08d94d9a875b
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: iGK3E5SP+lsVKSbhL4f6YZc7SM9xmCLchLl7SAygZ1f7VK1MnpH1MoPgUh+RsubbZ8nIfxw/Yh3/hZgL+DXWxtLBV+iW91T486+zrbavazTIeGcJK02NnDhFJvdj9EcGMTC18/2aqTZN+iQ66byUWxaEz3ykUVome+COCb7NoxnnS8XacFLGErGJNdR2cvFHpXsnFiLhxM8vOBUaiJSWAoTc61JYlFlGhB1ixaQxAFZlZPYXfVfghxDdH0cgyFz4RP/ejlaMbXY5akg67nyggObRqTu0mnIx/U6EZYbQLqHyFxWjkAynQCCnqip94cUzbmogoHYtHxeh+gS9eZndrZaJmKzeg3u4GoR70BjULlJ/lqqXqg9xTyjKeFLivOocVG2yBTlRvnc64T7pUV/rp9T9JCj7lblmc/Z6rfdUlvIZQS/5KWoHwIzyUpJmdGK1wM2LbgTyrLMmdZ6Pc2eh+PWxDvrpSsFc2iVTFoL5AUv6kC2lj7q5SHrrdfbSS4cDhauKMFJ9Qfsy8eEj5RQI1n/CsXgiYRKk7tKpSFGj0tRSrdQpYdn2Yoz4bp5HF4t2TNf6vXLme9gbserLlmLlx27Sslzs8EHuV3Wa5SRbSmZ6r6JajfaeE2/TOPKe5AdOqkoxqxO7MgRtaC24byy4Y+IyYCZHRS/hxOziWs6HE1hcNUL/6zwiA5/Cw5lepmKNcaQVODvgaVcvR+ZpSHx1Rxo4GlSO8cFC/IvdKaWKirI=
X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(4636009)(39850400004)(136003)(376002)(346002)(396003)(36840700001)(46966006)(356005)(8676002)(81166007)(7696005)(33656002)(450100002)(52536014)(70206006)(2906002)(8936002)(83380400001)(9686003)(316002)(55016002)(478600001)(5660300002)(47076005)(82310400003)(86362001)(54906003)(336012)(70586007)(36860700001)(4326008)(82740400003)(26005)(6506007)(186003)(53546011)(6862004)(90184004);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jul 2021 05:27:28.1235
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 5cdf645f-86f9-47a0-46bd-08d94d9a8fc8
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT062.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2539
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Marc,

Get it! Thanks for your explanation.

Thanks
Jianyong

> -----Original Message-----
> From: Marc Zyngier <maz@kernel.org>
> Sent: Thursday, July 22, 2021 4:11 PM
> To: Jianyong Wu <Jianyong.Wu@arm.com>
> Cc: James Morse <James.Morse@arm.com>; Andre Przywara
> <Andre.Przywara@arm.com>; lushenming@huawei.com;
> kvm@vger.kernel.org; kvmarm@lists.cs.columbia.edu; linux-
> doc@vger.kernel.org; linux-kernel@vger.kernel.org; Justin He
> <Justin.He@arm.com>
> Subject: Re: [PATCH] doc/arm: take care restore order of GICR_* in ITS
> restore
>
> On Thu, 22 Jul 2021 03:49:52 +0100,
> Jianyong Wu <Jianyong.Wu@arm.com> wrote:
> >
> > Hello Marc,
> >
> > > -----Original Message-----
> > > From: Marc Zyngier <maz@kernel.org>
> > > Sent: Wednesday, July 21, 2021 5:54 PM
> > > To: Jianyong Wu <Jianyong.Wu@arm.com>
> > > Cc: James Morse <James.Morse@arm.com>; Andre Przywara
> > > <Andre.Przywara@arm.com>; lushenming@huawei.com;
> > > kvm@vger.kernel.org; kvmarm@lists.cs.columbia.edu; linux-
> > > doc@vger.kernel.org; linux-kernel@vger.kernel.org; Justin He
> > > <Justin.He@arm.com>
> > > Subject: Re: [PATCH] doc/arm: take care restore order of GICR_* in
> > > ITS restore
> > >
> > > On Wed, 21 Jul 2021 10:20:19 +0100,
> > > Jianyong Wu <jianyong.wu@arm.com> wrote:
> > > >
> > > > When restore GIC/ITS, GICR_CTLR must be restored after
> > > > GICR_PROPBASER and GICR_PENDBASER. That is important, as both of
> > > > GICR_PROPBASER and GICR_PENDBASER will fail to be loaded when lpi
> > > > has enabled yet in GICR_CTLR. Keep the restore order above will avo=
id
> that issue.
> > > > Shout it out at the doc is very helpful that may avoid lots of debu=
g work.
> > >
> > > But that's something that is already mandated by the architecture, is=
n't it?
> > > See "5.1 LPIs" in the architecture spec:
> > >
> > > <quote>
> > >
> > > If GICR_PROPBASER is updated when GICR_CTLR.EnableLPIs =3D=3D 1, the
> > > effects are UNPREDICTABLE.
> > >
> > > [...]
> > >
> > > If GICR_PENDBASER is updated when GICR_CTLR.EnableLPIs =3D=3D 1, the
> > > effects are UNPREDICTABLE.
> > >
> >
> > I think this "UNPREDICTABLE" related with the "physical machine". Am I
> > right?
>
> No, you are unfortunately wrong. The architecture applies to *any*
> implementation, and makes no distinction between a HW implementation of
> a SW version. This is why we call it an architecture, and not an
> implementation.
>
> > In virtualization environment, kernel gives the definite answer that
> > we should not enable GICR_CTLR.EnableLPIs before restoring
> > GICR_PROPBASER(GICR_PENDBASER either) when restore GIC ITS in VMM,
> see
> > [1]. Thus, should we consider the virtualization environment as a
> > special case?
>
> Absolutely not.  If you start special casing things, then we end-up havin=
g
> stupidly designed SW that tries to do stupid things based on the supposed
> properties of an implementation.
>
> We follow the architecture, all the architecture, nothing but the archite=
cture.
> The architecture is the only barrier between insanity and pure madness! ;=
-)
>
> >
> > [1] linux/arch/arm64/kvm/vgic/vgic-mmio-v3.c
> > static void vgic_mmio_write_propbase(struct kvm_vcpu *vcpu,
> >                                      gpa_t addr, unsigned int len,
> >                                      unsigned long val) {
> >         struct vgic_dist *dist =3D &vcpu->kvm->arch.vgic;
> >         struct vgic_cpu *vgic_cpu =3D &vcpu->arch.vgic_cpu;
> >         u64 old_propbaser, propbaser;
> >
> >         /* Storing a value with LPIs already enabled is undefined */
> >         if (vgic_cpu->lpis_enabled)
> >            return;
> > ...
> > }
>
> Do you see how the kernel does exactly what the architecture says we can =
do?
> Ignoring the write is a perfectly valid implementation of UNPREDICTABLE.
>
> So what we do is completely in line with the architecture. As such, no ne=
ed to
> document it any further, everything is already where it should be. If
> someone tries to write code dealing with the GIC without understanding th=
e
> architecture, no amount of additional documentation will help.
>
> Thanks,
>
>       M.
>
> --
> Without deviation from the norm, progress is not possible.
IMPORTANT NOTICE: The contents of this email and any attachments are confid=
ential and may also be privileged. If you are not the intended recipient, p=
lease notify the sender immediately and do not disclose the contents to any=
 other person, use it for any purpose, or store or copy the information in =
any medium. Thank you.