Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Li, Xin3" <xin3.li@intel.com>
To:     Peter Zijlstra <peterz@infradead.org>,
        "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "hpa@zytor.com" <hpa@zytor.com>,
        "Christopherson,, Sean" <seanjc@google.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "Shankar, Ravi V" <ravi.v.shankar@intel.com>
Subject: RE: [RFC PATCH 22/32] x86/fred: FRED initialization code
Thread-Topic: [RFC PATCH 22/32] x86/fred: FRED initialization code
Thread-Index: AQHZFEEuSpGeNAxXE02epndn3lbDCa52hwiAgAAC2YCAAAH1gIABQ0VA
Date:   Wed, 21 Dec 2022 05:28:52 +0000
Message-ID: <BN6PR1101MB2161C51C6068026D9C442460A8EB9@BN6PR1101MB2161.namprd11.prod.outlook.com>
References: <20221220063658.19271-1-xin3.li@intel.com>
 <20221220063658.19271-23-xin3.li@intel.com>
 <Y6GELyEJeKY3dEqJ@hirez.programming.kicks-ass.net>
 <16972e64-7d7b-ad8c-f8dc-6dcab69e629e@citrix.com>
 <Y6GIN5Uf7Qd43A9U@hirez.programming.kicks-ass.net>
In-Reply-To: <Y6GIN5Uf7Qd43A9U@hirez.programming.kicks-ass.net>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?B652j4Ammo0w23NGj/BOqCC/SoT6jqt4uisbHO8D9+udj6sPZ05pRXLx5V?=
 =?iso-8859-1?Q?GIj7PX8Divp/ECiAQpynqGv+v4CJIlESzRJa3Cd+ajA/BwsvHUDeHG7DD6?=
 =?iso-8859-1?Q?ZKAGCym8lFjmKN6Zi1REzfhmAgaMuSYt/iqZD6gCVbl8r/+e0GfA4pV9eM?=
 =?iso-8859-1?Q?ov2Dy0GxDomqPXmhiwq+b3wGw0ww/NPUD6KK+hPd1VX64eYjGNA5DYifoz?=
 =?iso-8859-1?Q?+FJGgurtbRxhD7rrK0KzmCSgrdeldH5n3gVTMDTxbWT1wBrRQW/tZ3rLO1?=
 =?iso-8859-1?Q?H9SMQtWuEV/RTB22O7dNPfMFPueYgFu4DTc1ph+1aBfoSzBAaeRFrWMBGF?=
 =?iso-8859-1?Q?PDyv5GbBxMsiTN+oBPdUwneH4CglT3N/a1FHtGaCS519c+L9uVNb5z1Ooz?=
 =?iso-8859-1?Q?B0PRHDC534uBEr2kudTBM3cip+o4bJ/5emXrdp861ssuvyUMUCzSwRhEnz?=
 =?iso-8859-1?Q?HFkUVz18ytfBgFzCXdM0ne64vNnsWcezi8CuqIyCoGWRZNAu9xEF1eYxYO?=
 =?iso-8859-1?Q?ckoOgBtUbDimxC/q9Fif0M852p9wGp4j9edYO5TQlgYoKaMkgIAE/IlXkp?=
 =?iso-8859-1?Q?ZHSF6AQk8JLncaos0BiEz0JUibtoY5NARMQEO1y9yHbmSsMjZ8JiGp7u2v?=
 =?iso-8859-1?Q?1DubLuN5M/SncEVOEevA0sz6Wqcpk9IkYFk1oNH5KnfOL3WfWakuPECwbT?=
 =?iso-8859-1?Q?JlH36I5AeNmw+shYZqnS0V7Xu3PCVZBk+26fyRUZZw3EuE+uKegWXdq5XZ?=
 =?iso-8859-1?Q?0mtdkC0kDkORm2XbT4DXU94hWPzt+h0PWx75rLuC0KuTV1ZJWe46tNiBEZ?=
 =?iso-8859-1?Q?p1yGoP0Z/f4+qF6g1YG04EJ8JgHALu4ec2IqsQcvk8qu1yYZAcOU8xNg+2?=
 =?iso-8859-1?Q?ZZBJMWZ6ntLKt6jLQbwcC2ioYkAO2m4FKmL4n9hpKfesqMgKNZM/LQBWgf?=
 =?iso-8859-1?Q?wjCAGRcaqK5K+166Dss6ka7r+D5AWHhzovhLf+HpQDUGvUjRwjpvA8L2vd?=
 =?iso-8859-1?Q?CD59D8MbNAjgLxd4dhKOfmhhi0Y4jcu0PK/yVgp4nBQBzgoxv9sRbJXo44?=
 =?iso-8859-1?Q?A0WgY30fv3u4WlZ4sRWpfD3Yl4LYVhZRH3LRCR2YDCIjDMkHHLo4k5lIOt?=
 =?iso-8859-1?Q?v2fJI4HmG6BXQ8IBuRB3yh7m1uzbuX6yFVzXlm788POm+1oZo1GRIXlFnc?=
 =?iso-8859-1?Q?RrE6J5vy/HUni0+gF/SRRh3JcyPka8JuSmCicfG8qrTSKZPWJJS5NYKxRH?=
 =?iso-8859-1?Q?669grlwFMs+K/5o0nir8A4794KpZiXIEQptQsJksTgHYQdoxi299BHPZ+G?=
 =?iso-8859-1?Q?d6zVTyqBYtnqoRwplbSOGJYTSbMPFcuGQVn+aBXSdWPFUu6FB3faKx8Uf6?=
 =?iso-8859-1?Q?iMfKGU7kVQola73P2/m8Ku8CCK+a9zl//SbTcTPmWAoGNeX5vYzc+f3KC8?=
 =?iso-8859-1?Q?1Xh0z9UKsXvy1+j3fbVYXYZITYHNQyDWkm9EyzsLLE4j9dcJsK/GNd+Qb9?=
 =?iso-8859-1?Q?SkRZOSThGaokAcx2ir5FYcgGCJjfpoAi0wOFTqf4Yct3X6ijx0qAxZZSPK?=
 =?iso-8859-1?Q?TE8ad39Bl49gv6CJnWBcUDClV7HJbYTURBuTlYSzCRP6ro3qfyxzvZhUho?=
 =?iso-8859-1?Q?UtOQFUG75P0wwff+PLIw+yPtYcU3cqrKLn?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN6PR1101MB2161.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b0e36786-5c71-419a-39d4-08dae3143f08
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Dec 2022 05:28:52.2076
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dI6UoIK90K/FaX9bbpCkYw8SeFcMVG+WG/5RI/aaSXikDYmrURIqOXpTyo8MSiGCqj8PPAAvFc9wQ4HrOPG+rA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4993
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> > >> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
> > >> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
> > >> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
> > >> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
> > >> +	       FRED_STKLVL(X86_TRAP_DF,  3));
> > >> +
> > >> +	/* The FRED equivalents to IST stacks... */
> > >> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
> > >> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
> > >> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
> > > Not quite.. IIRC fred only switches to another stack when the level
> > > of the exception is higher. Specifically, if we trigger #DB while
> > > inside #NMI we will not switch to the #DB stack (since 1 < 2).

Yes, current stack level can only grow higher.

> >
> > There needs to be a new stack for #DF, and just possibly one for #MC.
> > NMI and #DB do not need separate stacks under FRED.
>=20
> True, there is very little need to use additional stacks with FRED.

Pretty much.

#DB/NMI from a ring 3 context uses CSL 0, and their CSLs increase only
when happening from a ring 0 context.

>=20
> > > Now, as mentioned elsewhere, it all nests a lot saner, but stack
> > > exhaustion is still a thing, given the above, what happens when a
> > > #DB hits an #NMI which tickles a #VE or something?
> > >
> > > I don't think we've increased the exception stack size, but perhaps
> > > we should for FRED?
> >
> > Not sure if it matters too much - it doesn't seem usefully different
> > to IDT delivery.=A0 #DB shouldn't get too deep, and NMI gets properly
> > inhibited now.
>=20
> Both #DB and #NMI can end up in perf, and all that goes quite deep :/

Can you please elaborate it a bit?