Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 20 Dec 2022 21:44:33 -0800
From:   "H. Peter Anvin" <hpa@zytor.com>
To:     "Li, Xin3" <xin3.li@intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        "andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "Christopherson,, Sean" <seanjc@google.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "Shankar, Ravi V" <ravi.v.shankar@intel.com>
Subject: RE: [RFC PATCH 22/32] x86/fred: FRED initialization code
User-Agent: K-9 Mail for Android
In-Reply-To: <BN6PR1101MB2161C51C6068026D9C442460A8EB9@BN6PR1101MB2161.namprd11.prod.outlook.com>
References: <20221220063658.19271-1-xin3.li@intel.com> <20221220063658.19271-23-xin3.li@intel.com> <Y6GELyEJeKY3dEqJ@hirez.programming.kicks-ass.net> <16972e64-7d7b-ad8c-f8dc-6dcab69e629e@citrix.com> <Y6GIN5Uf7Qd43A9U@hirez.programming.kicks-ass.net> <BN6PR1101MB2161C51C6068026D9C442460A8EB9@BN6PR1101MB2161.namprd11.prod.outlook.com>
Message-ID: <0D72CCA1-A0B6-487B-A6B9-7341020D28A2@zytor.com>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On December 20, 2022 9:28:52 PM PST, "Li, Xin3" <xin3=2Eli@intel=2Ecom> wro=
te:
>> > >> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
>> > >> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
>> > >> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
>> > >> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
>> > >> +	       FRED_STKLVL(X86_TRAP_DF,  3));
>> > >> +
>> > >> +	/* The FRED equivalents to IST stacks=2E=2E=2E */
>> > >> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
>> > >> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
>> > >> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
>> > > Not quite=2E=2E IIRC fred only switches to another stack when the l=
evel
>> > > of the exception is higher=2E Specifically, if we trigger #DB while
>> > > inside #NMI we will not switch to the #DB stack (since 1 < 2)=2E
>
>Yes, current stack level can only grow higher=2E
>
>> >
>> > There needs to be a new stack for #DF, and just possibly one for #MC=
=2E
>> > NMI and #DB do not need separate stacks under FRED=2E
>>=20
>> True, there is very little need to use additional stacks with FRED=2E
>
>Pretty much=2E
>
>#DB/NMI from a ring 3 context uses CSL 0, and their CSLs increase only
>when happening from a ring 0 context=2E
>
>>=20
>> > > Now, as mentioned elsewhere, it all nests a lot saner, but stack
>> > > exhaustion is still a thing, given the above, what happens when a
>> > > #DB hits an #NMI which tickles a #VE or something?
>> > >
>> > > I don't think we've increased the exception stack size, but perhaps
>> > > we should for FRED?
>> >
>> > Not sure if it matters too much - it doesn't seem usefully different
>> > to IDT delivery=2E=C2=A0 #DB shouldn't get too deep, and NMI gets pro=
perly
>> > inhibited now=2E
>>=20
>> Both #DB and #NMI can end up in perf, and all that goes quite deep :/
>
>Can you please elaborate it a bit?
>

Right, this is one major reason for putting #DB/NMI in a separate stack le=
vel=2E