Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 20 Dec 2022 11:02:31 +0100
From:   Peter Zijlstra <peterz@infradead.org>
To:     Andrew Cooper <Andrew.Cooper3@citrix.com>
Cc:     Xin Li <xin3.li@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "hpa@zytor.com" <hpa@zytor.com>,
        "seanjc@google.com" <seanjc@google.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "ravi.v.shankar@intel.com" <ravi.v.shankar@intel.com>
Subject: Re: [RFC PATCH 22/32] x86/fred: FRED initialization code
Message-ID: <Y6GIN5Uf7Qd43A9U@hirez.programming.kicks-ass.net>
References: <20221220063658.19271-1-xin3.li@intel.com>
 <20221220063658.19271-23-xin3.li@intel.com>
 <Y6GELyEJeKY3dEqJ@hirez.programming.kicks-ass.net>
 <16972e64-7d7b-ad8c-f8dc-6dcab69e629e@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <16972e64-7d7b-ad8c-f8dc-6dcab69e629e@citrix.com>
Precedence: bulk

On Tue, Dec 20, 2022 at 09:55:31AM +0000, Andrew Cooper wrote:
> On 20/12/2022 9:45 am, Peter Zijlstra wrote:
> > On Mon, Dec 19, 2022 at 10:36:48PM -0800, Xin Li wrote:
> >
> >> +	wrmsrl(MSR_IA32_FRED_STKLVLS,
> >> +	       FRED_STKLVL(X86_TRAP_DB,  1) |
> >> +	       FRED_STKLVL(X86_TRAP_NMI, 2) |
> >> +	       FRED_STKLVL(X86_TRAP_MC,  2) |
> >> +	       FRED_STKLVL(X86_TRAP_DF,  3));
> >> +
> >> +	/* The FRED equivalents to IST stacks... */
> >> +	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
> >> +	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
> >> +	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
> > Not quite.. IIRC fred only switches to another stack when the level of
> > the exception is higher. Specifically, if we trigger #DB while inside
> > #NMI we will not switch to the #DB stack (since 1 < 2).
> 
> There needs to be a new stack for #DF, and just possibly one for #MC.?
> NMI and #DB do not need separate stacks under FRED.

True, there is very little need to use additional stacks with FRED.

> > Now, as mentioned elsewhere, it all nests a lot saner, but stack
> > exhaustion is still a thing, given the above, what happens when a #DB
> > hits an #NMI which tickles a #VE or something?
> >
> > I don't think we've increased the exception stack size, but perhaps we
> > should for FRED?
> 
> Not sure if it matters too much - it doesn't seem usefully different to
> IDT delivery.? #DB shouldn't get too deep, and NMI gets properly
> inhibited now.

Both #DB and #NMI can end up in perf, and all that goes quite deep :/