Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 17 Apr 2020 13:12:55 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Joel Fernandes <joel@joelfernandes.org>
Cc:     vpillai <vpillai@digitalocean.com>,
        Nishanth Aravamudan <naravamudan@digitalocean.com>,
        Julien Desfossez <jdesfossez@digitalocean.com>,
        Tim Chen <tim.c.chen@linux.intel.com>, mingo@kernel.org,
        tglx@linutronix.de, pjt@google.com, torvalds@linux-foundation.org,
        linux-kernel@vger.kernel.org, fweisbec@gmail.com,
        keescook@chromium.org, kerrnel@google.com,
        Phil Auld <pauld@redhat.com>, Aaron Lu <aaron.lwe@gmail.com>,
        Aubrey Li <aubrey.intel@gmail.com>, aubrey.li@linux.intel.com,
        Valentin Schneider <valentin.schneider@arm.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
        Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [RFC PATCH 00/13] Core scheduling v5
Message-ID: <20200417111255.GZ20730@hirez.programming.kicks-ass.net>
References: <cover.1583332764.git.vpillai@digitalocean.com>
 <20200414142152.GV20730@hirez.programming.kicks-ass.net>
 <20200415163220.GA180518@google.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200415163220.GA180518@google.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Apr 15, 2020 at 12:32:20PM -0400, Joel Fernandes wrote:
> On Tue, Apr 14, 2020 at 04:21:52PM +0200, Peter Zijlstra wrote:
> > On Wed, Mar 04, 2020 at 04:59:50PM +0000, vpillai wrote:
> > > TODO
> > > ----
> > > - Work on merging patches that are ready to be merged
> > > - Decide on the API for exposing the feature to userland
> > > - Experiment with adding synchronization points in VMEXIT to mitigate
> > >   the VM-to-host-kernel leaking
> > 
> > VMEXIT is too late, you need to hook irq_enter(), which is what makes
> > the whole thing so horrible.
> 
> We came up with a patch to do this as well. Currently testing it more and it
> looks clean, will share it soon.

Thomas said we actually first do VMEXIT, and then enable interrupts. So
the VMEXIT thing should actually work, and that is indeed much saner
than sticking it in irq_enter().

It does however put yet more nails in the out-of-tree hypervisors.

> > > - Investigate the source of the overhead even when no tasks are tagged:
> > >   https://lkml.org/lkml/2019/10/29/242
> > 
> >  - explain why we're all still doing this ....
> > 
> > Seriously, what actual problems does it solve? The patch-set still isn't
> > L1TF complete and afaict it does exactly nothing for MDS.
> 
> The L1TF incompleteness is because of cross-HT attack from Guest vCPU
> attacker to an interrupt/softirq executing on the other sibling correct? The
> IRQ enter pausing the other sibling should fix that (which we will share in
> a future series revision after adequate testing).

Correct, the vCPU still running can glean host (kernel) state from the
sibling handling the interrupt in the host kernel.

> > Like I've written many times now, back when the world was simpler and
> > all we had to worry about was L1TF, core-scheduling made some sense, but
> > how does it make sense today?
> 
> For ChromeOS we're planning to tag each and every task seperately except for
> trusted processes, so we are isolating untrusted tasks even from each other.
> 
> Sorry if this sounds like pushing my usecase, but we do get parallelism
> advantage for the trusted tasks while still solving all security issues (for
> ChromeOS). I agree that cross-HT user <-> kernel MDS is still an issue if
> untrusted (tagged) tasks execute together on same core, but we are not
> planning to do that on our setup at least.

That doesn't completely solve things I think. Even if you run all
untrusted tasks as core exclusive, you still have a problem of them vs
interrupts on the other sibling.

You need to somehow arrange all interrupts to the core happen on the
same sibling that runs your untrusted task, such that the VERW on
return-to-userspace works as intended.

I suppose you can try and play funny games with interrupt routing tied
to the force-idle state, but I'm dreading what that'll look like. Or
were you going to handle this from your irq_enter() thing too?


Can someone go write up a document that very clearly states all the
problems and clearly explains how to use this feature?