DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B2388C08EC13
Date: Fri, 22 Sep 2017 09:33:05 -0300
From: Marcelo Tosatti <mtosatti@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, mingo@redhat.com,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [patch 3/3] x86: kvm guest side support for KVM_HC_RT_PRIO
 hypercall
Message-ID: <20170922123305.GB29608@amt.cnet>
References: <20170921113835.031375194@redhat.com>
 <20170921114039.466130276@redhat.com>
 <20170921133653.GO26248@char.us.oracle.com>
 <20170921140628.zliqlz7mrlqs5pzz@hirez.programming.kicks-ass.net>
 <20170922011039.GB20133@amt.cnet>
 <20170922100004.ydmaxvgpc2zx7j25@hirez.programming.kicks-ass.net>
 <20170922105609.deln6kylvvpaijg7@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170922105609.deln6kylvvpaijg7@hirez.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2022
Lines: 55

On Fri, Sep 22, 2017 at 12:56:09PM +0200, Peter Zijlstra wrote:
> On Fri, Sep 22, 2017 at 12:00:04PM +0200, Peter Zijlstra wrote:
> > On Thu, Sep 21, 2017 at 10:10:41PM -0300, Marcelo Tosatti wrote:
> > > When executing guest vcpu-0 with FIFO:1 priority, which is necessary
> > > to
> > > deal with the following situation:
> > > 
> > > VCPU-0 (housekeeping VCPU)              VCPU-1 (realtime VCPU)
> > > 
> > > raw_spin_lock(A)
> > > interrupted, schedule task T-1          raw_spin_lock(A) (spin)
> > > 
> > > raw_spin_unlock(A)
> > > 
> > > Certain operations must interrupt guest vcpu-0 (see trace below).
> > 
> > Those traces don't make any sense. All they include is kvm_exit and you
> > can't tell anything from that.
> > 
> > > To fix this issue, only change guest vcpu-0 to FIFO priority
> > > on spinlock critical sections (see patch).
> > 
> > This doesn't make sense. So you're saying that if you run all VCPUs as
> > FIFO things come apart? Why?
> > 
> > And why can't they still come apart when the guest holds a spinlock?
> 
> That is, running a RT guest and not having _all_ VCPUs being RT tasks on
> the host is absolutely and completely insane and broken.

Can you explain why, please?

> Fix whatever needs fixing to allow your VCPU0 to be RT, don't do insane
> things like this.

VCPU0 can be RT, but you'll get the following hang, if the emulator
thread is sharing a pCPU with VCPU0:

	1. submit IO.
	2. busy spin.

As executed by the guest vcpu (its a natural problem).

Do you have a better suggestion as how to fix the problem?

We can fix the BIOS, but userspace will still be allowed to
generate the code pattern above.

And increasing the priority of the emulator thread, at random times 
(so it can inject interrupts to vcpu-0), can cause it to interrupt 
vcpu-0 in a spinlock protected section.

The only other option is for customers to live with the decreased 
packing (that is require one pcpu for each vcpu, and an additional pcpu
for emulator threads). Is that what you are suggesting?