Date: Tue, 6 Dec 2016 13:43:55 -0800
From: yunhong jiang <yunhong.jiang@linux.intel.com>
To: Chris Metcalf <cmetcalf@mellanox.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: Questions on the task isolation patches
Message-ID: <20161206134355.193c752b@jnakajim-build>
In-Reply-To: <5dd4cbf7-d0c0-074a-c5bc-e2e09ec3dc75@mellanox.com>
References: <20161201142812.369f23f8@jnakajim-build>
        <5dd4cbf7-d0c0-074a-c5bc-e2e09ec3dc75@mellanox.com>
Organization: otc
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3797
Lines: 93

On Fri, 2 Dec 2016 13:58:08 -0500
Chris Metcalf <cmetcalf@mellanox.com> wrote:

> On 12/1/2016 5:28 PM, yunhong jiang wrote:
> > Hi, Chris
> >      I noticed your task isolation patch set at
> > https://lkml.org/lkml/2016/8/9/759 . Thanks a lot for the great
> > effort.
> >
> >      I checked the patch and have some questions about how to use
> > this functionality on the NFV environment. In the NFV scenario,
> > high speed network code runs on a VM user space and the VM is
> > hosted by KVM hypervisor. To achieve the high speed network, not
> > only the network code should be isolated, also the vCPU thread.
> 
> That's true.
> 
> >      I checked your patch to think how to isolate the vCPU thread,
> > and I have some questions and hope your hints:
> >
> > a) If the task isolation need prctl to mark itself as isolated,
> > possibly the vCPU thread can't achieve it. First, the vCPU thread
> > may need system service during OS booting time, also it's the
> > application, instead of the vCPU thread to decide if the vCPU
> > thread should be isolated. So possibly we need a mechanism so that
> > another process can set the vCPU thread's task isolation?
> 
> These are good questions.  I think that the we would probably want to
> add a KVM mode that did the prctl() before transitioning back to the

Would prctl() when back to gues be too heavy?

> guest.  But then, in the same way that we currently allow another
> prctl() from a task-isolated userspace process, we'd probably need to

You mean currently in your patch we alraedy can do the prctl from 3rd party
process to task-isolate a userspace process? Sorry that I didn't notice that
part.

> allow a KVM exit from a guest to happen without triggering any
> task-isolation errors.

yes.

> 
> Alternately, if we add the proposed NOSIG mode, then the hypervisor
> can use that, and simply stay in task-isolation mode no matter what
> system support is requested.

Yes, possibly we will keep the vCPU thread as always NOSIG mode.

> 
> Presumably in the guest, we'd run task isolation as well, but
> presumably we'd only exit to the hypervisor if we were already in the
> guest kernel?  Although I guess you could image providing trapping
> mmio mappings to the guest's userspace process that caused a
> hypervisor exit.  Perhaps we extend the hypervisor to know that a
> guest might support task isolation, and to generate a warning on
> behalf of the guest if the hypervisor sees an exit from the guest's
> userspace and the guest task is in task-isolation mode?
> 
> > b) I noticed that curently the task_isolation_prepare() is invoked
> > on exit_to_usermode_loop(), while we may need such job on VM Exit
> > procedure, or interrupt handling during VM exit handling.
> 
> Yes, something like that.  I am not super-familiar with the KVM
> internals (I did a KVM port to the tile architecture a while back, but
> I'd have to swap all that memory back in before I could even have a
> half-educated opinion.)

Got it.

> 
> >      Also, I'm also considering how to utilize this task_isolation
> > for network node which is not busy-loop, like the interrupt mode
> > DPDK code. In the interrupt mode DPDK code, the application may
> > yield the CPU if there is no network packet for a very long time
> > and it will re-enter the busy loop mode once new packet arrived.
> > This interrupt mode will be helpful for both power management and
> > resource utilization. Per my understanding, the application should
> > turn off the task isolation before yield the CPU and the restart
> > the task isolation after the new packet, right?
> 
> Yes, exactly.
> 
> I am not likely to pursue KVM myself, at least until the basic patch
> series has been accepted upstream.

Yes, and I will be more than happy to help on this.

Thanks
-jyh
>