Subject: Re: Questions on the task isolation patches
To: yunhong jiang <yunhong.jiang@linux.intel.com>
References: <20161201142812.369f23f8@jnakajim-build>
CC: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Paolo Bonzini" <pbonzini@redhat.com>
From: Chris Metcalf <cmetcalf@mellanox.com>
Message-ID: <5dd4cbf7-d0c0-074a-c5bc-e2e09ec3dc75@mellanox.com>
Date: Fri, 2 Dec 2016 13:58:08 -0500
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
 Thunderbird/45.5.1
MIME-Version: 1.0
In-Reply-To: <20161201142812.369f23f8@jnakajim-build>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 7bit
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Dec 2016 18:58:17.9977 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2759
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 3283
Lines: 69

On 12/1/2016 5:28 PM, yunhong jiang wrote:
> Hi, Chris
>      I noticed your task isolation patch set at
> https://lkml.org/lkml/2016/8/9/759 . Thanks a lot for the great effort.
>
>      I checked the patch and have some questions about how to use this
> functionality on the NFV environment. In the NFV scenario, high speed network
> code runs on a VM user space and the VM is hosted by KVM hypervisor. To achieve
> the high speed network, not only the network code should be isolated, also the
> vCPU thread.

That's true.

>      I checked your patch to think how to isolate the vCPU thread, and I have
> some questions and hope your hints:
>
> a) If the task isolation need prctl to mark itself as isolated,
> possibly the vCPU thread can't achieve it. First, the vCPU thread may
> need system service during OS booting time, also it's the application,
> instead of the vCPU thread to decide if the vCPU thread should be
> isolated. So possibly we need a mechanism so that another process can
> set the vCPU thread's task isolation?

These are good questions.  I think that the we would probably want to
add a KVM mode that did the prctl() before transitioning back to the
guest.  But then, in the same way that we currently allow another
prctl() from a task-isolated userspace process, we'd probably need to
allow a KVM exit from a guest to happen without triggering any
task-isolation errors.

Alternately, if we add the proposed NOSIG mode, then the hypervisor
can use that, and simply stay in task-isolation mode no matter what
system support is requested.

Presumably in the guest, we'd run task isolation as well, but
presumably we'd only exit to the hypervisor if we were already in the
guest kernel?  Although I guess you could image providing trapping
mmio mappings to the guest's userspace process that caused a
hypervisor exit.  Perhaps we extend the hypervisor to know that a
guest might support task isolation, and to generate a warning on
behalf of the guest if the hypervisor sees an exit from the guest's
userspace and the guest task is in task-isolation mode?

> b) I noticed that curently the task_isolation_prepare() is invoked on
> exit_to_usermode_loop(), while we may need such job on VM Exit procedure, or
> interrupt handling during VM exit handling.

Yes, something like that.  I am not super-familiar with the KVM
internals (I did a KVM port to the tile architecture a while back, but
I'd have to swap all that memory back in before I could even have a
half-educated opinion.)

>      Also, I'm also considering how to utilize this task_isolation for network
> node which is not busy-loop, like the interrupt mode DPDK code. In the
> interrupt mode DPDK code, the application may yield the CPU if there is no
> network packet for a very long time and it will re-enter the busy loop mode
> once new packet arrived. This interrupt mode will be helpful for both power
> management and resource utilization. Per my understanding, the application
> should turn off the task isolation before yield the CPU and the restart the
> task isolation after the new packet, right?

Yes, exactly.

I am not likely to pursue KVM myself, at least until the basic patch series
has been accepted upstream.

-- 
Chris Metcalf, Mellanox Technologies
http://www.mellanox.com