Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <2f191ddb-de89-52c0-e7da-26ac0239b8fe@efficios.com>
Date:   Fri, 11 Nov 2022 09:18:26 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.0
Subject: Re: [PATCH v5 08/24] sched: Introduce per memory space current
 virtual cpu id
Content-Language: en-US
To:     Andy Lutomirski <luto@amacapital.net>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        linux-kernel@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
        "Paul E . McKenney" <paulmck@kernel.org>,
        Boqun Feng <boqun.feng@gmail.com>,
        "H . Peter Anvin" <hpa@zytor.com>, Paul Turner <pjt@google.com>,
        linux-api@vger.kernel.org, Christian Brauner <brauner@kernel.org>,
        Florian Weimer <fw@deneb.enyo.de>, David.Laight@aculab.com,
        carlos@redhat.com, Peter Oskolkov <posk@posk.io>,
        Alexander Mikhalitsyn <alexander@mihalicyn.com>,
        Chris Kennelly <ckennelly@google.com>
References: <20221103200359.328736-1-mathieu.desnoyers@efficios.com>
 <20221103200359.328736-9-mathieu.desnoyers@efficios.com>
 <CALCETrW1doHX3=za+KDuB=4y+wHsnaZpVkDP3OhZXGrQU2iffw@mail.gmail.com>
From:   Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
In-Reply-To: <CALCETrW1doHX3=za+KDuB=4y+wHsnaZpVkDP3OhZXGrQU2iffw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2022-11-10 23:41, Andy Lutomirski wrote:
> On Thu, Nov 3, 2022 at 1:05 PM Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
>>
>> This feature allows the scheduler to expose a current virtual cpu id
>> to user-space. This virtual cpu id is within the possible cpus range,
>> and is temporarily (and uniquely) assigned while threads are actively
>> running within a memory space. If a memory space has fewer threads than
>> cores, or is limited to run on few cores concurrently through sched
>> affinity or cgroup cpusets, the virtual cpu ids will be values close
>> to 0, thus allowing efficient use of user-space memory for per-cpu
>> data structures.
>>
> 
> Just to check, is a "memory space" an mm?  I've heard these called
> "mms" or sometimes (mostly accurately) "processes" but never memory
> spaces.  Although I guess the clone(2) manpage says "memory space".

Yes, exactly.

I've had a hard time finding the right word there to describe the 
concept of a struct mm from a user-space point of view, and ended up 
finding that the clone(2) man page expresses the result of a clone 
system call with CLONE_VM set as sharing a "memory space", aka a mm_struct.

 From an internal kernel implementation perspective it is usually 
referred to as a "mm", but it's not a notion that appears to be exposed 
to user-space.

And unfortunately "process" can mean so many things other than a struct 
mm: is it a thread group ? Or just a group of processes sharing a file 
descriptor table ? Or sharing signal handlers ? How would you call a 
thread group (clone with CLONE_THREAD) that does not have CLONE_VM set ?

> 
> Also, in my mind "virtual cpu" is vCPU, which this isn't.  Maybe
> "compacted cpu" or something?  It's a strange sort of concept.

I've kept the same wording that has been introduced in 2011 by Paul 
Turner and used internally at Google since then, although it may be 
confusing if people expect kvm-vCPU and rseq-vcpu to mean the same 
thing. Both really end up providing the semantic of a virtually assigned 
cpu id (in opposition to the logical cpu id on the system), but this is 
much more involved in the case of KVM.

Thanks,

Mathieu


-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com