Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6;
Message-ID: <f0741a5b-229a-429a-8451-8af17261be9e@efficios.com>
Date:   Fri, 27 Oct 2023 12:35:56 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [POC][RFC][PATCH v2] sched: Extended Scheduler Time Slice
Content-Language: en-US
To:     Steven Rostedt <rostedt@goodmis.org>
Cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        Peter Zijlstra <peterz@infradead.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ankur Arora <ankur.a.arora@oracle.com>, linux-mm@kvack.org,
        x86@kernel.org, akpm@linux-foundation.org, luto@kernel.org,
        bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com,
        mingo@redhat.com, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de,
        jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com,
        boris.ostrovsky@oracle.com, konrad.wilk@oracle.com,
        jgross@suse.com, andrew.cooper3@citrix.com,
        Joel Fernandes <joel@joelfernandes.org>,
        Youssef Esmat <youssefesmat@chromium.org>,
        Vineeth Pillai <vineethrp@google.com>,
        Suleiman Souhlal <suleiman@google.com>,
        Ingo Molnar <mingo@kernel.org>,
        Daniel Bristot de Oliveira <bristot@kernel.org>
References: <20231025235413.597287e1@gandalf.local.home>
 <20231026105944.GJ33965@noisy.programming.kicks-ass.net>
 <20231026071413.4ed47b0e@gandalf.local.home>
 <f5b0fffa-423a-4571-be6c-383399274328@efficios.com>
 <CAHk-=whnyt2TccpDaWGTbDsVkKApL3c2FtDPMEwuTmeu_cEL8Q@mail.gmail.com>
 <7871472b-a0c4-4475-9671-69a3244f956d@efficios.com>
 <20231026164549.14d45c60@gandalf.local.home>
 <644da047-2f7a-4d55-a339-f2dc28d2c852@efficios.com>
 <20231027122442.5c76dd62@gandalf.local.home>
From:   Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
In-Reply-To: <20231027122442.5c76dd62@gandalf.local.home>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2023-10-27 12:24, Steven Rostedt wrote:
> On Fri, 27 Oct 2023 12:09:56 -0400
> Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:
> 
>>> I need to clear one bit while seeing if another bit is set. I could also
>>> use subl, as that would also atomically clear the bit.
>>
>> Ah ok, I did not get that you needed to test for a different bit than
>> the one you clear.
> 
> Yeah, maybe I'm not articulating the implementation as well.
> 
>    bit 0: Set by user space to tell the kernel it's in a critical section
> 
>    bit 1: Set by kernel that it gave user space extend time slice
> 
> Bit 1 will only be set by the kernel if bit 0 is set.
> 
> When entering a critical section, user space will set bit 0. When it leaves
> the critical section, it needs to clear bit 0, but also needs to handle the
> race condition from where it clears the bit and where the kernel could
> preempt it and set bit 1. Thus it needs an atomic operation to clear bit 0
> without affecting bit 1. Once bit 0 is cleared, it does not need to worry
> about bit 1 being set after that as the kernel will only set bit 1 if it
> sees that bit 0 was set. After user space clears bit 0, it must check bit 1
> to see if it should now schedule. And it's also up to user space to clear
> bit 1, but it can do that at any time with bit 0 cleared.
> 
>   extend() {
> 	cr_flags = 1;
>   }
> 
>   unextend() {
> 	cr_flags &= ~1;  /* Must be atomic */
> 	if (cr_flags & 2) {
> 		cr_flags = 0;  /* May not be necessary as it gets cleared by extend() */
> 		sched_yield();
> 	}
>   }
> 
> Does that make more sense?

Not really.

Please see my other email about the need for a reference count here, for
nested locks use-cases.

By "atomic" operation I suspect you only mean "single instruction" which can
alter the state of the field and keep its prior content in a register, not a
lock-prefixed atomic operation, right ?

The only reason why you have this asm trickiness is because both states
are placed into different bits from the same word, which is just an
optimization. You could achieve the same much more simply by splitting
this state in two different words, e.g.:

extend() {
   WRITE_ONCE(__rseq_abi->cr_nest, __rseq_abi->cr_nest + 1);
   barrier()
}

unextend() {
   barrier()
   WRITE_ONCE(__rseq_abi->cr_nest, __rseq_abi->cr_nest - 1);
   if (READ_ONCE(__rseq_abi->must_yield)) {
     WRITE_ONCE(__rseq_abi->must_yield, 0);
     sched_yield();
   }
}

Or am I missing something ?

Thanks,

Mathieu
   

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com