Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8;
References: <20230830184958.2333078-8-ankur.a.arora@oracle.com>
 <20230908070258.GA19320@noisy.programming.kicks-ass.net>
 <87zg1v3xxh.fsf@oracle.com>
 <CAHk-=whagwHrDxhjUVrRPhq78YC195KrSGzuC722-4MvAz40pw@mail.gmail.com>
 <87edj64rj1.fsf@oracle.com>
 <CAHk-=wi0bXpgULVVLc2AdJcta-fvQP7yyFQ_JtaoHUiPrqf--A@mail.gmail.com>
 <87zg1u1h5t.fsf@oracle.com>
 <CAHk-=whMkp68vNxVn1H3qe_P7n=X2sWPL9kvW22dsvMFH8FcQQ@mail.gmail.com>
 <20230911150410.GC9098@noisy.programming.kicks-ass.net>
 <87h6o01w1a.fsf@oracle.com>
 <20230912082606.GB35261@noisy.programming.kicks-ass.net>
 <87cyyfxd4k.ffs@tglx>
 <CAHk-=whnwC01m_1f-gaM1xbvvwzwTiKitrWniA-ChZv+bM03dg@mail.gmail.com>
 <87led2wdj0.ffs@tglx>
User-agent: mu4e 1.4.10; emacs 27.2
From:   Ankur Arora <ankur.a.arora@oracle.com>
To:     Thomas Gleixner <tglx@linutronix.de>
Cc:     Linus Torvalds <torvalds@linux-foundation.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Ankur Arora <ankur.a.arora@oracle.com>,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org,
        akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de,
        dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com,
        juri.lelli@redhat.com, vincent.guittot@linaro.org,
        willy@infradead.org, mgorman@suse.de, rostedt@goodmis.org,
        jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com,
        boris.ostrovsky@oracle.com, konrad.wilk@oracle.com,
        jgross@suse.com, andrew.cooper3@citrix.com
Subject: Re: [PATCH v2 7/9] sched: define TIF_ALLOW_RESCHED
In-reply-to: <87led2wdj0.ffs@tglx>
Date:   Wed, 20 Sep 2023 07:22:29 -0700
Message-ID: <878r90lyai.fsf@oracle.com>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8dtwZM5Cew7TtO49gE11AlNueeWDAf+ir3fxZzdHGzBTGwggd/7FR9l/IpHT?=
 =?us-ascii?Q?end6nM3FbMtEnAraF8Sv43N6wWkUnPdwXjkCyiXMMqgPPTG3w+DLNyzg8XUA?=
 =?us-ascii?Q?SUhH2zdeHwjFJDWsUEjtkfimVNR3Lq/0WRTmB8xbzTnv0vLxMKv/p6teGuIG?=
 =?us-ascii?Q?GIdi1PxsD5q147bYKYNFgYpEyWKdTOJwZ2R0M1pHZs2cZKNjm+l6eA5mTUcy?=
 =?us-ascii?Q?hQZcQe22ZKxLt6s/iv8GQyaO/rR86MlSZAO5cSBlP3X8eZUefGXu/xCCxJ8B?=
 =?us-ascii?Q?tS149M46ERyqtgpudVbLTz2bVIXgmya3eLcErLrnrmwrZ3dU4Lxz7h2Jeioe?=
 =?us-ascii?Q?UnaWZv9ZO0DgOJmjXVIWX6miTu8LOOKsAwRHEFAarMMD8CBSPdQv4Mj5TJG/?=
 =?us-ascii?Q?3ko8FzDos98MLiDBoSCHObKmyMN3xuiFx/qNvC+JpoqffUHgN/vNOQb81Huw?=
 =?us-ascii?Q?JOdWquAamDqV7XYlkz2B8cLPG6SEz4kwJPJbr7AEfLroQvCdkGoJ2Yu4bKuV?=
 =?us-ascii?Q?XUUir3BDM/gyVxt+P9g3/a8BBzCaiw1rS8TsmnVtgLJsLI2yEDQqf39JLxC9?=
 =?us-ascii?Q?yVm1fNA6ykIfJTZay9RmtNZ7hNeKfTyw6Q2RIuBOnrHdGidnJX3r7trGcrIp?=
 =?us-ascii?Q?jOHVJtskd3d/N/vlsx6Bpp/eBCYWxw7szigfzE/xL7fHvNBWgWQkAlgNToew?=
 =?us-ascii?Q?Cv9RlUaeiYmWjUAkGwlpDT39yLnFhRACUI0RKRbfQ5M6xdWkKqTf1wyzJedS?=
 =?us-ascii?Q?ueUDDYEeqrIMGsLrcVzW+8foC7smYWzHI1B4ESrSugZZ+jDWhsC6ROyfmNsL?=
 =?us-ascii?Q?3Ev1ZA+fhzhEqfv2/RjX+qrLEoNocRqfatAKVENMWTehj4IATGwqUDrvpPby?=
 =?us-ascii?Q?tBLnp47kldZB9OX+DOEWpimJYLQIEcDGWYslmrZzgfQFyQi3buYfwZkE/X9S?=
 =?us-ascii?Q?CgQkiEqaFhUB67StAfzAHrXh5EShN4hjkrzUfj49vvUoJFVGU9hkiZ89BSIM?=
 =?us-ascii?Q?l2gH6B623cy/vz1cvs8fAxzDqrL+YZP4WFgVnekE8Cw7vILTire5CD4+Prf6?=
 =?us-ascii?Q?U4il9GpLdMjrE6Bfi7TIn70YLwqgBBDKuQkAha4noPiJBy0Y6yu/hC3+DQ21?=
 =?us-ascii?Q?PARfzkmI4lUOjrs5bx0dxXFjp8rbjIWHcdVIKMatvLWbzBVzF/gxCk+lwtTn?=
 =?us-ascii?Q?Ye5UI9uauEbmrrKPdqaroQjqdtXAKnSAOzyTdFeiobuGNnmgfgg789Ux8kc1?=
 =?us-ascii?Q?azFRRFteMi92ObLoP3qj+TbUQ1Vpx0FYwBwIsxjSb+VymjXfWor2QXokqbg9?=
 =?us-ascii?Q?LUh9gcfK8PflCucDf9c9mcQh/FQjktKiiSokh/yG0zfdnScsOaYK1mQ3qZsW?=
 =?us-ascii?Q?SPpsiMGjPDEi/nlF+j+Vx+Nz2NJvc8NlGTQ+AGiMCinGZNQ8htOY7wdqRV6m?=
 =?us-ascii?Q?4nnJ982iEF818ImSTQqMCdTwbXhVbTqverErYPgTDaxQs8ZWdwlNA9R6r/te?=
 =?us-ascii?Q?hDDL/a92lA6/yYxlYxSBvkUUMNrlOODqXLpGUmfkeSY7ForugovbsiMRzrm/?=
 =?us-ascii?Q?vrbtVGTiD/WoBRxqpukCW8/B11o6C+psEAuX6uwfRYihLl1xdllTCwepd2ei?=
 =?us-ascii?Q?HQ=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?/x3Pase/AgcP62cDkiUA/cSZCkLyoOlNPQ+2Mhpv0idctAiHHAFa+c9oTjKM?=
 =?us-ascii?Q?zN/C0jQ2d3IC0uL746iLiz3ZvZoOdVjCNPG+qVa7RlFmZEUQ3TcMjc8KM/c0?=
 =?us-ascii?Q?Bfflt1BnXN1YFEzu5ifJJH2dnP3ybBlecGVPfIraaIVetOUEajb3BxZ/MjcM?=
 =?us-ascii?Q?j4XKZ6ynxpbevL2q/Ox3FWc/KOUah0FWditxHgeDiM9pZ23lUFPD5oOiul4P?=
 =?us-ascii?Q?TJvHbyJj36vhnHoM7UYy/vnbvbeKmsB2xZVr/vAsn/TLKZn3awXoYFlSvphS?=
 =?us-ascii?Q?D2+1w/YWey58mXR9/xRg1fTYF+ZMJP/dpeVggBChc9clVOfUSMMssFNpy16z?=
 =?us-ascii?Q?49JD9CjhE8cdE3U0IMKmGUWFVHBVVcyuoaT3IO7XNVeX02aoZpr15pAEae2H?=
 =?us-ascii?Q?HPijhV0unumz7yCtFMh2Kisz/sOA+4xTB1y6LjQrf2g+0R7Mr0ck5Y/zCgnT?=
 =?us-ascii?Q?dmjOhYcAZ68ZmQDHZzJpt8k0hHo691dc4YySNL6mJjqIcqSHDnbl6cLhuAeP?=
 =?us-ascii?Q?vNB0icLNiChJKkW3RDjGgH2JBR7F+0s6TpNq9CtY4Ork7W1UILhYeY6rObl8?=
 =?us-ascii?Q?dGvqBZT1WGxLazRaEhd2MiJ8ZRnAXoKdgTorYPYNKjstddcYiilO3Zgx/Z0C?=
 =?us-ascii?Q?kWe+itvKvRWSAUbpqKV6dKfIajCQ/6y+ESKFiVjMybJqhFmKn1EoiNUAbPbR?=
 =?us-ascii?Q?IFdUsUXzLWFn93+iqFqOFkrw3h7vuEMfa3W7nF9EpCWnaZkeccnCR/B0Dtex?=
 =?us-ascii?Q?11JIDWq4tf53Sq+fdAKTWmQ+ukLAzdgnOktFGgIITklju3kqJFD7tSY+2zTZ?=
 =?us-ascii?Q?EWvsStglmB+o4Ul1fiOpJ+RdYksAvsUFtiABHxEg1i0jKo0j1p8AkOxK6b1Y?=
 =?us-ascii?Q?uk/c0CCDT9KK92qRV4XtYnYgOtQm1i4nz71cqu4IjPk9R8tLgGnVQqw3S8Eo?=
 =?us-ascii?Q?07LX82Ccu3eXjGK8lzRRRhI2DN3zwjYf6bGlik/RCq3tuOojEpobiSyol4/0?=
 =?us-ascii?Q?GM2XFWYFanHPJ0fhYbGbh2+NV6XZaGtmV7h6PVbfVmb9CHWhZzBTUYEvMIBz?=
 =?us-ascii?Q?6nXoOV4u3LX4Worvz68avavu7tkZ/e3FHld9fk9UrVmA8JSC7XLDGTvKHVK8?=
 =?us-ascii?Q?UlnkFazWElmj2vOBXnogP6oNrHH7sjpL+42yC9hQVhkzvfPhPtKs0gHxgKZR?=
 =?us-ascii?Q?B6RO/GvameQ6w+NBdiaHNGCbc3UC+SRQABIEasceYF+wFZdNWr5kUo4qpoI?=
 =?us-ascii?Q?=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ba6390f3-a05a-4d1d-74e6-08dbb9e50670
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 14:22:31.1746
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: syksx0mxZdg9P4G4Zty34wpwgQWBPVPsprhAwZ7EUrQKZEfVQOEukuNsCkS+I3LovJma0WnyyO8pX3ARDnLndpmo9h4YaO9YTlH7sqEvAl8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR10MB6024
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.601,FMLib:17.11.176.26
 definitions=2023-09-20_05,2023-09-20_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxscore=0
 mlxlogscore=785 bulkscore=0 malwarescore=0 suspectscore=0 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000
 definitions=main-2309200117
X-Proofpoint-GUID: KEkIz4pqhcEnLIZJG9OCUgnO-KJJByse
X-Proofpoint-ORIG-GUID: KEkIz4pqhcEnLIZJG9OCUgnO-KJJByse
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Wed, 20 Sep 2023 07:23:28 -0700 (PDT)


Thomas Gleixner <tglx@linutronix.de> writes:

> So the decision matrix would be:
>
>                 Ret2user        Ret2kernel      PreemptCnt=0
>
> NEED_RESCHED       Y                Y               Y
> LAZY_RESCHED       Y                N               N
>
> That is completely independent of the preemption model and the
> differentiation of the preemption models happens solely at the scheduler
> level:

This is relatively minor, but do we need two flags? Seems to me we
can get to the same decision matrix by letting the scheduler fold
into the preempt-count based on current preemption model.

> PREEMPT_NONE sets only LAZY_RESCHED unless it needs to enforce the time
> slice where it sets NEED_RESCHED.

PREEMPT_NONE sets up TIF_NEED_RESCHED. For the time-slice expiry case,
also fold into preempt-count.

> PREEMPT_VOLUNTARY extends the NONE model so that the wakeup of RT class
> tasks or sporadic event tasks sets NEED_RESCHED too.

PREEMPT_NONE sets up TIF_NEED_RESCHED and also folds it for the
RT/sporadic tasks.

> PREEMPT_FULL always sets NEED_RESCHED like today.

Always fold the TIF_NEED_RESCHED into the preempt-count.

> We should be able merge the PREEMPT_NONE/VOLUNTARY behaviour so that we
> only end up with two variants or even subsume PREEMPT_FULL into that
> model because that's what is closer to the RT LAZY preempt behaviour,
> which has two goals:
>
>       1) Make low latency guarantees for RT workloads
>
>       2) Preserve the throughput for non-RT workloads
>
> But in any case this decision happens solely in the core scheduler code
> and nothing outside of it needs to be changed.
>
> So we not only get rid of the cond/might_resched() muck, we also get rid
> of the static_call/static_key machinery which drives PREEMPT_DYNAMIC.
> The only place which still needs that runtime tweaking is the scheduler
> itself.

True. The dynamic preemption could just become a scheduler tunable.

> Though it just occured to me that there are dragons lurking:
>
> arch/alpha/Kconfig:     select ARCH_NO_PREEMPT
> arch/hexagon/Kconfig:   select ARCH_NO_PREEMPT
> arch/m68k/Kconfig:      select ARCH_NO_PREEMPT if !COLDFIRE
> arch/um/Kconfig:        select ARCH_NO_PREEMPT
>
> So we have four architectures which refuse to enable preemption points,
> i.e. the only model they allow is NONE and they rely on cond_resched()
> for breaking large computations.
>
> But they support PREEMPT_COUNT, so we might get away with a reduced
> preemption point coverage:
>
>                 Ret2user        Ret2kernel      PreemptCnt=0
>
> NEED_RESCHED       Y                N               Y
> LAZY_RESCHED       Y                N               N

So from the discussion in the other thread, for the ARCH_NO_PREEMPT
configs that don't support preemption, we probably need a fourth
preemption model, say PREEMPT_UNSAFE.

These could use only the Ret2user preemption points and just fallback
to the !PREEMPT_COUNT primitives.

Thanks

--
ankur