Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 24 Jan 2022 11:03:06 +0100
From:   Peter Zijlstra <peterz@infradead.org>
To:     Mark Rutland <mark.rutland@arm.com>
Cc:     mingo@redhat.com, tglx@linutronix.de, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com, linux-kernel@vger.kernel.org,
        linux-mm@kvack.org, linux-api@vger.kernel.org, x86@kernel.org,
        pjt@google.com, posk@google.com, avagin@google.com,
        jannh@google.com, tdelisle@uwaterloo.ca, posk@posk.io
Subject: Re: [RFC][PATCH v2 5/5] sched: User Mode Concurency Groups
Message-ID: <20220124100306.GO20638@worktop.programming.kicks-ass.net>
References: <20220120155517.066795336@infradead.org>
 <20220120160822.914418096@infradead.org>
 <Yerl+ZrZ2qflIMyg@FVFF77S0Q05N>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Yerl+ZrZ2qflIMyg@FVFF77S0Q05N>
Precedence: bulk

On Fri, Jan 21, 2022 at 04:57:29PM +0000, Mark Rutland wrote:

> > @@ -221,8 +227,11 @@ static inline void local_irq_disable_exi
> >   */
> >  static inline void irqentry_irq_enable(struct pt_regs *regs)
> >  {
> > -	if (!regs_irqs_disabled(regs))
> > +	if (!regs_irqs_disabled(regs)) {
> >  		local_irq_enable();
> > +		if (user_mode(regs) && (current->flags & PF_UMCG_WORKER))
> > +			umcg_sys_enter(regs, -1);
> > +	}
> >  }
> 
> Perhaps it would make sense to have separate umcg_sys_enter(regs) and
> umcg_sys_enter_syscall(regs, syscallno)? Even if the former is just a wrapper,
> to make the entry/exit bits clearly correspond for all the !syscall cases?

Can do I suppose.

> Also, is the syscall case meant to nest within this, or syscall entry paths not
> supposed to call irqentry_irq_enable() ?

No nesting, syscall_ vs irqentry_. And you can't have a syscall and an
exception both be from user at the same time :-)

> >  /**
> > @@ -232,8 +241,11 @@ static inline void irqentry_irq_enable(s
> >   */
> >  static inline void irqentry_irq_disable(struct pt_regs *regs)
> >  {
> > -	if (!regs_irqs_disabled(regs))
> > +	if (!regs_irqs_disabled(regs)) {
> > +		if (user_mode(regs) && (current->flags & PF_UMCG_WORKER))
> > +			umcg_sys_exit(regs);
> >  		local_irq_disable();
> > +	}
> >  }
> 
> Do the umcg_sys_{enter,exit}() calls need to happen with IRQs unmasked?

Yes; both can end up blocking.

> * If not (and this nests): for arm64 these can live in our
>   enter_from_user_mode() and exit_to_user_mode() helpers.
> 
> * If so (or this doesn't nest): for arm64 we'd need to rework our
>   local_daif_{inherit,restore,mask}() calls to handle this, though I've been
>   meaning to do that anyway to handle pseudo-NMI better.
> 
> Either way, it looks like we'd need helpers along the lines of:
> 
> | static __always_inline void umcg_enter_from_user(struct pt_regs *regs)
> | {
> | 	if (current->flags & PF_UMCG_WORKER)
> | 		umcg_sys_enter(regs, -1);
> | }
> | 
> | static __always_inline void umcg_exit_to_user(struct pt_regs *regs)
> | {
> | 	if (current->flags & PF_UMCG_WORKER)
> | 		umcg_sys_exit(regs);
> | }

Would something like:

#ifndef arch_irqentry_irq_enter
static __always_inline bool arch_irqentry_irq_enter(struct pt_regs *regs)
{
	if (!regs_irqs_disabled(regs)) {
		local_irq_enable();
		return true;
	}
	return false;
}
#endif

static __always_inline void irqentry_irq_enter(struct pt_regs *regs)
{
	if (arch_irqentry_irq_inherit(regs)) {
		if (user_mode(regs) && (current->flags & PF_UMCG_WORKER))
			umcg_sys_enter(regs, -1);
	}
}

Work? Then arm64 can do:

static __always_inline bool arch_irqentry_irq_enter(struct pt_regs *regs)
{
	local_daif_inherit();
	return interrupts_enabled(regs);
}

or somesuch...