On Tue, 2023-09-19 at 14:00 +0100, Matthew Wilcox wrote:
> On Tue, Sep 19, 2023 at 02:30:59PM +0200, Thomas Gleixner wrote:
> > Though it just occured to me that there are dragons lurking:
> >
> > arch/alpha/Kconfig: select ARCH_NO_PREEMPT
> > arch/hexagon/Kconfig: select ARCH_NO_PREEMPT
> > arch/m68k/Kconfig: select ARCH_NO_PREEMPT if !COLDFIRE
> > arch/um/Kconfig: select ARCH_NO_PREEMPT
>
> Sounds like three-and-a-half architectures which could be queued up for
> removal right behind ia64 ...
The agreement to kill off ia64 wasn't an invitation to kill off other stuff
that people are still working on! Can we please not do this?
Thanks,
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer
`. `' Physicist
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Tue, Sep 19, 2023 at 03:37:24PM +0200, John Paul Adrian Glaubitz wrote:
> On Tue, 2023-09-19 at 14:00 +0100, Matthew Wilcox wrote:
> > On Tue, Sep 19, 2023 at 02:30:59PM +0200, Thomas Gleixner wrote:
> > > Though it just occured to me that there are dragons lurking:
> > >
> > > arch/alpha/Kconfig: select ARCH_NO_PREEMPT
> > > arch/hexagon/Kconfig: select ARCH_NO_PREEMPT
> > > arch/m68k/Kconfig: select ARCH_NO_PREEMPT if !COLDFIRE
> > > arch/um/Kconfig: select ARCH_NO_PREEMPT
> >
> > Sounds like three-and-a-half architectures which could be queued up for
> > removal right behind ia64 ...
>
> The agreement to kill off ia64 wasn't an invitation to kill off other stuff
> that people are still working on! Can we please not do this?
If you're working on one of them, then surely it's a simple matter of
working on adding CONFIG_PREEMPT support :-)
On Tue, 2023-09-19 at 15:42 +0200, Peter Zijlstra wrote:
> > The agreement to kill off ia64 wasn't an invitation to kill off other stuff
> > that people are still working on! Can we please not do this?
>
> If you're working on one of them, then surely it's a simple matter of
> working on adding CONFIG_PREEMPT support :-)
As Geert poined out, I'm not seeing anything particular problematic with the
architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
something about organizing KConfig files.
I find it a bit unfair that maintainers of architectures that have huge companies
behind them use their manpower to urge less popular architectures for removal just
because they don't have 150 people working on the port so they can keep up with
design changes quickly.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer
`. `' Physicist
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On 19/09/2023 14:42, Peter Zijlstra wrote:
> On Tue, Sep 19, 2023 at 03:37:24PM +0200, John Paul Adrian Glaubitz wrote:
>> On Tue, 2023-09-19 at 14:00 +0100, Matthew Wilcox wrote:
>>> On Tue, Sep 19, 2023 at 02:30:59PM +0200, Thomas Gleixner wrote:
>>>> Though it just occured to me that there are dragons lurking:
>>>>
>>>> arch/alpha/Kconfig: select ARCH_NO_PREEMPT
>>>> arch/hexagon/Kconfig: select ARCH_NO_PREEMPT
>>>> arch/m68k/Kconfig: select ARCH_NO_PREEMPT if !COLDFIRE
>>>> arch/um/Kconfig: select ARCH_NO_PREEMPT
>>>
>>> Sounds like three-and-a-half architectures which could be queued up for
>>> removal right behind ia64 ...
>>
>> The agreement to kill off ia64 wasn't an invitation to kill off other stuff
>> that people are still working on! Can we please not do this?
>
> If you're working on one of them, then surely it's a simple matter of
> working on adding CONFIG_PREEMPT support :-)
In the case of UML adding preempt will be quite difficult. I looked at this a few years back.
At the same time it is used for kernel test and other stuff. It is not exactly abandonware on a CPU found in archaeological artifacts of past civilizations like ia64.
>
> _______________________________________________
> linux-um mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-um
>
--
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/
On Tue, Sep 19, 2023 at 03:48:09PM +0200, John Paul Adrian Glaubitz wrote:
> On Tue, 2023-09-19 at 15:42 +0200, Peter Zijlstra wrote:
> > > The agreement to kill off ia64 wasn't an invitation to kill off other stuff
> > > that people are still working on! Can we please not do this?
> >
> > If you're working on one of them, then surely it's a simple matter of
> > working on adding CONFIG_PREEMPT support :-)
>
> As Geert poined out, I'm not seeing anything particular problematic with the
> architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
> something about organizing KConfig files.
The plan in the parent thread is to remove PREEMPT_NONE and
PREEMPT_VOLUNTARY and only keep PREEMPT_FULL.
> I find it a bit unfair that maintainers of architectures that have huge companies
> behind them use their manpower to urge less popular architectures for removal just
> because they don't have 150 people working on the port so they can keep up with
> design changes quickly.
PREEMPT isn't something new. Also, I don't think the arch part for
actually supporting it is particularly hard, mostly it is sticking the
preempt_schedule_irq() call in return from interrupt code path.
If you convert the arch to generic-entry (a much larger undertaking)
then you get this for free.
On Tue, 19 Sept 2023 at 06:48, John Paul Adrian Glaubitz
<[email protected]> wrote:
>
> As Geert poined out, I'm not seeing anything particular problematic with the
> architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
> something about organizing KConfig files.
It can definitely be problematic.
Not the Kconfig file part, and not the preempt count part itself.
But the fact that it has never been used and tested means that there
might be tons of "this architecture code knows it's not preemptible,
because this architecture doesn't support preemption".
So you may have basic architecture code that simply doesn't have the
"preempt_disable()/enable()" pairs that it needs.
PeterZ mentioned the generic entry code, which does this for the entry
path. But it actually goes much deeper: just do a
git grep preempt_disable arch/x86/kernel
and then do the same for some other architectures.
Looking at alpha, for example, there *are* hits for it, so at least
some of the code there clearly *tries* to do it. But does it cover all
the required parts? If it's never been tested, I'd be surprised if
it's all just ready to go.
I do think we'd need to basically continue to support ARCH_NO_PREEMPT
- and such architectures migth end up with the worst-cast latencies of
only scheduling at return to user space.
Linus
On Tue, Sep 19 2023 at 15:48, John Paul Adrian Glaubitz wrote:
> On Tue, 2023-09-19 at 15:42 +0200, Peter Zijlstra wrote:
>> > The agreement to kill off ia64 wasn't an invitation to kill off other stuff
>> > that people are still working on! Can we please not do this?
>>
>> If you're working on one of them, then surely it's a simple matter of
>> working on adding CONFIG_PREEMPT support :-)
>
> As Geert poined out, I'm not seeing anything particular problematic with the
> architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
> something about organizing KConfig files.
>
> I find it a bit unfair that maintainers of architectures that have huge companies
> behind them use their manpower to urge less popular architectures for removal just
> because they don't have 150 people working on the port so they can keep up with
> design changes quickly.
I don't urge for removal. I just noticed that these four architectures
lack PREEMPT support. The only thing which is missing is the actual
preemption point in the return to kernel code path.
But otherwise it should just work, which I obviously can't confirm :)
Even without that preemption point it should build and boot. There might
be some minor latency issues when that preemption point is not there,
but adding it is not rocket science either. It's probably about 10 lines
of ASM code, if at all.
Though not adding that might cause a blocking issue for the rework of
the whole preemption logic in order to remove the sprinkled around
cond_resched() muck or force us to maintain some nasty workaround just
for the benefit of a few stranglers.
So I can make the same argument the other way around, that it's
unjustified that some architectures which are just supported for
nostalgia throw roadblocks into kernel developemnt.
If my ALPHA foo wouldn't be very close to zero, I'd write that ASM hack
myself, but that's going to cost more of my and your time than it's
worth the trouble,
Hmm. I could delegate that to Linus, he might still remember :)
Thanks,
tglx
On Tue, Sep 19 2023 at 15:21, Anton Ivanov wrote:
> On 19/09/2023 14:42, Peter Zijlstra wrote:
>> If you're working on one of them, then surely it's a simple matter of
>> working on adding CONFIG_PREEMPT support :-)
>
> In the case of UML adding preempt will be quite difficult. I looked at
> this a few years back.
What's so difficult about it?
Thanks,
tglx
On Tue, Sep 19 2023 at 10:25, Linus Torvalds wrote:
> On Tue, 19 Sept 2023 at 06:48, John Paul Adrian Glaubitz
> <[email protected]> wrote:
>>
>> As Geert poined out, I'm not seeing anything particular problematic with the
>> architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
>> something about organizing KConfig files.
>
> It can definitely be problematic.
>
> Not the Kconfig file part, and not the preempt count part itself.
>
> But the fact that it has never been used and tested means that there
> might be tons of "this architecture code knows it's not preemptible,
> because this architecture doesn't support preemption".
>
> So you may have basic architecture code that simply doesn't have the
> "preempt_disable()/enable()" pairs that it needs.
>
> PeterZ mentioned the generic entry code, which does this for the entry
> path. But it actually goes much deeper: just do a
>
> git grep preempt_disable arch/x86/kernel
>
> and then do the same for some other architectures.
>
> Looking at alpha, for example, there *are* hits for it, so at least
> some of the code there clearly *tries* to do it. But does it cover all
> the required parts? If it's never been tested, I'd be surprised if
> it's all just ready to go.
>
> I do think we'd need to basically continue to support ARCH_NO_PREEMPT
> - and such architectures migth end up with the worst-cast latencies of
> only scheduling at return to user space.
The only thing these architectures should gain is the preempt counter
itself, but yes the extra preemption points are not mandatory to
have, i.e. we simply do not enable them for the nostalgia club.
The removal of cond_resched() might cause latencies, but then I doubt
that these museus pieces are used for real work :)
Thanks,
tglx
From: Linus Torvalds
> Sent: 19 September 2023 18:25
>
> On Tue, 19 Sept 2023 at 06:48, John Paul Adrian Glaubitz
> <[email protected]> wrote:
> >
> > As Geert poined out, I'm not seeing anything particular problematic with the
> > architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
> > something about organizing KConfig files.
>
> It can definitely be problematic.
>
> Not the Kconfig file part, and not the preempt count part itself.
>
> But the fact that it has never been used and tested means that there
> might be tons of "this architecture code knows it's not preemptible,
> because this architecture doesn't support preemption".
Do distos even build x86 kernels with PREEMPT_FULL?
I know I've had issues with massive latencies caused graphics driver
forcing write-backs of all the framebuffer memory.
(I think it is a failed attempt to fix a temporary display corruption.)
OTOH SMP support and CONFIG_RT will test most of the code.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
* Thomas Gleixner <[email protected]> wrote:
> On Tue, Sep 19 2023 at 10:25, Linus Torvalds wrote:
> > On Tue, 19 Sept 2023 at 06:48, John Paul Adrian Glaubitz
> > <[email protected]> wrote:
> >>
> >> As Geert poined out, I'm not seeing anything particular problematic with the
> >> architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
> >> something about organizing KConfig files.
> >
> > It can definitely be problematic.
> >
> > Not the Kconfig file part, and not the preempt count part itself.
> >
> > But the fact that it has never been used and tested means that there
> > might be tons of "this architecture code knows it's not preemptible,
> > because this architecture doesn't support preemption".
> >
> > So you may have basic architecture code that simply doesn't have the
> > "preempt_disable()/enable()" pairs that it needs.
> >
> > PeterZ mentioned the generic entry code, which does this for the entry
> > path. But it actually goes much deeper: just do a
> >
> > git grep preempt_disable arch/x86/kernel
> >
> > and then do the same for some other architectures.
> >
> > Looking at alpha, for example, there *are* hits for it, so at least
> > some of the code there clearly *tries* to do it. But does it cover all
> > the required parts? If it's never been tested, I'd be surprised if
> > it's all just ready to go.
> >
> > I do think we'd need to basically continue to support ARCH_NO_PREEMPT
> > - and such architectures migth end up with the worst-cast latencies of
> > only scheduling at return to user space.
>
> The only thing these architectures should gain is the preempt counter
> itself, [...]
And if any of these machines are still used, there's the small benefit of
preempt_count increasing debuggability of scheduling in supposedly
preempt-off sections that were ignored silently previously, as most of
these architectures do not even enable CONFIG_DEBUG_ATOMIC_SLEEP=y in their
defconfigs:
$ for ARCH in alpha hexagon m68k um; do git grep DEBUG_ATOMIC_SLEEP arch/$ARCH; done
$
Plus the efficiency of CONFIG_DEBUG_ATOMIC_SLEEP=y is much reduced on
non-PREEMPT kernels to begin with: it will basically only detect scheduling
in hardirqs-off critical sections.
So IMHO there's a distinct debuggability & robustness plus in enabling the
preemption count on all architectures, even if they don't or cannot use the
rescheduling points.
> [...] but yes the extra preemption points are not mandatory to have, i.e.
> we simply do not enable them for the nostalgia club.
>
> The removal of cond_resched() might cause latencies, but then I doubt
> that these museus pieces are used for real work :)
I'm not sure we should initially remove *explicit* legacy cond_resched()
points, except from high-freq paths where they hurt - and of course remove
them from might_sleep().
Thanks,
Ingo
On 19/09/2023 15:16, Peter Zijlstra wrote:
> On Tue, Sep 19, 2023 at 03:48:09PM +0200, John Paul Adrian Glaubitz wrote:
>> On Tue, 2023-09-19 at 15:42 +0200, Peter Zijlstra wrote:
>>>> The agreement to kill off ia64 wasn't an invitation to kill off other stuff
>>>> that people are still working on! Can we please not do this?
>>>
>>> If you're working on one of them, then surely it's a simple matter of
>>> working on adding CONFIG_PREEMPT support :-)
>>
>> As Geert poined out, I'm not seeing anything particular problematic with the
>> architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
>> something about organizing KConfig files.
>
> The plan in the parent thread is to remove PREEMPT_NONE and
> PREEMPT_VOLUNTARY and only keep PREEMPT_FULL.
>
>> I find it a bit unfair that maintainers of architectures that have huge companies
>> behind them use their manpower to urge less popular architectures for removal just
>> because they don't have 150 people working on the port so they can keep up with
>> design changes quickly.
>
> PREEMPT isn't something new. Also, I don't think the arch part for
> actually supporting it is particularly hard, mostly it is sticking the
> preempt_schedule_irq() call in return from interrupt code path.
That calls local_irq_enable() which does various signal related/irq pending work on UML. That in turn does no like being invoked again (as you may have already been invoked out of that) in the IRQ return path.
So it is likely to end up being slightly more difficult than that for UML - it will need to be wrapped so it can be invoked from the "host" side signal code as well as invoked with some additional checks to avoid making a hash out of the IRQ handling.
It may be necessary to modify some of the existing reentrancy prevention logic in the signal handlers as well and change it to make use of the preempt count instead of its own flags/counters.
>
> If you convert the arch to generic-entry (a much larger undertaking)
> then you get this for free.
>
> _______________________________________________
> linux-um mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-um
>
--
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/
On Tue, Sep 19 2023 at 10:25, Linus Torvalds wrote:
> PeterZ mentioned the generic entry code, which does this for the entry
> path. But it actually goes much deeper: just do a
>
> git grep preempt_disable arch/x86/kernel
>
> and then do the same for some other architectures.
>
> Looking at alpha, for example, there *are* hits for it, so at least
> some of the code there clearly *tries* to do it. But does it cover all
> the required parts? If it's never been tested, I'd be surprised if
> it's all just ready to go.
Interestingly enough m68k has zero instances, but it supports PREEMPT on
the COLDFIRE subarchitecture...
On Tue, Sep 19, 2023, at 10:16, Peter Zijlstra wrote:
> On Tue, Sep 19, 2023 at 03:48:09PM +0200, John Paul Adrian Glaubitz wrote:
>> On Tue, 2023-09-19 at 15:42 +0200, Peter Zijlstra wrote:
>> > > The agreement to kill off ia64 wasn't an invitation to kill off other stuff
>> > > that people are still working on! Can we please not do this?
>> >
>> > If you're working on one of them, then surely it's a simple matter of
>> > working on adding CONFIG_PREEMPT support :-)
>>
>> As Geert poined out, I'm not seeing anything particular problematic with the
>> architectures lacking CONFIG_PREEMPT at the moment. This seems to be more
>> something about organizing KConfig files.
>
> The plan in the parent thread is to remove PREEMPT_NONE and
> PREEMPT_VOLUNTARY and only keep PREEMPT_FULL.
...
>
> PREEMPT isn't something new. Also, I don't think the arch part for
> actually supporting it is particularly hard, mostly it is sticking the
> preempt_schedule_irq() call in return from interrupt code path.
>
> If you convert the arch to generic-entry (a much larger undertaking)
> then you get this for free.
I checked the default configurations for both in-kernel targets and
general-purpose distros and was surprised to learn that very few
actually turn on full preemption by default:
- All distros I looked at (rhel, debian, opensuse) use PREEMPT_VOLUNTARY
by default, though they usually also set PREEMPT_DYNAMIC to let users
override it at boot time.
- The majority (220) of all defconfig files in the kernel don't select
any preemption options, and just get PREEMPT_NONE automatically.
This includes the generic configs for armv7, s390 and mips.
- A small number (24) set PREEMPT_VOLUNTARY, but this notably
includes x86 and ppc64. x86 is the only one of those that sets
PREEMPT_DYNAMIC
- CONFIG_PREEMPT=y (full preemption) is used on 89 defconfigs,
including arm64 and a lot of the older arm32, arc and
mips platforms.
If we want to have a chance of removing both PREEMPT_NONE and
PREEMPT_VOLUNTARY, I think we should start with changing the
defaults first, so defconfigs that don't specify anything else
get PREEMPT=y, and distros that use PREEMPT_VOLUNTARY use it
use it in the absence of a command line argument. If that
doesn't cause too many regressions, the next step might be
to hide the choice under CONFIG_EXPERT until all m68k and
alpha no longer require PREEMPT_NONE.
Arnd