by Ingo Molnar

[permalink] [raw]

Subject: Re: x86/Voyager

* James Bottomley <[email protected]> wrote:

> > > But no-one's yet made any argument for why it's a worthwhile thing
> > > to be doing.
> >
> > Because the sub-arch code is butt-ugly.
> >
> > x86 subarchitectures are a hack that should never have gone upstream -
> > and we are now reversing that braindamage, step by step.
> > Subarchitectures are a compile-time "HAL", but a highly
> > non-transparent one at that. They complicates the x86 architecture in
> > a couple of key structures and very fundamentally so - and that
> > results in continued complications in critical areas of the x86 code.
>
> Right: it's a compile time HAL. Your "cleanup" is to convert it to a
> runtime one, which is a lot more complex because voyager still has to
> influence the x86 path in those locations regardless: it's not an x86 PC
> architecture and so is never going to be able to boot through an APIC/MP
> table SMP boot sequence.
>
> I'm not actually bothered by the complexity, though ... it would be cool
> for me to have a kernel that boots on both voyager and a PC. What
> worries me is the cost (both in terms of execution time and maintenance
> burden) this would impose on the standard PC path.

That cost is negligible. Voyager already uses smp_ops which solves most of
the callbacks. The remaining non-slowpath (non-init, non-shutdown, etc.)
bits are, roughly:

flush_tlb_[all|current_task|mm|page]()
smp_apic_timer_interrupt

Which can either reuse PARAVIRT with a Voyager-specific custom template
for these methods, or, if that's easier, we can add x86_quirk handlers as
well.

There is near zero overhead for normal PCs, ~5 functions will have:

if (unlikely(x86_quirks.smp_apic_timer_interrupt)) {
... quirk-path ...
}

type of constructs.

> The other thing I will point out is that if you think a runtime HAL
> simplification could be applied to the current kernel, a compile time
> HAL could equally well be done.

There are numerous advantages. Just a few of them, from the top of my
head:

- The point is to not split the build space on such a fundamental level -
testing is way too complex already.

- Runtime quirks tend to be tested far more than build-time quirks: for
example because a test kernel can just include all the runtime quirks
all at once - while it cannot possibly include all the
subarchitectures.

- It is far more apparent from the source code what happens if quirks are
out and visible. A compile-time 'HAL' is far less transparent: it is
very easy to miss that a function is present in multiple copies and
behaves in different ways, dependent on which subarch we are building
for. Developers tend to concentrate on a single piece of code, so
consolidating code flows is a development advantage.

- Runtime quirks tend to be much more usable. A distro kernel can include
all the quirks with near zero impact. Now in the specific case of
x86/Voyager this is probably not a big factor: as you seem to be owning
the last two (one?) working Voyager box in existence that runs
development kernels? But it is a factor in the general push to
eliminate subarchitectures.

(For similarly good reasons most of the hw quirks in the normal Linux
driver space are done via runtime constructs and not via build-time
constructs.)

Anyway, none of this is really new. We eliminated the ES7000, the VISWS
and the RDC321X subarchitectures already - Voyager is the holdout.

Ingo