On Fri, May 29, 2009 at 01:01:18PM +0100, George Dunlap wrote:
>
> If we take him at his word, that the root issue is that he fundamentally
> dislikes the design choice of running Linux-as-hypervisor-component,
> then we have a difference of opinion and we're just going to have to
> agree to disagree. But there are reasons to include it anyway,
> including benefits to existing Xen users and potential Xen users (who
> have decided not to use KVM for whatever reason), and the idea of
> survival-of-the-fittest: Xen and KVM have made different design choices,
> let's let them both grow and see which one thrives. If KVM's design is
> unilaterally superior, eventually Xen will die off. But I suspect that
> there's significant demand in the OSS virtualization ecology for both
> approaches, and the world will be the worse for dom0 support being
> out-of-tree.
>
Three years ago, when I was hired by Red Hat, I was put on the Virt team,
and I had to work on Xen. I found it an awkward community to say the least.
But I'll refrain from talking about that experience.
Before I was hired, I was full time developing the -rt patch. I was accustom
to the way the Linux development worked, and felt comfortable with it. I was
very pleased when I left the virt team to go back to work on the -rt patch.
Just before I left, KVM came out. I started playing with it and I once again
felt comfortable in that development. I probably would not have mind working
in the virt team if it was KVM that I was working on. I guess the point I'm
trying to make here is that KVM is developed in a Linux community, Xen is not.
The major difference between KVM and Xen is that KVM _is_ part of Linux. Xen
is not. The reason that this matters is that if we need to make a change to
the way Linux works we can simply make KVM handle the change. That is, you
could think of it as Dom0 and the hypervisor would always be in sync.
If we were to break an interface with Dom0 for Xen then we would have a bunch
of people crying foul about us breaking a defined API. One of Thomas's complaints
(and a valid one) is that once Linux supports an external API it must always
keep it compatible. This will hamper new development in Linux if the APIs are
scattered throughout the kernel without much thought.
Now here's a crazy solution. Merge the Xen hypervisor into Linux ;-)
Give full ownership of Xen to the Linux community. One of your people could be
a maintainer. This way the API between Dom0 and the hypervisor would be an internal
one. If you needed to upgrade Dom0, you also must upgrade the hypervisor, but that
would be fine since the hypervisor would also be in the Kernel proper.
This may not solve all the issues that the x86 maintainers have with the Dom0
patches, but it may help solve the API one.
Yeah, I know, I'll be having snowball fights with Saddam before that happens.
-- Steve
* Steven Rostedt <[email protected]> wrote:
> Now here's a crazy solution. Merge the Xen hypervisor into Linux
> ;-)
That's not that crazy - it's the right technical solution if DOM0 is
desired for upstream. From what i've seen in DOM0 land the incestous
dependencies are really only long-term manageable if the whole thing
is in a single tree.
A lot of Xen legacies could be dropped: the crazy ring1 hack on
32-bit, the various wide interfaces to make pure-software
virtualization limp along. All major CPUs shipped with hardware
virtualization support in the past 2-3 years, so the availability of
VMX and SVM can be taken for granted for such a project.
That cuts down on a fair amount of crap. A lot of code on the Linux
side could be reused, and a pure CONFIG_PCI=y (all other things
disabled) would provide a "slim hypervisor" instance with a very
small and concentrated code base. (That 'slim hypervisor' might even
be built with CONFIG_NOMMU.)
That way dom0 would be a natural extension: a minimal interface
between Linux-Xen-minimal and the dom0 guest instance.
It's a sane technical model IMO, and makes dom0 a lot more
palatable. Having in-tree competition to KVM would also obviously be
good to Linux in general.
Ingo
On Tue, 2 Jun 2009, Steven Rostedt wrote:
> If we were to break an interface with Dom0 for Xen then we would have a bunch
> of people crying foul about us breaking a defined API. One of Thomas's complaints
> (and a valid one) is that once Linux supports an external API it must always
> keep it compatible. This will hamper new development in Linux if the APIs are
> scattered throughout the kernel without much thought.
>
> Now here's a crazy solution. Merge the Xen hypervisor into Linux ;-)
Not that crazy as you might think.
> Give full ownership of Xen to the Linux community. One of your people could be
> a maintainer. This way the API between Dom0 and the hypervisor would be an internal
s/API/ABI/ :)
> one. If you needed to upgrade Dom0, you also must upgrade the hypervisor, but that
> would be fine since the hypervisor would also be in the Kernel proper.
>
> This may not solve all the issues that the x86 maintainers have with the Dom0
> patches, but it may help solve the API one.
In fact it would resolve the ABI problem once and forever as we could
fix hypervisor / dom0 in sync. hypervisor and dom0 need to run in
lock-step anyway if you want to make useful progress aside of
maintaining versioned interfaces which are known to bloat rapidly.
It's not a big deal to set a flag day which says: update hypervisor
and (dom0) kernel in one go.
Thanks,
tglx
That sound you heard was 10000 [email protected]
all having heart attacks at once.
Need I say more.
> -----Original Message-----
> From: Ingo Molnar [mailto:[email protected]]
> Sent: Tuesday, June 02, 2009 5:29 PM
> To: Steven Rostedt
> Cc: George Dunlap; David Miller; [email protected]; Dan Magenheimer;
> [email protected]; [email protected]; [email protected];
> [email protected]; Keir Fraser;
> [email protected]; [email protected]; Kurt Hackel; Ian Pratt;
> [email protected]; ksrinivasan; [email protected];
> [email protected]; Stephen Spector; Jens Axboe;
> [email protected]
> Subject: Merge Xen (the hypervisor) into Linux
>
>
>
> * Steven Rostedt <[email protected]> wrote:
>
> > Now here's a crazy solution. Merge the Xen hypervisor into Linux
> > ;-)
>
> That's not that crazy - it's the right technical solution if DOM0 is
> desired for upstream. From what i've seen in DOM0 land the incestous
> dependencies are really only long-term manageable if the whole thing
> is in a single tree.
>
> A lot of Xen legacies could be dropped: the crazy ring1 hack on
> 32-bit, the various wide interfaces to make pure-software
> virtualization limp along. All major CPUs shipped with hardware
> virtualization support in the past 2-3 years, so the availability of
> VMX and SVM can be taken for granted for such a project.
>
> That cuts down on a fair amount of crap. A lot of code on the Linux
> side could be reused, and a pure CONFIG_PCI=y (all other things
> disabled) would provide a "slim hypervisor" instance with a very
> small and concentrated code base. (That 'slim hypervisor' might even
> be built with CONFIG_NOMMU.)
>
> That way dom0 would be a natural extension: a minimal interface
> between Linux-Xen-minimal and the dom0 guest instance.
>
> It's a sane technical model IMO, and makes dom0 a lot more
> palatable. Having in-tree competition to KVM would also obviously be
> good to Linux in general.
>
> Ingo
>
On Tue, 2 Jun 2009, Dan Magenheimer wrote:
> That sound you heard was 10000 [email protected]
> all having heart attacks at once.
>
> Need I say more.
Well, you might answer the question whether you are the only survivor
of that mass heart attack. In case you are the only one we can simply
assume that 99.99% of the user base is gone and we can stop the merge
discussion completely. Otherwise we try to find the survivors which
have to contribute more than the tabloid pattern.
Thanks,
tglx
[ Speaking as me, no regard to $EMPLOYER ]
On Wed, Jun 03, 2009 at 01:28:43AM +0200, Ingo Molnar wrote:
> A lot of Xen legacies could be dropped: the crazy ring1 hack on
> 32-bit, the various wide interfaces to make pure-software
> virtualization limp along. All major CPUs shipped with hardware
> virtualization support in the past 2-3 years, so the availability of
> VMX and SVM can be taken for granted for such a project.
The biggest reason I personally want Xen to be in mainline is
PVM. Dropping PVM is, to me, pretty much saying "let's merge Xen
without taking the useful parts."
I have only two large machines I control. They're too big to
run as single hosts - it's a waste - but I can leverage cluster testing
by virtualizing them.
The first machine has HVM support. The early kind. It's about
2 years old. It's so dreadfully slow that I had to go to PVM. That
runs at very good speeds and I've stopped noticing the virtualization.
The only problem I have is managing the hypervisor bits, because they're
out of tree.
Now, perhaps that could be fixed. Someone told me that older
HVM boxen can't be fixed; you need a very recent VMX/SVM to perform
well. But if it is fixable, then perhaps future plans shouldn't worry
about it.
The second machine is pre-HVM by a short period. It is not even
three years old. I can't run HVM on it, at all. I can either run PVM
or I can't virtualize. It has fast CPUs and many GB of RAM. I can do
an entire four node cluster test on it, with serious (read, memory
intensive) software. In a PVM-less world, this machine becomes a
single cluster node, and I have to go find three more machines. Of
course, if I had infinite machines, I wouldn't be worrying about this at
all.
So I want to see PVM continue for a long time. I'd like it to
be something I can get with mainline Linux. I don't care if it is dom0,
dom0 and the hypervisor, whatever. I just don't want to have to be
patching out-of-tree patches for a pretty basic functionality.
I don't see 2-3 years as a time frame to assume "everyone has
one." Otherwise, why does Linux have code for x86_32? Everyone's had a
64bit system for at least that long. Sure, that's a straw man. It goes
both ways.
Like Chris said, if we have technical hurdles for Xen to cross,
let's get them out in the open and fixed. If previous Xen developer
interaction has left a bad taste in people's mouths, then the current
crew has to make it up to us. But we have to be willing to notice
they're doing so.
At the end of the day, I want to use Linux on my systems.
Joel
--
"I almost ran over an angel
He had a nice big fat cigar.
'In a sense,' he said, 'You're alone here
So if you jump, you'd best jump far.'"
Joel Becker
Principal Software Developer
Oracle
E-mail: [email protected]
Phone: (650) 506-8127
On Tue, 2 Jun 2009, Joel Becker wrote:
> [ Speaking as me, no regard to $EMPLOYER ]
>
> On Wed, Jun 03, 2009 at 01:28:43AM +0200, Ingo Molnar wrote:
>> A lot of Xen legacies could be dropped: the crazy ring1 hack on
>> 32-bit, the various wide interfaces to make pure-software
>> virtualization limp along. All major CPUs shipped with hardware
>> virtualization support in the past 2-3 years, so the availability of
>> VMX and SVM can be taken for granted for such a project.
>
> The biggest reason I personally want Xen to be in mainline is
> PVM. Dropping PVM is, to me, pretty much saying "let's merge Xen
> without taking the useful parts."
> So I want to see PVM continue for a long time. I'd like it to
> be something I can get with mainline Linux. I don't care if it is dom0,
> dom0 and the hypervisor, whatever. I just don't want to have to be
> patching out-of-tree patches for a pretty basic functionality.
> I don't see 2-3 years as a time frame to assume "everyone has
> one." Otherwise, why does Linux have code for x86_32? Everyone's had a
> 64bit system for at least that long. Sure, that's a straw man. It goes
> both ways.
it's always easier to continue to support stuff that you already have in
place than it is to add new things.
if the non PVM stuff could be added to the kernel, how much would that
simplify the code needed to support PVM? would that reduce the amount of
effort that the Xen people need to spend to something that would mean that
they would be able to keep up with fairly recent kernels?
or what about getting the non PVM version in, and then making the seperate
argument to add PVM support with a different config option ('xen support
for older CPU's, note there is a performance degredation if this option is
selected'), distros could support Xen in their main kernel package on new
hardware, and users like you could enable the slower version.
David Lang
note: I am not an approver in this process, just an interested observer
(who doesn't use Xen)
On Tue, Jun 02, 2009 at 05:00:21PM -0700, Dan Magenheimer wrote:
> That sound you heard was 10000 [email protected]
> all having heart attacks at once.
>
> Need I say more.
So maybe I'm stupid, but why would they be having heart attacks?
It seems like a decent solutoin to me. What's being proposed would
make the dom0/hypervisor interface an internal once, always subject to
change. What's wrong with that? Presumably the domU/hypervisor
interface would have to be remain stable, but why is the
dom0/hypervisor interface have to be sacred and unchanging? I don't
understand the concern.
- Ted
On Tue, 2 Jun 2009, Theodore Tso wrote:
> On Tue, Jun 02, 2009 at 05:00:21PM -0700, Dan Magenheimer wrote:
> > That sound you heard was 10000 [email protected]
> > all having heart attacks at once.
> >
> > Need I say more.
>
> So maybe I'm stupid, but why would they be having heart attacks?
Maybe because they asked for an apple and got an apple pie?
That is, they are pushing hard for an interface for Dom0, and Ingo just
agreed to take it along with the entire Xen hypervisor ;-)
>
> It seems like a decent solutoin to me. What's being proposed would
> make the dom0/hypervisor interface an internal once, always subject to
> change. What's wrong with that? Presumably the domU/hypervisor
> interface would have to be remain stable, but why is the
> dom0/hypervisor interface have to be sacred and unchanging? I don't
> understand the concern.
I know I said it was a crazy idea, but the craziness was not with the
technical side, or even if it is the correct thing to do. I just don't see
the Xen team cooperating with the Linux team. But maybe those are the old
days. Perhaps the rightful place for the Xen hypervisor is in Linux. Xen
is GPL right? Thus we could do this even with out the permission from
Citrix.
The Dom0 push of Xen just seems too much like Linux being Xen's sex
slave, when it should be the other way around. By Linux acquiring the Xen
hypervisor, then I can imaging much more progress in the area of Xen. KVM
may be a competitor, but the two may also be able to share code thus both
could benefit.
I'm not as turned off by Paravirt as others (although I've had my cursing
at it), but with Xen inside Linux, we can tame the damage. Progress of Xen
would speed up since there would be no barrier with the changes in Linux
with the changes in Xen. That is, they will always be compatible.
-- Steve
> > On Tue, Jun 02, 2009 at 05:00:21PM -0700, Dan Magenheimer wrote:
> > > That sound you heard was 10000 [email protected]
> > > all having heart attacks at once.
> > >
> > > Need I say more.
> >
> > So maybe I'm stupid, but why would they be having heart attacks?
>
> Maybe because they asked for an apple and got an apple pie?
>
> That is, they are pushing hard for an interface for Dom0, and
> Ingo just
> agreed to take it along with the entire Xen hypervisor ;-)
Um, no, he did not. He and Avi suggested that Xen be completely
rearchitected to suit Linux's preferences.
A hypervisor is not an operating system. Yes there is
similarity in a number of pieces of code. But there's
some similarity between Java and Linux too...
> Perhaps the rightful place for the Xen hypervisor is in
> Linux. Xen
> is GPL right? Thus we could do this even with out the permission from
> Citrix.
(tongue firmly in cheek in case you might assume otherwise)
Linux is GPL right? Perhaps the rightful place for the Linux
operating system is part of Java. Thus we could do this even
with out the permission from Ingo.
> I just don't see
> the Xen team cooperating with the Linux team. But maybe those
> are the old days.
Yes, let's fix that. Let's start turning this discussion towards
how we can cooperate better.
> The Dom0 push of Xen just seems too much like Linux being Xen's sex
> slave, when it should be the other way around.
I can certainly see how it might feel that way, but it needn't
be... nor the other way around. But in the end, only the end users
matter. If we can't cooperate, we simply cede the war to Windows
and Hyper-V.
From: Dan Magenheimer <[email protected]>
Date: Tue, 2 Jun 2009 21:49:58 -0700 (PDT)
> A hypervisor is not an operating system.
This is a pretty bogus statement if you ask me.
A hypervisor a software system that provides seperation between
protection realms.
It also handles exceptions and "system calls" on behalf of the other
protection realms.
I personally don't see the difference at all. And since many
hypervisors even do cpu scheduling, the fundamental differences
converge to almost nothing.
On Tue, 2 Jun 2009, David Miller wrote:
> From: Dan Magenheimer <[email protected]>
> Date: Tue, 2 Jun 2009 21:49:58 -0700 (PDT)
>
> > A hypervisor is not an operating system.
>
> This is a pretty bogus statement if you ask me.
>
> A hypervisor a software system that provides seperation between
> protection realms.
>
> It also handles exceptions and "system calls" on behalf of the other
> protection realms.
>
> I personally don't see the difference at all. And since many
> hypervisors even do cpu scheduling, the fundamental differences
> converge to almost nothing.
I recently sat in an Operating Systems class where the Professor was an
old IBM retiree, that worked on the 390 system way back when. He would
argue the point that an Operating System must do at least two things,
schedule tasks and manage paging. The Xen hypervisor does both, thus in
his eyes, it is indeed an Operating System.
-- Steve
P.S. he also thought that filesystem management does not have to be a
duty of the OS and he hated the fact he had to teach it ;-)
On Tue, 2 Jun 2009, Dan Magenheimer wrote:
> > > On Tue, Jun 02, 2009 at 05:00:21PM -0700, Dan Magenheimer wrote:
> > > > That sound you heard was 10000 [email protected]
> > > > all having heart attacks at once.
> > > >
> > > > Need I say more.
> > >
> > > So maybe I'm stupid, but why would they be having heart attacks?
> >
> > Maybe because they asked for an apple and got an apple pie?
> >
> > That is, they are pushing hard for an interface for Dom0, and
> > Ingo just
> > agreed to take it along with the entire Xen hypervisor ;-)
>
> Um, no, he did not. He and Avi suggested that Xen be completely
> rearchitected to suit Linux's preferences.
I was being a bit tongue in cheek with that comment too.
>
> A hypervisor is not an operating system.
You say potato I say potato (Hmm, that doesn't work in text)
> Yes there is
> similarity in a number of pieces of code. But there's
> some similarity between Java and Linux too...
Java can run on hardware?
>
> > Perhaps the rightful place for the Xen hypervisor is in
> > Linux. Xen
> > is GPL right? Thus we could do this even with out the permission from
> > Citrix.
>
> (tongue firmly in cheek in case you might assume otherwise)
> Linux is GPL right? Perhaps the rightful place for the Linux
> operating system is part of Java. Thus we could do this even
> with out the permission from Ingo.
If Java became GPL it could very well do that.
>
> > I just don't see
> > the Xen team cooperating with the Linux team. But maybe those
> > are the old days.
>
> Yes, let's fix that. Let's start turning this discussion towards
> how we can cooperate better.
Sure.
>
> > The Dom0 push of Xen just seems too much like Linux being Xen's sex
> > slave, when it should be the other way around.
>
> I can certainly see how it might feel that way, but it needn't
> be... nor the other way around. But in the end, only the end users
> matter. If we can't cooperate, we simply cede the war to Windows
> and Hyper-V.
When I suggest that Xen be merged into Linux, I did not mean it had to be
like KVM or lguest where the Linux would boot up and run Xen. I mean that
Xen could still be a micro kernel. The difference would be that its source
would live in the kernel proper. linux.git/xen? This way the ABI between
Xen and Dom0 would always be in sync.
We could even link it in to the vmlinuz, instead of needing the separate
xen.gz to load first. The vmlinuz could then expand into a Xen
hypervisor, and also load the Dom0 with it. One image for both entities.
If you want Dom0 ABI in, you have to expect it to change without notice.
If this breaks Xen, then we don't want to hear any complaints. This means
that users of Xen would need to make sure that they have both the most
recent on hypervisor and kernel and hope that they match.
With the combined image we then get the two to always be together, and no
problems with the users.
What's the issue with this? You get to keep your "micro hypervisor" design
that has been stated to be the superior method.
-- Steve
Hi,
> It seems like a decent solutoin to me. What's being proposed would
> make the dom0/hypervisor interface an internal once, always subject to
> change. What's wrong with that?
Linux is not the only player here. NetBSD can run as dom0 guest.
Solaris can run as dom0 guest too. Thus making the dom0/xen interface
private to linux and xen isn't going to fly.
cheers,
Gerd
> The biggest reason I personally want Xen to be in mainline is
> PVM. Dropping PVM is, to me, pretty much saying "let's merge Xen
> without taking the useful parts."
PVM is and has been for a long time a messaging parallel machine. Can you
not misuse the abbreviation in confusing ways (especially in email I read
in the morning ;))
Merging just hardware assisted vm support initially might be a perfectly
sensible path.
> Like Chris said, if we have technical hurdles for Xen to cross,
> let's get them out in the open and fixed. If previous Xen developer
> interaction has left a bad taste in people's mouths, then the current
> crew has to make it up to us. But we have to be willing to notice
> they're doing so.
Start by changing the mentality. Right now much of the patched code looks
like "We made a decision years ago when creating Xen. Now we need to
force that code we wrote into Linux somehow".
Stuff gets merged a lot better if the thinking is "how do we make the
minimal changes to the existing kernel, cleanly and with minimal
inter-relationships". Only after that do you worry about whether
the existing in kernel interfaces are right.
There is a simple reason for this: Changing an interface in the kernel is
a consensus finding process around all visible users of the interface.
It's much easier to do that as a follow up. That way you can bench
alternatives, test if it harms any of the users and merge change sets
that span all the various users of the interface in one go.
It's also frequently the case that when you have a simple clean interface
that doesn't fit some in tree users it becomes blindly obvious what it
should look like.
So I would suggest the path is
- Use existing interfaces
- Merge chunks of the Xen code without worrying too much about performance
in Xen but worry in detail about bare metal performance
- Don't worry about "hard" problems initially - eg with PAE just use the
paravirt CPUID hook and deny having PAE to begin with
- Where there isn't a clean simple interface try as hard as possible to
build some glue code using existing interfaces in the kernel
When it works, doesn't harm bare metal performance and is merged then go
back and worry about the harder stuff, optimisation and fine tuning. It
doesn't even need to be able to run all guests or all configurations
initially.
Also please can folks get out of the "how do we merge Xen" mentality into
the "How do we create dom0 functionality for Xen in Linux" - don't
pre-suppose the existing implementation is right.
Alan
Ingo Molnar wrote:
> A lot of Xen legacies could be dropped: the crazy ring1 hack on
> 32-bit, the various wide interfaces to make pure-software
> virtualization limp along. All major CPUs shipped with hardware
> virtualization support in the past 2-3 years, so the availability of
> VMX and SVM can be taken for granted for such a project.
What a great idea, and while we're doing this let's also drop support
for legacy stuff like PATA and i8042 in mainline. Noone will need it
anyway because their successors are on the market for years... let's
just take it for granted that everyone is using SATA and USB nowadays!
Best regards,
Christian
> Linux is not the only player here. NetBSD can run as dom0 guest.
> Solaris can run as dom0 guest too. Thus making the dom0/xen interface
> private to linux and xen isn't going to fly.
It does not however preclude fixing the dom0 interface.
Anyway we deal with unfixable interfaces on a regular basis with device
hardware. What we don't do is screw up the kernel handling garbage
hardware. We dump the adaption on the driver.
Same with Xen, impedance matching Xen's interface with the kernel is (at
least initialy) something that belongs entirely in the Xen glue, or to
get started initially by just turning off stuff.
MTRR, PAE etc can all be turned off for the purpose an initial merge.
On 06/03/09 10:47, Alan Cox wrote:
>> Linux is not the only player here. NetBSD can run as dom0 guest.
>> Solaris can run as dom0 guest too. Thus making the dom0/xen interface
>> private to linux and xen isn't going to fly.
>
> It does not however preclude fixing the dom0 interface.
It wasn't my intention to imply that. The interface can be extended
when needed. PAT support will probably be such a case. Changing it in
incompatible ways isn't going to work though.
> MTRR, PAE etc can all be turned off for the purpose an initial merge.
s/PAE/PAT/? PAE is mandatory ...
Having not-yet supported stuff disabled initially is sensible IMHO. Can
be done for MTRR and PAT. Is already done for MSI ;)
The lapic/ioapic stuff must be sorted though because otherwise you can't
boot the box at all. I think the same is true for the swiotlb bits.
cheers,
Gerd
On 03/06/2009 10:09, "Gerd Hoffmann" <[email protected]> wrote:
> On 06/03/09 10:47, Alan Cox wrote:
>>> Linux is not the only player here. NetBSD can run as dom0 guest.
>>> Solaris can run as dom0 guest too. Thus making the dom0/xen interface
>>> private to linux and xen isn't going to fly.
>>
>> It does not however preclude fixing the dom0 interface.
>
> It wasn't my intention to imply that. The interface can be extended
> when needed. PAT support will probably be such a case. Changing it in
> incompatible ways isn't going to work though.
We're happy to change interfaces where we agree that makes sense.
Compatibility is our own (Xen's) problem of course, and it's generally not
an insurmountable problem -- worst case we can launch dom0 in a varying
environment dependent on a Xen-specific elf note, for example.
-- Keir
On Wed, Jun 03, 2009 at 11:09:39AM +0200, Gerd Hoffmann wrote:
> On 06/03/09 10:47, Alan Cox wrote:
>>> Linux is not the only player here. NetBSD can run as dom0 guest.
>>> Solaris can run as dom0 guest too. Thus making the dom0/xen interface
>>> private to linux and xen isn't going to fly.
>>
>> It does not however preclude fixing the dom0 interface.
>
> It wasn't my intention to imply that. The interface can be extended
> when needed. PAT support will probably be such a case. Changing it in
> incompatible ways isn't going to work though.
But that means that if there is some fundamentally broken piece of
dom0 design, that the Linux kernel will be stuck with it ***forever***
and it will contaminate code paths and make the code harder to
maintain ***forever*** if we consent to the Xen merge? Is that really
what you are saying? Be careful how you answer that....
- Ted
On 03/06/2009 12:15, "Theodore Tso" <[email protected]> wrote:
>>> It does not however preclude fixing the dom0 interface.
>>
>> It wasn't my intention to imply that. The interface can be extended
>> when needed. PAT support will probably be such a case. Changing it in
>> incompatible ways isn't going to work though.
>
> But that means that if there is some fundamentally broken piece of
> dom0 design, that the Linux kernel will be stuck with it ***forever***
> and it will contaminate code paths and make the code harder to
> maintain ***forever*** if we consent to the Xen merge? Is that really
> what you are saying? Be careful how you answer that....
It's not true, if you are prepared for a new dom0 kernel to require a new
version of Xen (which seems not unreasonable). We're happy to make
reasonable interface changes, and deal with compatibility issues as
necessary within Xen.
-- Keir
On 06/03/09 13:15, Theodore Tso wrote:
> On Wed, Jun 03, 2009 at 11:09:39AM +0200, Gerd Hoffmann wrote:
>> It wasn't my intention to imply that. The interface can be extended
>> when needed. PAT support will probably be such a case. Changing it in
>> incompatible ways isn't going to work though.
>
> But that means that if there is some fundamentally broken piece of
> dom0 design, that the Linux kernel will be stuck with it ***forever***
> and it will contaminate code paths and make the code harder to
> maintain ***forever*** if we consent to the Xen merge?
No. Xen is stuck with it forever (or at least for a few releases).
Even when adding new & better dom0/xen interfaces in the merge process
Xen has to keep the old ones to handle the other dom0 guests (NetBSD,
Solaris, old 2.6.18 out-of-tree linux kernel). Pretty much like the
linux kernel has to keep old syscalls to not break the ABI for the
applications, xen has to maintain old hypercalls[1].
Other way around: Apps can use new system calls only when running one
recent kernels, and they have to deal with -ENOSYS. Likewise it might
be that the pv_ops-based dom0 kernel can provide some features only when
running on a recent hypervisor. That will likely be the case for PAT.
cheers,
Gerd
[1] and other interfaces like trap'n'emulate certain instructions.
Steven Rostedt wrote:
> What's the issue with this? You get to keep your "micro hypervisor" design
> that has been stated to be the superior method.
>
It is a very interesting idea, but it would still be basically a
completely new project. If someone started such a project, they could
probably cannibalize a lot of Xen's existing code (a funny boomerang,
since Xen cannibalized Linux's code when it started), but it would still
require a lot of work and re-writing, and the result would be a lot
different than Xen is now. It would be years before it was ready to be
used in a production system. It's not really realistic to expect all
the Xen developers and users to drop Xen development, shift gears into
this new project, and wait until it's ready to be used. (That's not to
say that the idea has no merit, just that Xen as it is wouldn't go away
until it this hypothetical linux hypervisor component was mature enough
for users and developers to jump onto.)
Yeah, lots of interesting implications for such a project.
Having a separate component to be a hypervisor, even if in the same
tree, would mean we could have dedicated hypervisor schedulers, &c.
They could (conceivably) work more closely with the dom0 scheduler to
make things more efficient.
As others have said, it would limit the ability of such a hypervisor to
be used with other dom0 operatings systems. Fixing the ABI sufficiently
so that others can use it might be possible, but it seems to me unlikely
to meet with much success without a lot of committment on both sides
(i.e., w/in Linux and within other OS communities).
I'm not sure that it would turn out quite the way some people expect,
though. From a technical perspective, I'm not sure getting rid of the
"ring 1 hack" or requiring HVM support would be the best design choice
for such a project. And it's hard to predict what kinds of technical,
political, or cultural issues, directions, or potential dead-ends a
project might take.
From all angles, it's too risky to just abandon the current Xen
codebase until this hypothetical linux hypervisor component has shown
itself to be viable.
-George
Ingo Molnar wrote:
> A lot of Xen legacies could be dropped: the crazy ring1 hack on
> 32-bit, the various wide interfaces to make pure-software
> virtualization limp along. All major CPUs shipped with hardware
> virtualization support in the past 2-3 years, so the availability of
> VMX and SVM can be taken for granted for such a project.
That's a pretty bold statement. I have five x86 machines in my house
currently being used, and none of them support VMX/SVM.
At least some Lenovo laptops disable VMX in the BIOS with no way to
enable it. Some of the Core2Duo chips don't support VMX at all.
I think Xen without paravirtualization would be a serious degradation of
usefulness.
Chris
On Wed, 03 Jun 2009 11:31:13 -0600
"Chris Friesen" <[email protected]> wrote:
> Ingo Molnar wrote:
>
> > A lot of Xen legacies could be dropped: the crazy ring1 hack on
> > 32-bit, the various wide interfaces to make pure-software
> > virtualization limp along. All major CPUs shipped with hardware
> > virtualization support in the past 2-3 years, so the availability of
> > VMX and SVM can be taken for granted for such a project.
>
> That's a pretty bold statement. I have five x86 machines in my house
> currently being used, and none of them support VMX/SVM.
>
> At least some Lenovo laptops disable VMX in the BIOS with no way to
> enable it. Some of the Core2Duo chips don't support VMX at all.
Ditto some Atom cpus which in turn means you can't run kvm on all the
netbooks right now - which is one place its very useful.
> I think Xen without paravirtualization would be a serious degradation of
> usefulness.
At that point you can just use kvm anyway.
On Wed, Jun 03, 2009 at 01:03:51PM +0100, George Dunlap wrote:
> It is a very interesting idea, but it would still be basically a
> completely new project. If someone started such a project, they could
> probably cannibalize a lot of Xen's existing code (a funny boomerang,
> since Xen cannibalized Linux's code when it started), but it would still
> require a lot of work and re-writing, and the result would be a lot
> different than Xen is now. It would be years before it was ready to be
> used in a production system.
You might be surprised; if we started with a working dom0/xen pair,
and there were people working on it to clean up dom0/xen interface,
treating it as an internal Linux interface with an eye towards
minimizing contamination of core kernel code, the Linux model of
development can go pretty fast. Compare and contrast it with the
***years*** of calendar time and decades of wasted man-years of
engineering effort needed to port and backport and maintain dom0
support with Linux. Given that experience, I could easily see how
some might assume that it would take years to significantly improve
things, but I suspect if xen were merged into mainline with the
assumption that it could be arbitrarily changed to make things sane,
with the primary interface that needed backwards compatibility care
being the xen/domU interface, I expect things would go pretty fast.
What would be lost is dom0 support for other OS's, but really, is that
such a major loss? Linux has far better device driver support than
Solaris or FreeBSD, so there is really that much gain in using some
other OS for dom0?
- Ted
On Wed, 3 Jun 2009, Christian Tramnitz wrote:
>
> What a great idea, and while we're doing this let's also drop support
> for legacy stuff like PATA and i8042 in mainline. Noone will need it
> anyway because their successors are on the market for years... let's
> just take it for granted that everyone is using SATA and USB nowadays!
Have you noticed how PATA and i8042 don't screw up anything else?
You're totally missing the problem. If Xen was a single driver thing, we
wouldn't have this discussion. But as is, Xen craps all over OTHER PEOPLES
CODE. When those people then aren't interested in Xen, why is anybody
surprised that people aren't excited?
Linus
Linus Torvalds, le Thu 04 Jun 2009 11:53:45 -0700, a ?crit :
> On Wed, 3 Jun 2009, Christian Tramnitz wrote:
> >
> > What a great idea, and while we're doing this let's also drop support
> > for legacy stuff like PATA and i8042 in mainline. Noone will need it
> > anyway because their successors are on the market for years... let's
> > just take it for granted that everyone is using SATA and USB nowadays!
>
> Have you noticed how PATA and i8042 don't screw up anything else?
Right. We should get rid of all the HIGHMEM kmap crap that cripples all
the code.
Samuel
From: Samuel Thibault <[email protected]>
Date: Fri, 5 Jun 2009 02:09:10 +0200
> Linus Torvalds, le Thu 04 Jun 2009 11:53:45 -0700, a ?crit :
>> On Wed, 3 Jun 2009, Christian Tramnitz wrote:
>> >
>> > What a great idea, and while we're doing this let's also drop support
>> > for legacy stuff like PATA and i8042 in mainline. Noone will need it
>> > anyway because their successors are on the market for years... let's
>> > just take it for granted that everyone is using SATA and USB nowadays!
>>
>> Have you noticed how PATA and i8042 don't screw up anything else?
>
> Right. We should get rid of all the HIGHMEM kmap crap that cripples all
> the code.
The kmap interfaces are pretty damn clean if you ask me. Especially
compared to the abortion Xen plops into the x86 platform code.
So, keep searching for an argument where none exists.
On Fri, 5 Jun 2009, Samuel Thibault wrote:
>
> Right. We should get rid of all the HIGHMEM kmap crap that cripples all
> the code.
Now you're starting to understand.
However, the difference between Xen and highmem (which I do hate, and
which took a long time and lots of effort to get done) is how many people
care. And in particular how many kernel developers do.
Until you can face these obvious facts, please just shut up. Ok?
Linsu