32-bit KVM has extra complications in the code due to:
- different ways to write 64-bit values in VMCS
- different handling of DS and ES selectors as well as FS/GS bases
- lack of CR8 and EFER
- lack of XFD
- impossibility of writing 64-bit PTEs atomically
The last is the big one, because it prevents from using the TDP MMU
unconditionally.
Since 32-bit processors with virtualization extensions are a historical
curiosity, and even 32-bit userspace can only deal with small guests
due to limited address space, it makes sense to restrict KVM to 64-bit
hosts and kernels.
In preparation for that make 32-bit KVM depend on CONFIG_BROKEN, and
opportunistically rephrase the help message---processors with
virtualization extensions are supported even if they are not
"fairly recent".
Signed-off-by: Paolo Bonzini <[email protected]>
---
arch/x86/kvm/Kconfig | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index e3cbd7706136..a107df22ffee 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -21,6 +21,7 @@ if VIRTUALIZATION
config KVM
tristate "Kernel-based Virtual Machine (KVM) support"
depends on HAVE_KVM
+ depends on X86_64 || BROKEN
depends on HIGH_RES_TIMERS
depends on X86_LOCAL_APIC
select PREEMPT_NOTIFIERS
@@ -50,12 +51,13 @@ config KVM
select HAVE_KVM_PM_NOTIFIER if PM
help
Support hosting fully virtualized guest machines using hardware
- virtualization extensions. You will need a fairly recent
- processor equipped with virtualization extensions. You will also
- need to select one or more of the processor modules below.
+ virtualization extensions. You will also need to select one or
+ more of the processor modules below.
This module provides access to the hardware capabilities through
- a character device node named /dev/kvm.
+ a character device node named /dev/kvm. You might have to enable
+ virtualization extensions in the firmware setup if the device is
+ not visible.
To compile this as a module, choose M here: the module
will be called kvm.
--
2.31.1
On Mon, Sep 26, 2022, Paolo Bonzini wrote:
> 32-bit KVM has extra complications in the code due to:
>
> - different ways to write 64-bit values in VMCS
>
> - different handling of DS and ES selectors as well as FS/GS bases
>
> - lack of CR8 and EFER
>
> - lack of XFD
>
More for the list:
- SVM is effectively restricted to PAE kernels due to NX requirements
> - impossibility of writing 64-bit PTEs atomically
It's not impossible, just ugly. KVM could use CMPXCHG8B to do all of the accesses
for the TDP MMU, including the non-atomic reads and writes.
> The last is the big one, because it prevents from using the TDP MMU
> unconditionally.
As above, if the TDP MMU really is the sticking point, that's solvable.
The real justification for deprecating 32-bit KVM is that, outside of KVM developers,
literally no one uses 32-bit KVM. I.e. any amount of effort that is required to
continue supporting 32-bit kernels is a complete waste of resources.
On Tue, 2022-09-27 at 17:07 +0000, Sean Christopherson wrote:
> On Mon, Sep 26, 2022, Paolo Bonzini wrote:
> > 32-bit KVM has extra complications in the code due to:
> >
> > - different ways to write 64-bit values in VMCS
> >
> > - different handling of DS and ES selectors as well as FS/GS bases
> >
> > - lack of CR8 and EFER
> >
> > - lack of XFD
> >
>
> More for the list:
>
> - SVM is effectively restricted to PAE kernels due to NX requirements
>
> > - impossibility of writing 64-bit PTEs atomically
>
> It's not impossible, just ugly. KVM could use CMPXCHG8B to do all of the accesses
> for the TDP MMU, including the non-atomic reads and writes.
>
> > The last is the big one, because it prevents from using the TDP MMU
> > unconditionally.
>
> As above, if the TDP MMU really is the sticking point, that's solvable.
>
> The real justification for deprecating 32-bit KVM is that, outside of KVM developers,
> literally no one uses 32-bit KVM. I.e. any amount of effort that is required to
> continue supporting 32-bit kernels is a complete waste of resources.
>
I also think that outside KVM developers nobody should be using KVM on 32 bit host.
However for _developement_ I think that 32 bit KVM support is very useful, as it
allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while,
and can even probably be useful to some users (e.g running some legacy stuff in a VM,
which includes a hypervisor, especially to run really legacy OSes / custom bare metal software,
using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but
other 32 bit nested hypervisors can be useful.
Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long
as current kernel works, it is useful to use the same kernel on host and guest.
Best regards,
Maxim Levitsky
On 9/28/22 09:10, Maxim Levitsky wrote:
> I also think that outside KVM developers nobody should be using KVM on 32 bit host.
>
> However for_developement_ I think that 32 bit KVM support is very useful, as it
> allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while,
> and can even probably be useful to some users (e.g running some legacy stuff in a VM,
> which includes a hypervisor, especially to run really legacy OSes / custom bare metal software,
> using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but
> other 32 bit nested hypervisors can be useful.
>
> Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long
> as current kernel works, it is useful to use the same kernel on host and guest.
Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT
reinjection. :) But really the ultimate solution to this would be to
improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit.
Paolo
On 9/27/22 19:07, Sean Christopherson wrote:
> On Mon, Sep 26, 2022, Paolo Bonzini wrote:
>> 32-bit KVM has extra complications in the code due to:
>>
>> - different ways to write 64-bit values in VMCS
>>
>> - different handling of DS and ES selectors as well as FS/GS bases
>>
>> - lack of CR8 and EFER
>>
>> - lack of XFD
>>
>
> More for the list:
>
> - SVM is effectively restricted to PAE kernels due to NX requirements
True, but I'm not sure how it complicates the code? Do you mean having
to kmap/kunmap, and if so are you thinking of making KVM depend on !HIGHMEM?
>> - impossibility of writing 64-bit PTEs atomically
>
> It's not impossible, just ugly. KVM could use CMPXCHG8B to do all of the accesses
> for the TDP MMU, including the non-atomic reads and writes.
Ok, rephrased:
==========
Breakage in 32-bit KVM is somewhat rare, but it did happen and
developers themselves found out a few months later. This means that it
is very unlikely that it has any users. 32-bit processors with
virtualization extensions are a historical curiosity; 32-bit userspace
can only deal with small guests due to limited address space.
Hence, it makes sense to restrict x86 KVM to 64-bit hosts and kernels.
This removes all the conditional code to deal with the above
differences, and it also enables unconditional use of the TDP MMU:
making it work on 32-bit systems would require contortions using
CMPXCHG8B to write 64-bit PTEs atomically, and they are simply not
worth it.
==========
>> The last is the big one, because it prevents from using the TDP MMU
>> unconditionally.
>
> As above, if the TDP MMU really is the sticking point, that's solvable.
Yeah, but until now the maintainability cost of keeping 32-bit on life
support was small. Using CMPXCHG8B in the TDP MMU is possibly worse
than having two MMUs for the two-dimensional paging case, therefore
dropping the old x86 direct MMU for me is what tips the balance in favor
of removal.
(Once David submits his MMU callbacks work, I still want to see if it's
possible to preserve the "old" x86 direct MMU, for example for use in
pKVM; however, it would not be in arch/x86/).
Paolo
> The real justification for deprecating 32-bit KVM is that, outside of KVM developers,
> literally no one uses 32-bit KVM. I.e. any amount of effort that is required to
> continue supporting 32-bit kernels is a complete waste of resources.
>
On Wed, Sep 28, 2022, Paolo Bonzini wrote:
> On 9/28/22 09:10, Maxim Levitsky wrote:
> > I also think that outside KVM developers nobody should be using KVM on 32 bit host.
> >
> > However for_developement_ I think that 32 bit KVM support is very useful, as it
> > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while,
> > and can even probably be useful to some users (e.g running some legacy stuff in a VM,
> > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software,
> > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but
> > other 32 bit nested hypervisors can be useful.
> >
> > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long
> > as current kernel works, it is useful to use the same kernel on host and guest.
>
> Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT
> reinjection. :) But really the ultimate solution to this would be to
> improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit.
Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors,
but the maintenance cost is painfully high.
On Wed, 2022-09-28 at 16:12 +0000, Sean Christopherson wrote:
> On Wed, Sep 28, 2022, Paolo Bonzini wrote:
> > On 9/28/22 09:10, Maxim Levitsky wrote:
> > > I also think that outside KVM developers nobody should be using KVM on 32 bit host.
> > >
> > > However for_developement_ I think that 32 bit KVM support is very useful, as it
> > > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while,
> > > and can even probably be useful to some users (e.g running some legacy stuff in a VM,
> > > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software,
> > > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but
> > > other 32 bit nested hypervisors can be useful.
> > >
> > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long
> > > as current kernel works, it is useful to use the same kernel on host and guest.
> >
> > Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT
> > reinjection. :) But really the ultimate solution to this would be to
> > improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit.
>
> Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors,
> but the maintenance cost is painfully high.
>
But is it actually? I test it routinely and it it does work quite well IMHO.
I don't remember that there were that much breakage lately in this area.
As far as my opinion goes I do volunteer to test this code more often,
and I do not want to see the 32 bit KVM support be removed *yet*.
Best regards,
Maxim Levitsky
On Wed, 2022-09-28 at 16:12 +0000, Sean Christopherson wrote:
> On Wed, Sep 28, 2022, Paolo Bonzini wrote:
> > On 9/28/22 09:10, Maxim Levitsky wrote:
> > > I also think that outside KVM developers nobody should be using KVM on 32 bit host.
> > >
> > > However for_developement_ I think that 32 bit KVM support is very useful, as it
> > > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while,
> > > and can even probably be useful to some users (e.g running some legacy stuff in a VM,
> > > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software,
> > > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but
> > > other 32 bit nested hypervisors can be useful.
> > >
> > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long
> > > as current kernel works, it is useful to use the same kernel on host and guest.
> >
> > Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT
> > reinjection. :) But really the ultimate solution to this would be to
> > improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit.
>
> Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors,
> but the maintenance cost is painfully high.
>
But is is actually? I test it routinely and it it does work quite well IMHO.
As far as my opinion goes I do volunteer to test this code more often,
and I do not want to see the 32 bit KVM support be removed *yet*.
Best regards,
Maxim Levitsky
On Wed, Sep 28, 2022, Maxim Levitsky wrote:
> On Wed, 2022-09-28 at 16:12 +0000, Sean Christopherson wrote:
> > On Wed, Sep 28, 2022, Paolo Bonzini wrote:
> > > On 9/28/22 09:10, Maxim Levitsky wrote:
> > > > I also think that outside KVM developers nobody should be using KVM on 32 bit host.
> > > >
> > > > However for_developement_ I think that 32 bit KVM support is very useful, as it
> > > > allows to smoke test the support for 32 bit nested hypervisors, which I do once in a while,
> > > > and can even probably be useful to some users (e.g running some legacy stuff in a VM,
> > > > which includes a hypervisor, especially to run really legacy OSes / custom bare metal software,
> > > > using an old hypervisor) - or in other words, 32 bit nested KVM is mostly useless, but
> > > > other 32 bit nested hypervisors can be useful.
> > > >
> > > > Yes, I can always use an older 32 bit kernel in a guest with KVM support, but as long
> > > > as current kernel works, it is useful to use the same kernel on host and guest.
> > >
> > > Yeah, I would use older 32 bit kernels just like I use RHEL4 to test PIT
> > > reinjection. :) But really the ultimate solution to this would be to
> > > improve kvm-unit-tests so that we can compile vmx.c and svm.c for 32-bit.
> >
> > Agreed. I too use 32-bit KVM to validate KVM's handling of 32-bit L1 hypervisors,
> > but the maintenance cost is painfully high.
> >
>
> But is it actually? I test it routinely and it it does work quite well IMHO.
> I don't remember that there were that much breakage lately in this area.
Oh, I didn't mean that it actually requires a lot of attention in terms of bug
fixes, what I meant by "maintenance cost" is the cost of testing that all the
flavors of 32-bit KVM actually work. That can be automated to some extent, but
there's a non-trivial cost to maintaining all that automation.
> As far as my opinion goes I do volunteer to test this code more often,
> and I do not want to see the 32 bit KVM support be removed *yet*.
Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
coverage. But I do think it should an "off-by-default" sort of thing. Maybe
BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
On 9/28/22 19:55, Sean Christopherson wrote:
>> As far as my opinion goes I do volunteer to test this code more often,
>> and I do not want to see the 32 bit KVM support be removed*yet*.
>
> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
> coverage. But I do think it should an "off-by-default" sort of thing. Maybe
> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
what's wrong with booting an older guest?
Paolo
On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
> On 9/28/22 19:55, Sean Christopherson wrote:
> > > As far as my opinion goes I do volunteer to test this code more often,
> > > and I do not want to see the 32 bit KVM support be removed*yet*.
> >
> > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
> > coverage. But I do think it should an "off-by-default" sort of thing. Maybe
> > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
>
> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
> what's wrong with booting an older guest?
From my point of view, using the same kernel source for host and the guest
is easier because you know that both kernels behave the same.
About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
these days compiles a 32 bit kernel, knows what they are doing.
I personally would wait few more releases when there is a pressing reason to remove
this support.
AFAIK, it is not really possible to remove most of the legacy direct mmu
because shadowing mmu still can use it (I think Sean told me that once).
Best regards,
Maxim Levitsky
>
> Paolo
>
On 9/29/22 15:52, Maxim Levitsky wrote:
> On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
>> On 9/28/22 19:55, Sean Christopherson wrote:
>>>> As far as my opinion goes I do volunteer to test this code more often,
>>>> and I do not want to see the 32 bit KVM support be removed*yet*.
>>>
>>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
>>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe
>>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
>>
>> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
>> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
>> what's wrong with booting an older guest?
>
> From my point of view, using the same kernel source for host and the guest
> is easier because you know that both kernels behave the same.
It is certainly easier, but it is less correct. You don't cover
anything that KVM doesn't use.
> About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
> one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
> these days compiles a 32 bit kernel, knows what they are doing.
>
> I personally would wait few more releases when there is a pressing reason to remove
> this support.
>
> AFAIK, it is not really possible to remove most of the legacy direct mmu
> because shadowing mmu still can use it (I think Sean told me that once).
Yeah, it won't let us remove a lot of code but there are several logic
cleanups that become possible if the TDP case can just assume the TDP
MMU is there. For example, there is no reason to have a cpu_role (as
opposed to an mmu_page_role for the root) if you are building HPA->GPA
page tables.
(Which reminds me that toggling CR0.WP is still a hog with the TDP MMU).
Paolo
On 29/09/2022 15.52, Maxim Levitsky wrote:
> On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
>> On 9/28/22 19:55, Sean Christopherson wrote:
>>>> As far as my opinion goes I do volunteer to test this code more often,
>>>> and I do not want to see the 32 bit KVM support be removed*yet*.
>>>
>>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
>>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe
>>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
>>
>> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
>> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
>> what's wrong with booting an older guest?
>
>>From my point of view, using the same kernel source for host and the guest
> is easier because you know that both kernels behave the same.
>
> About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
> one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
> these days compiles a 32 bit kernel, knows what they are doing.
>
> I personally would wait few more releases when there is a pressing reason to remove
> this support.
FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit
KVM support from the kernel. The QEMU project currently struggles badly with
keeping everything tested in the CI in a reasonable amount of time. The
32-bit KVM kernel support is the only reason to keep the qemu-system-i386
binary around - everything else can be covered with the qemu-system-x86_64
binary that is a superset of the -i386 variant (except for the KVM part as
far as I know).
Sure, we could also drop qemu-system-i386 from the CI without dropping the
32-bit KVM code in the kernel, but I guess things will rather bitrot there
even faster in that case, so I'd appreciate if the kernel could drop the
32-bit in the near future, too.
Thomas
On Fri, Feb 17, 2023, Thomas Huth wrote:
> On 29/09/2022 15.52, Maxim Levitsky wrote:
> > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
> > > On 9/28/22 19:55, Sean Christopherson wrote:
> > > > > As far as my opinion goes I do volunteer to test this code more often,
> > > > > and I do not want to see the 32 bit KVM support be removed*yet*.
> > > >
> > > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
> > > > coverage. But I do think it should an "off-by-default" sort of thing. Maybe
> > > > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
> > >
> > > Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
> > > coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
> > > what's wrong with booting an older guest?
> >
> > > From my point of view, using the same kernel source for host and the guest
> > is easier because you know that both kernels behave the same.
> >
> > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
> > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
> > these days compiles a 32 bit kernel, knows what they are doing.
> >
> > I personally would wait few more releases when there is a pressing reason to remove
> > this support.
>
> FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit
> KVM support from the kernel. The QEMU project currently struggles badly with
> keeping everything tested in the CI in a reasonable amount of time. The
> 32-bit KVM kernel support is the only reason to keep the qemu-system-i386
> binary around - everything else can be covered with the qemu-system-x86_64
> binary that is a superset of the -i386 variant (except for the KVM part as
> far as I know).
> Sure, we could also drop qemu-system-i386 from the CI without dropping the
> 32-bit KVM code in the kernel, but I guess things will rather bitrot there
> even faster in that case, so I'd appreciate if the kernel could drop the
> 32-bit in the near future, too.
Ya, I would happily drop support for 32-bit kernels today, the only sticking point
is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather
large point. :-(
On 22/02/2023 23.27, Sean Christopherson wrote:
> On Fri, Feb 17, 2023, Thomas Huth wrote:
>> On 29/09/2022 15.52, Maxim Levitsky wrote:
>>> On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
>>>> On 9/28/22 19:55, Sean Christopherson wrote:
>>>>>> As far as my opinion goes I do volunteer to test this code more often,
>>>>>> and I do not want to see the 32 bit KVM support be removed*yet*.
>>>>>
>>>>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
>>>>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe
>>>>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
>>>>
>>>> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
>>>> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
>>>> what's wrong with booting an older guest?
>>>
>>>> From my point of view, using the same kernel source for host and the guest
>>> is easier because you know that both kernels behave the same.
>>>
>>> About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
>>> one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
>>> these days compiles a 32 bit kernel, knows what they are doing.
>>>
>>> I personally would wait few more releases when there is a pressing reason to remove
>>> this support.
>>
>> FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit
>> KVM support from the kernel. The QEMU project currently struggles badly with
>> keeping everything tested in the CI in a reasonable amount of time. The
>> 32-bit KVM kernel support is the only reason to keep the qemu-system-i386
>> binary around - everything else can be covered with the qemu-system-x86_64
>> binary that is a superset of the -i386 variant (except for the KVM part as
>> far as I know).
>> Sure, we could also drop qemu-system-i386 from the CI without dropping the
>> 32-bit KVM code in the kernel, but I guess things will rather bitrot there
>> even faster in that case, so I'd appreciate if the kernel could drop the
>> 32-bit in the near future, too.
>
> Ya, I would happily drop support for 32-bit kernels today, the only sticking point
> is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather
> large point. :-(
From your point of view, would it be OK if QEMU dropped qemu-system-i386?
I.e. would it be fine to use older versions of QEMU only for that test
coverage (or do you even use a different userspace for testing that)?
Thomas
On Thu, 2023-02-23 at 08:01 +0100, Thomas Huth wrote:
> On 22/02/2023 23.27, Sean Christopherson wrote:
> > On Fri, Feb 17, 2023, Thomas Huth wrote:
> > > On 29/09/2022 15.52, Maxim Levitsky wrote:
> > > > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
> > > > > On 9/28/22 19:55, Sean Christopherson wrote:
> > > > > > > As far as my opinion goes I do volunteer to test this code more often,
> > > > > > > and I do not want to see the 32 bit KVM support be removed*yet*.
> > > > > >
> > > > > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
> > > > > > coverage. But I do think it should an "off-by-default" sort of thing. Maybe
> > > > > > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
> > > > >
> > > > > Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
> > > > > coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
> > > > > what's wrong with booting an older guest?
> > > > > From my point of view, using the same kernel source for host and the guest
> > > > is easier because you know that both kernels behave the same.
> > > >
> > > > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
> > > > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
> > > > these days compiles a 32 bit kernel, knows what they are doing.
> > > >
> > > > I personally would wait few more releases when there is a pressing reason to remove
> > > > this support.
> > >
> > > FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit
> > > KVM support from the kernel. The QEMU project currently struggles badly with
> > > keeping everything tested in the CI in a reasonable amount of time. The
> > > 32-bit KVM kernel support is the only reason to keep the qemu-system-i386
> > > binary around - everything else can be covered with the qemu-system-x86_64
> > > binary that is a superset of the -i386 variant (except for the KVM part as
> > > far as I know).
> > > Sure, we could also drop qemu-system-i386 from the CI without dropping the
> > > 32-bit KVM code in the kernel, but I guess things will rather bitrot there
> > > even faster in that case, so I'd appreciate if the kernel could drop the
> > > 32-bit in the near future, too.
> >
> > Ya, I would happily drop support for 32-bit kernels today, the only sticking point
> > is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather
> > large point. :-(
>
> From your point of view, would it be OK if QEMU dropped qemu-system-i386?
> I.e. would it be fine to use older versions of QEMU only for that test
> coverage (or do you even use a different userspace for testing that)?
>
> Thomas
>
From my point of view qemu-system-x86_64 does run 32 bit guests just fine.
The only exception that I know is that gdbstub is somewhat broken, but that can be probably
fixed.
Best regards,
Maxim Levitsky
On Thu, Feb 23, 2023, Maxim Levitsky wrote:
> On Thu, 2023-02-23 at 08:01 +0100, Thomas Huth wrote:
> > On 22/02/2023 23.27, Sean Christopherson wrote:
> > > On Fri, Feb 17, 2023, Thomas Huth wrote:
> > > > On 29/09/2022 15.52, Maxim Levitsky wrote:
> > > > > On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
> > > > > > On 9/28/22 19:55, Sean Christopherson wrote:
> > > > > > > > As far as my opinion goes I do volunteer to test this code more often,
> > > > > > > > and I do not want to see the 32 bit KVM support be removed*yet*.
> > > > > > >
> > > > > > > Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
> > > > > > > coverage. But I do think it should an "off-by-default" sort of thing. Maybe
> > > > > > > BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
> > > > > >
> > > > > > Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
> > > > > > coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
> > > > > > what's wrong with booting an older guest?
> > > > > > From my point of view, using the same kernel source for host and the guest
> > > > > is easier because you know that both kernels behave the same.
> > > > >
> > > > > About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
> > > > > one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
> > > > > these days compiles a 32 bit kernel, knows what they are doing.
> > > > >
> > > > > I personally would wait few more releases when there is a pressing reason to remove
> > > > > this support.
> > > >
> > > > FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit
> > > > KVM support from the kernel. The QEMU project currently struggles badly with
> > > > keeping everything tested in the CI in a reasonable amount of time. The
> > > > 32-bit KVM kernel support is the only reason to keep the qemu-system-i386
> > > > binary around - everything else can be covered with the qemu-system-x86_64
> > > > binary that is a superset of the -i386 variant (except for the KVM part as
> > > > far as I know).
> > > > Sure, we could also drop qemu-system-i386 from the CI without dropping the
> > > > 32-bit KVM code in the kernel, but I guess things will rather bitrot there
> > > > even faster in that case, so I'd appreciate if the kernel could drop the
> > > > 32-bit in the near future, too.
> > >
> > > Ya, I would happily drop support for 32-bit kernels today, the only sticking point
> > > is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather
> > > large point. :-(
> >
> > From your point of view, would it be OK if QEMU dropped qemu-system-i386?
> > I.e. would it be fine to use older versions of QEMU only for that test
> > coverage (or do you even use a different userspace for testing that)?
For me personally, I have no objection to dropping qemu-system-i386 support in
future QEMU releases. I update my 32-bit images very, very infrequently, so I
probably wouldn't even notice for like 5 years :-)
> From my point of view qemu-system-x86_64 does run 32 bit guests just fine.
Right, but unless I seriously misunderstand what qemu-system-x86_64 ecompasses,
it can't be used to run guests of 32-bit _hosts_, which is what we need to test
shadowing of 32-bit NPT.
On 23/02/2023 23.10, Sean Christopherson wrote:
> On Thu, Feb 23, 2023, Maxim Levitsky wrote:
>> On Thu, 2023-02-23 at 08:01 +0100, Thomas Huth wrote:
>>> On 22/02/2023 23.27, Sean Christopherson wrote:
>>>> On Fri, Feb 17, 2023, Thomas Huth wrote:
>>>>> On 29/09/2022 15.52, Maxim Levitsky wrote:
>>>>>> On Thu, 2022-09-29 at 15:26 +0200, Paolo Bonzini wrote:
>>>>>>> On 9/28/22 19:55, Sean Christopherson wrote:
>>>>>>>>> As far as my opinion goes I do volunteer to test this code more often,
>>>>>>>>> and I do not want to see the 32 bit KVM support be removed*yet*.
>>>>>>>>
>>>>>>>> Yeah, I 100% agree that it shouldn't be removed until we have equivalent test
>>>>>>>> coverage. But I do think it should an "off-by-default" sort of thing. Maybe
>>>>>>>> BROKEN is the wrong dependency though? E.g. would EXPERT be a better option?
>>>>>>>
>>>>>>> Yeah, maybe EXPERT is better but I'm not sure of the equivalent test
>>>>>>> coverage. 32-bit VMX/SVM kvm-unit-tests are surely a good idea, but
>>>>>>> what's wrong with booting an older guest?
>>>>>>> From my point of view, using the same kernel source for host and the guest
>>>>>> is easier because you know that both kernels behave the same.
>>>>>>
>>>>>> About EXPERT, IMHO these days most distros already dropped 32 bit suport thus anyway
>>>>>> one needs to compile a recent 32 bit kernel manually - thus IMHO whoever
>>>>>> these days compiles a 32 bit kernel, knows what they are doing.
>>>>>>
>>>>>> I personally would wait few more releases when there is a pressing reason to remove
>>>>>> this support.
>>>>>
>>>>> FWIW, from the QEMU perspective, it would be very helpful to remove 32-bit
>>>>> KVM support from the kernel. The QEMU project currently struggles badly with
>>>>> keeping everything tested in the CI in a reasonable amount of time. The
>>>>> 32-bit KVM kernel support is the only reason to keep the qemu-system-i386
>>>>> binary around - everything else can be covered with the qemu-system-x86_64
>>>>> binary that is a superset of the -i386 variant (except for the KVM part as
>>>>> far as I know).
>>>>> Sure, we could also drop qemu-system-i386 from the CI without dropping the
>>>>> 32-bit KVM code in the kernel, but I guess things will rather bitrot there
>>>>> even faster in that case, so I'd appreciate if the kernel could drop the
>>>>> 32-bit in the near future, too.
>>>>
>>>> Ya, I would happily drop support for 32-bit kernels today, the only sticking point
>>>> is the lack of 32-bit shadow paging test coverage, which unfortunately is a rather
>>>> large point. :-(
>>>
>>> From your point of view, would it be OK if QEMU dropped qemu-system-i386?
>>> I.e. would it be fine to use older versions of QEMU only for that test
>>> coverage (or do you even use a different userspace for testing that)?
>
> For me personally, I have no objection to dropping qemu-system-i386 support in
> future QEMU releases. I update my 32-bit images very, very infrequently, so I
> probably wouldn't even notice for like 5 years :-)
>
>> From my point of view qemu-system-x86_64 does run 32 bit guests just fine.
>
> Right, but unless I seriously misunderstand what qemu-system-x86_64 ecompasses,
> it can't be used to run guests of 32-bit _hosts_, which is what we need to test
> shadowing of 32-bit NPT.
That's what I've been told in the past, too, and that's why I asked. Thanks
for the clarification!
To summarize: My takeaway is that nobody really needs qemu-system-i386
anymore for recent development - the remaining 32-bit KVM use cases can be
done with older versions of QEMU instead, thus it should be fine for the
QEMU project to drop qemu-system-i386 nowadays.
Thanks,
Thomas