2010-07-02 04:42:26

by Xiao Guangrong

[permalink] [raw]
Subject: [PATCH] KVM: IOAPIC: only access APIC registers one dword at a time

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time.
For example, software should never access byte 2 from the Data register before
accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
programming model in this case.

So, this patch removes other width access

Signed-off-by: Xiao Guangrong <[email protected]>
---
virt/kvm/ioapic.c | 10 +++-------
1 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/virt/kvm/ioapic.c b/virt/kvm/ioapic.c
index 1149c60..f96816f 100644
--- a/virt/kvm/ioapic.c
+++ b/virt/kvm/ioapic.c
@@ -306,14 +306,10 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
spin_unlock(&ioapic->lock);

switch (len) {
- case 8:
- *(u64 *) val = result;
- break;
- case 1:
- case 2:
case 4:
- memcpy(val, (char *)&result, len);
+ *(u32 *) val = result;
break;
+
default:
printk(KERN_WARNING "ioapic: wrong length %d\n", len);
}
@@ -332,7 +328,7 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
(void*)addr, len, val);
ASSERT(!(addr & 0xf)); /* check alignment */

- if (len == 4 || len == 8)
+ if (len == 4)
data = *(u32 *) val;
else {
printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);
--
1.6.1.2


2010-07-02 07:31:24

by Jin Dongming

[permalink] [raw]
Subject: Re: [PATCH] KVM: IOAPIC: only access APIC registers one dword at a time

Hi, Xiao Guangrong

Xiao Guangrong wrote:
> The IOAPIC spec says:
>
> When accessing these registers, accesses must be done one dword at a time.
> For example, software should never access byte 2 from the Data register before
> accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
> programming model in this case.

I could understand what you described on the above, but I don't think it is
the best method to fix it. Could you consider a nice one?

>
> So, this patch removes other width access
>
> Signed-off-by: Xiao Guangrong <[email protected]>
> ---
> virt/kvm/ioapic.c | 10 +++-------
> 1 files changed, 3 insertions(+), 7 deletions(-)
>
> diff --git a/virt/kvm/ioapic.c b/virt/kvm/ioapic.c
> index 1149c60..f96816f 100644
> --- a/virt/kvm/ioapic.c
> +++ b/virt/kvm/ioapic.c
> @@ -306,14 +306,10 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
> spin_unlock(&ioapic->lock);
>
> switch (len) {
> - case 8:
> - *(u64 *) val = result;
> - break;
> - case 1:
> - case 2:
> case 4:
> - memcpy(val, (char *)&result, len);

Here the parameter len is not used for reading data from ioapic register, before switch case
the value of ioapic register has been read by ioapic_read_indirect().


> + *(u32 *) val = result;
> break;
> +
> default:
> printk(KERN_WARNING "ioapic: wrong length %d\n", len);

And I think here is not good to print warning message. Maybe it is better to do
such kind of checking at the first of this function before ioapic_read_indirect().

> }
> @@ -332,7 +328,7 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
> (void*)addr, len, val);
> ASSERT(!(addr & 0xf)); /* check alignment */
>
> - if (len == 4 || len == 8)
> + if (len == 4)
> data = *(u32 *) val;
> else {
> printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);

So I hope you could read the source code again. I think you can use a better method to fix it.

Best Regards,
Jin Dongming

2010-07-02 07:41:52

by Xiao Guangrong

[permalink] [raw]
Subject: Re: [PATCH] KVM: IOAPIC: only access APIC registers one dword at a time



Jin Dongming wrote:

>>
>> switch (len) {
>> - case 8:
>> - *(u64 *) val = result;
>> - break;
>> - case 1:
>> - case 2:
>> case 4:
>> - memcpy(val, (char *)&result, len);
>
> Here the parameter len is not used for reading data from ioapic register, before switch case
> the value of ioapic register has been read by ioapic_read_indirect().
>

Yeah, it's right, but it's rarely operation that guest using incorrect width to access
the registers, so i don't think it's worthy to move the width judgment before the real
registers read.

>
>> + *(u32 *) val = result;
>> break;
>> +
>> default:
>> printk(KERN_WARNING "ioapic: wrong length %d\n", len);
>
> And I think here is not good to print warning message. Maybe it is better to do
> such kind of checking at the first of this function before ioapic_read_indirect().
>

ditto

Thanks for your review, Jin!

2010-07-02 08:04:12

by Xiao Guangrong

[permalink] [raw]
Subject: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time.
For example, software should never access byte 2 from the Data register before
accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
programming model in this case.

So, this patch removes other width access

Signed-off-by: Xiao Guangrong <[email protected]>
---
virt/kvm/ioapic.c | 20 +++++++-------------
1 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/virt/kvm/ioapic.c b/virt/kvm/ioapic.c
index 1149c60..6610d11 100644
--- a/virt/kvm/ioapic.c
+++ b/virt/kvm/ioapic.c
@@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
ioapic_debug("addr %lx\n", (unsigned long)addr);
ASSERT(!(addr & 0xf)); /* check alignment */

+ if (len != 4) {
+ printk(KERN_WARNING "ioapic: wrong length %d\n", len);
+ return 0;
+ }
+
addr &= 0xff;
spin_lock(&ioapic->lock);
switch (addr) {
@@ -305,18 +310,7 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
}
spin_unlock(&ioapic->lock);

- switch (len) {
- case 8:
- *(u64 *) val = result;
- break;
- case 1:
- case 2:
- case 4:
- memcpy(val, (char *)&result, len);
- break;
- default:
- printk(KERN_WARNING "ioapic: wrong length %d\n", len);
- }
+ *(u32 *) val = result;
return 0;
}

@@ -332,7 +326,7 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
(void*)addr, len, val);
ASSERT(!(addr & 0xf)); /* check alignment */

- if (len == 4 || len == 8)
+ if (len == 4)
data = *(u32 *) val;
else {
printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);
--
1.6.1.2

2010-07-02 17:45:29

by Marcelo Tosatti

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time

On Fri, Jul 02, 2010 at 04:00:12PM +0800, Xiao Guangrong wrote:
> The IOAPIC spec says:
>
> When accessing these registers, accesses must be done one dword at a time.
> For example, software should never access byte 2 from the Data register before
> accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
> programming model in this case.
>
> So, this patch removes other width access
>
> Signed-off-by: Xiao Guangrong <[email protected]>
> ---
> virt/kvm/ioapic.c | 20 +++++++-------------
> 1 files changed, 7 insertions(+), 13 deletions(-)
>
> diff --git a/virt/kvm/ioapic.c b/virt/kvm/ioapic.c
> index 1149c60..6610d11 100644
> --- a/virt/kvm/ioapic.c
> +++ b/virt/kvm/ioapic.c
> @@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
> ioapic_debug("addr %lx\n", (unsigned long)addr);
> ASSERT(!(addr & 0xf)); /* check alignment */
>
> + if (len != 4) {
> + printk(KERN_WARNING "ioapic: wrong length %d\n", len);
> + return 0;
> + }
> +

Just remove the printks please, as guests can flood hosts dmesg.

2010-07-03 08:24:20

by Xiao Guangrong

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time



Marcelo Tosatti wrote:

>> + if (len != 4) {
>> + printk(KERN_WARNING "ioapic: wrong length %d\n", len);
>> + return 0;
>> + }
>> +
>
> Just remove the printks please, as guests can flood hosts dmesg.
>

OK, please review this one, thanks, Marcelo!

Subject: [PATCH v3] KVM: IOAPIC: only access APIC registers one dword at a time

The IOAPIC spec says:
When accessing these registers, accesses must be done one dword at a time.
For example, software should never access byte 2 from the Data register before
accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
programming model in this case.

So, this patch removes other width access

Signed-off-by: Xiao Guangrong <[email protected]>
---
virt/kvm/ioapic.c | 22 ++++++----------------
1 files changed, 6 insertions(+), 16 deletions(-)

diff --git a/virt/kvm/ioapic.c b/virt/kvm/ioapic.c
index 1149c60..aed91fd 100644
--- a/virt/kvm/ioapic.c
+++ b/virt/kvm/ioapic.c
@@ -288,6 +288,9 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
ioapic_debug("addr %lx\n", (unsigned long)addr);
ASSERT(!(addr & 0xf)); /* check alignment */

+ if (len != 4)
+ return 0;
+
addr &= 0xff;
spin_lock(&ioapic->lock);
switch (addr) {
@@ -305,18 +308,7 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
}
spin_unlock(&ioapic->lock);

- switch (len) {
- case 8:
- *(u64 *) val = result;
- break;
- case 1:
- case 2:
- case 4:
- memcpy(val, (char *)&result, len);
- break;
- default:
- printk(KERN_WARNING "ioapic: wrong length %d\n", len);
- }
+ *(u32 *) val = result;
return 0;
}

@@ -332,12 +324,10 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
(void*)addr, len, val);
ASSERT(!(addr & 0xf)); /* check alignment */

- if (len == 4 || len == 8)
+ if (len == 4)
data = *(u32 *) val;
- else {
- printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);
+ else
return 0;
- }

addr &= 0xff;
spin_lock(&ioapic->lock);
--
1.6.1.2

2010-07-03 10:11:25

by Avi Kivity

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time

On 07/02/2010 11:00 AM, Xiao Guangrong wrote:
> The IOAPIC spec says:
>
> When accessing these registers, accesses must be done one dword at a time.
> For example, software should never access byte 2 from the Data register before
> accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
> programming model in this case.
>
> So, this patch removes other width access
>
>

The ioapic code also implements the ia64 iosapic. I'm guessing that
does support 64-bit accesses. Please check the iosapic documentation.

There might be guests that use incorrect access despite the
documentation; if real hardware supports it, it should work. So we need
to start with just a warning, and allow the access. Later we can drop
the invalid access.

> @@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
> ioapic_debug("addr %lx\n", (unsigned long)addr);
> ASSERT(!(addr& 0xf)); /* check alignment */
>
> + if (len != 4) {
> + printk(KERN_WARNING "ioapic: wrong length %d\n", len);
> + return 0;
> + }
> +
>

Guest triggered, so needs to be rate limited.


--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

2010-07-05 03:50:59

by Xiao Guangrong

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time



Avi Kivity wrote:
> On 07/02/2010 11:00 AM, Xiao Guangrong wrote:
>> The IOAPIC spec says:
>>
>> When accessing these registers, accesses must be done one dword at a
>> time.
>> For example, software should never access byte 2 from the Data
>> register before
>> accessing bytes 0 and 1. The hardware will not attempt to recover from
>> a bad
>> programming model in this case.
>>
>> So, this patch removes other width access
>>
>>
>
> The ioapic code also implements the ia64 iosapic. I'm guessing that
> does support 64-bit accesses. Please check the iosapic documentation.
>

The iosapic also using 32-bit to access registers:

All registers are accessed using 32-bit uncacheable loads and stores to a reserved memory location
in system memory. This implies that to modify a field (e.g., a bit or a byte) in any register, the
whole 32-bit register must be read, the field modified, and the 32 bits written back. Partial register
access, or non-aligned register access, are implementation-defined by the I/O xAPIC and will not
be compatible across different implementations. Also, registers that are described as 64 bits wide
are accessed as multiple independent 32-bit registers.

[ From << Intel® Itanium® Processor Family Interrupt Architecture Guide >>, P2-6 ]

> There might be guests that use incorrect access despite the
> documentation; if real hardware supports it, it should work. So we need
> to start with just a warning, and allow the access. Later we can drop
> the invalid access.

If the OS contravene the spec, i thinks it's the OS's bug, also, i have tested some versions
windows/linux guests, it's no broken, can we directly drop the other wide access?

>
>> @@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device
>> *this, gpa_t addr, int len,
>> ioapic_debug("addr %lx\n", (unsigned long)addr);
>> ASSERT(!(addr& 0xf)); /* check alignment */
>>
>> + if (len != 4) {
>> + printk(KERN_WARNING "ioapic: wrong length %d\n", len);
>> + return 0;
>> + }
>> +
>>
>
> Guest triggered, so needs to be rate limited.

Yeah, will using printk_ratelimit cooperate with it.

2010-07-05 06:24:11

by Gleb Natapov

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time

On Mon, Jul 05, 2010 at 11:47:06AM +0800, Xiao Guangrong wrote:
> > There might be guests that use incorrect access despite the
> > documentation; if real hardware supports it, it should work. So we need
> > to start with just a warning, and allow the access. Later we can drop
> > the invalid access.
>
> If the OS contravene the spec, i thinks it's the OS's bug, also, i have tested some versions
> windows/linux guests, it's no broken, can we directly drop the other wide access?
>
If reality contradicts spec we should emulate reality. I vaguely
remember some guest using memcpy on apic page for instance.

--
Gleb.

2010-07-05 07:54:28

by Avi Kivity

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time

On 07/05/2010 06:47 AM, Xiao Guangrong wrote:
>
> Avi Kivity wrote:
>
>> On 07/02/2010 11:00 AM, Xiao Guangrong wrote:
>>
>>> The IOAPIC spec says:
>>>
>>> When accessing these registers, accesses must be done one dword at a
>>> time.
>>> For example, software should never access byte 2 from the Data
>>> register before
>>> accessing bytes 0 and 1. The hardware will not attempt to recover from
>>> a bad
>>> programming model in this case.
>>>
>>> So, this patch removes other width access
>>>
>>>
>>>
>> The ioapic code also implements the ia64 iosapic. I'm guessing that
>> does support 64-bit accesses. Please check the iosapic documentation.
>>
>>
> The iosapic also using 32-bit to access registers:
>
> All registers are accessed using 32-bit uncacheable loads and stores to a reserved memory location
> in system memory. This implies that to modify a field (e.g., a bit or a byte) in any register, the
> whole 32-bit register must be read, the field modified, and the 32 bits written back. Partial register
> access, or non-aligned register access, are implementation-defined by the I/O xAPIC and will not
> be compatible across different implementations. Also, registers that are described as 64 bits wide
> are accessed as multiple independent 32-bit registers.
>
> [ From<< Intel® Itanium® Processor Family Interrupt Architecture Guide>>, P2-6 ]
>

Ok.

>> There might be guests that use incorrect access despite the
>> documentation; if real hardware supports it, it should work. So we need
>> to start with just a warning, and allow the access. Later we can drop
>> the invalid access.
>>
> If the OS contravene the spec, i thinks it's the OS's bug, also, i have tested some versions
> windows/linux guests, it's no broken, can we directly drop the other wide access?
>

Well, there's the spec and there's real life, but in this case we can
try and if we see a problem we'll re-add the other access length.


--
error compiling committee.c: too many arguments to function

2010-07-05 08:00:17

by Xiao Guangrong

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time



Avi Kivity wrote:

>
>>> There might be guests that use incorrect access despite the
>>> documentation; if real hardware supports it, it should work. So we need
>>> to start with just a warning, and allow the access. Later we can drop
>>> the invalid access.
>>>
>> If the OS contravene the spec, i thinks it's the OS's bug, also, i
>> have tested some versions
>> windows/linux guests, it's no broken, can we directly drop the other
>> wide access?
>>
>
> Well, there's the spec and there's real life, but in this case we can
> try and if we see a problem we'll re-add the other access length.
>

OK, i'll do it as your suggestion, thanks.

2010-07-05 08:25:15

by Avi Kivity

[permalink] [raw]
Subject: Re: [PATCH v2] KVM: IOAPIC: only access APIC registers one dword at a time

On 07/05/2010 10:56 AM, Xiao Guangrong wrote:
>
>> Well, there's the spec and there's real life, but in this case we can
>> try and if we see a problem we'll re-add the other access length.
>>
>>
> OK, i'll do it as your suggestion, thanks.
>

I meant that you can drop access length != 4, same as in your original
patch. Only change needed is printk rate limit.

--
error compiling committee.c: too many arguments to function