2008-10-14 07:40:33

by Nicolas Bareil

[permalink] [raw]
Subject: USB problem on x86_64: nommu_map_single() issue?

Hi

On 2.6.27, I can't use any USB device (mass storage, mouse) anymore on my x86_64 system, the devices
are not detected: lsusb returns nothing except the hub.

I have the following logs in kern.log when booting with my USB mouse plugged :

Oct 13 15:43:48 brew kernel: USB Universal Host Controller Interface driver v3.0
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.0: setting latency timer to 64
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.0: UHCI Host Controller
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 1
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.0: irq 16, io base 0x00002000
Oct 13 15:43:48 brew kernel: usb usb1: configuration #1 chosen from 1 choice
Oct 13 15:43:48 brew kernel: hub 1-0:1.0: USB hub found
Oct 13 15:43:48 brew kernel: hub 1-0:1.0: 2 ports detected
Oct 13 15:43:48 brew kernel: intel_rng: Firmware space is locked read-only. If you can't or
Oct 13 15:43:48 brew kernel: intel_rng: don't want to disable this in firmware setup, and if
Oct 13 15:43:48 brew kernel: intel_rng: you are certain that your system has a functional
Oct 13 15:43:48 brew kernel: intel_rng: RNG, try using the 'no_fwh_detect' option.
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.1: setting latency timer to 64
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.1: UHCI Host Controller
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 2
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.1: irq 19, io base 0x00002020
Oct 13 15:43:48 brew kernel: usb usb2: configuration #1 chosen from 1 choice
Oct 13 15:43:48 brew kernel: hub 2-0:1.0: USB hub found
Oct 13 15:43:48 brew kernel: hub 2-0:1.0: 2 ports detected
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.2: setting latency timer to 64
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.2: UHCI Host Controller
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 3
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.2: irq 18, io base 0x00002040
Oct 13 15:43:48 brew kernel: usb usb3: configuration #1 chosen from 1 choice
Oct 13 15:43:48 brew kernel: hub 3-0:1.0: USB hub found
Oct 13 15:43:48 brew kernel: hub 3-0:1.0: 2 ports detected
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.3: PCI INT D -> GSI 22 (level, low) -> IRQ 22
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.3: setting latency timer to 64
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.3: UHCI Host Controller
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 4
Oct 13 15:43:48 brew kernel: uhci_hcd 0000:00:1d.3: irq 22, io base 0x00002060
Oct 13 15:43:48 brew kernel: usb usb4: configuration #1 chosen from 1 choice
Oct 13 15:43:48 brew kernel: hub 4-0:1.0: USB hub found
Oct 13 15:43:48 brew kernel: hub 4-0:1.0: 2 ports detected
Oct 13 15:43:48 brew kernel: usb 2-1: new low speed USB device using uhci_hcd and address 2
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 12e8b78e0+8 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 11f0fd480+64 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 12e8b78e0+8 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 11f0fd480+64 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 12e8b78e0+8 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 11f0fd480+64 of device mask ffffffff
Oct 13 15:43:48 brew kernel: HDA Intel 0000:00:1b.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
Oct 13 15:43:48 brew kernel: HDA Intel 0000:00:1b.0: setting latency timer to 64
Oct 13 15:43:48 brew kernel: usb 2-1: device descriptor read/64, error -32
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: setting latency timer to 64
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: EHCI Host Controller
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 5
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: debug port 1
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: irq 16, io mem 0xf0204000
Oct 13 15:43:48 brew kernel: ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
Oct 13 15:43:48 brew kernel: usb usb5: configuration #1 chosen from 1 choice
Oct 13 15:43:48 brew kernel: hub 5-0:1.0: USB hub found
Oct 13 15:43:48 brew kernel: hub 5-0:1.0: 8 ports detected
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 12e9de620+8 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 11ed07d80+64 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 12e9de620+8 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 11ed07d80+64 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 12e9de620+8 of device mask ffffffff
Oct 13 15:43:48 brew kernel: nommu_map_single: overflow 11ed07d80+64 of device mask ffffffff
Oct 13 15:43:48 brew kernel: hub 2-0:1.0: unable to enumerate USB device on port 1
...

The whole kern.log and my config are available here :
http://chdir.org/~nbareil/kern.log
http://chdir.or/~nbareil/config-2.6.27-brew

Thanks!


2008-10-14 09:50:23

by Oliver Neukum

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

Am Dienstag, 14. Oktober 2008 09:20:45 schrieb Nicolas Bareil:
> Hi
>
> On 2.6.27, I can't use any USB device (mass storage, mouse) anymore on my x86_64 system, the devices
> are not detected: lsusb returns nothing except the hub.

Is this a regression? Did it work on earlier kernels?
>
> I have the following logs in kern.log when booting with my USB mouse plugged :

For confirmation boot with mem=2G on the kernel command line?

Regards
Oliver

2008-10-14 10:45:21

by Nicolas Bareil

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

Hi Oliver,

On Tue, Oct 14, 2008 at 11:48:47AM +0200, Oliver Neukum wrote:
> > On 2.6.27, I can't use any USB device (mass storage, mouse) anymore on my x86_64 system, the devices
> > are not detected: lsusb returns nothing except the hub.
>
> Is this a regression? Did it work on earlier kernels?

Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.

> > I have the following logs in kern.log when booting with my USB mouse plugged :
>
> For confirmation boot with mem=2G on the kernel command line?

With this workaround, the USB works with the following logs (don't know if it's useful):

Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: setting latency timer to 64
Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: EHCI Host Controller
Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: debug port 1
Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: irq 16, io mem 0xf0204000
Oct 14 12:30:22 brew kernel: USB Universal Host Controller Interface driver v3.0
Oct 14 12:30:22 brew kernel: ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
Oct 14 12:30:22 brew kernel: usb usb1: configuration #1 chosen from 1 choice
Oct 14 12:30:22 brew kernel: hub 1-0:1.0: USB hub found
Oct 14 12:30:22 brew kernel: hub 1-0:1.0: 8 ports detected
Oct 14 12:30:22 brew kernel: intel_rng: Firmware space is locked read-only. If you can't or
Oct 14 12:30:22 brew kernel: intel_rng: don't want to disable this in firmware setup, and if
Oct 14 12:30:22 brew kernel: intel_rng: you are certain that your system has a functional
Oct 14 12:30:22 brew kernel: intel_rng: RNG, try using the 'no_fwh_detect' option.
Oct 14 12:30:22 brew kernel: HDA Intel 0000:00:1b.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
Oct 14 12:30:22 brew kernel: HDA Intel 0000:00:1b.0: setting latency timer to 64
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.0: setting latency timer to 64
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.0: UHCI Host Controller
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.0: irq 16, io base 0x00002000
Oct 14 12:30:22 brew kernel: usb usb2: configuration #1 chosen from 1 choice
Oct 14 12:30:22 brew kernel: hub 2-0:1.0: USB hub found
Oct 14 12:30:22 brew kernel: hub 2-0:1.0: 2 ports detected
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.1: setting latency timer to 64
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.1: UHCI Host Controller
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.1: irq 19, io base 0x00002020
Oct 14 12:30:22 brew kernel: usb usb3: configuration #1 chosen from 1 choice
Oct 14 12:30:22 brew kernel: hub 3-0:1.0: USB hub found
Oct 14 12:30:22 brew kernel: hub 3-0:1.0: 2 ports detected
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.2: setting latency timer to 64
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.2: UHCI Host Controller
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.2: irq 18, io base 0x00002040
Oct 14 12:30:22 brew kernel: usb usb4: configuration #1 chosen from 1 choice
Oct 14 12:30:22 brew kernel: hub 4-0:1.0: USB hub found
Oct 14 12:30:22 brew kernel: hub 4-0:1.0: 2 ports detected
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.3: PCI INT D -> GSI 22 (level, low) -> IRQ 22
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.3: setting latency timer to 64
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.3: UHCI Host Controller
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 5
Oct 14 12:30:22 brew kernel: uhci_hcd 0000:00:1d.3: irq 22, io base 0x00002060
Oct 14 12:30:22 brew kernel: usb usb5: configuration #1 chosen from 1 choice
Oct 14 12:30:22 brew kernel: hub 5-0:1.0: USB hub found
Oct 14 12:30:22 brew kernel: hub 5-0:1.0: 2 ports detected
Oct 14 12:30:22 brew kernel: usb 3-1: new low speed USB device using uhci_hcd and address 2

Thank you for your time!

2008-10-14 11:00:45

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

On Tue, 14 Oct 2008 12:45:00 +0200
Nicolas Bareil <[email protected]> wrote:

> Hi Oliver,
>
> On Tue, Oct 14, 2008 at 11:48:47AM +0200, Oliver Neukum wrote:
> > > On 2.6.27, I can't use any USB device (mass storage, mouse) anymore on my x86_64 system, the devices
> > > are not detected: lsusb returns nothing except the hub.
> >
> > Is this a regression? Did it work on earlier kernels?
>
> Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.

With old kernels, you can find something like the following line in
the boot log?

PCI-DMA: Using software bounce buffering for IO (SWIOTLB)

2008-10-14 11:01:54

by Oliver Neukum

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

Am Dienstag, 14. Oktober 2008 12:45:00 schrieb Nicolas Bareil:

Hello,

> Hi Oliver,
>
> On Tue, Oct 14, 2008 at 11:48:47AM +0200, Oliver Neukum wrote:
> > > On 2.6.27, I can't use any USB device (mass storage, mouse) anymore on my x86_64 system, the devices
> > > are not detected: lsusb returns nothing except the hub.
> >
> > Is this a regression? Did it work on earlier kernels?
>
> Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.

This a an IOMMU problem, not really a USB problem. Can you bisect the problem?

Regards
Oliver

2008-10-14 11:05:55

by Nicolas Bareil

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

On Tue, Oct 14, 2008 at 08:00:09PM +0900, FUJITA Tomonori wrote:
> > > Is this a regression? Did it work on earlier kernels?
> >
> > Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.
>
> With old kernels, you can find something like the following line in
> the boot log?
>
> PCI-DMA: Using software bounce buffering for IO (SWIOTLB)

Indeed, I have this line:

Oct 14 12:46:29 brew kernel: [ 0.004000] Calgary: detecting Calgary via BIOS EBDA area
Oct 14 12:46:29 brew kernel: [ 0.004000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
Oct 14 12:46:29 brew kernel: [ 0.004000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Oct 14 12:46:29 brew kernel: [ 0.004000] Placing software IO TLB between 0x4000000 - 0x8000000
Oct 14 12:46:29 brew kernel: [ 0.004000] Memory: 4055544k/4980736k available (2225k kernel code, 138092k reserved, 1079k data, 392k init)
Oct 14 12:46:29 brew kernel: [ 0.004000] CPA: page pool initialized 1 of 1 pages preallocated

The full kern.log is available here: http://chdir.org/~nbareil/kern.log-2.6.26-1

Thanks!

2008-10-14 11:15:34

by FUJITA Tomonori

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

On Tue, 14 Oct 2008 13:05:39 +0200
Nicolas Bareil <[email protected]> wrote:

> On Tue, Oct 14, 2008 at 08:00:09PM +0900, FUJITA Tomonori wrote:
> > > > Is this a regression? Did it work on earlier kernels?
> > >
> > > Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.
> >
> > With old kernels, you can find something like the following line in
> > the boot log?
> >
> > PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
>
> Indeed, I have this line:
>
> Oct 14 12:46:29 brew kernel: [ 0.004000] Calgary: detecting Calgary via BIOS EBDA area
> Oct 14 12:46:29 brew kernel: [ 0.004000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
> Oct 14 12:46:29 brew kernel: [ 0.004000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
> Oct 14 12:46:29 brew kernel: [ 0.004000] Placing software IO TLB between 0x4000000 - 0x8000000
> Oct 14 12:46:29 brew kernel: [ 0.004000] Memory: 4055544k/4980736k available (2225k kernel code, 138092k reserved, 1079k data, 392k init)
> Oct 14 12:46:29 brew kernel: [ 0.004000] CPA: page pool initialized 1 of 1 pages preallocated

Probably, the changes to the initial memory setup code breaks the
following code:

void __init pci_swiotlb_init(void)
{
/* don't initialize swiotlb if iommu=off (no_iommu=1) */
if (!iommu_detected && !no_iommu && max_pfn > MAX_DMA32_PFN)
swiotlb = 1;


SWIOTLB should be used for your box but somehow pci-nommu is used.


> The whole kern.log and my config are available here :
> http://chdir.org/~nbareil/kern.log

The following part in the 2.6.27 boot log looks suspicious:

Oct 13 15:43:48 brew kernel: last_pfn = 0x130000 max_arch_pfn = 0x3ffffffff
Oct 13 15:43:48 brew kernel: x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
Oct 13 15:43:48 brew kernel: last_pfn = 0xcffc2 max_arch_pfn = 0x3ffffffff


CC'ed Yinghai and Ingo, who might have clues.

2008-10-14 13:43:54

by Nicolas Bareil

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

On Tue, Oct 14, 2008 at 01:00:28PM +0200, Oliver Neukum wrote:
> > Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.
>
> This a an IOMMU problem, not really a USB problem. Can you bisect the problem?

Ok, after 24 reboots, I have the following commit:

commit 6f21d806f698a5c437dcae1f03e8dd73e3f5aab1
Merge: e59e14b... 9f72632...
Author: Linus Torvalds <[email protected]>
Date: Sun Sep 21 12:40:56 2008 -0700

Merge branch 'kvm-updates/2.6.27' of git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm

* 'kvm-updates/2.6.27' of git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm:
KVM: ia64: 'struct fdesc' build fix

Do you think I'm good to bisect that tree or do you have an idea?

2008-10-14 14:15:45

by Nicolas Bareil

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

On Tue, Oct 14, 2008 at 03:43:41PM +0200, Nicolas Bareil wrote:
> commit 6f21d806f698a5c437dcae1f03e8dd73e3f5aab1
> Merge: e59e14b... 9f72632...
> Author: Linus Torvalds <[email protected]>
> Date: Sun Sep 21 12:40:56 2008 -0700
>
> Merge branch 'kvm-updates/2.6.27' of git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm
>
> * 'kvm-updates/2.6.27' of git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm:
> KVM: ia64: 'struct fdesc' build fix
>
> Do you think I'm good to bisect that tree or do you have an idea?

Strangely, if I disable MMC, there is no error at all. I'm currently
booting on a 2.6.27 kernel and everything seems to be ok. My kernel log
is at http://chdir.org/~nbareil/kern.log.2.6.27.without.mmc for
information.

Thanks

2008-10-14 17:11:26

by Yinghai Lu

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

On Tue, Oct 14, 2008 at 4:14 AM, FUJITA Tomonori
<[email protected]> wrote:
> On Tue, 14 Oct 2008 13:05:39 +0200
> Nicolas Bareil <[email protected]> wrote:
>
>> On Tue, Oct 14, 2008 at 08:00:09PM +0900, FUJITA Tomonori wrote:
>> > > > Is this a regression? Did it work on earlier kernels?
>> > >
>> > > Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.
>> >
>> > With old kernels, you can find something like the following line in
>> > the boot log?
>> >
>> > PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
>>
>> Indeed, I have this line:
>>
>> Oct 14 12:46:29 brew kernel: [ 0.004000] Calgary: detecting Calgary via BIOS EBDA area
>> Oct 14 12:46:29 brew kernel: [ 0.004000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
>> Oct 14 12:46:29 brew kernel: [ 0.004000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
>> Oct 14 12:46:29 brew kernel: [ 0.004000] Placing software IO TLB between 0x4000000 - 0x8000000
>> Oct 14 12:46:29 brew kernel: [ 0.004000] Memory: 4055544k/4980736k available (2225k kernel code, 138092k reserved, 1079k data, 392k init)
>> Oct 14 12:46:29 brew kernel: [ 0.004000] CPA: page pool initialized 1 of 1 pages preallocated
>
> Probably, the changes to the initial memory setup code breaks the
> following code:
>
> void __init pci_swiotlb_init(void)
> {
> /* don't initialize swiotlb if iommu=off (no_iommu=1) */
> if (!iommu_detected && !no_iommu && max_pfn > MAX_DMA32_PFN)
> swiotlb = 1;
>
>
> SWIOTLB should be used for your box but somehow pci-nommu is used.
>
>
>> The whole kern.log and my config are available here :
>> http://chdir.org/~nbareil/kern.log
>
> The following part in the 2.6.27 boot log looks suspicious:
>
> Oct 13 15:43:48 brew kernel: last_pfn = 0x130000 max_arch_pfn = 0x3ffffffff
that is max_pfn
> Oct 13 15:43:48 brew kernel: x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
> Oct 13 15:43:48 brew kernel: last_pfn = 0xcffc2 max_arch_pfn = 0x3ffffffff
that is max_low_pfn

YH

2008-10-28 09:07:52

by Jouni Malinen

[permalink] [raw]
Subject: Re: USB problem on x86_64: nommu_map_single() issue?

On Tue, Oct 14, 2008 at 1:00 PM, Oliver Neukum <[email protected]> wrote:
> Am Dienstag, 14. Oktober 2008 12:45:00 schrieb Nicolas Bareil:
>> On Tue, Oct 14, 2008 at 11:48:47AM +0200, Oliver Neukum wrote:
>> > > On 2.6.27, I can't use any USB device (mass storage, mouse) anymore on my x86_64 system, the devices
>> > > are not detected: lsusb returns nothing except the hub.
>> >
>> > Is this a regression? Did it work on earlier kernels?
>>
>> Yes this is a regression: USB works in 2.6.26.x with (almost) the same configuration.
>
> This a an IOMMU problem, not really a USB problem. Can you bisect the problem?

It looks like I'm seeing something similar with the current
linux-2.6.git. However, in my case, 2.6.27 seemed to work and I
started seeing this only after the 2.6.27 release (though, git bisect
ended up in 2.6.27-rc3). In addition, I'm seeing way more issues than
just USB dying (both wired and wireless networking were hosed; r8169
was returning random memory or all zeroes in the payload of the
frames, etc.).

After a painful git bisect (two other bugs that prevented boot and
some build issues in the tested commits) the first bad commit turned
out to be cf169702ba6928cee9d4f4adf3e932b643b8db7a (see below; author
cc'ed). This is only adding a new PCI device id, so the commit itself
seems to just trigger the issue somewhere else.

I've verified that I can get rid of the nommu_map_single() messages by
reverting this patch (i.e., made kernel not find the northbridge) with
the current linux-2.6.git head. Obviously, that does not sound like a
proper fix here. Would anyone have any idea on what is the real issue
here or what could be done to find it?

I'm seeing this issue on a HP Pavilion dv5 laptop that has AMD Turion
X2 Ultra Dual-Core Mobile ZM-80. lspci shows the PCI id that was added
(1022:1303). mem=2G works around the problem. pci_swiotlb_init() did
not set swiotlb to 1, but hardcoding it to 1 there did not help
either.


$ git bisect bad
cf169702ba6928cee9d4f4adf3e932b643b8db7a is first bad commit
commit cf169702ba6928cee9d4f4adf3e932b643b8db7a
Author: Joerg Roedel <[email protected]>
Date: Tue Sep 2 13:13:40 2008 +0200

x86, gart: add detection of AMD family 0x11 northbridges

This patch adds the detection of the northbridges in the AMD family 0x11
processors. It also fixes the magic numbers there while changing this code.

Signed-off-by: Joerg Roedel <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

:040000 040000 183c03461f96be8bbf96cb51bae9bb0fb93dae89
58638c91f03393225f3c48cb12476466f9c5e2be M arch

----------------------------- arch/x86/kernel/k8.c -----------------------------
index 7377ccb..304d8ba 100644
@@ -16,8 +16,9 @@ EXPORT_SYMBOL(num_k8_northbridges);
static u32 *flush_words;

struct pci_device_id k8_nb_ids[] = {
- { PCI_DEVICE(PCI_VENDOR_ID_AMD, 0x1103) },
- { PCI_DEVICE(PCI_VENDOR_ID_AMD, 0x1203) },
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_K8_NB_MISC) },
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_10H_NB_MISC) },
+ { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_11H_NB_MISC) },
{}
};
EXPORT_SYMBOL(k8_nb_ids);


- Jouni

2008-10-28 15:14:30

by Joerg Roedel

[permalink] [raw]
Subject: [PATCH] x86, gart: fix gart detection for Fam11h CPUs

The AMD Fam11h CPUs have a K8 northbridge. This northbridge is different
from other familys because it lacks GART support (as I just learned).
But the kernel implicitly expects a GART if it finds an AMD northbridge.
Fix this by removing the Fam11h northbridge id from the scan list of K8
northbridges. This patch also changes the message in the GART driver
about missing K8 northbridges to tell that the GART is missing which is
the correct information in this case.

Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/k8.c | 1 -
arch/x86/kernel/pci-gart_64.c | 2 +-
2 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/k8.c b/arch/x86/kernel/k8.c
index 304d8ba..cbc4332 100644
--- a/arch/x86/kernel/k8.c
+++ b/arch/x86/kernel/k8.c
@@ -18,7 +18,6 @@ static u32 *flush_words;
struct pci_device_id k8_nb_ids[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_K8_NB_MISC) },
{ PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_10H_NB_MISC) },
- { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_11H_NB_MISC) },
{}
};
EXPORT_SYMBOL(k8_nb_ids);
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index e3f75bb..a42b02b 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -744,7 +744,7 @@ void __init gart_iommu_init(void)
long i;

if (cache_k8_northbridges() < 0 || num_k8_northbridges == 0) {
- printk(KERN_INFO "PCI-GART: No AMD northbridge found.\n");
+ printk(KERN_INFO "PCI-GART: No AMD GART found.\n");
return;
}

--
1.5.6.4

2008-10-28 16:00:51

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PATCH] x86, gart: fix gart detection for Fam11h CPUs


* Joerg Roedel <[email protected]> wrote:

> The AMD Fam11h CPUs have a K8 northbridge. This northbridge is
> different from other familys because it lacks GART support (as I
> just learned). But the kernel implicitly expects a GART if it finds
> an AMD northbridge. Fix this by removing the Fam11h northbridge id
> from the scan list of K8 northbridges. This patch also changes the
> message in the GART driver about missing K8 northbridges to tell
> that the GART is missing which is the correct information in this
> case.
>
> Signed-off-by: Joerg Roedel <[email protected]>
> ---
> arch/x86/kernel/k8.c | 1 -
> arch/x86/kernel/pci-gart_64.c | 2 +-
> 2 files changed, 1 insertions(+), 2 deletions(-)

applied to tip/x86/urgent, thanks Joerg!

Ingo

2008-10-28 16:58:13

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] x86, gart: fix gart detection for Fam11h CPUs

On Tue, Oct 28, 2008 at 04:13:54PM +0100, Joerg Roedel wrote:
> The AMD Fam11h CPUs have a K8 northbridge. This northbridge is different
> from other familys because it lacks GART support (as I just learned).
> But the kernel implicitly expects a GART if it finds an AMD northbridge.
> Fix this by removing the Fam11h northbridge id from the scan list of K8
> northbridges. This patch also changes the message in the GART driver
> about missing K8 northbridges to tell that the GART is missing which is
> the correct information in this case.
>
> Signed-off-by: Joerg Roedel <[email protected]>

Looks like this needs to be added to the 2.6.27-stable tree also, right?

thanks,

greg k-h

> ---
> arch/x86/kernel/k8.c | 1 -
> arch/x86/kernel/pci-gart_64.c | 2 +-
> 2 files changed, 1 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/kernel/k8.c b/arch/x86/kernel/k8.c
> index 304d8ba..cbc4332 100644
> --- a/arch/x86/kernel/k8.c
> +++ b/arch/x86/kernel/k8.c
> @@ -18,7 +18,6 @@ static u32 *flush_words;
> struct pci_device_id k8_nb_ids[] = {
> { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_K8_NB_MISC) },
> { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_10H_NB_MISC) },
> - { PCI_DEVICE(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_11H_NB_MISC) },
> {}
> };
> EXPORT_SYMBOL(k8_nb_ids);
> diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
> index e3f75bb..a42b02b 100644
> --- a/arch/x86/kernel/pci-gart_64.c
> +++ b/arch/x86/kernel/pci-gart_64.c
> @@ -744,7 +744,7 @@ void __init gart_iommu_init(void)
> long i;
>
> if (cache_k8_northbridges() < 0 || num_k8_northbridges == 0) {
> - printk(KERN_INFO "PCI-GART: No AMD northbridge found.\n");
> + printk(KERN_INFO "PCI-GART: No AMD GART found.\n");
> return;
> }
>
> --
> 1.5.6.4
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-usb" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2008-10-29 09:56:13

by Joerg Roedel

[permalink] [raw]
Subject: Re: [PATCH] x86, gart: fix gart detection for Fam11h CPUs

On Tue, Oct 28, 2008 at 09:52:36AM -0700, Greg KH wrote:
> On Tue, Oct 28, 2008 at 04:13:54PM +0100, Joerg Roedel wrote:
> > The AMD Fam11h CPUs have a K8 northbridge. This northbridge is different
> > from other familys because it lacks GART support (as I just learned).
> > But the kernel implicitly expects a GART if it finds an AMD northbridge.
> > Fix this by removing the Fam11h northbridge id from the scan list of K8
> > northbridges. This patch also changes the message in the GART driver
> > about missing K8 northbridges to tell that the GART is missing which is
> > the correct information in this case.
> >
> > Signed-off-by: Joerg Roedel <[email protected]>
>
> Looks like this needs to be added to the 2.6.27-stable tree also, right?

No, this patch not. The PCI id was added during the .28 merge window.

Joerg

--
| AMD Saxony Limited Liability Company & Co. KG
Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
System | Register Court Dresden: HRA 4896
Research | General Partner authorized to represent:
Center | AMD Saxony LLC (Wilmington, Delaware, US)
| General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy

2008-10-29 23:37:04

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] x86, gart: fix gart detection for Fam11h CPUs

On Wed, Oct 29, 2008 at 10:55:25AM +0100, Joerg Roedel wrote:
> On Tue, Oct 28, 2008 at 09:52:36AM -0700, Greg KH wrote:
> > On Tue, Oct 28, 2008 at 04:13:54PM +0100, Joerg Roedel wrote:
> > > The AMD Fam11h CPUs have a K8 northbridge. This northbridge is different
> > > from other familys because it lacks GART support (as I just learned).
> > > But the kernel implicitly expects a GART if it finds an AMD northbridge.
> > > Fix this by removing the Fam11h northbridge id from the scan list of K8
> > > northbridges. This patch also changes the message in the GART driver
> > > about missing K8 northbridges to tell that the GART is missing which is
> > > the correct information in this case.
> > >
> > > Signed-off-by: Joerg Roedel <[email protected]>
> >
> > Looks like this needs to be added to the 2.6.27-stable tree also, right?
>
> No, this patch not. The PCI id was added during the .28 merge window.

Ah, you are right, sorry about that, was looking at the wrong tree here.

greg k-h