Hello,
yesterday we came across another MM related problem. This time its related to
the bttv-driver. We do very simple things:
First load it:
# modprobe bttv
Jan 14 17:28:14 fred kernel: bttv: driver version 0.7.83 loaded
Jan 14 17:28:14 fred kernel: bttv: using 2 buffers with 2080k (4160k total) for
capture Jan 14 17:28:14 fred kernel: bttv: Host bridge is VIA Technologies,
Inc. VT82C693A/694x [Apollo PRO133x] Jan 14 17:28:14 fred kernel: bttv: Host
bridge is VIA Technologies, Inc. VT82C686 [Apollo Super ACPI] Jan 14 17:28:14
fred kernel: bttv: Bt8xx card found (0). Jan 14 17:28:14 fred kernel: bttv0:
Bt848 (rev 18) at 00:0c.0, irq: 16, latency: 32, memory: 0xef000000 Jan 14
17:28:14 fred kernel: bttv0: using: BT848A( *** UNKNOWN/GENERIC **)
[card=0,autodetected] Jan 14 17:28:14 fred kernel: i2c-core.o: adapter bt848 #0
registered as adapter 0. Jan 14 17:28:14 fred kernel: bttv0: i2c: checking for
MSP34xx @ 0x80... not found Jan 14 17:28:14 fred kernel: bttv0: i2c: checking
for TDA9875 @ 0xb0... not found Jan 14 17:28:14 fred kernel: bttv0: i2c:
checking for TDA7432 @ 0x8a... not found Jan 14 17:28:14 fred kernel:
i2c-core.o: driver i2c TV tuner driver registered. Jan 14 17:28:14 fred kernel:
tuner: chip found @ 0xc2 Jan 14 17:28:14 fred kernel: bttv0: i2c attach
[client=(unset),ok] Jan 14 17:28:14 fred kernel: i2c-core.o: client [(unset)]
registered to adapter [bt848 #0](pos. 0).
Then try to use it:
# xawtv
Jan 14 17:46:53 fred kernel: bttv: vmalloc_32(4259840) failed
... and exit of course.
Uh?
Interesting about it:
Does not work with 2.4.17 nor 2.4.18-pre3, but does work with 2.4.10-SUSE.
(Same setup, only booting another kernel)
Any hints?
Regards,
Stephan
> yesterday we came across another MM related problem. This time its related to
> the bttv-driver. We do very simple things:
Do you trust your kernel build ? The bt848 drivers are working beautifully
for me in 2.4.18pre
> > The bt848 drivers are working beautifully
> > for me in 2.4.18pre
>
> Well, I had a quick look at the code, and it seems that vmalloc is just
> failing, the source line is obvious./proc/meminfo before modprobe and xawtv:
>
> total: used: free: shared: buffers: cached:
> Mem: 1054728192 120070144 934658048 0 10420224 65257472
> Swap: 1085652992 0 1085652992
>
> Can this be highmem-related?
That would make complete sense if so. The bttv uses vmalloc_32(), as the
card has 32bit limits, and I am not running bttv (nor I suspect are most
people) with highmem enabled
On Mon, 14 Jan 2002 17:40:24 +0000 (GMT)
Alan Cox <[email protected]> wrote:
> > yesterday we came across another MM related problem. This time its related
to> > the bttv-driver. We do very simple things:
>
> Do you trust your kernel build ?
Yes.
> The bt848 drivers are working beautifully
> for me in 2.4.18pre
Well, I had a quick look at the code, and it seems that vmalloc is just
failing, the source line is obvious./proc/meminfo before modprobe and xawtv:
total: used: free: shared: buffers: cached:
Mem: 1054728192 120070144 934658048 0 10420224 65257472
Swap: 1085652992 0 1085652992
MemTotal: 1030008 kB
MemFree: 912752 kB
MemShared: 0 kB
Buffers: 10176 kB
Cached: 63728 kB
SwapCached: 0 kB
Active: 26056 kB
Inactive: 73692 kB
HighTotal: 131056 kB
HighFree: 36720 kB
LowTotal: 898952 kB
LowFree: 876032 kB
SwapTotal: 1060208 kB
SwapFree: 1060208 kB
And afterwards:
total: used: free: shared: buffers: cached:
Mem: 1054728192 120537088 934191104 0 10612736 65273856
Swap: 1085652992 0 1085652992
MemTotal: 1030008 kB
MemFree: 912296 kB
MemShared: 0 kB
Buffers: 10364 kB
Cached: 63744 kB
SwapCached: 0 kB
Active: 26140 kB
Inactive: 73812 kB
HighTotal: 131056 kB
HighFree: 36608 kB
LowTotal: 898952 kB
LowFree: 875688 kB
SwapTotal: 1060208 kB
SwapFree: 1060208 kB
Can this be highmem-related?
Regards,
Stephan
On Mon, 14 Jan 2002 17:56:54 +0000 (GMT)
Alan Cox <[email protected]> wrote:
> > > The bt848 drivers are working beautifully
> > > for me in 2.4.18pre
> >
> > Well, I had a quick look at the code, and it seems that vmalloc is just
> > failing, the source line is obvious./proc/meminfo before modprobe and
xawtv:> >
> > total: used: free: shared: buffers: cached:
> > Mem: 1054728192 120070144 934658048 0 10420224 65257472
> > Swap: 1085652992 0 1085652992
> >
> > Can this be highmem-related?
>
> That would make complete sense if so. The bttv uses vmalloc_32(), as the
> card has 32bit limits, and I am not running bttv (nor I suspect are most
> people) with highmem enabled
Ok, we re-checked without highmem: it's still the same problem. I try to find
out what's so special about 2.4.10-SUSE...
Sorry for this dumb newbie question: is there an easy way (/proc?) to find out
how much vmalloc space is used/left?
Regards,
Stephan
On Mon, 14 Jan 2002 17:56:54 +0000 (GMT)
Alan Cox <[email protected]> wrote:
> > Can this be highmem-related?
>
> That would make complete sense if so. The bttv uses vmalloc_32(), as the
> card has 32bit limits, and I am not running bttv (nor I suspect are most
> people) with highmem enabled
Ok. I tracked it down. It is definitely VMALLOC-stuff. I increased the
VMALLOC_RESERVE from 128 to 256 MB and now it _works_. Here is a list of loaded
modules, maybe one (or several) of those are known to be vmalloc-fans :-)
Module Size Used by
tuner 8048 1 (autoclean)
bttv 60848 0
i2c-algo-bit 7040 1 [bttv]
i2c-core 12224 0 [tuner bttv i2c-algo-bit]
videodev 4768 2 [bttv]
NVdriver 720128 14 (autoclean)
parport_pc 12432 1 (autoclean)
lp 5984 0 (autoclean)
parport 12736 1 (autoclean) [parport_pc lp]
nfs 73024 2 (autoclean)
lockd 47056 1 (autoclean) [nfs]
sunrpc 62496 1 (autoclean) [nfs lockd]
ipv6 158464 -1 (autoclean)
uhci 24896 0 (unused)
usbcore 48640 1 [uhci]
3c59x 25312 1 (autoclean)
emu10k1 58080 0
sound 54064 0 [emu10k1]
ac97_codec 9504 0 [emu10k1]
hisax 168112 4
isdn 115088 6 [hisax]
slhc 4304 0 [isdn]
serial 44128 0 (autoclean)
How can I find out?
Regards,
Stephan
> bttv 60848 0
bttv wants a reasonable amount
> NVdriver 720128 14 (autoclean)
Thats my guess. Mapping all the memory on your geofarce will not be a small
amount.
Alan
On Mon, 14 Jan 2002 20:03:49 +0000 (GMT)
Alan Cox <[email protected]> wrote:
> > bttv 60848 0
>
> bttv wants a reasonable amount
>
> > NVdriver 720128 14 (autoclean)
>
> Thats my guess. Mapping all the memory on your geofarce will not be a small
> amount.
Ok. So what do we do about it? I mean there are possibly some more people out
there with such a problem, or - to my prediction - there will be more in the
future. I see to possibilities:
1) simply increase it overall. I have not the slightest idea what the drawbacks
are. 2) make it configurable (looks like general setup to me).
I could provide a patch for either. Do we want that?
Regards,
Stephan
> Ok. So what do we do about it? I mean there are possibly some more people out
> there with such a problem, or - to my prediction - there will be more in the
> future. I see to possibilities:
> 1) simply increase it overall. I have not the slightest idea what the drawbacks
> are. 2) make it configurable (looks like general setup to me).
Making it bigger reduces that amount of ram directly mapped by the kernel
which hits performance (nastily for 2.4 not so bad for 2.5)
On 20020114 Alan Cox wrote:
>> Ok. So what do we do about it? I mean there are possibly some more people out
>> there with such a problem, or - to my prediction - there will be more in the
>> future. I see to possibilities:
>> 1) simply increase it overall. I have not the slightest idea what the drawbacks
>> are. 2) make it configurable (looks like general setup to me).
>
>Making it bigger reduces that amount of ram directly mapped by the kernel
>which hits performance (nastily for 2.4 not so bad for 2.5)
(Sorry for joning so late to the thread...)
It is wokring fine for me, under 2.4.18-pre3 + NVidia. The difference is
that I am using version 0.7.87 (see http://giga.cps.unizar.es/~magallon/linux/2.4.18-pre3/)
I have just checked the bttv page (http://bytesex.org/bttv/) and there is
a newer version (0.7.88). What comes in .17 is 0.7.83. I have not
noticed anything relevant in changelog, but you can try...
lsmod:
werewolf:~# lsmod
Module Size Used by Tainted: P
tuner 8548 1 (autoclean)
tvaudio 9312 0 (autoclean) (unused)
msp3400 14768 1 (autoclean)
bttv 63424 0 (autoclean)
i2c-algo-bit 7244 1 (autoclean) [bttv]
videodev 4960 3 (autoclean) [bttv]
emu10k1 57728 1
sound 55052 0 [emu10k1]
ac97_codec 9472 0 [emu10k1]
soundcore 3524 7 [emu10k1 sound]
serial 45792 7
isa-pnp 28968 0 [serial]
w83781d 17792 0 (unused)
i2c-proc 6112 0 [w83781d]
i2c-isa 1188 0 (unused)
i2c-piix4 3908 0 (unused)
i2c-core 13408 0 [tuner tvaudio msp3400 bttv i2c-algo-bit w83781d i2c-proc i2c-isa i2c-piix4]
NVdriver 821600 14
agpgart 17024 3
ne2k-pci 4960 1
8390 6608 0 [ne2k-pci]
--
J.A. Magallon # Let the source be with you...
mailto:[email protected]
Mandrake Linux release 8.2 (Cooker) for i586
Linux werewolf 2.4.18-pre3-beo #5 SMP Sun Jan 13 02:14:04 CET 2002 i686
> On Mon, 14 Jan 2002, Stephan von Krawczynski wrote:
> >
> > Ok. So what do we do about it? I mean there are possibly some more
people out
> > there with such a problem, or - to my prediction - there will be
more in the
> > future. I see to possibilities:
> > 1) simply increase it overall. I have not the slightest idea what
the drawbacks
> > are. 2) make it configurable (looks like general setup to me).
> >
> > I could provide a patch for either. Do we want that?
>
> I was waiting for hpa to jump on you for suggesting that! but since
> he hasn't yet done so (publicly), here's his mail when I suggested
it
> a month ago (as a temporary aid while tracking down a similar
problem).
>
> That problem was from NTFS overusing vmalloc (and I think Anton has
> now posted the fix), but NTFS wasn't in your list, and I assume you
> didn't have it in statically.
Nope, no ntfs involved here.
I read the former thread. It was in my mind when trying the increased
VMALLOC_RESERVE, and voila, it worked again. I must admit in your case
I was very much thinking that ntfs should be fixed, and I am pleased
the maintainer thought the same :-)
Unfortunately I do not have a good idea about what to do in days where
the graphics cards have more ram onboard than my last workstation had
in total. If I understood Alans' opinion, he thinks basically the
same, what the heck can you do about such monster cards? I don't know,
other than adjusting the RESERVE-area. I have not really thought about
it right now, but I suspect it would be rather impossible to make a
somehow runtime-scalable area working, which would of course be the
complete solution to the underlying problem.
So, what's left? At least configurable? Anybody with a better idea?
Regards,
Stephan
>
> >Making it bigger reduces that amount of ram directly mapped by the kernel
> >which hits performance (nastily for 2.4 not so bad for 2.5)
>
> (Sorry for joning so late to the thread...)
> It is wokring fine for me, under 2.4.18-pre3 + NVidia. The difference is
> that I am using version 0.7.87 (see http://giga.cps.unizar.es/~magallon/linux/2.4.18-pre3/)
> I have just checked the bttv page (http://bytesex.org/bttv/) and there is
> a newer version (0.7.88). What comes in .17 is 0.7.83. I have not
> noticed anything relevant in changelog, but you can try...
MM wise it shouldn't make a difference whenever you are using 0.7.83 or
0.7.88 (I've mailed 0.7.88 patches to macelo for 2.4.18 btw). The 0.8.x
versions have a complete different way to do the memory management.
Gerd
--
#define ENOCLUE 125 /* userland programmer induced race condition */
On 15 Jan 2002 10:17:03 GMT
Gerd Knorr <[email protected]> wrote:
> MM wise it shouldn't make a difference whenever you are using 0.7.83 or
> 0.7.88 (I've mailed 0.7.88 patches to macelo for 2.4.18 btw). The 0.8.x
> versions have a complete different way to do the memory management.
No vmallocs?
Regards,
Stephan
On Tue, Jan 15, 2002 at 12:14:24PM +0100, Stephan von Krawczynski wrote:
> On 15 Jan 2002 10:17:03 GMT
> Gerd Knorr <[email protected]> wrote:
>
> > MM wise it shouldn't make a difference whenever you are using 0.7.83 or
> > 0.7.88 (I've mailed 0.7.88 patches to macelo for 2.4.18 btw). The 0.8.x
> > versions have a complete different way to do the memory management.
>
> No vmallocs?
Yes. Instead of remapping vmalloced kernel memory it gives you shared
anonymous pages, then does zerocopy DMA using kiobufs. You may run in
trouble with >4GB machines.
Gerd
--
#define ENOCLUE 125 /* userland programmer induced race condition */
On Tue, 15 Jan 2002 14:20:17 +0100
Gerd Knorr <[email protected]> wrote:
> On Tue, Jan 15, 2002 at 12:14:24PM +0100, Stephan von Krawczynski wrote:
> > On 15 Jan 2002 10:17:03 GMT
> > Gerd Knorr <[email protected]> wrote:
> >
> > > MM wise it shouldn't make a difference whenever you are using 0.7.83 or
> > > 0.7.88 (I've mailed 0.7.88 patches to macelo for 2.4.18 btw). The 0.8.x
> > > versions have a complete different way to do the memory management.
> >
> > No vmallocs?
>
> Yes. Instead of remapping vmalloced kernel memory it gives you shared
> anonymous pages, then does zerocopy DMA using kiobufs. You may run in
> trouble with >4GB machines.
Interesting.
What's the problem on > 4GB ?
Regards,
Stephan
On Tue, Jan 15, 2002 at 02:46:52PM +0100, Stephan von Krawczynski wrote:
> On Tue, 15 Jan 2002 14:20:17 +0100
> Gerd Knorr <[email protected]> wrote:
>
> > Yes. Instead of remapping vmalloced kernel memory it gives you shared
> > anonymous pages, then does zerocopy DMA using kiobufs. You may run in
> > trouble with >4GB machines.
>
> Interesting.
> What's the problem on > 4GB ?
The bt878/848 is a 32bit PCI device, it simply can't go DMA to main
memory above 4GB. At least on ia32, on architecures with a iommu
(sparc, ...) it should work without trouble.
Gerd
--
#define ENOCLUE 125 /* userland programmer induced race condition */
On Tue, 15 Jan 2002 16:16:53 +0100
Gerd Knorr <[email protected]> wrote:
> On Tue, Jan 15, 2002 at 02:46:52PM +0100, Stephan von Krawczynski wrote:
> > On Tue, 15 Jan 2002 14:20:17 +0100
> > Gerd Knorr <[email protected]> wrote:
> >
> > > Yes. Instead of remapping vmalloced kernel memory it gives you shared
> > > anonymous pages, then does zerocopy DMA using kiobufs. You may run in
> > > trouble with >4GB machines.
> >
> > Interesting.
> > What's the problem on > 4GB ?
>
> The bt878/848 is a 32bit PCI device, it simply can't go DMA to main
> memory above 4GB. At least on ia32, on architecures with a iommu
> (sparc, ...) it should work without trouble.
Sorry, maybe I should have clarified: what is the problem with allocating those
pages below 4GB in a >4GB box?
Regards,
Stephan
> > > > Yes. Instead of remapping vmalloced kernel memory it gives you shared
> > > > anonymous pages, then does zerocopy DMA using kiobufs. You may run in
> > > > trouble with >4GB machines.
> > >
> > > Interesting.
> > > What's the problem on > 4GB ?
> >
> > The bt878/848 is a 32bit PCI device, it simply can't go DMA to main
> > memory above 4GB. At least on ia32, on architecures with a iommu
> > (sparc, ...) it should work without trouble.
>
> Sorry, maybe I should have clarified: what is the problem with allocating those
> pages below 4GB in a >4GB box?
do_anonymous_page() may give you pages above 4GB. Need to write my own
nopage handler to fix this.
Gerd
--
#define ENOCLUE 125 /* userland programmer induced race condition */
On Wed, 16 Jan 2002 10:57:47 +0100
Gerd Knorr <[email protected]> wrote:
> > > > > Yes. Instead of remapping vmalloced kernel memory it gives you
shared> > > > > anonymous pages, then does zerocopy DMA using kiobufs. You may
run in> > > > > trouble with >4GB machines.
> > > >
> > > > Interesting.
> > > > What's the problem on > 4GB ?
> > >
> > > The bt878/848 is a 32bit PCI device, it simply can't go DMA to main
> > > memory above 4GB. At least on ia32, on architecures with a iommu
> > > (sparc, ...) it should work without trouble.
> >
> > Sorry, maybe I should have clarified: what is the problem with allocating
those> > pages below 4GB in a >4GB box?
>
> do_anonymous_page() may give you pages above 4GB.
Guess it is a bit ... flaky ... to just do a
page = alloc_page(GFP_HIGHUSER);
if (!page)
goto no_mem;
in
do_anonymous_page()
I guess it should be somehow configurable (parameter). Your idea of using it
may grow and mem too, for sure.
Regards,
Stephan