4Kb kernel stacks are the future on i386, and it seems the problems it
initially caused are now sorted out.
Does anyone knows about any currently unsolved problems?
I'd like to:
- get a patch into on of the next -mm kernels that unconditionally
enables 4KSTACKS
- if there won't be new reports of breakages, send a patch to
completely remove !4KSTACKS for 2.6.15
In -mm, Reiser4 still has a dependency on !4KSTACKS.
I've mentioned this at least twice to the Reiser4 people, and they
should check why this dependency is still there and if there are still
stack issues in Reiser4 fix them.
If not people using Reiser4 on i386 will have to decide whether to
switch the filesystem or the architecture...
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
On Fri, Sep 02, 2005 at 02:39:15AM +0200, Adrian Bunk wrote:
> 4Kb kernel stacks are the future on i386, and it seems the problems
> it initially caused are now sorted out.
Not entirely.
XFS when mixed with raid/lvm/nfs still blows up. It's probably not
alone in this respect but worse than ext2/3.
On Thu, Sep 01, 2005 at 10:33:56PM -0700, Chris Wedgwood wrote:
> On Fri, Sep 02, 2005 at 02:39:15AM +0200, Adrian Bunk wrote:
>
> > 4Kb kernel stacks are the future on i386, and it seems the problems
> > it initially caused are now sorted out.
>
> Not entirely.
>
> XFS when mixed with raid/lvm/nfs still blows up. It's probably not
> alone in this respect but worse than ext2/3.
To clarify, you mean AND not OR (/) there -- in other words,
raid[+raid]+dm[+dm]+xfs+nfs can be fatal, yes.
We have no known failing cases other than several-deep stacked
driver cases which are also likely evil for other filesystems,
as you say. Which is not to say I'm in support of this patch..
just don't keep dissing XFS for this, we went and made a number
of at-times-painful changes to make this option work as best it
can.
cheers.
--
Nathan
On Friday September 2, [email protected] wrote:
> On Thu, Sep 01, 2005 at 10:33:56PM -0700, Chris Wedgwood wrote:
> > On Fri, Sep 02, 2005 at 02:39:15AM +0200, Adrian Bunk wrote:
> >
> > > 4Kb kernel stacks are the future on i386, and it seems the problems
> > > it initially caused are now sorted out.
> >
> > Not entirely.
> >
> > XFS when mixed with raid/lvm/nfs still blows up. It's probably not
> > alone in this respect but worse than ext2/3.
>
> To clarify, you mean AND not OR (/) there -- in other words,
> raid[+raid]+dm[+dm]+xfs+nfs can be fatal, yes.
It should be reasonably simple to remove this problem of stacked
drivers.
There really isn't any need for md and dm (or md and md or ..) to use
the stack and the same time.
The following patch, which I posted in November last year, arranges
that if generic_make_request is called recursively, then the instead
of doing anything, it just takes a copy of the 'bio', and deals with
it when the parent request finishes.
So it effectively converts the 'and' back to an 'or'.
It didn't get much of an enthusiastic response then. Maybe it is time
to try again. Anyone interested?
(I don't know if the patch still applies, but it should come close).
NeilBrown
==============================================
Signed-off-by: Neil Brown <[email protected]>
### Diffstat output
./drivers/block/ll_rw_blk.c | 38 +++++++++++++++++++++++++++++++++++++-
./include/linux/sched.h | 3 +++
2 files changed, 40 insertions(+), 1 deletion(-)
diff ./drivers/block/ll_rw_blk.c~current~ ./drivers/block/ll_rw_blk.c
--- ./drivers/block/ll_rw_blk.c~current~ 2004-11-16 15:55:55.000000000 +1100
+++ ./drivers/block/ll_rw_blk.c 2004-11-25 10:05:14.000000000 +1100
@@ -2609,7 +2609,7 @@ static inline void block_wait_queue_runn
* bi_sector for remaps as it sees fit. So the values of these fields
* should NOT be depended on after the call to generic_make_request.
*/
-void generic_make_request(struct bio *bio)
+static inline void __generic_make_request(struct bio *bio)
{
request_queue_t *q;
sector_t maxsector;
@@ -2686,6 +2686,42 @@ end_io:
} while (ret);
}
+/*
+ * We only want one ->make_request_fn to be active at a time,
+ * else stack usage with stacked devices could be a problem.
+ * So use current->bio_{list,tail} to keep a list of requests
+ * submited by a make_request_fn function.
+ * current->bio_tail is also used as a flag to say if
+ * generic_make_request is currently activce in this task or not.
+ * If it is NULL, then no make_request is active. If it is non-NULL,
+ * then a make_request is active, and new requests should be added
+ * at the tail
+ */
+void generic_make_request(struct bio *bio)
+{
+ if (current->bio_tail) {
+ /* make_request is active */
+ *(current->bio_tail) = bio;
+ bio->bi_next = NULL;
+ current->bio_tail = &bio->bi_next;
+ return;
+ }
+ /* not active yet, make it active */
+ current->bio_list = NULL;
+ current->bio_tail = & current->bio_list;
+ __generic_make_request(bio);
+ while (current->bio_list) {
+ bio = current->bio_list;
+ current->bio_list = bio->bi_next;
+ if (bio->bi_next == NULL)
+ current->bio_tail = ¤t->bio_list;
+ else
+ bio->bi_next = NULL;
+ __generic_make_request(bio);
+ }
+ current->bio_tail = NULL; /* deactivate */
+}
+
EXPORT_SYMBOL(generic_make_request);
/**
diff ./include/linux/sched.h~current~ ./include/linux/sched.h
--- ./include/linux/sched.h~current~ 2004-11-25 09:57:07.000000000 +1100
+++ ./include/linux/sched.h 2004-11-25 09:57:34.000000000 +1100
@@ -649,6 +649,9 @@ struct task_struct {
/* journalling filesystem info */
void *journal_info;
+
+/* stacked block device info */
+ struct bio *bio_list, **bio_tail;
/* VM state */
struct reclaim_state *reclaim_state;
Hello
Adrian Bunk wrote:
> 4Kb kernel stacks are the future on i386, and it seems the problems it
> initially caused are now sorted out.
>
> Does anyone knows about any currently unsolved problems?
>
> I'd like to:
> - get a patch into on of the next -mm kernels that unconditionally
> enables 4KSTACKS
> - if there won't be new reports of breakages, send a patch to
> completely remove !4KSTACKS for 2.6.15
>
> In -mm, Reiser4 still has a dependency on !4KSTACKS.
> I've mentioned this at least twice to the Reiser4 people, and they
> should check why this dependency is still there and if there are still
> stack issues in Reiser4 fix them.
>
We are about to send reiser4 update which makes it to not depend on !4kstack.
> If not people using Reiser4 on i386 will have to decide whether to
> switch the filesystem or the architecture...
>
> cu
> Adrian
>
Adrian Bunk wrote:
>4Kb kernel stacks are the future on i386, and it seems the problems it
>initially caused are now sorted out.
>
>Does anyone knows about any currently unsolved problems?
>
>I'd like to:
>- get a patch into on of the next -mm kernels that unconditionally
> enables 4KSTACKS
>- if there won't be new reports of breakages, send a patch to
> completely remove !4KSTACKS for 2.6.15
>
>In -mm, Reiser4 still has a dependency on !4KSTACKS.
>I've mentioned this at least twice to the Reiser4 people, and they
>should check why this dependency is still there and if there are still
>stack issues in Reiser4 fix them.
>
>If not people using Reiser4 on i386 will have to decide whether to
>switch the filesystem or the architecture...
>
>cu
>Adrian
>
>
>
Can you wait just one month after we get in for this? Or even 2 weeks.
vs needs a weekend, etc.....
On Fri, 2005-09-02 at 02:39 +0200, Adrian Bunk wrote:
> 4Kb kernel stacks are the future on i386, and it seems the problems it
> initially caused are now sorted out.
>
> Does anyone knows about any currently unsolved problems?
ndiswrapper
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Lee Revell wrote:
> On Fri, 2005-09-02 at 02:39 +0200, Adrian Bunk wrote:
>
>>4Kb kernel stacks are the future on i386, and it seems the problems it
>>initially caused are now sorted out.
>>
>>Does anyone knows about any currently unsolved problems?
>
>
> ndiswrapper
While I agree ndiswrapper has a use ... I don't think we should
base kernel development upon messing with something that is designed
to run a windows driver in linux ...
// Stefan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFDGqPiBrn2kJu9P78RAgO4AJ9r6FNwB+72iRmdcMoxP0vi8gTDUQCfeUG5
5Qbcq/o/Zao79JPEVqOmH+M=
=xpUz
-----END PGP SIGNATURE-----
On Sun, 2005-09-04 at 09:36 +0200, Stefan Smietanowski wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Lee Revell wrote:
> > On Fri, 2005-09-02 at 02:39 +0200, Adrian Bunk wrote:
> >
> >>4Kb kernel stacks are the future on i386, and it seems the problems it
> >>initially caused are now sorted out.
> >>
> >>Does anyone knows about any currently unsolved problems?
> >
> >
> > ndiswrapper
>
> While I agree ndiswrapper has a use ... I don't think we should
> base kernel development upon messing with something that is designed
> to run a windows driver in linux ...
Good point, but I don't think we should needlessly render people's
hardware inoperable either.
Lee
2005/9/4, Lee Revell <[email protected]>:
> On Fri, 2005-09-02 at 02:39 +0200, Adrian Bunk wrote:
> > 4Kb kernel stacks are the future on i386, and it seems the problems it
> > initially caused are now sorted out.
> >
> > Does anyone knows about any currently unsolved problems?
>
> ndiswrapper
Just a thought : why couldn't ndiswrapper set apart some piece
of memory and use it as the stack by changing the esp register
before executing windows code.
Like http://article.gmane.org/gmane.linux.drivers.ndiswrapper.general/4737
It's dirty, I know, but after all they are executing win32 code ...
Why wouldn't this work ?
--
Guillaume
Adrian Bunk wrote (ao):
> 4Kb kernel stacks are the future on i386, and it seems the problems it
> initially caused are now sorted out.
>
> Does anyone knows about any currently unsolved problems?
>
> I'd like to:
> - get a patch into on of the next -mm kernels that unconditionally
> enables 4KSTACKS
> - if there won't be new reports of breakages, send a patch to
> completely remove !4KSTACKS for 2.6.15
>
> In -mm, Reiser4 still has a dependency on !4KSTACKS.
> I've mentioned this at least twice to the Reiser4 people, and they
> should check why this dependency is still there and if there are still
> stack issues in Reiser4 fix them.
>
> If not people using Reiser4 on i386 will have to decide whether to
> switch the filesystem or the architecture...
FWIW: I use Reiser4 for several months now on several i386 systems, and
I always remove the "&& !4STACKS" from fs/reiser4/Kconfig and enable
CONFIG_4KSTACKS:
# zgrep -E 'REISER4|4KSTACKS' /proc/config.gz
CONFIG_REISER4_FS=y
# CONFIG_REISER4_DEBUG is not set
CONFIG_4KSTACKS=y
Also using lvm2, or raid1, or raid5, on ATA, SATA or SCSI. I haven't
experienced any problems yet, but the systems don't see heavy usage too.
Kind regards, Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
>Please don't tell me to "care for closed-source drivers".
ndiswrapper is NOT closed source. And I'm not asking you to "care".
>I don't want the pain of debugging crashes on the machines which run unknown code
>in kernel space.
I'm not asking you to debug crashes. I'm simply requesting that the
kernel stack size situation remain as it is: with 8K as the default
and 4K configurable.
>vda
I code, therefore I am
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Alex Davis wrote:
>>Please don't tell me to "care for closed-source drivers".
>
> ndiswrapper is NOT closed source. And I'm not asking you to "care".
While ndiswrapper might not be closed source, I would not call the
windows driver it loads open source ...
There is really no difference between madwifi+closed source hal
to ndiswrapper+windows driver.
Both are just as (non-)debuggable.
Ok, madwifi exposes more of it's driver source to us, but still.
// Stefan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
iD8DBQFDGyrOBrn2kJu9P78RAvIxAJ426eTZbTtB5v92A+ipxsKEFW4sPACgnYna
iXGFMduZgJ92UriSHvwTW3g=
=YcU6
-----END PGP SIGNATURE-----
* Stefan Smietanowski ([email protected]) wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Alex Davis wrote:
> >>Please don't tell me to "care for closed-source drivers".
> >
> > ndiswrapper is NOT closed source. And I'm not asking you to "care".
>
> While ndiswrapper might not be closed source, I would not call the
> windows driver it loads open source ...
Would it be possible for ndiswrapper to provide a seperate stack
to the windows code so this problem is seperable?
How debuggable is stack overrun? I mean if I am the unlucky
admin who decides to setup a crypto/raid/stripe/thing setup
which runs out of stack somewhere how easy will it be
for someone who doesn't know the innards to know what
happened as opposed to running into a random oops?
Dave
--
-----Open up your eyes, open up your mind, open up your code -------
/ Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \
\ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex /
\ _________________________|_____ http://www.treblig.org |_______/
Guillaume Chazarain wrote:
> Just a thought : why couldn't ndiswrapper set apart some piece
> of memory and use it as the stack by changing the esp register
> before executing windows code.
>
> Like http://article.gmane.org/gmane.linux.drivers.ndiswrapper.general/4737
>
> It's dirty, I know, but after all they are executing win32 code ...
>
> Why wouldn't this work ?
I think this would be a good idea. I don't see any reason in principle
why the ndiswrapper code couldn't use a separate stack for the Win32
driver code. Sharing the stack between the Linux kernel and whatever
junk is going on inside the Windows driver seems rather inherently fragile..
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
On Sun, Sep 04, 2005 at 04:23:04AM -0400, Lee Revell wrote:
> On Sun, 2005-09-04 at 09:36 +0200, Stefan Smietanowski wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Lee Revell wrote:
> > > On Fri, 2005-09-02 at 02:39 +0200, Adrian Bunk wrote:
> > >
> > >>4Kb kernel stacks are the future on i386, and it seems the problems it
> > >>initially caused are now sorted out.
> > >>
> > >>Does anyone knows about any currently unsolved problems?
> > >
> > >
> > > ndiswrapper
> >
> > While I agree ndiswrapper has a use ... I don't think we should
> > base kernel development upon messing with something that is designed
> > to run a windows driver in linux ...
>
> Good point, but I don't think we should needlessly render people's
> hardware inoperable either.
The NdisWrapper FAQ already tells you that you need a patch for some of
the binary-only Windows drivers that require more than 8kB stacks.
And the fact that NdisWrapper is mostly working hinders the development
of open source drivers for this hardware.
> Lee
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
Bas Westerbaan <[email protected]> wrote:
> > Yes you are. You're asking for 4KSTACKS config option to maintained
> > and it is not something you get for free. Besides, if it is indeed
> > ripped out of mainline kernel, you can always keep it a separate patch
> > for ndiswrapper.
> Though 4K stacks are used a lot, they probably aren't used on all
> configurations yet. Other situations may arise where 8K stacks may be
> preferred. It is too early to kill 8K stacks imho.
At least Fedora ships 4Kstacks kernel for quite a while now. No, it is not
"everywhere", but close.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
>The NdisWrapper FAQ already tells you that you need a patch for some of
>the binary-only Windows drivers that require more than 8kB stacks.
>
>And the fact that NdisWrapper is mostly working hinders the development
>of open source drivers for this hardware.
If the hardware manufacturer won't give you the spec's then writing a driver
is going to be pretty difficult, if not impossible. Reverse-engineering
may be possible, but still....
I believe it's the lack of spec's, rather than the existence of ndiswrapper
and driverloader, that hinder driver development.
-Alex
I code, therefore I am
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
On Sun, Sep 04, 2005 at 06:30:29PM -0700, Alex Davis wrote:
> >The NdisWrapper FAQ already tells you that you need a patch for some of
> >the binary-only Windows drivers that require more than 8kB stacks.
> >
> >And the fact that NdisWrapper is mostly working hinders the development
> >of open source drivers for this hardware.
>
> If the hardware manufacturer won't give you the spec's then writing a driver
> is going to be pretty difficult, if not impossible. Reverse-engineering
> may be possible, but still....
>
> I believe it's the lack of spec's, rather than the existence of ndiswrapper
> and driverloader, that hinder driver development.
How do you put pressure on hardware manufacturers for getting them to
release the specs?
If they are able to write "supported by Linux" on their products anyway
because there's a driver that runs under NdisWrapper?
Or if people return/don't buy the hardware because of missing Linux
support reducing the revenue of the hardware manufacturer by some $$ ?
> -Alex
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
On Mon, 5 Sep 2005, Adrian Bunk wrote:
> How do you put pressure on hardware manufacturers for getting them to
> release the specs?
>
> If they are able to write "supported by Linux" on their products anyway
> because there's a driver that runs under NdisWrapper?
that's specious... they can put "supported by Linux" on their products by
supplying a binary-only driver too... ndiswrapper is orthogonal to that
problem.
on a tangent... has there been any further research/work on page
clustering? wli's patches haven't been updated in a while... it's another
way to supply larger stacks. (hell with 16KiB "pages" i'd be stoked to
"waste" the bottom 4KiB of each stack with an unmapped page to ensure
overflow is detected.)
-dean
Dave Jones wrote:
>- NDISwrapper / driverloader.
> (Shock, horror - no-one cares).
Shock, horror. Someone DOES care: everyone who uses ndiswrapper or
driverloader, whether they know it or not. Are you proposing that
we punish the end-users because of the obstinence of the hardware
manufacturers? If/when native drivers are written, maybe we can
revisit this.
I code, therefore I am
______________________________________________________
Click here to donate to the Hurricane Katrina relief effort.
http://store.yahoo.com/redcross-donate3/
On Sun, September 4, 2005 10:00 pm, Alex Davis said:
> Dave Jones wrote:
>>- NDISwrapper / driverloader.
>> (Shock, horror - no-one cares).
>
> Shock, horror. Someone DOES care: everyone who uses ndiswrapper or
> driverloader, whether they know it or not. Are you proposing that
> we punish the end-users because of the obstinence of the hardware
> manufacturers? If/when native drivers are written, maybe we can
> revisit this.
Continuing the promotion of binary-only drivers _hurts_ the demand for
(and thus the development of) open source drivers. Read the comment from
Dave as something like "Nobody who matters, with regard to kernel
development, cares about NDISwrapper". If _you_ care, fork your own tree
and maintain the patch as necessary.
Regards,
Sean
--- Sean <[email protected]> wrote:
> On Sun, September 4, 2005 10:00 pm, Alex Davis said:
> > Dave Jones wrote:
> >>- NDISwrapper / driverloader.
> >> (Shock, horror - no-one cares).
> >
> > Shock, horror. Someone DOES care: everyone who uses ndiswrapper or
> > driverloader, whether they know it or not. Are you proposing that
> > we punish the end-users because of the obstinence of the hardware
> > manufacturers? If/when native drivers are written, maybe we can
> > revisit this.
>
>
> Continuing the promotion of binary-only drivers _hurts_ the demand for
> (and thus the development of) open source drivers. Read the comment from
> Dave as something like "Nobody who matters, with regard to kernel
> development, cares about NDISwrapper". If _you_ care, fork your own tree
> and maintain the patch as necessary.
>
> Regards,
> Sean
Sean:
Linux isn't just used by kernel developers. It's that attitude that helps insure
Linux will always have a small userbase. Lack of numbers just gives the manufacturers
another reason not to care about us. Joe User doesn't care about our philosophical
issues, nor should he. When I install Linux on someone's machine, he wants it to work
NOW, not in some 'perfect-world' future.
-Alex
-Alex
>
I code, therefore I am
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
On Sun, September 4, 2005 10:29 pm, Alex Davis said:
> Linux isn't just used by kernel developers. It's that attitude that
> helps insure Linux will always have a small userbase. Lack of numbers
> just gives the manufacturers another reason not to care about us. Joe
> User doesn't care about our philosophical issues, nor should he. When
> I install Linux on someone's machine, he wants it to work NOW, not in
> some 'perfect-world' future.
Alex,
I think you're wrong on just about all counts. The developers get to
_dictate_ what goes into Linux because they're footing the bill. Their
past performance shows that your worries are unfounded (as the vast
acceptance of Linux long _before_ binary-only drivers even existed,
shows).
Please stop the hand wringing over not being able to convince Joe User to
install Linux today. It will happen naturally when it's appropriate,
Linux is steadily growing in power and acceptance without binary drivers.
Trying to artificially speed up the process by destroying the very thing
that makes Linux an important force is actually counterproductive.
It's not a philosophical issue, it's what Linux _is_: an open source
operating system! That's what the developers are working on; not your
half-baked vision. If you want to create some bastardized version and are
willing to commit dollars and effort to maintaining the code needed to do
so, feel free.
Regards,
Sean
--- Sean <[email protected]> wrote:
> On Sun, September 4, 2005 10:29 pm, Alex Davis said:
>
> > Linux isn't just used by kernel developers. It's that attitude that
> > helps insure Linux will always have a small userbase. Lack of numbers
> > just gives the manufacturers another reason not to care about us. Joe
> > User doesn't care about our philosophical issues, nor should he. When
> > I install Linux on someone's machine, he wants it to work NOW, not in
> > some 'perfect-world' future.
>
> Alex,
>
>
> Please stop the hand wringing over not being able to convince Joe User to
> install Linux today. It will happen naturally when it's appropriate,
It will never be 'appropriate' if the system doesn't somehow work on Joe's
hardware. We currently have something that works. In my opinion it's
pointless to take that away. The manufacturers will still stone-wall us
regardless of ndiswrapper's existence. They were doing it before ndis-
wrapper existed.
> Linux is steadily growing in power and acceptance without binary drivers.
> Trying to artificially speed up the process by destroying the very thing
> that makes Linux an important force is actually counterproductive.
Please explain how Linux can be an 'important force' if people can't
use it? Wireless networking is very important to people.
> It's not a philosophical issue, it's what Linux _is_: an open source
> operating system! That's what the developers are working on; not your
> half-baked vision.
Um, ever hear of 'compromise'?? All I'm saying is let people use what
currently works until we can get an open-source solution. Ndiswrapper's
existence is not stopping you (or anyone else) from pestering manufacturers
for spec's and writing drivers. I look at ndiswrapper as a stop-gap solution.
Hey, even Linus himself has said 'better a sub-optimal solution than no solution'.
> If you want to create some bastardized version and are
> willing to commit dollars and effort to maintaining the code needed to do
> so, feel free.
>
> Regards,
> Sean
>
>
>
I code, therefore I am
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
On Sun, September 4, 2005 11:41 pm, Alex Davis said:
> It will never be 'appropriate' if the system doesn't somehow work on Joe's
> hardware. We currently have something that works. In my opinion it's
> pointless to take that away. The manufacturers will still stone-wall us
> regardless of ndiswrapper's existence. They were doing it before ndis-
> wrapper existed.
There are lots and lots of systems where Linux works. Encouraging users
to buy hardware that works with Linux, can only help. Encouraging them
that it doesn't matter and that binary-only drivers are a good
alternative, can only hurt.
> Please explain how Linux can be an 'important force' if people can't
> use it? Wireless networking is very important to people.
Lots of people can and do use Linux without ANY binary drivers. There
are Wireless choices that don't require binary only drivers.
> Um, ever hear of 'compromise'?? All I'm saying is let people use what
> currently works until we can get an open-source solution. Ndiswrapper's
> existence is not stopping you (or anyone else) from pestering
> manufacturers
> for spec's and writing drivers. I look at ndiswrapper as a stop-gap
> solution.
> Hey, even Linus himself has said 'better a sub-optimal solution than no
> solution'.
Nobody is stopping anyone from using what "currently works", there will be
lots of like minded people to provide crap kernels and shitty binary
drivers to people who don't know better. So don't worry, your well
intentioned vision of the future will survive the removal of 8K stacks
from the kernel.
Regards,
Sean
On Sep 4, 2005, at 23:41:58, Alex Davis wrote:
> --- Sean <[email protected]> wrote:
>> It's not a philosophical issue, it's what Linux _is_: an open source
>> operating system! That's what the developers are working on; not your
>> half-baked vision.
> Um, ever hear of 'compromise'?? All I'm saying is let people use what
> currently works until we can get an open-source solution.
> Ndiswrapper's
> existence is not stopping you (or anyone else) from pestering
> manufacturers
> for spec's and writing drivers. I look at ndiswrapper as a stop-gap
> solution.
> Hey, even Linus himself has said 'better a sub-optimal solution
> than no solution'.
In any case, this discussion is moot because the kernel API is changing
for the better and there is a clearly defined fix for ndiswrapper that
will allow it to continue to work even with the new interface: allocate
a separate ndiswrapper stack (IE: Not the kernel stacks). The kernel is
under no obligation not to break out-of-tree drivers, etc, even semi-
non-
-binary-only ones such as ndiswrapper. Figure out how to fix it and
move on!
Cheers,
Kyle Moffett
--
Q: Why do programmers confuse Halloween and Christmas?
A: Because OCT 31 == DEC 25.
--- Sean <[email protected]> wrote:
> On Sun, September 4, 2005 11:41 pm, Alex Davis said:
>
> > It will never be 'appropriate' if the system doesn't somehow work on Joe's
> > hardware. We currently have something that works. In my opinion it's
> > pointless to take that away. The manufacturers will still stone-wall us
> > regardless of ndiswrapper's existence. They were doing it before ndis-
> > wrapper existed.
>
> There are lots and lots of systems where Linux works. Encouraging users
> to buy hardware that works with Linux, can only help. Encouraging them
> that it doesn't matter and that binary-only drivers are a good
> alternative, can only hurt.
>
> > Please explain how Linux can be an 'important force' if people can't
> > use it? Wireless networking is very important to people.
>
> Lots of people can and do use Linux without ANY binary drivers. There
> are Wireless choices that don't require binary only drivers.
What if you don't have a choice? When someone comes to me with their laptop
containing a built-in wireless card not natively supported by Linux, am I
supposed to tell them "go buy a Linux-supported card" when there's a way
I can make their existing card work? I don't think so.
> > Um, ever hear of 'compromise'?? All I'm saying is let people use what
> > currently works until we can get an open-source solution. Ndiswrapper's
> > existence is not stopping you (or anyone else) from pestering
> > manufacturers
> > for spec's and writing drivers. I look at ndiswrapper as a stop-gap
> > solution.
> > Hey, even Linus himself has said 'better a sub-optimal solution than no
> > solution'.
>
> Nobody is stopping anyone from using what "currently works", there will be
> lots of like minded people to provide crap kernels and shitty binary
> drivers to people who don't know better.
> So don't worry, your well
> intentioned vision of the future will survive the removal of 8K stacks
> from the kernel.
> Regards,
> Sean
>
>
>
I code, therefore I am
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
On Mon, September 5, 2005 12:03 am, Alex Davis said:
>
> What if you don't have a choice? When someone comes to me with their
> laptop
> containing a built-in wireless card not natively supported by Linux, am I
> supposed to tell them "go buy a Linux-supported card" when there's a way
> I can make their existing card work? I don't think so.
You always have a choice in life. Nobody is stopping you from doing what
_you_ choose to do. That doesn't mean that developers who are concerned
with the creation and promotion of open source should care one whit about
your particular take on the situation. Go do whatever you want just
don't expect the open source developers to pay for it; you maintain the
crufty patches yourself.
If you want to see the stupidity of your argument imagine someone going to
Microsoft and saying... "look, if you'd simply stop charging for MS
Windows more people would embrace it!" Do you think therefore microsoft
should stop charging for Windows???
What if I wrote an email to them and said... "A friend of mine brings me
his laptop to install Windows on it, but he can't afford a copy of
Windows!!!" Do you think they should stop charging for Windows to help
you out?? Get real.
Please just do whatever you want and stop hoping that open source
developers will ever care about your choice to embrace binary-only
drivers.
Cheers,
Sean
On Sun, Sep 04, 2005 at 08:41:58PM -0700, Alex Davis wrote:
> It will never be 'appropriate' if the system doesn't somehow work on Joe's
> hardware. We currently have something that works.
"works".
As someone who gets to read a lot of bug reports from end-users,
this thing is far from perfect judging by the number of tainted
oopses I've seen, and not all of them look like stack size issues.
The fact is these situations are undebuggable. Where does that
leave users ?
> Please explain how Linux can be an 'important force' if people can't
> use it? Wireless networking is very important to people.
There are plenty of cards available, which work just fine with
free drivers.
> Um, ever hear of 'compromise'?? All I'm saying is let people use what
> currently works until we can get an open-source solution. Ndiswrapper's
> existence is not stopping you (or anyone else) from pestering manufacturers
> for spec's and writing drivers. I look at ndiswrapper as a stop-gap solution.
A lot of users look at it as nirvana. I've seen users report bugs against
ancient kernels, that are extremely likely to be fixed in later kernels,
yet they're unwilling to move to a newer kernel due to them being tied
to a driver that only works on an older kernel, despite the fact that there
are known security and data corruption issues with the older kernels.
Helping the cause of binary (or part binary) solutions doesn't solve anything.
It brings nothing but unsolvable problems, and upset users when their problems
can't get fixed.
Dave
Hi,
On Mon, Sep 05, 2005 at 12:12:56AM -0400, Sean wrote:
> On Mon, September 5, 2005 12:03 am, Alex Davis said:
> >
> > What if you don't have a choice? When someone comes to me with their
> > laptop
> > containing a built-in wireless card not natively supported by Linux, am I
> > supposed to tell them "go buy a Linux-supported card" when there's a way
> > I can make their existing card work? I don't think so.
>
> You always have a choice in life. Nobody is stopping you from doing what
> _you_ choose to do. That doesn't mean that developers who are concerned
> with the creation and promotion of open source should care one whit about
> your particular take on the situation. Go do whatever you want just
> don't expect the open source developers to pay for it; you maintain the
> crufty patches yourself.
Well, to be fair, most laptop users today are in companies which provide
them with the model the staff has chosen for all the employees. And employees
try to install Linux on them anyway. That's how you end up with things like
ndiswrapper, because the people who make those notebooks for companies don't
care at all about their customers ; what they want is negociate with the
staff to sell them 2000 laptops, and that's all.
However, those employees generally start with "easy to install" distros,
such as fedora or similar. Those don't ship with standard kernels anyway,
and they provide what is necessary to stay compatible with the tools which
support crappy hardware. So (and it's sad), all those people don't care at
all about kernel development, nor do they care about 2.6.13, 2.6.14, etc...
It's just *us* who tell them "ah, you can't do that because you use a
prehistorical kernel, let me install you the last one", and then and only
then, that fails and we whine about the crappy hardware and the "easy to
install" distro which seem to be made to work together.
So I tend to consider that ndiswrapper users won't care about what will
be in 2.6.14, but only about the fact that *their distro* will still
support ndiswrapper. Of course, it would have been easier if the kernel
would have stabilized and was only in maintenance mode, not breaking
anything on every new release, but Linus has decided differently, so be
it. Those users did not have a choice in the beginning, otherwise they
would have chosen a laptop supported by Linux. And honnestly, I know
plenty of them. In fact, it's the other way around. I know very few
people who can actually choose their laptop.
So you're both half-right from your point of view. But you're both half-wrong
too : no, people can't always choose, and no, people who don't choose their
laptop are not impacted by development kernels. So let's turn the page and
wait for a stable kernel.
Regards,
Willy
Hi Dave,
On Mon, Sep 05, 2005 at 12:26:41AM -0400, Dave Jones wrote:
(...)
> A lot of users look at it as nirvana. I've seen users report bugs against
> ancient kernels, that are extremely likely to be fixed in later kernels,
> yet they're unwilling to move to a newer kernel due to them being tied
> to a driver that only works on an older kernel, despite the fact that there
> are known security and data corruption issues with the older kernels.
>
> Helping the cause of binary (or part binary) solutions doesn't solve anything.
> It brings nothing but unsolvable problems, and upset users when their problems
> can't get fixed.
The problem is not whether to support or not to support binary only drivers,
I personnally am fairly against such drivers and don't have any on any of my
systems. But the problem is that something that works in a *stable* series
kernel does not work anymore in the next *stable* series which should
theorically only fix bugs. And that's what upsets your users. As you said,
when you tell them to upgrade, they refuse because the only thing they need
to make their laptop work will not work anymore. So when they have spent
tens of hours of research on the net to find *a way* to install Linux on
their laptop with network support, they get used to it. When you tell them
that the upgrade will fix their oopses but will not support their network
card anymore, of course they don't want to upgrade ! And don't tell me that
they just have to by another card, I know users of laptops with only one
pcmcia slot, and this one does not work at all. But even with the oopses,
they are happy not to be forced to work in windows. You know, when they
proudly show me that Linux finally "works" on their laptop, and that they
had to install a ton of binary drivers and ndiswrapper, I whine a lot
telling them that it's not "Linux" they have installed, but sort of a
thing that will ressemble it and which will probably be even less stable
than windows. But they don't care, because they did not have choice.
Willy
On Mon, September 5, 2005 12:36 am, Willy Tarreau said:
> Well, to be fair, most laptop users today are in companies which provide
> them with the model the staff has chosen for all the employees. And
> employees
> try to install Linux on them anyway. That's how you end up with things
> like
> ndiswrapper, because the people who make those notebooks for companies
> don't
> care at all about their customers ; what they want is negociate with the
> staff to sell them 2000 laptops, and that's all.
Companies that provide laptops to their employees tend to frown on the
users installing a bunch of stuff anyway. If the company was buying the
laptop to run Linux it would be spec'd appropriately.
But the real crux of the argument here is not whether or not people should
ever use binary-only drivers, it's whether the open source kernel
developers should spend any time worrying about it or not.
Cheers,
Sean
On Mon, Sep 05, 2005 at 12:47:03AM -0400, Sean wrote:
> On Mon, September 5, 2005 12:36 am, Willy Tarreau said:
>
> > Well, to be fair, most laptop users today are in companies which provide
> > them with the model the staff has chosen for all the employees. And
> > employees
> > try to install Linux on them anyway. That's how you end up with things
> > like
> > ndiswrapper, because the people who make those notebooks for companies
> > don't
> > care at all about their customers ; what they want is negociate with the
> > staff to sell them 2000 laptops, and that's all.
>
> Companies that provide laptops to their employees tend to frown on the
> users installing a bunch of stuff anyway.
But how do you think Linux has penetrated the enterprise market ???
We all have put dual boots on every windows machine we had access to,
eventhough this was clearly forbidden. And after repeatedly showing
to the staff that you saved their day with your Linux, they finally
start to get interested to it. One of my best customers has finally
bought about one hundred RHEL3 licences to replace amateur installs
on production machines. They would never have looked at it without
us braving unauthorized dual boots.
> If the company was buying the
> laptop to run Linux it would be spec'd appropriately.
>
> But the real crux of the argument here is not whether or not people should
> ever use binary-only drivers, it's whether the open source kernel
> developers should spend any time worrying about it or not.
I think we should not worry about it, but we should not deliberately break
it in a stable series when that does not bring anything. The fact that
Adrian proposed to completely remove the option is sad. It's in the windows
world that you can't choose. In Linux, you make menuconfig and choose what
suits your needs.
Regards,
Willy
On Mon, September 5, 2005 1:01 am, Willy Tarreau said:
> But how do you think Linux has penetrated the enterprise market ???
> We all have put dual boots on every windows machine we had access to,
> eventhough this was clearly forbidden. And after repeatedly showing
> to the staff that you saved their day with your Linux, they finally
> start to get interested to it. One of my best customers has finally
> bought about one hundred RHEL3 licences to replace amateur installs
> on production machines. They would never have looked at it without
> us braving unauthorized dual boots.
And how impressed would they have been if you had to install a bunch of
binary drivers to get the job done? Do you think all the problems that
are associated with binary drivers would not have bitten you or your
company?
What you're talking about did happen, but it didn't happen on laptops, and
it didn't happen with binary drivers. It happened naturally when Linux
grew mature enough to support servers without the need for binary-only
hacks.
Trying to accelerate the acceptance of Linux on the desktop with binary
hacks is simply counterproductive.
> I think we should not worry about it, but we should not deliberately break
> it in a stable series when that does not bring anything. The fact that
> Adrian proposed to completely remove the option is sad. It's in the
> windows
> world that you can't choose. In Linux, you make menuconfig and choose what
> suits your needs.
That's the beauty of open source; nobody is being deprived of a choice.
Anyone can simply patch their kernel with 8K support if that's what they
need. But as has been aptly pointed out by others in this thread, 8K
stacks aren't really needed at all, even for NDISwrapper.
4K stacks were NOT invented to hassle binary-only driver owners, they were
created to solve problems associated with 8K stacks.. Leaving the 8K
stack option in the kernel signals that it is a _good_ option. That just
isn't true, so the option should be removed or at least marked BROKEN.
Cheers,
Sean
On Mon, Sep 05, 2005 at 01:31:29AM -0400, Sean wrote:
> On Mon, September 5, 2005 1:01 am, Willy Tarreau said:
>
> > But how do you think Linux has penetrated the enterprise market ???
> > We all have put dual boots on every windows machine we had access to,
> > eventhough this was clearly forbidden. And after repeatedly showing
> > to the staff that you saved their day with your Linux, they finally
> > start to get interested to it. One of my best customers has finally
> > bought about one hundred RHEL3 licences to replace amateur installs
> > on production machines. They would never have looked at it without
> > us braving unauthorized dual boots.
>
> And how impressed would they have been if you had to install a bunch of
> binary drivers to get the job done? Do you think all the problems that
> are associated with binary drivers would not have bitten you or your
> company?
They don't mind. Those people install Checkpoint on Linux. A product of
which previous releases were even incompatible with security updates !
> What you're talking about did happen, but it didn't happen on laptops, and
> it didn't happen with binary drivers. It happened naturally when Linux
> grew mature enough to support servers without the need for binary-only
> hacks.
You don't get it. One of my collegue installed quagga on his binary-only
laptop. Now that he found it very useful and showed how to can be used,
he's planning on setting up servers to run it to bring new services. So
his *crappy* laptop full of binary-only drivers and relying on ndiswrapper
allows him to build a proof of concept and get the possibility to run it
on real hardware.
> Trying to accelerate the acceptance of Linux on the desktop with binary
> hacks is simply counterproductive.
>
> > I think we should not worry about it, but we should not deliberately break
> > it in a stable series when that does not bring anything. The fact that
> > Adrian proposed to completely remove the option is sad. It's in the
> > windows
> > world that you can't choose. In Linux, you make menuconfig and choose what
> > suits your needs.
>
> That's the beauty of open source; nobody is being deprived of a choice.
> Anyone can simply patch their kernel with 8K support if that's what they
> need. But as has been aptly pointed out by others in this thread, 8K
> stacks aren't really needed at all, even for NDISwrapper.
Except that someone has to maintain the patch, because with the speed the
kernel is changing, a patch against 2.6.14 will not work on 2.6.15. I've
tried UML on 2.4 in the past, and I've learned the hard way not to rely
on patches for old features when developpers consider them obsolete (2.4
support ended when 2.6 began).
> 4K stacks were NOT invented to hassle binary-only driver owners, they were
> created to solve problems associated with 8K stacks.. Leaving the 8K
> stack option in the kernel signals that it is a _good_ option. That just
> isn't true, so the option should be removed or at least marked BROKEN.
Keeping it marked as BROKEN is fine. It may even taint the kernel, that's
not a problem. Removing it is wrong.
Regards,
Willy
On Mon, September 5, 2005 2:04 am, Willy Tarreau said:
> They don't mind. Those people install Checkpoint on Linux. A product of
> which previous releases were even incompatible with security updates !
Heh.. well all the power to them. Still no reason for an open source
developer to waste one second worrying about it though.
> You don't get it. One of my collegue installed quagga on his binary-only
> laptop. Now that he found it very useful and showed how to can be used,
> he's planning on setting up servers to run it to bring new services. So
> his *crappy* laptop full of binary-only drivers and relying on ndiswrapper
> allows him to build a proof of concept and get the possibility to run it
> on real hardware.
Same comment. Someone should create a distribution dedicated to this
market if it's that important.
> Except that someone has to maintain the patch, because with the speed the
> kernel is changing, a patch against 2.6.14 will not work on 2.6.15. I've
> tried UML on 2.4 in the past, and I've learned the hard way not to rely
> on patches for old features when developpers consider them obsolete (2.4
> support ended when 2.6 began).
If enough people are interested in it, it can't be that hard to organize
external maintenance of the patch. Even these long conversations on the
mailing list show why the bloody thing should be ejected already. Move it
to it's own tree with it's own mailing list etc..
> Keeping it marked as BROKEN is fine. It may even taint the kernel, that's
> not a problem. Removing it is wrong.
I agree with you that it should taint at a minimum. But should something
that taints the kernel even be part of the mainline source?
Cheers,
Sean
On 9/5/05, Willy Tarreau <[email protected]> wrote:
> Except that someone has to maintain the patch, because with the speed the
> kernel is changing, a patch against 2.6.14 will not work on 2.6.15.
Indeed. It has to be maintained in tree as well and I don't see any
justification for making mainline care about !4KB_STACKS just for the
sake of ndiswrapper. While I appreciate that ndiswrapper is useful for
some people, it should IMHO be maintained as an out-of-tree patch
(stack size included).
Pekka
Willy Tarreau wrote (ao):
> So you're both half-right from your point of view. But you're both half-wrong
> too : no, people can't always choose, and no, people who don't choose their
> laptop are not impacted by development kernels. So let's turn the page and
> wait for a stable kernel.
If the company provides a laptop that doesn't mean that the user can't
choose anymore. He can buy his own laptop. Companies don't allow you to
fiddle with the installed OS anyway.
Kind regards, Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
On Mon, Sep 05, 2005 at 08:41:36AM +0200, Sander wrote:
> Willy Tarreau wrote (ao):
> > So you're both half-right from your point of view. But you're both half-wrong
> > too : no, people can't always choose, and no, people who don't choose their
> > laptop are not impacted by development kernels. So let's turn the page and
> > wait for a stable kernel.
>
> If the company provides a laptop that doesn't mean that the user can't
> choose anymore. He can buy his own laptop. Companies don't allow you to
> fiddle with the installed OS anyway.
On what planet do you live ? When you're a "consultant" (I hate this word),
you take the laptop you're assigned, and use it at customers'. And when
you're fed up with windows (ie: after about 2 hours of work with its shitty
hyperterminal and telnet client), you take any Linux CD and scratch the
install to be able to work for what you're paid, whatever the guy who
installed windows thinks about it. And when he complains to the staff,
you tell them to call the customer to be aware of the time they made you
lose with their inappropriate tools. If my laptop cannot run a decent
telnet client, tftp server, tcpdump, hping, ssh client/server, terminal
emulator, a very fast tcp/ip stack which by the way supports adding and
removing thousands of IP addresses without rebooting, then let it die.
And as I said, I am one of the lucky enough to be able to choose. That's
not the case for most of my coworkers.
Regards,
Willy
Willy Tarreau <[email protected]> wrote:
> On Mon, Sep 05, 2005 at 12:47:03AM -0400, Sean wrote:
[...]
> > But the real crux of the argument here is not whether or not people should
> > ever use binary-only drivers, it's whether the open source kernel
> > developers should spend any time worrying about it or not.
> I think we should not worry about it, but we should not deliberately break
> it in a stable series when that does not bring anything.
4Kstacks sure /does/ bring something. Quite a lot, actually. Please stop
pretending that the kernel crowd is out to get innocent users just for fun
and games. There /are/ technical reasons behind the change, the change
/has/ been tested for a long time and remaining bugs have (all but) been
flushed out, and now the time for the final step has come (get rid of the
old ways once and for all).
> The fact that
> Adrian proposed to completely remove the option is sad.
I for one don't see why.
> It's in the windows
> world that you can't choose. In Linux, you make menuconfig and choose what
> suits your needs.
When it goes away, you can fork your own version of the kernel with the
legacy option, or even figure out how to fix the offending user that needs
it (funny, that was exactly what was supposed to happen in the meantime).
You could even try to brib^Wconvince a kernel developer to do the above for
you, or hire a competent hacker. Or even pool your money with others that
see the same need and put out a call for getting it done.
Perhaps even better, put pressure on the vendors that don't want to give
out specs. Find out why, try to find out if something can be worked out
(specs under NDA, but GPLed driver, perhaps).
Yes, Linux /is/ full of options.
Just go and try to make some piece of ancient hardware work on some of the
propietary systems. /There/ you get no chance but "Oh, just change your
machine".
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
On Sul, 2005-09-04 at 07:51 -0700, Alex Davis wrote:
> I'm not asking you to debug crashes. I'm simply requesting that the
> kernel stack size situation remain as it is: with 8K as the default
> and 4K configurable.
How about the relevant question - why hasn't someone fixed ndiswrapper
yet - its been pending for months.
Alan Cox <[email protected]> writes:
> On Sul, 2005-09-04 at 07:51 -0700, Alex Davis wrote:
> > I'm not asking you to debug crashes. I'm simply requesting that the
> > kernel stack size situation remain as it is: with 8K as the default
> > and 4K configurable.
>
> How about the relevant question - why hasn't someone fixed ndiswrapper
> yet - its been pending for months.
AFAIK with interrupt stacks it shouldn't be a big issue to switch
to a private bigger stack. ndiswrapper just needs to have its own private
way to do "current" which accesses thread_info at the bottom of the stack.
The old problem used to be that interrupts use current too, but
the i386 irq stack code solves this already.
I suppose the real reason is that most people who are capable
to do these are too scared of the general concept of ndiswrapper
and don't touch it with a fire pole, and the others are unable to do it.
BTW Windows actually seems to have 12k of stack so even 8k is not
quite safe (if that word can be applied to ndiswrapper at all)
In general I think 4k stack is a bad move though - the VM stalls
argument is a red hering because if that really was such a big issue
(which my perception is that it is not) then it has to be fixed for
other architectures which need double page stack (like x86-64)
anyways. And the problem of running with tight stack is that you'll
always run into corner cases (e.g. involving more complicated stacking
cases or error handling paths etc.) where you overflow and which will
be hard to fix.
Running with tight stack is just a fundamentally fragile configuration
and will come back to bite you later. Even with 8k we regularly
had overflows reported and with 4k it will be much worse.
-Andi
On Tuesday 06 September 2005 07:37, Andi Kleen wrote:
> Running with tight stack is just a fundamentally fragile configuration
> and will come back to bite you later. Even with 8k we regularly
> had overflows reported and with 4k it will be much worse.
I think one of the reasons is:
"No matter how big stack is, there are always careless
code which can overflow it. 4k, 8k, 64k (hypotetically),
we still must keep stack size in mind when coding.
So, since we already are writing stack size aware code,
why not pick minimum realistically working stack size? Looks
like we can make 4k stack work, and it's naturally smallest
sensible (and in fact easiest to allocate/manage) stack for i386.
So be it, let's use 4k."
I suspect Windows went the opposite way. I bet they already went
thru several iterations of "ouch these drivers from BogoSoft
can overflow stack on nt N, let's bump up stack size for
our new shiny nt N+1".
--
vda
On Tuesday 06 September 2005 08:39, Denis Vlasenko wrote:
> I think one of the reasons is:
>
> "No matter how big stack is, there are always careless
> code which can overflow it. 4k, 8k, 64k (hypotetically),
> we still must keep stack size in mind when coding.
>
> So, since we already are writing stack size aware code,
> why not pick minimum realistically working stack size? Looks
> like we can make 4k stack work, and it's naturally smallest
> sensible (and in fact easiest to allocate/manage) stack for i386.
> So be it, let's use 4k."
Better engineering is to take the minimum size and then
add some safety margin. Running with the minimum only
is ok when you have the infrastructure to prove maximum
stack usage, but Linux is far too complex for that. In general
running with no safety margin is not a good idea.
BTW your minimum doesn't seem to be everybody's
minimum - everytime you see someone advocating
the smallest stacks they go "it's fine for us, but don't
use A, B, C, D" (and I expect these lists to get longer
with more experience).
The trend in Linux clearly seems to be towards more
complex code. While that is not always a good thing but
it seems to be unstoppable (everybody wants their
favourite features and the merging machinery is well oiled
and in full action). And doing all that will need more
stack space over time so shrinking is clearly the wrong
strategic direction.
At some point we undoubtedly will need to increase it further,
the logical point would be when Linux switches to larger softpage
sizes.
-Andi
On Tue, 2005-09-06 at 09:13 +0200, Andi Kleen wrote:
> At some point we undoubtedly will need to increase it further,
> the logical point would be when Linux switches to larger softpage
> sizes.
Is this really a "when"?
Hugh and wli were both working on this and IIRC neither could
show enough justification to get people interested in it and
get it merged (maybe apart from helping stupidly sized PAE systems
limp along)
And that was even before page size reductions and objrmap came
along, which makes the potential gain even smaller.
Are there still good reasons to have such a thing?
Nick
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
> Are there still good reasons to have such a thing?
I think so yes. It just doesn't make much sense to handle larger 64bit
setups with multiple GB of memory with 4K pages. While larger softpage
size will not increase TLB usage it will give you less cache use and in
general less cycles while accessing mem_map and while doing VM operations in
general. Other benefits are faster TLB flush (I saw some bottlenecks
with that in 2.4 on large systems)
Simple example: you have to handle a few GB of memory in the LRU
list. Doing it with half as many objects (8k page instead of 4k) is much more
efficient.
-Andi
In-Reply-To: <[email protected]>
On Sun, 04 Sep 2005 at 15:49:11 +0100, Alan Cox wrote:
> The question is whether ndiswrapper can do stack switching itself. Since
> as I understand it the NT stack is way more than 8K.
W2K usable kernel stack is about two pages. I'm not clear whether
this is really:
a) three pages with huge amount of fixed info, or
b) two pages with tiny info like Linux
__
Chuck
El Tue, 06 Sep 2005 17:32:57 +1000,
Nick Piggin <[email protected]> escribi?:
> Are there still good reasons to have such a thing?
Bigger block sizes was one of their features.
Andi Kleen wrote:
> AFAIK with interrupt stacks it shouldn't be a big issue to switch
> to a private bigger stack. ndiswrapper just needs to have its own private
> way to do "current" which accesses thread_info at the bottom of the stack.
I am developer of ndiswrapper and just caught up with this discussion. I am
interested in providing private stack for ndiswrapper. I am not familiar
with linux kernel internals to understand your proposal. Could you give me
details please: If you can give a rough sketch of idea, I can implement it.
Better yet, if you (or anyone else) can provide an implementation (not
necessarily against ndsiwrapper, but a proof of concept), it will be
greatly appreciated - this should also help any other projects that need
more than 4k stack.
Thanks,
Giri.
2005/9/6, Giridhar Pemmasani <[email protected]>:
> Andi Kleen wrote:
>
> > AFAIK with interrupt stacks it shouldn't be a big issue to switch
> > to a private bigger stack. ndiswrapper just needs to have its own private
> > way to do "current" which accesses thread_info at the bottom of the stack.
>
> I am developer of ndiswrapper and just caught up with this discussion. I am
> interested in providing private stack for ndiswrapper. I am not familiar
> with linux kernel internals to understand your proposal. Could you give me
> details please: If you can give a rough sketch of idea, I can implement it.
> Better yet, if you (or anyone else) can provide an implementation (not
> necessarily against ndsiwrapper, but a proof of concept), it will be
> greatly appreciated - this should also help any other projects that need
> more than 4k stack.
>
You may take a look at how the IRQ stacks work on x86, e.g.
http://lxr.linux.no/source/arch/i386/kernel/irq.c#L48
The idea of switching stacks first sounds appealing, but it is not
that trivial. The tricky part comes with current/current_thread_info.
After you switched to a private stack and called some Windows driver
function, that code may call back to the ndiswrapper API which, in
turn, may jump into some kernel functions. If that kernel code calls
current_thread_info while your differently sized stack is active,
current_thread_info will not be able to evaluate a correct thread_info
address (see http://lxr.linux.no/source/include/asm-i386/thread_info.h#L88).
The only way I see is to switch stacks back on ndiswrapper API entry.
But managing all those stacks correctly is challenging, as you will
likely not want to create a new stack on each switching point. Rather,
maintaining them per context (IRQ, tasklet, kernel thread, user
process, <stuff I forgot>) is required in order to save memory and
avoid shortages in atomic contexts.
Jan
On Mon, Sep 05, 2005 at 12:26:41AM -0400, Dave Jones wrote:
> As someone who gets to read a lot of bug reports from end-users,
> this thing is far from perfect judging by the number of tainted
> oopses I've seen, and not all of them look like stack size issues.
It would make sense to use 4KB pages with a guard page to catch unexpected
stack overflows. Then we'd at least catch some of the binary only modules
with Oops that clearly show who is at fault.
> Helping the cause of binary (or part binary) solutions doesn't solve anything.
> It brings nothing but unsolvable problems, and upset users when their problems
> can't get fixed.
Definately.
-ben
--
"Time is what keeps everything from happening all at once." -- John Wheeler
Jan Kiszka wrote:
> The only way I see is to switch stacks back on ndiswrapper API entry.
> But managing all those stacks correctly is challenging, as you will
> likely not want to create a new stack on each switching point. Rather,
This is what I had in mind before I saw this thread here. I, in fact, did
some work along those lines, but it is even more complicated than you
mentioned here: Windows uses different calling conventions (STDCALL,
FASTCALL, CDECL) so switching stacks by copying arguments/results gets
complicated. So I gave up on that approach. For X86-64 drivers we use
similar approach, but for that there is only one calling convention and we
don't need to switch stacks, but reshuffle arguments on stack / in
registers.
I am still hoping that Andi's approach is possible (I don't understand how
we can make kernel see current info from private stack).
Giri
On Tue, 6 Sep 2005, Giridhar Pemmasani wrote:
> Jan Kiszka wrote:
>
>> The only way I see is to switch stacks back on ndiswrapper API entry.
>> But managing all those stacks correctly is challenging, as you will
>> likely not want to create a new stack on each switching point. Rather,
>
> This is what I had in mind before I saw this thread here. I, in fact, did
> some work along those lines, but it is even more complicated than you
> mentioned here: Windows uses different calling conventions (STDCALL,
> FASTCALL, CDECL) so switching stacks by copying arguments/results gets
> complicated. So I gave up on that approach. For X86-64 drivers we use
> similar approach, but for that there is only one calling convention and we
> don't need to switch stacks, but reshuffle arguments on stack / in
> registers.
>
> I am still hoping that Andi's approach is possible (I don't understand how
> we can make kernel see current info from private stack).
>
> Giri
You can't without copying info from one stack to another. There are
other problems, also, the only place you can get data for a stack in the
kernel dynamically is from kmalloc(GFP_ATOMIC). Other kmalloc() data
are paged which may (will) cause a double-fault if you use it for
a stack. You are not going to get much more than a page of GFP_ATOMIC
data so you can't really make a larger stack than the existing
process/kernel stack.
I have tried to just allocate data when a module is installed (in the
.bss or .data segments as static data). Unfortunately, some kernel
code traps this as a "triple fault" if I try to use it for a stack,
even though the kernel segments for ES, SS, DS, all point to the
same area(s).
I think the purpose of compressing the stack was to get rid of
NDIS, but that's only a theory. Currently, they did a good job
of it!
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction.
.
I apologize for the following. I tried to kill it with the above dot :
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On Tue, Sep 06, 2005 at 02:42:02PM -0400, linux-os (Dick Johnson) wrote:
> I think
... and we have clear winner for the most incredible statement made on l-k.
2005/9/6, Giridhar Pemmasani <[email protected]>:
> Jan Kiszka wrote:
>
> > The only way I see is to switch stacks back on ndiswrapper API entry.
> > But managing all those stacks correctly is challenging, as you will
> > likely not want to create a new stack on each switching point. Rather,
>
> This is what I had in mind before I saw this thread here. I, in fact, did
> some work along those lines, but it is even more complicated than you
> mentioned here: Windows uses different calling conventions (STDCALL,
> FASTCALL, CDECL) so switching stacks by copying arguments/results gets
> complicated. So I gave up on that approach. For X86-64 drivers we use
> similar approach, but for that there is only one calling convention and we
> don't need to switch stacks, but reshuffle arguments on stack / in
> registers.
>
> I am still hoping that Andi's approach is possible (I don't understand how
> we can make kernel see current info from private stack).
>
The more I think about this the more it becomes clear that this path
will be too winding, especially when compared to the effort needed to
patch 8K (or more) back into the kernel as an intermediate workaround.
Moving the Windows code out of the kernel to userspace should be more
helpful on the long term. It would take a small stub, something like
tun/tap devices with wireless extensions, plus something to forward
PCI interrupts, and you could start hacking the wrapper in a save
harbour. If libusb is already prepared for such a task is not yet
clear to me (at least it is lacking USB 2.0 according to the docs).
But time will likely solve this as well.
Jan
On Tue, 6 Sep 2005, Jan Kiszka wrote:
> 2005/9/6, Giridhar Pemmasani <[email protected]>:
>> Jan Kiszka wrote:
>>
>>> The only way I see is to switch stacks back on ndiswrapper API entry.
>>> But managing all those stacks correctly is challenging, as you will
>>> likely not want to create a new stack on each switching point. Rather,
>>
>> This is what I had in mind before I saw this thread here. I, in fact, did
>> some work along those lines, but it is even more complicated than you
>> mentioned here: Windows uses different calling conventions (STDCALL,
>> FASTCALL, CDECL) so switching stacks by copying arguments/results gets
>> complicated. So I gave up on that approach. For X86-64 drivers we use
>> similar approach, but for that there is only one calling convention and we
>> don't need to switch stacks, but reshuffle arguments on stack / in
>> registers.
>>
>> I am still hoping that Andi's approach is possible (I don't understand how
>> we can make kernel see current info from private stack).
>>
>
> The more I think about this the more it becomes clear that this path
> will be too winding, especially when compared to the effort needed to
> patch 8K (or more) back into the kernel as an intermediate workaround.
>
> Moving the Windows code out of the kernel to userspace should be more
> helpful on the long term. It would take a small stub, something like
> tun/tap devices with wireless extensions, plus something to forward
> PCI interrupts, and you could start hacking the wrapper in a save
> harbour. If libusb is already prepared for such a task is not yet
> clear to me (at least it is lacking USB 2.0 according to the docs).
> But time will likely solve this as well.
>
> Jan
> -
The attached device-driver allocates a 0x4000 length
stack in an ioctl() procedure, uses that stack, then
returns to the old stack before returning from the
ioctl(). Since the data-space for the stack is allocated
only once (when the module is installed), the runtime
overhead is not very great. The driver runs on 2.6.13.
This just shows that it can be done! This doesn't mean
that one should actually do this!
Script started on Tue 06 Sep 2005 04:09:32 PM EDT
[root@chaos driver]# make install
Analogic StkDev : Initialization complete
[root@chaos driver]# ./tester
[root@chaos driver]# ./tester
[root@chaos driver]# ./tester
[root@chaos driver]# make remove
StkDev 6020 0 - Live 0xf0a34000
Analogic StkDev : Initialization complete
Doing something scary on the new stack!!
Actually survived!
Doing something scary on the new stack!!
Actually survived!
Doing something scary on the new stack!!
Actually survived!
Analogic StkDev : Module removed
[root@chaos driver]# exit
Script done on Tue 06 Sep 2005 04:09:55 PM EDT
Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction.
.
I apologize for the following. I tried to kill it with the above dot :
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On Tuesday 06 September 2005 13:23, Giridhar Pemmasani wrote:
> Jan Kiszka wrote:
> > The only way I see is to switch stacks back on ndiswrapper API entry.
> > But managing all those stacks correctly is challenging,
There are only two stacks involved, the normal kernel stack and your new ndis
stack. You save ESP of the kernel stack at the base of the ndis stack. When
the Windows code calls your api, you get the ndis ESP, load the kernel ESP
from the base of the ndis stack, push the ndis ESP so you can get back to the
ndis code later, and continue on your merry way.
> > as you will likely not want to create a new stack on each switching
> > point...
You will allocate your own stack once on driver initialization.
> This is what I had in mind before I saw this thread here. I, in fact, did
> some work along those lines, but it is even more complicated than you
> mentioned here: Windows uses different calling conventions (STDCALL,
> FASTCALL, CDECL) so switching stacks by copying arguments/results gets
> complicated.
I missed something there. You would switch stacks before calling the Windows
code and after the Windows code calls you (and respective returns) so you are
always in your own code when you switch, hence you know how to copy the
parameters.
> I am still hoping that Andi's approach is possible (I don't understand how
> we can make kernel see current info from private stack).
He suggested you use your own private variant of current which would
presumeably read a copy of current you stored at the bottom of your own
stack. But I don't see why your code would ever need current while you are
on the private ndis stack.
Andi, their stack will have to have a valid thread_info->task because
interrupts will use it. Out of interest, could you please explain what for?
Code like u32 stack[THREAD_SIZE/sizeof(u32)] is violated by a different sized
stack, but apparently not in any way that matters.
By the way, I use ndis_wrapper, thanks a lot you guys!
Regards,
Daniel
On Wednesday 07 September 2005 00:19, Daniel Phillips wrote:
> Andi, their stack will have to have a valid thread_info->task because
> interrupts will use it. Out of interest, could you please explain what
> for?
No, with 4k interrupts run on their own stack with their own thread_info
Or rather they mostly do. Currently do_IRQ does irq_enter which refers
thread_info before switching to the interrupt stack, that order would likely
need to be
exchanged.
-Andi
Daniel> There are only two stacks involved, the normal kernel
Daniel> stack and your new ndis stack. You save ESP of the kernel
Daniel> stack at the base of the ndis stack. When the Windows
Daniel> code calls your api, you get the ndis ESP, load the kernel
Daniel> ESP from the base of the ndis stack, push the ndis ESP so
Daniel> you can get back to the ndis code later, and continue on
Daniel> your merry way.
[...]
Daniel> You will allocate your own stack once on driver
Daniel> initialization.
I'm not quite sure it's this trivial. Obviously there are more than
two stacks involved, since there is more than one kernel stack! (One
per task plus IRQ stacks) This is more than just a theoretical
problem. It seems entirely possible that more than one task could
be in the driver, and clearly they each need their own stack.
So it's going to be at least a little harder than allocating a single
stack for NDIS use when the driver starts up.
I personally like the idea raised elsewhere in this thread of running
the Windows driver in userspace by proxying interrupts, PCI access,
etc. That seems more robust and probably allows some cool reverse
engineering hacks.
- R.
On Tuesday 06 September 2005 18:21, Andi Kleen wrote:
> On Wednesday 07 September 2005 00:19, Daniel Phillips wrote:
> > Andi, their stack will have to have a valid thread_info->task because
> > interrupts will use it. Out of interest, could you please explain what
> > for?
>
> No, with 4k interrupts run on their own stack with their own thread_info
> Or rather they mostly do. Currently do_IRQ does irq_enter which refers
> thread_info before switching to the interrupt stack, that order would
> likely need to be exchanged.
But then how would thread_info->task on the irq stack ever get initialized?
My "what for" question was re why interrupt routines even need a valid
current. I see one answer out there on the web: statistical profiling. Is
that it?
Regards,
Daniel
On Tue, 6 Sep 2005 18:19:45 -0400, Daniel Phillips <[email protected]> said:
Daniel> There are only two stacks involved, the normal kernel stack
Daniel> and your new ndis stack. You save ESP of the kernel stack
Sadly, that is not the case: Some drivers, especially USB drivers,
create multiple threads (those threads are created through
ndiswrapper, so if any massaging needs to be done, ndiswrapper can do
that). However, it is not multiple threads that is the problem, but
switching of stacks. On transition from Windows function to Linux
function, we need to copy the arguments that are in Windows stack into
Linux stack so Linux function gets the arguments. We need to do same
thing in the reverse direction. As mentioned before, this gets
complicated with different calling conventions and different number of
arguments to functions. Moreover, if Linux kernel decides to suspend a
thread while it is executing Windows function, kernel sees an invalid
stack.
Daniel> You will allocate your own stack once on driver
Daniel> initialization.
Rather, whenever a new thread is created.
Daniel> I missed something there. You would switch stacks before
Daniel> calling the Windows code and after the Windows code calls
Daniel> you (and respective returns) so you are always in your own
Daniel> code when you switch, hence you know how to copy the
Daniel> parameters.
This is not impossible, but messy (even if the very idea ndiswrapper
itself sounds scary to some).
Daniel> By the way, I use ndis_wrapper, thanks a lot you guys!
Good to know you are happy.
--
Giri
On Tue, 2005-09-06 at 18:36 -0400, Daniel Phillips wrote:
> But then how would thread_info->task on the irq stack ever get initialized?
>
> My "what for" question was re why interrupt routines even need a valid
> current. I see one answer out there on the web: statistical profiling. Is
> that it?
scheduler_tick - This is called from the timer interrupt, and it
definitely needs a valid current.
-- Steve
On Tuesday 06 September 2005 18:28, Roland Dreier wrote:
> Daniel> There are only two stacks involved, the normal kernel
> Daniel> stack and your new ndis stack. You save ESP of the kernel
> Daniel> stack at the base of the ndis stack. When the Windows
> Daniel> code calls your api, you get the ndis ESP, load the kernel
> Daniel> ESP from the base of the ndis stack, push the ndis ESP so
> Daniel> you can get back to the ndis code later, and continue on
> Daniel> your merry way.
>
> [...]
>
> Daniel> You will allocate your own stack once on driver
> Daniel> initialization.
>
> I'm not quite sure it's this trivial. Obviously there are more than
> two stacks involved, since there is more than one kernel stack! (One
> per task plus IRQ stacks) This is more than just a theoretical
> problem. It seems entirely possible that more than one task could
> be in the driver, and clearly they each need their own stack.
Semaphore :-)
Do you expect this to be heavily contended? On a very quick run through the
code, it seems you don't hold any spinlocks going into the driver from
process context. Interrupts... they better fit into a 4K stack or it's game
over. Preemption while on the ndis stack... you can always disable
preemption in this region, but the semaphore should protect you. Task killed
while preempted... I dunno.
> So it's going to be at least a little harder than allocating a single
> stack for NDIS use when the driver starts up.
>
> I personally like the idea raised elsewhere in this thread of running
> the Windows driver in userspace by proxying interrupts, PCI access,
> etc. That seems more robust and probably allows some cool reverse
> engineering hacks.
I expect the userspace approach will be a lot more work and a lot more
overhead too, but then again it sounds like loads of fun.
Regards,
Daniel
Daniel Phillips wrote:
> There are only two stacks involved, the normal kernel stack and your new ndis
> stack. You save ESP of the kernel stack at the base of the ndis stack. When
> the Windows code calls your api, you get the ndis ESP, load the kernel ESP
> from the base of the ndis stack, push the ndis ESP so you can get back to the
> ndis code later, and continue on your merry way.
With CONFIG_PREEMPT, this will still cause trouble due to lack
of "current" task info on the NDIS stack.
One option is to copy (duplicate) the bottom-of-stack info when
switching to the NDIS stack.
Another option is to stick a Mutex around any use of the NDIS stack
when calling into the foreign driver (might be done like this already??),
which will prevent PREEMPTion during the call.
Cheers
On Tue, 2005-09-06 at 21:59 -0400, Mark Lord wrote:
> Daniel Phillips wrote:
> > There are only two stacks involved, the normal kernel stack and your new ndis
> > stack. You save ESP of the kernel stack at the base of the ndis stack. When
> > the Windows code calls your api, you get the ndis ESP, load the kernel ESP
> > from the base of the ndis stack, push the ndis ESP so you can get back to the
> > ndis code later, and continue on your merry way.
>
> With CONFIG_PREEMPT, this will still cause trouble due to lack
> of "current" task info on the NDIS stack.
>
> One option is to copy (duplicate) the bottom-of-stack info when
> switching to the NDIS stack.
>
> Another option is to stick a Mutex around any use of the NDIS stack
> when calling into the foreign driver (might be done like this already??),
> which will prevent PREEMPTion during the call.
Isn't it bad to rely on taking a lock disabling preemption as a side
effect? For example it will break on PREEMPT_RT (not merged yet, but
still...). If you really just need preemption disabled/reenabled can't
you just do it explicitly?
Lee
On Tuesday 06 September 2005 21:59, Mark Lord wrote:
> Daniel Phillips wrote:
> > There are only two stacks involved, the normal kernel stack and your new
> > ndis stack. You save ESP of the kernel stack at the base of the ndis
> > stack. When the Windows code calls your api, you get the ndis ESP, load
> > the kernel ESP from the base of the ndis stack, push the ndis ESP so you
> > can get back to the ndis code later, and continue on your merry way.
I must have been smoking something when I convinced myself that the driver
can't call into the kernel without switching back to the kernel stack. But
this is wrong, as long as ->task and ->previous_esp are initialized, staying
on the bigger stack looks fine (previous_esp is apparently used only for
backtrace).
> With CONFIG_PREEMPT, this will still cause trouble due to lack
> of "current" task info on the NDIS stack.
>
> One option is to copy (duplicate) the bottom-of-stack info when
> switching to the NDIS stack.
Yes, just like do_IRQ.
> Another option is to stick a Mutex around any use of the NDIS stack
> when calling into the foreign driver (might be done like this already??),
There is no mutex now, but this is the easy way to get by with just one ndis
stack.
> which will prevent PREEMPTion during the call.
We have preempt_enable/disable for that. But I am not sure preemption needs
to be disabled.
Regards,
Daniel
On Wednesday 07 September 2005 00:16, Daniel Phillips wrote:
> ...as long as ->task and ->previous_esp are initialized,
> staying on the bigger stack looks fine (previous_esp is apparently used
> only for backtrace) ... just like do_IRQ.
Ahem, but let me note before somebody else does: it isn't interrupt context,
it is normal process context - while an interrupt can ignore most of the
thread_info fields, a normal process has to worry about all 9. To be on the
safe side, the first 8 need to be copied into and out of the ndis stack, with
preempt disabled until after the stack switch.
Regards,
Daniel
Alex Davis wrote:
>>Please don't tell me to "care for closed-source drivers".
>
> ndiswrapper is NOT closed source. And I'm not asking you to "care".
>
>
>>I don't want the pain of debugging crashes on the machines which run unknown code
>>in kernel space.
>
> I'm not asking you to debug crashes. I'm simply requesting that the
> kernel stack size situation remain as it is: with 8K as the default
> and 4K configurable.
I can be happy with 4K as the default, everything I use *except*
ndiswrapper seems to run fine (I don't currently need fancy filesystems)
but laptops seem to include a lot of unsupported hardware, which can't
be replaced due to resources (money, slots, batter life).
--
-bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me
Jan Kiszka wrote:
> 2005/9/6, Giridhar Pemmasani <[email protected]>:
>
>>Jan Kiszka wrote:
>>
>>
>>>The only way I see is to switch stacks back on ndiswrapper API entry.
>>>But managing all those stacks correctly is challenging, as you will
>>>likely not want to create a new stack on each switching point. Rather,
>>
>>This is what I had in mind before I saw this thread here. I, in fact, did
>>some work along those lines, but it is even more complicated than you
>>mentioned here: Windows uses different calling conventions (STDCALL,
>>FASTCALL, CDECL) so switching stacks by copying arguments/results gets
>>complicated. So I gave up on that approach. For X86-64 drivers we use
>>similar approach, but for that there is only one calling convention and we
>>don't need to switch stacks, but reshuffle arguments on stack / in
>>registers.
>>
>>I am still hoping that Andi's approach is possible (I don't understand how
>>we can make kernel see current info from private stack).
>>
>
>
> The more I think about this the more it becomes clear that this path
> will be too winding, especially when compared to the effort needed to
> patch 8K (or more) back into the kernel as an intermediate workaround.
Is there a technical reason ("hard to implement" is a practical reason)
why all stacks need to be the same size?
--
-bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me
2005/9/7, Bill Davidsen <[email protected]>:
> Jan Kiszka wrote:
> > 2005/9/6, Giridhar Pemmasani <[email protected]>:
> >
> >>Jan Kiszka wrote:
> >>
> >>
> >>>The only way I see is to switch stacks back on ndiswrapper API entry.
> >>>But managing all those stacks correctly is challenging, as you will
> >>>likely not want to create a new stack on each switching point. Rather,
> >>
> >>This is what I had in mind before I saw this thread here. I, in fact, did
> >>some work along those lines, but it is even more complicated than you
> >>mentioned here: Windows uses different calling conventions (STDCALL,
> >>FASTCALL, CDECL) so switching stacks by copying arguments/results gets
> >>complicated. So I gave up on that approach. For X86-64 drivers we use
> >>similar approach, but for that there is only one calling convention and we
> >>don't need to switch stacks, but reshuffle arguments on stack / in
> >>registers.
> >>
> >>I am still hoping that Andi's approach is possible (I don't understand how
> >>we can make kernel see current info from private stack).
> >>
> >
> >
> > The more I think about this the more it becomes clear that this path
> > will be too winding, especially when compared to the effort needed to
> > patch 8K (or more) back into the kernel as an intermediate workaround.
>
> Is there a technical reason ("hard to implement" is a practical reason)
> why all stacks need to be the same size?
>
Because of
static inline struct thread_info *current_thread_info(void)
{
struct thread_info *ti;
__asm__("andl %%esp,%0; ":"=r" (ti) : "" (~(THREAD_SIZE - 1)));
return ti;
}
[include/asm-i386/thread_info.h]
which assumes that it can "round down" the stack pointer and then will
find the thread_info of the current context there. Only works for
identically sized stacks. Note that this function is heavily used in
the kernel, either directly or indirectly. You cannot avoid it.
My current assessment regarding differently sized threads for
ndiswrapper: not feasible with vanilla kernels.
Jan
> > Is there a technical reason ("hard to implement" is a practical reason)
> > why all stacks need to be the same size?
>
> Because of
>
> static inline struct thread_info *current_thread_info(void)
> {
> struct thread_info *ti;
> __asm__("andl %%esp,%0; ":"=r" (ti) : "" (~(THREAD_SIZE - 1)));
> return ti;
> }
> [include/asm-i386/thread_info.h]
>
> which assumes that it can "round down" the stack pointer and then will
> find the thread_info of the current context there. Only works for
> identically sized stacks. Note that this function is heavily used in
> the kernel, either directly or indirectly. You cannot avoid it.
>
> My current assessment regarding differently sized threads for
> ndiswrapper: not feasible with vanilla kernels.
If so, it is not because of this. It just means you have to go back to the
idea of switching back to the original stack when the Windows driver calls
into the ndis API. (It must have been way too late last night when I claimed
the second stack switch wasn't necessary.)
Other issues:
- Use a semaphore to serialize access to a single ndis stack... any
spinlock or interrupt state issues? (I didn't notice any.)
- Copy parameters across the stack switch - a little tricky, but far from
the trickiest bit of glue in the kernel
- Preempt - looks like it has to be disabled from switching to the ndis
stack to switching back because of the thread_info problem
- It is best for Linux when life is a little hard for binary-only drivers,
but not completely impossible. When the smoke clears, ndis wrapper will
be slightly slower than before and we will be slightly closer to having
some native drivers. In the meantime, keeping the thing alive without
impacting core is an interesting puzzle.
Regards,
Daniel
On Wednesday 07 September 2005 15:52, Daniel Phillips wrote:
Ah, there's another issue: an interrupt can come in when esp is on the ndis
stack and above THREAD_SIZE, so do_IRQ will not find thread_info. Sorry,
this one is nasty.
Regards,
Daniel
2005/9/7, Daniel Phillips <[email protected]>:
> > > Is there a technical reason ("hard to implement" is a practical reason)
> > > why all stacks need to be the same size?
> >
> > Because of
> >
> > static inline struct thread_info *current_thread_info(void)
> > {
> > struct thread_info *ti;
> > __asm__("andl %%esp,%0; ":"=r" (ti) : "" (~(THREAD_SIZE - 1)));
> > return ti;
> > }
> > [include/asm-i386/thread_info.h]
> >
> > which assumes that it can "round down" the stack pointer and then will
> > find the thread_info of the current context there. Only works for
> > identically sized stacks. Note that this function is heavily used in
> > the kernel, either directly or indirectly. You cannot avoid it.
> >
> > My current assessment regarding differently sized threads for
> > ndiswrapper: not feasible with vanilla kernels.
>
> If so, it is not because of this. It just means you have to go back to the
> idea of switching back to the original stack when the Windows driver calls
> into the ndis API. (It must have been way too late last night when I claimed
> the second stack switch wasn't necessary.)
That alone doesn't help. You would also have to disable at least bh
while windows functions are being executed, otherwise the scheduler
may crash when trying to switch to some other task. Even worse with 4K
stacks: take a look at how do_IRQ works, the common interrupt entry
point. It calls current_thread_info, thus expects to preempt normal
kernel stacks. So, should we execute Windows driver code with IRQs
off? I don't think so.
>
> Other issues:
>
> - Use a semaphore to serialize access to a single ndis stack... any
> spinlock or interrupt state issues? (I didn't notice any.)
Too simple approach. You cannot deny that two or more functions of
ndiswrapper or the windows driver are being executed concurrently.
>
> - Copy parameters across the stack switch - a little tricky, but far from
> the trickiest bit of glue in the kernel
>
> - Preempt - looks like it has to be disabled from switching to the ndis
> stack to switching back because of the thread_info problem
>
> - It is best for Linux when life is a little hard for binary-only drivers,
> but not completely impossible. When the smoke clears, ndis wrapper will
> be slightly slower than before and we will be slightly closer to having
> some native drivers. In the meantime, keeping the thing alive without
> impacting core is an interesting puzzle.
Ndiswrapper is already slower than native drivers are, also due to
horribly implemented Windows drivers btw (the ndis model itself isn't
that bad, though).
After all the issues coming up, I'm even more convinced that any
effort on stack switching is better spent on moving ndiswrapper to
userspace - also expensive but safer in many ways.
Jan
2005/9/7, Daniel Phillips <[email protected]>:
> On Wednesday 07 September 2005 15:52, Daniel Phillips wrote:
> Ah, there's another issue: an interrupt can come in when esp is on the ndis
> stack and above THREAD_SIZE, so do_IRQ will not find thread_info. Sorry,
> this one is nasty.
>
Oh, you already got it as well.
Jan
Jan Kiszka wrote:
> Ndiswrapper is already slower than native drivers are, also due to
> horribly implemented Windows drivers btw (the ndis model itself isn't
> that bad, though).
Do you have any evidence to back your claims? What tests did you do to say
that ndiswrapper is slower than native driver? Under X86-64 there is some
overhead due to reshuffling of arguments, but it is so little that I doubt
if it can be measured.
I agree that some drivers, especially old drivers, are nasty from
ndiswrapper's point of view (e.g., a driver uses exception handling
mechanism through "fs" register, which won't work in Linux kernel space,
and another driver allocates huge chunk of memory in atomic context etc.),
but they are valid under Windows.
I have seen some FUD about ndiswrapper - either on madwifi or acx100 project
sites sometime back there was a list of reasons why one shouldn't use
ndiswrapper, one of them is the overhead due to binary emulation. And that
was from a WINE developer! If someone has
ideological/moral/ethical/operational etc issues against using ndiswrapper,
that is perfectly valid. But if you accept that NDIS isn't bad, I don't see
why ndiswrapper would suck (the idea I mean). I myself have gripes with
NDIS for wireless devices though - no support for monitor mode, quality
level etc.
Before someone jumps, let me clarify that although I toil with development
of ndiswrapper, I encourage people to use open source drivers when there is
a choice (e.g., prism54, hostap drivers). But, as pointed out by others,
there is lot of wireless networking hardware with no specifications and
drivers. ndiswrapper is just for such and should a vendor provide
documentation, I am sure someone will write open source driver for it. In
my experience, however, vendors don't seem to be interested much with Linux
support. I have requested quite a few of them to send me sample cards (the
ones for which no native drivers exist) so ndiswrpaper can support them.
Except for US Robotics, not a single vendor cared. A D-Link tech support
guy even laughed at the idea!
In short, if you are buying hardware to run Linux, check first if there are
open source drivers for it. However, if you have a wireless card that
doesn't have native drivers, and if you can't/won't change hardware and you
want to use it under Linux, then ndiswrapper may get the job done.
Not all of the contents in this article is in response to the quoted
article, but I just wanted to clear some misconceptions about goals/use of
ndiswrapper that I have noticed.
Thanks
Giri
2005/9/7, Giridhar Pemmasani <[email protected]>:
> Jan Kiszka wrote:
>
> > Ndiswrapper is already slower than native drivers are, also due to
> > horribly implemented Windows drivers btw (the ndis model itself isn't
> > that bad, though).
>
> Do you have any evidence to back your claims? What tests did you do to say
> that ndiswrapper is slower than native driver? Under X86-64 there is some
> overhead due to reshuffling of arguments, but it is so little that I doubt
> if it can be measured.
Giri, I'm not attacking your project. You know I'm sharing your
pragmatic view. Performance is a pure technical issue.
Yes, I can provide some numbers around atheros devices (10-20%
speed-up). And yes, I can explain why ndiswrapper suffers from certain
differences of the NDIS driver model compared to the one of Linux
(just consider what had to be moved to tasklets). But I think this
would better be continued on the ndiswrapper list than here.
Jan
--- Bill Davidsen <[email protected]> wrote:
> Alex Davis wrote:
> >>Please don't tell me to "care for closed-source drivers".
> >
> > ndiswrapper is NOT closed source. And I'm not asking you to "care".
> >
> >
> >>I don't want the pain of debugging crashes on the machines which run unknown code
> >>in kernel space.
> >
> > I'm not asking you to debug crashes. I'm simply requesting that the
> > kernel stack size situation remain as it is: with 8K as the default
> > and 4K configurable.
>
> I can be happy with 4K as the default, everything I use *except*
> ndiswrapper seems to run fine (I don't currently need fancy filesystems)
> but laptops seem to include a lot of unsupported hardware, which can't
> be replaced due to resources (money, slots, batter life).
> --
I could live with any default, as long as it's configurable.
The intent here, however, is to take away the option. That's
what I have an issue with.
Is there any problem caused by letting stack size be
configurable to any (sane) arbitrary maximum value
(e.g. 32K)?
> -bill davidsen ([email protected])
> "The secret to procrastination is to put things off until the
> last possible moment - but no longer" -me
>
I code, therefore I am
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
On Thu, 8 Sep 2005 01:25:20 +0200, Jan Kiszka <[email protected]> said:
Jan> Yes, I can provide some numbers around atheros devices (10-20%
Jan> speed-up). And yes, I can explain why ndiswrapper suffers from
Are you comparing atheros Windows driver with ndiswrapper or madwifi
with ndiswrapper? If it is latter, it is understandable. If it is
former, it is not what I heard from others.
--
Giri
Jan Kiszka wrote:
> 2005/9/7, Bill Davidsen <[email protected]>:
>
>>Is there a technical reason ("hard to implement" is a practical reason)
>>why all stacks need to be the same size?
>>
>
>
> Because of
>
> static inline struct thread_info *current_thread_info(void)
> {
> struct thread_info *ti;
> __asm__("andl %%esp,%0; ":"=r" (ti) : "" (~(THREAD_SIZE - 1)));
> return ti;
> }
> [include/asm-i386/thread_info.h]
>
> which assumes that it can "round down" the stack pointer and then will
> find the thread_info of the current context there. Only works for
> identically sized stacks. Note that this function is heavily used in
> the kernel, either directly or indirectly. You cannot avoid it.
Avoiding it isn't necessary, a config option to read a variable
THREAD_SIZE would solve this part of it. I looked at the implications of
this, and other than a slight overhead it looks as if the problem is
where to put the size :-( I think it could be done given that all
threads of a process would have the same size, but I am *not* suggesting
that it should be done.
>
> My current assessment regarding differently sized threads for
> ndiswrapper: not feasible with vanilla kernels.
Have to agree, it would take way more effort than it's worth, given that
the need can be avoided by allowing user config of the stack size. It
does look relatively easy to allow larger stack sizes, though. Wasn't
there one driver so ill-behaved it wouldn't even run in an 8k stack?
Andi made the point that allocating server size memory in 4k blocks
results in a lot of them, resulting in increased overhead in some
places. Thinking about configuring the stack size, maybe it would be
useful to make memory allocation block size independent of the hardware
and let both be set over a wide range in config.
--
-bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me
Willy Tarreau wrote:
> Hi,
>
> On Mon, Sep 05, 2005 at 12:12:56AM -0400, Sean wrote:
>
>>On Mon, September 5, 2005 12:03 am, Alex Davis said:
>>
>>>What if you don't have a choice? When someone comes to me with their
>>>laptop
>>>containing a built-in wireless card not natively supported by Linux, am I
>>>supposed to tell them "go buy a Linux-supported card" when there's a way
>>>I can make their existing card work? I don't think so.
>>
>>You always have a choice in life. Nobody is stopping you from doing what
>>_you_ choose to do. That doesn't mean that developers who are concerned
>>with the creation and promotion of open source should care one whit about
>>your particular take on the situation. Go do whatever you want just
>>don't expect the open source developers to pay for it; you maintain the
>>crufty patches yourself.
>
>
> Well, to be fair, most laptop users today are in companies which provide
> them with the model the staff has chosen for all the employees. And employees
> try to install Linux on them anyway. That's how you end up with things like
> ndiswrapper, because the people who make those notebooks for companies don't
> care at all about their customers ; what they want is negociate with the
> staff to sell them 2000 laptops, and that's all.
>
> However, those employees generally start with "easy to install" distros,
> such as fedora or similar. Those don't ship with standard kernels anyway,
> and they provide what is necessary to stay compatible with the tools which
> support crappy hardware. So (and it's sad), all those people don't care at
> all about kernel development, nor do they care about 2.6.13, 2.6.14, etc...
>
> It's just *us* who tell them "ah, you can't do that because you use a
> prehistorical kernel, let me install you the last one", and then and only
> then, that fails and we whine about the crappy hardware and the "easy to
> install" distro which seem to be made to work together.
>
> So I tend to consider that ndiswrapper users won't care about what will
> be in 2.6.14, but only about the fact that *their distro* will still
> support ndiswrapper. Of course, it would have been easier if the kernel
> would have stabilized and was only in maintenance mode, not breaking
> anything on every new release, but Linus has decided differently, so be
> it. Those users did not have a choice in the beginning, otherwise they
> would have chosen a laptop supported by Linux. And honnestly, I know
> plenty of them. In fact, it's the other way around. I know very few
> people who can actually choose their laptop.
And in many cases there is a cost difference between the Linux
compatible unit and what's on sale with every feature you want but the
supported wireless. And since a fair number of nice machines come with a
single slot, adding a card is not always an option, there may be
something else there.
If the machine which runs Linux costs anywhere near the cost of a
Windows license, whoever pays the bills has a real incentive to turn to
the dark side.
>
> So you're both half-right from your point of view. But you're both half-wrong
> too : no, people can't always choose, and no, people who don't choose their
> laptop are not impacted by development kernels. So let's turn the page and
> wait for a stable kernel.
It appears that Linus has decided that there are not going to be any
such from kernel.org. That's a bad thing for users, to choose between
obsolete and stable, but it's his kernel.
I wouldn't go shouting "fork" so loud, a major fork would kill Linux as
dead as BSD, a family of neat operating systems, none of which is widely
used the way Linux is, and which are compatible with each other mainly
at the acronym level, all are called BSD.
Question: does colinux get around all this crap by using the Windows
drivers and running in essentially microkernel mode? Windows as the new
NDISwrapper ;-)
--
-bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me
Horst von Brand wrote:
> Just go and try to make some piece of ancient hardware work on some of the
> propietary systems. /There/ you get no chance but "Oh, just change your
> machine".
Is "maintain your own operating system" really better in your mind? Does
that sound like a viable alternative?
It's not old hardware that's the problem, it's new hardware which isn't
supported. And unlike desktops, there's a lot less in the way of
hardware options on a lappie, you take what you get, you get what you
can afford, and often that means running on what someone else chooses.
--
-bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me
Bill Davidsen wrote:
> It appears that Linus has decided that there are not going to be any
> such from kernel.org. That's a bad thing for users, to choose between
> obsolete and stable, but it's his kernel.
Most users are best served to stick with the latest kernel *from their
distro*. It's only the developers and the distro maintainers that need
to worry about tracking the latest kernel.
Of course there are a lot of enthusiasts that do it as well, but there
has been a caveat for quite a while that if you track the bleeding edge,
you really should be on the mailing lists so that you know what's going
on and which issues might bite you.
There is a cost to maintaining features in the kernel, even something
like multiple stack sizes. Whenever you have a config option, you need
to worry about making sure that everything works both ways. This takes
time and resources. There's always going to be a push to simplify things.
Chris
On Thu, September 8, 2005 4:18 pm, Bill Davidsen said:
> Is "maintain your own operating system" really better in your mind? Does
> that sound like a viable alternative?
No, that's a false choice. You folks just need to convince your
distribution to apply the patches you "need" or create your own
distribution. That does not mean this cruft should be in the mainline
kernel.
> It's not old hardware that's the problem, it's new hardware which isn't
> supported. And unlike desktops, there's a lot less in the way of
> hardware options on a lappie, you take what you get, you get what you
> can afford, and often that means running on what someone else chooses.
So?
Cheers,
Sean
On Fri, Sep 02, 2005 at 04:40:54PM +1000 Neil Brown wrote:
> On Friday September 2, [email protected] wrote:
> > On Thu, Sep 01, 2005 at 10:33:56PM -0700, Chris Wedgwood wrote:
> > > On Fri, Sep 02, 2005 at 02:39:15AM +0200, Adrian Bunk wrote:
> > >
> > > > 4Kb kernel stacks are the future on i386, and it seems the problems
> > > > it initially caused are now sorted out.
> > >
> > > Not entirely.
> > >
> > > XFS when mixed with raid/lvm/nfs still blows up. It's probably not
> > > alone in this respect but worse than ext2/3.
> >
> > To clarify, you mean AND not OR (/) there -- in other words,
> > raid[+raid]+dm[+dm]+xfs+nfs can be fatal, yes.
>
> It should be reasonably simple to remove this problem of stacked
> drivers.
> There really isn't any need for md and dm (or md and md or ..) to use
> the stack and the same time.
>
Sorry to bump in so late - but there seems to be a reporter who is
suffering from these issues now:
http://bugzilla.kernel.org/show_bug.cgi?id=5210
On Tue, Sep 13, 2005 at 10:05:52AM +0200, Alexander Nyberg wrote:
> On Fri, Sep 02, 2005 at 04:40:54PM +1000 Neil Brown wrote:
>
> > On Friday September 2, [email protected] wrote:
> > > On Thu, Sep 01, 2005 at 10:33:56PM -0700, Chris Wedgwood wrote:
> > > > On Fri, Sep 02, 2005 at 02:39:15AM +0200, Adrian Bunk wrote:
> > > >
> > > > > 4Kb kernel stacks are the future on i386, and it seems the problems
> > > > > it initially caused are now sorted out.
> > > >
> > > > Not entirely.
> > > >
> > > > XFS when mixed with raid/lvm/nfs still blows up. It's probably not
> > > > alone in this respect but worse than ext2/3.
> > >
> > > To clarify, you mean AND not OR (/) there -- in other words,
> > > raid[+raid]+dm[+dm]+xfs+nfs can be fatal, yes.
> >
> > It should be reasonably simple to remove this problem of stacked
> > drivers.
> > There really isn't any need for md and dm (or md and md or ..) to use
> > the stack and the same time.
> >
>
> Sorry to bump in so late - but there seems to be a reporter who is
> suffering from these issues now:
>
> http://bugzilla.kernel.org/show_bug.cgi?id=5210
Nathan, can you look into this bug?
TIA
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
On Sun, Oct 02, 2005 at 12:50:12AM +0200, Adrian Bunk wrote:
> On Tue, Sep 13, 2005 at 10:05:52AM +0200, Alexander Nyberg wrote:
> >
> > http://bugzilla.kernel.org/show_bug.cgi?id=5210
>
> Nathan, can you look into this bug?
>
OK - looks like a scheduling-while-atomic warning on a metdata
buffer read, then a panic (sometime later?) at do_page_fault
with no useful stack trace (removed by bug reporter? - thats
the one you need...). The first stack trace (atomic/shedule)
would not be >4K afaict, so thats a red herring I think.
cheers.
--
Nathan