2003-05-30 23:27:42

by Andrew Morton

[permalink] [raw]
Subject: must-fix, version 6


There are probably quite a few things here which are in progress or are
already fixed. Could people please send updates.

I shall separate the "must fix" list from the "late features and speedups"
list. It's a bit confusing having them combined.

I have attempted to prioritise the "late features and speedups" items along
the following lines:

PRI1: We're totally lame if this doesn't get in
PRI2: Would be nice
PRI3: Not very important

I basically guessed. Feel free to argue.



ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/must-fix/



Changes since version 5:

--- /tmp/must-fix-5.txt Fri May 30 16:29:05 2003
+++ /tmp/must-fix-6.txt Fri May 30 16:29:12 2003
@@ -79,21 +79,21 @@
o CD burning. There are still a few quirks to solve wrt SG_IO and ide-cd.

Jens: The basic hang has been solved (double fault in ide-cd), there still
seems to be some cases that don't work too well. Don't really have a
handle on those :/

o lmb: Last time I looked at the multipath code (2.5.50 or so) it also
looked pretty broken; I plan to port forward the changes we did on 2.4
before KS.

-o elevator-noop is broken.
+o loop.c: fix http://bugzilla.kernel.org/show_bug.cgi?id=192

drivers/input/
~~~~~~~~~~~~~~

o rmk: unconverted keyboard/mouse drivers (there's a deadline of 2.6.0
currently on these remaining in my/Linus' tree.)

o viro: large absence of locking.

o synaptic touchpad support
@@ -231,70 +231,60 @@
doesn't get wedged.

o qlogic drivers: merge qlogicisp, feral with a view to dropping qlogicfc
and qlogicisp

o jejb: and merge the qla2xxx too

fs/
~~~

-o ext3 data=journal mode is bust.
+o ext3 data=journal mode is bust. (fix is in progress)

o ext3/htree readdir can return "." and ".." in unexpected order, which
might break buggy userspace apps. Ted has a fix planned.


o AIO/direct-IO writes can race with truncate and wreck filesystems.

- o Easy fix is to only allow the feature for S_ISBLK files.
+ o Easy fix is to add an rwsem to the inode.

o hch: devfs: there's a fundamental lookup vs devfsd race that's only
fixable by introducing a lookup vs devfs deadlock. I can't see how this is
fixable without getting rid of the current devfsd design. Mandrake seems
to have a workaround for this so this is at least not triggered so easily,
but that's not what I'd consider a fix..

o viro: fs/char_dev.c needs removal of aeb stuff and merge of cdev-cidr.
In progress.

o forward-port sct's O_DIRECT fixes

o viro: there is some generic stuff for namei/namespace/super, but that's a
slow-merge and can go in 2.6 just fine

o andi: also soft needs to be fixed - there are quite a lot of
uninterruptible waits in sunrpc/nfs

o trond: NFS has a mmap-versus-truncate problem

-kernel/sched.c/
-~~~~~~~~~~~~~~~
-
-o O(1) scheduler starvation, poor behaviour seems unresolved.
-
- Jens: "I've been running 2.5.67-mm3 on my workstation for two days, and
- it still doesn't feel as good as 2.4. It's not a disaster like some
- revisisons ago, but it still has occasional CPU "stalls" where it feels
- like a process waits for half a second of so for CPU time. That's is very
- noticable."
-
- Also see Mike Galbraith's work.
-
- Conclusion: the scheduler has issues, lots of people working on it. Rick
- Lindsley, Andrew Theurer.
+kernel/sched.c
+~~~~~~~~~~~~~~

o "Persistent starvation"

http://www.hpl.hp.com/research/linux/kernel/o1-starve.php

- ingo: "this is mostly invalid".
+ ingo: "basically by calling sleep(1) in an infinite loop you can end up
+ expiring yourself. The testcode (test-starve.c) triggers this. This is
+ solved by going to sub-timeslices. Which i've got done a few weeks ago and
+ it has seen some testing by others as well.

o Overeager affinity in presence of repeated yields

http://www.hpl.hp.com/research/linux/kernel/o1-openmp.php

ingo: this is valid. fix is in progress.

o The "thud.c" test app. This is a exploit for the interactivity
estimator. it's unlikely to bite in real-world cases. Needs watching.
Can be ameliorated by setting nice values.
@@ -313,35 +303,49 @@
IMHO

(alan has patch)

o nasty task refcounting bug is taking ages to track down. (bugzilla ref?)

o viro: core sysctl code is racy. And its interaction wiuth sysfs

o gettimeofday goes backwards. Merge up David M-T's fixes?

+o Daniel Jacobowitz <[email protected]>: when CLONE_DETACHED threads were
+ removed from /proc several approaches were suggested to let procps find out
+ about them and none of them were implemented. There's some real potential
+ for badness with these mostly-invisible processes. Something needs to be
+ added so that we can display and detect them.
+
+lib/kobject.c
+~~~~~~~~~~~~~
+
+o kobject refcounting (comments from Al Viro):
+
+ _anything_ can grab a temporary reference to kobject. IOW, if kobject is
+ embedded into something that could be freed - it _MUST_ have a destructor
+ and that destructor _MUST_ be the destructor for containing object.
+
+ Any violation of the above (and we already have a bunch of those) is a
+ user-triggerable memory corruption.
+
+ We can tolerate it for a while in 2.5 (e.g. during work on susbsystem we
+ can decide to switch to that way of handling objects and have subsystem
+ vulnerable for a while), but all such windows must be closed before 2.6 and
+ during 2.6 we can't open them at all.
+
mm/
~~~

o Overcommit accounting gets wrong answers

- o underestimates reclaimable slab, gives bogus failures when
- dcache&icache are large.
-
o gets confused by reclaimable-but-not-freed truncated ext3 pages.
- Lame fix exists in -mm.
-
-o Proper user level no overcommit also requires a root margin adding
-
-o There's a vmalloc race. David Woodhouse has a patch, but it had a
- problem. Need to revisit it.

o GFP_DMA32 (or something like that). Lots of ideas. jejb, zaitcev,
willy, arjan, wli.

o access_process_vm() doesn't flush right. We probably need new flushing
primitives to do this (davem?)


modules
~~~~~~~
@@ -449,110 +453,149 @@

o A couple of hundred real looking bugzilla bugs

o viro: cdev rework. Main group is pretty stable and I hope to feed it to
Linus RSN. That's cdev-cidr and ->i_cdev/->i_cindex stuff


Not-ready features and speedups
===============================

+Legend:
+
+PRI1: We're totally lame if this doesn't get in
+PRI2: Would be nice
+PRI3: Not very important

drivers/block/
~~~~~~~~~~~~~~

o Framework for selecting IO schedulers. This is the main one really.
Once this is in place we can drop in new schedulers any old time, no risk.

+ PRI1
+
o Anticipatory scheduler. Working OK now, still has problems with seeky
OLTP-style loads.

+ PRI1
+
o CFQ scheduler. Seems to work but Jens planning significant rework.

+ PRI2
+
o cryptoloop: jmorris: There's no cryptoloop in the 2.4 mainline kernel,
but I think every distro ships some version. It would probably be useful
to have crypto natively supported in 2.6, with backward compatibility for
the majority of 2.4 users.

problem: lack of a loop maintainer

+ PRI2
+
o viro: paride drivers need a big cleanup

+ PRI2
+
drivers/char/rtc/
~~~~~~~~~~~~~~~~~

-o rmk: I think we need a generic RTC driver (which is backed by real RTCs).
- Integrator-based stuff has a 32-bit 1Hz counter RTC with alarm, as has the
- SA11xx, and probably PXA. There's another implementation for the RiscPC
- and ARM26 stuff. I'd rather not see 4 implementations of the RTC userspace
- API, but one common implementation so that stuff gets done in a consistent
- way.
-
- We postponed this at the beginning of 2.4 until 2.5 happened. We're now
- at 2.5, and I'm about to add at least one more (the Integrator
- implementation.) This isn't sane imo.
+o rmk, trini: add support for alarms to the existing generic rtc driver.
+
+ PRI2

device mapper
~~~~~~~~~~~~~

o ioctl interface cleanup patch is ready (redo the structure layouts)

+ PRI1
+
o A port of the 2.4 snapshot target is in progress

+ PRI1
+
o the fs interface to dm needs to be redone. gregkh was going to work on
this. viro is interested in seeing work thus-far.

+ PRI2
+
drivers/net/wireless/
~~~~~~~~~~~~~~~~~~~~~

(Jean Tourrilhes <[email protected]>)

o get latest orinoco changes from David.

+ PRI1
+
o get the latest airo.c fixes from CVS. This will hopefully fix problems
people have reported on the LKML.

+ PRI1
+
o get HostAP driver in the kernel. No consolidation of the 802.11
management across driver can happen until this one is in (which is probably
2.7.X material). I think Jouni is mostly ready but didn't find time for
it.

+ PRI2
+
o get more wireless drivers into the kernel. The most "integrable" drivers
at this point seem the NWN driver, Pavel's Spectrum driver and the Atmel
driver.

+ PRI1
+
o The last two drivers mentioned above are held up by firmware issues (see
flamewar on LKML a few days ago). So maybe fixing those firmware issues
should be a requirement for 2.6.X, because we can expect more wireless
devices to need firmware upload at startup coming to market.

+ (in progress?)
+
+ PRI1
+
drivers/usb/gadget/
~~~~~~~~~~~~~~~~~~~

o rmk: SA11xx USB client/gadget code (David B has been doing some work on
this, and keeps trying to prod me, but unfortunately I haven't had the time
to look at his work, sorry David.)

+ PRI3
+
fs/
~~~

o ext3 lock_kernel() removal: that part works OK and is mergeable. But
we'll also need to make lock_journal() a spinlock, and that's deep surgery.

+ Patches exist in -mm.
+
+ PRI1
+
o ext3 and ext2 block allocators have serious failure modes - interleaved
allocations.

+ PRI3
+
o 32bit quota needs a lot more testing but may work now

+ PRI2
+
o Integrate Chris Mason's 2.4 reiserfs ordered data and data journaling
patches. They make reiserfs a lot safer.

+ Ordered: PRI2
+ data journalled: PRI3
+
o (Trond:) Yes: I'm still working on an atomic "open()", i.e. one
where we short-circuit the usual VFS path_walk() + lookup() +
permission() + create() + .... bullsh*t...

I have several reasons for wanting to do this (all of
them related to NFS of course, but much of the reasoning applies
to *all* networked file systems).

1) The above sequence is simply not atomic on *any* networked
filesystem.
@@ -567,295 +610,427 @@
doesn't pass an 'EXCLUSIVE' flag down to the filesystems.

4) (NFS specific?) open() has very different cache consistency
requirements when compared to most other VFS operations.

I'd very much like for something like Peter Braam's 'lookup with
intent' or (better yet) for a proper dentry->open() to be integrated with
path_walk()/open_namei(). I'm still working on the latter (Peter has
already completed the lookup with intent stuff).

+ PRI2 (?)
+
+o (Chuck Lever <[email protected]>): NFS O_DIRECT support must be
+ completed. The best approach is to fall back to something like the 2.4 NFS
+ O_DIRECT support, which issues RPCs synchronously and uses the RPC
+ completion mechanism to wait for I/O completion.
+
+ PRI2
+
o rmk: update acorn partition parsing code - making all acorn schemes
appear in check.c so we don't have to duplicate the scanning of multiple
types, and adding support for eesox partitions.

+ PRI2
+
o atomic i_size patches

+ PRI1
+
o viro: cleaning up options-parsers in filesystems. (patch exists, needs
porting).

+ PRI2
+
o aio: fs IO isn't async at present. suparna has restart patches, they're
in -mm. Need to get Ben to review/comment.

+ PRI1.

kernel/
~~~~~~~

o rusty: Zippel's Reference count simplification. Tricky code, but cuts
about 120 lines from module.c. Patch exists, needs stressing.

+ PRI3
+
o rusty: /proc/kallsyms. What most people really wanted from /proc/ksyms.
Patch exists.

+ PRI3
+
o rusty: Fix module-failed-init races by starting module "disabled". Patch
exists, requires some subsystems (ie. add_partition) to explicitly say
"make module live now". Without patch we are no worse off than 2.4 etc.

+ PRI1
+
o Integrate userspace irq balancing daemon.

-o kexec. Seems to work, is in -mm.
+ PRI2
+
+o kexec. Seems to work, was in -mm.
+
+ PRI3

o rmk: modules / /proc/kcore / vmalloc This needs sorting and testing to
ensure that stuff like gdb vmlinux /proc/kcore works as expected. I
believe this is the only show stopper preventing any ARM platform being
built in Linus' kernel.

+ Patch exists in -mm, nobody has tested it for the above afaik.
+
+ PRI1
+
o kcore is a problem for ia64 (Tony Luck)

+ Patch exists in -mm.
+
+ PRI1
+
o rmk: lib/inflate.c must not use static variables (causes these to be
referenced via GOTOFF relocations in PIC decompressor. We have a PIC
decompressor to avoid having to hard code a per platform zImage link
address into the makefiles.)

+ PRI2
+
o klibc merge?

+ PRI2
+
mm/
~~~

o objrmap: concerns over page reclaim performance at high sharing levels,
and interoperation with nonlinear mappings is hairy.

o Reintroduce and make /proc/sys/vm/freepages writable again so that boxes
can be tuned for heavy interrupt load.

+ Patch exists in -mm.
+
+ PRI1
+
o oxymoron's async write-error-handling patch

+ PRI1
+
+o dropbehind for large files
+
+ PRI2
+
net/
~~~~

(davem)

o Real serious use of IPSEC is hampered by lack of MPLS support. MPLS is a
switching technology that works by switching based upon fixed length labels
prepended to packets. Many people use this and IPSEC to implement VPNs
over public networks, it is also used for things like traffic engineering.

A good reference site is:

http://www.mplsrc.com/

Anyways, an existing (crappy) implementation exists. I've almost
completed a rewrite, I should have something in the tree next week.

+ PRI1
+
o Sometimes we generate IP fragments when it truly isn't necessary.

The way IP fragmentation is specified, each fragment must be modulo 8
bytes in length. So suppose the device has an MTU that is not 0 modulo 8,
ethernet even classifies in this way. 1500 == (8 * 187) + 4

Our IP fragmenting engine can fragment on packets that are sized within
the last modulo 8 bytes of the MTU. This happens in obscure cases, but it
does happen.

I've proposed a fix to Alexey, whereby very late in the output path we
check the packet, if we fragmented but the data length would fit into the
MTU we unfragment the packet.

This is low priority, because technically it creates suboptimal behavior
rather than mis-operation.

+ PRI1
+
net/*/netfilter/
~~~~~~~~~~~~~~~~

o Lots of misc. cleanups, which are happening slowly.

+ PRI2
+
o davem: Netfilter needs to stop linearizing packets as much as possible.

Zerocopy output packets are basically undone by netfilter becuase all of
it assumed it was working with linear socket buffers.

Rusty is fixing this piece by piece. He is nearly done with this work.

+ PRI1
+
power management
~~~~~~~~~~~~~~~~

(Pat) There is some preliminary work at bk://ldm.bkbits.net/linux-2.5-power,
though I'm currently in the process of reworking it.

It includes:

o New device power management core code, both for individual devices,
and for global state transitions.

+ PRI1
+
o A generic user interface for triggering system power state transitions.

+ PRI1
+
o Arch-independent code for performing state transitions, that calls
platform-specific methods along the way.

+ PRI1
+
o A better suspend-to-disk mechanism than swsusp.

There are various other details to be worked out, which are the real fun
part. And of course, driver support, but that is something that can happen
at any time.

(Alan)

+ PRI2
+
o Frame buffer restore codepaths (that requires some deep PCI magic)

+ PRI2
+
o XFree86 hooks

+ PRI2
+
o AGP restoration

+ PRI2
+
o DRI restoration

(davej/Alan: not super-critical, can crash laptop on restore. davej
looking into it.)

+ PRI2
+
o IDE suspend/resume without races (Ben is looking at this a little)

+ PRI2
+
o Pat: There are already CPU device structures; MTRRs should be a
dynamically registered interface of CPUs, which implies there needs
to be some other glue to know that there are MTRRs that need to be
saved/restored.

+ PRI1
+
global
~~~~~~

o 64-bit dev_t. Seems almost ready, but it's not really known how much
work is still to do. Patches exist in -mm but with the recent rise of the
neo-viro I'm not sure where things are at.

+ PRI1
+
o We need a kernel side API for reporting error events to userspace (could
be async to 2.6 itself)

(Prototype core based on netlink exists)

+ PRI2
+
o Kai: Introduce a sane, easy and standard way to build external modules

+ PRI2
+
o Kai: Allow separate src/objdir

+ PRI2
+
o general confusion over firmware policy:

o do we mandate that it be uploaded from userspace?

o Is binary-blob-in-kernel-image OK?

o Each driver (wireless, scsi, etc) seems to do it in a different,
private manner.

gregkh: patch exists, drivers can be ported to use new infrastructure at
any time.

+ PRI1
+
o larger cpumask_t - supporting more than BITS_PER_LONG CPUs.

wli: patch exists. ia32, ppc are done. ppc64 in progress. Needs work
for other architectures.

+ PRI1
+
+o pavel: ioctl32 emulation should be shared across architectures. (patch
+ exists).
+
+ PRI2
+
drivers
~~~~~~~

-o Some network drivers don't even build
-
o Alan: Cardbus/PCMCIA requires all Russell's stuff is merged to do
multiheader right and so on

+ PRI1
+
drivers/acpi/
~~~~~~~~~~~~~

-o davej: ACPI has a number of failures right now. There are a number of
- entries in bugzilla which could all be the same bug. It manifests as a
- "network card doesn't recieve packets" booting with 'acpi=off noapic' fixes
- it.
-
- alan: VIA APIC stuff is one bit of this, there are also some other
+o alan: VIA APIC stuff is one bit of this, there are also some other
reports that were caused by ACPI not setting level v edge trigger some
times

-o davej: There's also another nasty 'doesnt boot' bug which quite a few
- people (myself included) are seeing on some boxes (especially laptops).
+ PRI1

o mochel: it seems the acpi irq routing code could use a serious rewrite.

+ grover: The problem is the ACPI irq routing code is trying to piggyback
+ on the existing MPS-specific data structures, and it's generally a hack.
+ So yes mochel is right, but it is also purging MPS-ities from common code
+ as well. I've done some preliminary work in this area and it doesn't seem
+ to break anything (yet) but a rewrite in this area imho should not be
+ rushed out the door. And, I think the above bugs can be fixed w/o the
+ rewrite.
+
+ PRI2
+
o mochel: ACPI suspend doesn't work. Important, not cricital. Pat is
working it.

+ PRI2
+
drivers/block/
~~~~~~~~~~~~~~

o Floppy is almost unusably buggy still

akpm: we need more people to test & report.

+ alan: "Floppy has worked for me since the patches that went in 2.5.69-ac
+ and I think -bk somewhere"
+
+ PRI1
+
drivers/char/
~~~~~~~~~~~~~

o Alan: Multiple serious bugs in the DRI drivers (most now with patches
thankfully). "The badness I know about is almost entirely IRQ mishandling.
DRI failing to mask PCI irqs on exit paths."

- (might be fixed due to DRI updates?)
+ (This is understood and fixed in DRI CVS)
+
+ PRI1

o Various suspect things in AGP.

+ PRI1
+
drivers/ide/
~~~~~~~~~~~~

(Alan)

o IDE requires bio walking

"Bartlomiej has IDE multisector working" (does that mean it's fixed?)

+ PRI1

o IDE PIO has occasional unexplained PIO disk eating reports

+ PRI1
+
o IDE has multiple zillions of races/hangs in 2.5 still

+ PRI1
+
o IDE scsi needs rewriting

+ PRI2
+
o IDE needs significant reworking to handle Simplex right

+ PRI2
+
o IDE hotplug handling for 2.5 is completely broken still

+ PRI2
+
o There are lots of other IDE bugs that wont go away until the taskfile
stuff is included, the locking bugs that allow any user to hang the IDE
layer in 2.5, and some other updates are forward ported. (esp. HPT372N).

+ PRI1
+
drivers/isdn/
~~~~~~~~~~~~~

(Kai, rmk)

o isdn_tty locking is completely broken (cli() and friends)

+ PRI2
+
o fix lots of remaining bugs in the isdn link layer / hisax protocol layer
/ hisax subdrivers, so that at least 99% of the users have a usable ISDN
subsystem

+ PRI1
+
o fix other drivers

+ PRI2
+
o lots more cleanups, adaption to recent APIs etc

+ PRI3
+
o fixup tty-based ISDN drivers which provide TIOCM* ioctls (see my recent
3-set patch for serial stuff)

Alternatively, we could re-introduce the fallback to driver ioctl parsing
for these if not enough drivers get updated.

+ PRI3
+
drivers/net/
~~~~~~~~~~~~

o davej: Either Wireless network drivers or PCMCIA broke somewhen. A
configuration that worked fine under 2.4 doesn't receive any packets. Need
to look into this more to make sure I don't have any misconfiguration that
just 'happened to work' under 2.4

+ PRI1

drivers/scsi/
~~~~~~~~~~~~~

o qlogic follies:

- jejb: Merge the feral driver. It covers all qlogic chips: 1020 all
the way up to 23xxx. mjacob is promising a "major" rewrite which
eliminates this as a candidate for immediate inclusion. Panics on my
parisc hardware, works on my ia64. BK tree is
@@ -866,81 +1041,193 @@
until I at least see how Qlogic responds to the issues. Can't currently
build this for my only fibre card (a qla2100). BK tree is at
http://linux-scsi.bkbits.net/scsi-qla2xxx-2.5

- I think the best plan currently is not to merge either of these, but
keep shadow BK trees for them (thus holding out the possibility of
merger) to see how they evolve. I agree with hch that feral seems to be
in the better shape but, barring directions to the contrary, I can't see
why both shouldn't be included eventually.

+ PRI2
+
arch/i386/
~~~~~~~~~~

o Also PC9800 merge needs finishing to the point we want for 2.6 (not all).

+ PRI3
+
o ES7000 wants merging (now we are all happy with it). That shouldn't be a
big problem.

+ PRI2
+
o davej: PAT support (for mtrr exhaustion w/ AGP)

+ PRI2
+
o 2.5.x won't boot on some 440GX

alan: Problem understood now, feasible fix in 2.4/2.4-ac. (440GX has two
IRQ routers, we use the $PIR table with the PIIX, but the 440GX doesnt use
the PIIX for its IRQ routing). Fall back to BIOS for 440GX works and Intel
concurs.

+ PRI1
+
o 2.5.x doesn't handle VIA APIC right yet.

1. We must write the PCI_INTERRUPT_LINE

2. We have quirk handlers that seem to trash it.

+ PRI1
+
o ACPI needs the relax patches merging to work on lots of laptops

+ alan: ACPI relax stuff is in 2.4-ac, compaq workaround is in next -ac
+ coming. These seem to deliver the goods - toshibas now work a treat. Some
+ other relax bits are being discussed (assume local0 starts 0 etc) and
+ progress looks great. This can occur before 2.6 or during.
+
+ PRI1
+
o ECC driver questions are not yet sorted (DaveJ is working on this) (Dan
Hollis)

+ alan: ECC - I have some test bits from Dan's stuff - they need no kernel
+ core changes for most platforms. That means we can treat it as a random
+ driver merge.
+
+ PRI3
+
+o alan: 2.4 has some fixes for tsc handling bugs. One where some bioses in
+ SMM mode mess up our toggle on the time high/low or mangle the counter and
+ one where a few chips need religious use of _p for timer access and we
+ don't do that. This is forward porting little bits of fixup.
+
+ ACPI HZ stuff we can't trap - a lot of ACPI is implemented as outb's
+ triggering SMM traps
+
+ PRI1
+
arch/x86_64/
~~~~~~~~~~~~

(Andi)

o time handling is broken. Need to move up 2.4 time.c code.

+ PRI1
+
o Another report of a crash at shutdown on Simics with no iommu when all
memory was used. Could be related to the one above.

+ PRI1
+
o NMI watchdog seems to tick too fast

+ PRI2
+
o not very well tested. probably more bugs lurking.

+ PRI1
+
o need to coredump 64bit vsyscall code with dwarf2

+ PRI2
+
o move 64bit signal trampolines into vsyscall code and add dwarf2 for it.

+ PRI1
+
o describe kernel assembly with dwarf2 annotations for kgdb (currently
waiting on some binutils changes for this)

+ PRI3
+
arch/alpha/
~~~~~~~~~~~

o rth: Ptrace writes are broken. This means we can't (reliably) set
breakpoints or modify variables from gdb.

+ PRI1
+
arch/arm/
~~~~~~~~~

o rmk: missing raw keyboard translation tables for all ARM machines.
Haven't even looked into this at all. This could be messy since there
isn't an ARM architecture standard. I'm presently hoping that it won't be
an issue. If it does, I guess we'll see drivers/char/keyboard.c explode.

+ PRI2
+
arch/others/
~~~~~~~~~~~~

o SH/SH-64 need resyncing, as do some other ports. No impact on
mainstream platforms hopefully.

+ PRI2
+
o IA64 needs merging, has impact on core code

+ PRI1
+
+arch/s390/
+~~~~~~~~~
+
+o A nastly memory management problem causes random crashes. These appear
+ to be fixed/hidden by the objrmap patch, more investigation is needed.
+
+ PRI1
+
+drivers/s390/
+~~~~~~~~~~~~~
+
+o Early userspace and 64 bit dev_t will allow the removal of most of
+ dasd_devmap.c and dasd_genhd.c.
+
+ PRI2
+
+o The 3270 console driver needs to be replaced with a working one
+ (prototype is there, needs to be finished).
+
+ PRI2
+
+o Minor interface changes are pending in cio/ when the z990 machines are
+ out.
+
+ There are some more things being worked on that are either post-2.6.0 or
+ are likely to remain outside of the official kernel (i.e. not for your
+ list):
+
+ PRI3
+
+o Jan Glauber is working on a fix for the timer issues related to running
+ on virtualized CPUs (wall-clock vs. cpu time).
+
+ PRI1
+
+o new zfcp fibre channel driver
+
+ PRI3
+
+o the qeth driver will become GPL soon
+
+ PRI3
+
+o a block device driver for ramdisks shared among virtual machines
+
+ PRI3
+
+o driver for crypto hardware
+
+ PRI2
+
+o 'claw' network device driver
+
+ PRI3
+


2003-05-30 23:30:26

by Andrew Morton

[permalink] [raw]
Subject: Re: must-fix, version 6


The should-fix list, version 6.


Not-ready features and speedups
===============================

Legend:

PRI1: We're totally lame if this doesn't get in
PRI2: Would be nice
PRI3: Not very important

drivers/block/
~~~~~~~~~~~~~~

o Framework for selecting IO schedulers. This is the main one really.
Once this is in place we can drop in new schedulers any old time, no risk.

PRI1

o Anticipatory scheduler. Working OK now, still has problems with seeky
OLTP-style loads.

PRI1

o CFQ scheduler. Seems to work but Jens planning significant rework.

PRI2

o cryptoloop: jmorris: There's no cryptoloop in the 2.4 mainline kernel,
but I think every distro ships some version. It would probably be useful
to have crypto natively supported in 2.6, with backward compatibility for
the majority of 2.4 users.

problem: lack of a loop maintainer

PRI2

o viro: paride drivers need a big cleanup

PRI2

drivers/char/rtc/
~~~~~~~~~~~~~~~~~

o rmk, trini: add support for alarms to the existing generic rtc driver.

PRI2

device mapper
~~~~~~~~~~~~~

o ioctl interface cleanup patch is ready (redo the structure layouts)

PRI1

o A port of the 2.4 snapshot target is in progress

PRI1

o the fs interface to dm needs to be redone. gregkh was going to work on
this. viro is interested in seeing work thus-far.

PRI2

drivers/net/wireless/
~~~~~~~~~~~~~~~~~~~~~

(Jean Tourrilhes <[email protected]>)

o get latest orinoco changes from David.

PRI1

o get the latest airo.c fixes from CVS. This will hopefully fix problems
people have reported on the LKML.

PRI1

o get HostAP driver in the kernel. No consolidation of the 802.11
management across driver can happen until this one is in (which is probably
2.7.X material). I think Jouni is mostly ready but didn't find time for
it.

PRI2

o get more wireless drivers into the kernel. The most "integrable" drivers
at this point seem the NWN driver, Pavel's Spectrum driver and the Atmel
driver.

PRI1

o The last two drivers mentioned above are held up by firmware issues (see
flamewar on LKML a few days ago). So maybe fixing those firmware issues
should be a requirement for 2.6.X, because we can expect more wireless
devices to need firmware upload at startup coming to market.

(in progress?)

PRI1

drivers/usb/gadget/
~~~~~~~~~~~~~~~~~~~

o rmk: SA11xx USB client/gadget code (David B has been doing some work on
this, and keeps trying to prod me, but unfortunately I haven't had the time
to look at his work, sorry David.)

PRI3

fs/
~~~

o ext3 lock_kernel() removal: that part works OK and is mergeable. But
we'll also need to make lock_journal() a spinlock, and that's deep surgery.

Patches exist in -mm.

PRI1

o ext3 and ext2 block allocators have serious failure modes - interleaved
allocations.

PRI3

o 32bit quota needs a lot more testing but may work now

PRI2

o Integrate Chris Mason's 2.4 reiserfs ordered data and data journaling
patches. They make reiserfs a lot safer.

Ordered: PRI2
data journalled: PRI3

o (Trond:) Yes: I'm still working on an atomic "open()", i.e. one
where we short-circuit the usual VFS path_walk() + lookup() +
permission() + create() + .... bullsh*t...

I have several reasons for wanting to do this (all of
them related to NFS of course, but much of the reasoning applies
to *all* networked file systems).

1) The above sequence is simply not atomic on *any* networked
filesystem.

2) It introduces a sh*tload of completely unnecessary RPC calls (why
do a 'permission' RPC call when the server is in *any* case going to
tell you whether or not this operations is allowed. Why do a
'lookup()' when the 'create()' call can be made to tell you whether or
not a file already exists).

3) It is incompatible with some operations: the current create()
doesn't pass an 'EXCLUSIVE' flag down to the filesystems.

4) (NFS specific?) open() has very different cache consistency
requirements when compared to most other VFS operations.

I'd very much like for something like Peter Braam's 'lookup with
intent' or (better yet) for a proper dentry->open() to be integrated with
path_walk()/open_namei(). I'm still working on the latter (Peter has
already completed the lookup with intent stuff).

PRI2 (?)

o (Chuck Lever <[email protected]>): NFS O_DIRECT support must be
completed. The best approach is to fall back to something like the 2.4 NFS
O_DIRECT support, which issues RPCs synchronously and uses the RPC
completion mechanism to wait for I/O completion.

PRI2

o rmk: update acorn partition parsing code - making all acorn schemes
appear in check.c so we don't have to duplicate the scanning of multiple
types, and adding support for eesox partitions.

PRI2

o atomic i_size patches

PRI1

o viro: cleaning up options-parsers in filesystems. (patch exists, needs
porting).

PRI2

o aio: fs IO isn't async at present. suparna has restart patches, they're
in -mm. Need to get Ben to review/comment.

PRI1.

kernel/
~~~~~~~

o rusty: Zippel's Reference count simplification. Tricky code, but cuts
about 120 lines from module.c. Patch exists, needs stressing.

PRI3

o rusty: /proc/kallsyms. What most people really wanted from /proc/ksyms.
Patch exists.

PRI3

o rusty: Fix module-failed-init races by starting module "disabled". Patch
exists, requires some subsystems (ie. add_partition) to explicitly say
"make module live now". Without patch we are no worse off than 2.4 etc.

PRI1

o Integrate userspace irq balancing daemon.

PRI2

o kexec. Seems to work, was in -mm.

PRI3

o rmk: modules / /proc/kcore / vmalloc This needs sorting and testing to
ensure that stuff like gdb vmlinux /proc/kcore works as expected. I
believe this is the only show stopper preventing any ARM platform being
built in Linus' kernel.

Patch exists in -mm, nobody has tested it for the above afaik.

PRI1

o kcore is a problem for ia64 (Tony Luck)

Patch exists in -mm.

PRI1

o rmk: lib/inflate.c must not use static variables (causes these to be
referenced via GOTOFF relocations in PIC decompressor. We have a PIC
decompressor to avoid having to hard code a per platform zImage link
address into the makefiles.)

PRI2

o klibc merge?

PRI2

mm/
~~~

o objrmap: concerns over page reclaim performance at high sharing levels,
and interoperation with nonlinear mappings is hairy.

o Reintroduce and make /proc/sys/vm/freepages writable again so that boxes
can be tuned for heavy interrupt load.

Patch exists in -mm.

PRI1

o oxymoron's async write-error-handling patch

PRI1

o dropbehind for large files

PRI2

net/
~~~~

(davem)

o Real serious use of IPSEC is hampered by lack of MPLS support. MPLS is a
switching technology that works by switching based upon fixed length labels
prepended to packets. Many people use this and IPSEC to implement VPNs
over public networks, it is also used for things like traffic engineering.

A good reference site is:

http://www.mplsrc.com/

Anyways, an existing (crappy) implementation exists. I've almost
completed a rewrite, I should have something in the tree next week.

PRI1

o Sometimes we generate IP fragments when it truly isn't necessary.

The way IP fragmentation is specified, each fragment must be modulo 8
bytes in length. So suppose the device has an MTU that is not 0 modulo 8,
ethernet even classifies in this way. 1500 == (8 * 187) + 4

Our IP fragmenting engine can fragment on packets that are sized within
the last modulo 8 bytes of the MTU. This happens in obscure cases, but it
does happen.

I've proposed a fix to Alexey, whereby very late in the output path we
check the packet, if we fragmented but the data length would fit into the
MTU we unfragment the packet.

This is low priority, because technically it creates suboptimal behavior
rather than mis-operation.

PRI1

net/*/netfilter/
~~~~~~~~~~~~~~~~

o Lots of misc. cleanups, which are happening slowly.

PRI2

o davem: Netfilter needs to stop linearizing packets as much as possible.

Zerocopy output packets are basically undone by netfilter becuase all of
it assumed it was working with linear socket buffers.

Rusty is fixing this piece by piece. He is nearly done with this work.

PRI1

power management
~~~~~~~~~~~~~~~~

(Pat) There is some preliminary work at bk://ldm.bkbits.net/linux-2.5-power,
though I'm currently in the process of reworking it.

It includes:

o New device power management core code, both for individual devices,
and for global state transitions.

PRI1

o A generic user interface for triggering system power state transitions.

PRI1

o Arch-independent code for performing state transitions, that calls
platform-specific methods along the way.

PRI1

o A better suspend-to-disk mechanism than swsusp.

There are various other details to be worked out, which are the real fun
part. And of course, driver support, but that is something that can happen
at any time.

(Alan)

PRI2

o Frame buffer restore codepaths (that requires some deep PCI magic)

PRI2

o XFree86 hooks

PRI2

o AGP restoration

PRI2

o DRI restoration

(davej/Alan: not super-critical, can crash laptop on restore. davej
looking into it.)

PRI2

o IDE suspend/resume without races (Ben is looking at this a little)

PRI2

o Pat: There are already CPU device structures; MTRRs should be a
dynamically registered interface of CPUs, which implies there needs
to be some other glue to know that there are MTRRs that need to be
saved/restored.

PRI1

global
~~~~~~

o 64-bit dev_t. Seems almost ready, but it's not really known how much
work is still to do. Patches exist in -mm but with the recent rise of the
neo-viro I'm not sure where things are at.

PRI1

o We need a kernel side API for reporting error events to userspace (could
be async to 2.6 itself)

(Prototype core based on netlink exists)

PRI2

o Kai: Introduce a sane, easy and standard way to build external modules

PRI2

o Kai: Allow separate src/objdir

PRI2

o general confusion over firmware policy:

o do we mandate that it be uploaded from userspace?

o Is binary-blob-in-kernel-image OK?

o Each driver (wireless, scsi, etc) seems to do it in a different,
private manner.

gregkh: patch exists, drivers can be ported to use new infrastructure at
any time.

PRI1

o larger cpumask_t - supporting more than BITS_PER_LONG CPUs.

wli: patch exists. ia32, ppc are done. ppc64 in progress. Needs work
for other architectures.

PRI1

o pavel: ioctl32 emulation should be shared across architectures. (patch
exists).

PRI2

drivers
~~~~~~~

o Alan: Cardbus/PCMCIA requires all Russell's stuff is merged to do
multiheader right and so on

PRI1

drivers/acpi/
~~~~~~~~~~~~~

o alan: VIA APIC stuff is one bit of this, there are also some other
reports that were caused by ACPI not setting level v edge trigger some
times

PRI1

o mochel: it seems the acpi irq routing code could use a serious rewrite.

grover: The problem is the ACPI irq routing code is trying to piggyback
on the existing MPS-specific data structures, and it's generally a hack.
So yes mochel is right, but it is also purging MPS-ities from common code
as well. I've done some preliminary work in this area and it doesn't seem
to break anything (yet) but a rewrite in this area imho should not be
rushed out the door. And, I think the above bugs can be fixed w/o the
rewrite.

PRI2

o mochel: ACPI suspend doesn't work. Important, not cricital. Pat is
working it.

PRI2

drivers/block/
~~~~~~~~~~~~~~

o Floppy is almost unusably buggy still

akpm: we need more people to test & report.

alan: "Floppy has worked for me since the patches that went in 2.5.69-ac
and I think -bk somewhere"

PRI1

drivers/char/
~~~~~~~~~~~~~

o Alan: Multiple serious bugs in the DRI drivers (most now with patches
thankfully). "The badness I know about is almost entirely IRQ mishandling.
DRI failing to mask PCI irqs on exit paths."

(This is understood and fixed in DRI CVS)

PRI1

o Various suspect things in AGP.

PRI1

drivers/ide/
~~~~~~~~~~~~

(Alan)

o IDE requires bio walking

"Bartlomiej has IDE multisector working" (does that mean it's fixed?)

PRI1

o IDE PIO has occasional unexplained PIO disk eating reports

PRI1

o IDE has multiple zillions of races/hangs in 2.5 still

PRI1

o IDE scsi needs rewriting

PRI2

o IDE needs significant reworking to handle Simplex right

PRI2

o IDE hotplug handling for 2.5 is completely broken still

PRI2

o There are lots of other IDE bugs that wont go away until the taskfile
stuff is included, the locking bugs that allow any user to hang the IDE
layer in 2.5, and some other updates are forward ported. (esp. HPT372N).

PRI1

drivers/isdn/
~~~~~~~~~~~~~

(Kai, rmk)

o isdn_tty locking is completely broken (cli() and friends)

PRI2

o fix lots of remaining bugs in the isdn link layer / hisax protocol layer
/ hisax subdrivers, so that at least 99% of the users have a usable ISDN
subsystem

PRI1

o fix other drivers

PRI2

o lots more cleanups, adaption to recent APIs etc

PRI3

o fixup tty-based ISDN drivers which provide TIOCM* ioctls (see my recent
3-set patch for serial stuff)

Alternatively, we could re-introduce the fallback to driver ioctl parsing
for these if not enough drivers get updated.

PRI3

drivers/net/
~~~~~~~~~~~~

o davej: Either Wireless network drivers or PCMCIA broke somewhen. A
configuration that worked fine under 2.4 doesn't receive any packets. Need
to look into this more to make sure I don't have any misconfiguration that
just 'happened to work' under 2.4

PRI1

drivers/scsi/
~~~~~~~~~~~~~

o qlogic follies:

- jejb: Merge the feral driver. It covers all qlogic chips: 1020 all
the way up to 23xxx. mjacob is promising a "major" rewrite which
eliminates this as a candidate for immediate inclusion. Panics on my
parisc hardware, works on my ia64. BK tree is
http://linux-scsi.bkbits.net/scsi-isp-2.5

- qla2xxx: only for FC chips. Has significant build issues. hch
promises to send me a "must fix" list for this. I plan not to merge this
until I at least see how Qlogic responds to the issues. Can't currently
build this for my only fibre card (a qla2100). BK tree is at
http://linux-scsi.bkbits.net/scsi-qla2xxx-2.5

- I think the best plan currently is not to merge either of these, but
keep shadow BK trees for them (thus holding out the possibility of
merger) to see how they evolve. I agree with hch that feral seems to be
in the better shape but, barring directions to the contrary, I can't see
why both shouldn't be included eventually.

PRI2

arch/i386/
~~~~~~~~~~

o Also PC9800 merge needs finishing to the point we want for 2.6 (not all).

PRI3

o ES7000 wants merging (now we are all happy with it). That shouldn't be a
big problem.

PRI2

o davej: PAT support (for mtrr exhaustion w/ AGP)

PRI2

o 2.5.x won't boot on some 440GX

alan: Problem understood now, feasible fix in 2.4/2.4-ac. (440GX has two
IRQ routers, we use the $PIR table with the PIIX, but the 440GX doesnt use
the PIIX for its IRQ routing). Fall back to BIOS for 440GX works and Intel
concurs.

PRI1

o 2.5.x doesn't handle VIA APIC right yet.

1. We must write the PCI_INTERRUPT_LINE

2. We have quirk handlers that seem to trash it.

PRI1

o ACPI needs the relax patches merging to work on lots of laptops

alan: ACPI relax stuff is in 2.4-ac, compaq workaround is in next -ac
coming. These seem to deliver the goods - toshibas now work a treat. Some
other relax bits are being discussed (assume local0 starts 0 etc) and
progress looks great. This can occur before 2.6 or during.

PRI1

o ECC driver questions are not yet sorted (DaveJ is working on this) (Dan
Hollis)

alan: ECC - I have some test bits from Dan's stuff - they need no kernel
core changes for most platforms. That means we can treat it as a random
driver merge.

PRI3

o alan: 2.4 has some fixes for tsc handling bugs. One where some bioses in
SMM mode mess up our toggle on the time high/low or mangle the counter and
one where a few chips need religious use of _p for timer access and we
don't do that. This is forward porting little bits of fixup.

ACPI HZ stuff we can't trap - a lot of ACPI is implemented as outb's
triggering SMM traps

PRI1

arch/x86_64/
~~~~~~~~~~~~

(Andi)

o time handling is broken. Need to move up 2.4 time.c code.

PRI1

o Another report of a crash at shutdown on Simics with no iommu when all
memory was used. Could be related to the one above.

PRI1

o NMI watchdog seems to tick too fast

PRI2

o not very well tested. probably more bugs lurking.

PRI1

o need to coredump 64bit vsyscall code with dwarf2

PRI2

o move 64bit signal trampolines into vsyscall code and add dwarf2 for it.

PRI1

o describe kernel assembly with dwarf2 annotations for kgdb (currently
waiting on some binutils changes for this)

PRI3

arch/alpha/
~~~~~~~~~~~

o rth: Ptrace writes are broken. This means we can't (reliably) set
breakpoints or modify variables from gdb.

PRI1

arch/arm/
~~~~~~~~~

o rmk: missing raw keyboard translation tables for all ARM machines.
Haven't even looked into this at all. This could be messy since there
isn't an ARM architecture standard. I'm presently hoping that it won't be
an issue. If it does, I guess we'll see drivers/char/keyboard.c explode.

PRI2

arch/others/
~~~~~~~~~~~~

o SH/SH-64 need resyncing, as do some other ports. No impact on
mainstream platforms hopefully.

PRI2

o IA64 needs merging, has impact on core code

PRI1

arch/s390/
~~~~~~~~~

o A nastly memory management problem causes random crashes. These appear
to be fixed/hidden by the objrmap patch, more investigation is needed.

PRI1

drivers/s390/
~~~~~~~~~~~~~

o Early userspace and 64 bit dev_t will allow the removal of most of
dasd_devmap.c and dasd_genhd.c.

PRI2

o The 3270 console driver needs to be replaced with a working one
(prototype is there, needs to be finished).

PRI2

o Minor interface changes are pending in cio/ when the z990 machines are
out.

There are some more things being worked on that are either post-2.6.0 or
are likely to remain outside of the official kernel (i.e. not for your
list):

PRI3

o Jan Glauber is working on a fix for the timer issues related to running
on virtualized CPUs (wall-clock vs. cpu time).

PRI1

o new zfcp fibre channel driver

PRI3

o the qeth driver will become GPL soon

PRI3

o a block device driver for ramdisks shared among virtual machines

PRI3

o driver for crypto hardware

PRI2

o 'claw' network device driver

PRI3


2003-05-30 23:28:19

by Andrew Morton

[permalink] [raw]
Subject: Re: must-fix, version 6


Must-fix bugs
=============

drivers/char/
~~~~~~~~~~~~~

o TTY locking is broken.

o see FIXME in do_tty_hangup(). This causes ppp BUGs in local_bh_enable()

o Other problems: aviro, dipankar, Alan have details.

o somebody will have to document the tty driver and ldisc API

o Lack of test cases and/or stress tests is a problem. Contributions and
suggestions are sought.

o Lots of drivers are using cli/sti and are broken.

o willy: random.c is completely lockfree, and not in a good way. i had
some patches but nothing got seriously tested.

drivers/tty
~~~~~~~~~~~

o viro: we need to fix refcounting for tty_driver (oopsable race, must fix
anyway, hopefully about a week until it's merged) then we can do
tty/misc/upper levels of sound and hopefully upper level of USB.

USB is a place where we _really_ need to deal with dynamic allocation of
device numbers and that will bite.

drivers/block/
~~~~~~~~~~~~~~

o RAID0 dies on strangely aligned BIOs

o Need to hoist BIO-split code out of device mapper, use that.

arjan: "if we add that function, we must be sure that it can split on
not-a-page boundaries too otherwise it's useless for a bunch of things"

(neilb)

1/ RAID5 should work fine. It accepts any sort of bio and always
submits a 1-page bio to the underlying device, and if my
understanding is correct, every device must be able to handle a
single page bio, no matter what the alignment (which is why raid0
has a problem - it doesn't).

2/ RAID1 works pretty well. The only improvement needed is to define
a merge_bvec_fn function which passes the question down to lower
layers. This should be easy except for the small fact that it is
impossible :-) There is no enforced pairing between calls to
merge_bvec_fn and submit_bh, so it is possible that a hot spare
with different restrictions could get swapped in between the one
and the other and could confuse things. I suspect that can be
worked around somehow though...

Someone sent me a patch that is sorely needed - it allows you
to simply call blk_queue_stack() (or somethink like that), and it will
get your stacked limits set appropriately.

3/ I just realised that raid0 is easier than I had previously
thought. We don't need the completely functional bio splitting
that dm has. We only need to be able to split a bio that has just
one page as the use of merge_bvec_fn will ensure that we never get
a larger bio that we cannot handle. And splitting a bio with only
one page is a lot easier. I now have code in my tree that
implements this quite cleanly and will probably post a patch
during the week.

o ideraid hasn't been ported to 2.5 at all yet.

We need to understand whether the proposed BIO split code will suffice
for this.

o CD burning. There are still a few quirks to solve wrt SG_IO and ide-cd.

Jens: The basic hang has been solved (double fault in ide-cd), there still
seems to be some cases that don't work too well. Don't really have a
handle on those :/

o lmb: Last time I looked at the multipath code (2.5.50 or so) it also
looked pretty broken; I plan to port forward the changes we did on 2.4
before KS.

o loop.c: fix http://bugzilla.kernel.org/show_bug.cgi?id=192

drivers/input/
~~~~~~~~~~~~~~

o rmk: unconverted keyboard/mouse drivers (there's a deadline of 2.6.0
currently on these remaining in my/Linus' tree.)

o viro: large absence of locking.

o synaptic touchpad support

Jens Taprogge <[email protected]> is working on this.

o andi: also the input keyboard stuff still has unusably obscure config
options for standard PC hardware.

o viro: parport is nearly as bad as that and there the code is more hairy.
IMO parport is more of "figure out what API changes are needed for its
users, get them done ASAP, then fix generic layer at leisure"

drivers/misc/
~~~~~~~~~~~~~

o rmk: UCB1[23]00 drivers, currently sitting in drivers/misc in the ARM
tree. (touchscreen, audio, gpio, type device.)

These need to be moved out of drivers/misc/ and into real places

o viro: actually, misc.c has a good chance to die. With cdev-cidr that's
trivial.

drivers/net/
~~~~~~~~~~~~

o rmk: network drivers. ARM people like to add tonnes of #ifdefs into
these to customise them to their hardware platform (eg, chip access
methods, addresses, etc.) I cope with this by not integrating them into my
tree. The result is that many ARM platforms can't be built from even my
tree without extra patches. This isn't sane, and has bred a culture of
network drivers not being submitted. I don't see this changing for 2.6
though.

drivers/net/irda/
~~~~~~~~~~~~~~~~~

o dongle drivers need to be converted to sir-dev

o irport need to be converted to sir-kthread

o new drivers (irtty-sir/smsc-ircc2/donauboe) need more testing

o rmk: Refuse IrDA initialisation if sizeof(structures) is incorrect (I'm
not sure if we still need this; I think gcc 2.95.3 on ARM shows this
problem though.)

drivers/pci/
~~~~~~~~~~~~

o alan: Some cardbus crashes the system

(bugzilla, please?)

o We have multiple drivers walking the pci device lists and also using
things like pci_find_device in unsafe ways with no refcounting. I think
we have to make pci_find_device etc refcount somewhere and add
pci_device_put as was done with networking.
http://bugzilla.kernel.org/show_bug.cgi?id=709

(gregkh will work on this)

o willy: PCI Domain support. The 'must-fix' bit of this is getting sysfs
to present the right interface to userspace so we can adapt pciutils & X to
use it.

drivers/pcmcia/
~~~~~~~~~~~~~~~

o alan: Most drivers crash the system on eject randomly with timer bugs. I
think after RMK's stuff is in most of the pcmcia/cardbus ones go except the
locking disaster.

(rmk, brodo: in progress)

drivers/pld/
~~~~~~~~~~~~

o rmk: EPXA (ARM platform) PLD hotswap drivers (drivers/pld)

(rmk: will work out what to do here. maybe drivers/arm/)

drivers/video/
~~~~~~~~~~~~~~

o Lots of drivers don't compile, others do but don't work.

drivers/scsi/
~~~~~~~~~~~~~

o hch: large parts of the locking are hosed or not existant

(Mike Anderson, Patrick Mansfield, Badari Pulavarty)

o shost->my_devices isn't locked down at all

o the host list ist locked but not refcounted, mess can happen when the
spinlock is dropped

o there are lots of members of struct Scsi_Host/scsi_device/scsi_cmnd
with very unclear locking, many of them probably want to become
atomic_t's or bitmaps (for the 1bit bitfields).

o there's lots of volatile abuse in the scsi code that needs to be
thought about.

o there's some global variables incremented without any locks

o Convert am53c974, dpt_i2o, initio and pci2220i to DMA-mapping

o Make inia100, cpqfc, pci2000 and dc390t compile

o Convert

wd33c99 based: a2091 a3000 gpv11 mvme174 sgiwd93 53c7xx based:
amiga7xxx bvme6000 mvme16x initio am53c974 pci2000 pci2220i qla1280
sym53c8xx dc390t

To new error handling

I think the sym53c8xx could probably be pulled out of the tree because
the sym_2 replaces it. I'm also looking at converting the qla1280.

It also might be possible to shift the 53c7xx based drivers over to
53c700 which does the new EH stuff, but I don't have the hardware to check
such a shift.

For the non-compiling stuff, I've probably missed a few that just aren't
compilable on my platforms, so any updates would be welcome. Also, are
some of our non-compiling or unconverted drivers obsolete?

o rmk: I have a pending todo: I need to put the scsi error handling through
a workout on my scsi bus from hell to make sure it does the right thing and
doesn't get wedged.

o qlogic drivers: merge qlogicisp, feral with a view to dropping qlogicfc
and qlogicisp

o jejb: and merge the qla2xxx too

fs/
~~~

o ext3 data=journal mode is bust. (fix is in progress)

o ext3/htree readdir can return "." and ".." in unexpected order, which
might break buggy userspace apps. Ted has a fix planned.


o AIO/direct-IO writes can race with truncate and wreck filesystems.

o Easy fix is to add an rwsem to the inode.

o hch: devfs: there's a fundamental lookup vs devfsd race that's only
fixable by introducing a lookup vs devfs deadlock. I can't see how this is
fixable without getting rid of the current devfsd design. Mandrake seems
to have a workaround for this so this is at least not triggered so easily,
but that's not what I'd consider a fix..

o viro: fs/char_dev.c needs removal of aeb stuff and merge of cdev-cidr.
In progress.

o forward-port sct's O_DIRECT fixes

o viro: there is some generic stuff for namei/namespace/super, but that's a
slow-merge and can go in 2.6 just fine

o andi: also soft needs to be fixed - there are quite a lot of
uninterruptible waits in sunrpc/nfs

o trond: NFS has a mmap-versus-truncate problem

kernel/sched.c
~~~~~~~~~~~~~~

o "Persistent starvation"

http://www.hpl.hp.com/research/linux/kernel/o1-starve.php

ingo: "basically by calling sleep(1) in an infinite loop you can end up
expiring yourself. The testcode (test-starve.c) triggers this. This is
solved by going to sub-timeslices. Which i've got done a few weeks ago and
it has seen some testing by others as well.

o Overeager affinity in presence of repeated yields

http://www.hpl.hp.com/research/linux/kernel/o1-openmp.php

ingo: this is valid. fix is in progress.

o The "thud.c" test app. This is a exploit for the interactivity
estimator. it's unlikely to bite in real-world cases. Needs watching.
Can be ameliorated by setting nice values.

o generic interactivity problems need watching. We've closed down a number
of items recently without introducing new ones, so i'm confident this is
heading in the right direction.

kernel/
~~~~~~~

o Alan: 32bit uid support is *still* broken for process accounting.

Create a 32bit uid, turn accounting on. Shock horror it doesn't work
because the field is 16bit. We need an acct structure flag day for 2.6
IMHO

(alan has patch)

o nasty task refcounting bug is taking ages to track down. (bugzilla ref?)

o viro: core sysctl code is racy. And its interaction wiuth sysfs

o gettimeofday goes backwards. Merge up David M-T's fixes?

o Daniel Jacobowitz <[email protected]>: when CLONE_DETACHED threads were
removed from /proc several approaches were suggested to let procps find out
about them and none of them were implemented. There's some real potential
for badness with these mostly-invisible processes. Something needs to be
added so that we can display and detect them.

lib/kobject.c
~~~~~~~~~~~~~

o kobject refcounting (comments from Al Viro):

_anything_ can grab a temporary reference to kobject. IOW, if kobject is
embedded into something that could be freed - it _MUST_ have a destructor
and that destructor _MUST_ be the destructor for containing object.

Any violation of the above (and we already have a bunch of those) is a
user-triggerable memory corruption.

We can tolerate it for a while in 2.5 (e.g. during work on susbsystem we
can decide to switch to that way of handling objects and have subsystem
vulnerable for a while), but all such windows must be closed before 2.6 and
during 2.6 we can't open them at all.

mm/
~~~

o Overcommit accounting gets wrong answers

o gets confused by reclaimable-but-not-freed truncated ext3 pages.

o GFP_DMA32 (or something like that). Lots of ideas. jejb, zaitcev,
willy, arjan, wli.

o access_process_vm() doesn't flush right. We probably need new flushing
primitives to do this (davem?)


modules
~~~~~~~

(Rusty)

o The .modinfo patch needs to go in. It's trivial, but it's the major
missing functionality vs. 2.4. Keeps bouncing off Linus.

o __module_get(): "I know I have a refcount already and I don't care
if they're doing rmmod --wait, gimme.". Keeps bouncing off Linus.

o Per-cpu support inside modules (have patch, in testing).

o shemminger: The module remove rework that Rusty and Dave are working on
needs to be fixed before 2.6. Right now, it is impossible to write a
protocol or network device that can be safely unloaded when it is a module.

See:
http://www.osdl.org/archive/shemminger/modules.html

(This is "two stage unload")

net/
~~~~

(davem)

o UDP apps can in theory deadlock, because the ip_append_data path can end
up sleeping while the socket lock is held.

It is OK to sleep with the socket held held, normally. But in this case
the sleep happens while waiting for socket memory/space to become
available, if another context needs to take the socket lock to free up the
space we could hang.

I sent a rough patch on how to fix this to Alexey, and he is analyzing
the situation. I expect a final fix from him next week or so.

o Semantics for IPSEC during operations such as TCP connect suck currently.

When we first try to connect to a destination, we may need to ask the
IPSEC key management daemon to resolve the IPSEC routes for us. For the
purposes of what the kernel needs to do, you can think of it like ARP. We
can't send the packet out properly until we resolve the path.

What happens now for IPSEC is basically this:

O_NONBLOCK: returns -EAGAIN over and over until route is resolved

!O_NONBLOCK: Sleeps until route is resolved

These semantics are total crap. The solution, which Alexey is working
on, is to allow incomplete routes to exist. These "incomplete" routes
merely put the packet onto a "resolution queue", and once the key manager
does it's thing we finish the output of the packet. This is precisely how
ARP works.

I don't know when Alexey will be done with this.

o There are those mysterious TCP hangs of established state sockets.
Someone has to get a good log in order for us to effectively debug this.

net/*/netfilter/
~~~~~~~~~~~~~~~~

(Rusty)

o Handle non-linear skbs everywhere. This is going in via Dave now.

o Rework conntrack hashing.

o Module relationship bogosity fix (trivial, have patch).

sound/
~~~~~~

o rmk: several OSS drivers for SA11xx-based hardware in need of
ALSA-ification and L3 bus support code for these.

o rmk: linux/sound/drivers/mpu401/mpu401.c and
linux/sound/drivers/virmidi.c complained about 'errno' at some time in the
past, need to confirm whether this is still a problem.

o rmk: need to complete ALSA-ification of the WaveArtist driver for both
NetWinder and other stuff (there's some fairly fundamental differences in
the way the mixer needs to be handled for the NetWinder.)


(Issues with forward-porting 2.4 bugfixes.)
(Killing off OSS is 2.7 material)


global
~~~~~~

o Lots of 2.4 fixes including some security are not in 2.5

o HZ=1000 caused lots of lost timer interrupts. ACPI or SMM. (andi,
jstultz, arjan)

o There are about 60 or 70 security related checks that need doing
(copy_user etc) from Stanford tools. (badari is looking into this, and
hollisb)

o A couple of hundred real looking bugzilla bugs

o viro: cdev rework. Main group is pretty stable and I hope to feed it to
Linus RSN. That's cdev-cidr and ->i_cdev/->i_cindex stuff




2003-05-31 03:15:12

by William Lee Irwin III

[permalink] [raw]
Subject: Re: must-fix, version 6

On Fri, May 30, 2003 at 04:37:20PM -0700, Andrew Morton wrote:
> o GFP_DMA32 (or something like that). Lots of ideas. jejb, zaitcev,
> willy, arjan, wli.

Specifically, 64-bit systems need to be able to enforce 32-bit
addressing limits for device metadata like network cards' ring buffers
and SCSI command descriptors.

This is very distinct from 32-bit lowmem exhaustion issues; on 32-bit
the command descriptors and ring buffers are O(devices) and hence not
a lowmem pressure issue (well, the O(devices) bit could be but we might
as well say we don't support it and no one has asked for it to be
handled). Actual data having io done to it is unrelated to this.


-- wli

2003-05-31 09:33:55

by Christoph Hellwig

[permalink] [raw]
Subject: Re: must-fix, version 6

On Fri, May 30, 2003 at 04:37:20PM -0700, Andrew Morton wrote:
> + PRI3
> +
> o 32bit quota needs a lot more testing but may work now
>
> + PRI2

I think we can remove this. It's shipped with the vendor trees, -ac
and the XFS tree for a while now and got a fair amount of testing.

> o IDE requires bio walking
>
> "Bartlomiej has IDE multisector working" (does that mean it's fixed?)
>
> + PRI1

bio walking seems to be in now.

> +o new zfcp fibre channel driver
> +
> + PRI3

Is there a 2.5 patch of this somewhere?

2003-05-31 15:45:39

by Arnd Bergmann

[permalink] [raw]
Subject: Re: must-fix, version 6

Christoph Hellwig wrote:

> On Fri, May 30, 2003 at 04:37:20PM -0700, Andrew Morton wrote:

>> o Minor interface changes are pending in cio/ when the z990 machines are
>> out.

This should be at least PRI2, as it will change the user-visible interface,
e.g. the bus_id format for CCW devices.

>> There are some more things being worked on that are either post-2.6.0 or
>> are likely to remain outside of the official kernel (i.e. not for your
>> list):
>>
>> PRI3

The 'not for your list' actually referred to the items below...

>> o Jan Glauber is working on a fix for the timer issues related to running
>> on virtualized CPUs (wall-clock vs. cpu time).
>>
>> PRI1

Keep this as PRI3. It would be really nice to have, but I don't expect
it to be stable before 2.6.0. The current code is the same as in 2.4.

>> o the qeth driver will become GPL soon
>>
>> PRI3
>>
>> o a block device driver for ramdisks shared among virtual machines
>>
>> PRI3

Ok. We're only waiting for the OK from our lawyers before these
can be submitted. The qeth driver is ugly, but is the one for
the most common hardware, so it would be nice to have it included
eventually.

>>
>> o driver for crypto hardware
>>
>> PRI2
>>
>> o 'claw' network device driver
>>
>> PRI3
>>
>> o new zfcp fibre channel driver
>>
>> PRI3

These three drivers are not fully ported to 2.5 as they are less
important and we have no plan of submitting them for either 2.4
or 2.[567]. The code is also too ugly to be accepted anyway, so you
can drop them from the list. Vendors can pick up the source
from out developerworks site, as always...

> Is there a 2.5 patch of this somewhere?
No. An official release of the zfcp driver should be available at
the end of this year for linux-2.6. If you want to look at the
code, Heiko Carstens <[email protected]> can probably
send you a snapshot of his current working version.

Arnd <><

2003-05-31 16:11:20

by Ingo Oeser

[permalink] [raw]
Subject: Re: must-fix, version 6

Hi Andrew,
hi Al,
hi LKML,

On Fri, May 30, 2003 at 04:37:20PM -0700, Andrew Morton wrote:
> +lib/kobject.c
> +~~~~~~~~~~~~~
> +
> +o kobject refcounting (comments from Al Viro):
> +
> + _anything_ can grab a temporary reference to kobject. IOW, if kobject is
> + embedded into something that could be freed - it _MUST_ have a destructor
> + and that destructor _MUST_ be the destructor for containing object.
> +
> + Any violation of the above (and we already have a bunch of those) is a
> + user-triggerable memory corruption.
> +
> + We can tolerate it for a while in 2.5 (e.g. during work on susbsystem we
> + can decide to switch to that way of handling objects and have subsystem
> + vulnerable for a while), but all such windows must be closed before 2.6 and
> + during 2.6 we can't open them at all.
> +

I think a simple variant of this is to extend the kobject
structure with a backlink void* to the container of this object.

That way:
- we can use kfree() as the default destructor
- use special destructors and pass the right object
- warn and do the most useful thing (a kfree), until all
users are converted to either pass kfree or sth. else as an
constructor.

This solution becomes non-sense, if we T <<< I, where T is the
amount of unique kobject type and I the sum of instances of all
types.

But even then, there can be a transitional model like the above,
but with a "default destructor" needing the container and the
kobject passed, which can later be reduced trivially to just have
the kobject passed and the container being caclulated with
"containerof" in the destructor with knowlegde about the type
where the kobject is in.

The latter solution needs lots of code to be written and is a
huge janitorial work. The backlink solution is much simpler, but
memory consuming.

Comments appreciated.

Regards

Ingo Oeser

2003-05-31 19:18:08

by Christoph Hellwig

[permalink] [raw]
Subject: Re: must-fix, version 6

On Sat, May 31, 2003 at 05:26:12PM +0200, Arnd Bergmann wrote:
> >> o new zfcp fibre channel driver
> >>
> >> PRI3
>
> These three drivers are not fully ported to 2.5 as they are less
> important and we have no plan of submitting them for either 2.4
> or 2.[567]. The code is also too ugly to be accepted anyway, so you
> can drop them from the list. Vendors can pick up the source
> from out developerworks site, as always...

So instead of fixing the code you rely on vendors merging crap?
what a wonderful idea, let's hope RH, SuSE & co reject it...

2003-05-31 19:22:56

by Eric Benson

[permalink] [raw]
Subject: Re: must-fix, version 6

A fully-functioning ide-scsi is extremely important. The application
cdrecord and its derivatives require a SCSI interface to work. I don't
know of any other CD-R/RW recording application in common use on Linux.
Many distributions add ide-scsi boot parameters when they discover an
IDE CD-R drive. Even though the newest development versions of cdrecord
along with the newest versions of kernel 2.5 do not require ide-scsi,
the installed base is enormous. This will represent a significant
barrier to widespread adoption of 2.6 if currently installed versions of
cdrecord with currently configured boot parameters must be updated.

2003-05-31 23:08:51

by Arnd Bergmann

[permalink] [raw]
Subject: Re: must-fix, version 6

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 31 May 2003 21:31, Christoph Hellwig wrote:

> So instead of fixing the code you rely on vendors merging crap?
> what a wonderful idea, let's hope RH, SuSE & co reject it...

Exactly. Only speaking for myself: I don't have access to the hardware,
the code is outside of the official tree and the maintainers don't
care if it ever gets merged. To me, that's not much different from all
the other obscure stuff in the vendor kernels.

Note that all the /important/ s390 drivers are in 2.5 (except qeth and
the new 3270 -- both are being worked on) and in reasonably good
shape now, mostly due to the fact that Martin rewrote 80% of the code
during the last year.

For zfcp, the chances of getting it cleaned up during 2.7 are good and
the code for 2.6 will already be much better than what we have now.
For claw and z90crypt we still need a volunteer who will either take
over maintainership or teach the current maintainers.

Arnd <><
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE+2TFw5t5GS2LDRf4RAgNDAJ9kK++TCYCn6kaRM1w1ZlyrT98swQCffO4t
4eGUYyCX8bYnuQCE+iHsUqg=
=oWCi
-----END PGP SIGNATURE-----

2003-05-31 23:56:40

by Daniel Phillips

[permalink] [raw]
Subject: Re: must-fix, version 6

On Saturday 31 May 2003 01:37, Andrew Morton wrote:
> device mapper
> ~~~~~~~~~~~~~
>
> o ioctl interface cleanup patch is ready (redo the structure layouts)
>
> + PRI1
> +
> o A port of the 2.4 snapshot target is in progress

o A port of the 2.4 snapshot and mirror targets is in progress

Regards,

Daniel