2002-10-25 05:34:30

by Andrew Morton

[permalink] [raw]
Subject: 2.5.44-mm5


url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.44/2.5.44-mm5/

We seem to have found the dud patch. Things should be a little
more stable...

The CONFIG_PREEMPT+SMP problem I was having went away when gcc-2.95.3
was used in place of 2.91.66. Which is a bit of a problem because
_someone_ has to keep an eye on 2.91.66 compatibility as long as it
continues to be required for sparc builds.

There is a memory corruption problem happening deep in the scsi code
when the hot-n-cold pages patch is present. Current theory is that
this is a use-after-free problem in scsi, and that the hot-n-cold
pages patch is exposing it because its LIFO page queue will hand
out a page immediately after this cpu freed it.

The workaround is a patch -R of

page-reservation.patch
pagevec-hot-cold-hint.patch
readahead-cold-pages.patch
hot_cold_pages.patch

in that order.



Added since 2.5.44-mm4:

+generic-nonlinear-mappings-D0.patch

Non-file-contiguous mmap support. Back in.

+md-01-driverfs-core.patch
+md-02-driverfs-topology.patch
+md-03-numa-meminfo.patch
+md-04-memblk_online_map.patch
+md-05-node_online_map.patch

Export lots of NUMA info via driverfs

+ipclock-2544mm4.patch

Reduced lock contention in the IPC code

+ipc-akpm.patch

Reduced inlines inthe IPC code

+kstat.patch

Use the per_cpu infrastructure for what remains of the old kernel_stat
accounting.

+kstat-arch.patch

Teach the non-ia32 architectures about the new kernel_stat regime.



All 107 patches:

rcu-idle-fix.patch
RCU idle detection fix

read-barrier-depends.patch
read_barrier_depends fix

deferred-lru-add-fix.patch
fix for adding mapped pages to the active list

ide-warnings.patch
Fix some IDE compile warnings

dmi-warning.patch
fix a compile warning in dmi_scan.c

scsi-reboot-fix.patch

kgdb.patch

misc.patch
misc fixes

ramfs-aops.patch
Move ramfs address_space ops into libfs

ramfs-prepare-write-speedup.patch
correctness fixes in libfs address_space ops

pipe-fix.patch
use correct wakeups in fs/pipe.c

dio-submit-fix.patch
rework direct-io for bio_add_page

dio-fine-alignment.patch
Allow O_DIRECT to use 512-byte alignment

file_ra_state_init.patch
Add a function to initialise file readahead state

less-unlikelies.patch
reduced buslocked traffic in the page allocator

running-iowait.patch
expose nr_running and nr_iowait task counts in /proc

intel-user-copy-taka.patch
Faster copy_*_user for Intel ia32 CPUs

shrink_slab-overflow.patch

uaccess-uninline.patch

ingo-oom-kill.patch
oom-killer changes for threaded apps

unbloat-pid.patch
Reduce RAM use in kernel/pid.c

per-cpu-ratelimits.patch

per-cpu-warning.patch
Fix per-cpu compile warnings on UP

cpuup-notifiers.patch
extended cpu hotplug notifiers

per-cpu-02-rcu.patch
cpu_possible rcu per_cpu data

per-cpu-03-timer.patch
cpu_possible timer percpu data

per-cpu-04-tasklet.patch
cpu_possible tasklet percpu data

per-cpu-05-bh.patch
cpu_possible bh_accounting

export-per-cpu-symbol.patch
create EXPORT_PER_CPU_SYMBOL

per-cpu-page_state.patch

add_timer_on.patch
add_timer_on(): function to start a timer on a particular CPU

slab-split-01-rename.patch
slab cleanup: rename static functions

slab-split-02-SMP.patch
slab: enable the cpu arrays on uniprocessor

slab-split-03-tail.patch
slab: reduced internal fragmentation

slab-split-04-drain.patch
slab: take the spinlock in the drain function.

slab-split-05-name.patch
slab: remove spaces from /proc identifiers

slab-split-06-mand-cpuarray.patch
slab: cleanups and speedups

slab-split-07-inline.patch
slab: uninline poisoning checks

slab-split-08-reap.patch
slab: reap timers

slab-timer.patch

slab-use-sem.patch

slab-cleanup.patch
Slab cleanup

slab-per-cpu.patch
Use CPU notifiers in slab

ingo-mmap-speedup.patch
Ingo's mmap speedup

mm-inlines.patch
remove some inlines from mm/*

o_streaming.patch
O_STREAMING support

shmem_getpage-unlock_page.patch
tmpfs 1/9 shmem_getpage unlock_page

shmem_getpage-beyond-eof.patch
tmpfs 2/9 shmem_getpage beyond eof

shmem_getpage-reading-holes.patch
tmpfs 3/9 shmem_getpage reading holes

shmem-fs-cleanup.patch
tmpfs 4/9 shmem fs cleanup

shmem_file_sendfile.patch
tmpfs 5/9 shmem_file_sendfile

shmem_file_write-update.patch
tmpfs 6/9 shmem_file_write update

shmem_getpage-flush_dcache.patch
tmpfs 7/9 shmem_getpage flush_dcache

loopable-tmpfs.patch
tmpfs 8/9 loopable tmpfs

event-II.patch
f_version/i_version cleanups

event-ext2.patch
f_version/i_version cleanups: ext2

mod_timer-race.patch

blkdev-o_direct-short-read.patch
Fix O_DIRECT blockdev reads at end-of-device

orlov-allocator.patch

blk-queue-bounce.patch
inline blk_queue_bounce

lseek-ext2_readdir.patch
remove lock_kernel() from ext2_readdir()

decoded-wchan-output.patch
pre-decoded wchan output in /proc/pid/wchan

write-deadlock.patch
Fix the generic_file_write-from-same-mmapped-page deadlock

rd-cleanup.patch
Cleanup and fix the ramdisk driver (doesn't work right yet)

hugetlb-prefault.patch
hugetlbpages: factor out some code for hugetlbfs

hugetlb-header-split.patch
Move hugetlb declarations into their own header

htlb-update.patch
hugetlb fixes and cleanups

hugetlb-page-count.patch
fix hugetlb thinko

hugetlbfs.patch
hugetlbfs file system

hugetlb-shm.patch
hugetlbfs backing for SYSV shared memory

truncate-bkl.patch
don't take the BKL in inode_setattr

akpm-deadline.patch
deadline scheduler tweaks

pipe-speedup.patch
user faster wakeups in the pipe code

dcache_rcu.patch
Use RCU for dcache

mpopulate.patch
remap_file_pages

shmem_populate.patch
tmpfs 9/9 Ingo's shmem_populate

ext23-acl-xattr-01.patch

ext23-acl-xattr-02.patch

ext23-acl-xattr-03.patch

ext23-acl-xattr-04.patch

ext23-acl-xattr-05.patch

ext23-acl-xattr-06.patch

ext23-acl-xattr-07.patch

ext23-acl-xattr-08.patch

ext23-acl-xattr-09.patch

ext23-acl-xattr-10.patch

ext23-acl-xattr-11.patch

ext2-mount-fix.patch

acl-xattr-on.patch
turn on posix acls and extended attributes

rmqueue_bulk.patch
bulk page allocator

free_pages_bulk.patch
Bulk page freeing function

hot_cold_pages.patch
Hot/Cold pages and zone->lock amortisation

readahead-cold-pages.patch
Use cache-cold pages for pagecache reads.

pagevec-hot-cold-hint.patch
hot/cold hints for truncate and page reclaim

page-reservation.patch
Page reservation API

wli-show_free_areas.patch
show_free_areas extensions

shpte-ng.patch

generic-nonlinear-mappings-D0.patch
generic nonlinear mappings

md-01-driverfs-core.patch
Core driverfs Topology

md-02-driverfs-topology.patch
i386 driverfs Topology

md-03-numa-meminfo.patch
NUMA meminfo for driverfs Topology

md-04-memblk_online_map.patch
create memblk_online_map

md-05-node_online_map.patch
create node_online_map

ipclock-2544mm4.patch
IPC lock contention reduction

ipc-akpm.patch

kstat.patch

kstat-arch.patch


2002-10-25 11:54:57

by Con Kolivas

[permalink] [raw]
Subject: Re: 2.5.44-mm5


Compile failure (gcc3.2):

gcc -Wp,-MD,init/.version.o.d -D__KERNEL__ -Iinclude -Wall -Wstrict-prototypes
-Wno-trigraphs -O2 -fno-strict-aliasing -fno-common -pipe
-mpreferred-stack-boundary=2 -march=i686 -Iarch/i386/mach-generic
-fomit-frame-pointer -nostdinc -iwithprefix include -DKBUILD_BASENAME=version
-c -o init/version.o init/version.c
ld -m elf_i386 -r -o init/built-in.o init/main.o init/version.o init/do_mounts.o
ld -m elf_i386 -e stext -T arch/i386/vmlinux.lds.s
arch/i386/kernel/head.o arch/i386/kernel/init_task.o init/built-in.o
--start-group arch/i386/kernel/built-in.o arch/i386/mm/built-in.o
arch/i386/mach-generic/built-in.o kernel/built-in.o mm/built-in.o
fs/built-in.o ipc/built-in.o security/built-in.o lib/lib.a
arch/i386/lib/lib.a drivers/built-in.o sound/built-in.o
arch/i386/pci/built-in.o arch/i386/oprofile/built-in.o net/built-in.o
--end-group -o .tmp_vmlinux1
kernel/built-in.o: In function `sched_init':
kernel/built-in.o(.init.text+0xc4): undefined reference to `init_kstat'
make: *** [.tmp_vmlinux1] Error 1

2002-10-25 12:27:17

by Dipankar Sarma

[permalink] [raw]
Subject: Re: 2.5.44-mm5

On Fri, Oct 25, 2002 at 12:04:22PM +0000, Con Kolivas wrote:
>
> Compile failure (gcc3.2):
>
> gcc -Wp,-MD,init/.version.o.d -D__KERNEL__ -Iinclude -Wall -Wstrict-prototypes
> -Wno-trigraphs -O2 -fno-strict-aliasing -fno-common -pipe
> -mpreferred-stack-boundary=2 -march=i686 -Iarch/i386/mach-generic
> -fomit-frame-pointer -nostdinc -iwithprefix include -DKBUILD_BASENAME=version
> -c -o init/version.o init/version.c
> ld -m elf_i386 -r -o init/built-in.o init/main.o init/version.o init/do_mounts.o
> ld -m elf_i386 -e stext -T arch/i386/vmlinux.lds.s
> arch/i386/kernel/head.o arch/i386/kernel/init_task.o init/built-in.o
> --start-group arch/i386/kernel/built-in.o arch/i386/mm/built-in.o
> arch/i386/mach-generic/built-in.o kernel/built-in.o mm/built-in.o
> fs/built-in.o ipc/built-in.o security/built-in.o lib/lib.a
> arch/i386/lib/lib.a drivers/built-in.o sound/built-in.o
> arch/i386/pci/built-in.o arch/i386/oprofile/built-in.o net/built-in.o
> --end-group -o .tmp_vmlinux1
> kernel/built-in.o: In function `sched_init':
> kernel/built-in.o(.init.text+0xc4): undefined reference to `init_kstat'
> make: *** [.tmp_vmlinux1] Error 1

The patch below should fix your problem, hopefully. Although I
don't understand why kstat initialization isn't in common code.
I will try to fix it the right way later.

Thanks
--
Dipankar Sarma <[email protected]> http://lse.sourceforge.net
Linux Technology Center, IBM Software Lab, Bangalore, India.



diff -urN linux-2.5.44-mm5/kernel/sched.c linux-2.5.44-mm5-fix/kernel/sched.c
--- linux-2.5.44-mm5/kernel/sched.c Fri Oct 25 15:11:06 2002
+++ linux-2.5.44-mm5-fix/kernel/sched.c Fri Oct 25 15:40:01 2002
@@ -2160,6 +2160,8 @@
* Don't use in new code.
*/
spinlock_t kernel_flag __cacheline_aligned_in_smp = SPIN_LOCK_UNLOCKED;
+#else
+static inline void init_kstat(void) { }
#endif

void __init sched_init(void)

2002-10-25 18:27:17

by Daniel Phillips

[permalink] [raw]
Subject: Re: 2.5.44-mm5

On Friday 25 October 2002 07:40, Andrew Morton wrote:
> url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.44/2.5.44-mm5/
>
> We seem to have found the dud patch. Things should be a little
> more stable...
>
> The CONFIG_PREEMPT+SMP problem I was having went away when gcc-2.95.3
> was used in place of 2.91.66. Which is a bit of a problem because
> _someone_ has to keep an eye on 2.91.66 compatibility as long as it
> continues to be required for sparc builds.

Didn't davem say something about being ready to move to a more recent
compiler, or does my memory not serve correctly?

--
Daniel

2002-10-25 19:03:30

by Paul Larson

[permalink] [raw]
Subject: Re: 2.5.44-mm5 - ltp-cvs (current) - 98.74% Pass

2.5.44-mm5 does not have any new or unexpected LTP failures above
2.5.44-vanilla. These results are the same for both.

These results are on an 8-way PIII-700 16 GB. Preempt off, PAE on,
HUGETLB on, Shared PTE on

tag=nanosleep02 stime=1035560360 dur=1 exit=exited stat=1 core=no cu=0
cs=0
tag=personality02 stime=1035560367 dur=0 exit=exited stat=1 core=no cu=0
cs=0
tag=pread02 stime=1035560372 dur=0 exit=exited stat=1 core=no cu=1 cs=1
tag=pwrite02 stime=1035560372 dur=0 exit=exited stat=1 core=no cu=0 cs=1
tag=writev01 stime=1035560429 dur=0 exit=exited stat=1 core=no cu=0 cs=1
tag=dio04 stime=1035561857 dur=0 exit=exited stat=1 core=no cu=0 cs=0
tag=dio10 stime=1035561870 dur=400 exit=exited stat=1 core=no cu=124
cs=465
tag=sem02 stime=1035562935 dur=20 exit=exited stat=1 core=no cu=0 cs=0

---test_start---
tag=nanosleep02 stime=1035525897
cmdline="nanosleep02"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
nanosleep02 1 FAIL : Remaining sleep time 4001000 usec doesn't
match with the expected 3998337 usec time
nanosleep02 1 FAIL : child process exited abnormally
---execution_status---
duration=1 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=1
---test_end---
---test_start---
tag=personality02 stime=1035525903
cmdline="personality02"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
personality02 1 FAIL : call failed - errno = 0 - Success
---execution_status---
duration=0 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=0
---test_end---
---test_start---
tag=pread02 stime=1035525908
cmdline="pread02"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
pread02 1 PASS : pread() fails, file descriptor is a PIPE or
FIFO, errno:29
pread02 2 FAIL : pread() returned 0, expected -1, errno:22
---execution_status---
duration=0 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=0
---test_end---
---test_start---
tag=pwrite02 stime=1035525908
cmdline="pwrite02"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
pwrite02 1 PASS : file descriptor is a PIPE or FIFO, errno:29
caught SIGXFSZ
pwrite02 2 FAIL : specified offset is -ve or invalid, unexpected
errno:27, expected:22
pwrite02 3 PASS : file descriptor is bad, errno:9
---execution_status---
duration=0 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=0
---test_end---
---test_start---
tag=writev01 stime=1035525965
cmdline="writev01"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
writev01 0 INFO : Enter Block 1
writev01 0 INFO : Received EINVAL as expected
writev01 0 INFO : block 1 PASSED
writev01 0 INFO : Exit block 1
writev01 0 INFO : Enter block 2
writev01 1 FAIL : writev() failed unexpectedly
writev01 0 INFO : block 2 FAILED
writev01 0 INFO : Exit block 2
writev01 0 INFO : Enter block 3
writev01 0 INFO : block 3 PASSED
writev01 0 INFO : Exit block 3
writev01 0 INFO : Enter block 4
writev01 0 INFO : Received EBADF as expected
writev01 0 INFO : block 4 PASSED
writev01 0 INFO : Exit block 4
writev01 0 INFO : Enter block 5
writev01 0 INFO : Received EINVAL as expected
writev01 0 INFO : block 5 PASSED
writev01 0 INFO : Exit block 5
writev01 0 INFO : Enter block 6
writev01 2 PASS : writev() wrote 0 iovectors
writev01 0 INFO : block 6 PASSED
writev01 0 INFO : Exit block 6
writev01 0 INFO : Enter block 7
writev01 3 PASS : writev passed writing 64 bytes, followed by two
NULL vectors
writev01 0 INFO : block 7 PASSED
writev01 0 INFO : Exit block 7
writev01 0 INFO : Enter block 8
writev01 0 INFO : Received EPIPE as expected
writev01 0 INFO : block 8 PASSED
writev01 0 INFO : Exit block 8
---execution_status---
duration=0 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=0
---test_end---
---test_start---
tag=dio04 stime=1035527610
cmdline="diotest4"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
[9] open /dev/null:Invalid argument
---execution_status---
duration=1 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=38
---test_end---
---test_start---
tag=dio10 stime=1035527680
cmdline="diotest4 -b 65536"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
[9] open /dev/null:Invalid argument
---execution_status---
duration=1444 termination_type=exited termination_id=1 corefile=no
cutime=127 cstime=118842
---test_end---
---test_start---
tag=sem02 stime=1035530115
cmdline="sem02"
contacts=""
analysis=exit
initiation_status="ok"
---test_output---
Waiter, pid = 5116
Poster, pid = 5117, posting
Poster posted
Poster exiting
Waiter waiting, pid = 5116
sem02: FAIL
Waiter done waiting
---execution_status---
duration=20 termination_type=exited termination_id=1 corefile=no
cutime=0 cstime=1
---test_end---

Thanks,
Paul Larson

2002-10-28 08:53:03

by Ravikiran G Thirumalai

[permalink] [raw]
Subject: Re: 2.5.44-mm5

On Fri, Oct 25, 2002 at 12:36:45PM +0000, Dipankar Sarma wrote:
> ...
> The patch below should fix your problem, hopefully. Although I
> don't understand why kstat initialization isn't in common code.

My mistake...it should be in common code. I missed the #endif after
kernel_flag definition....and was dumb enough to think sched_init
was defined only for CONFIG_SMP || CONFIG_PREEMPT....sheesh..
Here's the fix...moves init_kstat to common code; applies on mm6

Thanks,
Kiran


--- mm6.orig/kernel/sched.c Mon Oct 28 11:36:57 2002
+++ mm6.fix/kernel/sched.c Mon Oct 28 11:47:39 2002
@@ -2117,6 +2117,18 @@
#endif

#if CONFIG_SMP || CONFIG_PREEMPT
+/*
+ * The 'big kernel lock'
+ *
+ * This spinlock is taken and released recursively by lock_kernel()
+ * and unlock_kernel(). It is transparently dropped and reaquired
+ * over schedule(). It is used to protect legacy code that hasn't
+ * been migrated to a proper locking design yet.
+ *
+ * Don't use in new code.
+ */
+spinlock_t kernel_flag __cacheline_aligned_in_smp = SPIN_LOCK_UNLOCKED;
+#endif

static void kstat_init_cpu(int cpu)
{
@@ -2149,21 +2161,6 @@
register_cpu_notifier(&kstat_nb);
}

-/*
- * The 'big kernel lock'
- *
- * This spinlock is taken and released recursively by lock_kernel()
- * and unlock_kernel(). It is transparently dropped and reaquired
- * over schedule(). It is used to protect legacy code that hasn't
- * been migrated to a proper locking design yet.
- *
- * Don't use in new code.
- */
-spinlock_t kernel_flag __cacheline_aligned_in_smp = SPIN_LOCK_UNLOCKED;
-#else
-static inline void init_kstat(void) { }
-#endif
-
void __init sched_init(void)
{
runqueue_t *rq;