Here is a listing of LTP results for the linux kernel. For the 2.6.x
series LTP results have been pretty constant, but they've gotten
interesting lately:
Patch Name TestReq# CPU PASS FAIL WARN BROK RunTime
----------------------------------------------------------------------
patch-2.4.27-rc2 294321 2-way 7226 6 3 6 69.0
linux-2.6.7 294027 2-way 7225 6 3 6 46.0
linux-2.6.7 294004 1-way 7225 6 3 6 42.2
patch-2.6.7-bk1 294069 2-way 7224 7 3 6 45.9
patch-2.6.7-bk2 294081 2-way 7224 7 3 6 45.9
patch-2.6.7-bk3 294103 2-way 7224 7 3 6 46.4
patch-2.6.7-bk4 294165 2-way 7187 7 3 6 48.7
patch-2.6.7-bk5 294181 2-way 7181 7 3 6 45.5
patch-2.6.7-bk6 294204 2-way 7224 7 3 6 47.1
patch-2.6.7-bk7 294228 2-way 7224 7 3 6 49.0
patch-2.6.7-bk8 294304 2-way 7223 10 3 7 47.5
patch-2.6.7-bk9 294333 2-way 7224 7 3 6 46.1
patch-2.6.7-bk10 294403 2-way 7223 10 3 7 42.9
patch-2.6.7-bk11 294423 2-way 7178 46 3 6 47.8
2.6.7-mm1 294146 2-way 7185 46 3 6 59.1
2.6.7-mm1 294126 1-way 7185 46 3 6 52.9
2.6.7-mm2 294271 2-way 7181 47 3 6 44.9
2.6.7-mm3 294363 1-way 7185 46 3 6 41.0
2.6.7-rc3-mm2 293949 2-way 7223 8 3 6 46.5
We usually always see 6-7 fails on the 2.6.x kernels, so the increase is
unusual.
I've generated some detailed LTP test result reports on a few of the
above runs, with specifics about the test runs and failures. These are
available here:
http://developer.osdl.org/bryce/ltp/
HTH,
Bryce
Bryce Harrington <[email protected]> wrote:
>
> Here is a listing of LTP results for the linux kernel. For the 2.6.x
> series LTP results have been pretty constant, but they've gotten
> interesting lately:
>
> Patch Name TestReq# CPU PASS FAIL WARN BROK RunTime
> ----------------------------------------------------------------------
> patch-2.4.27-rc2 294321 2-way 7226 6 3 6 69.0
> linux-2.6.7 294027 2-way 7225 6 3 6 46.0
> linux-2.6.7 294004 1-way 7225 6 3 6 42.2
> patch-2.6.7-bk1 294069 2-way 7224 7 3 6 45.9
> patch-2.6.7-bk2 294081 2-way 7224 7 3 6 45.9
> patch-2.6.7-bk3 294103 2-way 7224 7 3 6 46.4
> patch-2.6.7-bk4 294165 2-way 7187 7 3 6 48.7
> patch-2.6.7-bk5 294181 2-way 7181 7 3 6 45.5
> patch-2.6.7-bk6 294204 2-way 7224 7 3 6 47.1
> patch-2.6.7-bk7 294228 2-way 7224 7 3 6 49.0
> patch-2.6.7-bk8 294304 2-way 7223 10 3 7 47.5
> patch-2.6.7-bk9 294333 2-way 7224 7 3 6 46.1
> patch-2.6.7-bk10 294403 2-way 7223 10 3 7 42.9
> patch-2.6.7-bk11 294423 2-way 7178 46 3 6 47.8
> 2.6.7-mm1 294146 2-way 7185 46 3 6 59.1
> 2.6.7-mm1 294126 1-way 7185 46 3 6 52.9
> 2.6.7-mm2 294271 2-way 7181 47 3 6 44.9
> 2.6.7-mm3 294363 1-way 7185 46 3 6 41.0
> 2.6.7-rc3-mm2 293949 2-way 7223 8 3 6 46.5
>
> We usually always see 6-7 fails on the 2.6.x kernels, so the increase is
> unusual.
>
> I've generated some detailed LTP test result reports on a few of the
> above runs, with specifics about the test runs and failures. These are
> available here:
>
> http://developer.osdl.org/bryce/ltp/
Seems that the new failures are all related to failing to return -EFAULT
when passing a bad filename address into filesystem syscalls.
Perhaps this was fixed recently - they work OK here.
vmm:/usr/src/ltp-full-20031002# ./testcases/kernel/syscalls/access/access03
access03 1 PASS : access((char *)-1,R_OK) failed as expected with errno 14 (EFAULT) : Bad address
access03 2 PASS : access((char *)-1,W_OK) failed as expected with errno 14 (EFAULT) : Bad address
access03 3 PASS : access((char*)-1,X_OK) failed as expected with errno 14 (EFAULT) : Bad address
access03 4 PASS : access((char*)-1,F_OK) failed as expected with errno 14 (EFAULT) : Bad address
access03 5 PASS : access((char*)sbrk(0)+1,R_OK) failed as expected with errno 14 (EFAULT) : Bad address
access03 6 PASS : access((char*)sbrk(0)+1,W_OK) failed as expected with errno 14 (EFAULT) : Bad address
access03 7 PASS : access(high_address,X_OK) failed as expected with errno 14 (EFAULT) : Bad address
access03 8 PASS : access((char*)sbrk(0)+1,F_OK) failed as expected with errno 14 (EFAULT) : Bad address
vmm:/usr/src/ltp-full-20031002# ./testcases/kernel/syscalls/access/access05
access05 1 PASS : access() fails, Read Access denied on file, errno:13
access05 2 PASS : access() fails, Write Access denied on file, errno:13
access05 3 PASS : access() fails, Execute Access denied on file, errno:13
access05 4 PASS : access() fails, Access mode invalid, errno:22
access05 5 PASS : access() fails, Address beyond address space, errno:14
access05 6 PASS : access() fails, Negative address, errno:14
access05 7 PASS : access() fails, Pathname is empty, errno:2
access05 8 PASS : access() fails, Pathname too long, errno:36
vmm:/usr/src/ltp-full-20031002# ./testcases/kernel/syscalls/chdir/chdir04
chdir04 1 PASS : expected failure - errno = 36 : File name too long
chdir04 2 PASS : expected failure - errno = 2 : No such file or directory
chdir04 3 PASS : expected failure - errno = 14 : Bad address
Hi Andrew,
Here's an update with mm4 and bk12:
On Mon, 28 Jun 2004, Andrew Morton wrote:
> Bryce Harrington <[email protected]> wrote:
> >
> > Here is a listing of LTP results for the linux kernel. For the 2.6.x
> > series LTP results have been pretty constant, but they've gotten
> > interesting lately:
> >
> > Patch Name TestReq# CPU PASS FAIL WARN BROK RunTime
> > ----------------------------------------------------------------------
> > patch-2.4.27-rc2 294321 2-way 7226 6 3 6 69.0
> > linux-2.6.7 294027 2-way 7225 6 3 6 46.0
> > linux-2.6.7 294004 1-way 7225 6 3 6 42.2
> > patch-2.6.7-bk1 294069 2-way 7224 7 3 6 45.9
> > patch-2.6.7-bk2 294081 2-way 7224 7 3 6 45.9
> > patch-2.6.7-bk3 294103 2-way 7224 7 3 6 46.4
> > patch-2.6.7-bk4 294165 2-way 7187 7 3 6 48.7
> > patch-2.6.7-bk5 294181 2-way 7181 7 3 6 45.5
> > patch-2.6.7-bk6 294204 2-way 7224 7 3 6 47.1
> > patch-2.6.7-bk7 294228 2-way 7224 7 3 6 49.0
> > patch-2.6.7-bk8 294304 2-way 7223 10 3 7 47.5
> > patch-2.6.7-bk9 294333 2-way 7224 7 3 6 46.1
> > patch-2.6.7-bk10 294403 2-way 7223 10 3 7 42.9
> > patch-2.6.7-bk11 294423 2-way 7178 46 3 6 47.8
patch-2.6.7-bk12 294442 2-way 7178 46 3 6 47.6
> > 2.6.7-mm1 294146 2-way 7185 46 3 6 59.1
> > 2.6.7-mm1 294126 1-way 7185 46 3 6 52.9
> > 2.6.7-mm2 294271 2-way 7181 47 3 6 44.9
> > 2.6.7-mm3 294363 1-way 7185 46 3 6 41.0
2.6.7-mm4 294465 1-way 7178 46 3 6 42.1
> > 2.6.7-rc3-mm2 293949 2-way 7223 8 3 6 46.5
> >
> > I've generated some detailed LTP test result reports on a few of the
> > above runs, with specifics about the test runs and failures. These are
> > available here:
> >
> > http://developer.osdl.org/bryce/ltp/
>
> Seems that the new failures are all related to failing to return -EFAULT
> when passing a bad filename address into filesystem syscalls.
>
> Perhaps this was fixed recently - they work OK here.
We're seeing the same behavior on the mm4 and bk12 kernels as before.
It looks like you're using a version of LTP from 2003... If that's
true, could you try running the April 2004 version of LTP, just to rule
out it being a difference in test versions?
http://prdownloads.sourceforge.net/ltp/ltp-full-20040405.tgz?download
Bryce
> vmm:/usr/src/ltp-full-20031002# ./testcases/kernel/syscalls/access/access03
> access03 1 PASS : access((char *)-1,R_OK) failed as expected with errno 14 (EFAULT) : Bad address
> access03 2 PASS : access((char *)-1,W_OK) failed as expected with errno 14 (EFAULT) : Bad address
> access03 3 PASS : access((char*)-1,X_OK) failed as expected with errno 14 (EFAULT) : Bad address
> access03 4 PASS : access((char*)-1,F_OK) failed as expected with errno 14 (EFAULT) : Bad address
> access03 5 PASS : access((char*)sbrk(0)+1,R_OK) failed as expected with errno 14 (EFAULT) : Bad address
> access03 6 PASS : access((char*)sbrk(0)+1,W_OK) failed as expected with errno 14 (EFAULT) : Bad address
> access03 7 PASS : access(high_address,X_OK) failed as expected with errno 14 (EFAULT) : Bad address
> access03 8 PASS : access((char*)sbrk(0)+1,F_OK) failed as expected with errno 14 (EFAULT) : Bad address
> vmm:/usr/src/ltp-full-20031002# ./testcases/kernel/syscalls/access/access05
> access05 1 PASS : access() fails, Read Access denied on file, errno:13
> access05 2 PASS : access() fails, Write Access denied on file, errno:13
> access05 3 PASS : access() fails, Execute Access denied on file, errno:13
> access05 4 PASS : access() fails, Access mode invalid, errno:22
> access05 5 PASS : access() fails, Address beyond address space, errno:14
> access05 6 PASS : access() fails, Negative address, errno:14
> access05 7 PASS : access() fails, Pathname is empty, errno:2
> access05 8 PASS : access() fails, Pathname too long, errno:36
> vmm:/usr/src/ltp-full-20031002# ./testcases/kernel/syscalls/chdir/chdir04
> chdir04 1 PASS : expected failure - errno = 36 : File name too long
> chdir04 2 PASS : expected failure - errno = 2 : No such file or directory
> chdir04 3 PASS : expected failure - errno = 14 : Bad address
>
On Mon, Jun 28, 2004 at 06:46:15PM -0700, Bryce Harrington wrote:
> We usually always see 6-7 fails on the 2.6.x kernels, so the increase is
> unusual.
> I've generated some detailed LTP test result reports on a few of the
> above runs, with specifics about the test runs and failures. These are
> available here:
> http://developer.osdl.org/bryce/ltp/
This looks related to some widely-propagated change in errno return
value (probably originating from some centralized source and cascading
up the call chains).
-- wli