Thanks Andrew and Matthew.
We can now actually *without* rebooting the machine,
start the database.
stop the database, and
(gasp!) re-start the database without error. Progress. :)
(2.5.62 stock and 2.5.62-mm1 both fail this test)
Also,
we've run the OSDL dbt-2 workload on a 4-cpu machine,
(PIII @700mhz x4 w/3GB RAM Qlogic 2200 FC card)
using
linux-2.5.62 + flock() fix (plm patch id#1567)
- disk using qlogicfc driver
linux-2.5.62-mm1 + flock() fix (plm patch id#1567)
- disk using isp (new) driver
Results and bunch o' numbers at:
http://www.osdl.org/archive/cliffw/flock/flock-mm1
http://www.osdl.org/archive/cliffw/flock/flock-62
Caveats: This data is one run from each kernel,
results from repeated runs may vary, harmful or fatal if
swallowed. Merge of plm 1567 w/-mm1 performed by unskilled
labor. We have *not* tested this on the several other
wierdasp SCSI devices in house, more results maybe.
Quick observations:
DBT numbers are about the same (+/- 10%), but vmstat differs:
- There are certainly tunables, especially in /proc/vm/sys
that we haven't tried yet. Any suggestions on tunables
most appreciated.
- more procssess are runing w/-mm1 in each sample..
- this a cached run, io occurs mostly during load, except for db log io
If you want some numbers Right Now, these are from the midpoint
of each run, full files at above web link:
-----vmstat 60second interval, Linux-2.5.62-flk----------
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
4 8 0 992 298672 35192 1241232 0 0 165 12960 1945 5748 95 4 1
4 0 0 992 292888 35668 1241664 0 0 183 1159 1173 3487 95 2 3
5 0 0 992 284368 36324 1242108 0 0 185 1047 1162 3543 95 2 3
5 1 0 992 276816 36636 1242496 0 0 172 1028 1157 3438 95 2 3
5 0 0 992 269336 36972 1242896 0 0 160 1056 1160 3562 95 2 3
4 0 0 992 262048 37276 1243288 0 0 155 1041 1158 3480 95 2 3
4 1 0 992 254552 37756 1243716 0 0 144 1051 1158 3474 95 2 3
4 0 0 992 247776 38412 1244160 0 0 137 1061 1158 3473 95 2 3
----vmstat 60 second interval, Linux-2.5.62-mm1-flk---------
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
6 2 1 1100 9968 29624 1522312 0 0 209 1174 1276 3841 95 2 3
13 3 1 1100 10176 29696 1514056 0 0 190 1135 1268 3823 95 2 3
8 1 1 1100 9960 29852 1506376 0 0 190 1131 1266 3744 95 2 3
4 5 3 1100 10176 29884 1499152 0 0 155 1109 1260 3689 95 2 3
7 1 1 1100 10184 29972 1492920 0 0 156 1138 1262 3753 95 2 3
7 1 1 1100 9720 30244 1486636 0 0 150 1150 1266 3690 95 2 3
8 1 1 1100 14008 30220 1476588 0 0 131 1133 1260 3739 95 2 3
11 5 1 1100 10048 30392 1474692 0 0 123 7871 1768 5311 95 3 2
--------------------------
cliffw
[email protected]
Cliff White <[email protected]> wrote:
>
>
> Thanks Andrew and Matthew.
> We can now actually *without* rebooting the machine,
> start the database.
> stop the database, and
> (gasp!) re-start the database without error. Progress. :)
> (2.5.62 stock and 2.5.62-mm1 both fail this test)
Ah, that's good news. I have an update from Matthew which fixes
the confusion-at-reboot-time. Below.
> Also,
> we've run the OSDL dbt-2 workload on a 4-cpu machine,
> (PIII @700mhz x4 w/3GB RAM Qlogic 2200 FC card)
> using
>
> linux-2.5.62 + flock() fix (plm patch id#1567)
> - disk using qlogicfc driver
> linux-2.5.62-mm1 + flock() fix (plm patch id#1567)
> - disk using isp (new) driver
>
> Results and bunch o' numbers at:
> http://www.osdl.org/archive/cliffw/flock/flock-mm1
> http://www.osdl.org/archive/cliffw/flock/flock-62
>
> Caveats: This data is one run from each kernel,
> results from repeated runs may vary, harmful or fatal if
> swallowed. Merge of plm 1567 w/-mm1 performed by unskilled
> labor. We have *not* tested this on the several other
> wierdasp SCSI devices in house, more results maybe.
>
> Quick observations:
> DBT numbers are about the same (+/- 10%), but vmstat differs:
> - There are certainly tunables, especially in /proc/vm/sys
> that we haven't tried yet. Any suggestions on tunables
> most appreciated.
> - more procssess are runing w/-mm1 in each sample..
> - this a cached run, io occurs mostly during load, except for db log io
>
That all looks OK. You're sustaining almost 95% user CPU time across the
run. The -mm1 run is using more pagecache for some reason. Presumably
because of "more processes".
scsi/isp/isp_linux.c | 18 +++++++++---------
scsi/isp/isp_pci.c | 16 +++++++++++++---
2 files changed, 22 insertions(+), 12 deletions(-)
diff -puN drivers/scsi/isp/isp_linux.c~isp-update-1 drivers/scsi/isp/isp_linux.c
--- 25/drivers/scsi/isp/isp_linux.c~isp-update-1 2003-02-18 13:19:18.000000000 -0800
+++ 25-akpm/drivers/scsi/isp/isp_linux.c 2003-02-18 13:19:18.000000000 -0800
@@ -944,7 +944,7 @@ isplinux_abort(Scsi_Cmnd *Cmnd)
NewCmnd = isplinux_remove_from_doneq(Cmnd);
wqfnd++;
}
- ISP_UNLKU_SOFTC(isp);
+ ISP_UNLK_SOFTC(isp);
isp_prt(isp, ISP_LOGINFO,
"isplinux_abort: found %d:%p for non-running cmd for %d.%d.%d",
wqfnd, NewCmnd, XS_CHANNEL(Cmnd), XS_TGT(Cmnd), XS_LUN(Cmnd));
@@ -954,14 +954,14 @@ isplinux_abort(Scsi_Cmnd *Cmnd)
}
} else {
if (isp_control(isp, ISPCTL_ABORT_CMD, Cmnd)) {
- ISP_UNLKU_SOFTC(isp);
+ ISP_UNLK_SOFTC(isp);
ISP_DRIVER_EXIT_LOCK(isp);
return (FAILED);
}
if (isp->isp_nactive > 0)
isp->isp_nactive--;
isp_destroy_handle(isp, handle);
- ISP_UNLKU_SOFTC(isp);
+ ISP_UNLK_SOFTC(isp);
ISP_DRIVER_EXIT_LOCK(isp);
isp_prt(isp, ISP_LOGINFO,
"isplinux_abort: aborted running cmd (handle 0x%x) for %d.%d.%d",
@@ -989,9 +989,9 @@ isplinux_bdr(Scsi_Cmnd *Cmnd)
isp = XS_ISP(Cmnd);
arg = XS_CHANNEL(Cmnd) << 16 | XS_TGT(Cmnd);
ISP_DRIVER_ENTRY_LOCK(isp);
- ISP_LOCKU_SOFTC(isp);
+ ISP_LOCK_SOFTC(isp);
arg = isp_control(isp, ISPCTL_RESET_DEV, &arg);
- ISP_UNLKU_SOFTC(isp);
+ ISP_UNLK_SOFTC(isp);
ISP_DRIVER_EXIT_LOCK(isp);
isp_prt(isp, ISP_LOGINFO, "Bus Device Reset %succesfully sent to %d.%d.%d",
arg == 0? "s" : "uns", XS_CHANNEL(Cmnd), XS_TGT(Cmnd), XS_LUN(Cmnd));
@@ -1013,9 +1013,9 @@ isplinux_sreset(Scsi_Cmnd *Cmnd)
isp = XS_ISP(Cmnd);
arg = XS_CHANNEL(Cmnd);
ISP_DRIVER_ENTRY_LOCK(isp);
- ISP_LOCKU_SOFTC(isp);
+ ISP_LOCK_SOFTC(isp);
arg = isp_control(isp, ISPCTL_RESET_BUS, &arg);
- ISP_UNLKU_SOFTC(isp);
+ ISP_UNLK_SOFTC(isp);
ISP_DRIVER_EXIT_LOCK(isp);
isp_prt(isp, ISP_LOGINFO, "SCSI Bus Reset on Channel %d %succesful",
XS_CHANNEL(Cmnd), arg == 0? "s" : "uns");
@@ -1041,7 +1041,7 @@ isplinux_hreset(Scsi_Cmnd *Cmnd)
isp_prt(isp, ISP_LOGINFO, "Resetting Host Adapter");
ISP_DRIVER_ENTRY_LOCK(isp);
- ISP_LOCKU_SOFTC(isp);
+ ISP_LOCK_SOFTC(isp);
/*
* Save pending, running, and completed commands.
@@ -1069,7 +1069,7 @@ isplinux_hreset(Scsi_Cmnd *Cmnd)
isplinux_reinit(isp);
- ISP_UNLKU_SOFTC(isp);
+ ISP_UNLK_SOFTC(isp);
ISP_DRIVER_EXIT_LOCK(isp);
/*
diff -puN drivers/scsi/isp/isp_pci.c~isp-update-1 drivers/scsi/isp/isp_pci.c
--- 25/drivers/scsi/isp/isp_pci.c~isp-update-1 2003-02-18 13:19:18.000000000 -0800
+++ 25-akpm/drivers/scsi/isp/isp_pci.c 2003-02-18 13:20:11.000000000 -0800
@@ -377,12 +377,17 @@ isplinux_pci_addhost(Scsi_Host_Template
isp->isp_next = isplist;
isplist = isp;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,4)
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,5,61)
scsi_set_device(host, &pci_isp->pci_dev->dev);
+#else
+ scsi_set_pci_device(host, pci_isp->pci_dev);
+#endif
#endif
return (pci_isp);
}
-#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18)
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18) && \
+ LINUX_VERSION_CODE < KERNEL_VERSION(2,5,0)
#include <linux/reboot.h>
static int
isp_notify_reboot(struct notifier_block *ispnb, unsigned long Event, void *b)
@@ -513,7 +518,11 @@ isplinux_pci_detect(Scsi_Host_Template *
}
}
#endif
-#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18)
+ /*
+ * Don't do reboot notifier stuff for 2.5.X yet
+ */
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18) && \
+ LINUX_VERSION_CODE < KERNEL_VERSION(2,5,0)
if (isp_nfound) {
register_reboot_notifier(&isp_notifier);
}
@@ -551,7 +560,8 @@ isplinux_pci_release(struct Scsi_Host *h
RlsPages(FCPARAM(isp)->isp_scratch, 1);
}
#endif
-#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18)
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18) && \
+ LINUX_VERSION_CODE < KERNEL_VERSION(2,5,0)
if (--isp_nfound <= 0) {
unregister_reboot_notifier(&isp_notifier);
}
_
> Cliff White <[email protected]> wrote:
> >
> >
> > Thanks Andrew and Matthew.
> > We can now actually *without* rebooting the machine,
> > start the database.
> > stop the database, and
> > (gasp!) re-start the database without error. Progress. :)
> > (2.5.62 stock and 2.5.62-mm1 both fail this test)
>
> Ah, that's good news. I have an update from Matthew which fixes
> the confusion-at-reboot-time. Below.
Hmm..i'm not sure what that problem is, i don't think we've seen it.
With the below patch, we still are unable to stop and restart SAP DB,
so we still need Matthew Wilcox's flock fix merged (plm 1567)
thanks
cliffw
>
> > Also,
> > we've run the OSDL dbt-2 workload on a 4-cpu machine,
> > (PIII @700mhz x4 w/3GB RAM Qlogic 2200 FC card)
> > using
> >
> > linux-2.5.62 + flock() fix (plm patch id#1567)
> > - disk using qlogicfc driver
> > linux-2.5.62-mm1 + flock() fix (plm patch id#1567)
> > - disk using isp (new) driver
> >
> > Results and bunch o' numbers at:
> > http://www.osdl.org/archive/cliffw/flock/flock-mm1
> > http://www.osdl.org/archive/cliffw/flock/flock-62
> >
> > Caveats: This data is one run from each kernel,
> > results from repeated runs may vary, harmful or fatal if
> > swallowed. Merge of plm 1567 w/-mm1 performed by unskilled
> > labor. We have *not* tested this on the several other
> > wierdasp SCSI devices in house, more results maybe.
> >
> > Quick observations:
> > DBT numbers are about the same (+/- 10%), but vmstat differs:
> > - There are certainly tunables, especially in /proc/vm/sys
> > that we haven't tried yet. Any suggestions on tunables
> > most appreciated.
> > - more procssess are runing w/-mm1 in each sample..
> > - this a cached run, io occurs mostly during load, except for db log io
> >
>
> That all looks OK. You're sustaining almost 95% user CPU time across the
> run. The -mm1 run is using more pagecache for some reason. Presumably
> because of "more processes".
>
>
>
> scsi/isp/isp_linux.c | 18 +++++++++---------
> scsi/isp/isp_pci.c | 16 +++++++++++++---
> 2 files changed, 22 insertions(+), 12 deletions(-)
>
> diff -puN drivers/scsi/isp/isp_linux.c~isp-update-1 drivers/scsi/isp/isp_linux.c
> --- 25/drivers/scsi/isp/isp_linux.c~isp-update-1 2003-02-18 13:19:18.000000000 -0800
> +++ 25-akpm/drivers/scsi/isp/isp_linux.c 2003-02-18 13:19:18.000000000 -0800
> @@ -944,7 +944,7 @@ isplinux_abort(Scsi_Cmnd *Cmnd)
> NewCmnd = isplinux_remove_from_doneq(Cmnd);
> wqfnd++;
> }
> - ISP_UNLKU_SOFTC(isp);
> + ISP_UNLK_SOFTC(isp);
> isp_prt(isp, ISP_LOGINFO,
> "isplinux_abort: found %d:%p for non-running cmd for %d.%d.%d",
> wqfnd, NewCmnd, XS_CHANNEL(Cmnd), XS_TGT(Cmnd), XS_LUN(Cmnd));
> @@ -954,14 +954,14 @@ isplinux_abort(Scsi_Cmnd *Cmnd)
> }
> } else {
> if (isp_control(isp, ISPCTL_ABORT_CMD, Cmnd)) {
> - ISP_UNLKU_SOFTC(isp);
> + ISP_UNLK_SOFTC(isp);
> ISP_DRIVER_EXIT_LOCK(isp);
> return (FAILED);
> }
> if (isp->isp_nactive > 0)
> isp->isp_nactive--;
> isp_destroy_handle(isp, handle);
> - ISP_UNLKU_SOFTC(isp);
> + ISP_UNLK_SOFTC(isp);
> ISP_DRIVER_EXIT_LOCK(isp);
> isp_prt(isp, ISP_LOGINFO,
> "isplinux_abort: aborted running cmd (handle 0x%x) for %d.%d.%d",
> @@ -989,9 +989,9 @@ isplinux_bdr(Scsi_Cmnd *Cmnd)
> isp = XS_ISP(Cmnd);
> arg = XS_CHANNEL(Cmnd) << 16 | XS_TGT(Cmnd);
> ISP_DRIVER_ENTRY_LOCK(isp);
> - ISP_LOCKU_SOFTC(isp);
> + ISP_LOCK_SOFTC(isp);
> arg = isp_control(isp, ISPCTL_RESET_DEV, &arg);
> - ISP_UNLKU_SOFTC(isp);
> + ISP_UNLK_SOFTC(isp);
> ISP_DRIVER_EXIT_LOCK(isp);
> isp_prt(isp, ISP_LOGINFO, "Bus Device Reset %succesfully sent to %d.%d.%d",
> arg == 0? "s" : "uns", XS_CHANNEL(Cmnd), XS_TGT(Cmnd), XS_LUN(Cmnd));
> @@ -1013,9 +1013,9 @@ isplinux_sreset(Scsi_Cmnd *Cmnd)
> isp = XS_ISP(Cmnd);
> arg = XS_CHANNEL(Cmnd);
> ISP_DRIVER_ENTRY_LOCK(isp);
> - ISP_LOCKU_SOFTC(isp);
> + ISP_LOCK_SOFTC(isp);
> arg = isp_control(isp, ISPCTL_RESET_BUS, &arg);
> - ISP_UNLKU_SOFTC(isp);
> + ISP_UNLK_SOFTC(isp);
> ISP_DRIVER_EXIT_LOCK(isp);
> isp_prt(isp, ISP_LOGINFO, "SCSI Bus Reset on Channel %d %succesful",
> XS_CHANNEL(Cmnd), arg == 0? "s" : "uns");
> @@ -1041,7 +1041,7 @@ isplinux_hreset(Scsi_Cmnd *Cmnd)
> isp_prt(isp, ISP_LOGINFO, "Resetting Host Adapter");
>
> ISP_DRIVER_ENTRY_LOCK(isp);
> - ISP_LOCKU_SOFTC(isp);
> + ISP_LOCK_SOFTC(isp);
>
> /*
> * Save pending, running, and completed commands.
> @@ -1069,7 +1069,7 @@ isplinux_hreset(Scsi_Cmnd *Cmnd)
>
> isplinux_reinit(isp);
>
> - ISP_UNLKU_SOFTC(isp);
> + ISP_UNLK_SOFTC(isp);
> ISP_DRIVER_EXIT_LOCK(isp);
>
> /*
> diff -puN drivers/scsi/isp/isp_pci.c~isp-update-1 drivers/scsi/isp/isp_pci.c
> --- 25/drivers/scsi/isp/isp_pci.c~isp-update-1 2003-02-18 13:19:18.000000000 -0800
> +++ 25-akpm/drivers/scsi/isp/isp_pci.c 2003-02-18 13:20:11.000000000 -0800
> @@ -377,12 +377,17 @@ isplinux_pci_addhost(Scsi_Host_Template
> isp->isp_next = isplist;
> isplist = isp;
> #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,4)
> +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,5,61)
> scsi_set_device(host, &pci_isp->pci_dev->dev);
> +#else
> + scsi_set_pci_device(host, pci_isp->pci_dev);
> +#endif
> #endif
> return (pci_isp);
> }
>
> -#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18)
> +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18) && \
> + LINUX_VERSION_CODE < KERNEL_VERSION(2,5,0)
> #include <linux/reboot.h>
> static int
> isp_notify_reboot(struct notifier_block *ispnb, unsigned long Event, void *b)
> @@ -513,7 +518,11 @@ isplinux_pci_detect(Scsi_Host_Template *
> }
> }
> #endif
> -#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18)
> + /*
> + * Don't do reboot notifier stuff for 2.5.X yet
> + */
> +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18) && \
> + LINUX_VERSION_CODE < KERNEL_VERSION(2,5,0)
> if (isp_nfound) {
> register_reboot_notifier(&isp_notifier);
> }
> @@ -551,7 +560,8 @@ isplinux_pci_release(struct Scsi_Host *h
> RlsPages(FCPARAM(isp)->isp_scratch, 1);
> }
> #endif
> -#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18)
> +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,4,18) && \
> + LINUX_VERSION_CODE < KERNEL_VERSION(2,5,0)
> if (--isp_nfound <= 0) {
> unregister_reboot_notifier(&isp_notifier);
> }
>
> _
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>