I've collected some data from STP to see if it's useful or if there's
anything else that would be useful to collect. I've got some tests
queued up for the newer patches, but I wanted to put out what I had so
far.
METRICS OVER LAST 20 MINUTES:
--------------- -------- ----- ---- -------- -----------------------------------
Kernel Elevator NOTPM CPU% Blocks/s URL
--------------- -------- ----- ---- -------- -----------------------------------
2.5.68-mm2 as 1155 94.3 8940.2 http://khack.osdl.org/stp/271356/
2.5.68-mm2 deadline 1255 94.9 9598.7 http://khack.osdl.org/stp/271359/
FUNCTIONS SORTED BY TICKS:
-- ------------------------- ------- ------------------------- -------
# as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
-- ------------------------- ------- ------------------------- -------
1 default_idle 6103428 default_idle 5359025
2 bounce_copy_vec 86272 bounce_copy_vec 97696
3 schedule 63819 schedule 70114
4 __make_request 30397 __blk_queue_bounce 31167
5 __blk_queue_bounce 26962 scsi_request_fn 26623
6 scsi_request_fn 24845 __make_request 25012
7 do_softirq 21122 do_softirq 24623
8 scsi_end_request 14080 system_call 13056
9 system_call 12059 try_to_wake_up 12503
10 try_to_wake_up 11240 dio_bio_end_io 11511
--
Mark Wong - - [email protected]
Open Source Development Lab Inc - A non-profit corporation
15275 SW Koll Parkway - Suite H - Beaverton OR, 97006
(503)-626-2455 x 32 (office)
(503)-626-2436 (fax)
http://www.osdl.org/archive/markw/
On Wed, May 07, 2003 at 09:33:29AM -0700, [email protected] wrote:
> FUNCTIONS SORTED BY TICKS:
> -- ------------------------- ------- ------------------------- -------
> # as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
> -- ------------------------- ------- ------------------------- -------
> 1 default_idle 6103428 default_idle 5359025
> 2 bounce_copy_vec 86272 bounce_copy_vec 97696
> 3 schedule 63819 schedule 70114
> 4 __make_request 30397 __blk_queue_bounce 31167
> 5 __blk_queue_bounce 26962 scsi_request_fn 26623
> 6 scsi_request_fn 24845 __make_request 25012
> 7 do_softirq 21122 do_softirq 24623
> 8 scsi_end_request 14080 system_call 13056
> 9 system_call 12059 try_to_wake_up 12503
> 10 try_to_wake_up 11240 dio_bio_end_io 11511
You're already in deeper trouble than elevators can get you out of as
your driver is using bounce buffers. What hardware/driver are you using?
-- wli
On Wed, May 07 2003, [email protected] wrote:
> I've collected some data from STP to see if it's useful or if there's
> anything else that would be useful to collect. I've got some tests
> queued up for the newer patches, but I wanted to put out what I had so
> far.
>
>
> METRICS OVER LAST 20 MINUTES:
> --------------- -------- ----- ---- -------- -----------------------------------
> Kernel Elevator NOTPM CPU% Blocks/s URL
> --------------- -------- ----- ---- -------- -----------------------------------
> 2.5.68-mm2 as 1155 94.3 8940.2 http://khack.osdl.org/stp/271356/
> 2.5.68-mm2 deadline 1255 94.9 9598.7 http://khack.osdl.org/stp/271359/
>
> FUNCTIONS SORTED BY TICKS:
> -- ------------------------- ------- ------------------------- -------
> # as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
> -- ------------------------- ------- ------------------------- -------
> 1 default_idle 6103428 default_idle 5359025
> 2 bounce_copy_vec 86272 bounce_copy_vec 97696
> 3 schedule 63819 schedule 70114
> 4 __make_request 30397 __blk_queue_bounce 31167
> 5 __blk_queue_bounce 26962 scsi_request_fn 26623
> 6 scsi_request_fn 24845 __make_request 25012
uhh nasty, you are spending a lot of time bouncing. How much RAM is in
the machine, and what is the scsi hba?
--
Jens Axboe
On Wed, 2003-05-07 at 18:33, [email protected] wrote:
> I've collected some data from STP to see if it's useful or if there's
> anything else that would be useful to collect. I've got some tests
> queued up for the newer patches, but I wanted to put out what I had so
> far.
>
>
> METRICS OVER LAST 20 MINUTES:
> --------------- -------- ----- ---- -------- -----------------------------------
> Kernel Elevator NOTPM CPU% Blocks/s URL
> --------------- -------- ----- ---- -------- -----------------------------------
> 2.5.68-mm2 as 1155 94.3 8940.2 http://khack.osdl.org/stp/271356/
> 2.5.68-mm2 deadline 1255 94.9 9598.7 http://khack.osdl.org/stp/271359/
>
> FUNCTIONS SORTED BY TICKS:
> -- ------------------------- ------- ------------------------- -------
> # as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
> -- ------------------------- ------- ------------------------- -------
> 1 default_idle 6103428 default_idle 5359025
> 2 bounce_copy_vec 86272 bounce_copy_vec 97696
> 3 schedule 63819 schedule 70114
> 4 __make_request 30397 __blk_queue_bounce 31167
> 5 __blk_queue_bounce 26962 scsi_request_fn 26623
> 6 scsi_request_fn 24845 __make_request 25012
You are using scsi, what tcq depth are you using? AS doesn't like >4 or
something like that.
--
/Martin
On 7 May, Jens Axboe wrote:
> On Wed, May 07 2003, [email protected] wrote:
>> I've collected some data from STP to see if it's useful or if there's
>> anything else that would be useful to collect. I've got some tests
>> queued up for the newer patches, but I wanted to put out what I had so
>> far.
>>
>>
>> METRICS OVER LAST 20 MINUTES:
>> --------------- -------- ----- ---- -------- -----------------------------------
>> Kernel Elevator NOTPM CPU% Blocks/s URL
>> --------------- -------- ----- ---- -------- -----------------------------------
>> 2.5.68-mm2 as 1155 94.3 8940.2 http://khack.osdl.org/stp/271356/
>> 2.5.68-mm2 deadline 1255 94.9 9598.7 http://khack.osdl.org/stp/271359/
>>
>> FUNCTIONS SORTED BY TICKS:
>> -- ------------------------- ------- ------------------------- -------
>> # as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
>> -- ------------------------- ------- ------------------------- -------
>> 1 default_idle 6103428 default_idle 5359025
>> 2 bounce_copy_vec 86272 bounce_copy_vec 97696
>> 3 schedule 63819 schedule 70114
>> 4 __make_request 30397 __blk_queue_bounce 31167
>> 5 __blk_queue_bounce 26962 scsi_request_fn 26623
>> 6 scsi_request_fn 24845 __make_request 25012
>
> uhh nasty, you are spending a lot of time bouncing. How much RAM is in
> the machine, and what is the scsi hba?
The system has 4GB of memory and has a DECchip 21554 (aacraid) that the
external drives are connected to. Mark Haverkamp is currently trying to
address those bounce buffers.
On 7 May, Martin Josefsson wrote:
> On Wed, 2003-05-07 at 18:33, [email protected] wrote:
>> I've collected some data from STP to see if it's useful or if there's
>> anything else that would be useful to collect. I've got some tests
>> queued up for the newer patches, but I wanted to put out what I had so
>> far.
>>
>>
>> METRICS OVER LAST 20 MINUTES:
>> --------------- -------- ----- ---- -------- -----------------------------------
>> Kernel Elevator NOTPM CPU% Blocks/s URL
>> --------------- -------- ----- ---- -------- -----------------------------------
>> 2.5.68-mm2 as 1155 94.3 8940.2 http://khack.osdl.org/stp/271356/
>> 2.5.68-mm2 deadline 1255 94.9 9598.7 http://khack.osdl.org/stp/271359/
>>
>> FUNCTIONS SORTED BY TICKS:
>> -- ------------------------- ------- ------------------------- -------
>> # as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
>> -- ------------------------- ------- ------------------------- -------
>> 1 default_idle 6103428 default_idle 5359025
>> 2 bounce_copy_vec 86272 bounce_copy_vec 97696
>> 3 schedule 63819 schedule 70114
>> 4 __make_request 30397 __blk_queue_bounce 31167
>> 5 __blk_queue_bounce 26962 scsi_request_fn 26623
>> 6 scsi_request_fn 24845 __make_request 25012
>
> You are using scsi, what tcq depth are you using? AS doesn't like >4 or
> something like that.
I'm told that it's set to 128 with this bit of code:
if(dev->tagged_supported)
scsi_adjust_queue_depth(dev, MSG_ORDERED_TAG, 128);
On Wed, 2003-05-07 at 10:36, Jens Axboe wrote:
> On Wed, May 07 2003, [email protected] wrote:
> > On 7 May, Jens Axboe wrote:
> > > On Wed, May 07 2003, [email protected] wrote:
> > >> I've collected some data from STP to see if it's useful or if there's
> > >> anything else that would be useful to collect. I've got some tests
> > >> queued up for the newer patches, but I wanted to put out what I had so
> > >> far.
> > >>
> > >>
> > >> METRICS OVER LAST 20 MINUTES:
> > >> --------------- -------- ----- ---- -------- -----------------------------------
> > >> Kernel Elevator NOTPM CPU% Blocks/s URL
> > >> --------------- -------- ----- ---- -------- -----------------------------------
> > >> 2.5.68-mm2 as 1155 94.3 8940.2 http://khack.osdl.org/stp/271356/
> > >> 2.5.68-mm2 deadline 1255 94.9 9598.7 http://khack.osdl.org/stp/271359/
> > >>
> > >> FUNCTIONS SORTED BY TICKS:
> > >> -- ------------------------- ------- ------------------------- -------
> > >> # as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
> > >> -- ------------------------- ------- ------------------------- -------
> > >> 1 default_idle 6103428 default_idle 5359025
> > >> 2 bounce_copy_vec 86272 bounce_copy_vec 97696
> > >> 3 schedule 63819 schedule 70114
> > >> 4 __make_request 30397 __blk_queue_bounce 31167
> > >> 5 __blk_queue_bounce 26962 scsi_request_fn 26623
> > >> 6 scsi_request_fn 24845 __make_request 25012
> > >
> > > uhh nasty, you are spending a lot of time bouncing. How much RAM is in
> > > the machine, and what is the scsi hba?
> >
> > The system has 4GB of memory and has a DECchip 21554 (aacraid) that the
> > external drives are connected to. Mark Haverkamp is currently trying to
> > address those bounce buffers.
>
> aacraid actually looks sane enough. so you should just be able to set
> host->highmem_io and it should work.
That is what I thought, but I wanted to get a machine with lots of
memory to try it out on.
Mark.
--
Mark Haverkamp <[email protected]>
On Wed, May 07 2003, [email protected] wrote:
> On 7 May, Jens Axboe wrote:
> > On Wed, May 07 2003, [email protected] wrote:
> >> I've collected some data from STP to see if it's useful or if there's
> >> anything else that would be useful to collect. I've got some tests
> >> queued up for the newer patches, but I wanted to put out what I had so
> >> far.
> >>
> >>
> >> METRICS OVER LAST 20 MINUTES:
> >> --------------- -------- ----- ---- -------- -----------------------------------
> >> Kernel Elevator NOTPM CPU% Blocks/s URL
> >> --------------- -------- ----- ---- -------- -----------------------------------
> >> 2.5.68-mm2 as 1155 94.3 8940.2 http://khack.osdl.org/stp/271356/
> >> 2.5.68-mm2 deadline 1255 94.9 9598.7 http://khack.osdl.org/stp/271359/
> >>
> >> FUNCTIONS SORTED BY TICKS:
> >> -- ------------------------- ------- ------------------------- -------
> >> # as 2.5.68-mm2 ticks deadline 2.5.68-mm2 ticks
> >> -- ------------------------- ------- ------------------------- -------
> >> 1 default_idle 6103428 default_idle 5359025
> >> 2 bounce_copy_vec 86272 bounce_copy_vec 97696
> >> 3 schedule 63819 schedule 70114
> >> 4 __make_request 30397 __blk_queue_bounce 31167
> >> 5 __blk_queue_bounce 26962 scsi_request_fn 26623
> >> 6 scsi_request_fn 24845 __make_request 25012
> >
> > uhh nasty, you are spending a lot of time bouncing. How much RAM is in
> > the machine, and what is the scsi hba?
>
> The system has 4GB of memory and has a DECchip 21554 (aacraid) that the
> external drives are connected to. Mark Haverkamp is currently trying to
> address those bounce buffers.
aacraid actually looks sane enough. so you should just be able to set
host->highmem_io and it should work.
--
Jens Axboe
On Wed, 7 May 2003, Mark Haverkamp wrote:
> That is what I thought, but I wanted to get a machine with lots of
> memory to try it out on.
if(aac->pae_support)
pci_set_dma_mask(dev, 0xFFFFFFFFFFFFFFFFUL);
This probably wants 0xFFFFFFFFFFFFFFFFULL and checking pci_set_dma_mask()
return value.
--
function.linuxpower.ca
[email protected] wrote:
>I've collected some data from STP to see if it's useful or if there's
>anything else that would be useful to collect. I've got some tests
>queued up for the newer patches, but I wanted to put out what I had so
>far.
>
Thanks. It looks like AS isn't doing too badly here. Newer mm kernels
have some more AS changes, and the dynamic struct request patch which
would be good to test.
Are you using TCQ on your disks?
On 8 May, Nick Piggin wrote:
> [email protected] wrote:
>
>>I've collected some data from STP to see if it's useful or if there's
>>anything else that would be useful to collect. I've got some tests
>>queued up for the newer patches, but I wanted to put out what I had so
>>far.
>>
> Thanks. It looks like AS isn't doing too badly here. Newer mm kernels
> have some more AS changes, and the dynamic struct request patch which
> would be good to test.
>
> Are you using TCQ on your disks?
>
There's a queue depth being set. Is that a good indicator that TCQ is
being used? If not, I'd be happy to verify it.
Mark
On Thu, May 08 2003, [email protected] wrote:
> On 8 May, Nick Piggin wrote:
> > [email protected] wrote:
> >
> >>I've collected some data from STP to see if it's useful or if there's
> >>anything else that would be useful to collect. I've got some tests
> >>queued up for the newer patches, but I wanted to put out what I had so
> >>far.
> >>
> > Thanks. It looks like AS isn't doing too badly here. Newer mm kernels
> > have some more AS changes, and the dynamic struct request patch which
> > would be good to test.
> >
> > Are you using TCQ on your disks?
> >
>
> There's a queue depth being set. Is that a good indicator that TCQ is
> being used? If not, I'd be happy to verify it.
The queue depth being set, is the highest queueing depth that the scsi
mid layer will throw at you. The actual TCQ depth may be lower, depends
on the hardware. aacraid, iirc, has a pretty big depth so I woudldn't be
surprised if it could use all of those 128 tags.
It would be interesting to see a forced depth of 2 with AS against the
stock case of 128 (and deadline).
--
Jens Axboe