Message-ID: <49DA33EF.3020700@vlnb.net>
Date: Mon, 06 Apr 2009 20:55:11 +0400
From: Vladislav Bolkhovitin <vst@vlnb.net>
User-Agent: Thunderbird 2.0.0.21 (X11/20090320)
MIME-Version: 1.0
To: Tomasz Chmielewski <mangoo@wpkg.org>
CC: Bart Van Assche <bart.vanassche@gmail.com>, linux-kernel@vger.kernel.org,
       linux-scsi@vger.kernel.org, iscsitarget-devel@lists.sourceforge.net,
       James Bottomley <James.Bottomley@hansenpartnership.com>,
       scst-devel <scst-devel@lists.sourceforge.net>, stgt@vger.kernel.org
Subject: Re: [Scst-devel] [ANNOUNCE]: Comparison of features sets between
 	different SCSI targets (SCST, STGT, IET, LIO)
References: <49D7AD54.4060408@vlnb.net> <49D7B122.50103@wpkg.org> <e2e108260904050429lcd1870ag6656f36a170ccab3@mail.gmail.com> <49D9D9A0.40106@wpkg.org>
In-Reply-To: <49D9D9A0.40106@wpkg.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2942
Lines: 82

Tomasz Chmielewski, on 04/06/2009 02:29 PM wrote:
> Bart Van Assche schrieb:
> 
>> Hello Tomasz,
>>
>> It would be great if you could publish the details of your setup and
>> the tests you have run, such that we are able to reproduce the tests
>> and analyze the results further.
> 
> Here it is.
> 
> The target is running Debian Lenny 64bit userspace on an Intel Celeron 2.93GHz CPU, 2 GB RAM.
> 
> Initiator is running Debian Etch 64 bit userspace, open-iscsi 2.0-869, Intel Xeon 3050/2.13GHz, 8 GB RAM.
> 
> 
> Each test was repeated 6 times, "sync" was made and caches were dropped on both sides before each test was started.
> 
> dd parameters were like below, so 6.6 GB of data was read each time:
> 
> dd if=/dev/sdag of=/dev/null bs=64k count=100000
> 
> 
> Data was read from two block devices:
> - /dev/md0, which is RAID-1 on two ST31500341AS 1.5 TB drives
> - encrypted dm-crypt device which is on top of /dev/md0
> 
> Encrypted device was created with the following additional options passed to cryptsetup
> (it provides the most performance on systems where CPU is a bottleneck, but with decreased
> security when compared to default options):
> 
> -c aes-ecb-plain -s 128
> 
> 
> Generally, CPU on the target was a bottleneck, so I also tested the load on target.
> 
> 
> md0, crypt columns - averages from dd
> us, sy, id, wa - averages from vmstat
> 
> 
> 1. Disk speeds on the target
> 
> Raw performance: 102.17 MB/s
> Raw performance (encrypted):  50.21 MB/s
> 
> 
> 2. Read-ahead on the initiator: 256 (default); md0, crypt - MB/s
> 
>                            md0   us  sy  id  wa  | crypt   us  sy  id  wa  
> STGT                      50.63   4% 45% 18% 33% | 32.52    3% 62% 16% 19%
> SCST (debug + no patches) 43.75   0% 26% 30% 44% | 42.05    0% 84%  1% 15%
> SCST (fullperf + patches) 45.18   0% 25% 33% 42% | 44.12    0% 81%  2% 17%
> 
> 
> 3. Read-ahead on the initiator: 16384; md0, crypt - MB/s
> 
>                            md0   us  sy  id  wa  | crypt   us  sy  id  wa  
> STGT                      56.43   3% 55%  2% 40% | 46.90    3% 90%  3%  4%
> SCST (debug + no patches) 73.85   0% 58%  1% 41% | 42.70    0% 85%  0% 15%
> SCST (fullperf + patches) 76.27   0% 63%  1% 36% | 42.52    0% 85%  0% 15%

Good! You proved that:

1. SCST is capable to work much better than STGT: 35% for md and 37% for 
crypt considering maximum values.

2. Default read-ahead size isn't appropriate for remote data access 
cases and should be increased. I slowly have been discussing it in past 
few months with Wu Fengguang, the read-ahead maintainer.

Which IO scheduler on the target did you use? I guess, deadline? If so, 
you should try with CFQ as well.

Thanks,
Vlad

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/