Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756791AbaAHP0P (ORCPT ); Wed, 8 Jan 2014 10:26:15 -0500 Received: from bombadil.infradead.org ([198.137.202.9]:60416 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755301AbaAHP0N (ORCPT ); Wed, 8 Jan 2014 10:26:13 -0500 Date: Wed, 8 Jan 2014 07:26:10 -0800 From: Christoph Hellwig To: Sergey Meirovich Cc: Christoph Hellwig , Jan Kara , linux-scsi , Linux Kernel Mailing List , Gluk Subject: Re: Terrible performance of sequential O_DIRECT 4k writes in SAN environment. ~3 times slower then Solars 10 with the same HBA/Storage. Message-ID: <20140108152610.GA5863@infradead.org> References: <20140106201032.GA13491@quack.suse.cz> <20140107155830.GA28395@infradead.org> <20140108140307.GA588@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jan 08, 2014 at 04:43:07PM +0200, Sergey Meirovich wrote: > Results are almost the same: > 14.68Mb/sec 3758.02 Requests/sec > On my laptop SSD I get the following results (sometimes up to 200MB/s, sometimes down to 100MB/s, always in the 40k to 50k IOps range): time elapsed (sec.): 5 bandwidth (MiB/s): 160.00 IOps: 40960.00 The IOps are more than the hardware is physically capable of, but given that you didn't specify O_SYNC this seems sensible given that we never have to flush the disk cache. Could it be that your array has WCE=0? In Linux we'll never enable the cache automatically, but Solaris does at least when using ZFS. Try running: sdparm --set=WCE /dev/sdX and try again. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/