Hi fellows,
I'm been facing some lseek() troubles on a very light hardware (Atom E660) under heavy load (network + cpu + disk IOs). I'm using 3.2.32 on a 32bit Os with a local SSD as mass storage.
If a do open a block device like sdb1 and lseek SEEK_SET in it, some unexpected latencies occurs.
Using the same load, everything works perfectly by using contigous streams but once I do lseek it start to be laggy. I've been searching around for a while and finally found this message : https://lkml.org/lkml/2011/9/15/399 from Andy.
The description was very similar to what I experienced but the patch from Andy was on to the fs layer.
I've been looking the code for the block level layer and found the implementation is pretty different.
http://lxr.linux.no/#linux+v3.2.33/fs/read_write.c#L69
vs
http://lxr.linux.no/#linux+v3.2.33/fs/block_dev.c#L353
As I can see, we do first put the mutex, then i_size_read and then considering the kind of SEEK we want.
The semantic changes from the read_write implementation where it does the locking only for SEEK_CUR and i_size_read isn't executed for SEEK_SET.
So I really wonder if we shall rework this part to avoid the uncessary locking for all of them except SEEK_CUR and remove i_size_read from SEEK_SET. The i_size_read is also a matter as it does a memory barrier. On such low-end hardware I have, that could costs.
I can work on it and validate its performances unless the experts you are told me this is a mandatory feature.
Thanks for your attention and comments on this topic.
Erwan,
On Wed, 07 Nov 2012 21:48:17 +0100 Erwan Velu <[email protected]> wrote:
> Hi fellows,
>
> I'm been facing some lseek() troubles on a very light hardware (Atom E660) under heavy load (network + cpu + disk IOs). I'm using 3.2.32 on a 32bit Os with a local SSD as mass storage.
>
> If a do open a block device like sdb1 and lseek SEEK_SET in it, some unexpected latencies occurs.
> Using the same load, everything works perfectly by using contigous streams but once I do lseek it start to be laggy. I've been searching around for a while and finally found this message : https://lkml.org/lkml/2011/9/15/399 from Andy.
>
> The description was very similar to what I experienced but the patch from Andy was on to the fs layer.
>
> I've been looking the code for the block level layer and found the implementation is pretty different.
> http://lxr.linux.no/#linux+v3.2.33/fs/read_write.c#L69
> vs
> http://lxr.linux.no/#linux+v3.2.33/fs/block_dev.c#L353
>
> As I can see, we do first put the mutex, then i_size_read and then considering the kind of SEEK we want.
> The semantic changes from the read_write implementation where it does the locking only for SEEK_CUR and i_size_read isn't executed for SEEK_SET.
>
> So I really wonder if we shall rework this part to avoid the uncessary locking for all of them except SEEK_CUR and remove i_size_read from SEEK_SET. The i_size_read is also a matter as it does a memory barrier. On such low-end hardware I have, that could costs.
>
> I can work on it and validate its performances unless the experts you are told me this is a mandatory feature.
>
> Thanks for your attention and comments on this topic.
If your lseek()ing process is indeed blocking on i_mutex then something
else must be holding it. ie: there's some heavy I/O happening against
that device at the same time? Tell us more...
Another possibility is that the delay is not in lseek() but is actually
in the device open/close, doing lots of pagecache invalidation and/or
writeout. It used to be the case that blockdev close() would do a
heavyweight flush/invalidate, but I haven't checked lately.