Hi All,
In ext4 punch hole, we realized that the punch hole operation needs to
be done under i_mutex just like truncate. i_mutex for truncate is held
in the vfs layer, so we dont need to lock it at the file system layer,
but vfs does not lock i_mutex for fallocate. We can lock i_mutex for
fallocate at the fs layer, but question was raised then: should i_mutex
for fallocate be held in the vfs layer instead? I do not know if other
file systems need i_mutex to be locked for fallocate, or if they might
be locking it already, so I am doing some investigating on this idea,
and also the appropriate use of i_mutex in general. Can someone provide
some insight this topic? Thx!
Allison Henderson
Hi Allison,
there are races between read and punch hole too. After pages are
released and before punch hole hold i_data_sem, a page can be mapped
by read, then write can be done on the page.
Maybe we should separate punch hole from fallocate, and solve the
races in vfs layer.
Yongqiang.
On Thu, Sep 1, 2011 at 8:33 AM, Allison Henderson
<[email protected]> wrote:
> Hi All,
>
> In ext4 punch hole, we realized that the punch hole operation needs to be
> done under i_mutex just like truncate. ?i_mutex for truncate is held in the
> vfs layer, so we dont need to lock it at the file system layer, but vfs does
> not lock i_mutex for fallocate. ?We can lock i_mutex for fallocate at the fs
> layer, but question was raised then: should i_mutex for fallocate be held in
> the vfs layer instead? ?I do not know if other file systems need i_mutex to
> be locked for fallocate, or if they might be locking it already, so I am
> doing some investigating on this idea, and also the appropriate use of
> i_mutex in general. ?Can someone provide some insight this topic? ?Thx!
>
> Allison Henderson
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at ?http://www.tux.org/lkml/
>
--
Best Wishes
Yongqiang Yang
Oh, I meant for this to go to linux-fsdevel instead of linux-kernel, but
all feedback is welcome! :)
On 08/31/2011 05:33 PM, Allison Henderson wrote:
> Hi All,
>
> In ext4 punch hole, we realized that the punch hole operation needs to
> be done under i_mutex just like truncate. i_mutex for truncate is held
> in the vfs layer, so we dont need to lock it at the file system layer,
> but vfs does not lock i_mutex for fallocate. We can lock i_mutex for
> fallocate at the fs layer, but question was raised then: should i_mutex
> for fallocate be held in the vfs layer instead? I do not know if other
> file systems need i_mutex to be locked for fallocate, or if they might
> be locking it already, so I am doing some investigating on this idea,
> and also the appropriate use of i_mutex in general. Can someone provide
> some insight this topic? Thx!
>
> Allison Henderson
On Wed, Aug 31, 2011 at 05:33:25PM -0700, Allison Henderson wrote:
> Hi All,
>
> In ext4 punch hole, we realized that the punch hole operation needs
> to be done under i_mutex just like truncate. i_mutex for truncate
> is held in the vfs layer, so we dont need to lock it at the file
> system layer, but vfs does not lock i_mutex for fallocate. We can
> lock i_mutex for fallocate at the fs layer, but question was raised
> then: should i_mutex for fallocate be held in the vfs layer instead?
No.
> I do not know if other file systems need i_mutex to be locked for
> fallocate,
For one, XFS does not require i_mutex to be held for any extent
manipulation of any kind (allocation, truncation, hole punch,
unwritten extent conversion, etc).
Hence the current structure of having the filesystem take i_mutex if
it needs it to protect allocations against races is appropriate.
Cheers,
Dave.
--
Dave Chinner
[email protected]
On Wed, Aug 31, 2011 at 05:33:25PM -0700, Allison Henderson wrote:
> Hi All,
>
> In ext4 punch hole, we realized that the punch hole operation needs
> to be done under i_mutex just like truncate. i_mutex for truncate
> is held in the vfs layer, so we dont need to lock it at the file
> system layer, but vfs does not lock i_mutex for fallocate. We can
> lock i_mutex for fallocate at the fs layer, but question was raised
> then: should i_mutex for fallocate be held in the vfs layer instead?
> I do not know if other file systems need i_mutex to be locked for
> fallocate, or if they might be locking it already, so I am doing
> some investigating on this idea, and also the appropriate use of
> i_mutex in general. Can someone provide some insight this topic?
Don't do it.
i_mutex is already overloaded, and this does not fit into any
of the somewhat reasonable uses cases for it, which are:
a) for directories the VFS uses it to protect the tree topology
b) for regular files all generic I/O code currently uses it to
serialize writers.
c) the VFS uses it around truncate, and setxattr updates
d) filesystems abuse it for internal metadata in various places
As you can see right now we do not hold it over any file operation,
and I'm absolutely against adding that. I'd rather untange the
current uses, specificly:
- push synchronization of setattr into the filesystems
- push synchronization of xattr write operations into the filesystems
- move the read/write synchronization to a separate shared/exclusive
lock like it's already done in XFS, and like Lukas proposed for
ext4. This fixes the Posix compliance corner cases about reads
beeing atomic vs writes, simplifies direct I/O locking a lot,
and allows for more parallel direct I/O support like XFS supports.
- try to get rid of the abuses inside filesystems as much as possible.
On 09/01/2011 12:31 AM, Christoph Hellwig wrote:
> On Wed, Aug 31, 2011 at 05:33:25PM -0700, Allison Henderson wrote:
>> Hi All,
>>
>> In ext4 punch hole, we realized that the punch hole operation needs
>> to be done under i_mutex just like truncate. i_mutex for truncate
>> is held in the vfs layer, so we dont need to lock it at the file
>> system layer, but vfs does not lock i_mutex for fallocate. We can
>> lock i_mutex for fallocate at the fs layer, but question was raised
>> then: should i_mutex for fallocate be held in the vfs layer instead?
>> I do not know if other file systems need i_mutex to be locked for
>> fallocate, or if they might be locking it already, so I am doing
>> some investigating on this idea, and also the appropriate use of
>> i_mutex in general. Can someone provide some insight this topic?
>
> Don't do it.
>
> i_mutex is already overloaded, and this does not fit into any
> of the somewhat reasonable uses cases for it, which are:
>
> a) for directories the VFS uses it to protect the tree topology
> b) for regular files all generic I/O code currently uses it to
> serialize writers.
> c) the VFS uses it around truncate, and setxattr updates
> d) filesystems abuse it for internal metadata in various places
>
> As you can see right now we do not hold it over any file operation,
> and I'm absolutely against adding that. I'd rather untange the
> current uses, specificly:
>
> - push synchronization of setattr into the filesystems
> - push synchronization of xattr write operations into the filesystems
> - move the read/write synchronization to a separate shared/exclusive
> lock like it's already done in XFS, and like Lukas proposed for
> ext4. This fixes the Posix compliance corner cases about reads
> beeing atomic vs writes, simplifies direct I/O locking a lot,
> and allows for more parallel direct I/O support like XFS supports.
> - try to get rid of the abuses inside filesystems as much as possible.
>
Alrighty, this helps explain things! Thx all for the feedback! :)
On Wed, Aug 31, 2011 at 06:12:04PM -0700, Allison Henderson wrote:
> Oh, I meant for this to go to linux-fsdevel instead of linux-kernel, but
> all feedback is welcome! :)
>
> On 08/31/2011 05:33 PM, Allison Henderson wrote:
>> Hi All,
>>
>> In ext4 punch hole, we realized that the punch hole operation needs to
>> be done under i_mutex just like truncate. i_mutex for truncate is held
>> in the vfs layer, so we dont need to lock it at the file system layer,
>> but vfs does not lock i_mutex for fallocate. We can lock i_mutex for
>> fallocate at the fs layer, but question was raised then: should i_mutex
>> for fallocate be held in the vfs layer instead? I do not know if other
>> file systems need i_mutex to be locked for fallocate, or if they might
>> be locking it already, so I am doing some investigating on this idea,
>> and also the appropriate use of i_mutex in general. Can someone provide
>> some insight this topic? Thx!
>>
Btrfs has range locking for our extents, so we don't really need to be holding
the i_mutex, even tho it appears we do it anyway. So I'd rather this not be
moved to VFS for us who can do more fine grained locking. Thanks,
Josef