Alex Shi found a regression when doing ffsb test. The test has several threads,
and each thread creates a small file, write to it and then delete it. ffsb
reports about 20% regression and Alex bisected it to 43d2932d88e4. The test
will call __mark_inode_dirty 3 times. without this commit, we only take
inode_lock one time, while with it, we take the lock 3 times with flags (
I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased
too much. Below proposed patch fixes it.
fs is allocating blocks, which usually means file writes and the inode
will be dirtied soon. We fully dirty the inode to reduce some inode_lock
contention in several calls of __mark_inode_dirty.
Signed-off-by: Shaohua Li <[email protected]>
Signed-off-by: Alex Shi <[email protected]>
diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h
index d50ba85..8dd2cc3 100644
--- a/include/linux/quotaops.h
+++ b/include/linux/quotaops.h
@@ -275,7 +275,7 @@ static inline int dquot_alloc_space(struct inode *inode, qsize_t nr)
ret = dquot_alloc_space_nodirty(inode, nr);
if (!ret)
- mark_inode_dirty_sync(inode);
+ mark_inode_dirty(inode);
return ret;
}
Hi,
On Fri 20-08-10 16:49:43, Shaohua Li wrote:
> Alex Shi found a regression when doing ffsb test. The test has several threads,
> and each thread creates a small file, write to it and then delete it. ffsb
> reports about 20% regression and Alex bisected it to 43d2932d88e4. The test
> will call __mark_inode_dirty 3 times. without this commit, we only take
> inode_lock one time, while with it, we take the lock 3 times with flags (
> I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased
> too much. Below proposed patch fixes it.
Thanks for the analysis and the patch! With which filesystem have you
measured the results? And what kind of machine it was?
> fs is allocating blocks, which usually means file writes and the inode
> will be dirtied soon. We fully dirty the inode to reduce some inode_lock
> contention in several calls of __mark_inode_dirty.
Well, this is rather a workaround for a poor handling of inode dirty
bits. BTW, I think Nick's VFS scalability patches address inode_lock
contention as well so with them the contention you see should be reduced.
Anyway, I will take this patch for the time before inode_lock
contention improves and add a proper comment about this before
mark_inode_dirty.
Honza
>
> Signed-off-by: Shaohua Li <[email protected]>
> Signed-off-by: Alex Shi <[email protected]>
>
> diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h
> index d50ba85..8dd2cc3 100644
> --- a/include/linux/quotaops.h
> +++ b/include/linux/quotaops.h
> @@ -275,7 +275,7 @@ static inline int dquot_alloc_space(struct inode *inode, qsize_t nr)
>
> ret = dquot_alloc_space_nodirty(inode, nr);
> if (!ret)
> - mark_inode_dirty_sync(inode);
> + mark_inode_dirty(inode);
> return ret;
> }
>
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Tue 24-08-10 16:23:59, Jan Kara wrote:
> Hi,
>
> On Fri 20-08-10 16:49:43, Shaohua Li wrote:
> > Alex Shi found a regression when doing ffsb test. The test has several threads,
> > and each thread creates a small file, write to it and then delete it. ffsb
> > reports about 20% regression and Alex bisected it to 43d2932d88e4. The test
> > will call __mark_inode_dirty 3 times. without this commit, we only take
> > inode_lock one time, while with it, we take the lock 3 times with flags (
> > I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased
> > too much. Below proposed patch fixes it.
> Thanks for the analysis and the patch! With which filesystem have you
> measured the results? And what kind of machine it was?
>
> > fs is allocating blocks, which usually means file writes and the inode
> > will be dirtied soon. We fully dirty the inode to reduce some inode_lock
> > contention in several calls of __mark_inode_dirty.
> Well, this is rather a workaround for a poor handling of inode dirty
> bits. BTW, I think Nick's VFS scalability patches address inode_lock
> contention as well so with them the contention you see should be reduced.
> Anyway, I will take this patch for the time before inode_lock
> contention improves and add a proper comment about this before
> mark_inode_dirty.
Attached is a version of the patch that is in my tree now.
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Tue, Aug 24, 2010 at 10:23:59PM +0800, Jan Kara wrote:
> Hi,
>
> On Fri 20-08-10 16:49:43, Shaohua Li wrote:
> > Alex Shi found a regression when doing ffsb test. The test has several threads,
> > and each thread creates a small file, write to it and then delete it. ffsb
> > reports about 20% regression and Alex bisected it to 43d2932d88e4. The test
> > will call __mark_inode_dirty 3 times. without this commit, we only take
> > inode_lock one time, while with it, we take the lock 3 times with flags (
> > I_DIRTY_SYNC,I_DIRTY_PAGES,I_DIRTY). Perf shows the lock contention increased
> > too much. Below proposed patch fixes it.
> Thanks for the analysis and the patch! With which filesystem have you
> measured the results? And what kind of machine it was?
it's ext3 and 24 CPU system with two sockets.
> > fs is allocating blocks, which usually means file writes and the inode
> > will be dirtied soon. We fully dirty the inode to reduce some inode_lock
> > contention in several calls of __mark_inode_dirty.
> Well, this is rather a workaround for a poor handling of inode dirty
> bits. BTW, I think Nick's VFS scalability patches address inode_lock
> contention as well so with them the contention you see should be reduced.
yep, the VFS scalability patch might reduce this.
> Anyway, I will take this patch for the time before inode_lock
> contention improves and add a proper comment about this before
> mark_inode_dirty.
Thanks.