2021-09-06 03:08:21

by Baptiste Lepers

Subject: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

Per comment, record_root_in_trans orders the writes of the root->state
and root->last_trans:
set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
smp_wmb();
root->last_trans = trans->transid;

But the barrier that enforces the order on the read side is misplaced:
smp_rmb(); <-- misplaced
if (root->last_trans == trans->transid &&
<-- missing barrier here -->
!test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))

This patches fixes the ordering and wraps the racy accesses with
READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.

Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
Signed-off-by: Baptiste Lepers <[email protected]>
---
fs/btrfs/transaction.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 14b9fdc8aaa9..a609222e6704 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
(unsigned long)root->root_key.objectid,
BTRFS_ROOT_TRANS_TAG);
spin_unlock(&fs_info->fs_roots_radix_lock);
- root->last_trans = trans->transid;
+ WRITE_ONCE(root->last_trans, trans->transid);

/* this is pretty tricky. We don't want to
* take the relocation lock in btrfs_record_root_in_trans
@@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
struct btrfs_root *root)
{
struct btrfs_fs_info *fs_info = root->fs_info;
- int ret;
+ int ret, last_trans;

if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
return 0;
@@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
* see record_root_in_trans for comments about IN_TRANS_SETUP usage
* and barriers
*/
+ last_trans = READ_ONCE(root->last_trans);
smp_rmb();
- if (root->last_trans == trans->transid &&
+ if (last_trans == trans->transid &&
!test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
return 0;

--
2.17.1

2021-09-06 12:38:40

by David Sterba

Subject: Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

On Mon, Sep 06, 2021 at 11:25:59AM +1000, Baptiste Lepers wrote:
> Per comment, record_root_in_trans orders the writes of the root->state
> and root->last_trans:
> set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
> smp_wmb();
> root->last_trans = trans->transid;
>
> But the barrier that enforces the order on the read side is misplaced:
> smp_rmb(); <-- misplaced
> if (root->last_trans == trans->transid &&
> <-- missing barrier here -->
> !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>
> This patches fixes the ordering and wraps the racy accesses with
> READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
>
> Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
> Signed-off-by: Baptiste Lepers <[email protected]>
> ---
> fs/btrfs/transaction.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
> index 14b9fdc8aaa9..a609222e6704 100644
> --- a/fs/btrfs/transaction.c
> +++ b/fs/btrfs/transaction.c
> @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
> (unsigned long)root->root_key.objectid,
> BTRFS_ROOT_TRANS_TAG);
> spin_unlock(&fs_info->fs_roots_radix_lock);
> - root->last_trans = trans->transid;
> + WRITE_ONCE(root->last_trans, trans->transid);
>
> /* this is pretty tricky. We don't want to
> * take the relocation lock in btrfs_record_root_in_trans
> @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> struct btrfs_root *root)
> {
> struct btrfs_fs_info *fs_info = root->fs_info;
> - int ret;
> + int ret, last_trans;
>
> if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
> return 0;
> @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> * see record_root_in_trans for comments about IN_TRANS_SETUP usage
> * and barriers
> */
> + last_trans = READ_ONCE(root->last_trans);
> smp_rmb();
> - if (root->last_trans == trans->transid &&
> + if (last_trans == trans->transid &&
> !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))

Aren't the smp_rmb barriers supposed to be used before the condition?

2021-09-07 00:47:08

by Baptiste Lepers

Subject: Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

No, they need to be between the reads to have an effect. See
https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP
BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You
will see that the barriers are always between the ordered reads and
not before.

I think that Paul, the barrier guru, can confirm that the barrier was
misplaced in the original code? :)


On Tue, Sep 7, 2021 at 10:43 AM Baptiste Lepers
<[email protected]> wrote:
>
>
>
> On Mon, Sep 6, 2021 at 10:27 PM David Sterba <[email protected]> wrote:
>>
>> On Mon, Sep 06, 2021 at 11:25:59AM +1000, Baptiste Lepers wrote:
>> > Per comment, record_root_in_trans orders the writes of the root->state
>> > and root->last_trans:
>> > set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
>> > smp_wmb();
>> > root->last_trans = trans->transid;
>> >
>> > But the barrier that enforces the order on the read side is misplaced:
>> > smp_rmb(); <-- misplaced
>> > if (root->last_trans == trans->transid &&
>> > <-- missing barrier here -->
>> > !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>> >
>> > This patches fixes the ordering and wraps the racy accesses with
>> > READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
>> >
>> > Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
>> > Signed-off-by: Baptiste Lepers <[email protected]>
>> > ---
>> > fs/btrfs/transaction.c | 7 ++++---
>> > 1 file changed, 4 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
>> > index 14b9fdc8aaa9..a609222e6704 100644
>> > --- a/fs/btrfs/transaction.c
>> > +++ b/fs/btrfs/transaction.c
>> > @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
>> > (unsigned long)root->root_key.objectid,
>> > BTRFS_ROOT_TRANS_TAG);
>> > spin_unlock(&fs_info->fs_roots_radix_lock);
>> > - root->last_trans = trans->transid;
>> > + WRITE_ONCE(root->last_trans, trans->transid);
>> >
>> > /* this is pretty tricky. We don't want to
>> > * take the relocation lock in btrfs_record_root_in_trans
>> > @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>> > struct btrfs_root *root)
>> > {
>> > struct btrfs_fs_info *fs_info = root->fs_info;
>> > - int ret;
>> > + int ret, last_trans;
>> >
>> > if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
>> > return 0;
>> > @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
>> > * see record_root_in_trans for comments about IN_TRANS_SETUP usage
>> > * and barriers
>> > */
>> > + last_trans = READ_ONCE(root->last_trans);
>> > smp_rmb();
>> > - if (root->last_trans == trans->transid &&
>> > + if (last_trans == trans->transid &&
>> > !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>>
>> Aren't the smp_rmb barriers supposed to be used before the condition?
>
>
> No, they need to be between the reads to have an effect. See https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You will see that the barriers are always between the ordered reads and not before.
>
> I think that Paul, the barrier guru, can confirm that the barrier was misplaced in the original code? :)

2021-09-16 03:48:09

by Baptiste Lepers

Subject: Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

Just curious about the status of this patch. :) Let me know if you
need further information.

Thanks!

On Tue, Sep 7, 2021 at 10:44 AM Baptiste Lepers
<[email protected]> wrote:
>
> No, they need to be between the reads to have an effect. See
> https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP
> BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You
> will see that the barriers are always between the ordered reads and
> not before.
>
> I think that Paul, the barrier guru, can confirm that the barrier was
> misplaced in the original code? :)
>
>
> On Tue, Sep 7, 2021 at 10:43 AM Baptiste Lepers
> <[email protected]> wrote:
> >
> >
> >
> > On Mon, Sep 6, 2021 at 10:27 PM David Sterba <[email protected]> wrote:
> >>
> >> On Mon, Sep 06, 2021 at 11:25:59AM +1000, Baptiste Lepers wrote:
> >> > Per comment, record_root_in_trans orders the writes of the root->state
> >> > and root->last_trans:
> >> > set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
> >> > smp_wmb();
> >> > root->last_trans = trans->transid;
> >> >
> >> > But the barrier that enforces the order on the read side is misplaced:
> >> > smp_rmb(); <-- misplaced
> >> > if (root->last_trans == trans->transid &&
> >> > <-- missing barrier here -->
> >> > !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
> >> >
> >> > This patches fixes the ordering and wraps the racy accesses with
> >> > READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
> >> >
> >> > Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
> >> > Signed-off-by: Baptiste Lepers <[email protected]>
> >> > ---
> >> > fs/btrfs/transaction.c | 7 ++++---
> >> > 1 file changed, 4 insertions(+), 3 deletions(-)
> >> >
> >> > diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
> >> > index 14b9fdc8aaa9..a609222e6704 100644
> >> > --- a/fs/btrfs/transaction.c
> >> > +++ b/fs/btrfs/transaction.c
> >> > @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
> >> > (unsigned long)root->root_key.objectid,
> >> > BTRFS_ROOT_TRANS_TAG);
> >> > spin_unlock(&fs_info->fs_roots_radix_lock);
> >> > - root->last_trans = trans->transid;
> >> > + WRITE_ONCE(root->last_trans, trans->transid);
> >> >
> >> > /* this is pretty tricky. We don't want to
> >> > * take the relocation lock in btrfs_record_root_in_trans
> >> > @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> >> > struct btrfs_root *root)
> >> > {
> >> > struct btrfs_fs_info *fs_info = root->fs_info;
> >> > - int ret;
> >> > + int ret, last_trans;
> >> >
> >> > if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
> >> > return 0;
> >> > @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> >> > * see record_root_in_trans for comments about IN_TRANS_SETUP usage
> >> > * and barriers
> >> > */
> >> > + last_trans = READ_ONCE(root->last_trans);
> >> > smp_rmb();
> >> > - if (root->last_trans == trans->transid &&
> >> > + if (last_trans == trans->transid &&
> >> > !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
> >>
> >> Aren't the smp_rmb barriers supposed to be used before the condition?
> >
> >
> > No, they need to be between the reads to have an effect. See https://www.kernel.org/doc/Documentation/memory-barriers.txt §SMP BARRIER PAIRING ("When dealing with CPU-CPU interactions..."). You will see that the barriers are always between the ordered reads and not before.
> >
> > I think that Paul, the barrier guru, can confirm that the barrier was misplaced in the original code? :)

2021-09-16 09:33:26

by Filipe Manana

Subject: Re: [PATCH] btrfs: transaction: Fix misplaced barrier in btrfs_record_root_in_trans

On Mon, Sep 6, 2021 at 2:38 AM Baptiste Lepers
<[email protected]> wrote:
>
> Per comment, record_root_in_trans orders the writes of the root->state
> and root->last_trans:
> set_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state);
> smp_wmb();
> root->last_trans = trans->transid;
>
> But the barrier that enforces the order on the read side is misplaced:
> smp_rmb(); <-- misplaced
> if (root->last_trans == trans->transid &&
> <-- missing barrier here -->
> !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
>
> This patches fixes the ordering and wraps the racy accesses with
> READ_ONCE and WRITE_ONCE calls to avoid load/store tearing.
>
> Fixes: 7585717f304f5 ("Btrfs: fix relocation races")
> Signed-off-by: Baptiste Lepers <[email protected]>
> ---
> fs/btrfs/transaction.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
> index 14b9fdc8aaa9..a609222e6704 100644
> --- a/fs/btrfs/transaction.c
> +++ b/fs/btrfs/transaction.c
> @@ -437,7 +437,7 @@ static int record_root_in_trans(struct btrfs_trans_handle *trans,
> (unsigned long)root->root_key.objectid,
> BTRFS_ROOT_TRANS_TAG);
> spin_unlock(&fs_info->fs_roots_radix_lock);
> - root->last_trans = trans->transid;
> + WRITE_ONCE(root->last_trans, trans->transid);
>
> /* this is pretty tricky. We don't want to
> * take the relocation lock in btrfs_record_root_in_trans
> @@ -489,7 +489,7 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> struct btrfs_root *root)
> {
> struct btrfs_fs_info *fs_info = root->fs_info;
> - int ret;
> + int ret, last_trans;

last_trans should be u64, as root->last_trans is a u64.

Other than that it looks good to me.
Thanks.

>
> if (!test_bit(BTRFS_ROOT_SHAREABLE, &root->state))
> return 0;
> @@ -498,8 +498,9 @@ int btrfs_record_root_in_trans(struct btrfs_trans_handle *trans,
> * see record_root_in_trans for comments about IN_TRANS_SETUP usage
> * and barriers
> */
> + last_trans = READ_ONCE(root->last_trans);
> smp_rmb();
> - if (root->last_trans == trans->transid &&
> + if (last_trans == trans->transid &&
> !test_bit(BTRFS_ROOT_IN_TRANS_SETUP, &root->state))
> return 0;
>
> --
> 2.17.1
>


--
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”