On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai <[email protected]> wrote:
>
> From: Yu Kuai <[email protected]>
>
> 'pers->prepare_suspend' is introduced to prevent a deadlock for raid456,
> this change prepares to clean this up in later patches while refactoring
> mddev_suspend(). Specifically allow reshape to make progress while
> waiting for 'active_io' to be 0.
Hi Kuai
From my side, I can't understand the comments. The change has
relationship with pers->prepare_suspend? And why this change can
affect reshape? If this change indeed can affect these two things, can
you explain more?
>
> Signed-off-by: Yu Kuai <[email protected]>
> ---
> drivers/md/md.c | 2 +-
> drivers/md/md.h | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 9d8dff9d923c..7fa311a14317 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -355,7 +355,7 @@ static DEFINE_SPINLOCK(all_mddevs_lock);
> */
> static bool is_suspended(struct mddev *mddev, struct bio *bio)
> {
> - if (is_md_suspended(mddev))
> + if (is_md_suspended(mddev) || percpu_ref_is_dying(&mddev->active_io))
If we use mddev->suspended to judge if the raid is suspended, it
should be enough? Because mddev->suspended must be true when active_io
is dying.
Best Regards
Xiao
> return true;
> if (bio_data_dir(bio) != WRITE)
> return false;
> diff --git a/drivers/md/md.h b/drivers/md/md.h
> index b628c292506e..fb3b123f16dd 100644
> --- a/drivers/md/md.h
> +++ b/drivers/md/md.h
> @@ -584,7 +584,7 @@ static inline bool md_is_rdwr(struct mddev *mddev)
>
> static inline bool is_md_suspended(struct mddev *mddev)
> {
> - return percpu_ref_is_dying(&mddev->active_io);
> + return READ_ONCE(mddev->suspended);
> }
>
> static inline int __must_check mddev_lock(struct mddev *mddev)
> --
> 2.39.2
>
Hi,
在 2023/09/20 16:46, Xiao Ni 写道:
> On Mon, Aug 28, 2023 at 10:04 AM Yu Kuai <[email protected]> wrote:
>>
>> From: Yu Kuai <[email protected]>
>>
>> 'pers->prepare_suspend' is introduced to prevent a deadlock for raid456,
>> this change prepares to clean this up in later patches while refactoring
>> mddev_suspend(). Specifically allow reshape to make progress while
>> waiting for 'active_io' to be 0.
>
> Hi Kuai
>
>>From my side, I can't understand the comments. The change has
> relationship with pers->prepare_suspend? And why this change can
> affect reshape? If this change indeed can affect these two things, can
> you explain more?
First of all, 'prepare_suspend' is used to fix a deadlock in raid456:
1) suspend is waiting for normal io to be done.
mddev_suspend
mddev->suspended++ -> new sync_thread can't start
percpu_ref_kill(active_io)
wait_event(percpu_ref_is_zero(active_io))
2) normal io is waiting for reshape to make progress.
3) reshape is waiting for suspended array to be resumed.
md_check_recovery
if (is_md_suspended(mddev))
return
Then prepare_suspend will failed the io that is waiting for reshape to
make progress:
mddev_suspend
mddev->suspended++
percpu_ref_kill(active_io)
-> new io will be stuck in md_handle_request
pers->prepare_suspend() -> raid5_prepare_suspend
-> wake_up(wait_for_overlap)
// woke up
raid5_make_request
make_stripe_request
!reshape_inprogress(mddev) && reshape_disabled(mddev)
// return io error for the io that is waiting for
// reshape to make progress
wait_event(percpu_ref_is_zero(active_io))
With this patch and the new api to suspend array:
mddev_suspend
percpu_ref_kill(active_io)
wait_event(percpu_ref_is_zero(active_io))
-> while waiting for normal io to be done, new sync_thread can still
start, and reshape can still make progress.
mddev->suspended++
>
>>
>> Signed-off-by: Yu Kuai <[email protected]>
>> ---
>> drivers/md/md.c | 2 +-
>> drivers/md/md.h | 2 +-
>> 2 files changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/md/md.c b/drivers/md/md.c
>> index 9d8dff9d923c..7fa311a14317 100644
>> --- a/drivers/md/md.c
>> +++ b/drivers/md/md.c
>> @@ -355,7 +355,7 @@ static DEFINE_SPINLOCK(all_mddevs_lock);
>> */
>> static bool is_suspended(struct mddev *mddev, struct bio *bio)
>> {
>> - if (is_md_suspended(mddev))
>> + if (is_md_suspended(mddev) || percpu_ref_is_dying(&mddev->active_io))
>
> If we use mddev->suspended to judge if the raid is suspended, it
> should be enough? Because mddev->suspended must be true when active_io
> is dying.
In the new api, active_io is killed before increasing suspended, and the
difference is that the timing that array is suspended will be delayed
from the start of mddev_suspend() to when all dispatched io is done.
I think this is OK because this doesn't change behaviour when
mddev_suspend() returns.
Thanks,
Kuai
`
>
> Best Regards
> Xiao
>> return true;
>> if (bio_data_dir(bio) != WRITE)
>> return false;
>> diff --git a/drivers/md/md.h b/drivers/md/md.h
>> index b628c292506e..fb3b123f16dd 100644
>> --- a/drivers/md/md.h
>> +++ b/drivers/md/md.h
>> @@ -584,7 +584,7 @@ static inline bool md_is_rdwr(struct mddev *mddev)
>>
>> static inline bool is_md_suspended(struct mddev *mddev)
>> {
>> - return percpu_ref_is_dying(&mddev->active_io);
>> + return READ_ONCE(mddev->suspended);
>> }
>>
>> static inline int __must_check mddev_lock(struct mddev *mddev)
>> --
>> 2.39.2
>>
>
> .
>