Currently, active_aligned_reads and nr_pending used as counters are atomic types.
Therefore, when inc/dec in a multi-core results in conflicts.
To solve this problem, use "percpu_ref" counters that can avoid conflicts and maintain consistency.
Switch modes of percpu_ref to achieve both consistency and conflict avoidance.
During normal operations such as inc/dec, it operates as percpu mode.
When consistency is required, it operates as atomic mode.
The operations that require consistency are as follows:
$B!&(BZero check for the counter
$B!&(BAll operations in RAID 1/10
Patches 1, 3, 6 change active_aligned_reads, and patches 2, 4, 5, 7 to 11 change nr_pending.
nr_pending temporarily switch from percpu mode to atomic mode in patch 7.
This is to reduce the amount of changes from patches 8 to 10.
Finally, nr_pending switch from atomic mode to percpu mode in patch 11.
Keisuke TADA (11):
add infra for active_aligned_reads changes
add infra for nr_pending changes
workaround for inconsistency of config state in takeover
minimize execution of zero check for nr_pending
match the type of variables to percpu_ref
avoid conflicts in active_aligned_reads operations
change the type of nr_pending from atomic_t to percpu_ref
add atomic mode switching in RAID 1/10
add atomic mode switching when removing disk
add atomic mode switching when I/O completion
avoid conflicts in nr_pending operations
drivers/md/md-bitmap.c | 2 +-
drivers/md/md.c | 40 +++++++++++++++---
drivers/md/md.h | 62 ++++++++++++++++++++++++----
drivers/md/raid1.c | 37 +++++++++++------
drivers/md/raid10.c | 60 ++++++++++++++++-----------
drivers/md/raid5-cache.c | 4 +-
drivers/md/raid5.c | 89 +++++++++++++++++++++++++++-------------
drivers/md/raid5.h | 17 +++++++-
8 files changed, 230 insertions(+), 81 deletions(-)
base-commit: fcf3f7e2fc8a53a6140beee46ec782a4c88e4744
--
2.34.1
Hi,
$B:_(B 2024/03/26 18:29, tada keisuke $B<LF;(B:
> Currently, active_aligned_reads and nr_pending used as counters are atomic types.
> Therefore, when inc/dec in a multi-core results in conflicts.
I don't uderstand what's the problem here, what's wrong with atomic
types in a multi-core system?
Thanks,
Kuai
>> Currently, active_aligned_reads and nr_pending used as counters are atomic types.
>> Therefore, when inc/dec in a multi-core results in conflicts.
>
> I don't uderstand what's the problem here, what's wrong with atomic
> types in a multi-core system?
Hi Kuai
This patch is not a fix for the problem, but an improvement.
This patch makes the counter inc/dec faster.
For the same reason, active_io was changed from atomic_t to percpu_ref.
https://marc.info/?l=linux-raid&m=167514225920879
Thanks,
Keisuke
Thanks for the patchset!
On Tue, Mar 26, 2024 at 3:29 AM tada keisuke <[email protected]> wrote:
>
> Currently, active_aligned_reads and nr_pending used as counters are atomic types.
> Therefore, when inc/dec in a multi-core results in conflicts.
>
> To solve this problem, use "percpu_ref" counters that can avoid conflicts and maintain consistency.
>
> Switch modes of percpu_ref to achieve both consistency and conflict avoidance.
> During normal operations such as inc/dec, it operates as percpu mode.
> When consistency is required, it operates as atomic mode.
> The operations that require consistency are as follows:
> ・Zero check for the counter
> ・All operations in RAID 1/10
IIUC, the patchset is to resolve race conditions and thus improve performance.
If this is the case, could you please include some performance numbers in the
next version?
Thanks,
Song