Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230420112946.2869956-1-yukuai1@huaweicloud.com>
 <20230420112946.2869956-2-yukuai1@huaweicloud.com> <CAPhsuW5ifaGc47-vJWwbRyjgJHr3CJy+_zZ1wAL=FNxPOk-0WQ@mail.gmail.com>
 <b3393e4d-2fdb-41a6-54ba-fb564c484e56@huaweicloud.com>
In-Reply-To: <b3393e4d-2fdb-41a6-54ba-fb564c484e56@huaweicloud.com>
From:   Song Liu <song@kernel.org>
Date:   Mon, 24 Apr 2023 23:39:35 -0700
Message-ID: <CAPhsuW47GBgnofLrN8DVMkrBC=tVjhvoQroVZPRrMUHwGcOBLA@mail.gmail.com>
Subject: Re: [PATCH -next 1/8] md/raid10: prevent soft lockup while flush writes
To:     Yu Kuai <yukuai1@huaweicloud.com>
Cc:     neilb@suse.de, akpm@osdl.org, linux-raid@vger.kernel.org,
        linux-kernel@vger.kernel.org, yi.zhang@huawei.com,
        yangerkun@huawei.com, "yukuai (C)" <yukuai3@huawei.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Mon, Apr 24, 2023 at 11:16=E2=80=AFPM Yu Kuai <yukuai1@huaweicloud.com> =
wrote:
>
> Hi,
>
> =E5=9C=A8 2023/04/25 8:23, Song Liu =E5=86=99=E9=81=93:
> > On Thu, Apr 20, 2023 at 4:31=E2=80=AFAM Yu Kuai <yukuai1@huaweicloud.co=
m> wrote:
> >>
> >> From: Yu Kuai <yukuai3@huawei.com>
> >>
> >> Currently, there is no limit for raid1/raid10 plugged bio. While flush=
ing
> >> writes, raid1 has cond_resched() while raid10 doesn't, and too many
> >> writes can cause soft lockup.
> >>
> >> Follow up soft lockup can be triggered easily with writeback test for
> >> raid10 with ramdisks:
> >>
> >> watchdog: BUG: soft lockup - CPU#10 stuck for 27s! [md0_raid10:1293]
> >> Call Trace:
> >>   <TASK>
> >>   call_rcu+0x16/0x20
> >>   put_object+0x41/0x80
> >>   __delete_object+0x50/0x90
> >>   delete_object_full+0x2b/0x40
> >>   kmemleak_free+0x46/0xa0
> >>   slab_free_freelist_hook.constprop.0+0xed/0x1a0
> >>   kmem_cache_free+0xfd/0x300
> >>   mempool_free_slab+0x1f/0x30
> >>   mempool_free+0x3a/0x100
> >>   bio_free+0x59/0x80
> >>   bio_put+0xcf/0x2c0
> >>   free_r10bio+0xbf/0xf0
> >>   raid_end_bio_io+0x78/0xb0
> >>   one_write_done+0x8a/0xa0
> >>   raid10_end_write_request+0x1b4/0x430
> >>   bio_endio+0x175/0x320
> >>   brd_submit_bio+0x3b9/0x9b7 [brd]
> >>   __submit_bio+0x69/0xe0
> >>   submit_bio_noacct_nocheck+0x1e6/0x5a0
> >>   submit_bio_noacct+0x38c/0x7e0
> >>   flush_pending_writes+0xf0/0x240
> >>   raid10d+0xac/0x1ed0
> >
> > Is it possible to trigger this with a mdadm test?
> >
>
> The test I mentioned in patch 8 can trigger this problem reliablity, so
> I this add a new test can achieve this.

To be clear, by "mdadm test" I mean the tests included in mdadm:

https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/tests

Could you please try to add a test? If it works, we should add it to
mdadm.

Thanks,
Song