Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20181211103616.22137-1-anders.roxell@linaro.org> <613667c4-f850-d88c-7a2e-76dda5d966f5@arm.com>
In-Reply-To: <613667c4-f850-d88c-7a2e-76dda5d966f5@arm.com>
From:   Anders Roxell <anders.roxell@linaro.org>
Date:   Wed, 12 Dec 2018 19:15:36 +0100
Message-ID: <CADYN=9+FpcMUqtLgkvWQnf2uQOgS3VhwCLHmcriD_5LSVBZFKQ@mail.gmail.com>
Subject: Re: [PATCH] dma-debug: fix soft lockup when a lot of debug are enabled
To:     robin.murphy@arm.com
Cc:     hch@lst.de, Marek Szyprowski <m.szyprowski@samsung.com>,
        iommu@lists.linux-foundation.org,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        rostedt@goodmis.org
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, 11 Dec 2018 at 12:51, Robin Murphy <robin.murphy@arm.com> wrote:
>
> Hi Anders,

Hi Robin,

>
> On 11/12/2018 10:36, Anders Roxell wrote:
> > When running a kernel in qemu with enough debugging options (slub-debug,
> > ftrace, kcov, kasan, ubsan, ...) enabled together, that results in a
> > slow initcall. So a 'watchdog: BUG: soft lockup' happens:
> >
> > [   44.105619] Call trace:
> > [   44.106709]  __slab_alloc+0x70/0x88
> > [   44.107757]  kmem_cache_alloc_trace+0x13c/0x590
> > [   44.108764]  prealloc_memory+0xb8/0x240
> > [   44.109768]  dma_debug_init+0x174/0x1f0
> > [   44.110725]  do_one_initcall+0x430/0x8b0
> > [   44.111719]  do_initcall_level+0x548/0x5b4
> > [   44.112672]  do_initcalls+0x28/0x4c
> > [   44.113628]  do_basic_setup+0x34/0x3c
> > [   44.114559]  kernel_init_freeable+0x194/0x24c
> > [   44.115570]  kernel_init+0x24/0x18c
> > [   44.116498]  ret_from_fork+0x10/0x18
> >
> > Rework to call cond_resched(), in the function prealloc_memory()'s
> > for-loop.
>
> Funnily enough I've just done a bit of work in this area which should
> make allocation a lot more efficient - would you mind giving this series
> a spin on the same setup to see how much it helps the performance?
>
> https://lore.kernel.org/lkml/cover.1544447601.git.robin.murphy@arm.com/

I tried it on todays next tag and couldn't reproduce the soft lockup.

Cheers,
Anders

>
> Thanks,
> Robin.
>
> >
> > Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
> > ---
> >   kernel/dma/debug.c | 1 +
> >   1 file changed, 1 insertion(+)
> >
> > diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c
> > index 231ca4628062..2abdc265aec4 100644
> > --- a/kernel/dma/debug.c
> > +++ b/kernel/dma/debug.c
> > @@ -767,6 +767,7 @@ static int prealloc_memory(u32 num_entries)
> >       int i;
> >
> >       for (i = 0; i < num_entries; ++i) {
> > +             cond_resched();
> >               entry = kzalloc(sizeof(*entry), GFP_KERNEL);
> >               if (!entry)
> >                       goto out_err;
> >