Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
From:   Qian Cai <cai@lca.pw>
Mime-Version: 1.0 (1.0)
Subject: Re: [PATCH v3 1/3] mm: kmemleak: Make the tool tolerant to struct scan_area allocation failures
Date:   Thu, 3 Oct 2019 04:36:39 -0400
Message-Id: <934700AE-CD78-43B8-A59B-2E968DA44DBB@lca.pw>
References: <2ac37341-097e-17a2-fb6b-7912da9fa38e@ozlabs.ru>
Cc:     Catalin Marinas <catalin.marinas@arm.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org,
        Andrew Morton <akpm@linux-foundation.org>,
        Michal Hocko <mhocko@kernel.org>,
        Matthew Wilcox <willy@infradead.org>
In-Reply-To: <2ac37341-097e-17a2-fb6b-7912da9fa38e@ozlabs.ru>
To:     Alexey Kardashevskiy <aik@ozlabs.ru>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> On Oct 3, 2019, at 2:13 AM, Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>=20
> I came across this one while bisecting sudden drop in throughput of a 100G=
bit Mellanox CX4 ethernet card in a PPC POWER9
> system, the speed dropped from 100Gbit to about 40Gbit. Bisect pointed at d=
ba82d943177, this are the relevant config
> options:
>=20
> [fstn1-p1 kernel]$ grep KMEMLEAK ~/pbuild/kernel-le-4g/.config
> CONFIG_HAVE_DEBUG_KMEMLEAK=3Dy
> CONFIG_DEBUG_KMEMLEAK=3Dy
> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=3D16000
> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
> CONFIG_DEBUG_KMEMLEAK_AUTO_SCAN=3Dy
>=20
> Setting CONFIG_DEBUG_KMEMLEAK_MEM_POOL_SIZE=3D400 or even 4000 (this is wh=
at KMEMLEAK_EARLY_LOG_SIZE is now in the master)
> produces soft lockups on the recent upstream (sha1 a3c0e7b1fe1f):
>=20
> [c000001fde64fb60] [c000000000c24ed4] _raw_write_unlock_irqrestore+0x54/0x=
70
> [c000001fde64fb90] [c0000000004117e4] find_and_remove_object+0xa4/0xd0
> [c000001fde64fbe0] [c000000000411c74] delete_object_full+0x24/0x50
> [c000001fde64fc00] [c000000000411d28] __kmemleak_do_cleanup+0x88/0xd0
> [c000001fde64fc40] [c00000000012a1a4] process_one_work+0x374/0x6a0
> [c000001fde64fd20] [c00000000012a548] worker_thread+0x78/0x5a0
> [c000001fde64fdb0] [c000000000135508] kthread+0x198/0x1a0
> [c000001fde64fe20] [c00000000000b980] ret_from_kernel_thread+0x5c/0x7c
>=20
> KMEMLEAK_EARLY_LOG_SIZE=3D8000 works but slow.
>=20
> Interestingly KMEMLEAK_EARLY_LOG_SIZE=3D400 on dba82d943177 still worked a=
nd I saw my 100Gbit. Disabling KMEMLEAK also
> fixes the speed (apparently).
>=20
> Is that something expected? Thanks,

It is expected that a debug option like this will make the system slower.=