Received-SPF: pass (google.com: domain of linux-kernel+bounces-39668-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Fri, 26 Jan 2024 14:35:44 +0800
From: Oliver Sang <oliver.sang@intel.com>
To: Catalin Marinas <catalin.marinas@arm.com>
CC: <oe-lkp@lists.linux.dev>, <lkp@intel.com>, <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>, Waiman Long <longman@redhat.com>,
	<linux-mm@kvack.org>, <oliver.sang@intel.com>
Subject: Re: [linus:master] [kmemleak]  39042079a0:
 kernel-selftests.kvm.memslot_perf_test.fail
Message-ID: <ZbNSwITESw+/SfNy@xsang-OptiPlex-9020>
References: <202401251429.d3dea02b-oliver.sang@intel.com>
 <ZbJNtFO_N4pVuEUK@arm.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZbJNtFO_N4pVuEUK@arm.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?FCFegthiPZ4EaBoXowqpeRdgzoft09S1ZYDesce8jtSCEv8B+tDxFq+bN5Ex?=
 =?us-ascii?Q?/vTd03LMt1Djd4+tuyjKgpsAKHQ4JQRb4O7yQHHoKeNrp7UdRztkzNXef5ao?=
 =?us-ascii?Q?wT2KHbukkti5djgmA8ak9ytGX9xBhAW1HmOS7bK2k31olPnuonXJ0zZQfmZU?=
 =?us-ascii?Q?qDVhIHjA+uOUYo27n0m3JSBoprDdTpbAGLe/Jq611lTVaBTx/CG3MpxJL1jb?=
 =?us-ascii?Q?nWPF5/gRood3tNp8wXpzdA4lzkP8o4hhMkZbXeba7ahM2XR2BBt2acrRusqX?=
 =?us-ascii?Q?BFAkWlIKHQ+2ZBLLKspFo8hDCGlCEBEXg2+6CFM/daNTmJWsg6cUGnO7spiB?=
 =?us-ascii?Q?CuFUMkY+NErl/XMioLHB/HDbkhFFKumro/DhZ++GagyZGdMWlb0VfvlW3sN0?=
 =?us-ascii?Q?gZMS8bVm974ZAvOcXUp8IkA+WY1I0AZ4Vivrb2ypuVjmdqLoa5VfwLH3J8Uu?=
 =?us-ascii?Q?Yon0p6q6DdJKmaclr37eSjLIGm/eezUdP9yrWvrB2/Mq4FN1Hm/bsO2Xf/0l?=
 =?us-ascii?Q?t1ucwIv0pIdsIrYbuuRSX2XY9R+MyTKSIdAkNqZVh7ZDarYoWVlaOlDt0T5p?=
 =?us-ascii?Q?qSL9WySfaLu3ZyfDbX+fcdEZTu7xrJaVWFhr01/1ufBrV+pG4Rt3PWoT4S1a?=
 =?us-ascii?Q?rd5/m39ff4csKnOE+OBuiq20PXijq88zxdwWfRSU1otOi4u+buQVZt7MEJEF?=
 =?us-ascii?Q?WY+ogtPHhggE89WivE4i5w0VQGKn/1WyqhMCCE3EDBZzTJHZVMG298mV1e3c?=
 =?us-ascii?Q?vd9OcZjFrJRnYz1AMcE/HqSDt7ULnKZzo7LKkUxpuip1ct34mARPr7xOrqae?=
 =?us-ascii?Q?EPYFX5esM+stuc4jzpoNNTgdR2hcRvncUHoKblT+k8YDBvZEvBdbgeeyhgqH?=
 =?us-ascii?Q?l+KV4j+GYjgmhxHFY+9bx62xJJptntAN58tLVK/wm+M+gkM6DPlBFisllmR9?=
 =?us-ascii?Q?IXdPW6R2vA16gedf97LF1ddLUSkP+UnIJFYM62g2ckUfbZiboGQ1F3vpMgBB?=
 =?us-ascii?Q?p1F/UhDbzbboIOsi1izYxvkrGn4IR63iSMqplwiTCT736csLTgbN1EEj/sql?=
 =?us-ascii?Q?lM4o71NEgR3V0Jl2QESe6ORRwSr7zSF17UmiP83r1Z8e2BmaGMm/NGEbb6iL?=
 =?us-ascii?Q?bIKAsAiiM4syUr7FhBinEdFvCm76vJ0GmFd7+HXkU+simD7PTYg0X9rYFg0u?=
 =?us-ascii?Q?sP8KQZQnfHLmB7cMKL8EE50NosVgb6TxhH3gGQqLlKPdYipJnXmE/fU5E7C6?=
 =?us-ascii?Q?86NgGGZEALdc439nT6Gg8NN/KQFcrXntHriEPJedoH6cVjXZPAOD6VF8D3N6?=
 =?us-ascii?Q?Z/G0hye8pr3X9xGeBrIPpNKtBdIlQMhEJh4ig06NAJmSUCeLhkcySq59gfOk?=
 =?us-ascii?Q?PEKJyJ0UiQDB/17S3i/j+UNDrWE/TErQywDi6TJ618/n07IVoYUK8CU2+lMS?=
 =?us-ascii?Q?abKPPoQocKUii1nasn9MAe7yWoSy3F5B8mEPr8srwp8QkvP1q66kMHPLdkKB?=
 =?us-ascii?Q?iWUQOCxo5+W5mRW5kMsxmts3EJXkl0VUcNXJ5DM7qdJpe0l5UA56Lkq4wMC3?=
 =?us-ascii?Q?+r7BSVfoBEtz8q5PSuLF9QzTkyvXUl6U58KUwsjqBGaSnEFV7G5dXZorhLiS?=
 =?us-ascii?Q?ew=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5067f1c3-9328-46e6-f0ea-08dc1e390a3a
X-MS-Exchange-CrossTenant-AuthSource: LV3PR11MB8603.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2024 06:35:51.4350
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RAXwhIX/Ad12/AWhXy0kaUom5ovPQv1zKsVty8mjQ7l3wPm2YDKXTzR5JO5LjXrFAU48nKXg+0FasJX06cJO1A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB6244
X-OriginatorOrg: intel.com

hi, Catalin,

On Thu, Jan 25, 2024 at 12:01:56PM +0000, Catalin Marinas wrote:
> On Thu, Jan 25, 2024 at 03:34:37PM +0800, kernel test robot wrote:
> > kernel test robot noticed "kernel-selftests.kvm.memslot_perf_test.fail" on:
> > 
> > commit: 39042079a0c241d09fa6fc3bb67c2ddf60011d0f ("kmemleak: avoid RCU stalls when freeing metadata for per-CPU pointers")
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> [...]
> > The kernel config and materials to reproduce are available at:
> > https://download.01.org/0day-ci/archive/20240125/202401251429.d3dea02b-oliver.sang@intel.com
> [...]
> > # Testing RW performance with 1 runs, 5 seconds each
> > #
> > not ok 71 selftests: kvm: memslot_perf_test # TIMEOUT 120 seconds
> 
> I'm not sure how this relates to kmemleak, especially the commit above.
> It might as well be that the above kmemleak commit increases the system
> load and you trip over the timeout above. Is it still reproducible with
> a larger timeout (unfortunately I don't have the hardware to try to
> reproduce the problem)?
> 
> I can see a lockdep warning in the dmesg but that doesn't look related
> to kmemleak.

below things FYI.

(1) we set same timeout for 39042079a0 and its parent, test always can pass
on parent (by 20 runs), but almost always fail on 39042079a0 (18 out of 20
runs fail).

(2) while running tests, we use a kmemleak monitor.
for parent runs, seems no problem, but for 39042079a0 runs, it will capture
below log:

$ cat kmemleak
unreferenced object 0xff110001198a1720 (size 16):
  comm "swapper/0", pid 1, jiffies 4294783051
  hex dump (first 16 bytes):
    41 43 50 49 20 48 4d 41 54 00 6b 6b 6b 6b 6b a5  ACPI HMAT.kkkkk.
  backtrace (crc 45396b86):
    [<ffffffff81a97e28>] __kmem_cache_alloc_node+0x228/0x310
    [<ffffffff81936de5>] __kmalloc_node_track_caller+0x55/0xf0
    [<ffffffff819160b4>] kstrdup+0x34/0x60
    [<ffffffff81aada3b>] mt_set_default_dram_perf+0x3cb/0x460
    [<ffffffff8658a9a6>] hmat_init+0x2b6/0x3a0
    [<ffffffff81002c58>] do_one_initcall+0xd8/0x410
    [<ffffffff864aaa4a>] do_initcalls+0x1ca/0x390
    [<ffffffff864ab53e>] kernel_init_freeable+0x8ce/0xc80
    [<ffffffff83beed8f>] kernel_init+0x1f/0x1d0
    [<ffffffff810a07c1>] ret_from_fork+0x31/0x70
    [<ffffffff81006341>] ret_from_fork_asm+0x11/0x20

(3) we tried to disable the kmemleak monitor for 39042079a0, then
memslot_perf_test can pass now. there is no kmemleak dump.

(4) we also noticed a test can pass on 39042079a0 now, but always fail on
parent, no matter kmemleak monitor enable or disable.
(normally we won't report a commit if it makes some test start to pass, so this
information is not in original report)

f67f8d4a8c1e1ebc 39042079a0c241d09fa6fc3bb67
---------------- ---------------------------
       fail:runs  %reproduction    fail:runs
           |             |             |
           :20         150%          30:30    kernel-selftests.kvm.rseq_test.pass

> 
> -- 
> Catalin