Message-ID: <90bbe424-a522-b225-35c2-a3b3693271c7@oracle.com>
Date:   Mon, 13 Dec 2021 08:24:21 -0600
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.4.0
Subject: Re: [PATCH v3 5/5] mm/slub: do not create dma-kmalloc if no managed
 pages in DMA zone
Content-Language: en-US
To:     Baoquan He <bhe@redhat.com>, linux-kernel@vger.kernel.org
Cc:     linux-mm@kvack.org, akpm@linux-foundation.org, hch@lst.de,
        cl@linux.com, John.p.donnelly@oracle.com,
        kexec@lists.infradead.org, stable@vger.kernel.org,
        Pekka Enberg <penberg@kernel.org>,
        David Rientjes <rientjes@google.com>,
        Joonsoo Kim <iamjoonsoo.kim@lge.com>,
        Vlastimil Babka <vbabka@suse.cz>
References: <20211213122712.23805-1-bhe@redhat.com>
 <20211213122712.23805-6-bhe@redhat.com>
From:   john.p.donnelly@oracle.com
In-Reply-To: <20211213122712.23805-6-bhe@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Y2VtdUN1NXNFMGtSMnFDS1d2U0JpbStlOGZRZVU0VjFZQWh1ck5zTlIxUU4y?=
 =?utf-8?B?SFp3UTJMc04velRBTjlkYmZHZWFYanhZaHhPNWMzUEI4aVN4N0drLzVEUzlV?=
 =?utf-8?B?NlBTYTM2Mmd4K2RiYVMxenNUbC82bnZEa0pNSEVPend5U01FbS9KYWpwRE80?=
 =?utf-8?B?K3dueHVUK0hzRTdBQVYrVmtUeVhqMTRUYUk2WUVHWjhwaVlLbnd4a2VCR1Vt?=
 =?utf-8?B?clByUFdPM2J3V3E2aTNzZ1hyVVpMb0p1SFJDNElIYy9EU0hWWDZhVkRqaExu?=
 =?utf-8?B?Qk5QcWplTXpIcFF0Y2NIMVVCd213NXZ6Z1ltSEJuTUFndVNkUjV6SkhXQlZh?=
 =?utf-8?B?bXRMSEdVZDR2Z2xxV21pQ1dtajdGck1BY3BlS0xmMHJuUGQwZ2JHa2x0Y1l2?=
 =?utf-8?B?QnFlU3dwN2NLNTBoSGR2akNielVQckdLQUptSk51N24vaEVKNUQ5elZmbVM5?=
 =?utf-8?B?R3hELzhIMHBYVE1WTEhNTExoYWJMMnhQQVhaZ3ByVnN1V1ZjMUh1ZTR3ZXR1?=
 =?utf-8?B?dVZTcHZjRE92dGEycU0xajBGdnU3TkpZSmFra3hyK3hzMG0xakJHZnIzeGwv?=
 =?utf-8?B?R0s1T0VzQ1laU2wzbGFnZHBUcGx0NmlGRUQ5Sm16b2FyKzVPeFFjWnk5cHk5?=
 =?utf-8?B?K291S2c2UDBUMmp6YW1QWk5mU0tGakhkelhWTW1lSXBYaWRLbVR6V0tWbTRD?=
 =?utf-8?B?YlhXWTBwR3hTVGtZMmxWSzQwY3JUdEkzZFhWSDRBZE1NVGZ6RWwzMDh4bSs1?=
 =?utf-8?B?SXB2Tlc1TGlVcmdLbXpVSFpZejBTSHJQTVA2Ny96dkRqZ29lM0ZZaVpxcmRW?=
 =?utf-8?B?bVVBK3NPWGc1eUE4TFIvTUFlVXlza1BRLzRwOEpjT1dYaWlxWmRzUlo4dExo?=
 =?utf-8?B?QkpGeVBSR1ZoTjRhMk41WHdhYno2VVloUC8wdDFieU5Oa1p2QVpkU3pQU081?=
 =?utf-8?B?azZYZkNQdkdEUEdEVzk4UzdFSy9DdG0rdElVV1NwbFVqdkdNSGZUV25Xa1Jp?=
 =?utf-8?B?TkFibWtKRnM1WEY0RFc2NjNqQ3JpR1ZuYUU5dkpUOEw4QWJ6Zm1PL3ZaUVZU?=
 =?utf-8?B?aExPUm9pVzFWR241cDRQcmh6cEdyRUlEOUY5RmtvRzhGWmI1SlFYTjl2cGcz?=
 =?utf-8?B?N2Z1UHdWaTZlUmtSd3RmK085Ky9scUZUMFoySTdzNTBmRzErRXdpZU9QamhQ?=
 =?utf-8?B?eWJUNGF0NjVpTkJ6NXozeWRmVURacUFQNWdDWERPOUxnRHd4MDhnMEVvMTdn?=
 =?utf-8?B?aXRKOG0rU2UwTVRFMk9iem1iZWd1RnZ4N0h2dm03RG9nb2JWSmN2Y0ZnK29x?=
 =?utf-8?B?aGwrWEc4aTU5bEJ2R2t5NERIdFZnQnRaUXFIcm1HcnA3YjRzZmx2eXc3S3ZM?=
 =?utf-8?B?dzdkK0JSUDArQ09mTmtTazhPMXMvV3FUUmwrZXpIS2V4MExBSlVjNEQxcW05?=
 =?utf-8?B?SEZRZTd0RkczcE5kNUxqallTVmhYZGlQWXFhN0ZmV0oyRjdBbytaTDFBbzA5?=
 =?utf-8?B?UTRna29wUk8xSnJBQmZUaGxLU3JlZ0tZQlNlOWRzTm5qRjdTTnJEeHl3NHUx?=
 =?utf-8?B?Z3Y3eTEyQzBwbTdXVVJmM3NrTCt6WWZoTzdGSHlWM0N6YTZWMDEzekNLRk5M?=
 =?utf-8?B?a0EyL2dYNXpndFVUNnFBeWRJcmJiVGFBNUdDWDlubkdPVEFzSVQ5YWlTV2dL?=
 =?utf-8?B?UVJhNFczcjdNcmJwSHhKMVV3blRBVGJMREdtcHltZ25nLzc1bHJLbUp1dU9T?=
 =?utf-8?B?Q3VqTzVKYWpKLzZicXJFb1Zlbkk2WUxnNzc5aDRlTHpwZ0pkeFAwRGZwYzVY?=
 =?utf-8?B?Q25VcDdibjl3b2JNclc4ZE9rT0lkbm5iSGUxUGhla21hSTN0aWhIczR6Vnl5?=
 =?utf-8?B?Zmx1WnpqT050Wk90UFR4cEZieDFDdjc3ZjJZWTlMd0Qzc2hkQUpCQmZIbVFs?=
 =?utf-8?B?VFQ4TFMwVmRQZUo0bmI3bkZrcHorV2pjNW53Nzc1MlhBQlJiTCtnT3VxNGNB?=
 =?utf-8?B?YVlPalIzSzluTVhvckFZNVFpTmdOelluYmNlNW9SWWZnMUdqOXNrdHhpdHlJ?=
 =?utf-8?B?MTlHclQvL0hBZnoyRmJkeTI0YmZNWGN2UlZXQ0VUSmpmNHBQWUlna1lKOVcx?=
 =?utf-8?B?ZjFpNUpSOURtbUFrZ2ZRKytGM3loNXRpTmV5UXJtdkdLdW5NbTFxRkQzMGE5?=
 =?utf-8?B?RCtONkF3Yk5jUWxBdExGVXl4elhJTXpOb0lsL3NQWldNVFNNQy9pd3o1Y3RG?=
 =?utf-8?B?QmpjdFJveWJsaXA1cXEyN0xnVDRRPT0=?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dea187a0-4a6e-4250-89b4-08d9be4447ba
X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4722.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Dec 2021 14:24:32.1346
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: e3QOJutm1VZjtlFqg0IGERa7Wew7Tho11c+fYBc9r4zh7FGuHtUu5hf6XEz4miUINnz/9CvY9TSGwjCbbD14HSf4tF08ruQRGccGyFZq9cA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4435
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10196 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 suspectscore=0
 mlxlogscore=999 phishscore=0 malwarescore=0 bulkscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000
 definitions=main-2112130091
X-Proofpoint-GUID: VR6XnJmvztGOKdD22SKC8nwE-qkWRom9
X-Proofpoint-ORIG-GUID: VR6XnJmvztGOKdD22SKC8nwE-qkWRom9
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 12/13/21 6:27 AM, Baoquan He wrote:
> Dma-kmalloc will be created as long as CONFIG_ZONE_DMA is enabled.
> However, it will fail if DMA zone has no managed pages. The failure
> can be seen in kdump kernel of x86_64 as below:
> 
>   CPU: 0 PID: 65 Comm: kworker/u2:1 Not tainted 5.14.0-rc2+ #9
>   Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS RMLSDP.86I.R2.28.D690.1306271008 06/27/2013
>   Workqueue: events_unbound async_run_entry_fn
>   Call Trace:
>    dump_stack_lvl+0x57/0x72
>    warn_alloc.cold+0x72/0xd6
>    __alloc_pages_slowpath.constprop.0+0xf56/0xf70
>    __alloc_pages+0x23b/0x2b0
>    allocate_slab+0x406/0x630
>    ___slab_alloc+0x4b1/0x7e0
>    ? sr_probe+0x200/0x600
>    ? lock_acquire+0xc4/0x2e0
>    ? fs_reclaim_acquire+0x4d/0xe0
>    ? lock_is_held_type+0xa7/0x120
>    ? sr_probe+0x200/0x600
>    ? __slab_alloc+0x67/0x90
>    __slab_alloc+0x67/0x90
>    ? sr_probe+0x200/0x600
>    ? sr_probe+0x200/0x600
>    kmem_cache_alloc_trace+0x259/0x270
>    sr_probe+0x200/0x600
>    ......
>    bus_probe_device+0x9f/0xb0
>    device_add+0x3d2/0x970
>    ......
>    __scsi_add_device+0xea/0x100
>    ata_scsi_scan_host+0x97/0x1d0
>    async_run_entry_fn+0x30/0x130
>    process_one_work+0x2b0/0x5c0
>    worker_thread+0x55/0x3c0
>    ? process_one_work+0x5c0/0x5c0
>    kthread+0x149/0x170
>    ? set_kthread_struct+0x40/0x40
>    ret_from_fork+0x22/0x30
>   Mem-Info:
>   ......
> 
> The above failure happened when calling kmalloc() to allocate buffer with
> GFP_DMA. It requests to allocate slab page from DMA zone while no managed
> pages in there.
>   sr_probe()
>   --> get_capabilities()
>       --> buffer = kmalloc(512, GFP_KERNEL | GFP_DMA);
> 
> The DMA zone should be checked if it has managed pages, then try to create
> dma-kmalloc.
> 
> Fixes: 6f599d84231f ("x86/kdump: Always reserve the low 1M when the crashkernel option is specified")
> Cc: stable@vger.kernel.org
> Signed-off-by: Baoquan He <bhe@redhat.com>

  Acked-by: John Donnelly <john.p.donnelly@oracle.com>
  Tested-by:  John Donnelly <john.p.donnelly@oracle.com>

> Cc: Christoph Lameter <cl@linux.com>
> Cc: Pekka Enberg <penberg@kernel.org>
> Cc: David Rientjes <rientjes@google.com>
> Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> ---
>   mm/slab_common.c | 9 +++++++++
>   1 file changed, 9 insertions(+)
> 
> diff --git a/mm/slab_common.c b/mm/slab_common.c
> index e5d080a93009..ae4ef0f8903a 100644
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -878,6 +878,9 @@ void __init create_kmalloc_caches(slab_flags_t flags)
>   {
>   	int i;
>   	enum kmalloc_cache_type type;
> +#ifdef CONFIG_ZONE_DMA
> +	bool managed_dma;
> +#endif
>   
>   	/*
>   	 * Including KMALLOC_CGROUP if CONFIG_MEMCG_KMEM defined
> @@ -905,10 +908,16 @@ void __init create_kmalloc_caches(slab_flags_t flags)
>   	slab_state = UP;
>   
>   #ifdef CONFIG_ZONE_DMA
> +	managed_dma = has_managed_dma();
> +
>   	for (i = 0; i <= KMALLOC_SHIFT_HIGH; i++) {
>   		struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i];
>   
>   		if (s) {
> +			if (!managed_dma) {
> +				kmalloc_caches[KMALLOC_DMA][i] = kmalloc_caches[KMALLOC_NORMAL][i];
> +				continue;
> +			}
>   			kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache(
>   				kmalloc_info[i].name[KMALLOC_DMA],
>   				kmalloc_info[i].size,