Message-ID: <f7cb4884-72c5-dab3-c2e0-323d826febae@oracle.com>
Date:   Mon, 13 Dec 2021 08:23:18 -0600
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.4.0
Subject: Re: [PATCH v3 4/5] dma/pool: create dma atomic pool only if dma zone
 has managed pages
Content-Language: en-US
To:     Baoquan He <bhe@redhat.com>, linux-kernel@vger.kernel.org
Cc:     linux-mm@kvack.org, akpm@linux-foundation.org, hch@lst.de,
        cl@linux.com, John.p.donnelly@oracle.com,
        kexec@lists.infradead.org, stable@vger.kernel.org,
        Marek Szyprowski <m.szyprowski@samsung.com>,
        Robin Murphy <robin.murphy@arm.com>,
        iommu@lists.linux-foundation.org
References: <20211213122712.23805-1-bhe@redhat.com>
 <20211213122712.23805-5-bhe@redhat.com>
From:   john.p.donnelly@oracle.com
In-Reply-To: <20211213122712.23805-5-bhe@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Ty9SZkhTSytvQWszODVPejF1RmN5RkhreGdxV3Q1VVAzQ0FsOVNaczdsWFcv?=
 =?utf-8?B?ajNFWlZoblozS3IvQzhaZ2E0Z2RWTytydkNoeC9MSnZaWVN0Z3hXYTg0TGRO?=
 =?utf-8?B?V2RURU8rVkNJZHg1Vk9pdE1NOUUzVkxHSjQ1ZXRtR1NHdVYvRUFYUnllcjFJ?=
 =?utf-8?B?UHovRVpEZHAzSHBmNndrTHp2VUlUSkdvQllBcXhoakpIZlNzeHgwVUJqTlpS?=
 =?utf-8?B?R2hvUWJwRmswbkUwMnQrUlJQYk9GL0preXpIOG01U0hscG9BMC9oMHlTeG5i?=
 =?utf-8?B?YUVsazVoOG5xUytTVmh4YWdUendlNjI1VjdnN1NETzBSdVg0M0FkSWV6K1pK?=
 =?utf-8?B?RGg1UkV0aVo1VHVOVnpQekhaRXhuMlE1NS9JTGRqc0xid0xMMTVvN005VmZi?=
 =?utf-8?B?aGdRQjVtdUZsc1Y4a2pGRkZsWUdxNUZMZU5qaTh1TWF4NFhoU2Jua0NOTUli?=
 =?utf-8?B?VWxBQlhoaURFVWJnZkxDVHpieGo5NjBJTXJPbXY2SXlTTjc5SXNCNEYwaURR?=
 =?utf-8?B?TklQdXFHN0FiUHZ1em5EV2ZSVjNoSTFnc2hOaVFrbVNmVmZwMC9qWGxvSUht?=
 =?utf-8?B?bmVVWk1Jd3VWdklwbTg2eWplYTNqdGZqS0xSS0lLajdKRFhIOVZ6NXoxL09v?=
 =?utf-8?B?MVlEQzdacUNoZUg4Rmhkc3QwOHQwZU9RblBVaG4zTWkyWXpIRnc2ZlhFRXVM?=
 =?utf-8?B?RDROeTBQZHIzVndJSnc2MU53N3hIZzZhZ1ljNWRWTmhjSDBwQ1ZuOHBhbGpo?=
 =?utf-8?B?ZWw0OXFLWDYzUVZmU0p4K3FWb3IzMEpJTk9mVEZtQkMwaUhGcWFrc1ZLN0Jt?=
 =?utf-8?B?MjJhYkNLbjBualBzSHpwK1grQ1ZBcGI3dndidDhVODIyR3B2bmRvZDFQT3Rh?=
 =?utf-8?B?MHRBZlExWC9zVEpRMXdsb1ZsRkt5TGF5VTJKRW95YlBwd3RXWnFmS0tlQnBS?=
 =?utf-8?B?Q0FPU01jRW56Q3l5QTVrMXdKenV3dEhBdHhhM0ZwV2hKZURFSCtwOUhiZDZi?=
 =?utf-8?B?a2V4VkZndjF4c2NxSyt2d0E0MkNtRFVPa3R1Z3NlcUhGS2N5bGlhNzVBZ0VJ?=
 =?utf-8?B?ZDM4Wlhta2xqRlI2dE04MUpXMHlvUG9vMDhuVVB6eXRsTHB6VmNVQ0lEK0lP?=
 =?utf-8?B?eVlENURWSjVzUXdOMXZxb1pRaEtKME5DRnFIYmpBUEcrREZWM21KbDhJS1dr?=
 =?utf-8?B?RHlhcEFUZWNOdzB4MU5aTENlUDlhS05TK0VIMVRlY3hjVEIyRDhFZzN3TDlK?=
 =?utf-8?B?THpxajdBM0QzbHZVWml3d1RQN3Y2OXV6TVNDaWFmZm44eFQ4NkJTVTFkVkF5?=
 =?utf-8?B?bCtFemd0RHlEK3Zhemx3N2IrUGRNSGZMK2tTdUpjN1BFUlQ5ZGxIVENFaE4w?=
 =?utf-8?B?UWRGM1dzOHIzRlZnSlE5YWVYUE9Kb0tyYnk5MkNXWDAyK21NanZLNmlST3Q0?=
 =?utf-8?B?MVpCcmU2KytJdThKekRwMm5lYWl1bkxaSU5jOHhxc3M3SUVJVDBmY09RVjRT?=
 =?utf-8?B?YkZKVVowV1BrUlluVlpkRkNaL1JiRnRnRWI2dXRkMzBPdHhzQ0R5S1BFaWFp?=
 =?utf-8?B?WXpoSUwycTVmZ3FXZVk5c1luRVBiSjJQbGhsWmxCbmN0R3ZTSVk2QkRtYUpD?=
 =?utf-8?B?TDFsWTJaOVhCc01KSTF0T3F3b2IrOHNtTEMydTd0R1dEMHo0NFJ3Y0JrelI1?=
 =?utf-8?B?WmFzQ0Q1d3hRNThZSm5YaDlCdVYvSVpTWDJoSk1PWTArUnNNTUZBeHdyTnM0?=
 =?utf-8?B?Y2RMYkd2b09Fa08vSDk4VitqTEkydmxZdjdtdWVkN2tDemJKUE9kK2pUSi9p?=
 =?utf-8?B?ZEgzVURCd1diY1hyU1hRVStaUzBsT0loay9tUG0vSm9scUt1OE9ZOHVEd0lT?=
 =?utf-8?B?L2wwL3dFNGIxV0VHbkpucnFJVVZ1Rjl1SFJSMkhrV1hKN3VWeGtSVWJWUjFr?=
 =?utf-8?B?MWEydStIald3VzZkMVM1TkVDVnJFV0ZRdDRzem0wQjRiVzFDVEtSQkxsbzkr?=
 =?utf-8?B?TDBQTDdscjdNN1pkcUcrOCtYUG4veEN5dE04TGkxWVJlcnppRktjQm5KRnhO?=
 =?utf-8?B?M1VONWJpN3MxMUJydUdvV3MwdW5Ta1NOdEh4U2hYUkdTZ3VoRmo4MXB6Vi84?=
 =?utf-8?B?a2ZON2lXa1BSblB6TUlwWnVzY082UTg3RHlOUDFvRkQxdmd4bDgrM28weHFu?=
 =?utf-8?B?MUFsUFhXSTIyUkF3aHV1NXgvQ3Q1M0J6UUtCS2I3eS9qUE44anQwSUJTQ3JM?=
 =?utf-8?B?UW5xYmlLeUViSitDVEVwYnpJdG13PT0=?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7aefb268-0c4a-45fb-bd3c-08d9be442206
X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4722.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Dec 2021 14:23:28.8838
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rKqPb33m+Ba/LFNzhV+OMfypVv3e+H95tHxyRSWm86dxLg14u/n4hPEjwCXkgaytWDv7qQdmXbm4vi/Zc/lTJkKvcE6HmHxNAiiXGVkuAJU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR10MB1759
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10196 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 suspectscore=0
 mlxlogscore=999 phishscore=0 malwarescore=0 bulkscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000
 definitions=main-2112130091
X-Proofpoint-GUID: 6SNsDVUKNphjraKQSJ0BARU3u8bAnTGR
X-Proofpoint-ORIG-GUID: 6SNsDVUKNphjraKQSJ0BARU3u8bAnTGR
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 12/13/21 6:27 AM, Baoquan He wrote:
> Currently three dma atomic pools are initialized as long as the relevant
> kernel codes are built in. While in kdump kernel of x86_64, this is not
> right when trying to create atomic_pool_dma, because there's no managed
> pages in DMA zone. In the case, DMA zone only has low 1M memory presented
> and locked down by memblock allocator. So no pages are added into buddy
> of DMA zone. Please check commit f1d4d47c5851 ("x86/setup: Always reserve
> the first 1M of RAM").
> 
> Then in kdump kernel of x86_64, it always prints below failure message:
> 
>   DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
>   swapper/0: page allocation failure: order:5, mode:0xcc1(GFP_KERNEL|GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0
>   CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.13.0-0.rc5.20210611git929d931f2b40.42.fc35.x86_64 #1
>   Hardware name: Dell Inc. PowerEdge R910/0P658H, BIOS 2.12.0 06/04/2018
>   Call Trace:
>    dump_stack+0x7f/0xa1
>    warn_alloc.cold+0x72/0xd6
>    ? _raw_spin_unlock_irq+0x24/0x40
>    ? __alloc_pages_direct_compact+0x90/0x1b0
>    __alloc_pages_slowpath.constprop.0+0xf29/0xf50
>    ? __cond_resched+0x16/0x50
>    ? prepare_alloc_pages.constprop.0+0x19d/0x1b0
>    __alloc_pages+0x24d/0x2c0
>    ? __dma_atomic_pool_init+0x93/0x93
>    alloc_page_interleave+0x13/0xb0
>    atomic_pool_expand+0x118/0x210
>    ? __dma_atomic_pool_init+0x93/0x93
>    __dma_atomic_pool_init+0x45/0x93
>    dma_atomic_pool_init+0xdb/0x176
>    do_one_initcall+0x67/0x320
>    ? rcu_read_lock_sched_held+0x3f/0x80
>    kernel_init_freeable+0x290/0x2dc
>    ? rest_init+0x24f/0x24f
>    kernel_init+0xa/0x111
>    ret_from_fork+0x22/0x30
>   Mem-Info:
>   ......
>   DMA: failed to allocate 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocation
>   DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
> 
> Here, let's check if DMA zone has managed pages, then create atomic_pool_dma
> if yes. Otherwise just skip it.
> 
> Fixes: 6f599d84231f ("x86/kdump: Always reserve the low 1M when the crashkernel option is specified")
> Cc: stable@vger.kernel.org
> Signed-off-by: Baoquan He <bhe@redhat.com>


  Acked-by: John Donnelly <john.p.donnelly@oracle.com>
  Tested-by:  John Donnelly <john.p.donnelly@oracle.com>


> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: iommu@lists.linux-foundation.org
> ---
>   kernel/dma/pool.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
> index 5a85804b5beb..00df3edd6c5d 100644
> --- a/kernel/dma/pool.c
> +++ b/kernel/dma/pool.c
> @@ -206,7 +206,7 @@ static int __init dma_atomic_pool_init(void)
>   						    GFP_KERNEL);
>   	if (!atomic_pool_kernel)
>   		ret = -ENOMEM;
> -	if (IS_ENABLED(CONFIG_ZONE_DMA)) {
> +	if (has_managed_dma()) {
>   		atomic_pool_dma = __dma_atomic_pool_init(atomic_pool_size,
>   						GFP_KERNEL | GFP_DMA);
>   		if (!atomic_pool_dma)
> @@ -229,7 +229,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)
>   	if (prev == NULL) {
>   		if (IS_ENABLED(CONFIG_ZONE_DMA32) && (gfp & GFP_DMA32))
>   			return atomic_pool_dma32;
> -		if (IS_ENABLED(CONFIG_ZONE_DMA) && (gfp & GFP_DMA))
> +		if (atomic_pool_dma && (gfp & GFP_DMA))
>   			return atomic_pool_dma;
>   		return atomic_pool_kernel;
>   	}