Message-ID: <73cd623e-93cc-4109-710f-f9f4559b2eab@oracle.com>
Date:   Mon, 6 Dec 2021 21:54:24 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.0
Subject: Re: [PATCH RESEND v2 4/5] dma/pool: create dma atomic pool only if
 dma zone has managed pages
Content-Language: en-US
To:     Baoquan He <bhe@redhat.com>, linux-kernel@vger.kernel.org
Cc:     linux-mm@kvack.org, akpm@linux-foundation.org, hch@lst.de,
        robin.murphy@arm.com, cl@linux.com, penberg@kernel.org,
        rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz,
        m.szyprowski@samsung.com, kexec@lists.infradead.org,
        iommu@lists.linux-foundation.org
References: <20211207030750.30824-1-bhe@redhat.com>
 <20211207030750.30824-5-bhe@redhat.com>
From:   John Donnelly <John.p.donnelly@oracle.com>
In-Reply-To: <20211207030750.30824-5-bhe@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eitMYnlGQ3RUampFOUhzSkQwVkhjWUFQamE3ajI5RVRMWG1LZnpKcXhhNHNX?=
 =?utf-8?B?VThrM0gwTmhOU2E0ZFJEOFE1VW5UV1NvS3EvVDlwcFBxdzBsbE83Nm91RWN6?=
 =?utf-8?B?ZXdRQ2V0KzhWME43UzVmdS9RVjFYVldPanBVdkRnei9tcEc4ek1vTjVJQUlw?=
 =?utf-8?B?cWUyZzVPc1pyc05LWFhGNEtXOGdSRGpnNHhYZXhsdGhEUnRaZFNBaWtrbWdY?=
 =?utf-8?B?UVlFK2JpNHliZjU0aGFreFhla015ci9rZ2diL2YwVzd2bmtyTlByVW05bytT?=
 =?utf-8?B?VTlJa29QZDExZzNybUFuZ2tBRHN6dWh3YURiWlpxV1p5M3VIcHFMdTRIdkRp?=
 =?utf-8?B?T1kxODZvcThFbXpBU1I5N1lYY0xVWjV1emhKbUd6WExWUjZXSUxVZ21uWmJO?=
 =?utf-8?B?bTJqUllwQW9QV2hTTVM2MEV6RXVmcEdlVTBjcHpScE9UbFpZeFV4Y2ZoamRo?=
 =?utf-8?B?Y0taaUg0WWtiT2pqZHhMU3dCd1BwVmNQVlBZQjZBVjZMbkpLYlluaTk0NGU3?=
 =?utf-8?B?cldDQU5sNGIwWlkraDh2RzVBdnFCN25UV1F5UUNPNkNyRFFsZHJRQUJ4dlJZ?=
 =?utf-8?B?RDNTbGtpZVh2R2dpczh6SENWdFZpQlV3Y2hkNWtmUWQxc0gwNU91Q2lyL1Z4?=
 =?utf-8?B?cDRHblVBTGowV2NvYVJ0U3I4TnBodW8yT090OEZLUlBFTWFTMTBMUExkaks4?=
 =?utf-8?B?U1R3T0lzb1czS2Q2eno5Y0gwdldDM09nU09rdjBhY1lZU1o3NERUelpoZTFp?=
 =?utf-8?B?VDNRRjBOWXNzZmwxNU9iVnRwUEZNeVhSRU40UllYUHNKeDIxVVpnRkY5MVp6?=
 =?utf-8?B?NFRKaWdhTnRvYnc1cGVEZGNRZFBFakw1VHVHZTBsVWpBRTc5RDVyMFlKN2cv?=
 =?utf-8?B?T25paTBOTnJTdS8yL0I0V2F6SHpkVXBaL2ljelc4L21OUG9aZFcvbFJxOXFS?=
 =?utf-8?B?aXUxa1VWTGF5eDZVcFpaMkZJVVNNdkZWWXVuT3NQSFpKdkxFQ1c0c0dwTzl4?=
 =?utf-8?B?R0l2aEwxY25Ubm5aRUlGZ2VGMVVyYXBpR1B2MFo4T0M5Uzd1QnExNmZveHJS?=
 =?utf-8?B?WFhBNnQvcnBlYkhJMnhjWitSdXIwMXA4cFVQMEZzL0U2YkFOaGM1dWsweHJI?=
 =?utf-8?B?SnVMRzZJdEFudFdoTGZ1K0ZVZ2hyWGNKYW42MjUxQTJMdzBsd0h6UmhwSXN5?=
 =?utf-8?B?TCs0dkVvUlNldzg0enhEbUt0ZVBjbjJrbkNySDJMWi94OGpsTmQyYkxOWEtz?=
 =?utf-8?B?SWZhdU84WEtFcVR6WEtsNHNoZXRRN2dNYzEvbElzbktFYTJZS3dTVElSSGR0?=
 =?utf-8?B?ZUQ1cFBpZjlpMitnYUlqdmFWSU1wUjBIU3YwZUdmRVByUU1lR2g4cUd6QWpM?=
 =?utf-8?B?MG1BT1NOTTVmRHQ3aElSSGZYSFVRWGZrOWhLUkNiOGI1STFmZmx6bGZFamJv?=
 =?utf-8?B?MThBTkJtYWVYaXNvSmNzdlpIdzVqV2cxeHlHK0hxNlhVZFdEeFltQzJBSjMx?=
 =?utf-8?B?M3FQTzhJYlFxZytVWlB6aDhrcEU1RkgvTEIvRDNRTjBmaVpSSTFKVDNLWGZr?=
 =?utf-8?B?cllIcnlwT2JBRVRZN1dXdW42ZThLd2Y5UWhheEtzMGZENHZGNnhnU042d1Zy?=
 =?utf-8?B?ZDBpNzlTRHpVUm1mdW00djlGU1VxWllqbFdqQ3VoeGYydkpwSzZOaWdvcmFW?=
 =?utf-8?B?T21BQUIwSmZXQ0I1SUV3eW5ZbVc1SkVUTnFLV0pvVWtmSVhMZmxaYmg1K2Zz?=
 =?utf-8?B?dURFRnF5ZmRvblhJd2lsdE4wQVpLN0ZOZnpiS1ErQUltTUxpVXNHQ0ZzUWI2?=
 =?utf-8?B?SDFnNEFwbzJPbXA5QVJWa1VueE1RdlJ4cUdtR3d1Rjd0M3Y4R1NqZmJNWjY3?=
 =?utf-8?B?WnRYUGM2aEtDSyt4NzVOdll3aS9qTnUyWnphOTQwaHFJVTNJYXcxZDBTdU9G?=
 =?utf-8?B?dUxvWUw0S0U1Y3hIMGZraTVPTXA3T0N6Y2YvU3FRbW9FT3dqeUcySDg0c3Mr?=
 =?utf-8?B?UHB0eTIxYjhRZmVWZ0Z3cU4wbmhpNUdUWXlyZklwcUErQTAvSW9sdVc4d01s?=
 =?utf-8?B?ZitSU2Q2Qm1TaTNFZDltMTI3MHVKUy9yZ3JhS29ySHRoQkN1aFZ6RGdzR0pP?=
 =?utf-8?B?cDJRalZqZUlydEx5TVBSTHZTdndIRkhvRGRSM2x1WlBJODc5UFMvNForb2k4?=
 =?utf-8?B?bTlhbFFOMjl1K254a00xd29lYmxneFBNT0JoUDUwOFlMSUNZNXFac3RpYkdo?=
 =?utf-8?Q?YojWUHNqbGfOMCAWtAnkluqXlN+f7S2/iCXadcuEGs=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 49ca8ce6-d93d-43ac-5d31-08d9b93545fc
X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4722.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2021 03:54:30.9770
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VOiDtOLzijvtIt6VYJXaP7QR1gs3in55kG1vkFflKeHnKtHeVG9NkbHgnnJal7hYErhPtPFDNjUIhs/mAcgT4gUM2KQ6KL0PcJWAqxfXM+0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4514
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10190 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 suspectscore=0
 adultscore=0 mlxlogscore=999 bulkscore=0 malwarescore=0 mlxscore=0
 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2110150000 definitions=main-2112070022
X-Proofpoint-ORIG-GUID: CUK3o-6tpORcrz9LcYHbuLWCLMS3EojI
X-Proofpoint-GUID: CUK3o-6tpORcrz9LcYHbuLWCLMS3EojI
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 12/6/21 9:07 PM, Baoquan He wrote:
> Currently three dma atomic pools are initialized as long as the relevant
> kernel codes are built in. While in kdump kernel of x86_64, this is not
> right when trying to create atomic_pool_dma, because there's no managed
> pages in DMA zone. In the case, DMA zone only has low 1M memory presented
> and locked down by memblock allocator. So no pages are added into buddy
> of DMA zone. Please check commit f1d4d47c5851 ("x86/setup: Always reserve
> the first 1M of RAM").
> 
> Then in kdump kernel of x86_64, it always prints below failure message:
> 
>   DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
>   swapper/0: page allocation failure: order:5, mode:0xcc1(GFP_KERNEL|GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0
>   CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.16.0-rc3+ #1
>   Call Trace:
>    dump_stack+0x7f/0xa1
>    warn_alloc.cold+0x72/0xd6
>    ? _raw_spin_unlock_irq+0x24/0x40
>    ? __alloc_pages_direct_compact+0x90/0x1b0
>    __alloc_pages_slowpath.constprop.0+0xf29/0xf50
>    ? __cond_resched+0x16/0x50
>    ? prepare_alloc_pages.constprop.0+0x19d/0x1b0
>    __alloc_pages+0x24d/0x2c0
>    ? __dma_atomic_pool_init+0x93/0x93
>    alloc_page_interleave+0x13/0xb0
>    atomic_pool_expand+0x118/0x210
>    ? __dma_atomic_pool_init+0x93/0x93
>    __dma_atomic_pool_init+0x45/0x93
>    dma_atomic_pool_init+0xdb/0x176
>    do_one_initcall+0x67/0x320
>    ? rcu_read_lock_sched_held+0x3f/0x80
>    kernel_init_freeable+0x290/0x2dc
>    ? rest_init+0x24f/0x24f
>    kernel_init+0xa/0x111
>    ret_from_fork+0x22/0x30
>   Mem-Info:
>   ......
>   DMA: failed to allocate 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocation
>   DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
> 
> Here, let's check if DMA zone has managed pages, then create atomic_pool_dma
> if yes. Otherwise just skip it.
> 
> Signed-off-by: Baoquan He <bhe@redhat.com>
  Reviewed-by: John Donnelly <john.p.donnelly@oracle.com>
  Tested-by:  John Donnelly <john.p.donnelly@oracle.com>


> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: iommu@lists.linux-foundation.org
> ---
>   kernel/dma/pool.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c
> index 5a85804b5beb..00df3edd6c5d 100644
> --- a/kernel/dma/pool.c
> +++ b/kernel/dma/pool.c
> @@ -206,7 +206,7 @@ static int __init dma_atomic_pool_init(void)
>   						    GFP_KERNEL);
>   	if (!atomic_pool_kernel)
>   		ret = -ENOMEM;
> -	if (IS_ENABLED(CONFIG_ZONE_DMA)) {
> +	if (has_managed_dma()) {
>   		atomic_pool_dma = __dma_atomic_pool_init(atomic_pool_size,
>   						GFP_KERNEL | GFP_DMA);
>   		if (!atomic_pool_dma)
> @@ -229,7 +229,7 @@ static inline struct gen_pool *dma_guess_pool(struct gen_pool *prev, gfp_t gfp)
>   	if (prev == NULL) {
>   		if (IS_ENABLED(CONFIG_ZONE_DMA32) && (gfp & GFP_DMA32))
>   			return atomic_pool_dma32;
> -		if (IS_ENABLED(CONFIG_ZONE_DMA) && (gfp & GFP_DMA))
> +		if (atomic_pool_dma && (gfp & GFP_DMA))
>   			return atomic_pool_dma;
>   		return atomic_pool_kernel;
>   	}
>