Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RFC PATCH 7/8] hugetlb: add update_and_free_page_no_sleep for
 irq context
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     linux-mm@kvack.org, linux-kernel@vger.kernel.org,
        Michal Hocko <mhocko@suse.com>,
        Shakeel Butt <shakeelb@google.com>,
        Oscar Salvador <osalvador@suse.de>,
        David Hildenbrand <david@redhat.com>,
        Muchun Song <songmuchun@bytedance.com>,
        David Rientjes <rientjes@google.com>,
        Miaohe Lin <linmiaohe@huawei.com>,
        Matthew Wilcox <willy@infradead.org>,
        HORIGUCHI NAOYA <naoya.horiguchi@nec.com>,
        "Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>,
        Waiman Long <longman@redhat.com>, Peter Xu <peterx@redhat.com>,
        Mina Almasry <almasrymina@google.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Roman Gushchin <guro@fb.com>,
        Christoph Hellwig <hch@infradead.org>
References: <20210319224209.150047-1-mike.kravetz@oracle.com>
 <20210319224209.150047-8-mike.kravetz@oracle.com>
 <YFhYHZ9onwdZMeDi@hirez.programming.kicks-ass.net>
From:   Mike Kravetz <mike.kravetz@oracle.com>
Message-ID: <2383057a-29dc-383b-720f-7cdcdd015e40@oracle.com>
Date:   Mon, 22 Mar 2021 10:42:23 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.1.1
In-Reply-To: <YFhYHZ9onwdZMeDi@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [192.168.2.112] (50.38.35.18) by MW2PR16CA0056.namprd16.prod.outlook.com (2603:10b6:907:1::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3955.18 via Frontend Transport; Mon, 22 Mar 2021 17:42:25 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 13686aa6-258c-458b-d32e-08d8ed59db5f
X-MS-TrafficTypeDiagnostic: BYAPR10MB3352:
X-Microsoft-Antispam-PRVS: <BYAPR10MB335216E940365E182517CE6AE2659@BYAPR10MB3352.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:3513;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: PbQnQEpEkTw6Oh8yyAOoTs48c+9YIAZZJvP585FyDDcXlqlh2p2KWqfqG2aCgT7V7dr8YbiqhdwbsyBL3NXtmbxih9WwCkuErHilQbQEAumBpMGA6S74fhmy041Uj6MtX/TVK8IgQ9MwedIWTh4ki2lwMsmL2fBVG2+HeHUe2Sq6t60+YeV26doSD4f1mptNC72hPyewNwiqEqNxulE3Wx4n5JACHkJgBQccmvgTXEulG0h0wY/OAWfBhaMj+eZFjCPHr1RLDFjcekO1HKK6hDxuCFbsBuScafvCL0Q4jxillUL1YDmggLCYEUhg91Vy88tY5b6dbHKn16yjIw2WQJz324xTZskkt4f8RzQTseVann1QpYNDfuf2/Ih/HuQ6bjUZP51lCvu56OCJidYWhT+g0hMT63bgtHEn+DcCpKMxsejv1ug94ZO0niMXGKHUWKSfr0kskf2+fJ1cOIWK7zPsliAD/K8Y1PItLmjH/26KczZd4HeTiaWIiCZt8LUKyvgtm/oCaBW1JeWPF/Sh7IM5so0v8NPeOY/DscV5+dIrTrEYjYLjIWZA9c4bEW1WUIQyP0P0aagY6mmhA2qCTSxEmmJLqmzUHuiGt2B52zd4MzaXe8UOWUcLJlezBZbNZm88PVxKVr5R/rrw7WHIhoiIiRQyHnABCTUb9CGZtToNWCys3wTBpCVaxIVpyseqgVzLOwSPU+74BEO6KsiAcOvOjoXyBlgOyxu+G4Q+/3VPwqFLrijO68rGBn0kDdO07xCnVFuM1ZHDPaisIKbmmjhwwj6n+SfSvzvQiC2P0lM=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(376002)(39860400002)(396003)(346002)(366004)(8676002)(966005)(2906002)(8936002)(53546011)(186003)(36756003)(16526019)(52116002)(5660300002)(66946007)(4326008)(66476007)(66556008)(16576012)(26005)(316002)(31686004)(6916009)(38100700001)(2616005)(15650500001)(44832011)(31696002)(83380400001)(7416002)(478600001)(6486002)(86362001)(956004)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?SzlxcjYrSCs2cHR5YkhpRDc4R1h0SWZnSUdMT21oYWZhWWhaSkNmMWIwdFpF?=
 =?utf-8?B?U2RzaVdlTDFWZ1NrUzFIK1UxcmUrSnNubEdaNzZJa1FtM2NURGt4Z1ZYc2xx?=
 =?utf-8?B?cjlDN2poU1lZVnhxbjRNMTJrVXUvazFvdHh3aFJWRzlaL2VrL1BsbnZGcFo2?=
 =?utf-8?B?Smpsa1o4QWxoZEszdm5LbTZIaGZFd2NEVjdyeUdpRXFSRnFYckhRaWtxam9i?=
 =?utf-8?B?VXpjbUF6clRkd2NCbjNsMUVhQitqdUJ2MG50MEp6RzR5VHZjN1FuQm9CVGdT?=
 =?utf-8?B?RzgxcVpjUEVNaXJJdzV5eGxsT0NwZXN1dGtYLy91SWdqMm13Q2N6L0ppaFVm?=
 =?utf-8?B?Mm5mZ0lqZmN0N0JCMHhGeUZxRllDejBNcU5oQ0k1SWFuUWtyNmtHZ3BNMjlx?=
 =?utf-8?B?Z0w1aTFSNXFHMXduMWRJWWE1bWszaUNTNmFnTW1jbENVVHVjNW01MG5FTFZY?=
 =?utf-8?B?SXJESFFJWEtUa2NTQ1ozMTV6eDhUbWNWUmNPb21zcTlNYlNsQWhhUzVEMFVV?=
 =?utf-8?B?bFhkQ0J0bUoxM2RZRVk4alhpK2Roa0ZOdUFBalo0OHczeVZpaWdUTEVGQ09Y?=
 =?utf-8?B?VUdyV3psdWdOdjVBTTJTQnU2S0I3aVhXUVJzQkZsbE9CcDZVMmZ4Sm1mMGUw?=
 =?utf-8?B?a1J2WW1XdVhPdWtOSk9HZWFGUjQ2cEhVMzZlV0JUaFpFYmFrL2NOQWw5UlQ4?=
 =?utf-8?B?dG0yR0wrMUFMWms3MUpna3FiTFhOVVJ5SmhtR1EvUVBuZjB3VERhKzc5QTVL?=
 =?utf-8?B?V3Q1aGswSlJ2WEdubDlLM0VvMEkrdnBHMWhyMkxvbGlETHVsRWRweFpmQ08v?=
 =?utf-8?B?Y2x3S2JYUGcwMjFxeENPZW1tUmpBQ0l2MWNHTFVZMEtFMm1VTTZQWDRpd3pP?=
 =?utf-8?B?VmJNajhJUjBKRm9XeHhjcXdCdUZMdXdCSnU5V1VpekxiWjZFZ1d5VGdLRzgv?=
 =?utf-8?B?T0VpdFZldUhqdGgyc0hVNmQycUlWbUFiVGlBcEFPVkhtWVpqYkp5RGVXMlNT?=
 =?utf-8?B?SDkyOGNzZVVCQW1OdTk1YUhLbDJSTkpUOXo2UlhlWkZGbnFpLzRpcXJNNFVF?=
 =?utf-8?B?c21sKy9Mc1ZsNmNJSkFRb01DYlpLV1ZVblVNYURTZzIyUlY5YlMwUmlaSHlo?=
 =?utf-8?B?UmNvTjBoaVdLd1hJdG1VdnRRa05ndGY5MExtVStla2UvUnBjcmRlc3NBWkRn?=
 =?utf-8?B?QmpTMEswRDNieGd0bzVSZ05HelR3MmxmUjRIQzFqVExHUDBkdjJGU2pKY1FF?=
 =?utf-8?B?Ujh2aTIrbnRxL1dUL2l2RlZlT09RODQ0S1pCMVJKV3JSVDFibXJYci8rVXR4?=
 =?utf-8?B?Y0dNZWlMR1k0c1BrQWd1ZG54N0xUYXFjUlJBVzZGeGE4RTFMbFJJZkJ5Nncr?=
 =?utf-8?B?dVJMTHZaY2lZd2FFZEdDMllZbXFIVjdINVE1OWRkTzZJQVBkNlZWYmtPZVI1?=
 =?utf-8?B?MHBScys5SWNLdzY0VGVYQi9VZVpKY1ZDWUQyYVRSR1FaNmhBS25iOWkxN3po?=
 =?utf-8?B?SmdQbkl6N3dVTExVeGR0ekpMUDdBZ0dzb1ArRmJ0RHpxNlMxSUtmdUtIUVhG?=
 =?utf-8?B?SDdnV0VWSlpmOGpGeTNUN2lpUk5hQXB6cUhTUTlaOTVJMDJDZ3ZjZFdQdS9r?=
 =?utf-8?B?ZTVYK002Y1lNcUc5QldOWkNIbU1NRWNxOVFzZ283OG1xNzNWQTYrby9Pcmkz?=
 =?utf-8?B?Vmg2a0pCNGUvcUoxRE1aWFlEOVJvOVh0dU0wSjVRTUNsTStaOEpQNTJiMTNr?=
 =?utf-8?Q?BAwcPOb2mVNnyzAsO3pYET4VGZnetXezMpUw1uT?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 13686aa6-258c-458b-d32e-08d8ed59db5f
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Mar 2021 17:42:26.3607
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YhWfqSW9zQCKIEUAdnUNqE+nJWgQH38uozfW5xK7KVUZiL+LIVnVFmv0RtVq2YnNRzU/JU16iHXYyOFRMnZxpA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3352
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9931 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0
 mlxscore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2103220128
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9931 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 phishscore=0
 lowpriorityscore=0 suspectscore=0 clxscore=1011 priorityscore=1501
 spamscore=0 adultscore=0 impostorscore=0 mlxlogscore=999 mlxscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2009150000 definitions=main-2103220128
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Cc: Roman, Christoph

On 3/22/21 1:41 AM, Peter Zijlstra wrote:
> On Fri, Mar 19, 2021 at 03:42:08PM -0700, Mike Kravetz wrote:
>> The locks acquired in free_huge_page are irq safe.  However, in certain
>> circumstances the routine update_and_free_page could sleep.  Since
>> free_huge_page can be called from any context, it can not sleep.
>>
>> Use a waitqueue to defer freeing of pages if the operation may sleep.  A
>> new routine update_and_free_page_no_sleep provides this functionality
>> and is only called from free_huge_page.
>>
>> Note that any 'pages' sent to the workqueue for deferred freeing have
>> already been removed from the hugetlb subsystem.  What is actually
>> deferred is returning those base pages to the low level allocator.
> 
> So maybe I'm stupid, but why do you need that work in hugetlb? Afaict it
> should be in cma_release().

My thinking (which could be totally wrong) is that cma_release makes no
claims about calling context.  From the code, it is pretty clear that it
can only be called from task context with no locks held.  Although,
there could be code incorrectly calling it today hugetlb does.  Since
hugetlb is the only code with this new requirement, it should do the
work.

Wait!!!  That made me remember something.
Roman had code to create a non-blocking version of cma_release().
https://lore.kernel.org/linux-mm/20201022225308.2927890-1-guro@fb.com/

There were no objections, and Christoph even thought there may be
problems with callers of dma_free_contiguous.

Perhaps, we should just move forward with Roman's patches to create
cma_release_nowait() and avoid this workqueue stuff?
-- 
Mike Kravetz