Received: by 2002:a05:6a10:1d13:0:0:0:0 with SMTP id pp19csp433106pxb; Thu, 19 Aug 2021 03:08:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzY3mIFJ21DXBbBWKZ3OMN1EYNEjxWscSdDE5/uA9WEiXbxq+bzW1evTJH7Ir+MlfdrL2m7 X-Received: by 2002:a05:6402:3099:: with SMTP id de25mr15368943edb.36.1629367720732; Thu, 19 Aug 2021 03:08:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1629367720; cv=none; d=google.com; s=arc-20160816; b=0RlTIcNTfzPFW0iduuw3PpEeHCzceveRcW60LNa7l+NEsA1nkhEWN8+VnVd4k3tGHS Od/zv5w74tFowL8ZhXppoSf2RF73XZfj4l8QJVAxRLyQ7if35JonikcQnamKryqjDTXA UbxvjVD7YoYQPh0hB1eV82UaF0BOEXbaY6Kk7sR1UPGlaWABVckpm/+lmrvR3/2c0Ocv 7oK4EfASA6+glEJIV7pb3WHr5YPVgLf+Q+bX+xhtJcuEF1I9Zg98j6J0W3lgaHu1/DL0 ZbG1FU58rl4x/BkOsGAb2eURIOeXyFBTShRa2bNW18bWkqc7p8KO4N7Dnl3Iv/V6caWx dbkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:content-language :in-reply-to:mime-version:user-agent:date:message-id:organization :from:references:cc:to:subject:dkim-signature; bh=Opv2d9ol21udi4K0v30kbE3GUCegq3iBRM6wdcQpyVk=; b=X7Yqxm3W2Di1BXYGdzGcydjA7dJXJLNqgbzKQDcyq6/mA1TwOI95lq2xYpzvsxU79n 8wEApHp0KGUFFKLTjnG+mra0xyQzpTKeqLFWhuDZJu+St7QHWJN/RP7Gx+rOyYkoSH+4 PUDpvJZjOhgzvdnUGnu1kOBB+Ax3pylPdewn/AuHQqoE9A3E9zKUxDRzWS+B+Yam3XtS AaGEShaeNsVrLf54QfXsmS62cH+utfcIUBlJqY0d5CgqHMbxTcrr+nz+BHJO8ggbJpVk kMGM8Hdr5ptXH4msqmAqZ4ITeI/l2LDfIcC0hI2cb4hpatsJkQRSUdZ15XEU7BeJbKXh So+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HXzJmEAW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id ar8si2997296ejc.257.2021.08.19.03.08.17; Thu, 19 Aug 2021 03:08:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HXzJmEAW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237984AbhHSKHW (ORCPT + 99 others); Thu, 19 Aug 2021 06:07:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:43425 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237746AbhHSKHV (ORCPT ); Thu, 19 Aug 2021 06:07:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1629367605; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Opv2d9ol21udi4K0v30kbE3GUCegq3iBRM6wdcQpyVk=; b=HXzJmEAWYoOmyYEJhI6WLMBlnyUCgnVnsXrAayE8oDmK4SCc796d7uiYihvmMMLGEBXAv+ huIK4TVN7vHaI6Fi/WZFP+fXjDGn7XnuGY3Af2KKcaCt3OUqP9gMbyE+j7D1ICT5zxraP9 rbyEyXsQOBNejANgII12W14hLpw5XJo= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-491-FYtCgnA-O_24Z0mnxc5XPg-1; Thu, 19 Aug 2021 06:06:43 -0400 X-MC-Unique: FYtCgnA-O_24Z0mnxc5XPg-1 Received: by mail-wr1-f72.google.com with SMTP id p4-20020a5d59a4000000b00156992180dbso1527029wrr.10 for ; Thu, 19 Aug 2021 03:06:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=Opv2d9ol21udi4K0v30kbE3GUCegq3iBRM6wdcQpyVk=; b=H4UzCzyv0qNhZ9d0q4VFuynbVYARFwnoK08gwnDAxOgM9Zoi6RGcYhQkrDxTt7bKR8 l6GgquOLCXLWiq7t7buPzuelkMseIzuxxAZnC2/Q/B5EGp+5Ox664LXsNP78QvKCQWj3 8rYTl39OA624foKx9VKzetom6M0Mw1NO1xhZUY6XAgBJo48PM074ff6oDoq4qYQEznqk ZDbSj74pqE3JBuvGyT7vtvlXswtZv6qMh8P78yXJpiMz5Ev6h8fe0EhJBvCtbAjdZMIk d4pdSYjfVAodEx8znU0w9aJrc0+7poQYF01y70ZfvqY5MSjgF9Suo5vBVdAgx++84tjo u6Sg== X-Gm-Message-State: AOAM531DYq2HwKl9OASPI1c+hjRoj5VgqtaRRrdNaDDaWRcxwechuKdw ffbV1dczX6dWD9xsm5hGWXWeVKxLLo8cKboPB28NIO7/y+q8MeB8M9mprC1FjKDrGVo66wbeTTD NS6d0Z2rVhpq9l5Buwfb+S+Ul X-Received: by 2002:a5d:6483:: with SMTP id o3mr2757120wri.197.1629367602746; Thu, 19 Aug 2021 03:06:42 -0700 (PDT) X-Received: by 2002:a5d:6483:: with SMTP id o3mr2757086wri.197.1629367602415; Thu, 19 Aug 2021 03:06:42 -0700 (PDT) Received: from [192.168.3.132] (p5b0c6bd1.dip0.t-ipconnect.de. [91.12.107.209]) by smtp.gmail.com with ESMTPSA id y1sm2167170wmq.43.2021.08.19.03.06.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Aug 2021 03:06:41 -0700 (PDT) Subject: Re: [PATCH 1/5] mm: Add support for unaccepted memory To: Joerg Roedel Cc: Dave Hansen , Andi Kleen , "Kirill A. Shutemov" , Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Varad Gautam , Dario Faggioli , x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" References: <9748c07c-4e59-89d0-f425-c57f778d1b42@linux.intel.com> <17b6a3a3-bd7d-f57e-8762-96258b16247a@intel.com> <796a4b20-7fa3-3086-efa0-2f728f35ae06@linux.intel.com> <3caf5e73-c104-0057-680c-7851476e67ac@linux.intel.com> <25312492-5d67-e5b0-1a51-b6880f45a550@intel.com> From: David Hildenbrand Organization: Red Hat Message-ID: <39d09b80-f79e-0dab-2be7-31d4db43ab77@redhat.com> Date: Thu, 19 Aug 2021 12:06:40 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 19.08.21 11:55, Joerg Roedel wrote: > Hi David, > > On Tue, Aug 17, 2021 at 05:00:55PM +0200, David Hildenbrand wrote: >> Not sure if already discussed, but what about making sure that free pages >> are not a mixture (partially unaccepted, partially accepted). >> >> You'd have to expose the pages in that granularity to the buddy >> (__free_pages_core), indicating the state. You'd have to reject merging >> pages of differing acceptance state. >> >> Accepting a page would then be handled outside of the zone lock, completely >> controlled by the state. >> >> So a page in the buddy would either be completely accepted or completely >> unaccepted, signaled e.g., by PageOffline(). >> >> Consequently, when allocating a 4KiB page, you'd split an unaccepted 2MiB >> page into separate unaccepted pages. You'd grab one of the unaccepted 4KiB >> pages and accept it before initializing it and handing it out. > > Yes, that is the alternative to over-accepting memory on allocation. But > the problem here is that accepting/validating memory is an expensive > operation which also requires a hypercall. The hypercalls on SNP and TDX > can accept/validate multiple pages in one call. So the recommendation is > to accept memory in bigger chunks, like the 2MB that have been proposed. > The general idea would be to have one thread scanning the free page list and accepting pages in the background. Take a page out, accept it, release it back to the buddy. If we place accepted pages to the front of the free list, allocations would be quite fast. Sure, you'd have some slow early allocations, but I'd guess it really wouldn't make a big impact overall. We'd have to try. It's quite similar to the free page reporting infrastructure, except that with free page reporting we want to place pages to the tail, not to the front. That would be easy to add. > Only accepting memory in allocation size might be too slow, as there is > a lot of code doing order-0 allocations. I think this approach will also > be more intrusive to the page alloctor, as it needs more changes on the > free path to check for acceptance states before pages can be merged. That's already mostly done in this patch IIRC. -- Thanks, David / dhildenb