Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754506AbcDSQub (ORCPT ); Tue, 19 Apr 2016 12:50:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45521 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752540AbcDSQu3 (ORCPT ); Tue, 19 Apr 2016 12:50:29 -0400 Date: Tue, 19 Apr 2016 12:50:24 -0400 From: Andrea Arcangeli To: "Shi, Yang" Cc: "Kirill A. Shutemov" , Hugh Dickins , Andrew Morton , Dave Hansen , Vlastimil Babka , Christoph Lameter , Naoya Horiguchi , Jerome Marchand , Sasha Levin , Andres Lagar-Cavilla , Ning Qu , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCHv7 00/29] THP-enabled tmpfs/shmem using compound pages Message-ID: <20160419165024.GB24312@redhat.com> References: <1460766240-84565-1-git-send-email-kirill.shutemov@linux.intel.com> <571565F0.9070203@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <571565F0.9070203@linaro.org> User-Agent: Mutt/1.6.0 (2016-04-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2529 Lines: 64 Hello, On Mon, Apr 18, 2016 at 03:55:44PM -0700, Shi, Yang wrote: > Hi Kirill, > > Finally, I got some time to look into and try yours and Hugh's patches, > got two problems. One thing that come to mind to test is this: qemu with -machine accel=kvm -mem-path=/dev/shm/,share=on . The THP Compound approach in tmpfs may just happen to work already with KVM (or at worst it'd require minor adjustments) because it uses the exact same model KVM is already aware about from THP in anonymous memory, example from arch/x86/kvm/mmu.c: static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu, gfn_t *gfnp, kvm_pfn_t *pfnp, int *levelp) { kvm_pfn_t pfn = *pfnp; gfn_t gfn = *gfnp; int level = *levelp; /* * Check if it's a transparent hugepage. If this would be an * hugetlbfs page, level wouldn't be set to * PT_PAGE_TABLE_LEVEL and there would be no adjustment done * here. */ if (!is_error_noslot_pfn(pfn) && !kvm_is_reserved_pfn(pfn) && level == PT_PAGE_TABLE_LEVEL && PageTransCompound(pfn_to_page(pfn)) && !mmu_gfn_lpage_is_disallowed(vcpu, gfn, PT_DIRECTORY_LEVEL)) { Not using two different models between THP in tmpfs and THP in anon is essential not just to significantly reduce the size of the kernel code, but also because THP knowledge can't be self contained in the mm/shmem.c file. Having to support two different models would complicate things for secondary MMU drivers (i.e. mmu notifer users) like KVM who also need to create huge mapping in the shadow pagetable layer in arch/x86/kvm if the primary MMU allows for it. > x86-64 and ARM64 with yours and Hugh's patches (linux-next tree), I got > the program execution time reduced by ~12% on x86-64, it looks very > impressive. Agreed, both patchset are impressive works and achieving amazing results! My view is that in terms of long-lived computation from userland point of view, both models are malleable enough and could achieve everything we need in the end, but as far as the overall kernel efficiency is concerned the compound model will always retain a slight advantage in performance by leveraging a native THP compound refcounting that requires just one atomic_inc/dec per THP mapcount instead of 512 of them. Other advantages of the compound model is that it's half in code size despite already including khugepaged (i.e. the same split_huge_page works for both tmpfs and anon) and like said above it won't introduce much complications for drivers like KVM as the model didn't change. Thanks, Andrea