Received: by 2002:ac0:946b:0:0:0:0:0 with SMTP id j40csp801029imj; Thu, 7 Feb 2019 12:03:29 -0800 (PST) X-Google-Smtp-Source: AHgI3IYu3IxHWezSbeYDgzfYQw4CQa9w5ThESZ7LJjCcImmrckWC142yYhkiUbaxEkfdHmwQX4oy X-Received: by 2002:a63:5c41:: with SMTP id n1mr704302pgm.1.1549569809845; Thu, 07 Feb 2019 12:03:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549569809; cv=none; d=google.com; s=arc-20160816; b=zOO1kNyrtR5FdTpueNMLohj2UeFzrrXBbA6Kysoh1GygoD2owTpp3NnN+FmpFWBoYA X1tKYZ8blhkDG9gMoM0vhgEpPV4I6+86Cqwss+yA1RNmXelRtXjn79aNyT6f4lsNywXl Tg1DKVGE3iAJucGnyzEN04pDuE+YVVDesiQ7XlioxpwL2Q/cI56WSfQVl/1XRNU10wTK dVi8gEfRVG7B1JexWA4tGVJ+9mT4NwMa4QziPvf0GHeNLxGI8lcAwTqamm9XkB7Vx0Mm pu53fLEm0TMZeWP+jCT94V/9vE1HGo4cYEGgEUFbSVXM2CIDZn1vKo5lHMyxQ0dhj4c7 LgxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:subject:cc:to:from:date; bh=pnFWzImOoCKJOVougHoefMFBbDyHbsXZd3+mZnnIsd4=; b=N7XgfUPcgjIuPZ8Sd8Pf5Grj83OhzSO5swaeH9zjbbWmX0yiCMuOLfjGRn3YxfcBGl q2MvebuM5cC48sDgQd+FD799mbicNuuAonTMdZV11MJWNPgmWKclIjzVgxSDrS61dhhV Mu3qLYzTMwuco57QAU1CMmo/avCxf2pm5dDSOQY2F44sD55ESSbNv6eduFm5eZDuvZUb GeC10rr0iLIU8MCSNOOmB0wKrOZ/Ovz+0tECLROliN7Dc6JP1BbvAxBFDQmyLsvKkQEO Xhy/s9HC72Kyh7iTbu/o8GGYBrJL//wOLlIIkMDYuCbPy3obJGGhRnulMuo2sPjI+5V9 6oCg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w31si10279658pla.308.2019.02.07.12.03.14; Thu, 07 Feb 2019 12:03:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727294AbfBGUCi (ORCPT + 99 others); Thu, 7 Feb 2019 15:02:38 -0500 Received: from mx1.redhat.com ([209.132.183.28]:52272 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726512AbfBGUCi (ORCPT ); Thu, 7 Feb 2019 15:02:38 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 67BD28666B; Thu, 7 Feb 2019 20:02:37 +0000 (UTC) Received: from doriath (ovpn-116-107.phx2.redhat.com [10.3.116.107]) by smtp.corp.redhat.com (Postfix) with ESMTP id C1C2B600C3; Thu, 7 Feb 2019 20:02:35 +0000 (UTC) Date: Thu, 7 Feb 2019 15:02:04 -0500 From: Luiz Capitulino To: Alexander Duyck Cc: Alexander Duyck , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, rkrcmar@redhat.com, x86@kernel.org, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, pbonzini@redhat.com, tglx@linutronix.de, akpm@linux-foundation.org Subject: Re: [RFC PATCH 3/4] kvm: Add guest side support for free memory hints Message-ID: <20190207150204.7b305de7@doriath> In-Reply-To: <34c93e5a05a7dc93e38364733f8832f2e1b2dcb3.camel@linux.intel.com> References: <20190204181118.12095.38300.stgit@localhost.localdomain> <20190204181552.12095.46287.stgit@localhost.localdomain> <20190207132104.17a296da@doriath> <34c93e5a05a7dc93e38364733f8832f2e1b2dcb3.camel@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Thu, 07 Feb 2019 20:02:37 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 07 Feb 2019 10:44:11 -0800 Alexander Duyck wrote: > On Thu, 2019-02-07 at 13:21 -0500, Luiz Capitulino wrote: > > On Mon, 04 Feb 2019 10:15:52 -0800 > > Alexander Duyck wrote: > > > > > From: Alexander Duyck > > > > > > Add guest support for providing free memory hints to the KVM hypervisor for > > > freed pages huge TLB size or larger. I am restricting the size to > > > huge TLB order and larger because the hypercalls are too expensive to be > > > performing one per 4K page. Using the huge TLB order became the obvious > > > choice for the order to use as it allows us to avoid fragmentation of higher > > > order memory on the host. > > > > > > I have limited the functionality so that it doesn't work when page > > > poisoning is enabled. I did this because a write to the page after doing an > > > MADV_DONTNEED would effectively negate the hint, so it would be wasting > > > cycles to do so. > > > > > > Signed-off-by: Alexander Duyck > > > --- > > > arch/x86/include/asm/page.h | 13 +++++++++++++ > > > arch/x86/kernel/kvm.c | 23 +++++++++++++++++++++++ > > > 2 files changed, 36 insertions(+) > > > > > > diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h > > > index 7555b48803a8..4487ad7a3385 100644 > > > --- a/arch/x86/include/asm/page.h > > > +++ b/arch/x86/include/asm/page.h > > > @@ -18,6 +18,19 @@ > > > > > > struct page; > > > > > > +#ifdef CONFIG_KVM_GUEST > > > +#include > > > +extern struct static_key_false pv_free_page_hint_enabled; > > > + > > > +#define HAVE_ARCH_FREE_PAGE > > > +void __arch_free_page(struct page *page, unsigned int order); > > > +static inline void arch_free_page(struct page *page, unsigned int order) > > > +{ > > > + if (static_branch_unlikely(&pv_free_page_hint_enabled)) > > > + __arch_free_page(page, order); > > > +} > > > +#endif > > > + > > > #include > > > extern struct range pfn_mapped[]; > > > extern int nr_pfn_mapped; > > > diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c > > > index 5c93a65ee1e5..09c91641c36c 100644 > > > --- a/arch/x86/kernel/kvm.c > > > +++ b/arch/x86/kernel/kvm.c > > > @@ -48,6 +48,7 @@ > > > #include > > > > > > static int kvmapf = 1; > > > +DEFINE_STATIC_KEY_FALSE(pv_free_page_hint_enabled); > > > > > > static int __init parse_no_kvmapf(char *arg) > > > { > > > @@ -648,6 +649,15 @@ static void __init kvm_guest_init(void) > > > if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) > > > apic_set_eoi_write(kvm_guest_apic_eoi_write); > > > > > > + /* > > > + * The free page hinting doesn't add much value if page poisoning > > > + * is enabled. So we only enable the feature if page poisoning is > > > + * no present. > > > + */ > > > + if (!page_poisoning_enabled() && > > > + kvm_para_has_feature(KVM_FEATURE_PV_UNUSED_PAGE_HINT)) > > > + static_branch_enable(&pv_free_page_hint_enabled); > > > + > > > #ifdef CONFIG_SMP > > > smp_ops.smp_prepare_cpus = kvm_smp_prepare_cpus; > > > smp_ops.smp_prepare_boot_cpu = kvm_smp_prepare_boot_cpu; > > > @@ -762,6 +772,19 @@ static __init int kvm_setup_pv_tlb_flush(void) > > > } > > > arch_initcall(kvm_setup_pv_tlb_flush); > > > > > > +void __arch_free_page(struct page *page, unsigned int order) > > > +{ > > > + /* > > > + * Limit hints to blocks no smaller than pageblock in > > > + * size to limit the cost for the hypercalls. > > > + */ > > > + if (order < KVM_PV_UNUSED_PAGE_HINT_MIN_ORDER) > > > + return; > > > + > > > + kvm_hypercall2(KVM_HC_UNUSED_PAGE_HINT, page_to_phys(page), > > > + PAGE_SIZE << order); > > > > Does this mean that the vCPU executing this will get stuck > > here for the duration of the hypercall? Isn't that too long, > > considering that the zone lock is taken and madvise in the > > host block on semaphores? > > I'm pretty sure the zone lock isn't held when this is called. The lock > isn't acquired until later in the path. This gets executed just before > the page poisoning call which would take time as well since it would > have to memset an entire page. This function is called as a part of > free_pages_prepare, the zone locks aren't acquired until we are calling > into either free_one_page and a few spots before calling > __free_one_page. Yeah, you're right of course! I think mixed up __arch_free_page() and __free_one_page()... free_pages() code path won't take any locks up to calling __arch_free_page(). Sorry for the noise. > My other function in patch 4 which does this from inside of > __free_one_page does have to release the zone lock since it is taken > there. I haven't checked that one yet, I'll let you know if I have comments.