Received: by 2002:a05:6a10:c604:0:0:0:0 with SMTP id y4csp37203pxt; Thu, 5 Aug 2021 17:11:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxSj+1hHycVUwFWgDimE8KMyOml1USXYQJJ/DpmfIVaCQ1uA+4WX4khuNaaL1n3WILyooxq X-Received: by 2002:a6b:ea18:: with SMTP id m24mr148976ioc.76.1628208718200; Thu, 05 Aug 2021 17:11:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628208718; cv=none; d=google.com; s=arc-20160816; b=UBb8MN+WrjIGA/LhFu6JMiWcnmOQnR4p5e1d/mP36Aa+AjOwL60oRiXzKMEWng2FaE OxgZOI0ef7YiObsQbtWN+AgrvxtOVmhHAr7qOvORr4QgmX0CYC7zSZBlABUfJWtvhPJb SrSFFBE2wYmQLReUZ3hTGqd7+0LHYV4ZEJKCZ1X2rdAfoXerM4oHD+L7Woa3m821CojN 3Rr9xc5FCAU45tC9eivleSrRCc/InUPVG3+e9SlXLuX6awPdUR0SSRzYV+S3HnDwpoCH YPiIKfx2gBcL4thPHqARKj5GO/EMPt8oSf2rc/xienOmU5lV0yNtNuOxSi52zCzMAz+C VKEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=25dI2qwAi7TSMZKSjGmJYglQgYbrwOJcxN6GHCDQJtY=; b=c3dcQBok3QYUS8txCKSdNYrW8uTqhjUNFLKAUJp1fek5AGfdKAZYbWSFx36XwS5DMj 8/tCZj18UfCMYMULzCzw+81u+RAyITxO/0yDFnb8EZk6luEvRNL2PogryHoRkCdI6Gkd iuqVVC455BYcR5YVff3EWyZ7BhdBC4tt0wUpBpDWyaHiJKX4pdSDcIHHtTLXUCcbnbXu TzpOKKfoBjKEL4nxAoH+9CrhQsatkcjuyEyJ8GO19yFLbdqssl4qzr/MCCrYWCEGnGgK 0TKKYZlPzeIpjiQQNteH3dkbgEM+oZqJw9P+3lqwGkHaTZ7s5c+4KCwAQaoZi7P293NP zMtA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=uFTSZPbF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id n2si7318487jaj.0.2021.08.05.17.11.45; Thu, 05 Aug 2021 17:11:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=uFTSZPbF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239987AbhHERj3 (ORCPT + 99 others); Thu, 5 Aug 2021 13:39:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239557AbhHERj2 (ORCPT ); Thu, 5 Aug 2021 13:39:28 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D468C061765 for ; Thu, 5 Aug 2021 10:39:14 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id u13-20020a17090abb0db0290177e1d9b3f7so16625967pjr.1 for ; Thu, 05 Aug 2021 10:39:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=25dI2qwAi7TSMZKSjGmJYglQgYbrwOJcxN6GHCDQJtY=; b=uFTSZPbFCNjb65mQPn0Er3edoI5Tmo52V5M7wb5ifPkYDUeTgC/9MnNHvvfC5T4c6D fg9Hf9rnez9xcFsdyFWGWwMsaYVxXUtpI3M0YlwSoG00LC07NRKudSF7L7svQjkGaWlm SnDiKDpT/ncc0KXEeCLwfBU4UOqtHjsqxieGTHZe1/TRJQvEF3XilbYTdFQ6iTWGwGVO bf63iXpy7D3DFsaAmEVEOEROXRQ+dA8w5RfaWGPayfYPszXElngWAshhZgXFVYSGvIYI vAewl8g3XCgeJLYC7KC756Imz2qE8gu7BOzxLBF0o4EioAMBrMAR7grb9J+qR6SyWhXS ReZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=25dI2qwAi7TSMZKSjGmJYglQgYbrwOJcxN6GHCDQJtY=; b=PzB5FRWtZnKX5wZ1lMbsHet9ZFJ3e9ghvzNIhXd+t56dl97MaWCcH3VTYllLFrJum0 1YQl0UQAVxkqPzANqSKF4xqzSeSPq/TvUrRfxKfRlaweI6CfywA765E5bkJ1QGun40UR hILmp1Aa3/GDhJElP4GVy2/zcI+5K30Qzd00reryP3WC4k7D1k3flGvewdK59PolcWNt xdtLlQakXj/0iTcjzjEK5rdmIvFQCtt1ZVKdjkJhvc4eyTwNdPVtkbfMGatzN0qDz4bF Z/Wk/cWfyrkT8f9UMAhtMFfs3d3WeoGALq90NRcQr8n9RCp17DV23zk+3tIpYFet0j4l la5Q== X-Gm-Message-State: AOAM5328kL2XpMDubWcHgGiF+7QI32LZZNN3hi08piXuQ2PM5HdCzLzU lF9Zuo6X8Hd8WyhzQ21TVker0w== X-Received: by 2002:a17:90a:8049:: with SMTP id e9mr16826271pjw.160.1628185153210; Thu, 05 Aug 2021 10:39:13 -0700 (PDT) Received: from google.com (157.214.185.35.bc.googleusercontent.com. [35.185.214.157]) by smtp.gmail.com with ESMTPSA id j3sm7677397pfe.98.2021.08.05.10.39.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Aug 2021 10:39:12 -0700 (PDT) Date: Thu, 5 Aug 2021 17:39:08 +0000 From: Sean Christopherson To: "Edgecombe, Rick P" Cc: "Huang, Kai" , "erdemaktas@google.com" , "linux-kernel@vger.kernel.org" , "jmattson@google.com" , "kvm@vger.kernel.org" , "vkuznets@redhat.com" , "tglx@linutronix.de" , "ckuehl@redhat.com" , "joro@8bytes.org" , "x86@kernel.org" , "hpa@zytor.com" , "mingo@redhat.com" , "pbonzini@redhat.com" , "bp@alien8.de" , "Yamahata, Isaku" , "isaku.yamahata@gmail.com" , "wanpengli@tencent.com" Subject: Re: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits Message-ID: References: <20210805234424.d14386b79413845b990a18ac@intel.com> <78b802bbcf72a087bcf118340eae89f97024d09c.camel@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <78b802bbcf72a087bcf118340eae89f97024d09c.camel@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Aug 05, 2021, Edgecombe, Rick P wrote: > On Thu, 2021-08-05 at 16:06 +0000, Sean Christopherson wrote: > > On Thu, Aug 05, 2021, Kai Huang wrote: > > > And removing 'gfn_stolen_bits' in 'struct kvm_mmu_page' could also save > > > some memory. > > > > But I do like saving memory... One potentially bad idea would be to > > unionize gfn and stolen bits by shifting the stolen bits after they're > > extracted from the gpa, e.g. > > > > union { > > gfn_t gfn_and_stolen; > > struct { > > gfn_t gfn:52; > > gfn_t stolen:12; > > } > > }; > > > > the downsides being that accessing just the gfn would require an additional > > masking operation, and the stolen bits wouldn't align with reality. > > It definitely seems like the sp could be packed more efficiently. Yeah, in general it could be optimized. But for TDP/direct MMUs, we don't care thaaat much because there are relatively few shadow pages, versus indirect MMUs with thousands or tens of thousands of shadow pages. Of course, indirect MMUs are also the most gluttonous due to the unsync_child_bitmap, gfns, write flooding count, etc... If we really want to reduce the memory footprint for the common case (TDP MMU), the crud that's used only by indirect shadow pages could be shoved into a different struct by abusing the struct layout and and wrapping accesses to the indirect-only fields with casts/container_of and helpers, e.g. struct kvm_mmu_indirect_page { struct kvm_mmu_page this; gfn_t *gfns; unsigned int unsync_children; DECLARE_BITMAP(unsync_child_bitmap, 512); #ifdef CONFIG_X86_32 /* * Used out of the mmu-lock to avoid reading spte values while an * update is in progress; see the comments in __get_spte_lockless(). */ int clear_spte_count; #endif /* Number of writes since the last time traversal visited this page. */ atomic_t write_flooding_count; } > One other idea is the stolen bits could just be recovered from the role > bits with a helper, like how the page fault error code stolen bits > encoding version of this works. As in, a generic "stolen_gfn_bits" in the role instead of a per-feature role bit? That would avoid the problem of per-feature role bits leading to a pile of marshalling code, and wouldn't suffer the masking cost when accessing ->gfn, though I'm not sure that matters much. > If the stolen bits are not fed into the hash calculation though it > would change the behavior a bit. Not sure if for better or worse. Also > the calculation of hash collisions would need to be aware. The role is already factored into the collision logic. > FWIW, I kind of like something like Sean's proposal. It's a bit > convoluted, but there are more unused bits in the gfn than the role. And tightly bound, i.e. there can't be more than gfn_t gfn+gfn_stolen bits. > Also they are a little more related.