Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 16 Jun 2023 15:50:19 +0800
From:   Yan Zhao <yan.y.zhao@intel.com>
To:     Yuan Yao <yuan.yao@linux.intel.com>
CC:     <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <pbonzini@redhat.com>, <seanjc@google.com>, <chao.gao@intel.com>,
        <kai.huang@intel.com>, <robert.hoo.linux@gmail.com>
Subject: Re: [PATCH v3 09/11] KVM: x86/mmu: serialize vCPUs to zap gfn when
 guest MTRRs are honored
Message-ID: <ZIwUO/8W7dRYt2s8@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20230616023101.7019-1-yan.y.zhao@intel.com>
 <20230616023945.7570-1-yan.y.zhao@intel.com>
 <20230616074550.g2ikzbni2rjy7dfw@yy-desk-7060>
 <ZIwROWti5d0sCFwT@yzhao56-desk.sh.intel.com>
 <20230616080917.fhekzs2fyhqtbitx@yy-desk-7060>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230616080917.fhekzs2fyhqtbitx@yy-desk-7060>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?WwPxe3pVlaaTY0Xin4oS+XyjLTdyq9NlB9sBFSWQJiHqNOLrf5WigBlYXfAS?=
 =?us-ascii?Q?fHjU5sZ1Zf/b5pc+rTGjhdeCE1TmqLTABEaTDwSx0wtEsO6bqhsXRSx9RVOH?=
 =?us-ascii?Q?Wx8MCgFqYiAz6nCMmhU3Wab2BTquho1t9OIcwFQm2WkvJoHc1BlWZJXdRGA4?=
 =?us-ascii?Q?80zgtb3gHsZmG2OAxR6tl3xKiMVBgueFeGvVy2JhbqqGM3sUO9gd50p45j5v?=
 =?us-ascii?Q?qQJwOuVkXGZdmMNZh+/k/4MzBjSRcoGMLgCRBec4+HnomeBx9qjcJk6k4fLB?=
 =?us-ascii?Q?TO8L4VtSLPotylHFp5Cvq4TwNjQiKEHuKfgVJhwej3Zn3kPN/z+snrYsiRhF?=
 =?us-ascii?Q?lVRMiIKs0MOcuG1v8EEQRpCY52+p4ZaMRwRZFARxr7f8/TWdpflsz1dZ7o33?=
 =?us-ascii?Q?OJ3hITyhiabhvp7hLXO6ygqKepv/HyZiHpxRlLU9iK1EC8EsSe3kMpGG8qr0?=
 =?us-ascii?Q?3MlAweR8t7cId16aKOpFhRrS4YV/dU3UmI7aeuk4hdnRXk91XRtcP1YBqMVg?=
 =?us-ascii?Q?km1vcg+VRe5RoJ53KzlWojp9kJQ4u3gjnM9RdS1LF9aA61yuPVSYPPuxffMK?=
 =?us-ascii?Q?G+mz7AfzwNk8QgqCtUXUxlBka45ntUWSHZrEkAR0qCFf5Q8TD4lxok/ra5V2?=
 =?us-ascii?Q?XGXtMTg/gKEC/UQK1qqXCi9buK8Bh36Gy8ROv2W266Lr2mamKKwQHJ5FOCy+?=
 =?us-ascii?Q?TQ1UbbweaItadod2vVxDzOTj5OApAMFe6KPMJWw80rqxy81USXxFesuWc7eM?=
 =?us-ascii?Q?GOKV7Rl/cGTNcBE25gQe1dMhqptVFsy/5u2Oc2o8lAll5wIQFWWZ/RKzlUQ+?=
 =?us-ascii?Q?YGOlKEReEM3VI2281NUA07iMMwhcRadBE+/w9ve2K5++ibxwkvVEXJaC09z4?=
 =?us-ascii?Q?rUegInUqnIMmyNn+5gwyLltY9PcdVjy2XGuPZIfdSHlTnPwbdzWf4I34iWlB?=
 =?us-ascii?Q?c5enltFvS27iiXXvW2dyIusqQgamGIFqlWMpoVxKeWNyVXP5mjrpHKCAUchl?=
 =?us-ascii?Q?EyCIoLxtX+dOkFF50nReRPt6YWQdKp1XI33CvC03wmijsIyVQFnCndKWfA5g?=
 =?us-ascii?Q?o6zsGMW+kPD1RJYZoDNrGxr5CAcjpgP7RAzkGVFHaiYZqGe/GXdf2XCfhmbh?=
 =?us-ascii?Q?69y1QDMbBDubsfYhUXAOt0nUevvaAg+72ZUjPw14pxfZXDaVwPcgmJxLckvE?=
 =?us-ascii?Q?BBvGy8zitGLqMWEWt7ViZHkZcNijUXWCzyh+cThH+h+hB8dnmJhjgxeSAXBe?=
 =?us-ascii?Q?ybecdk9l+WoCMQNLlvpZaSeosrH++6blrCiULmiqSqvrs26LYj+4QJRtvOuV?=
 =?us-ascii?Q?WKkbySsgNIUeEnC+gEwvXb1/iTV4MG/cch8oEnTTtUZeZnOxo96zRxt80FbH?=
 =?us-ascii?Q?2NQtyydi4jzMZNRxG6F6bWOS9+dRzwHp/tp131JiF70dWPVPfgJHZjQRtGbZ?=
 =?us-ascii?Q?m0or5WOzl2OeDwt7sL2BsXhusGhgGOJ24j7+mz5jcIRpoa7L5Yc2Q6KaKi7J?=
 =?us-ascii?Q?TqS7eoYDj+iU/nGEgLYVFxRvQgQzgQm/dFKDisPeyrRtJtXol75pMdLJlJ2E?=
 =?us-ascii?Q?ZKIe2plPtvaJ7abRG6FSzsW9MfE8B0d0PMzHp8CE?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e81f7dd3-2afd-481c-1bfe-08db6e41dd0a
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2023 08:15:36.5600
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 26+STPTAwZXd1t5fkYzlhS+cmdQa3vrGECoPgOMzO2YzLnSSfSPwfCPFr6WPk0bAZ73z50saiLAVr56udKWRLA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB6223
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_PASS,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Jun 16, 2023 at 04:09:17PM +0800, Yuan Yao wrote:
> On Fri, Jun 16, 2023 at 03:37:29PM +0800, Yan Zhao wrote:
> > On Fri, Jun 16, 2023 at 03:45:50PM +0800, Yuan Yao wrote:
> > > > +/*
> > > > + * Add @range into kvm->arch.mtrr_zap_list and sort the list in
> > > > + * "length" ascending + "start" descending order, so that
> > > > + * ranges consuming more zap cycles can be dequeued later and their
> > > > + * chances of being found duplicated are increased.
> > > > + */
> > > > +static void kvm_add_mtrr_zap_list(struct kvm *kvm, struct mtrr_zap_range *range)
> > > > +{
> > > > +	struct list_head *head = &kvm->arch.mtrr_zap_list;
> > > > +	u64 len = range->end - range->start;
> > > > +	struct mtrr_zap_range *cur, *n;
> > > > +	bool added = false;
> > > > +
> > > > +	spin_lock(&kvm->arch.mtrr_zap_list_lock);
> > > > +
> > > > +	if (list_empty(head)) {
> > > > +		list_add(&range->node, head);
> > > > +		spin_unlock(&kvm->arch.mtrr_zap_list_lock);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	list_for_each_entry_safe(cur, n, head, node) {
> > > > +		u64 cur_len = cur->end - cur->start;
> > > > +
> > > > +		if (len < cur_len)
> > > > +			break;
> > > > +
> > > > +		if (len > cur_len)
> > > > +			continue;
> > > > +
> > > > +		if (range->start > cur->start)
> > > > +			break;
> > > > +
> > > > +		if (range->start < cur->start)
> > > > +			continue;
> > > > +
> > > > +		/* equal len & start, no need to add */
> > > > +		added = true;
> > >
> > > Possible/worth to ignore the range already covered
> > > by queued range ?
> >
> > I may not get you correctly, but
> > the "added" here means an queued range with exactly same start + len
> > found, so free and drop adding the new range here.
> 
> I mean drop adding three B below if A already in the queue:
> 
> |------A--------|
> |----B----|
> 
> |------A--------|
>       |----B----|
> 
> |------A--------|
>   |----B----|
> 
Oh, I implemented this way in my first version.
But it will complicate the logic and increase time holding spinlock.
And as usually in the zaps caused by MTRRs update and CR0.CD toggles,
the queued ranges are duplicated in different vCPUs and non-overlapping
in one vCPU, I turned to this simplier implemenation finally.