Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 11 Aug 2023 13:22:15 +0800
From:   Yan Zhao <yan.y.zhao@intel.com>
To:     Chao Gao <chao.gao@intel.com>
CC:     <linux-mm@kvack.org>, <linux-kernel@vger.kernel.org>,
        <kvm@vger.kernel.org>, <pbonzini@redhat.com>, <seanjc@google.com>,
        <mike.kravetz@oracle.com>, <apopple@nvidia.com>, <jgg@nvidia.com>,
        <rppt@kernel.org>, <akpm@linux-foundation.org>,
        <kevin.tian@intel.com>, <david@redhat.com>
Subject: Re: [RFC PATCH v2 0/5] Reduce NUMA balance caused TLB-shootdowns in
 a VM
Message-ID: <ZNXFhzwWIGAI8KP2@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20230810085636.25914-1-yan.y.zhao@intel.com>
 <ZNTtExiPZx4b180d@chao-email>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZNTtExiPZx4b180d@chao-email>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?phQigCwxc18JFkSYe6Cj0fgccNiPPsG6p/a18Geew76jd3qd2BQZqvnbdcVo?=
 =?us-ascii?Q?REagu0SE4O/j/BDUXa3YQG9i0aJuH4o+/DizH60BG+LMHbD4iz/3GcqqZU/4?=
 =?us-ascii?Q?g7TsnTi29mDH0tpmjpk0eELhfgxsfcrsWDs+O+ZRO6/LF36CKdO91UbuOitU?=
 =?us-ascii?Q?YZuCDuFUv3NjhswKCFeCBFW8ffnkV89yc20zsv8gmyt4beWP0gJMJib+m4kT?=
 =?us-ascii?Q?RwcWCjRrCSkc791qgTeeHq0IbiljFP/Eu4hBakEgzzdmY6GVzLs6d4v80Rjn?=
 =?us-ascii?Q?rBGXMz2hpUGu1v8Tj2SVN2i0jJRVpmQcNbB/nfrEs4WjZkH3ZTsIbeeDuwiW?=
 =?us-ascii?Q?57a+2jEqZ0CkjQe2tQieuEnA1UinfeLrQ3UqjiTdPOufLGTuty+DJhYGxw3T?=
 =?us-ascii?Q?oEIdMVUD2iBuxrDIPj6CRCWqIasFt0ox5GyFTMpzu0Y1Fai8w33cn/K/AwWr?=
 =?us-ascii?Q?lVJvwJUnUjZxjgAjt68xUKyW1WNfPhqDS2q1OgvkJiEqxiVInKudunG8Jg2K?=
 =?us-ascii?Q?UEFCfsWkeBm0kvDsc9WuT7soXJIQfyMd4GA43uoygoQVym1hnsUPy0NRNson?=
 =?us-ascii?Q?NI98W1OzKbGmAklre1pXS3mr8cq86TxBXtUk4HflJ5B+YmFw1iAkLr/i+Srh?=
 =?us-ascii?Q?9JGKRhu4eR1tJUCnyDR5l4ZLVhJdAOc4pRRbBV9U+d5fCSbyZlZUUPaUEC/T?=
 =?us-ascii?Q?sKEUMkhb886n/s1SxSRRbltdr5sfSQ+9N+KXHKOlLScc2on328apgq3cebks?=
 =?us-ascii?Q?YzMkRH0EGTMFkyGBmx9R+eVgwe97UVAwu/gTN7Cr3t/HzNmZVXF4tJ65p7Fp?=
 =?us-ascii?Q?rTfmUU8EYOgkXKpL9TgWzL84oQ6PnqWpZ9YpkI4WjuV1ViS7uJ++2z/iQSr6?=
 =?us-ascii?Q?k2NfTKkqPxP0r6VGsHuZ3spAyNnYNs2zIQ5JfMQGgaIGj3n0D2Vt9zbV6Xwz?=
 =?us-ascii?Q?QQro5Ku54j5U6efii7hazods4kUgRS5pVnByc9D3ajWnD/GLwWuFbFu3nzYy?=
 =?us-ascii?Q?gh0BmuTZPLbbpyk8jH7erq4e3JDBDki5AXNkqM7CQjzG6OkIpIMInINSWqKD?=
 =?us-ascii?Q?ywv312iWu7dsAza9JlSb66dEcRP8XIxHX62+QwYjUq1GSpXEnINCClEi3mQ/?=
 =?us-ascii?Q?PcQ5kumyv/jBStXJPcwiv+D4hdlSGv5y1ANp6jtJOVPCJ7YzYTxnGxMwLVfA?=
 =?us-ascii?Q?nJ6EKNSEWZLVeeyf2OlbSdjlpgeJH1HPuha+Jju2My5PDpsrWaTFJanY6FAa?=
 =?us-ascii?Q?iTLzBjYvKHxH7Tc3g/ehgUd3JpayUFOP3XA/HQwvCAaVCALlFJDardxz+GMF?=
 =?us-ascii?Q?pdqFd93dbpw7XTEyhfEuK+2N0a4Cf/eL2huIb+huM1I8OuL7C/wl84M/TswS?=
 =?us-ascii?Q?4oF6nm3FovCbfUFm7z70+MN0ltFjygqMuMUPvs4AQQpghE/Ms/5J+S1Bo5nE?=
 =?us-ascii?Q?pmlLOEN4fFcIrM6kTkXIx4fz5sXgcfxV1qjIExG/eulreakyP/ljgQFBmsSa?=
 =?us-ascii?Q?kjmg7Tz2Gc13vNrMXcl8zDEO8o49UtcAGg/m0fxY0EmMDE6Vs4yOtghXChb0?=
 =?us-ascii?Q?B6gh210FPhjKpYCG9o4qNbV1t7W4K5YLNYe0MghY?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3a058e89-e44a-4cde-9870-08db9a2eb225
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2023 05:49:15.2402
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UFufpkBpKxSVFoqVZ9UL/ATydQrkn+VIYuJeoYRXR3XVFAdLbxS86wfhSACk5WDm8sOFq+lh4Ojcna2C1SfJQA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB5138
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Aug 10, 2023 at 09:58:43PM +0800, Chao Gao wrote:
> On Thu, Aug 10, 2023 at 04:56:36PM +0800, Yan Zhao wrote:
> >This is an RFC series trying to fix the issue of unnecessary NUMA
> >protection and TLB-shootdowns found in VMs with assigned devices or VFIO
> >mediated devices during NUMA balance.
> >
> >For VMs with assigned devices or VFIO mediated devices, all or part of
> >guest memory are pinned for long-term.
> >
> >Auto NUMA balancing will periodically selects VMAs of a process and change
> >protections to PROT_NONE even though some or all pages in the selected
> >ranges are long-term pinned for DMAs, which is true for VMs with assigned
> >devices or VFIO mediated devices.
> >
> >Though this will not cause real problem because NUMA migration will
> >ultimately reject migration of those kind of pages and restore those
> >PROT_NONE PTEs, it causes KVM's secondary MMU to be zapped periodically
> >with equal SPTEs finally faulted back, wasting CPU cycles and generating
> >unnecessary TLB-shootdowns.
> 
> In my understanding, NUMA balancing also moves tasks closer to the memory
> they are accessing. Can this still work with this series applied?
> 
For pages protected with PROT_NONE in primary MMU in scanning phase, yes;
For pages not set to PROT_NONE, no.
Because looks this task_numa_migrate() is only triggered in next page
fault when PROT_NONE and accessible VMA is found.