Date:   Tue, 14 Feb 2023 15:12:11 +0100
From:   Peter Zijlstra <peterz@infradead.org>
To:     "Huang, Kai" <kai.huang@intel.com>
Cc:     "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Hansen, Dave" <dave.hansen@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Luck, Tony" <tony.luck@intel.com>,
        "david@redhat.com" <david@redhat.com>,
        "bagasdotme@gmail.com" <bagasdotme@gmail.com>,
        "ak@linux.intel.com" <ak@linux.intel.com>,
        "Wysocki, Rafael J" <rafael.j.wysocki@intel.com>,
        "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>,
        "Christopherson,, Sean" <seanjc@google.com>,
        "Chatre, Reinette" <reinette.chatre@intel.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "Yamahata, Isaku" <isaku.yamahata@intel.com>,
        "Shahar, Sagi" <sagis@google.com>,
        "imammedo@redhat.com" <imammedo@redhat.com>,
        "Gao, Chao" <chao.gao@intel.com>,
        "Brown, Len" <len.brown@intel.com>,
        "sathyanarayanan.kuppuswamy@linux.intel.com" 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        "Huang, Ying" <ying.huang@intel.com>,
        "Williams, Dan J" <dan.j.williams@intel.com>
Subject: Re: [PATCH v9 07/18] x86/virt/tdx: Do TDX module per-cpu
 initialization
Message-ID: <Y+uWu9hLMZ35JHlY@hirez.programming.kicks-ass.net>
References: <cover.1676286526.git.kai.huang@intel.com>
 <557c526a1190903d11d67c4e2c76e01f67f6eb15.1676286526.git.kai.huang@intel.com>
 <86a8fe2f-566a-d0b9-7a22-9b41c91796f8@intel.com>
 <BL1PR11MB59789A024CFA9E9DE715C2F1F7DD9@BL1PR11MB5978.namprd11.prod.outlook.com>
 <af6034c3-98a3-239b-2c79-ff878f5d9673@intel.com>
 <43fec733ea5331c6de4592dbf44a62e0c61eecb1.camel@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <43fec733ea5331c6de4592dbf44a62e0c61eecb1.camel@intel.com>
Precedence: bulk

On Tue, Feb 14, 2023 at 12:02:22AM +0000, Huang, Kai wrote:
> On Mon, 2023-02-13 at 14:43 -0800, Dave Hansen wrote:
> > On 2/13/23 13:19, Huang, Kai wrote:
> > > > On 2/13/23 03:59, Kai Huang wrote:
> > > > > To avoid duplicated code, add a
> > > > > helper to call SEAMCALL on all online cpus one by one but with a skip
> > > > > function to check whether to skip certain cpus, and use that helper to
> > > > > do the per-cpu initialization.
> > > > ...
> > > > > +/*
> > > > > + * Call @func on all online cpus one by one but skip those cpus
> > > > > + * when @skip_func is valid and returns true for them.
> > > > > + */
> > > > > +static int tdx_on_each_cpu_cond(int (*func)(void *), void *func_data,
> > > > > +                           bool (*skip_func)(int cpu, void *),
> > > > > +                           void *skip_data)
> > > > I only see one caller of this.  Where is the duplicated code?
> > > The other caller is in patch 15 (x86/virt/tdx: Configure global KeyID on all packages).
> > > 
> > > I kinda mentioned this in the changelog:
> > > 
> > >         " Similar to the per-cpu module initialization, a later step to config the key for the global KeyID..."
> > > 
> > > If we don't have this helper, then we can end up with having below loop in two functions:
> > > 
> > >         for_each_online(cpu) {
> > >                 if (should_skip(cpu))
> > >                         continue;
> > > 
> > >                 // call @func on @cpu.
> > >         }
> > 
> > I don't think saving two lines of actual code is worth the opacity that
> > results from this abstraction.
> 
> Alright thanks for the suggestion.  I'll remove this tdx_on_each_cpu_cond() and
> do directly.
> 
> But just checking:
> 
> LP.INIT can actually be called in parallel on different cpus (doesn't have to,
> of course), so we can actually just use on_each_cpu_cond() for LP.INIT:
> 
> 	on_each_cpu_cond(should_skip_cpu, smp_func_module_lp_init, NULL, true);
> 
> But IIUC Peter doesn't like using IPI and prefers using via work:
> 
> https://lore.kernel.org/lkml/Y30dujuXC8wlLwoQ@hirez.programming.kicks-ass.net/
> 
> So I used smp_call_on_cpu() here, which only calls @func on one cpu, but not a
> cpumask.  For LP.INIT ideally we can have something like:
> 
> 	schedule_on_cpu(struct cpumask *cpus, work_func_t func);
> 
> to call @func on a cpu set, but that doesn't exist now, and I don't think it's
> worth to introduce it?

schedule_on_each_cpu() exists and can easily be extended to take a cond
function if you so please.