Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 22 Jul 2019 21:34:14 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Nadav Amit <namit@vmware.com>
Cc:     Dave Hansen <dave.hansen@intel.com>,
        Andy Lutomirski <luto@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        the arch/x86 maintainers <x86@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        Rik van Riel <riel@surriel.com>,
        Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH v3 1/9] smp: Run functions concurrently in
 smp_call_function_many()
Message-ID: <20190722193414.GG6698@worktop.programming.kicks-ass.net>
References: <20190719005837.4150-1-namit@vmware.com>
 <20190719005837.4150-2-namit@vmware.com>
 <c02833c8-8c68-4331-03c7-d9e5eb2f285c@intel.com>
 <20190722181658.GA6698@worktop.programming.kicks-ass.net>
 <CF32049E-D6AA-4AD5-A276-0CEC84B6DB11@vmware.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CF32049E-D6AA-4AD5-A276-0CEC84B6DB11@vmware.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Jul 22, 2019 at 06:41:44PM +0000, Nadav Amit wrote:
> > On Jul 22, 2019, at 11:16 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> > 
> > On Fri, Jul 19, 2019 at 11:23:06AM -0700, Dave Hansen wrote:
> >> On 7/18/19 5:58 PM, Nadav Amit wrote:
> >>> @@ -624,16 +622,11 @@ EXPORT_SYMBOL(on_each_cpu);
> >>> void on_each_cpu_mask(const struct cpumask *mask, smp_call_func_t func,
> >>> 			void *info, bool wait)
> >>> {
> >>> -	int cpu = get_cpu();
> >>> +	preempt_disable();
> >>> 
> >>> -	smp_call_function_many(mask, func, info, wait);
> >>> -	if (cpumask_test_cpu(cpu, mask)) {
> >>> -		unsigned long flags;
> >>> -		local_irq_save(flags);
> >>> -		func(info);
> >>> -		local_irq_restore(flags);
> >>> -	}
> >>> -	put_cpu();
> >>> +	__smp_call_function_many(mask, func, func, info, wait);
> >>> +
> >>> +	preempt_enable();
> >>> }
> >> 
> >> The get_cpu() was missing it too, but it would be nice to add some
> >> comments about why preempt needs to be off.  I was also thinking it
> >> might make sense to do:
> >> 
> >> 	cfd = get_cpu_var(cfd_data);
> >> 	__smp_call_function_many(cfd, ...);
> >> 	put_cpu_var(cfd_data);
> >> 	
> >> instead of the explicit preempt_enable/disable(), but I don't feel too
> >> strongly about it.
> > 
> > It is also required for cpu hotplug.
> 
> But then smpcfd_dead_cpu() will not respect the “cpu” argument. Do you still
> prefer it this way (instead of the current preempt_enable() /
> preempt_disable())?

I just meant that the preempt_disable() (either form) is required for
hotplug (we must not send IPIs to offline CPUs, that gets things upset).

Personally I don't mind the bare preempt_disable() as you have; but I
think Dave's idea of a comment has merrit.