Date: Tue, 7 Jul 2015 19:10:16 +0200
From: Frederic Weisbecker <fweisbec@gmail.com>
To: Oleg Nesterov <oleg@redhat.com>
Cc: LKML <linux-kernel@vger.kernel.org>, Christoph Lameter <cl@linux.com>,
        Rik van Riel <riel@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH 3/3] kmod: Remove unecessary explicit wide CPU affinity
 setting
Message-ID: <20150707171014.GH20498@lerouge>
References: <1436196821-13962-1-git-send-email-fweisbec@gmail.com>
 <1436196821-13962-4-git-send-email-fweisbec@gmail.com>
 <20150707163030.GA16647@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20150707163030.GA16647@redhat.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2876
Lines: 69

On Tue, Jul 07, 2015 at 06:30:30PM +0200, Oleg Nesterov wrote:
> On 07/06, Frederic Weisbecker wrote:
> >
> > The call_usermodehelper_exec_[a]sync() kernel threads are created by
> > khelper precisely because
> 
> I think khelper should simply die. It doesn't make any sense today,
> kmod.c should use some system wq instead. But see below.
> 
> > Not only useless it even breaks nohz full. The housekeeping work
> > (general kernel internal code that user doesn't care much about) is
> > handled by a reduced set of CPUs in nohz full, precisely those that are
> > not included by nohz_full= kernel parameters. For example unbound
> > workqueues are handled by housekeeping CPUs.
> 
> Confused... I do not see how workqueue_attrs->cpumask can depend on
> tick_nohz_full_mask or housekeeping_mask. Could you explain?

People who want CPU isolation will likely write
/sys/devices/virtual/workqueue/cpumask to a reduced set of CPUs, typically
CPU 0 that is used for housekeeping in nohz full.

In fact we should add the code which initialize wq_unbound_cpumask
to housekeeping_mask automatically.

So this cpumask is inherited to khelper because it is a singlethread
workqueue.

> 
> > @@ -223,9 +223,6 @@ static int call_usermodehelper_exec_async(void *data)
> >  	flush_signal_handlers(current, 1);
> >  	spin_unlock_irq(&current->sighand->siglock);
> >
> > -	/* We can run anywhere, unlike our parent keventd(). */
> > -	set_cpus_allowed_ptr(current, cpu_all_mask);
> > -
> 
> I think this is fine, ->no_numa is true for khelper.
> 
> But this means that after this change kmod.c can't use a system wq,
> ->no_numa is false by default. And khelper is no_numa only because
> it is __WQ_ORDERED, but kmod.c doesn't need really need__WQ_ORDERED,
> except, again, this implies ->no_numa == T.

I'm not sure what means no_numa in the context of workqueues, I guess
it's about having system workqueues bound to one CPU or several in the
same nodes. But indeed we can't use system workqueues because they are
per-cpu and we inherit that. And it's ridiculous to call set_cpus_allowed_ptr()
to fix that.

Hence why we use a singlethread, even though we don't care about ordering.
Ok I guess that's more or less what you just said :o)

> So perhaps init_workqueues() should create another global
> WQ_UNBOUND/no_numa workqueue_struct so that we could kill khelper_wq?
> Or kmod.c can use system_unbound_wq, but then we need to keep this
> set_cpus_allowed_ptr().
> 
> To me khelper_wq looks just annoying.

That's a good idea. I can do that!

Perhaps queuing there would be done through schedule_work_unbound() ?
Or schedule_work_no_numa()?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/