Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 6 Apr 2021 14:59:57 -0400
From:   Dan Schatzberg <schatzberg.dan@gmail.com>
To:     Hillf Danton <hdanton@sina.com>
Cc:     Jens Axboe <axboe@kernel.dk>, linux-block@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH 1/3] loop: Use worker per cgroup instead of kworker
Message-ID: <YGyvra69F/DIa7KI@dschatzberg-fedora-PC0Y6AEN.dhcp.thefacebook.com>
References: <20210402191638.3249835-1-schatzberg.dan@gmail.com>
 <20210403020902.1384-1-hdanton@sina.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210403020902.1384-1-hdanton@sina.com>
Precedence: bulk

Hi Hillf, thanks for the review

On Sat, Apr 03, 2021 at 10:09:02AM +0800, Hillf Danton wrote:
> On Fri,  2 Apr 2021 12:16:32 Dan Schatzberg wrote:
> > +queue_work:
> > +	if (worker) {
> > +		/*
> > +		 * We need to remove from the idle list here while
> > +		 * holding the lock so that the idle timer doesn't
> > +		 * free the worker
> > +		 */
> > +		if (!list_empty(&worker->idle_list))
> > +			list_del_init(&worker->idle_list);
> 
> Nit, only queue work if the worker is inactive - otherwise it is taking
> care of the cmd_list.

By worker is inactive, you mean worker is on the idle_list? Yes, I
think you're right that queue_work() is unnecessary in that case since
each worker checks empty cmd_list then adds itself to idle_list under
the lock.

> 
> > +		work = &worker->work;
> > +		cmd_list = &worker->cmd_list;
> > +	} else {
> > +		work = &lo->rootcg_work;
> > +		cmd_list = &lo->rootcg_cmd_list;
> > +	}
> > +	list_add_tail(&cmd->list_entry, cmd_list);
> > +	queue_work(lo->workqueue, work);
> > +	spin_unlock_irq(&lo->lo_work_lock);
> >  }
> [...]
> > +	/*
> > +	 * We only add to the idle list if there are no pending cmds
> > +	 * *and* the worker will not run again which ensures that it
> > +	 * is safe to free any worker on the idle list
> > +	 */
> > +	if (worker && !work_pending(&worker->work)) {
> 
> The empty cmd_list is a good enough reason for worker to become idle.

This is only true with the above change to avoid a gratuitous
queue_work(), right? Otherwise we run the risk of freeing a worker
concurrently with loop_process_work() being invoked.