Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RFC PATCH v3 3/3] blk-mq: Lockout tagset iterator when exiting
 elevator
To:     John Garry <john.garry@huawei.com>, hare@suse.de,
        ming.lei@redhat.com, axboe@kernel.dk, hch@lst.de
Cc:     linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        pragalla@codeaurora.org, kashyap.desai@broadcom.com,
        yuyufen@huawei.com
References: <1614957294-188540-1-git-send-email-john.garry@huawei.com>
 <1614957294-188540-4-git-send-email-john.garry@huawei.com>
From:   Bart Van Assche <bvanassche@acm.org>
Message-ID: <48a3cf78-3f6d-c13c-bca2-1f8277817b45@acm.org>
Date:   Fri, 5 Mar 2021 20:43:03 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <1614957294-188540-4-git-send-email-john.garry@huawei.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 3/5/21 7:14 AM, John Garry wrote:
> diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
> index 7ff1b20d58e7..5950fee490e8 100644
> --- a/block/blk-mq-tag.c
> +++ b/block/blk-mq-tag.c
> @@ -358,11 +358,16 @@ void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset,
>  {
>  	int i;
>  
> +	if (!atomic_inc_not_zero(&tagset->iter_usage_counter))
> +		return;
> +
>  	for (i = 0; i < tagset->nr_hw_queues; i++) {
>  		if (tagset->tags && tagset->tags[i])
>  			__blk_mq_all_tag_iter(tagset->tags[i], fn, priv,
>  					      BT_TAG_ITER_STARTED);
>  	}
> +
> +	atomic_dec(&tagset->iter_usage_counter);
>  }
>  EXPORT_SYMBOL(blk_mq_tagset_busy_iter);

This changes the behavior of blk_mq_tagset_busy_iter(). What will e.g.
happen if the mtip driver calls blk_mq_tagset_busy_iter(&dd->tags,
mtip_abort_cmd, dd) concurrently with another blk_mq_tagset_busy_iter()
call and if that causes all mtip_abort_cmd() calls to be skipped?

> +	while (atomic_cmpxchg(&set->iter_usage_counter, 1, 0) != 1);

Isn't it recommended to call cpu_relax() inside busy-waiting loops?

>  	blk_mq_sched_free_requests(q);
>  	__elevator_exit(q, e);
>  
> +	atomic_set(&set->iter_usage_counter, 1);

Can it happen that the above atomic_set() call happens while a
blk_mq_tagset_busy_iter() call is in progress? Should that atomic_set()
call perhaps be changed into an atomic_inc() call?

Thanks,

Bart.