Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20200121192540.51642-1-muraliraja.muniraju@rubrik.com> <88d16046-f9aa-d5e8-1b1c-7c3ff9516290@kernel.dk>
In-Reply-To: <88d16046-f9aa-d5e8-1b1c-7c3ff9516290@kernel.dk>
From:   Muraliraja Muniraju <muraliraja.muniraju@rubrik.com>
Date:   Thu, 23 Jan 2020 11:25:57 -0800
Message-ID: <CAByjrT9=pZGOwDFnJQ60aG-EznV2QK+DQT3a1NEDJr3RU0K_Gw@mail.gmail.com>
Subject: Re: [PATCH] Adding multiple workers to the loop device.
To:     Jens Axboe <axboe@kernel.dk>
Cc:     linux-block <linux-block@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

I see that the kernel is already is using the multi queues with the
number of hardware queues is 1. But the problem IMO is that the worker
seems to be processing 1 request at a time, to parallelize requests
and have more concurrency more workers needs to be added. I also tried
increasing the nr_hw_queues without increasing the number of workers,
I did not see any difference in performance and it stayed the same. It
allows to queue more requests but it is processed one at a time. I
have not tried with enabling BLK_MQ_F_BLOCKING though. I see that it
can schedule requests early.

On Thu, Jan 23, 2020 at 10:59 AM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 1/21/20 12:25 PM, muraliraja.muniraju wrote:
> > Current loop device implementation has a single kthread worker and
> > drains one request at a time to completion. If the underneath device is
> > slow then this reduces the concurrency significantly. To help in these
> > cases, adding multiple loop workers increases the concurrency. Also to
> > retain the old behaviour the default number of loop workers is 1 and can
> > be tuned via the ioctl.
>
> Have you considered using blk-mq for this? Right now loop just does
> some basic checks and then queues for a thread. If you bump nr_hw_queues
> up (provide a parameter for that) and set BLK_MQ_F_BLOCKING in the
> tag flags, then that might be a more viable approach for handling this.
>
> --
> Jens Axboe
>