Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <1609845423-110410-1-git-send-email-john.garry@huawei.com>
In-Reply-To: <1609845423-110410-1-git-send-email-john.garry@huawei.com>
From:   Jinpu Wang <jinpu.wang@cloud.ionos.com>
Date:   Mon, 11 Jan 2021 12:57:46 +0100
Message-ID: <CAMGffEmJwH26VJm+Pr8FA5Dk0HxZstcuj+3S_5zK+RM9SahU0w@mail.gmail.com>
Subject: Re: [RFC/RFT PATCH] scsi: pm8001: Expose HW queues for pm80xx hw
To:     John Garry <john.garry@huawei.com>,
        Viswas G <Viswas.G@microchip.com>
Cc:     "James E.J. Bottomley" <jejb@linux.ibm.com>,
        "Martin K. Petersen" <martin.petersen@oracle.com>,
        akshatzen@google.com, Ruksar.devadi@microchip.com,
        Radha Ramachandran <radha@google.com>, bjashnani@google.com,
        vishakhavc@google.com, Ashokkumar.N@microchip.com,
        Linux SCSI Mailinglist <linux-scsi@vger.kernel.org>,
        open list <linux-kernel@vger.kernel.org>,
        Hannes Reinecke <hare@suse.de>,
        Kashyap Desai <kashyap.desai@broadcom.com>, ming.lei@redhat.com
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Hi John,


On Tue, Jan 5, 2021 at 12:21 PM John Garry <john.garry@huawei.com> wrote:
>
> In commit 05c6c029a44d ("scsi: pm80xx: Increase number of supported
> queues"), support for 80xx chip was improved by enabling multiple HW
> queues.
>
> In this, like other SCSI MQ HBA drivers, the HW queues were not exposed
> to upper layer, and instead the driver managed the queues internally.
>
> However, this management duplicates blk-mq code. In addition, the HW queue
> management is sub-optimal for a system where the number of CPUs exceeds
> the HW queues - this is because queues are selected in a round-robin
> fashion, when it would be better to make adjacent CPUs submit on the same
> queue. And finally, the affinity of the completion queue interrupts is not
> set to mirror the cpu<->HQ queue mapping, which is suboptimal.
>
> As such, for when MSIX is supported, expose HW queues to upper layer. Flag
> PCI_IRQ_AFFINITY is set for allocating the MSIX vectors to automatically
> assign affinity for the completion queue interrupts.
>
> Signed-off-by: John Garry <john.garry@huawei.com>
>
> ---
> I sent as an RFC/RFT as I have no HW to test. In addition, since HW queue
> #0 is used always for internal commands (like in send_task_abort()), if
> all CPUs associated with HW queue #0 are offlined, the interrupt for that
> queue will be shutdown, and no CPUs would be available to service any
> internal commands completion. To solve that, we need [0] merged first and
> switch over to use the new API. But we can still test performance in the
> meantime.
>
> I assume someone else is making the change to use the request tag for IO
> tag management.
>
> [0] https://lore.kernel.org/linux-scsi/47ba045e-a490-198b-1744-529f97192d3b@suse.de/
Thanks for the patch, maybe Viswas can help to test?

Regards!