Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
IronPort-SDR: LyPXteM28wOwsKIA89neZZhgGjl0t6rr8MvIyEhl0PfvRLuagwLBJn77YRuoTKfUdNKrsln6R6
 WehZ1eruJZj8eAexfTY9G273ip8BPBefBb5Rr7kGSfK50YpOYr0zhqJB8v5wY12zff1LjVs5/o
 C18a9A1zstIokWjlv4ahGT27MfW61whPVw/wEkQZwB6+w0nFFAgm0nIP6KQNg+eGqu1Wj1pRDH
 Ms6/Zyjhp2O+Cwr/e191252fG9G/8SJoA7WD0mNIzqDpLOkbil97rGolfSYu5CvjIhIFpO8FGg
 uz+CRL9Ej84OH6IegkHuVo1+
IronPort-SDR: O98aWuCFfvvs7Ud7Da6lFeRs4E+rAtCWFfsuucZd+0ifBRqRyQBOjxVFw/HWXjUtmhYqk5E+cZ
 pOOfkwvjimEYlr+qBYWAw/q5heyj1I3qMc2XPiceEpmFrU9+mEeNkJcCv1lA4fycFhtrXqmhrv
 kHwFqsZ6zScGtKgHx4G3M0AGyDVVWMiisbPx/o1xPliw3JJ9FF6ANBNwiJWGVqGe0Bn7Kryaei
 II5WDLpR/NosrrrjgmZBhnUQ9wX0iBO5KuxGxvDcD5Uz0G/AukEHtrf033i5r1/qwAeF80Hqis
 bY8=
WDCIronportException: Internal
Message-ID: <62b801e8-66b6-0af7-b0c9-195823bf9f62@opensource.wdc.com>
Date:   Mon, 11 Jul 2022 19:40:22 +0900
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH v5 0/5] DMA mapping changes for SCSI core
Content-Language: en-US
To:     John Garry <john.garry@huawei.com>,
        "Martin K. Petersen" <martin.petersen@oracle.com>,
        Christoph Hellwig <hch@lst.de>
Cc:     joro@8bytes.org, will@kernel.org, jejb@linux.ibm.com,
        m.szyprowski@samsung.com, robin.murphy@arm.com,
        linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-ide@vger.kernel.org, iommu@lists.linux-foundation.org,
        iommu@lists.linux.dev, linux-scsi@vger.kernel.org,
        linuxarm@huawei.com
References: <1656590892-42307-1-git-send-email-john.garry@huawei.com>
 <b5f80062-e8ef-9597-1b0c-393140950dfb@huawei.com>
 <20220706134447.GA23753@lst.de> <yq1y1x47jgn.fsf@ca-mkp.ca.oracle.com>
 <5fd4814a-81b1-0e71-58e0-57a747eb684e@huawei.com>
 <6367a264-a3d3-8857-9b5a-2afcd25580cb@opensource.wdc.com>
 <a415e4a1-72ce-53e1-437a-fc7e56e4b913@huawei.com>
From:   Damien Le Moal <damien.lemoal@opensource.wdc.com>
Organization: Western Digital Research
In-Reply-To: <a415e4a1-72ce-53e1-437a-fc7e56e4b913@huawei.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On 7/11/22 16:36, John Garry wrote:
> On 11/07/2022 00:08, Damien Le Moal wrote:
>>> Ah, I think that I misunderstood Damien's question. I thought he was
>>> asking why not keep shost max_sectors at dma_max_mapping_size() and t=
hen
>>> init each sdev request queue max hw sectors at dma_opt_mapping_size()=
.
>> I was suggesting the reverse:)  Keep the device hard limit
>> (max_hw_sectors) to the max dma mapping and set the soft limit
>> (max_sectors) to the optimal dma mapping size.
>=20
> Sure, but as I mentioned below, I only see a small % of requests whose=20
> mapping size exceeds max_sectors but that still causes a big performanc=
e=20
> hit. So that is why I want to set the hard limit as the optimal dma=20
> mapping size.

How can you possibly end-up with requests larger than max_sectors ? BIO
split is done using this limit, right ? Or is it that request merging is
allowed up to max_hw_sectors even if the resulting request size exceeds
max_sectors ?

>=20
> Indeed, the IOMMU IOVA caching limit is already the same as default=20
> max_sectors for the disks in my system - 128Kb for 4k page size.
>=20
>>
>>> But he seems that you want to know why not have the request queue max
>>> sectors at dma_opt_mapping_size(). The answer is related to meaning o=
f
>>> dma_opt_mapping_size(). If we get any mappings which exceed this size
>>> then it can have a big dma mapping performance hit. So I set max hw
>>> sectors at this =E2=80=98opt=E2=80=99 mapping size to ensure that we =
get no mappings
>>> which exceed this size. Indeed, I think max sectors is 128Kb today fo=
r
>>> my host, which would be same as dma_opt_mapping_size() value with an
>>> IOMMU enabled. And I find that only a small % of request size may exc=
eed
>>> this 128kb size, but it still has a big performance impact.
>>>
>=20
> Thanks,
> John


--=20
Damien Le Moal
Western Digital Research