Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date:   Mon, 16 Sep 2019 15:16:21 +0530
From:   Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
To:     Robin Murphy <robin.murphy@arm.com>
Cc:     Vivek Gautam <vivek.gautam@codeaurora.org>, joro@8bytes.org,
        agross@kernel.org, will.deacon@arm.com,
        iommu@lists.linux-foundation.org, bjorn.andersson@linaro.org,
        linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-msm-owner@vger.kernel.org
Subject: Re: [PATCH v4 3/3] iommu: arm-smmu-impl: Add sdm845 implementation
 hook
In-Reply-To: <9fb7d18c-e292-cbc9-aa6d-d85465ea249e@arm.com>
References: <20190823063248.13295-1-vivek.gautam@codeaurora.org>
 <20190823063248.13295-4-vivek.gautam@codeaurora.org>
 <9fb7d18c-e292-cbc9-aa6d-d85465ea249e@arm.com>
Message-ID: <f234e891bc16a1869ba8a929e52a49f7@codeaurora.org>
User-Agent: Roundcube Webmail/1.2.5
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Robin,

On 2019-09-10 18:56, Robin Murphy wrote:
> On 23/08/2019 07:32, Vivek Gautam wrote:
>> Add reset hook for sdm845 based platforms to turn off
>> the wait-for-safe sequence.
>> 
>> Understanding how wait-for-safe logic affects USB and UFS performance
>> on MTP845 and DB845 boards:
>> 
>> Qcom's implementation of arm,mmu-500 adds a WAIT-FOR-SAFE logic
>> to address under-performance issues in real-time clients, such as
>> Display, and Camera.
>> On receiving an invalidation requests, the SMMU forwards SAFE request
>> to these clients and waits for SAFE ack signal from real-time clients.
>> The SAFE signal from such clients is used to qualify the start of
>> invalidation.
>> This logic is controlled by chicken bits, one for each - MDP 
>> (display),
>> IFE0, and IFE1 (camera), that can be accessed only from secure 
>> software
>> on sdm845.
>> 
>> This configuration, however, degrades the performance of non-real time
>> clients, such as USB, and UFS etc. This happens because, with 
>> wait-for-safe
>> logic enabled the hardware tries to throttle non-real time clients 
>> while
>> waiting for SAFE ack signals from real-time clients.
>> 
>> On mtp845 and db845 devices, with wait-for-safe logic enabled by the
>> bootloaders we see degraded performance of USB and UFS when kernel
>> enables the smmu stage-1 translations for these clients.
>> Turn off this wait-for-safe logic from the kernel gets us back the 
>> perf
>> of USB and UFS devices until we re-visit this when we start seeing 
>> perf
>> issues on display/camera on upstream supported SDM845 platforms.
>> The bootloaders on these boards implement secure monitor callbacks to
>> handle a specific command - QCOM_SCM_SVC_SMMU_PROGRAM with which the
>> logic can be toggled.
>> 
>> There are other boards such as cheza whose bootloaders don't enable 
>> this
>> logic. Such boards don't implement callbacks to handle the specific 
>> SCM
>> call so disabling this logic for such boards will be a no-op.
>> 
>> This change is inspired by the downstream change from Patrick Daly
>> to address performance issues with display and camera by handling
>> this wait-for-safe within separte io-pagetable ops to do TLB
>> maintenance. So a big thanks to him for the change and for all the
>> offline discussions.
>> 
>> Without this change the UFS reads are pretty slow:
>> $ time dd if=/dev/sda of=/dev/zero bs=1048576 count=10 conv=sync
>> 10+0 records in
>> 10+0 records out
>> 10485760 bytes (10.0MB) copied, 22.394903 seconds, 457.2KB/s
>> real    0m 22.39s
>> user    0m 0.00s
>> sys     0m 0.01s
>> 
>> With this change they are back to rock!
>> $ time dd if=/dev/sda of=/dev/zero bs=1048576 count=300 conv=sync
>> 300+0 records in
>> 300+0 records out
>> 314572800 bytes (300.0MB) copied, 1.030541 seconds, 291.1MB/s
>> real    0m 1.03s
>> user    0m 0.00s
>> sys     0m 0.54s
>> 
>> Signed-off-by: Vivek Gautam <vivek.gautam@codeaurora.org>
>> ---
>>   drivers/iommu/arm-smmu-impl.c | 27 ++++++++++++++++++++++++++-
> 
> I'd be inclined to introduce the inevitable arm-smmu-qcom.c from the
> start, and save worrying about moving this out later. Other than that,
> though, the general self-contained shape of it all is every bit as
> beautiful as I'd hoped :D
> 

Have posted v5 with your suggestion.

Thanks,
Sai

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member
of Code Aurora Forum, hosted by The Linux Foundation