Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: dma_mask limited to 32-bits with OF platform device
From:   Roger Quadros <rogerq@ti.com>
To:     Robin Murphy <robin.murphy@arm.com>,
        Rob Herring <robh+dt@kernel.org>,
        Christoph Hellwig <hch@lst.de>
CC:     =?UTF-8?Q?P=c3=a9ter_Ujfalusi?= <peter.ujfalusi@ti.com>,
        Murali Karicheri <m-karicheri2@ti.com>,
        "Nori, Sekhar" <nsekhar@ti.com>, "Anna, Suman" <s-anna@ti.com>,
        Stefan Wahren <stefan.wahren@i2se.com>,
        =?UTF-8?Q?Andreas_F=c3=a4rber?= <afaerber@suse.de>,
        Hans Verkuil <hverkuil@xs4all.nl>,
        <devicetree@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Nishanth Menon <nm@ti.com>,
        "hdegoede@redhat.com" <hdegoede@redhat.com>,
        Vignesh Raghavendra <vigneshr@ti.com>
References: <c1c75923-3094-d3fc-fe8e-ee44f17b1a0a@ti.com>
 <3a91f306-f544-a63c-dfe2-7eae7b32bcca@arm.com>
 <56314192-f3c6-70c5-6b9a-3d580311c326@ti.com>
 <9bd83815-6f54-2efb-9398-42064f73ab1c@arm.com>
 <20200217132133.GA27134@lst.de> <b3c56884-128e-a7e1-2e09-0e8de3c3512d@ti.com>
 <CAL_JsqLxECRKWG3SoORADtZ-gVbqCHyx9mhGzrCPO+X=--w8AQ@mail.gmail.com>
 <15d0ac5f-4919-5852-cd95-93c24d8bdbb9@ti.com>
 <827fa19d-1990-16bc-33f5-fc82ac0d4a8a@arm.com>
 <3d8ea578-2ecb-1126-3bf0-2dc695687245@ti.com>
Message-ID: <e898cf0a-6f52-8550-c73e-b78bc413bcc7@ti.com>
Date:   Tue, 3 Mar 2020 10:27:04 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.4.1
MIME-Version: 1.0
In-Reply-To: <3d8ea578-2ecb-1126-3bf0-2dc695687245@ti.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Robin, Christoph,

On 26/02/2020 13:33, Roger Quadros wrote:
> Hi,
> 
> On 19/02/2020 17:25, Robin Murphy wrote:
>> On 19/02/2020 2:29 pm, Roger Quadros wrote:
>>> Rob,
>>>
>>> On 18/02/2020 19:22, Rob Herring wrote:
>>>> On Tue, Feb 18, 2020 at 2:28 AM Roger Quadros <rogerq@ti.com> wrote:
>>>>>
>>>>> Chrishtoph,
>>>>>
>>>>> The branch works fine for SATA on DRA7 with CONFIG_LPAE once I
>>>>> have the below DT fix.
>>>>>
>>>>> Do you intend to send these fixes to -stable?
>>>>>
>>>>> ------------------------- arch/arm/boot/dts/dra7.dtsi -------------------------
>>>>> index d78b684e7fca..853ecf3cfb37 100644
>>>>> @@ -645,6 +645,8 @@
>>>>>                  sata: sata@4a141100 {
>>>>>                          compatible = "snps,dwc-ahci";
>>>>>                          reg = <0x4a140000 0x1100>, <0x4a141100 0x7>;
>>>>> +                       #size-cells = <2>;
>>>>> +                       dma-ranges = <0x00000000 0x00000000 0x1 0x00000000>;
>>>>
>>>> dma-ranges should be in the parent (bus) node, not the device node.
>>>
>>> I didn't understand why.
>>>
>>> There are many devices on the parent bus node and all devices might not have the 32-bit DMA limit
>>> the SATA controller has.
>>>
>>> SATA controller is the bus master and the ATA devices are children of the SATA controller.
>>
>> But SATA is not a memory-mapped bus - in the context of MMIO, the AHCI is the bus-master device, not a bridge or level of interconnect. The DeviceTree spec[1] clearly defines dma-ranges as an address translation between a "parent bus" and a "child bus".
>>
>> If in the worst case this address-limited interconnect really only exists between the AHCI's master interface and everything else in the system, then you'll have to describe it explicitly to meet DT's expectation of a "bus" (e.g. [2]). Yes, it's a bit clunky, but any scheme has its edge cases.
>>
>>>  From Documentation/devicetree/booting-without-of.txt
>>>
>>> * DMA Bus master
>>> Optional property:
>>> - dma-ranges: <prop-encoded-array> encoded as arbitrary number of triplets of
>>>          (child-bus-address, parent-bus-address, length). Each triplet specified
>>>          describes a contiguous DMA address range.
>>>          The dma-ranges property is used to describe the direct memory access (DMA)
>>>          structure of a memory-mapped bus whose device tree parent can be accessed
>>>          from DMA operations originating from the bus. It provides a means of
>>>          defining a mapping or translation between the physical address space of
>>>          the bus and the physical address space of the parent of the bus.
>>>          (for more information see the Devicetree Specification)
>>>
>>> * DMA Bus child
>>> Optional property:
>>> - dma-ranges: <empty> value. if present - It means that DMA addresses
>>>          translation has to be enabled for this device.
>>
>> Disregarding that this was apparently never in ePAPR, so not grandfathered in to DTSpec, and effectively nobody ever has actually followed it (oh, if only...), note "<empty>" - that still doesn't imply that a *non-empty* dma-ranges would be valid on device nodes.
>>
>> Robin.
>>
>> [1] https://www.devicetree.org/specifications/
>> [2] https://lore.kernel.org/lkml/20181010120737.30300-20-laurentiu.tudor@nxp.com/
> 
> With the patch (in the end). dev->bus_dma_limit is still set to 0 and so is not being used.
> 
> from of_dma_configure()
>          ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
> ...
>          /* ...but only set bus limit if we found valid dma-ranges earlier */
>          if (!ret)
>                  dev->bus_dma_limit = end;
> 
> There is no other place bus_dma_limit is set. Looks like every device should inherit that
> from it's parent right?

Any ideas how to expect this to work?

> 
> diff --git a/arch/arm/boot/dts/dra7.dtsi b/arch/arm/boot/dts/dra7.dtsi
> index 64a0f90f5b52..5418c31d4da7 100644
> --- a/arch/arm/boot/dts/dra7.dtsi
> +++ b/arch/arm/boot/dts/dra7.dtsi
> @@ -680,15 +680,22 @@
>           };
> 
>           /* OCP2SCP3 */
> -        sata: sata@4a141100 {
> -            compatible = "snps,dwc-ahci";
> -            reg = <0x4a140000 0x1100>, <0x4a141100 0x7>;
> -            interrupts = <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>;
> -            phys = <&sata_phy>;
> -            phy-names = "sata-phy";
> -            clocks = <&l3init_clkctrl DRA7_L3INIT_SATA_CLKCTRL 8>;
> -            ti,hwmods = "sata";
> -            ports-implemented = <0x1>;
> +        sata_aux_bus {
> +            #address-cells = <2>;
> +            #size-cells = <2>;
> +            compatible = "simple-bus";
> +            ranges = <0x0 0x0 0x4a140000 0x0 0x1200>;
> +            dma-ranges = <0x0 0x0 0x0 0x0 0x1 0x00000000>;
> +            sata: sata@4a141100 {
> +                compatible = "snps,dwc-ahci";
> +                reg = <0x0 0x0 0x0 0x1100>, <0x0 0x1100 0x0 0x7>;
> +                interrupts = <GIC_SPI 49 IRQ_TYPE_LEVEL_HIGH>;
> +                phys = <&sata_phy>;
> +                phy-names = "sata-phy";
> +                clocks = <&l3init_clkctrl DRA7_L3INIT_SATA_CLKCTRL 8>;
> +                ti,hwmods = "sata";
> +                ports-implemented = <0x1>;
> +            };
>           };
> 
>           /* OCP2SCP1 */
> 

-- 
cheers,
-roger
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki.
Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki