2009-06-11 04:15:44

by Kumar Gala

[permalink] [raw]
Subject: Re: [PATCH 1/6] rapidio: add common mapping APIs for RapidIO memory access


On May 12, 2009, at 3:35 AM, Li Yang wrote:

> Add the mapping functions used to support direct IO memory access of
> rapidIO.
>
> Signed-off-by: Zhang Wei <[email protected]>
> Signed-off-by: Li Yang <[email protected]>

Use inbnd/outbnd instead of inb/outb which make one think of byte
level io accessors.

As I look at this I don't think this is the correct API. I think we
should be using the DMA mapping API to hide these details. The
concept of mapping like this seems to be more a function of FSL's
Address translation/mapping unit (ATMU) than anything specific to the
RIO bus standard.

> ---
> drivers/rapidio/rio.c | 95 ++++++++++++++++++++++++++++++++++++++
> +++++++++
> include/linux/rio.h | 25 ++++++++++++
> include/linux/rio_drv.h | 24 +++++++++---
> 3 files changed, 138 insertions(+), 6 deletions(-)

- k


2009-06-11 09:48:26

by Li Yang

[permalink] [raw]
Subject: RE: [PATCH 1/6] rapidio: add common mapping APIs for RapidIO memory access

>On May 12, 2009, at 3:35 AM, Li Yang wrote:
>
>> Add the mapping functions used to support direct IO memory access of
>> rapidIO.
>>
>> Signed-off-by: Zhang Wei <[email protected]>
>> Signed-off-by: Li Yang <[email protected]>
>
>Use inbnd/outbnd instead of inb/outb which make one think of
>byte level io accessors.
>
>As I look at this I don't think this is the correct API. I
>think we should be using the DMA mapping API to hide these
>details. The concept of mapping like this seems to be more a
>function of FSL's Address translation/mapping unit (ATMU) than
>anything specific to the RIO bus standard.

This is a separate RIO block level ATMU. Although it looks like the
system level ATMU, system ATMU doesn't have the knowledge of rapidIO
target device ID. The mapping need to be dynamic, as it's easy to have
more RIO devices than the outbound windows.

- Leo

2009-06-11 13:34:48

by Kumar Gala

[permalink] [raw]
Subject: Re: [PATCH 1/6] rapidio: add common mapping APIs for RapidIO memory access


On Jun 11, 2009, at 4:47 AM, Li Yang-R58472 wrote:

>> On May 12, 2009, at 3:35 AM, Li Yang wrote:
>>
>>> Add the mapping functions used to support direct IO memory access of
>>> rapidIO.
>>>
>>> Signed-off-by: Zhang Wei <[email protected]>
>>> Signed-off-by: Li Yang <[email protected]>
>>
>> Use inbnd/outbnd instead of inb/outb which make one think of
>> byte level io accessors.
>>
>> As I look at this I don't think this is the correct API. I
>> think we should be using the DMA mapping API to hide these
>> details. The concept of mapping like this seems to be more a
>> function of FSL's Address translation/mapping unit (ATMU) than
>> anything specific to the RIO bus standard.
>
> This is a separate RIO block level ATMU. Although it looks like the
> system level ATMU, system ATMU doesn't have the knowledge of rapidIO
> target device ID. The mapping need to be dynamic, as it's easy to
> have
> more RIO devices than the outbound windows.

I understand that. What I'm saying is the RIO block level ATMU is a
Freescale specific detail and not part of any standard RIO bus
programming model. We have mapping APIs that we can connect to for
this via the DMA API layer.

- k

2009-06-12 13:35:34

by Li Yang

[permalink] [raw]
Subject: Re: [PATCH 1/6] rapidio: add common mapping APIs for RapidIO memory access

On Thu, Jun 11, 2009 at 9:32 PM, Kumar Gala<[email protected]> wrote:
>
> On Jun 11, 2009, at 4:47 AM, Li Yang-R58472 wrote:
>
>>> On May 12, 2009, at 3:35 AM, Li Yang wrote:
>>>
>>>> Add the mapping functions used to support direct IO memory access of
>>>> rapidIO.
>>>>
>>>> Signed-off-by: Zhang Wei <[email protected]>
>>>> Signed-off-by: Li Yang <[email protected]>
>>>
>>> Use inbnd/outbnd instead of inb/outb which make one think of
>>> byte level io accessors.
>>>
>>> As I look at this I don't think this is the correct API.  I
>>> think we should be using the DMA mapping API to hide these
>>> details.  The concept of mapping like this seems to be more a
>>> function of FSL's Address translation/mapping unit (ATMU) than
>>> anything specific to the RIO bus standard.
>>
>> This is a separate RIO block level ATMU.  Although it looks like the
>> system level ATMU, system ATMU doesn't have the knowledge of rapidIO
>> target device ID.  The mapping need to be dynamic, as it's easy to have
>> more RIO devices than the outbound windows.
>
> I understand that.  What I'm saying is the RIO block level ATMU is a
> Freescale specific detail and not part of any standard RIO bus programming
> model.  We have mapping APIs that we can connect to for this via the DMA API
> layer.

Ok, I see your point now. Do you mean dma_map_*() for DMA API layer?
But in my understanding the current dma_map_*() APIs are preparing
local memory for device to access which is similar to the inbound
case. Is it suitable to also use them for mapping device's space for
CPU access? Can you give an example of using this API for Address
Translation and Mapping purpose?

- Leo

2009-06-12 14:01:45

by Kumar Gala

[permalink] [raw]
Subject: Re: [PATCH 1/6] rapidio: add common mapping APIs for RapidIO memory access


On Jun 12, 2009, at 8:27 AM, Li Yang wrote:

> On Thu, Jun 11, 2009 at 9:32 PM, Kumar
> Gala<[email protected]> wrote:
>>
>> On Jun 11, 2009, at 4:47 AM, Li Yang-R58472 wrote:
>>
>>>> On May 12, 2009, at 3:35 AM, Li Yang wrote:
>>>>
>>>>> Add the mapping functions used to support direct IO memory
>>>>> access of
>>>>> rapidIO.
>>>>>
>>>>> Signed-off-by: Zhang Wei <[email protected]>
>>>>> Signed-off-by: Li Yang <[email protected]>
>>>>
>>>> Use inbnd/outbnd instead of inb/outb which make one think of
>>>> byte level io accessors.
>>>>
>>>> As I look at this I don't think this is the correct API. I
>>>> think we should be using the DMA mapping API to hide these
>>>> details. The concept of mapping like this seems to be more a
>>>> function of FSL's Address translation/mapping unit (ATMU) than
>>>> anything specific to the RIO bus standard.
>>>
>>> This is a separate RIO block level ATMU. Although it looks like the
>>> system level ATMU, system ATMU doesn't have the knowledge of rapidIO
>>> target device ID. The mapping need to be dynamic, as it's easy to
>>> have
>>> more RIO devices than the outbound windows.
>>
>> I understand that. What I'm saying is the RIO block level ATMU is a
>> Freescale specific detail and not part of any standard RIO bus
>> programming
>> model. We have mapping APIs that we can connect to for this via
>> the DMA API
>> layer.
>
> Ok, I see your point now. Do you mean dma_map_*() for DMA API layer?
> But in my understanding the current dma_map_*() APIs are preparing
> local memory for device to access which is similar to the inbound
> case. Is it suitable to also use them for mapping device's space for
> CPU access? Can you give an example of using this API for Address
> Translation and Mapping purpose?

Yes, I meant the dma_map_*() API. Any system with a true IOMMU uses
the dma_map_ layer as the way to do address translation.

- k

2009-06-15 10:39:10

by Li Yang

[permalink] [raw]
Subject: Re: [PATCH 1/6] rapidio: add common mapping APIs for RapidIO memory access

On Fri, Jun 12, 2009 at 9:58 PM, Kumar Gala<[email protected]> wrote:
>
> On Jun 12, 2009, at 8:27 AM, Li Yang wrote:
>
>> On Thu, Jun 11, 2009 at 9:32 PM, Kumar Gala<[email protected]>
>> wrote:
>>>
>>> On Jun 11, 2009, at 4:47 AM, Li Yang-R58472 wrote:
>>>
>>>>> On May 12, 2009, at 3:35 AM, Li Yang wrote:
>>>>>
>>>>>> Add the mapping functions used to support direct IO memory access of
>>>>>> rapidIO.
>>>>>>
>>>>>> Signed-off-by: Zhang Wei <[email protected]>
>>>>>> Signed-off-by: Li Yang <[email protected]>
>>>>>
>>>>> Use inbnd/outbnd instead of inb/outb which make one think of
>>>>> byte level io accessors.
>>>>>
>>>>> As I look at this I don't think this is the correct API.  I
>>>>> think we should be using the DMA mapping API to hide these
>>>>> details.  The concept of mapping like this seems to be more a
>>>>> function of FSL's Address translation/mapping unit (ATMU) than
>>>>> anything specific to the RIO bus standard.
>>>>
>>>> This is a separate RIO block level ATMU.  Although it looks like the
>>>> system level ATMU, system ATMU doesn't have the knowledge of rapidIO
>>>> target device ID.  The mapping need to be dynamic, as it's easy to have
>>>> more RIO devices than the outbound windows.
>>>
>>> I understand that.  What I'm saying is the RIO block level ATMU is a
>>> Freescale specific detail and not part of any standard RIO bus
>>> programming
>>> model.  We have mapping APIs that we can connect to for this via the DMA
>>> API
>>> layer.
>>
>> Ok, I see your point now. Do you mean dma_map_*() for DMA API layer?
>> But in my understanding the current dma_map_*() APIs are preparing
>> local memory for device to access which is similar to the inbound
>> case.  Is it suitable to also use them for mapping device's space for
>> CPU access?  Can you give an example of using this API for Address
>> Translation and Mapping purpose?
>
> Yes, I meant the dma_map_*() API.  Any system with a true IOMMU uses the
> dma_map_ layer as the way to do address translation.

IOMMU case is not very similar to the RapidIO scenario. RapidIO
mapping is more like PCI address space mapping.

To be specific, the DMA API return dma_addr_t not the rapidIO address
type. And they can only handle inbound mapping not both ways. I
don't think the DMA API is competent enough to be used here for RIO
mapping. Unless we have a more universal mapping API, it can be
justified to create its own API.

- Leo