2017-06-08 08:28:37

by Vladimir Murzin

[permalink] [raw]
Subject: Re: [PATCH v5 0/7] ARM: Fix dma_alloc_coherent() and friends for NOMMU

Ping!

On 24/05/17 11:24, Vladimir Murzin wrote:
> Short story:
>
> Without these patches coherent DMA is broken for AndrĂ¡s and Alexandre,
> so they cannot safely enable DMA on their platforms.
>
> Patches have been circulated on a list since last year without much
> attention to changes in dma-coherent.c and dma-noop.c. Meanwhile, ARM
> bits have been reviewed and there is no strict objection to get them
> merged. Unfortunately, applying only ARM bits doesn't help much and
> the original issue would still exist.
>
> Please, let me know how to move with this fix forward?
>
> Long story:
>
> It seems that addition of cache support for M-class CPUs uncovered
> latent bug in DMA usage. NOMMU memory model has been treated as being
> always consistent; however, for R/M CPU classes memory can be covered
> by MPU which in turn might configure RAM as Normal i.e. bufferable and
> cacheable. It breaks dma_alloc_coherent() and friends, since data can
> stuck in caches now or be buffered.
>
> This patch set is trying to address the issue by providing region of
> memory suitable for consistent DMA operations. It is supposed that
> such region is marked by MPU as non-cacheable. Robin suggested to
> advertise such memory as reserved shared-dma-pool, rather then using
> homebrew command line option, and extend dma-coherent to provide
> default DMA area in the similar way as it is done for CMA (PATCH
> 4/7). It allows us to offload all bookkeeping on generic coherent DMA
> framework, and it seems that it might be reused by other architectures
> like c6x and blackfin.
>
> While reviewing/testing previous versions of the patch set it turned
> out that dma-coherent does not take into account "dma-ranges" device
> tree property, so it is addressed in PATCH 3/7.
>
> For ARM, dedicated DMA region is required for cases other than:
> - MMU/MPU is off
> - cpu is v7m w/o cache support
> - device is coherent
>
> In case any of the above conditions is true dma operations are forced
> to be coherent and wired with dma_noop_ops.
>
> To make life easier NOMMU dma operations are kept in separate
> compilation unit.
>
> Since the issue was reported at the same time as Benjamin sent his
> patch [1] to allow mmap for NOMMU, his case is also addressed in this
> series (PATCH 1/7 and PATCH 2/7).
>
> Thanks!
>
> [1] http://www.armlinux.org.uk/developer/patches/viewpatch.php?id=8633/1
>
> Cc: Joerg Roedel <[email protected]>
> Cc: Christian Borntraeger <[email protected]>
> Cc: Michal Nazarewicz <[email protected]>
> Cc: Marek Szyprowski <[email protected]>
> Cc: Alan Stern <[email protected]>
> Cc: Yoshinori Sato <[email protected]>
> Cc: Rich Felker <[email protected]>
> Cc: Roger Quadros <[email protected]>
> Cc: Greg Kroah-Hartman <[email protected]>
> Cc: Rob Herring <[email protected]>
> Cc: Mark Rutland <[email protected]>
> Cc: Doug Ledford <[email protected]>
>
> Changelog:
> v4 -> v5
> - rebased on v4.12-rc2
> - updated description for CONFIG_ARM_DMA_MEM_BUFFERABLE
>
> v3 -> v4
> - rebased on v4.11-rc7
> - made CONFIG_ARM_DMA_MEM_BUFFERABLE optional for CPU_V7M
> - added Arnd's Acked-by
>
> v2 -> v3
> - fixed warnings reported by Alexandre and kbuild robot
>
> v1 -> v2
> - rebased on v4.11-rc1
> - added Robin's Reviewed-by
> - dedicated flag is introduced to use dev->dma_pfn_offset
> rather than mem->device_base in case memory region is
> configured via device tree (so Tested-by discarded there)
>
> RFC v6 -> v1
> - dropped RFC tag
> - added Alexandre's Tested-by
>
> Vladimir Murzin (7):
> dma: Take into account dma_pfn_offset
> dma: Add simple dma_noop_mmap
> drivers: dma-coherent: Account dma_pfn_offset when used with device
> tree
> drivers: dma-coherent: Introduce default DMA pool
> ARM: NOMMU: Introduce dma operations for noMMU
> ARM: NOMMU: Set ARM_DMA_MEM_BUFFERABLE for M-class cpus
> ARM: dma-mapping: Remove traces of NOMMU code
>
> .../bindings/reserved-memory/reserved-memory.txt | 3 +
> arch/arm/Kconfig | 1 +
> arch/arm/include/asm/dma-mapping.h | 2 +-
> arch/arm/mm/Kconfig | 8 +-
> arch/arm/mm/Makefile | 5 +-
> arch/arm/mm/dma-mapping-nommu.c | 253 +++++++++++++++++++++
> arch/arm/mm/dma-mapping.c | 29 +--
> drivers/base/dma-coherent.c | 74 +++++-
> lib/dma-noop.c | 29 ++-
> 9 files changed, 359 insertions(+), 45 deletions(-)
> create mode 100644 arch/arm/mm/dma-mapping-nommu.c
>


2017-06-08 16:26:34

by Russell King (Oracle)

[permalink] [raw]
Subject: Re: [PATCH v5 0/7] ARM: Fix dma_alloc_coherent() and friends for NOMMU

Well, I've no objection to this, but it does need acks from other
people before I can apply it.

There's two patches that touch drivers/base that need Greg's ack.

I'm not sure what's happening with lib/dma-noop.c, there doesn't
appear to be a maintainer list for it, so I guess that's a
free-for-all.

On Thu, Jun 08, 2017 at 09:28:30AM +0100, Vladimir Murzin wrote:
> Ping!
>
> On 24/05/17 11:24, Vladimir Murzin wrote:
> > Short story:
> >
> > Without these patches coherent DMA is broken for Andr?s and Alexandre,
> > so they cannot safely enable DMA on their platforms.
> >
> > Patches have been circulated on a list since last year without much
> > attention to changes in dma-coherent.c and dma-noop.c. Meanwhile, ARM
> > bits have been reviewed and there is no strict objection to get them
> > merged. Unfortunately, applying only ARM bits doesn't help much and
> > the original issue would still exist.
> >
> > Please, let me know how to move with this fix forward?
> >
> > Long story:
> >
> > It seems that addition of cache support for M-class CPUs uncovered
> > latent bug in DMA usage. NOMMU memory model has been treated as being
> > always consistent; however, for R/M CPU classes memory can be covered
> > by MPU which in turn might configure RAM as Normal i.e. bufferable and
> > cacheable. It breaks dma_alloc_coherent() and friends, since data can
> > stuck in caches now or be buffered.
> >
> > This patch set is trying to address the issue by providing region of
> > memory suitable for consistent DMA operations. It is supposed that
> > such region is marked by MPU as non-cacheable. Robin suggested to
> > advertise such memory as reserved shared-dma-pool, rather then using
> > homebrew command line option, and extend dma-coherent to provide
> > default DMA area in the similar way as it is done for CMA (PATCH
> > 4/7). It allows us to offload all bookkeeping on generic coherent DMA
> > framework, and it seems that it might be reused by other architectures
> > like c6x and blackfin.
> >
> > While reviewing/testing previous versions of the patch set it turned
> > out that dma-coherent does not take into account "dma-ranges" device
> > tree property, so it is addressed in PATCH 3/7.
> >
> > For ARM, dedicated DMA region is required for cases other than:
> > - MMU/MPU is off
> > - cpu is v7m w/o cache support
> > - device is coherent
> >
> > In case any of the above conditions is true dma operations are forced
> > to be coherent and wired with dma_noop_ops.
> >
> > To make life easier NOMMU dma operations are kept in separate
> > compilation unit.
> >
> > Since the issue was reported at the same time as Benjamin sent his
> > patch [1] to allow mmap for NOMMU, his case is also addressed in this
> > series (PATCH 1/7 and PATCH 2/7).
> >
> > Thanks!
> >
> > [1] http://www.armlinux.org.uk/developer/patches/viewpatch.php?id=8633/1
> >
> > Cc: Joerg Roedel <[email protected]>
> > Cc: Christian Borntraeger <[email protected]>
> > Cc: Michal Nazarewicz <[email protected]>
> > Cc: Marek Szyprowski <[email protected]>
> > Cc: Alan Stern <[email protected]>
> > Cc: Yoshinori Sato <[email protected]>
> > Cc: Rich Felker <[email protected]>
> > Cc: Roger Quadros <[email protected]>
> > Cc: Greg Kroah-Hartman <[email protected]>
> > Cc: Rob Herring <[email protected]>
> > Cc: Mark Rutland <[email protected]>
> > Cc: Doug Ledford <[email protected]>
> >
> > Changelog:
> > v4 -> v5
> > - rebased on v4.12-rc2
> > - updated description for CONFIG_ARM_DMA_MEM_BUFFERABLE
> >
> > v3 -> v4
> > - rebased on v4.11-rc7
> > - made CONFIG_ARM_DMA_MEM_BUFFERABLE optional for CPU_V7M
> > - added Arnd's Acked-by
> >
> > v2 -> v3
> > - fixed warnings reported by Alexandre and kbuild robot
> >
> > v1 -> v2
> > - rebased on v4.11-rc1
> > - added Robin's Reviewed-by
> > - dedicated flag is introduced to use dev->dma_pfn_offset
> > rather than mem->device_base in case memory region is
> > configured via device tree (so Tested-by discarded there)
> >
> > RFC v6 -> v1
> > - dropped RFC tag
> > - added Alexandre's Tested-by
> >
> > Vladimir Murzin (7):
> > dma: Take into account dma_pfn_offset
> > dma: Add simple dma_noop_mmap
> > drivers: dma-coherent: Account dma_pfn_offset when used with device
> > tree
> > drivers: dma-coherent: Introduce default DMA pool
> > ARM: NOMMU: Introduce dma operations for noMMU
> > ARM: NOMMU: Set ARM_DMA_MEM_BUFFERABLE for M-class cpus
> > ARM: dma-mapping: Remove traces of NOMMU code
> >
> > .../bindings/reserved-memory/reserved-memory.txt | 3 +
> > arch/arm/Kconfig | 1 +
> > arch/arm/include/asm/dma-mapping.h | 2 +-
> > arch/arm/mm/Kconfig | 8 +-
> > arch/arm/mm/Makefile | 5 +-
> > arch/arm/mm/dma-mapping-nommu.c | 253 +++++++++++++++++++++
> > arch/arm/mm/dma-mapping.c | 29 +--
> > drivers/base/dma-coherent.c | 74 +++++-
> > lib/dma-noop.c | 29 ++-
> > 9 files changed, 359 insertions(+), 45 deletions(-)
> > create mode 100644 arch/arm/mm/dma-mapping-nommu.c
> >
>

--
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

2017-06-15 07:25:20

by Vladimir Murzin

[permalink] [raw]
Subject: Re: [PATCH v5 0/7] ARM: Fix dma_alloc_coherent() and friends for NOMMU

Greg?

On 08/06/17 17:25, Russell King - ARM Linux wrote:
> Well, I've no objection to this, but it does need acks from other
> people before I can apply it.
>
> There's two patches that touch drivers/base that need Greg's ack.
>
> I'm not sure what's happening with lib/dma-noop.c, there doesn't
> appear to be a maintainer list for it, so I guess that's a
> free-for-all.
>
> On Thu, Jun 08, 2017 at 09:28:30AM +0100, Vladimir Murzin wrote:
>> Ping!
>>
>> On 24/05/17 11:24, Vladimir Murzin wrote:
>>> Short story:
>>>
>>> Without these patches coherent DMA is broken for Andr?s and Alexandre,
>>> so they cannot safely enable DMA on their platforms.
>>>
>>> Patches have been circulated on a list since last year without much
>>> attention to changes in dma-coherent.c and dma-noop.c. Meanwhile, ARM
>>> bits have been reviewed and there is no strict objection to get them
>>> merged. Unfortunately, applying only ARM bits doesn't help much and
>>> the original issue would still exist.
>>>
>>> Please, let me know how to move with this fix forward?
>>>
>>> Long story:
>>>
>>> It seems that addition of cache support for M-class CPUs uncovered
>>> latent bug in DMA usage. NOMMU memory model has been treated as being
>>> always consistent; however, for R/M CPU classes memory can be covered
>>> by MPU which in turn might configure RAM as Normal i.e. bufferable and
>>> cacheable. It breaks dma_alloc_coherent() and friends, since data can
>>> stuck in caches now or be buffered.
>>>
>>> This patch set is trying to address the issue by providing region of
>>> memory suitable for consistent DMA operations. It is supposed that
>>> such region is marked by MPU as non-cacheable. Robin suggested to
>>> advertise such memory as reserved shared-dma-pool, rather then using
>>> homebrew command line option, and extend dma-coherent to provide
>>> default DMA area in the similar way as it is done for CMA (PATCH
>>> 4/7). It allows us to offload all bookkeeping on generic coherent DMA
>>> framework, and it seems that it might be reused by other architectures
>>> like c6x and blackfin.
>>>
>>> While reviewing/testing previous versions of the patch set it turned
>>> out that dma-coherent does not take into account "dma-ranges" device
>>> tree property, so it is addressed in PATCH 3/7.
>>>
>>> For ARM, dedicated DMA region is required for cases other than:
>>> - MMU/MPU is off
>>> - cpu is v7m w/o cache support
>>> - device is coherent
>>>
>>> In case any of the above conditions is true dma operations are forced
>>> to be coherent and wired with dma_noop_ops.
>>>
>>> To make life easier NOMMU dma operations are kept in separate
>>> compilation unit.
>>>
>>> Since the issue was reported at the same time as Benjamin sent his
>>> patch [1] to allow mmap for NOMMU, his case is also addressed in this
>>> series (PATCH 1/7 and PATCH 2/7).
>>>
>>> Thanks!
>>>
>>> [1] http://www.armlinux.org.uk/developer/patches/viewpatch.php?id=8633/1
>>>
>>> Cc: Joerg Roedel <[email protected]>
>>> Cc: Christian Borntraeger <[email protected]>
>>> Cc: Michal Nazarewicz <[email protected]>
>>> Cc: Marek Szyprowski <[email protected]>
>>> Cc: Alan Stern <[email protected]>
>>> Cc: Yoshinori Sato <[email protected]>
>>> Cc: Rich Felker <[email protected]>
>>> Cc: Roger Quadros <[email protected]>
>>> Cc: Greg Kroah-Hartman <[email protected]>
>>> Cc: Rob Herring <[email protected]>
>>> Cc: Mark Rutland <[email protected]>
>>> Cc: Doug Ledford <[email protected]>
>>>
>>> Changelog:
>>> v4 -> v5
>>> - rebased on v4.12-rc2
>>> - updated description for CONFIG_ARM_DMA_MEM_BUFFERABLE
>>>
>>> v3 -> v4
>>> - rebased on v4.11-rc7
>>> - made CONFIG_ARM_DMA_MEM_BUFFERABLE optional for CPU_V7M
>>> - added Arnd's Acked-by
>>>
>>> v2 -> v3
>>> - fixed warnings reported by Alexandre and kbuild robot
>>>
>>> v1 -> v2
>>> - rebased on v4.11-rc1
>>> - added Robin's Reviewed-by
>>> - dedicated flag is introduced to use dev->dma_pfn_offset
>>> rather than mem->device_base in case memory region is
>>> configured via device tree (so Tested-by discarded there)
>>>
>>> RFC v6 -> v1
>>> - dropped RFC tag
>>> - added Alexandre's Tested-by
>>>
>>> Vladimir Murzin (7):
>>> dma: Take into account dma_pfn_offset
>>> dma: Add simple dma_noop_mmap
>>> drivers: dma-coherent: Account dma_pfn_offset when used with device
>>> tree
>>> drivers: dma-coherent: Introduce default DMA pool
>>> ARM: NOMMU: Introduce dma operations for noMMU
>>> ARM: NOMMU: Set ARM_DMA_MEM_BUFFERABLE for M-class cpus
>>> ARM: dma-mapping: Remove traces of NOMMU code
>>>
>>> .../bindings/reserved-memory/reserved-memory.txt | 3 +
>>> arch/arm/Kconfig | 1 +
>>> arch/arm/include/asm/dma-mapping.h | 2 +-
>>> arch/arm/mm/Kconfig | 8 +-
>>> arch/arm/mm/Makefile | 5 +-
>>> arch/arm/mm/dma-mapping-nommu.c | 253 +++++++++++++++++++++
>>> arch/arm/mm/dma-mapping.c | 29 +--
>>> drivers/base/dma-coherent.c | 74 +++++-
>>> lib/dma-noop.c | 29 ++-
>>> 9 files changed, 359 insertions(+), 45 deletions(-)
>>> create mode 100644 arch/arm/mm/dma-mapping-nommu.c
>>>
>>
>

2017-06-20 12:59:17

by Vladimir Murzin

[permalink] [raw]
Subject: Re: [PATCH v5 0/7] ARM: Fix dma_alloc_coherent() and friends for NOMMU

+Christoph since he is trying to consolidate work in this area [1].

Christoph, any chance you can look into patches touching
dma-{coherent,noop}.c?

[1] https://patchwork.ozlabs.org/patch/778254/

Thanks!
Vladimir

On 15/06/17 08:25, Vladimir Murzin wrote:
> Greg?
>
> On 08/06/17 17:25, Russell King - ARM Linux wrote:
>> Well, I've no objection to this, but it does need acks from other
>> people before I can apply it.
>>
>> There's two patches that touch drivers/base that need Greg's ack.
>>
>> I'm not sure what's happening with lib/dma-noop.c, there doesn't
>> appear to be a maintainer list for it, so I guess that's a
>> free-for-all.
>>
>> On Thu, Jun 08, 2017 at 09:28:30AM +0100, Vladimir Murzin wrote:
>>> Ping!
>>>
>>> On 24/05/17 11:24, Vladimir Murzin wrote:
>>>> Short story:
>>>>
>>>> Without these patches coherent DMA is broken for Andr?s and Alexandre,
>>>> so they cannot safely enable DMA on their platforms.
>>>>
>>>> Patches have been circulated on a list since last year without much
>>>> attention to changes in dma-coherent.c and dma-noop.c. Meanwhile, ARM
>>>> bits have been reviewed and there is no strict objection to get them
>>>> merged. Unfortunately, applying only ARM bits doesn't help much and
>>>> the original issue would still exist.
>>>>
>>>> Please, let me know how to move with this fix forward?
>>>>
>>>> Long story:
>>>>
>>>> It seems that addition of cache support for M-class CPUs uncovered
>>>> latent bug in DMA usage. NOMMU memory model has been treated as being
>>>> always consistent; however, for R/M CPU classes memory can be covered
>>>> by MPU which in turn might configure RAM as Normal i.e. bufferable and
>>>> cacheable. It breaks dma_alloc_coherent() and friends, since data can
>>>> stuck in caches now or be buffered.
>>>>
>>>> This patch set is trying to address the issue by providing region of
>>>> memory suitable for consistent DMA operations. It is supposed that
>>>> such region is marked by MPU as non-cacheable. Robin suggested to
>>>> advertise such memory as reserved shared-dma-pool, rather then using
>>>> homebrew command line option, and extend dma-coherent to provide
>>>> default DMA area in the similar way as it is done for CMA (PATCH
>>>> 4/7). It allows us to offload all bookkeeping on generic coherent DMA
>>>> framework, and it seems that it might be reused by other architectures
>>>> like c6x and blackfin.
>>>>
>>>> While reviewing/testing previous versions of the patch set it turned
>>>> out that dma-coherent does not take into account "dma-ranges" device
>>>> tree property, so it is addressed in PATCH 3/7.
>>>>
>>>> For ARM, dedicated DMA region is required for cases other than:
>>>> - MMU/MPU is off
>>>> - cpu is v7m w/o cache support
>>>> - device is coherent
>>>>
>>>> In case any of the above conditions is true dma operations are forced
>>>> to be coherent and wired with dma_noop_ops.
>>>>
>>>> To make life easier NOMMU dma operations are kept in separate
>>>> compilation unit.
>>>>
>>>> Since the issue was reported at the same time as Benjamin sent his
>>>> patch [1] to allow mmap for NOMMU, his case is also addressed in this
>>>> series (PATCH 1/7 and PATCH 2/7).
>>>>
>>>> Thanks!
>>>>
>>>> [1] http://www.armlinux.org.uk/developer/patches/viewpatch.php?id=8633/1
>>>>
>>>> Cc: Joerg Roedel <[email protected]>
>>>> Cc: Christian Borntraeger <[email protected]>
>>>> Cc: Michal Nazarewicz <[email protected]>
>>>> Cc: Marek Szyprowski <[email protected]>
>>>> Cc: Alan Stern <[email protected]>
>>>> Cc: Yoshinori Sato <[email protected]>
>>>> Cc: Rich Felker <[email protected]>
>>>> Cc: Roger Quadros <[email protected]>
>>>> Cc: Greg Kroah-Hartman <[email protected]>
>>>> Cc: Rob Herring <[email protected]>
>>>> Cc: Mark Rutland <[email protected]>
>>>> Cc: Doug Ledford <[email protected]>
>>>>
>>>> Changelog:
>>>> v4 -> v5
>>>> - rebased on v4.12-rc2
>>>> - updated description for CONFIG_ARM_DMA_MEM_BUFFERABLE
>>>>
>>>> v3 -> v4
>>>> - rebased on v4.11-rc7
>>>> - made CONFIG_ARM_DMA_MEM_BUFFERABLE optional for CPU_V7M
>>>> - added Arnd's Acked-by
>>>>
>>>> v2 -> v3
>>>> - fixed warnings reported by Alexandre and kbuild robot
>>>>
>>>> v1 -> v2
>>>> - rebased on v4.11-rc1
>>>> - added Robin's Reviewed-by
>>>> - dedicated flag is introduced to use dev->dma_pfn_offset
>>>> rather than mem->device_base in case memory region is
>>>> configured via device tree (so Tested-by discarded there)
>>>>
>>>> RFC v6 -> v1
>>>> - dropped RFC tag
>>>> - added Alexandre's Tested-by
>>>>
>>>> Vladimir Murzin (7):
>>>> dma: Take into account dma_pfn_offset
>>>> dma: Add simple dma_noop_mmap
>>>> drivers: dma-coherent: Account dma_pfn_offset when used with device
>>>> tree
>>>> drivers: dma-coherent: Introduce default DMA pool
>>>> ARM: NOMMU: Introduce dma operations for noMMU
>>>> ARM: NOMMU: Set ARM_DMA_MEM_BUFFERABLE for M-class cpus
>>>> ARM: dma-mapping: Remove traces of NOMMU code
>>>>
>>>> .../bindings/reserved-memory/reserved-memory.txt | 3 +
>>>> arch/arm/Kconfig | 1 +
>>>> arch/arm/include/asm/dma-mapping.h | 2 +-
>>>> arch/arm/mm/Kconfig | 8 +-
>>>> arch/arm/mm/Makefile | 5 +-
>>>> arch/arm/mm/dma-mapping-nommu.c | 253 +++++++++++++++++++++
>>>> arch/arm/mm/dma-mapping.c | 29 +--
>>>> drivers/base/dma-coherent.c | 74 +++++-
>>>> lib/dma-noop.c | 29 ++-
>>>> 9 files changed, 359 insertions(+), 45 deletions(-)
>>>> create mode 100644 arch/arm/mm/dma-mapping-nommu.c
>>>>
>>>
>>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>

2017-06-20 13:00:03

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH v5 0/7] ARM: Fix dma_alloc_coherent() and friends for NOMMU

On Tue, Jun 20, 2017 at 01:59:10PM +0100, Vladimir Murzin wrote:
> +Christoph since he is trying to consolidate work in this area [1].
>
> Christoph, any chance you can look into patches touching
> dma-{coherent,noop}.c?
>
> [1] https://patchwork.ozlabs.org/patch/778254/

Sure, will do.