2021-07-30 03:18:55

by Guangbin Huang

[permalink] [raw]
Subject: [PATCH net-next 2/4] io: add function to flush the write combine buffer to device immediately

From: Xiongfeng Wang <[email protected]>

Device registers can be mapped as write-combine type. In this case, data
are not written into the device immediately. They are temporarily stored
in the write combine buffer and written into the device when the buffer
is full. But in some situation, we need to flush the write combine
buffer to device immediately for better performance. So we add a general
function called 'flush_wc_write()'. We use DGH instruction to implement
this function for ARM64.

Signed-off-by: Xiongfeng Wang <[email protected]>
Signed-off-by: Guangbin Huang <[email protected]>
---
arch/arm64/include/asm/io.h | 2 ++
include/linux/io.h | 6 ++++++
2 files changed, 8 insertions(+)

diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
index 7fd836bea7eb..5315d023b2dd 100644
--- a/arch/arm64/include/asm/io.h
+++ b/arch/arm64/include/asm/io.h
@@ -112,6 +112,8 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
#define __iowmb() dma_wmb()
#define __iomb() dma_mb()

+#define flush_wc_write() dgh()
+
/*
* Relaxed I/O memory access primitives. These follow the Device memory
* ordering rules but do not guarantee any ordering relative to Normal memory
diff --git a/include/linux/io.h b/include/linux/io.h
index 9595151d800d..469d53444218 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -166,4 +166,10 @@ static inline void arch_io_free_memtype_wc(resource_size_t base,
}
#endif

+/* IO barriers */
+
+#ifndef flush_wc_write
+#define flush_wc_write() do { } while (0)
+#endif
+
#endif /* _LINUX_IO_H */
--
2.8.1



2021-07-30 09:03:25

by Will Deacon

[permalink] [raw]
Subject: Re: [PATCH net-next 2/4] io: add function to flush the write combine buffer to device immediately

Hi,

On Fri, Jul 30, 2021 at 11:14:22AM +0800, Guangbin Huang wrote:
> From: Xiongfeng Wang <[email protected]>
>
> Device registers can be mapped as write-combine type. In this case, data
> are not written into the device immediately. They are temporarily stored
> in the write combine buffer and written into the device when the buffer
> is full. But in some situation, we need to flush the write combine
> buffer to device immediately for better performance. So we add a general
> function called 'flush_wc_write()'. We use DGH instruction to implement
> this function for ARM64.
>
> Signed-off-by: Xiongfeng Wang <[email protected]>
> Signed-off-by: Guangbin Huang <[email protected]>
> ---
> arch/arm64/include/asm/io.h | 2 ++
> include/linux/io.h | 6 ++++++
> 2 files changed, 8 insertions(+)

-ENODOCUMENTATION

> diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
> index 7fd836bea7eb..5315d023b2dd 100644
> --- a/arch/arm64/include/asm/io.h
> +++ b/arch/arm64/include/asm/io.h
> @@ -112,6 +112,8 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
> #define __iowmb() dma_wmb()
> #define __iomb() dma_mb()
>
> +#define flush_wc_write() dgh()

I think it would be worthwhile to look at what architectures other than
arm64 offer here. For example, is there anything similar to this on riscv,
x86 or power? Doing a quick survery of what's out there might help us define
a macro that can be used across multiple architectures.

Thanks,

Will

> /*
> * Relaxed I/O memory access primitives. These follow the Device memory
> * ordering rules but do not guarantee any ordering relative to Normal memory
> diff --git a/include/linux/io.h b/include/linux/io.h
> index 9595151d800d..469d53444218 100644
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -166,4 +166,10 @@ static inline void arch_io_free_memtype_wc(resource_size_t base,
> }
> #endif
>
> +/* IO barriers */
> +
> +#ifndef flush_wc_write
> +#define flush_wc_write() do { } while (0)
> +#endif
> +
> #endif /* _LINUX_IO_H */
> --
> 2.8.1
>

2021-07-30 09:43:31

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH net-next 2/4] io: add function to flush the write combine buffer to device immediately

On Fri, Jul 30, 2021 at 11:14:22AM +0800, Guangbin Huang wrote:
> From: Xiongfeng Wang <[email protected]>
>
> Device registers can be mapped as write-combine type. In this case, data
> are not written into the device immediately. They are temporarily stored
> in the write combine buffer and written into the device when the buffer
> is full. But in some situation, we need to flush the write combine
> buffer to device immediately for better performance. So we add a general
> function called 'flush_wc_write()'. We use DGH instruction to implement
> this function for ARM64.

Isn't this slightly misleading? IIUC DGH does not guarantee flushing, it
just prevents writes merging (maybe this was already discussed on the
previous RFC).

--
Catalin

2021-10-11 16:29:45

by Guangbin Huang

[permalink] [raw]
Subject: Re: [PATCH net-next 2/4] io: add function to flush the write combine buffer to device immediately



On 2021/7/30 17:00, Will Deacon wrote:
> Hi,
>
> On Fri, Jul 30, 2021 at 11:14:22AM +0800, Guangbin Huang wrote:
>> From: Xiongfeng Wang <[email protected]>
>>
>> Device registers can be mapped as write-combine type. In this case, data
>> are not written into the device immediately. They are temporarily stored
>> in the write combine buffer and written into the device when the buffer
>> is full. But in some situation, we need to flush the write combine
>> buffer to device immediately for better performance. So we add a general
>> function called 'flush_wc_write()'. We use DGH instruction to implement
>> this function for ARM64.
>>
>> Signed-off-by: Xiongfeng Wang <[email protected]>
>> Signed-off-by: Guangbin Huang <[email protected]>
>> ---
>> arch/arm64/include/asm/io.h | 2 ++
>> include/linux/io.h | 6 ++++++
>> 2 files changed, 8 insertions(+)
>
> -ENODOCUMENTATION
>
Hi Will, may I consult you which document file is good to add documentation?

>> diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
>> index 7fd836bea7eb..5315d023b2dd 100644
>> --- a/arch/arm64/include/asm/io.h
>> +++ b/arch/arm64/include/asm/io.h
>> @@ -112,6 +112,8 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
>> #define __iowmb() dma_wmb()
>> #define __iomb() dma_mb()
>>
>> +#define flush_wc_write() dgh()
>
> I think it would be worthwhile to look at what architectures other than
> arm64 offer here. For example, is there anything similar to this on riscv,
> x86 or power? Doing a quick survery of what's out there might help us define
> a macro that can be used across multiple architectures.
>
> Thanks,
>
> Will
>
>> /*
>> * Relaxed I/O memory access primitives. These follow the Device memory
>> * ordering rules but do not guarantee any ordering relative to Normal memory
>> diff --git a/include/linux/io.h b/include/linux/io.h
>> index 9595151d800d..469d53444218 100644
>> --- a/include/linux/io.h
>> +++ b/include/linux/io.h
>> @@ -166,4 +166,10 @@ static inline void arch_io_free_memtype_wc(resource_size_t base,
>> }
>> #endif
>>
>> +/* IO barriers */
>> +
>> +#ifndef flush_wc_write
>> +#define flush_wc_write() do { } while (0)
>> +#endif
>> +
>> #endif /* _LINUX_IO_H */
>> --
>> 2.8.1
>>
> .
>

2021-10-15 10:41:18

by Xiongfeng Wang

[permalink] [raw]
Subject: Re: Re: [PATCH net-next 2/4] io: add function to flush the write combine buffer to device immediately

Hi, Will

On 2021/7/30 17:00, Will Deacon wrote:
> Hi,
>
> On Fri, Jul 30, 2021 at 11:14:22AM +0800, Guangbin Huang wrote:
>> From: Xiongfeng Wang <[email protected]>
>>
>> Device registers can be mapped as write-combine type. In this case, data
>> are not written into the device immediately. They are temporarily stored
>> in the write combine buffer and written into the device when the buffer
>> is full. But in some situation, we need to flush the write combine
>> buffer to device immediately for better performance. So we add a general
>> function called 'flush_wc_write()'. We use DGH instruction to implement
>> this function for ARM64.
>>
>> Signed-off-by: Xiongfeng Wang <[email protected]>
>> Signed-off-by: Guangbin Huang <[email protected]>
>> ---
>> arch/arm64/include/asm/io.h | 2 ++
>> include/linux/io.h | 6 ++++++
>> 2 files changed, 8 insertions(+)
>
> -ENODOCUMENTATION
>
>> diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h
>> index 7fd836bea7eb..5315d023b2dd 100644
>> --- a/arch/arm64/include/asm/io.h
>> +++ b/arch/arm64/include/asm/io.h
>> @@ -112,6 +112,8 @@ static inline u64 __raw_readq(const volatile void __iomem *addr)
>> #define __iowmb() dma_wmb()
>> #define __iomb() dma_mb()
>>
>> +#define flush_wc_write() dgh()
>
> I think it would be worthwhile to look at what architectures other than
> arm64 offer here. For example, is there anything similar to this on riscv,
> x86 or power? Doing a quick survery of what's out there might help us define
> a macro that can be used across multiple architectures.

I searched in 'barrier.h' of different architectures and didn't find similar
merge preventing instructions. Could you give me some advice on naming this
common interface ?

Thanks,
Xiongfeng

>
> Thanks,
>
> Will
>
>> /*
>> * Relaxed I/O memory access primitives. These follow the Device memory
>> * ordering rules but do not guarantee any ordering relative to Normal memory
>> diff --git a/include/linux/io.h b/include/linux/io.h
>> index 9595151d800d..469d53444218 100644
>> --- a/include/linux/io.h
>> +++ b/include/linux/io.h
>> @@ -166,4 +166,10 @@ static inline void arch_io_free_memtype_wc(resource_size_t base,
>> }
>> #endif
>>
>> +/* IO barriers */
>> +
>> +#ifndef flush_wc_write
>> +#define flush_wc_write() do { } while (0)
>> +#endif
>> +
>> #endif /* _LINUX_IO_H */
>> --
>> 2.8.1
>>
> .
>