2020-08-20 05:06:42

by George Cherian

[permalink] [raw]
Subject: [PATCHv2] PCI: Add pci_iounmap

In case if any architecture selects CONFIG_GENERIC_PCI_IOMAP and not
CONFIG_GENERIC_IOMAP, then the pci_iounmap function is reduced to a NULL
function. Due to this the managed release variants or even the explicit
pci_iounmap calls doesn't really remove the mappings.

This issue is seen on an arm64 based system. arm64 by default selects
only CONFIG_GENERIC_PCI_IOMAP and not CONFIG_GENERIC_IOMAP from this
'commit cb61f6769b88 ("ARM64: use GENERIC_PCI_IOMAP")'

Simple bind/unbind test of any pci driver using pcim_iomap/pci_iomap,
would lead to the following error message after long hour tests

"allocation failed: out of vmalloc space - use vmalloc=<size> to
increase size."

Signed-off-by: George Cherian <[email protected]>
---
* Changes from v1
- Fix the 0-day compilation error.
- Mark the lib/iomap pci_iounmap call as weak incase
if any architecture have there own implementation.

include/asm-generic/io.h | 4 ++++
lib/pci_iomap.c | 10 ++++++++++
2 files changed, 14 insertions(+)

diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h
index dabf8cb7203b..5986b37226b7 100644
--- a/include/asm-generic/io.h
+++ b/include/asm-generic/io.h
@@ -915,12 +915,16 @@ static inline void iowrite64_rep(volatile void __iomem *addr,
struct pci_dev;
extern void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long max);

+#ifdef CONFIG_GENERIC_PCI_IOMAP
+extern void pci_iounmap(struct pci_dev *dev, void __iomem *p);
+#else
#ifndef pci_iounmap
#define pci_iounmap pci_iounmap
static inline void pci_iounmap(struct pci_dev *dev, void __iomem *p)
{
}
#endif
+#endif /* CONFIG_GENERIC_PCI_IOMAP */
#endif /* CONFIG_GENERIC_IOMAP */

/*
diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
index 2d3eb1cb73b8..ecd1eb3f6c25 100644
--- a/lib/pci_iomap.c
+++ b/lib/pci_iomap.c
@@ -134,4 +134,14 @@ void __iomem *pci_iomap_wc(struct pci_dev *dev, int bar, unsigned long maxlen)
return pci_iomap_wc_range(dev, bar, 0, maxlen);
}
EXPORT_SYMBOL_GPL(pci_iomap_wc);
+
+#ifndef CONFIG_GENERIC_IOMAP
+#define pci_iounmap pci_iounmap
+void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr);
+void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr)
+{
+ iounmap(addr);
+}
+EXPORT_SYMBOL(pci_iounmap);
+#endif
#endif /* CONFIG_PCI */
--
2.25.1


2020-08-21 00:00:44

by Bjorn Helgaas

[permalink] [raw]
Subject: Re: [PATCHv2] PCI: Add pci_iounmap

[+cc Michael, author of 66eab4df288a ("lib: add GENERIC_PCI_IOMAP")]

On Thu, Aug 20, 2020 at 10:33:06AM +0530, George Cherian wrote:
> In case if any architecture selects CONFIG_GENERIC_PCI_IOMAP and not
> CONFIG_GENERIC_IOMAP, then the pci_iounmap function is reduced to a NULL
> function. Due to this the managed release variants or even the explicit
> pci_iounmap calls doesn't really remove the mappings.
>
> This issue is seen on an arm64 based system. arm64 by default selects
> only CONFIG_GENERIC_PCI_IOMAP and not CONFIG_GENERIC_IOMAP from this
> 'commit cb61f6769b88 ("ARM64: use GENERIC_PCI_IOMAP")'
>
> Simple bind/unbind test of any pci driver using pcim_iomap/pci_iomap,
> would lead to the following error message after long hour tests
>
> "allocation failed: out of vmalloc space - use vmalloc=<size> to
> increase size."
>
> Signed-off-by: George Cherian <[email protected]>
> ---
> * Changes from v1
> - Fix the 0-day compilation error.
> - Mark the lib/iomap pci_iounmap call as weak incase
> if any architecture have there own implementation.
>
> include/asm-generic/io.h | 4 ++++
> lib/pci_iomap.c | 10 ++++++++++
> 2 files changed, 14 insertions(+)
>
> diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h
> index dabf8cb7203b..5986b37226b7 100644
> --- a/include/asm-generic/io.h
> +++ b/include/asm-generic/io.h
> @@ -915,12 +915,16 @@ static inline void iowrite64_rep(volatile void __iomem *addr,
> struct pci_dev;
> extern void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long max);
>
> +#ifdef CONFIG_GENERIC_PCI_IOMAP
> +extern void pci_iounmap(struct pci_dev *dev, void __iomem *p);
> +#else
> #ifndef pci_iounmap
> #define pci_iounmap pci_iounmap
> static inline void pci_iounmap(struct pci_dev *dev, void __iomem *p)
> {
> }
> #endif
> +#endif /* CONFIG_GENERIC_PCI_IOMAP */
> #endif /* CONFIG_GENERIC_IOMAP */
>
> /*
> diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
> index 2d3eb1cb73b8..ecd1eb3f6c25 100644
> --- a/lib/pci_iomap.c
> +++ b/lib/pci_iomap.c
> @@ -134,4 +134,14 @@ void __iomem *pci_iomap_wc(struct pci_dev *dev, int bar, unsigned long maxlen)
> return pci_iomap_wc_range(dev, bar, 0, maxlen);
> }
> EXPORT_SYMBOL_GPL(pci_iomap_wc);
> +
> +#ifndef CONFIG_GENERIC_IOMAP
> +#define pci_iounmap pci_iounmap
> +void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr);
> +void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr)
> +{
> + iounmap(addr);
> +}
> +EXPORT_SYMBOL(pci_iounmap);
> +#endif

I completely agree that this looks like a leak that needs to be fixed.

But my head hurts after trying to understand pci_iomap() and
pci_iounmap(). I hate to add even more #ifdefs here. Can't we
somehow rationalize this and put pci_iounmap() next to pci_iomap()?

66eab4df288a ("lib: add GENERIC_PCI_IOMAP") moved pci_iomap() from
lib/iomap.c to lib/pci_iomap.c, but left pci_iounmap() in lib/iomap.c.
There must be some good reason why they're separated, but I don't know
what it is.

> #endif /* CONFIG_PCI */
> --
> 2.25.1
>

2020-08-21 17:07:50

by George Cherian

[permalink] [raw]
Subject: Re: [PATCHv2] PCI: Add pci_iounmap

Hi Bjorn,

> -----Original Message-----
> From: Bjorn Helgaas <[email protected]>
> Sent: Friday, August 21, 2020 3:26 AM
> To: George Cherian <[email protected]>
> Cc: [email protected]; [email protected]; linux-
> [email protected]; [email protected]; [email protected]; Michael S.
> Tsirkin <[email protected]>
> Subject: [EXT] Re: [PATCHv2] PCI: Add pci_iounmap
>
> [+cc Michael, author of 66eab4df288a ("lib: add GENERIC_PCI_IOMAP")]
>
> On Thu, Aug 20, 2020 at 10:33:06AM +0530, George Cherian wrote:
> > In case if any architecture selects CONFIG_GENERIC_PCI_IOMAP and not
> > CONFIG_GENERIC_IOMAP, then the pci_iounmap function is reduced to a
> > NULL function. Due to this the managed release variants or even the
> > explicit pci_iounmap calls doesn't really remove the mappings.
> >
> > This issue is seen on an arm64 based system. arm64 by default selects
> > only CONFIG_GENERIC_PCI_IOMAP and not CONFIG_GENERIC_IOMAP
> from this
> > 'commit cb61f6769b88 ("ARM64: use GENERIC_PCI_IOMAP")'
> >
> > Simple bind/unbind test of any pci driver using pcim_iomap/pci_iomap,
> > would lead to the following error message after long hour tests
> >
> > "allocation failed: out of vmalloc space - use vmalloc=<size> to
> > increase size."
> >
> > Signed-off-by: George Cherian <[email protected]>
> > ---
> > * Changes from v1
> > - Fix the 0-day compilation error.
> > - Mark the lib/iomap pci_iounmap call as weak incase
> > if any architecture have there own implementation.
> >
> > include/asm-generic/io.h | 4 ++++
> > lib/pci_iomap.c | 10 ++++++++++
> > 2 files changed, 14 insertions(+)
> >
> > diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index
> > dabf8cb7203b..5986b37226b7 100644
> > --- a/include/asm-generic/io.h
> > +++ b/include/asm-generic/io.h
> > @@ -915,12 +915,16 @@ static inline void iowrite64_rep(volatile void
> > __iomem *addr, struct pci_dev; extern void __iomem *pci_iomap(struct
> > pci_dev *dev, int bar, unsigned long max);
> >
> > +#ifdef CONFIG_GENERIC_PCI_IOMAP
> > +extern void pci_iounmap(struct pci_dev *dev, void __iomem *p); #else
> > #ifndef pci_iounmap
> > #define pci_iounmap pci_iounmap
> > static inline void pci_iounmap(struct pci_dev *dev, void __iomem *p)
> > { } #endif
> > +#endif /* CONFIG_GENERIC_PCI_IOMAP */
> > #endif /* CONFIG_GENERIC_IOMAP */
> >
> > /*
> > diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c index
> > 2d3eb1cb73b8..ecd1eb3f6c25 100644
> > --- a/lib/pci_iomap.c
> > +++ b/lib/pci_iomap.c
> > @@ -134,4 +134,14 @@ void __iomem *pci_iomap_wc(struct pci_dev
> *dev, int bar, unsigned long maxlen)
> > return pci_iomap_wc_range(dev, bar, 0, maxlen); }
> > EXPORT_SYMBOL_GPL(pci_iomap_wc);
> > +
> > +#ifndef CONFIG_GENERIC_IOMAP
> > +#define pci_iounmap pci_iounmap
> > +void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr);
> > +void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr) {
> > + iounmap(addr);
> > +}
> > +EXPORT_SYMBOL(pci_iounmap);
> > +#endif
>
> I completely agree that this looks like a leak that needs to be fixed.
>
> But my head hurts after trying to understand pci_iomap() and
> pci_iounmap(). I hate to add even more #ifdefs here. Can't we somehow
> rationalize this and put pci_iounmap() next to pci_iomap()?

Yes, that makes more sense than having #ifdefs here.
I will re-spin and send out another version.
>
> 66eab4df288a ("lib: add GENERIC_PCI_IOMAP") moved pci_iomap() from
> lib/iomap.c to lib/pci_iomap.c, but left pci_iounmap() in lib/iomap.c.
> There must be some good reason why they're separated, but I don't know
> what it is.
>
> > #endif /* CONFIG_PCI */
> > --
> > 2.25.1
> >

2020-08-23 08:08:47

by Michael S. Tsirkin

[permalink] [raw]
Subject: Re: [PATCHv2] PCI: Add pci_iounmap

On Thu, Aug 20, 2020 at 04:55:49PM -0500, Bjorn Helgaas wrote:
> [+cc Michael, author of 66eab4df288a ("lib: add GENERIC_PCI_IOMAP")]
>
> On Thu, Aug 20, 2020 at 10:33:06AM +0530, George Cherian wrote:
> > In case if any architecture selects CONFIG_GENERIC_PCI_IOMAP and not
> > CONFIG_GENERIC_IOMAP, then the pci_iounmap function is reduced to a NULL
> > function. Due to this the managed release variants or even the explicit
> > pci_iounmap calls doesn't really remove the mappings.
> >
> > This issue is seen on an arm64 based system. arm64 by default selects
> > only CONFIG_GENERIC_PCI_IOMAP and not CONFIG_GENERIC_IOMAP from this
> > 'commit cb61f6769b88 ("ARM64: use GENERIC_PCI_IOMAP")'
> >
> > Simple bind/unbind test of any pci driver using pcim_iomap/pci_iomap,
> > would lead to the following error message after long hour tests
> >
> > "allocation failed: out of vmalloc space - use vmalloc=<size> to
> > increase size."
> >
> > Signed-off-by: George Cherian <[email protected]>
> > ---
> > * Changes from v1
> > - Fix the 0-day compilation error.
> > - Mark the lib/iomap pci_iounmap call as weak incase
> > if any architecture have there own implementation.
> >
> > include/asm-generic/io.h | 4 ++++
> > lib/pci_iomap.c | 10 ++++++++++
> > 2 files changed, 14 insertions(+)
> >
> > diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h
> > index dabf8cb7203b..5986b37226b7 100644
> > --- a/include/asm-generic/io.h
> > +++ b/include/asm-generic/io.h
> > @@ -915,12 +915,16 @@ static inline void iowrite64_rep(volatile void __iomem *addr,
> > struct pci_dev;
> > extern void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long max);
> >
> > +#ifdef CONFIG_GENERIC_PCI_IOMAP
> > +extern void pci_iounmap(struct pci_dev *dev, void __iomem *p);
> > +#else
> > #ifndef pci_iounmap
> > #define pci_iounmap pci_iounmap
> > static inline void pci_iounmap(struct pci_dev *dev, void __iomem *p)
> > {
> > }
> > #endif
> > +#endif /* CONFIG_GENERIC_PCI_IOMAP */
> > #endif /* CONFIG_GENERIC_IOMAP */
> >
> > /*
> > diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
> > index 2d3eb1cb73b8..ecd1eb3f6c25 100644
> > --- a/lib/pci_iomap.c
> > +++ b/lib/pci_iomap.c
> > @@ -134,4 +134,14 @@ void __iomem *pci_iomap_wc(struct pci_dev *dev, int bar, unsigned long maxlen)
> > return pci_iomap_wc_range(dev, bar, 0, maxlen);
> > }
> > EXPORT_SYMBOL_GPL(pci_iomap_wc);
> > +
> > +#ifndef CONFIG_GENERIC_IOMAP
> > +#define pci_iounmap pci_iounmap
> > +void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr);
> > +void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr)
> > +{
> > + iounmap(addr);
> > +}
> > +EXPORT_SYMBOL(pci_iounmap);
> > +#endif
>
> I completely agree that this looks like a leak that needs to be fixed.
>
> But my head hurts after trying to understand pci_iomap() and
> pci_iounmap(). I hate to add even more #ifdefs here. Can't we
> somehow rationalize this and put pci_iounmap() next to pci_iomap()?
>
> 66eab4df288a ("lib: add GENERIC_PCI_IOMAP") moved pci_iomap() from
> lib/iomap.c to lib/pci_iomap.c, but left pci_iounmap() in lib/iomap.c.
> There must be some good reason why they're separated, but I don't know
> what it is.

My recollection is vaguely that map code was more or less
the same across architectures, but unmap was often different,
so I only moved map into the library.

> > #endif /* CONFIG_PCI */
> > --
> > 2.25.1
> >