Currently, neither the vio_bus or vio_driver structures provide support
for a shutdown() routine.
Add support for shutdown() by allowing drivers to provide a
implementation via function pointer in their vio_driver struct and
provide a proper implementation in the driver template for the vio_bus
that calls a vio drivers shutdown() if defined.
In the case that no shutdown() is defined by a vio driver and a kexec is
in progress we implement a big hammer that calls remove() to ensure no
further DMA for the devices is possible.
Signed-off-by: Tyrel Datwyler <[email protected]>
---
arch/powerpc/include/asm/vio.h | 1 +
arch/powerpc/platforms/pseries/vio.c | 16 ++++++++++++++++
2 files changed, 17 insertions(+)
diff --git a/arch/powerpc/include/asm/vio.h b/arch/powerpc/include/asm/vio.h
index 721c0d6715ac..e7479a4abf96 100644
--- a/arch/powerpc/include/asm/vio.h
+++ b/arch/powerpc/include/asm/vio.h
@@ -114,6 +114,7 @@ struct vio_driver {
const struct vio_device_id *id_table;
int (*probe)(struct vio_dev *dev, const struct vio_device_id *id);
void (*remove)(struct vio_dev *dev);
+ void (*shutdown)(struct vio_dev *dev);
/* A driver must have a get_desired_dma() function to
* be loaded in a CMO environment if it uses DMA.
*/
diff --git a/arch/powerpc/platforms/pseries/vio.c b/arch/powerpc/platforms/pseries/vio.c
index 9cb4fc839fd5..d122b8644319 100644
--- a/arch/powerpc/platforms/pseries/vio.c
+++ b/arch/powerpc/platforms/pseries/vio.c
@@ -22,6 +22,7 @@
#include <linux/mm.h>
#include <linux/dma-map-ops.h>
#include <linux/kobject.h>
+#include <linux/kexec.h>
#include <asm/iommu.h>
#include <asm/dma.h>
@@ -1278,6 +1279,20 @@ static int vio_bus_remove(struct device *dev)
return 0;
}
+static void vio_bus_shutdown(struct device *dev)
+{
+ struct vio_dev *viodev = to_vio_dev(dev);
+ struct vio_driver *viodrv;
+
+ if (dev->driver) {
+ viodrv = to_vio_driver(dev->driver);
+ if (viodrv->shutdown)
+ viodrv->shutdown(viodev);
+ else if (kexec_in_progress)
+ vio_bus_remove(dev);
+ }
+}
+
/**
* vio_register_driver: - Register a new vio driver
* @viodrv: The vio_driver structure to be registered.
@@ -1613,6 +1628,7 @@ struct bus_type vio_bus_type = {
.match = vio_bus_match,
.probe = vio_bus_probe,
.remove = vio_bus_remove,
+ .shutdown = vio_bus_shutdown,
};
/**
--
2.27.0
On 4/1/21 5:13 PM, Tyrel Datwyler wrote:
> Currently, neither the vio_bus or vio_driver structures provide support
> for a shutdown() routine.
>
> Add support for shutdown() by allowing drivers to provide a
> implementation via function pointer in their vio_driver struct and
> provide a proper implementation in the driver template for the vio_bus
> that calls a vio drivers shutdown() if defined.
>
> In the case that no shutdown() is defined by a vio driver and a kexec is
> in progress we implement a big hammer that calls remove() to ensure no
> further DMA for the devices is possible.
>
> Signed-off-by: Tyrel Datwyler <[email protected]>
> ---
Ping... any comments, problems with this approach?
-Tyrel
Tyrel Datwyler <[email protected]> writes:
> On 4/1/21 5:13 PM, Tyrel Datwyler wrote:
>> Currently, neither the vio_bus or vio_driver structures provide support
>> for a shutdown() routine.
>>
>> Add support for shutdown() by allowing drivers to provide a
>> implementation via function pointer in their vio_driver struct and
>> provide a proper implementation in the driver template for the vio_bus
>> that calls a vio drivers shutdown() if defined.
>>
>> In the case that no shutdown() is defined by a vio driver and a kexec is
>> in progress we implement a big hammer that calls remove() to ensure no
>> further DMA for the devices is possible.
>>
>> Signed-off-by: Tyrel Datwyler <[email protected]>
>> ---
>
> Ping... any comments, problems with this approach?
The kexec part seems like a bit of a hack.
It also doesn't help for kdump, when none of the shutdown code is run.
How many drivers do we have? Can we just implement a proper shutdown for
them?
cheers
On 4/17/21 5:30 AM, Michael Ellerman wrote:
> Tyrel Datwyler <[email protected]> writes:
>> On 4/1/21 5:13 PM, Tyrel Datwyler wrote:
>>> Currently, neither the vio_bus or vio_driver structures provide support
>>> for a shutdown() routine.
>>>
>>> Add support for shutdown() by allowing drivers to provide a
>>> implementation via function pointer in their vio_driver struct and
>>> provide a proper implementation in the driver template for the vio_bus
>>> that calls a vio drivers shutdown() if defined.
>>>
>>> In the case that no shutdown() is defined by a vio driver and a kexec is
>>> in progress we implement a big hammer that calls remove() to ensure no
>>> further DMA for the devices is possible.
>>>
>>> Signed-off-by: Tyrel Datwyler <[email protected]>
>>> ---
>>
>> Ping... any comments, problems with this approach?
>
> The kexec part seems like a bit of a hack.
>
> It also doesn't help for kdump, when none of the shutdown code is run.
If I understand correctly for kdump we have a reserved memory space where the
kdump kernel is loaded, but for kexec the memory region isn't reserved ahead of
time meaning we can try and load the kernel over potential memory used for DMA
by the current kernel. Please correct me if I've got that wrong.
>
> How many drivers do we have? Can we just implement a proper shutdown for
> them?
Well that is the end goal. I just don't currently have the bandwidth to do each
driver myself with a proper shutdown sequence, and thought this was a launching
off point to at least introduce the shutdown callback to the VIO bus.
Off the top of my head we have 3 storage drivers, 2 network drivers, vtpm, vmc,
pseries_rng, nx, nx842, hvcs, hvc_vio.
I can drop the kexec_in_progress hammer and just have each driver call remove()
themselves in their shutdown function. Leave it to each maintainer to decide if
remove() is enough or if there is a more lightweight quiesce sequence they
choose to implement.
-Tyrel
>
> cheers
>
Tyrel Datwyler <[email protected]> writes:
> On 4/17/21 5:30 AM, Michael Ellerman wrote:
>> Tyrel Datwyler <[email protected]> writes:
>>> On 4/1/21 5:13 PM, Tyrel Datwyler wrote:
>>>> Currently, neither the vio_bus or vio_driver structures provide support
>>>> for a shutdown() routine.
>>>>
>>>> Add support for shutdown() by allowing drivers to provide a
>>>> implementation via function pointer in their vio_driver struct and
>>>> provide a proper implementation in the driver template for the vio_bus
>>>> that calls a vio drivers shutdown() if defined.
>>>>
>>>> In the case that no shutdown() is defined by a vio driver and a kexec is
>>>> in progress we implement a big hammer that calls remove() to ensure no
>>>> further DMA for the devices is possible.
>>>>
>>>> Signed-off-by: Tyrel Datwyler <[email protected]>
>>>> ---
>>>
>>> Ping... any comments, problems with this approach?
>>
>> The kexec part seems like a bit of a hack.
>>
>> It also doesn't help for kdump, when none of the shutdown code is run.
>
> If I understand correctly for kdump we have a reserved memory space where the
> kdump kernel is loaded, but for kexec the memory region isn't reserved ahead of
> time meaning we can try and load the kernel over potential memory used for DMA
> by the current kernel.
That's correct.
>> How many drivers do we have? Can we just implement a proper shutdown for
>> them?
>
> Well that is the end goal. I just don't currently have the bandwidth to do each
> driver myself with a proper shutdown sequence, and thought this was a launching
> off point to at least introduce the shutdown callback to the VIO bus.
Fair enough.
> Off the top of my head we have 3 storage drivers, 2 network drivers, vtpm, vmc,
> pseries_rng, nx, nx842, hvcs, hvc_vio.
>
> I can drop the kexec_in_progress hammer and just have each driver call remove()
> themselves in their shutdown function. Leave it to each maintainer to decide if
> remove() is enough or if there is a more lightweight quiesce sequence they
> choose to implement.
That's OK, you've convinced me. I'll take it as-is.
Eventually it would be good for drivers to implement shutdown in the
optimal way for their device, but that can be done incrementally.
cheers
On Thu, 1 Apr 2021 18:13:25 -0600, Tyrel Datwyler wrote:
> Currently, neither the vio_bus or vio_driver structures provide support
> for a shutdown() routine.
>
> Add support for shutdown() by allowing drivers to provide a
> implementation via function pointer in their vio_driver struct and
> provide a proper implementation in the driver template for the vio_bus
> that calls a vio drivers shutdown() if defined.
>
> [...]
Applied to powerpc/next.
[1/1] powerpc/pseries: Add shutdown() to vio_driver and vio_bus
https://git.kernel.org/powerpc/c/39d0099f94390eb7a677e1a5c9bb56a4daa242a1
cheers