From: Diana Craciun <[email protected]>
The vfio-mc bus driver needs some additional services to be exported by the
mc-bus driver like:
- a way to reset the DPRC container
- driver_override support
- functions to setup/tear down a DPRC
- functions for allocating the pool of interrupts. In case of VFIO the
interrupts are not configured at probe time, but later by userspace
request
v2 -> v3
- Add a new version for dprc_get_obj_region
- Export the cacheability bus specific bits defines
v1 -> v2
- Remove driver_override propagation through various functions
- Cache the DPRC API version
The patches are related with "vfio/fsl-mc: VFIO support for FSL-MC
devices" patches, but the series were split because they are targeting
different subsystems. However, the mc-bus patches may suffer changes
when addressing the VFIO review comments.
Bharat Bhushan (3):
bus/fsl-mc: add support for 'driver_override' in the mc-bus
bus/fsl-mc: Add dprc-reset-container support
bus/fsl-mc: Extend ICID size from 16bit to 32bit
Diana Craciun (10):
bus/fsl-mc: Do no longer export the total number of irqs outside
dprc_scan_objects
bus/fsl-mc: Add a new parameter to dprc_scan_objects function
bus/fsl-mc: Set the QMAN/BMAN region flags
bus/fsl-mc: Cache the DPRC API version
bus/fsl-mc: Export a dprc scan function to be used by multiple
entities
bus/fsl-mc: Export a cleanup function for DPRC
bus/fsl-mc: Add a container setup function
bus/fsl_mc: Do not rely on caller to provide non NULL mc_io
bus/fsl-mc: Export IRQ pool handling functions to be used by VFIO
bus/fsl-mc: Add a new version for dprc_get_obj_region command
drivers/bus/fsl-mc/dprc-driver.c | 181 ++++++++++++++++----------
drivers/bus/fsl-mc/dprc.c | 141 ++++++++++++++++----
drivers/bus/fsl-mc/fsl-mc-allocator.c | 12 +-
drivers/bus/fsl-mc/fsl-mc-bus.c | 64 ++++++++-
drivers/bus/fsl-mc/fsl-mc-private.h | 31 ++---
drivers/bus/fsl-mc/mc-io.c | 7 +-
include/linux/fsl/mc.h | 37 +++++-
7 files changed, 348 insertions(+), 125 deletions(-)
--
2.17.1
From: Diana Craciun <[email protected]>
There are already firmware API commands that have multiple
versions. For each multiple version command, another command
to retrieve the API version is issued. This may introduce an important
overhead. The version does not change while the system is running,
so the DPRC API version can be safely cached.
Signed-off-by: Diana Craciun <[email protected]>
---
drivers/bus/fsl-mc/dprc.c | 30 +++++++++++++++++++++---------
1 file changed, 21 insertions(+), 9 deletions(-)
diff --git a/drivers/bus/fsl-mc/dprc.c b/drivers/bus/fsl-mc/dprc.c
index 602f030d84eb..01bf41743efc 100644
--- a/drivers/bus/fsl-mc/dprc.c
+++ b/drivers/bus/fsl-mc/dprc.c
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause)
/*
* Copyright 2013-2016 Freescale Semiconductor Inc.
+ * Copyright 2020 NXP
*
*/
#include <linux/kernel.h>
@@ -8,6 +9,13 @@
#include "fsl-mc-private.h"
+/**
+ * cache the DPRC version to reduce the number of commands
+ * towards the mc firmware
+ */
+static u16 dprc_major_ver;
+static u16 dprc_minor_ver;
+
/**
* dprc_open() - Open DPRC object for use
* @mc_io: Pointer to MC portal's I/O object
@@ -443,15 +451,19 @@ int dprc_get_obj_region(struct fsl_mc_io *mc_io,
struct fsl_mc_command cmd = { 0 };
struct dprc_cmd_get_obj_region *cmd_params;
struct dprc_rsp_get_obj_region *rsp_params;
- u16 major_ver, minor_ver;
int err;
- /* prepare command */
- err = dprc_get_api_version(mc_io, 0,
- &major_ver,
- &minor_ver);
- if (err)
- return err;
+ /**
+ * If the DPRC object version was not yet cached, cache it now.
+ * Otherwise use the already cached value.
+ */
+ if (!dprc_major_ver && !dprc_minor_ver) {
+ err = dprc_get_api_version(mc_io, 0,
+ &dprc_major_ver,
+ &dprc_minor_ver);
+ if (err)
+ return err;
+ }
/**
* MC API version 6.3 introduced a new field to the region
@@ -459,7 +471,7 @@ int dprc_get_obj_region(struct fsl_mc_io *mc_io,
* address is set to zero to indicate it needs to be obtained elsewhere
* (typically the device tree).
*/
- if (major_ver > 6 || (major_ver == 6 && minor_ver >= 3))
+ if (dprc_major_ver > 6 || (dprc_major_ver == 6 && dprc_minor_ver >= 3))
cmd.header =
mc_encode_cmd_header(DPRC_CMDID_GET_OBJ_REG_V2,
cmd_flags, token);
@@ -483,7 +495,7 @@ int dprc_get_obj_region(struct fsl_mc_io *mc_io,
rsp_params = (struct dprc_rsp_get_obj_region *)cmd.params;
region_desc->base_offset = le64_to_cpu(rsp_params->base_offset);
region_desc->size = le32_to_cpu(rsp_params->size);
- if (major_ver > 6 || (major_ver == 6 && minor_ver >= 3))
+ if (dprc_major_ver > 6 || (dprc_major_ver == 6 && dprc_minor_ver >= 3))
region_desc->base_address = le64_to_cpu(rsp_params->base_addr);
else
region_desc->base_address = 0;
--
2.17.1
From: Bharat Bhushan <[email protected]>
This patch is required for vfio-fsl-mc meta driver to successfully bind
layerscape container devices for device passthrough. This patch adds
a mechanism to allow a layerscape device to specify a driver rather than
a layerscape driver provide a device match.
Example to allow a device (dprc.1) to specifically bind
with driver (vfio-fsl-mc):-
- echo vfio-fsl-mc > /sys/bus/fsl-mc/devices/dprc.1/driver_override
- echo dprc.1 > /sys/bus/fsl-mc/drivers/fsl_mc_dprc/unbind
- echo dprc.1 > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind
Signed-off-by: Bharat Bhushan <[email protected]>
Signed-off-by: Laurentiu Tudor <[email protected]>
Signed-off-by: Diana Craciun <[email protected]>
---
drivers/bus/fsl-mc/fsl-mc-bus.c | 54 +++++++++++++++++++++++++++++++++
include/linux/fsl/mc.h | 2 ++
2 files changed, 56 insertions(+)
diff --git a/drivers/bus/fsl-mc/fsl-mc-bus.c b/drivers/bus/fsl-mc/fsl-mc-bus.c
index 40526da5c6a6..3a86f5087717 100644
--- a/drivers/bus/fsl-mc/fsl-mc-bus.c
+++ b/drivers/bus/fsl-mc/fsl-mc-bus.c
@@ -3,6 +3,7 @@
* Freescale Management Complex (MC) bus driver
*
* Copyright (C) 2014-2016 Freescale Semiconductor, Inc.
+ * Copyright 2019-2020 NXP
* Author: German Rivera <[email protected]>
*
*/
@@ -71,6 +72,12 @@ static int fsl_mc_bus_match(struct device *dev, struct device_driver *drv)
struct fsl_mc_driver *mc_drv = to_fsl_mc_driver(drv);
bool found = false;
+ /* When driver_override is set, only bind to the matching driver */
+ if (mc_dev->driver_override) {
+ found = !strcmp(mc_dev->driver_override, mc_drv->driver.name);
+ goto out;
+ }
+
if (!mc_drv->match_id_table)
goto out;
@@ -135,8 +142,52 @@ static ssize_t modalias_show(struct device *dev, struct device_attribute *attr,
}
static DEVICE_ATTR_RO(modalias);
+static ssize_t driver_override_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
+ char *driver_override, *old = mc_dev->driver_override;
+ char *cp;
+
+ if (WARN_ON(dev->bus != &fsl_mc_bus_type))
+ return -EINVAL;
+
+ if (count >= (PAGE_SIZE - 1))
+ return -EINVAL;
+
+ driver_override = kstrndup(buf, count, GFP_KERNEL);
+ if (!driver_override)
+ return -ENOMEM;
+
+ cp = strchr(driver_override, '\n');
+ if (cp)
+ *cp = '\0';
+
+ if (strlen(driver_override)) {
+ mc_dev->driver_override = driver_override;
+ } else {
+ kfree(driver_override);
+ mc_dev->driver_override = NULL;
+ }
+
+ kfree(old);
+
+ return count;
+}
+
+static ssize_t driver_override_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
+
+ return snprintf(buf, PAGE_SIZE, "%s\n", mc_dev->driver_override);
+}
+static DEVICE_ATTR_RW(driver_override);
+
static struct attribute *fsl_mc_dev_attrs[] = {
&dev_attr_modalias.attr,
+ &dev_attr_driver_override.attr,
NULL,
};
@@ -706,6 +757,9 @@ EXPORT_SYMBOL_GPL(fsl_mc_device_add);
*/
void fsl_mc_device_remove(struct fsl_mc_device *mc_dev)
{
+ kfree(mc_dev->driver_override);
+ mc_dev->driver_override = NULL;
+
/*
* The device-specific remove callback will get invoked by device_del()
*/
diff --git a/include/linux/fsl/mc.h b/include/linux/fsl/mc.h
index 2b5f8366dbe1..4e9b570a1845 100644
--- a/include/linux/fsl/mc.h
+++ b/include/linux/fsl/mc.h
@@ -161,6 +161,7 @@ struct fsl_mc_obj_desc {
* @regions: pointer to array of MMIO region entries
* @irqs: pointer to array of pointers to interrupts allocated to this device
* @resource: generic resource associated with this MC object device, if any.
+ * @driver_override: driver name to force a match
*
* Generic device object for MC object devices that are "attached" to a
* MC bus.
@@ -194,6 +195,7 @@ struct fsl_mc_device {
struct fsl_mc_device_irq **irqs;
struct fsl_mc_resource *resource;
struct device_link *consumer_link;
+ char *driver_override;
};
#define to_fsl_mc_device(_dev) \
--
2.17.1
From: Diana Craciun <[email protected]>
Prepare the dprc_scan_objects function to be used by
the VFIO mc driver code. The function is used to scan the mc
objects by the bus driver. The same functionality is
needed by the VFIO mc driver, but in this case the
interrupt configuration is delayed until the userspace
configures the interrupts. In order to use the same function
in both drivers add a new parameter.
Signed-off-by: Diana Craciun <[email protected]>
---
drivers/bus/fsl-mc/dprc-driver.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/drivers/bus/fsl-mc/dprc-driver.c b/drivers/bus/fsl-mc/dprc-driver.c
index 035b220779d0..7a8061224df8 100644
--- a/drivers/bus/fsl-mc/dprc-driver.c
+++ b/drivers/bus/fsl-mc/dprc-driver.c
@@ -198,6 +198,8 @@ static void dprc_add_new_devices(struct fsl_mc_device *mc_bus_dev,
* dprc_scan_objects - Discover objects in a DPRC
*
* @mc_bus_dev: pointer to the fsl-mc device that represents a DPRC object
+ * @alloc_interrupts: if true the function allocates the interrupt pool,
+ * otherwise the interrupt allocation is delayed
*
* Detects objects added and removed from a DPRC and synchronizes the
* state of the Linux bus driver, MC by adding and removing
@@ -211,7 +213,8 @@ static void dprc_add_new_devices(struct fsl_mc_device *mc_bus_dev,
* populated before they can get allocation requests from probe callbacks
* of the device drivers for the non-allocatable devices.
*/
-static int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev)
+static int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev,
+ bool alloc_interrupts)
{
int num_child_objects;
int dprc_get_obj_failures;
@@ -299,7 +302,7 @@ static int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev)
irq_count, FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS);
}
- if (!mc_bus->irq_resources) {
+ if (alloc_interrupts && !mc_bus->irq_resources) {
error = fsl_mc_populate_irq_pool(mc_bus,
FSL_MC_IRQ_POOL_MAX_TOTAL_IRQS);
if (error < 0)
@@ -339,7 +342,7 @@ static int dprc_scan_container(struct fsl_mc_device *mc_bus_dev)
* Discover objects in the DPRC:
*/
mutex_lock(&mc_bus->scan_mutex);
- error = dprc_scan_objects(mc_bus_dev);
+ error = dprc_scan_objects(mc_bus_dev, true);
mutex_unlock(&mc_bus->scan_mutex);
if (error < 0) {
fsl_mc_cleanup_all_resource_pools(mc_bus_dev);
@@ -409,7 +412,7 @@ static irqreturn_t dprc_irq0_handler_thread(int irq_num, void *arg)
DPRC_IRQ_EVENT_OBJ_DESTROYED |
DPRC_IRQ_EVENT_OBJ_CREATED)) {
- error = dprc_scan_objects(mc_dev);
+ error = dprc_scan_objects(mc_dev, true);
if (error < 0) {
/*
* If the error is -ENXIO, we ignore it, as it indicates
--
2.17.1
Hi Diana
On Mon, Jul 06, 2020 at 03:42:33PM +0300, Diana Craciun wrote:
> From: Bharat Bhushan <[email protected]>
>
> This patch is required for vfio-fsl-mc meta driver to successfully bind
> layerscape container devices for device passthrough. This patch adds
> a mechanism to allow a layerscape device to specify a driver rather than
> a layerscape driver provide a device match.
>
> Example to allow a device (dprc.1) to specifically bind
> with driver (vfio-fsl-mc):-
> - echo vfio-fsl-mc > /sys/bus/fsl-mc/devices/dprc.1/driver_override
> - echo dprc.1 > /sys/bus/fsl-mc/drivers/fsl_mc_dprc/unbind
> - echo dprc.1 > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind
Something i asked before, why is this buried in a driver, when it
could be put in the driver core. This is not the only driver doing
this, so it does make sense to share the code and bugs.
Andrew
Hi Andrew,
I apologize, I somehow missed you previous comment.
OK, I see that there are a number of drivers using the same code. In
case we share the code, probably we need to store the driver_override
value in the struct device (not in the particular driver device
structure as it is done now).
Diana
On 7/6/2020 4:57 PM, Andrew Lunn wrote:
> Hi Diana
>
> On Mon, Jul 06, 2020 at 03:42:33PM +0300, Diana Craciun wrote:
>> From: Bharat Bhushan <[email protected]>
>>
>> This patch is required for vfio-fsl-mc meta driver to successfully bind
>> layerscape container devices for device passthrough. This patch adds
>> a mechanism to allow a layerscape device to specify a driver rather than
>> a layerscape driver provide a device match.
>>
>> Example to allow a device (dprc.1) to specifically bind
>> with driver (vfio-fsl-mc):-
>> - echo vfio-fsl-mc > /sys/bus/fsl-mc/devices/dprc.1/driver_override
>> - echo dprc.1 > /sys/bus/fsl-mc/drivers/fsl_mc_dprc/unbind
>> - echo dprc.1 > /sys/bus/fsl-mc/drivers/vfio-fsl-mc/bind
> Something i asked before, why is this buried in a driver, when it
> could be put in the driver core. This is not the only driver doing
> this, so it does make sense to share the code and bugs.
>
> Andrew