Currently a user can only configure the IRQ CPU affinity of a device
via the global /proc/irq/../smp_afinity interface, however this
interface changes the affinity globally across all subsystems connected
to the device.
Historically, this API is useful for single function devices since,
generally speaking, the queue structure created on top of the IRQ
vectors is predictable enough that this control is usable.
However, with complex multi-subsystem devices, like mlx5, the
assignment of queues at every layer throughout the software stack is
complex and there are multiple queues, each for different usage, over
the same IRQ. Hence, a simple fiddling of the base IRQ is no longer
effective.
As an example mlx5 SF's can share MSI-X IRQ between themselves, which
means that currently user doesn't have control over which SF to use
which CPU set. Hence, an application and IRQ can run on different
CPUs, which leads to lower performance, as shown in the bellow table.
application=netperf, SF-IRQ channel affinity latecy(usec)
(lower is better)
cpu=0 (numa=0) cpu={0} cpu={0} 14.417
cpu=8 (numa=0) cpu={0} cpu={0} 15.114 (+5%)
cpu=1 (numa=1) cpu={0} cpu={0} 17.784 (+30%)
This series is a start at resolving this problem by inverting the
control of the affinities. Instead of having the user go around behind
the driver and adjusting the IRQs the driver already created we want
to have the user tell the software layer what CPUs to use and the
software layer will manage this. The suggested command will then trickle
down to the PCI driver which will create/share MSI-X IRQs and resources
to achieve it. In the mlx5 SF example the involved software components
would be devlink, rdma, vdpa and netdev.
This series introduces a devlink control that assigns a CPU set to the
cross-subsystem mlx5_core PCI function device. This can be used either
on PF, VF or SF and restricts all the software layers above it to the
given CPU set.
For specified CPU, SF either uses an existing IRQ affiliated to the CPU
or a new IRQ available from the device. For example if user gives
affinity 3 (11 in binary), SF will create driver internal required
completion EQ, attached to these specific CPU's IRQ.
If SF is already fully probed, devlink reload is required for
cpu_affinity to take effect.
The following command sets the affinity of mlx5 PF/VF/SF.
devlink command structure:
$ devlink dev param set auxiliary/mlx5_core.sf.4 name cpu_affinity value \
[cpu_bitmask] cmode driverinit
Applications that want to restrict a SF or VF HW to a CPU set, for
instance container workloads, can make use of this API to easily
achieve it.
Shay Drory (4):
net netlink: Introduce NLA_BITFIELD type
devlink: Add support for NLA_BITFIELD for devlink param
devlink: Add new cpu_affinity generic device param
net/mlx5: Support cpu_affinity devlink dev param
.../networking/devlink/devlink-params.rst | 5 +
Documentation/networking/devlink/mlx5.rst | 3 +
.../net/ethernet/mellanox/mlx5/core/devlink.c | 123 +++++++++++++++
.../net/ethernet/mellanox/mlx5/core/devlink.h | 2 +
drivers/net/ethernet/mellanox/mlx5/core/eq.c | 39 +++++
.../ethernet/mellanox/mlx5/core/mlx5_core.h | 2 +
.../ethernet/mellanox/mlx5/core/mlx5_irq.h | 5 +-
.../net/ethernet/mellanox/mlx5/core/pci_irq.c | 85 +++++++++-
include/net/devlink.h | 22 +++
include/net/netlink.h | 30 ++++
include/uapi/linux/netlink.h | 10 ++
lib/nlattr.c | 145 +++++++++++++++++-
net/core/devlink.c | 143 +++++++++++++++--
net/netlink/policy.c | 4 +
14 files changed, 594 insertions(+), 24 deletions(-)
--
2.21.3
Add new device generic parameter to configure device affinity.
A user who wishes to customize the affinity of the device can do it
using below example.
$ devlink dev param set auxiliary/mlx5_core.sf.4 name cpu_affinity \
value [cpu_bitmask] cmode driverinit
$ devlink dev reload pci/0000:06:00.0
At this point devlink instance will use the customize affinity.
Signed-off-by: Shay Drory <[email protected]>
---
Documentation/networking/devlink/devlink-params.rst | 5 +++++
include/net/devlink.h | 4 ++++
net/core/devlink.c | 5 +++++
3 files changed, 14 insertions(+)
diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst
index 4e01dc32bc08..2f9f5baf4373 100644
--- a/Documentation/networking/devlink/devlink-params.rst
+++ b/Documentation/networking/devlink/devlink-params.rst
@@ -137,3 +137,8 @@ own name.
* - ``event_eq_size``
- u32
- Control the size of asynchronous control events EQ.
+ * - ``cpu_affinity``
+ - Bitfield
+ - control the cpu affinity of the device. user is able to change cpu
+ affinity also via procfs interface (/proc/irq/\*/smp_affinity). This will
+ overwrite the devlink setting.
diff --git a/include/net/devlink.h b/include/net/devlink.h
index f411482f716d..595a4d54a2bd 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -466,6 +466,7 @@ enum devlink_param_generic_id {
DEVLINK_PARAM_GENERIC_ID_ENABLE_IWARP,
DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE,
DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE,
+ DEVLINK_PARAM_GENERIC_ID_CPU_AFFINITY,
/* add new param generic ids above here*/
__DEVLINK_PARAM_GENERIC_ID_MAX,
@@ -524,6 +525,9 @@ enum devlink_param_generic_id {
#define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME "event_eq_size"
#define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE DEVLINK_PARAM_TYPE_U32
+#define DEVLINK_PARAM_GENERIC_CPU_AFFINITY_NAME "cpu_affinity"
+#define DEVLINK_PARAM_GENERIC_CPU_AFFINITY_TYPE DEVLINK_PARAM_TYPE_BITFIELD
+
#define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \
{ \
.id = DEVLINK_PARAM_GENERIC_ID_##_id, \
diff --git a/net/core/devlink.c b/net/core/devlink.c
index 3d7e27abc487..d2dfd9a88eb1 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -4477,6 +4477,11 @@ static const struct devlink_param devlink_param_generic[] = {
.name = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME,
.type = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE,
},
+ {
+ .id = DEVLINK_PARAM_GENERIC_ID_CPU_AFFINITY,
+ .name = DEVLINK_PARAM_GENERIC_CPU_AFFINITY_NAME,
+ .type = DEVLINK_PARAM_GENERIC_CPU_AFFINITY_TYPE,
+ },
};
static int devlink_param_generic_verify(const struct devlink_param *param)
--
2.21.3
Generic bitfield attribute content sent to the kernel by user.
With this netlink attr type the user can either set or unset a
bitmap in the kernel.
This attribute is an extension (dynamic array) of NLA_BITFIELD32,
and have similar checks and policies.
Signed-off-by: Shay Drory <[email protected]>
Reviewed-by: Moshe Shemesh <[email protected]>
---
include/net/netlink.h | 30 ++++++++
include/uapi/linux/netlink.h | 10 +++
lib/nlattr.c | 145 ++++++++++++++++++++++++++++++++++-
net/netlink/policy.c | 4 +
4 files changed, 185 insertions(+), 4 deletions(-)
diff --git a/include/net/netlink.h b/include/net/netlink.h
index 7a2a9d3144ba..52a0bcccae36 100644
--- a/include/net/netlink.h
+++ b/include/net/netlink.h
@@ -180,6 +180,7 @@ enum {
NLA_S32,
NLA_S64,
NLA_BITFIELD32,
+ NLA_BITFIELD,
NLA_REJECT,
__NLA_TYPE_MAX,
};
@@ -235,12 +236,16 @@ enum nla_policy_validation {
* given type fits, using it verifies minimum length
* just like "All other"
* NLA_BITFIELD32 Unused
+ * NLA_BITFIELD Maximum length of attribute payload
* NLA_REJECT Unused
* All other Minimum length of attribute payload
*
* Meaning of validation union:
* NLA_BITFIELD32 This is a 32-bit bitmap/bitselector attribute and
* `bitfield32_valid' is the u32 value of valid flags
+ * NLA_BITFIELD This is a dynamic array of 32-bit bitmap/bitselector
+ * attribute and `arr_bitfield32_valid' is the u32
+ * values array of valid flags.
* NLA_REJECT This attribute is always rejected and `reject_message'
* may point to a string to report as the error instead
* of the generic one in extended ACK.
@@ -318,6 +323,7 @@ struct nla_policy {
u16 len;
union {
const u32 bitfield32_valid;
+ const u32 *arr_bitfield32_valid;
const u32 mask;
const char *reject_message;
const struct nla_policy *nested_policy;
@@ -363,6 +369,8 @@ struct nla_policy {
_NLA_POLICY_NESTED_ARRAY(ARRAY_SIZE(policy) - 1, policy)
#define NLA_POLICY_BITFIELD32(valid) \
{ .type = NLA_BITFIELD32, .bitfield32_valid = valid }
+#define NLA_POLICY_BITFIELD(valid, size) \
+ { .type = NLA_BITFIELD, .arr_bitfield32_valid = valid, .len = size }
#define __NLA_IS_UINT_TYPE(tp) \
(tp == NLA_U8 || tp == NLA_U16 || tp == NLA_U32 || tp == NLA_U64)
@@ -1545,6 +1553,19 @@ static inline int nla_put_bitfield32(struct sk_buff *skb, int attrtype,
return nla_put(skb, attrtype, sizeof(tmp), &tmp);
}
+/**
+ * nla_put_bitfield - Add a bitfield netlink attribute to a socket buffer
+ * @skb: socket buffer to add attribute to
+ * @attrtype: attribute type
+ * @bitfield: bitfield
+ */
+static inline int nla_put_bitfield(struct sk_buff *skb, int attrtype,
+ const struct nla_bitfield *bitfield)
+{
+ return nla_put(skb, attrtype, bitfield->size * sizeof(struct nla_bitfield32)
+ + sizeof(*bitfield), bitfield);
+}
+
/**
* nla_get_u32 - return payload of u32 attribute
* @nla: u32 netlink attribute
@@ -1738,6 +1759,15 @@ static inline struct nla_bitfield32 nla_get_bitfield32(const struct nlattr *nla)
return tmp;
}
+struct nla_bitfield *nla_bitfield_alloc(__u64 nbits);
+void nla_bitfield_free(struct nla_bitfield *bitfield);
+void nla_bitfield_to_bitmap(unsigned long *bitmap,
+ struct nla_bitfield *bitfield);
+void nla_bitfield_from_bitmap(struct nla_bitfield *bitfield,
+ unsigned long *bitmap, __u64 bitmap_nbits);
+bool nla_bitfield_len_is_valid(struct nla_bitfield *bitfield, size_t user_len);
+bool nla_bitfield_nbits_valid(struct nla_bitfield *bitfield, size_t nbits);
+
/**
* nla_memdup - duplicate attribute memory (kmemdup)
* @src: netlink attribute to duplicate from
diff --git a/include/uapi/linux/netlink.h b/include/uapi/linux/netlink.h
index 4c0cde075c27..a11bb91e3386 100644
--- a/include/uapi/linux/netlink.h
+++ b/include/uapi/linux/netlink.h
@@ -252,6 +252,14 @@ struct nla_bitfield32 {
__u32 selector;
};
+/* Generic bitmap attribute content sent to the kernel.
+ * The size is the number of elements in the array.
+ */
+struct nla_bitfield {
+ __u64 size;
+ struct nla_bitfield32 data[0];
+};
+
/*
* policy descriptions - it's specific to each family how this is used
* Normally, it should be retrieved via a dump inside another attribute
@@ -283,6 +291,7 @@ struct nla_bitfield32 {
* entry has attributes again, the policy for those inner ones
* and the corresponding maxtype may be specified.
* @NL_ATTR_TYPE_BITFIELD32: &struct nla_bitfield32 attribute
+ * @NL_ATTR_TYPE_BITFIELD: &struct nla_bitfield attribute
*/
enum netlink_attribute_type {
NL_ATTR_TYPE_INVALID,
@@ -307,6 +316,7 @@ enum netlink_attribute_type {
NL_ATTR_TYPE_NESTED_ARRAY,
NL_ATTR_TYPE_BITFIELD32,
+ NL_ATTR_TYPE_BITFIELD,
};
/**
diff --git a/lib/nlattr.c b/lib/nlattr.c
index 86029ad5ead4..6d20bf38850b 100644
--- a/lib/nlattr.c
+++ b/lib/nlattr.c
@@ -58,11 +58,9 @@ static int __nla_validate_parse(const struct nlattr *head, int len, int maxtype,
struct netlink_ext_ack *extack,
struct nlattr **tb, unsigned int depth);
-static int validate_nla_bitfield32(const struct nlattr *nla,
- const u32 valid_flags_mask)
+static int validate_bitfield32(const struct nla_bitfield32 *bf,
+ const u32 valid_flags_mask)
{
- const struct nla_bitfield32 *bf = nla_data(nla);
-
if (!valid_flags_mask)
return -EINVAL;
@@ -81,6 +79,33 @@ static int validate_nla_bitfield32(const struct nlattr *nla,
return 0;
}
+static int validate_nla_bitfield32(const struct nlattr *nla,
+ const u32 valid_flags_mask)
+{
+ const struct nla_bitfield32 *bf = nla_data(nla);
+
+ return validate_bitfield32(bf, valid_flags_mask);
+}
+
+static int validate_nla_bitfield(const struct nlattr *nla,
+ const u32 *valid_flags_masks,
+ const u16 nbits)
+{
+ struct nla_bitfield *bf = nla_data(nla);
+ int err;
+ int i;
+
+ if (!nla_bitfield_len_is_valid(bf, nla_len(nla)) ||
+ !nla_bitfield_nbits_valid(bf, nbits))
+ return -EINVAL;
+ for (i = 0; i < bf->size; i++) {
+ err = validate_bitfield32(&bf->data[i], valid_flags_masks[i]);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
static int nla_validate_array(const struct nlattr *head, int len, int maxtype,
const struct nla_policy *policy,
struct netlink_ext_ack *extack,
@@ -422,6 +447,12 @@ static int validate_nla(const struct nlattr *nla, int maxtype,
goto out_err;
break;
+ case NLA_BITFIELD:
+ err = validate_nla_bitfield(nla, pt->arr_bitfield32_valid, pt->len);
+ if (err)
+ goto out_err;
+ break;
+
case NLA_NUL_STRING:
if (pt->len)
minlen = min_t(int, attrlen, pt->len + 1);
@@ -839,6 +870,112 @@ int nla_strcmp(const struct nlattr *nla, const char *str)
}
EXPORT_SYMBOL(nla_strcmp);
+/**
+ * nla_bitfield_alloc - Alloc struct nla_bitfield
+ * @nbits: number of bits to accommodate
+ */
+struct nla_bitfield *nla_bitfield_alloc(__u64 nbits)
+{
+ struct nla_bitfield *bitfield;
+ size_t bitfield_size;
+ size_t bitfield_len;
+
+ bitfield_len = DIV_ROUND_UP(nbits, BITS_PER_TYPE(u32));
+ bitfield_size = bitfield_len * sizeof(struct nla_bitfield32) +
+ sizeof(*bitfield);
+ bitfield = kzalloc(bitfield_size, GFP_KERNEL);
+ if (bitfield)
+ bitfield->size = bitfield_len;
+ return bitfield;
+}
+EXPORT_SYMBOL(nla_bitfield_alloc);
+
+/**
+ * nla_bitfield_free - Free struct nla_bitfield
+ * @bitfield: the bitfield to free
+ */
+void nla_bitfield_free(struct nla_bitfield *bitfield)
+{
+ kfree(bitfield);
+}
+EXPORT_SYMBOL(nla_bitfield_free);
+
+/**
+ * nla_bitfield_to_bitmap - Convert bitfield to bitmap
+ * @bitmap: bitmap to copy to (dst)
+ * @bitfield: bitfield to be copied (src)
+ */
+void nla_bitfield_to_bitmap(unsigned long *bitmap,
+ struct nla_bitfield *bitfield)
+{
+ int i, j;
+ u32 tmp;
+
+ for (i = 0; i < bitfield->size; i++) {
+ tmp = bitfield->data[i].value & bitfield->data[i].selector;
+ for (j = 0; j < BITS_PER_TYPE(u32); j++)
+ if (tmp & (1 << j))
+ set_bit(j + i * BITS_PER_TYPE(u32), bitmap);
+ }
+}
+EXPORT_SYMBOL(nla_bitfield_to_bitmap);
+
+/**
+ * nla_bitfield_from_bitmap - Convert bitmap to bitfield
+ * @bitfield: bitfield to copy to (dst)
+ * @bitmap: bitmap to be copied (src)
+ * @bitmap_nbits: len of bitmap
+ */
+void nla_bitfield_from_bitmap(struct nla_bitfield *bitfield,
+ unsigned long *bitmap, __u64 bitmap_nbits)
+{
+ long size;
+ int i, j;
+
+ size = DIV_ROUND_UP(bitmap_nbits, BITS_PER_TYPE(u32));
+ for (i = 0; i < size; i++) {
+ for (j = 0; j < min_t(__u64, bitmap_nbits, BITS_PER_TYPE(u32)); j++)
+ if (test_bit(j + i * BITS_PER_TYPE(u32), bitmap))
+ bitfield->data[i].value |= 1 << j;
+ bitfield->data[i].selector = bitmap_nbits >= BITS_PER_TYPE(u32) ?
+ UINT_MAX : (1 << bitmap_nbits) - 1;
+ bitmap_nbits -= BITS_PER_TYPE(u32);
+ }
+}
+EXPORT_SYMBOL(nla_bitfield_from_bitmap);
+
+/**
+ * nla_bitfield_len_is_valid - validate the len of the bitfield
+ * @bitfield: bitfield to validate
+ * @user_len: len of the nla.
+ */
+bool nla_bitfield_len_is_valid(struct nla_bitfield *bitfield, size_t user_len)
+{
+ return !(user_len % sizeof(bitfield->data[0]) ||
+ sizeof(bitfield->data[0]) * bitfield->size +
+ sizeof(*bitfield) != user_len);
+}
+EXPORT_SYMBOL(nla_bitfield_len_is_valid);
+
+/**
+ * nla_bitfield_nbits_valid - validate the len of the bitfield vs a given nbits
+ * @bitfield: bitfield to validate
+ * @nbits: number of bits the user wants to use.
+ */
+bool nla_bitfield_nbits_valid(struct nla_bitfield *bitfield, size_t nbits)
+{
+ u32 *last_value = &bitfield->data[bitfield->size - 1].value;
+ u32 last_bit;
+
+ if (BITS_PER_TYPE(u32) * (bitfield->size - 1) > nbits)
+ return false;
+
+ nbits -= BITS_PER_TYPE(u32) * (bitfield->size - 1);
+ last_bit = find_last_bit((unsigned long *)last_value, BITS_PER_TYPE(u32));
+ return last_bit == BITS_PER_TYPE(u32) ? true : last_bit <= nbits - 1;
+}
+EXPORT_SYMBOL(nla_bitfield_nbits_valid);
+
#ifdef CONFIG_NET
/**
* __nla_reserve - reserve room for attribute on the skb
diff --git a/net/netlink/policy.c b/net/netlink/policy.c
index 8d7c900e27f4..c9fffb3b8045 100644
--- a/net/netlink/policy.c
+++ b/net/netlink/policy.c
@@ -227,6 +227,7 @@ int netlink_policy_dump_attr_size_estimate(const struct nla_policy *pt)
case NLA_STRING:
case NLA_NUL_STRING:
case NLA_BINARY:
+ case NLA_BITFIELD:
/* maximum is common, u32 min-length/max-length */
return common + 2 * nla_attr_size(sizeof(u32));
case NLA_FLAG:
@@ -338,11 +339,14 @@ __netlink_policy_dump_write_attr(struct netlink_policy_dump_state *state,
break;
case NLA_STRING:
case NLA_NUL_STRING:
+ case NLA_BITFIELD:
case NLA_BINARY:
if (pt->type == NLA_STRING)
type = NL_ATTR_TYPE_STRING;
else if (pt->type == NLA_NUL_STRING)
type = NL_ATTR_TYPE_NUL_STRING;
+ else if (pt->type == NLA_BITFIELD)
+ type = NL_ATTR_TYPE_BITFIELD;
else
type = NL_ATTR_TYPE_BINARY;
--
2.21.3
Add devlink support for param of type NLA_BITFIELD.
kernel user need to provide a bitmap to devlink.
Signed-off-by: Shay Drory <[email protected]>
Reviewed-by: Moshe Shemesh <[email protected]>
---
include/net/devlink.h | 18 ++++++
net/core/devlink.c | 138 ++++++++++++++++++++++++++++++++++++------
2 files changed, 139 insertions(+), 17 deletions(-)
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 8d5349d2fb68..f411482f716d 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -372,6 +372,7 @@ enum devlink_param_type {
DEVLINK_PARAM_TYPE_U32,
DEVLINK_PARAM_TYPE_STRING,
DEVLINK_PARAM_TYPE_BOOL,
+ DEVLINK_PARAM_TYPE_BITFIELD,
};
union devlink_param_value {
@@ -380,6 +381,7 @@ union devlink_param_value {
u32 vu32;
char vstr[__DEVLINK_PARAM_MAX_STRING_VALUE];
bool vbool;
+ unsigned long *vbitmap;
};
struct devlink_param_gset_ctx {
@@ -412,6 +414,8 @@ struct devlink_flash_notify {
* @generic: indicates if the parameter is generic or driver specific
* @type: parameter type
* @supported_cmodes: bitmap of supported configuration modes
+ * @nbits: number of bits this param need to use, if this param is
+ * of dynamic len.
* @get: get parameter value, used for runtime and permanent
* configuration modes
* @set: set parameter value, used for runtime and permanent
@@ -427,6 +431,7 @@ struct devlink_param {
bool generic;
enum devlink_param_type type;
unsigned long supported_cmodes;
+ u64 nbits;
int (*get)(struct devlink *devlink, u32 id,
struct devlink_param_gset_ctx *ctx);
int (*set)(struct devlink *devlink, u32 id,
@@ -542,6 +547,19 @@ enum devlink_param_generic_id {
.validate = _validate, \
}
+#define DEVLINK_PARAM_DYNAMIC_GENERIC(_id, _cmodes, _get, _set, _validate, _nbits)\
+{ \
+ .id = DEVLINK_PARAM_GENERIC_ID_##_id, \
+ .name = DEVLINK_PARAM_GENERIC_##_id##_NAME, \
+ .type = DEVLINK_PARAM_GENERIC_##_id##_TYPE, \
+ .generic = true, \
+ .supported_cmodes = _cmodes, \
+ .nbits = _nbits, \
+ .get = _get, \
+ .set = _set, \
+ .validate = _validate, \
+}
+
/* Part number, identifier of board design */
#define DEVLINK_INFO_VERSION_GENERIC_BOARD_ID "board.id"
/* Revision of board design */
diff --git a/net/core/devlink.c b/net/core/devlink.c
index fcd9f6d85cf1..3d7e27abc487 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -4568,6 +4568,8 @@ devlink_param_type_to_nla_type(enum devlink_param_type param_type)
return NLA_STRING;
case DEVLINK_PARAM_TYPE_BOOL:
return NLA_FLAG;
+ case DEVLINK_PARAM_TYPE_BITFIELD:
+ return NLA_BITFIELD;
default:
return -EINVAL;
}
@@ -4575,11 +4577,13 @@ devlink_param_type_to_nla_type(enum devlink_param_type param_type)
static int
devlink_nl_param_value_fill_one(struct sk_buff *msg,
- enum devlink_param_type type,
+ const struct devlink_param *param,
enum devlink_param_cmode cmode,
union devlink_param_value val)
{
struct nlattr *param_value_attr;
+ struct nla_bitfield *bitfield;
+ int err;
param_value_attr = nla_nest_start_noflag(msg,
DEVLINK_ATTR_PARAM_VALUE);
@@ -4589,7 +4593,7 @@ devlink_nl_param_value_fill_one(struct sk_buff *msg,
if (nla_put_u8(msg, DEVLINK_ATTR_PARAM_VALUE_CMODE, cmode))
goto value_nest_cancel;
- switch (type) {
+ switch (param->type) {
case DEVLINK_PARAM_TYPE_U8:
if (nla_put_u8(msg, DEVLINK_ATTR_PARAM_VALUE_DATA, val.vu8))
goto value_nest_cancel;
@@ -4612,6 +4616,17 @@ devlink_nl_param_value_fill_one(struct sk_buff *msg,
nla_put_flag(msg, DEVLINK_ATTR_PARAM_VALUE_DATA))
goto value_nest_cancel;
break;
+ case DEVLINK_PARAM_TYPE_BITFIELD:
+ bitfield = nla_bitfield_alloc(param->nbits);
+ if (!bitfield)
+ return -ENOMEM;
+ nla_bitfield_from_bitmap(bitfield, val.vbitmap, param->nbits);
+ err = nla_put_bitfield(msg, DEVLINK_ATTR_PARAM_VALUE_DATA,
+ bitfield);
+ nla_bitfield_free(bitfield);
+ if (err)
+ goto value_nest_cancel;
+ break;
}
nla_nest_end(msg, param_value_attr);
@@ -4623,6 +4638,24 @@ devlink_nl_param_value_fill_one(struct sk_buff *msg,
return -EMSGSIZE;
}
+static int devlink_param_value_get(const struct devlink_param *param,
+ union devlink_param_value *value)
+{
+ if (param->type == DEVLINK_PARAM_TYPE_BITFIELD) {
+ value->vbitmap = bitmap_zalloc(param->nbits, GFP_KERNEL);
+ if (!value->vbitmap)
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+static void devlink_param_value_put(const struct devlink_param *param,
+ union devlink_param_value *value)
+{
+ if (param->type == DEVLINK_PARAM_TYPE_BITFIELD)
+ bitmap_free(value->vbitmap);
+}
+
static int devlink_nl_param_fill(struct sk_buff *msg, struct devlink *devlink,
unsigned int port_index,
struct devlink_param_item *param_item,
@@ -4645,14 +4678,22 @@ static int devlink_nl_param_fill(struct sk_buff *msg, struct devlink *devlink,
if (!devlink_param_cmode_is_supported(param, i))
continue;
if (i == DEVLINK_PARAM_CMODE_DRIVERINIT) {
- if (!param_item->driverinit_value_valid)
- return -EOPNOTSUPP;
+ if (!param_item->driverinit_value_valid) {
+ err = -EOPNOTSUPP;
+ goto param_value_put;
+ }
param_value[i] = param_item->driverinit_value;
} else {
ctx.cmode = i;
- err = devlink_param_get(devlink, param, &ctx);
+ err = devlink_param_value_get(param, ¶m_value[i]);
if (err)
- return err;
+ goto param_value_put;
+ ctx.val = param_value[i];
+ err = devlink_param_get(devlink, param, &ctx);
+ if (err) {
+ devlink_param_value_put(param, ¶m_value[i]);
+ goto param_value_put;
+ }
param_value[i] = ctx.val;
}
param_value_set[i] = true;
@@ -4660,7 +4701,7 @@ static int devlink_nl_param_fill(struct sk_buff *msg, struct devlink *devlink,
hdr = genlmsg_put(msg, portid, seq, &devlink_nl_family, flags, cmd);
if (!hdr)
- return -EMSGSIZE;
+ goto genlmsg_put_err;
if (devlink_nl_put_handle(msg, devlink))
goto genlmsg_cancel;
@@ -4693,10 +4734,13 @@ static int devlink_nl_param_fill(struct sk_buff *msg, struct devlink *devlink,
for (i = 0; i <= DEVLINK_PARAM_CMODE_MAX; i++) {
if (!param_value_set[i])
continue;
- err = devlink_nl_param_value_fill_one(msg, param->type,
+ err = devlink_nl_param_value_fill_one(msg, param,
i, param_value[i]);
if (err)
goto values_list_nest_cancel;
+ if (i != DEVLINK_PARAM_CMODE_DRIVERINIT)
+ devlink_param_value_put(param, ¶m_value[i]);
+ param_value_set[i] = false;
}
nla_nest_end(msg, param_values_list);
@@ -4710,7 +4754,13 @@ static int devlink_nl_param_fill(struct sk_buff *msg, struct devlink *devlink,
nla_nest_cancel(msg, param_attr);
genlmsg_cancel:
genlmsg_cancel(msg, hdr);
- return -EMSGSIZE;
+genlmsg_put_err:
+ err = -EMSGSIZE;
+param_value_put:
+ for (i = 0; i <= DEVLINK_PARAM_CMODE_MAX; i++)
+ if (i != DEVLINK_PARAM_CMODE_DRIVERINIT && param_value_set[i])
+ devlink_param_value_put(param, ¶m_value[i]);
+ return err;
}
static void devlink_param_notify(struct devlink *devlink,
@@ -4815,6 +4865,9 @@ devlink_param_type_get_from_info(struct genl_info *info,
case NLA_FLAG:
*param_type = DEVLINK_PARAM_TYPE_BOOL;
break;
+ case NLA_BITFIELD:
+ *param_type = DEVLINK_PARAM_TYPE_BITFIELD;
+ break;
default:
return -EINVAL;
}
@@ -4827,6 +4880,7 @@ devlink_param_value_get_from_info(const struct devlink_param *param,
struct genl_info *info,
union devlink_param_value *value)
{
+ struct nla_bitfield *bitfield;
struct nlattr *param_data;
int len;
@@ -4863,6 +4917,18 @@ devlink_param_value_get_from_info(const struct devlink_param *param,
return -EINVAL;
value->vbool = nla_get_flag(param_data);
break;
+ case DEVLINK_PARAM_TYPE_BITFIELD:
+ bitfield = nla_data(param_data);
+
+ if (!nla_bitfield_len_is_valid(bitfield, nla_len(param_data)) ||
+ !nla_bitfield_nbits_valid(bitfield, param->nbits))
+ return -EINVAL;
+ value->vbitmap = bitmap_zalloc(param->nbits, GFP_KERNEL);
+ if (!value->vbitmap)
+ return -ENOMEM;
+
+ nla_bitfield_to_bitmap(value->vbitmap, bitfield);
+ break;
}
return 0;
}
@@ -4936,33 +5002,48 @@ static int __devlink_nl_cmd_param_set_doit(struct devlink *devlink,
if (param->validate) {
err = param->validate(devlink, param->id, value, info->extack);
if (err)
- return err;
+ goto out;
}
- if (!info->attrs[DEVLINK_ATTR_PARAM_VALUE_CMODE])
- return -EINVAL;
+ if (!info->attrs[DEVLINK_ATTR_PARAM_VALUE_CMODE]) {
+ err = -EINVAL;
+ goto out;
+ }
cmode = nla_get_u8(info->attrs[DEVLINK_ATTR_PARAM_VALUE_CMODE]);
- if (!devlink_param_cmode_is_supported(param, cmode))
- return -EOPNOTSUPP;
+ if (!devlink_param_cmode_is_supported(param, cmode)) {
+ err = -EOPNOTSUPP;
+ goto out;
+ }
if (cmode == DEVLINK_PARAM_CMODE_DRIVERINIT) {
if (param->type == DEVLINK_PARAM_TYPE_STRING)
strcpy(param_item->driverinit_value.vstr, value.vstr);
+ else if (param->type == DEVLINK_PARAM_TYPE_BITFIELD)
+ bitmap_copy(param_item->driverinit_value.vbitmap,
+ value.vbitmap,
+ param_item->param->nbits);
else
param_item->driverinit_value = value;
param_item->driverinit_value_valid = true;
} else {
- if (!param->set)
- return -EOPNOTSUPP;
+ if (!param->set) {
+ err = -EOPNOTSUPP;
+ goto out;
+ }
ctx.val = value;
ctx.cmode = cmode;
err = devlink_param_set(devlink, param, &ctx);
if (err)
- return err;
+ goto out;
}
+ devlink_param_value_put(param, &value);
devlink_param_notify(devlink, port_index, param_item, cmd);
return 0;
+
+out:
+ devlink_param_value_put(param, &value);
+ return err;
}
static int devlink_nl_cmd_param_set_doit(struct sk_buff *skb,
@@ -10098,6 +10179,8 @@ static int devlink_param_verify(const struct devlink_param *param)
{
if (!param || !param->name || !param->supported_cmodes)
return -EINVAL;
+ if (param->type == DEVLINK_PARAM_TYPE_BITFIELD && !param->nbits)
+ return -EINVAL;
if (param->generic)
return devlink_param_generic_verify(param);
else
@@ -10188,6 +10271,16 @@ int devlink_param_register(struct devlink *devlink,
return -ENOMEM;
param_item->param = param;
+ if (param_item->param->type == DEVLINK_PARAM_TYPE_BITFIELD &&
+ devlink_param_cmode_is_supported(param_item->param,
+ DEVLINK_PARAM_CMODE_DRIVERINIT)) {
+ param_item->driverinit_value.vbitmap =
+ bitmap_zalloc(param_item->param->nbits, GFP_KERNEL);
+ if (!param_item->driverinit_value.vbitmap) {
+ kfree(param_item);
+ return -ENOMEM;
+ }
+ }
list_add_tail(¶m_item->list, &devlink->param_list);
return 0;
@@ -10210,6 +10303,10 @@ void devlink_param_unregister(struct devlink *devlink,
devlink_param_find_by_name(&devlink->param_list, param->name);
WARN_ON(!param_item);
list_del(¶m_item->list);
+ if (param_item->param->type == DEVLINK_PARAM_TYPE_BITFIELD &&
+ devlink_param_cmode_is_supported(param_item->param,
+ DEVLINK_PARAM_CMODE_DRIVERINIT))
+ bitmap_free(param_item->driverinit_value.vbitmap);
kfree(param_item);
}
EXPORT_SYMBOL_GPL(devlink_param_unregister);
@@ -10244,6 +10341,10 @@ int devlink_param_driverinit_value_get(struct devlink *devlink, u32 param_id,
if (param_item->param->type == DEVLINK_PARAM_TYPE_STRING)
strcpy(init_val->vstr, param_item->driverinit_value.vstr);
+ else if (param_item->param->type == DEVLINK_PARAM_TYPE_BITFIELD)
+ bitmap_copy(init_val->vbitmap,
+ param_item->driverinit_value.vbitmap,
+ param_item->param->nbits);
else
*init_val = param_item->driverinit_value;
@@ -10280,6 +10381,9 @@ int devlink_param_driverinit_value_set(struct devlink *devlink, u32 param_id,
if (param_item->param->type == DEVLINK_PARAM_TYPE_STRING)
strcpy(param_item->driverinit_value.vstr, init_val.vstr);
+ else if (param_item->param->type == DEVLINK_PARAM_TYPE_BITFIELD)
+ bitmap_copy(param_item->driverinit_value.vbitmap,
+ init_val.vbitmap, param_item->param->nbits);
else
param_item->driverinit_value = init_val;
param_item->driverinit_value_valid = true;
--
2.21.3