It's been a while since the last time sw_flow_key was made smaller in
1139e241ec43 ("openvswitch: Compact sw_flow_key."), and it has seen five
patches adding new members since then.
With the current linux-next kernel and gcc-6.0 on ARM, this has tipped
us slightly over the stack frame warning limit of 1024 bytes:
net/openvswitch/datapath.c: In function 'ovs_flow_cmd_set':
net/openvswitch/datapath.c:1202:1: error: the frame size of 1032 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]
This slightly rearranges the members in struct sw_flow_key to minimize
the amount of padding we have between them, bringing us again slightly
below the warning limit (checking all files against 128 bytes limit):
datapath.c: In function 'get_flow_actions.constprop':
datapath.c:1083:1: warning: the frame size of 464 bytes is larger than 128 bytes
datapath.c: In function 'ovs_flow_cmd_new':
datapath.c:1061:1: warning: the frame size of 984 bytes is larger than 128 bytes
datapath.c: In function 'ovs_flow_cmd_del':
datapath.c:1336:1: warning: the frame size of 528 bytes is larger than 128 bytes
datapath.c: In function 'ovs_flow_cmd_get':
datapath.c:1261:1: warning: the frame size of 504 bytes is larger than 128 bytes
datapath.c: In function 'ovs_flow_cmd_set':
datapath.c:1202:1: warning: the frame size of 1016 bytes is larger than 128 bytes
datapath.c: In function 'queue_gso_packets':
datapath.c:379:1: warning: the frame size of 472 bytes is larger than 128 bytes
flow_table.c: In function 'masked_flow_lookup':
flow_table.c:489:1: warning: the frame size of 488 bytes is larger than 128 bytes
flow_netlink.c: In function 'validate_and_copy_set_tun':
flow_netlink.c:1994:1: warning: the frame size of 512 bytes is larger than 128 bytes
This means it's still too large really, we just don't warn about it any more,
and will get the warning again once another member is added. My patch is a
band-aid at best, but more work is needed here. One problem is that
ovs_flow_cmd_new and ovs_flow_cmd_set have two copies of struct sw_flow_key on
the stack, one of them nested within sw_flow_mask. If we could reduce that to
a single instance, the stack usage would be cut in half here.
Signed-off-by: Arnd Bergmann <[email protected]>
Fixes: 00a93babd06a ("openvswitch: add tunnel protocol to sw_flow_key")
Fixes: c2ac66735870 ("openvswitch: Allow matching on conntrack label")
Fixes: 182e3042e15d ("openvswitch: Allow matching on conntrack mark")
Fixes: 7f8a436eaa2c ("openvswitch: Add conntrack action")
Fixes: 971427f353f3 ("openvswitch: Add recirc and hash action.")
---
net/openvswitch/flow.h | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/net/openvswitch/flow.h b/net/openvswitch/flow.h
index 1d055c559eaf..41d15c50a43f 100644
--- a/net/openvswitch/flow.h
+++ b/net/openvswitch/flow.h
@@ -63,7 +63,11 @@ struct sw_flow_key {
u32 skb_mark; /* SKB mark. */
u16 in_port; /* Input switch port (or DP_MAX_PORTS). */
} __packed phy; /* Safe when right after 'tun_key'. */
- u8 tun_proto; /* Protocol of encapsulating tunnel. */
+ struct {
+ __be16 src; /* TCP/UDP/SCTP source port. */
+ __be16 dst; /* TCP/UDP/SCTP destination port. */
+ __be16 flags; /* TCP flags. */
+ } tp;
u32 ovs_flow_hash; /* Datapath computed hash value. */
u32 recirc_id; /* Recirculation ID. */
struct {
@@ -83,11 +87,6 @@ struct sw_flow_key {
u8 frag; /* One of OVS_FRAG_TYPE_*. */
} ip;
};
- struct {
- __be16 src; /* TCP/UDP/SCTP source port. */
- __be16 dst; /* TCP/UDP/SCTP destination port. */
- __be16 flags; /* TCP flags. */
- } tp;
union {
struct {
struct {
@@ -114,11 +113,12 @@ struct sw_flow_key {
};
struct {
/* Connection tracking fields. */
- u16 zone;
u32 mark;
+ u16 zone;
u8 state;
struct ovs_key_ct_labels labels;
} ct;
+ u8 tun_proto; /* Protocol of encapsulating tunnel. */
} __aligned(BITS_PER_LONG/8); /* Ensure that we can do comparisons as longs. */
--
2.7.0
On Fri, Mar 18, 2016 at 6:34 AM, Arnd Bergmann <[email protected]> wrote:
> This means it's still too large really, we just don't warn about it any more,
> and will get the warning again once another member is added. My patch is a
> band-aid at best, but more work is needed here. One problem is that
> ovs_flow_cmd_new and ovs_flow_cmd_set have two copies of struct sw_flow_key on
> the stack, one of them nested within sw_flow_mask. If we could reduce that to
> a single instance, the stack usage would be cut in half here.
I think it should be possible to reduce those two functions to only
use a single instance of the structure.
For new flows, we're already allocating the key structure on the heap,
it seems like we could just translate the key into that rather than to
intermediate location on the stack. And when setting flows, I'm not
sure that we really need the mask at all - we don't do anything with
it other than check it against the actions but we really should be
using the original mask for that rather than the new one.
It seems like it would be preferable to go down that route rather than
this patch, since as you say, this patch is really only suppressing
the warning and doesn't have a significant impact on the actual stack
consumption. Plus, the ordering of the flow layout affects how much
data needs to be compared during packet lookup, so this change will
likely be bad for forwarding performance.