On my server, I found a general protection fault in kernel message.
Bellow is the detailed information.
[ 34.234846] general protection fault: 0000 [#1] SMP PTI
[ 34.235498] CPU: 0 PID: 147 Comm: kworker/u2:3 Not tainted
4.20.0-rc3-next-20181120 #23
[ 34.236461] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
rel-1.11.0-0-g63451fca13-prebuilt.qemu-project.org 04/01/2014
[ 34.238216] Workqueue: netns cleanup_net
[ 34.238623] RIP: 0010:unregister_sysctl_table+0x13/0x80
[ 34.239202] Code: 6d ff ff ff 48 c7 c7 60 b1 07 83 bd f4 ff ff ff e8 22 1d a7
00 eb c5 0f 1f 44 00 00 41 55 48 85 ff 41 54 55 53 48 89 fb 74 30 <48> 8b 7f 20
e8 04 f1 ff ff 83 f8 01 7f 29 48 c7 c7 60 b1 07 83 e8
[ 34.241920] RSP: 0018:ffffc9000022fda8 EFLAGS: 00010206
[ 34.242496] RAX: 0000000000000000 RBX: 0000d2f000002328 RCX: 0000000000000000
[ 34.243480] RDX: 000000000000001c RSI: ffffffff82999d00 RDI: 0000d2f000002328
[ 34.244311] RBP: ffffc9000022fe30 R08: 000000000000000a R09: 0000000000002800
[ 34.245274] R10: 000000000000024a R11: ffffea0000f64a40 R12: ffffffff8294a658
[ 34.246191] R13: ffffffff8294a660 R14: ffffffff82941e00 R15: ffffc9000022fe30
[ 34.247217] FS: 0000000000000000(0000) GS:ffff88803ea00000(0000)
knlGS:0000000000000000
[ 34.248230] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 34.248984] CR2: 00007faa1819b2a8 CR3: 0000000002828005 CR4: 00000000003606f0
[ 34.249845] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 34.250695] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 34.251535] Call Trace:
[ 34.251848] nf_ct_l4proto_pernet_unregister_one+0x45/0x60
[ 34.252479] proto_gre_net_exit+0x18/0x90
[ 34.252888] ops_exit_list.isra.8+0x33/0x60
[ 34.253332] cleanup_net+0x195/0x2a0
[ 34.253698] process_one_work+0x15f/0x360
[ 34.254190] worker_thread+0x49/0x3e0
[ 34.254544] kthread+0xf5/0x130
[ 34.254966] ? process_one_work+0x360/0x360
[ 34.255401] ? kthread_park+0x80/0x80
[ 34.255916] ret_from_fork+0x35/0x40
[ 34.256269] Modules linked in:
[ 34.256582] ---[ end trace be3904a1ee0bddf8 ]---
[ 34.257080] RIP: 0010:unregister_sysctl_table+0x13/0x80
[ 34.257697] Code: 6d ff ff ff 48 c7 c7 60 b1 07 83 bd f4 ff ff ff e8 22 1d a7
00 eb c5 0f 1f 44 00 00 41 55 48 85 ff 41 54 55 53 48 89 fb 74 30 <48> 8b 7f 20
e8 04 f1 ff ff 83 f8 01 7f 29 48 c7 c7 60 b1 07 83 e8
[ 34.260268] RSP: 0018:ffffc9000022fda8 EFLAGS: 00010206
[ 34.260864] RAX: 0000000000000000 RBX: 0000d2f000002328 RCX: 0000000000000000
[ 34.261717] RDX: 000000000000001c RSI: ffffffff82999d00 RDI: 0000d2f000002328
[ 34.262569] RBP: ffffc9000022fe30 R08: 000000000000000a R09: 0000000000002800
[ 34.263592] R10: 000000000000024a R11: ffffea0000f64a40 R12: ffffffff8294a658
[ 34.264449] R13: ffffffff8294a660 R14: ffffffff82941e00 R15: ffffc9000022fe30
[ 34.265295] FS: 0000000000000000(0000) GS:ffff88803ea00000(0000)
knlGS:0000000000000000
[ 34.266395] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 34.267044] CR2: 00007faa1819b2a8 CR3: 0000000002828005 CR4: 00000000003606f0
[ 34.267936] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 34.268881] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
This general protection fault is in function unregister_sysctl_table(),
because 'header' is a pointer that isn't kmalloced.
If some modules(in this case, it is GRE) forget to kmemdup sysctl table,
'pn->ctl_table' will be NULL in function nf_ct_l4proto_register_sysctl(),
and then register_net_sysctl() can't be executed, so the 'header' in
__register_sysctl_table() will never be assigned,
header = kzalloc(sizeof(struct ctl_table_header) +
sizeof(struct ctl_node)*nr_entries, GFP_KERNEL);
But pn->users will be incremented as the return value of
nf_ct_l4proto_register_sysctl() is 0.
As a result of that behavior, when doing unregister,
unregister_net_sysctl_table() will be executed. Then we will access a
pointer that isn't assigned. That's why general protection fault occurs.
This patch is to fix this general protection fault issue.
After this patch, an error message will be printed to indicate some error
happens, for example, in this case bellow message will be printed,
"nf_conntrack_gre4: pernet registration failed."
Signed-off-by: Yafang Shao <[email protected]>
---
net/netfilter/nf_conntrack_proto.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c
index 40643af..154e8c0 100644
--- a/net/netfilter/nf_conntrack_proto.c
+++ b/net/netfilter/nf_conntrack_proto.c
@@ -192,8 +192,12 @@ int nf_ct_l4proto_register_sysctl(struct net *net,
pn->ctl_table = NULL;
}
}
+ } else {
+ /* in case any module doesn't kmemdup sysctl table */
+ err = -ENOENT;
}
#endif /* CONFIG_SYSCTL */
+
return err;
}
--
1.8.3.1
After this patch, there will be two sysctl knobs for GRE.
net.netfilter.nf_conntrack_gre_timeout_replied = 180
net.netfilter.nf_conntrack_gre_timeout_unreplied = 30
Signed-off-by: Yafang Shao <[email protected]>
---
net/netfilter/nf_conntrack_proto_gre.c | 43 +++++++++++++++++++++++++++++++++-
1 file changed, 42 insertions(+), 1 deletion(-)
diff --git a/net/netfilter/nf_conntrack_proto_gre.c b/net/netfilter/nf_conntrack_proto_gre.c
index 2a5e56c..a70894e 100644
--- a/net/netfilter/nf_conntrack_proto_gre.c
+++ b/net/netfilter/nf_conntrack_proto_gre.c
@@ -320,9 +320,50 @@ static int gre_timeout_nlattr_to_obj(struct nlattr *tb[],
};
#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */
+#ifdef CONFIG_SYSCTL
+static struct ctl_table gre_sysctl_table[] = {
+ {
+ .procname = "nf_conntrack_gre_timeout_unreplied",
+ .maxlen = sizeof(unsigned int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_jiffies,
+ },
+ {
+ .procname = "nf_conntrack_gre_timeout_replied",
+ .maxlen = sizeof(unsigned int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_jiffies,
+ },
+ {}
+};
+#endif /* CONFIG_SYSCTL */
+
+static int gre_kmemdup_sysctl_table(struct net *net, struct nf_proto_net *nf,
+ struct netns_proto_gre *net_gre)
+{
+#ifdef CONFIG_SYSCTL
+ int i;
+
+ if (nf->ctl_table)
+ return 0;
+
+ nf->ctl_table = kmemdup(gre_sysctl_table,
+ sizeof(gre_sysctl_table),
+ GFP_KERNEL);
+ if (!nf->ctl_table)
+ return -ENOMEM;
+
+ for (i = 0; i < GRE_CT_MAX; i++)
+ nf->ctl_table[i].data = &net_gre->gre_timeouts[i];
+#endif
+
+ return 0;
+}
+
static int gre_init_net(struct net *net)
{
struct netns_proto_gre *net_gre = gre_pernet(net);
+ struct nf_proto_net *nf = &net_gre->nf;
int i;
rwlock_init(&net_gre->keymap_lock);
@@ -330,7 +371,7 @@ static int gre_init_net(struct net *net)
for (i = 0; i < GRE_CT_MAX; i++)
net_gre->gre_timeouts[i] = gre_timeouts[i];
- return 0;
+ return gre_kmemdup_sysctl_table(net, nf, net_gre);
}
/* protocol helper struct */
--
1.8.3.1
All of the callers will execute drop_sysctl_table() whatever the callee
insert_header() successes or fails.
So we can't execute drop_sysctl_table() in insert_header().
Signed-off-by: Yafang Shao <[email protected]>
---
fs/proc/proc_sysctl.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c
index 89921a0..9aeb750 100644
--- a/fs/proc/proc_sysctl.c
+++ b/fs/proc/proc_sysctl.c
@@ -241,7 +241,6 @@ static int insert_header(struct ctl_dir *dir, struct ctl_table_header *header)
if (header->ctl_table == sysctl_mount_point)
clear_empty_dir(dir);
header->parent = NULL;
- drop_sysctl_table(&dir->header);
return err;
}
--
1.8.3.1
If pn is NULL, it will return 0.
That's not proper. We should return an error.
Signed-off-by: Yafang Shao <[email protected]>
---
net/netfilter/nf_conntrack_proto.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c
index 316fef3..3caf137 100644
--- a/net/netfilter/nf_conntrack_proto.c
+++ b/net/netfilter/nf_conntrack_proto.c
@@ -252,6 +252,7 @@ int nf_ct_l4proto_pernet_register_one(struct net *net,
goto out;
}
+ ret = -EINVAL;
pn = nf_ct_l4proto_net(net, l4proto);
if (pn == NULL)
goto out;
--
1.8.3.1
nf_ct_l4proto_net() may return NULL.
That may happens if some module forget to set both l4proto->get_net_proto
and l4proto->net_id.
We'd check the return value here, in case crash happens.
Signed-off-by: Yafang Shao <[email protected]>
---
net/netfilter/nf_conntrack_proto.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c
index 154e8c0..316fef3 100644
--- a/net/netfilter/nf_conntrack_proto.c
+++ b/net/netfilter/nf_conntrack_proto.c
@@ -946,6 +946,9 @@ int nf_conntrack_proto_pernet_init(struct net *net)
struct nf_proto_net *pn = nf_ct_l4proto_net(net,
&nf_conntrack_l4proto_generic);
+ if (pn == NULL)
+ return -EINVAL;
+
err = nf_conntrack_l4proto_generic.init_net(net);
if (err < 0)
return err;
--
1.8.3.1
On Wed, Dec 05, 2018 at 08:56:29PM +0800, Yafang Shao wrote:
> nf_ct_l4proto_net() may return NULL.
> That may happens if some module forget to set both l4proto->get_net_proto
> and l4proto->net_id.
> We'd check the return value here, in case crash happens.
>
> Signed-off-by: Yafang Shao <[email protected]>
> ---
> net/netfilter/nf_conntrack_proto.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c
> index 154e8c0..316fef3 100644
> --- a/net/netfilter/nf_conntrack_proto.c
> +++ b/net/netfilter/nf_conntrack_proto.c
> @@ -946,6 +946,9 @@ int nf_conntrack_proto_pernet_init(struct net *net)
> struct nf_proto_net *pn = nf_ct_l4proto_net(net,
> &nf_conntrack_l4proto_generic);
There is another spot missing in this file that is missing the check
for NULL.
We can probably simplify all this is we place the gre conntracker in
the conntrack core, as it's been suggested already. It's the only one
remaining as a module and it now supports for IPv6, so it's probably
better follow that path.
On Thu, Dec 13, 2018 at 7:26 AM Pablo Neira Ayuso <[email protected]> wrote:
>
> On Wed, Dec 05, 2018 at 08:56:29PM +0800, Yafang Shao wrote:
> > nf_ct_l4proto_net() may return NULL.
> > That may happens if some module forget to set both l4proto->get_net_proto
> > and l4proto->net_id.
> > We'd check the return value here, in case crash happens.
> >
> > Signed-off-by: Yafang Shao <[email protected]>
> > ---
> > net/netfilter/nf_conntrack_proto.c | 3 +++
> > 1 file changed, 3 insertions(+)
> >
> > diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c
> > index 154e8c0..316fef3 100644
> > --- a/net/netfilter/nf_conntrack_proto.c
> > +++ b/net/netfilter/nf_conntrack_proto.c
> > @@ -946,6 +946,9 @@ int nf_conntrack_proto_pernet_init(struct net *net)
> > struct nf_proto_net *pn = nf_ct_l4proto_net(net,
> > &nf_conntrack_l4proto_generic);
>
> There is another spot missing in this file that is missing the check
> for NULL.
>
> We can probably simplify all this is we place the gre conntracker in
> the conntrack core, as it's been suggested already. It's the only one
> remaining as a module and it now supports for IPv6, so it's probably
> better follow that path.
Got it.
Thanks for your explanation.
Thanks
Yafang