2005-03-11 07:23:38

by Dmitry Torokhov

[permalink] [raw]
Subject: Last night Linus bk - netfilter busted?

Hi,

My box gets stuck while booting (actually starting ntpd) whith tonight
pull from Linus. It looks like it is spinning in ipt_do_table when I do
SysRq-P. No call trace though.

Anyone else seeing it? Any ideas?

--
Dmitry


2005-03-11 09:09:57

by Steven Rostedt

[permalink] [raw]
Subject: [Announce] Stream line modules from .config with streamline_config.pl


Hi all,

I'm not sure if someone else did this, but I wrote this simple script that
turns off the modules from your config file that you are not currently
using. After downloading a new kernel I usually use the .config from my
Debian distribution. But this usually has way too many modules turned on
and my compile takes much longer than I prefer. So instead of searching
for all the modules that I need in "make *config" I wrote this script to
do it for me.

It's a small simple perl script (that's why it's attached and not on some
web site), and it can easily have options added to it, but it's good
enough for me, so that's where I left it.

Here's how it works and what it does.

1. Boot up the kernel that has the modules you want to load.
2. cd to the directory that holds the source of the kernel you
just booted.
3. If it's not already there, copy the config file you want to start
with to this directory, as .config.
4. Make sure all the modules you want/need are loaded.
5. run this script redirecting the output to another file.
6. copy this other file to .config (backing up your old one if you want).
7. Run "make oldconfig".

What this script does is simply, reads all the makefiles in the current
directory tree (find . -name Makefile). Searches each of these Makefiles
for the string "obj-$(CONFIG.*) += ..." (the real regular expression is
more complex, and the file is attached if you want to see it). It then
stores the object files associated to the CONFIG_.* and it handles
multiple lines that end with "\".

Then it runs "lsmod" to get what modules are loaded.

Finally it reads the .config file in the directory and prints it to
standard output. When it finds a CONFIG_.*=m it checks to see if that
config had an object from lsmod associated to it, if so, then it prints it
as is, otherwise it turns it off.

Here's what I did with my Debian distribution:

cd /usr/src/linux-2.6.10
cp /boot/config-2.6.10-1-686-smp .config
~/bin/streamline_config > config_strip
mv .config config_sav
mv config_strip .config
make oldconfig

Now this is the config file that I start with when downloading other
kernels.

Obviously if you don't load all the modules you want, or later buy a
new device that needs a module that wasn't loaded, you will need to figure
out what module to add and compile it. Or use the saved config again
(you did save it?) and do this all over.

Well, do what you want with this, it's an unrescricted license. Comments?

If someone else had already done something like this, let me know. But I
wrote this and a colleage of mine suggested to send it here. So here it
is.

Cheers,

-- Steve


Attachments:
streamline_config.pl (2.91 kB)

2005-03-11 14:01:06

by Patrick McHardy

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

Dmitry Torokhov wrote:
> My box gets stuck while booting (actually starting ntpd) whith tonight
> pull from Linus. It looks like it is spinning in ipt_do_table when I do
> SysRq-P. No call trace though.

Please post your ruleset and .config. A backtrace would also be
useful.

> Anyone else seeing it? Any ideas?

Works fine here. You could try if reverting one of these two patches
helps (second one only if its a SMP box).

[email protected], 2005-03-09 20:28:17-08:00, [email protected]
[NETFILTER]: Reduce call chain length in netfilter (take 2)

[email protected], 2005-03-03 23:15:48+01:00, [email protected]
[NETFILTER]: Reduce netfilter memory use on MP systems

Regards
Patrick

2005-03-11 19:06:42

by David Miller

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

On Fri, 11 Mar 2005 15:00:56 +0100
Patrick McHardy <[email protected]> wrote:

> Works fine here. You could try if reverting one of these two patches
> helps (second one only if its a SMP box).
>
> [email protected], 2005-03-09 20:28:17-08:00, [email protected]
> [NETFILTER]: Reduce call chain length in netfilter (take 2)

It's this change, I know it is, because Linus sees the same problem
on his workstation.

You wouldn't happen to be seeing this problem on a PPC box would
you? Since Linus's machine is a PPC machine too, that would support
my theory that this could be a compiler issue on that platform.

Damn, wait, Patrick, I think I know what's happening. The iptables
IPT_* verdicts are dependant upon the NF_* values, and they don't
cope with Bart's changes I bet. Can you figure out what the exact
error would be? This kind of issue would explain the looping inside
of ipt_do_table(), wouldn't it?

2005-03-11 19:10:46

by Dmitry Torokhov

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

On Friday 11 March 2005 13:51, David S. Miller wrote:
> On Fri, 11 Mar 2005 15:00:56 +0100
> Patrick McHardy <[email protected]> wrote:
>
> > Works fine here. You could try if reverting one of these two patches
> > helps (second one only if its a SMP box).
> >
> > [email protected], 2005-03-09 20:28:17-08:00, [email protected]
> > [NETFILTER]: Reduce call chain length in netfilter (take 2)
>
> It's this change, I know it is, because Linus sees the same problem
> on his workstation.
>
> You wouldn't happen to be seeing this problem on a PPC box would
> you? Since Linus's machine is a PPC machine too, that would support
> my theory that this could be a compiler issue on that platform.
>

No, it is regular PIII laptop (preempt, UP).

> Damn, wait, Patrick, I think I know what's happening. The iptables
> IPT_* verdicts are dependant upon the NF_* values, and they don't
> cope with Bart's changes I bet. Can you figure out what the exact
> error would be? This kind of issue would explain the looping inside
> of ipt_do_table(), wouldn't it?
>

--
Dmitry

2005-03-11 19:31:48

by Sergey Vlasov

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

On Fri, 11 Mar 2005 10:51:36 -0800 David S. Miller wrote:

> On Fri, 11 Mar 2005 15:00:56 +0100
> Patrick McHardy <[email protected]> wrote:
>
> > Works fine here. You could try if reverting one of these two patches
> > helps (second one only if its a SMP box).
> >
> > [email protected], 2005-03-09 20:28:17-08:00, [email protected]
> > [NETFILTER]: Reduce call chain length in netfilter (take 2)
>
> It's this change, I know it is, because Linus sees the same problem
> on his workstation.
>
> You wouldn't happen to be seeing this problem on a PPC box would
> you? Since Linus's machine is a PPC machine too, that would support
> my theory that this could be a compiler issue on that platform.
>
> Damn, wait, Patrick, I think I know what's happening. The iptables
> IPT_* verdicts are dependant upon the NF_* values, and they don't
> cope with Bart's changes I bet. Can you figure out what the exact
> error would be? This kind of issue would explain the looping inside
> of ipt_do_table(), wouldn't it?

This is not just some buggy code - that patch also breaks interfaces:

include/linux/netfilter_ipv4/ip_tables.h:
#define IPT_RETURN (-NF_MAX_VERDICT - 1)

And this value is visible in userspace. Therefore we cannot modify
NF_MAX_VERDICT without breaking all existing iptables binaries.


Attachments:
(No filename) (1.28 kB)
(No filename) (189.00 B)
Download all attachments

2005-03-11 20:54:33

by Patrick McHardy

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2005/03/11 21:41:01+01:00 [email protected]
# [NETFILTER]: Fix iptables userspace compatibility breakage
#
# Signed-off-by: Patrick McHardy <[email protected]>
#
# include/linux/netfilter_ipv6/ip6_tables.h
# 2005/03/11 21:40:52+01:00 [email protected] +1 -1
# [NETFILTER]: Fix iptables userspace compatibility breakage
#
# Signed-off-by: Patrick McHardy <[email protected]>
#
# include/linux/netfilter_ipv4/ip_tables.h
# 2005/03/11 21:40:52+01:00 [email protected] +1 -1
# [NETFILTER]: Fix iptables userspace compatibility breakage
#
# Signed-off-by: Patrick McHardy <[email protected]>
#
diff -Nru a/include/linux/netfilter_ipv4/ip_tables.h b/include/linux/netfilter_ipv4/ip_tables.h
--- a/include/linux/netfilter_ipv4/ip_tables.h 2005-03-11 21:41:32 +01:00
+++ b/include/linux/netfilter_ipv4/ip_tables.h 2005-03-11 21:41:32 +01:00
@@ -166,7 +166,7 @@
#define IPT_CONTINUE 0xFFFFFFFF

/* For standard target */
-#define IPT_RETURN (-NF_MAX_VERDICT - 1)
+#define IPT_RETURN (-NF_REPEAT - 1)

/* TCP matching stuff */
struct ipt_tcp
diff -Nru a/include/linux/netfilter_ipv6/ip6_tables.h b/include/linux/netfilter_ipv6/ip6_tables.h
--- a/include/linux/netfilter_ipv6/ip6_tables.h 2005-03-11 21:41:32 +01:00
+++ b/include/linux/netfilter_ipv6/ip6_tables.h 2005-03-11 21:41:32 +01:00
@@ -166,7 +166,7 @@
#define IP6T_CONTINUE 0xFFFFFFFF

/* For standard target */
-#define IP6T_RETURN (-NF_MAX_VERDICT - 1)
+#define IP6T_RETURN (-NF_REPEAT - 1)

/* TCP matching stuff */
struct ip6t_tcp


Attachments:
x (1.56 kB)

2005-03-11 21:24:29

by Herbert Xu

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

Patrick McHardy <[email protected]> wrote:
>
> You're right, good catch. IPT_RETURN is interpreted internally by
> ip_tables, but since the value changed it isn't recognized by ip_tables
> anymore and returned to nf_iterate() as NF_REPEAT. This patch restores
> the old value.

Please fix netfilter_arp while you're at it since it does exactly
the same thing.

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2005-03-11 23:20:24

by Patrick McHardy

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2005/03/11 23:54:54+01:00 [email protected]
# [NETFILTER]: Fix iptables userspace compatibility breakage
#
# Signed-off-by: Patrick McHardy <[email protected]>
#
# include/linux/netfilter_ipv6/ip6_tables.h
# 2005/03/11 23:54:44+01:00 [email protected] +1 -1
# [NETFILTER]: Fix iptables userspace compatibility breakage
#
# Signed-off-by: Patrick McHardy <[email protected]>
#
# include/linux/netfilter_ipv4/ip_tables.h
# 2005/03/11 23:54:44+01:00 [email protected] +1 -1
# [NETFILTER]: Fix iptables userspace compatibility breakage
#
# Signed-off-by: Patrick McHardy <[email protected]>
#
# include/linux/netfilter_arp/arp_tables.h
# 2005/03/11 23:54:44+01:00 [email protected] +1 -1
# [NETFILTER]: Fix iptables userspace compatibility breakage
#
# Signed-off-by: Patrick McHardy <[email protected]>
#
diff -Nru a/include/linux/netfilter_arp/arp_tables.h b/include/linux/netfilter_arp/arp_tables.h
--- a/include/linux/netfilter_arp/arp_tables.h 2005-03-11 23:55:09 +01:00
+++ b/include/linux/netfilter_arp/arp_tables.h 2005-03-11 23:55:09 +01:00
@@ -154,7 +154,7 @@
#define ARPT_CONTINUE 0xFFFFFFFF

/* For standard target */
-#define ARPT_RETURN (-NF_MAX_VERDICT - 1)
+#define ARPT_RETURN (-NF_REPEAT - 1)

/* The argument to ARPT_SO_GET_INFO */
struct arpt_getinfo
diff -Nru a/include/linux/netfilter_ipv4/ip_tables.h b/include/linux/netfilter_ipv4/ip_tables.h
--- a/include/linux/netfilter_ipv4/ip_tables.h 2005-03-11 23:55:09 +01:00
+++ b/include/linux/netfilter_ipv4/ip_tables.h 2005-03-11 23:55:09 +01:00
@@ -166,7 +166,7 @@
#define IPT_CONTINUE 0xFFFFFFFF

/* For standard target */
-#define IPT_RETURN (-NF_MAX_VERDICT - 1)
+#define IPT_RETURN (-NF_REPEAT - 1)

/* TCP matching stuff */
struct ipt_tcp
diff -Nru a/include/linux/netfilter_ipv6/ip6_tables.h b/include/linux/netfilter_ipv6/ip6_tables.h
--- a/include/linux/netfilter_ipv6/ip6_tables.h 2005-03-11 23:55:09 +01:00
+++ b/include/linux/netfilter_ipv6/ip6_tables.h 2005-03-11 23:55:09 +01:00
@@ -166,7 +166,7 @@
#define IP6T_CONTINUE 0xFFFFFFFF

/* For standard target */
-#define IP6T_RETURN (-NF_MAX_VERDICT - 1)
+#define IP6T_RETURN (-NF_REPEAT - 1)

/* TCP matching stuff */
struct ip6t_tcp


Attachments:
x (2.23 kB)

2005-03-13 06:54:13

by Dmitry Torokhov

[permalink] [raw]
Subject: Re: Last night Linus bk - netfilter busted?

On Friday 11 March 2005 17:55, Patrick McHardy wrote:
> Herbert Xu wrote:
> > Patrick McHardy <[email protected]> wrote:
> >
> >>You're right, good catch. IPT_RETURN is interpreted internally by
> >>ip_tables, but since the value changed it isn't recognized by ip_tables
> >>anymore and returned to nf_iterate() as NF_REPEAT. This patch restores
> >>the old value.
> >
> >
> > Please fix netfilter_arp while you're at it since it does exactly
> > the same thing.
>
> New patch attached, thanks.
>

If this is of any interest, yesterday's pull from Linux plus this patch
seem to be working fine here.

Thank you.

--
Dmitry