Hi,
I have a linux box which acts as gateway and b/w manager for our wireless
network. I have 3 Interfaces ( one not is use yet). I have created 3 aliases
on the interface connected to the wireless (with 3 different class C
address).
I have around 350-400 wireless clients with over 10Mbps traffic at peak
hours.
These days I have observed some abnormal behavior.
System:
Dell P4 Poweredge
Red Hat 8.0 with 2.4.25 kernel
Lan Cards:
1. Intel 82540EM Gigabit Ethernet Controller (Intenet)
2. 3com 3c905B Cyclone (Wireless)
The problem is:
* Everything works fine for about 5-10 mins then all of a sudden TCP services
are not accessable.
* For some reason TCP times out. However at the same time ping,traceroute and
dns trace works without any problem.
* The connected TCP sokets keeps working without any problem. I verified this
by using Msn chat. I observerd that I chat session ( which I had started
when everything was normal) continued without any problem however I was not
able to initiate a new chat session.
Result of tcpdum on eth0 (internet) and eth2(wireless)
* I can see tcp SYN request coming in from eth2 (from client) and going out
via eth0 to destination.
* I can see tcp SYN+ACK coming in from eth0. But it doesn't get out through
eth2 !!! However other udp,icmp pkts can.
src 69.56.37.146 , dst 64.236.16.52
this is tcpdump on eth0 (internet side)
17:14:32.003370 69.56.37.146.3170 > 64.236.16.52.http: S
2446361626:2446361626(0) win 64240 <mss 1460,nop,nop,sackOK> (DF)
17:14:32.004694 64.236.16.52.http > 69.56.37.146.3170: S
3641974278:3641974278(0) ack 2446361627 win 65535 <mss 1460> (DF)
17:14:34.920578 69.56.37.146.3170 > 64.236.16.52.http: S
2446361626:2446361626(0) win 64240 <mss 1460,nop,nop,sackOK> (DF)
17:14:34.921800 64.236.16.52.http > 69.56.37.146.3170: S
3641974278:3641974278(0) ack 2446361627 win 65535 <mss 1460> (DF)
17:14:37.915564 64.236.16.52.http > 69.56.37.146.3170: S
3641974278:3641974278(0) ack 2446361627 win 65535 <mss 1460> (DF)
17:14:40.955491 69.56.37.146.3170 > 64.236.16.52.http: S
2446361626:2446361626(0) win 64240 <mss 1460,nop,nop,sackOK> (DF)
17:14:40.957682 64.236.16.52.http > 69.56.37.146.3170: S
3641974278:3641974278(0) ack 2446361627 win 65535 <mss 1460> (DF)
this is tcpdump on eth2 (client side)
17:14:32.003179 69.56.37.146.3170 > 64.236.16.52.http: S
2446361626:2446361626(0) win 64240 <mss 1460,nop,nop,sackOK> (DF)
17:14:34.920212 69.56.37.146.3170 > 64.236.16.52.http: S
2446361626:2446361626(0) win 64240 <mss 1460,nop,nop,sackOK> (DF)
17:14:40.955257 69.56.37.146.3170 > 64.236.16.52.http: S
2446361626:2446361626(0) win 64240 <mss 1460,nop,nop,sackOK> (DF)
As you can see i am getting the ACK from source from seq. 2446361626 (with
ack 2446361627 ) on eth0. But it is not going out via eth2!!
After sometime everthing starts working normally again. So what may be the
problem? I have problem only with the tcp protocol and not with other. I
haven't changed any tcp memory related settings.
net.ipv4.tcp_rmem = 4096 87380 174760
net.ipv4.tcp_wmem = 4096 87380 174760
net.ipv4.tcp_mem = 195584 196096 196608
net.core.optmem_max = 10240
net.core.rmem_default = 65535
net.core.wmem_default = 65535
net.core.rmem_max = 131071
net.core.wmem_max = 131071
Has anyone come across this probelm? any suggestion?
Thanks
Rohit
> * Everything works fine for about 5-10 mins then all of a sudden TCP services
> are not accessable.
> * For some reason TCP times out. However at the same time ping,traceroute and
> dns trace works without any problem.
> * The connected TCP sokets keeps working without any problem. I verified this
> by using Msn chat. I observerd that I chat session ( which I had started
> when everything was normal) continued without any problem however I was not
> able to initiate a new chat session.
Are you using session tracking. The symptoms you describe are
classically those of session tracking nat/firewalling/whatever running
out of table entries and being unable to allow new connections.
On Thu, 02 Sep 2004 11:56:59 +0100, Alan Cox <[email protected]> wrote:
> > * Everything works fine for about 5-10 mins then all of a sudden TCP services
> > are not accessable.
> > * For some reason TCP times out. However at the same time ping,traceroute and
> > dns trace works without any problem.
> > * The connected TCP sokets keeps working without any problem. I verified this
> > by using Msn chat. I observerd that I chat session ( which I had started
> > when everything was normal) continued without any problem however I was not
> > able to initiate a new chat session.
>
> Are you using session tracking. The symptoms you describe are
> classically those of session tracking nat/firewalling/whatever running
> out of table entries and being unable to allow new connections.
>
No, it is not running any session tracking (ip_conntrack) neither it
does nat. It is just a firewall with around 1600 rules in FORWARD
mangle table and around 1500 rules in FORWARD filter table. Out of
1500 rules , 1377 rules are MAC filter rules.
And it had 3 alias address for the interface conneted to the wirelss.
regards,
Rohit
>
On Thu, 2 Sep 2004 23:33, Rohit Neupane wrote:
> On Thu, 02 Sep 2004 11:56:59 +0100, Alan Cox <[email protected]>
wrote:
> > > * Everything works fine for about 5-10 mins then all of a sudden TCP
> > > services are not accessable.
> > > * For some reason TCP times out. However at the same time
> > > ping,traceroute and dns trace works without any problem.
> > > * The connected TCP sokets keeps working without any problem. I
> > > verified this by using Msn chat. I observerd that I chat session (
> > > which I had started when everything was normal) continued without any
> > > problem however I was not able to initiate a new chat session.
> >
> > Are you using session tracking. The symptoms you describe are
> > classically those of session tracking nat/firewalling/whatever running
> > out of table entries and being unable to allow new connections.
>
> No, it is not running any session tracking (ip_conntrack) neither it
> does nat. It is just a firewall with around 1600 rules in FORWARD
> mangle table and around 1500 rules in FORWARD filter table. Out of
> 1500 rules , 1377 rules are MAC filter rules.
> And it had 3 alias address for the interface conneted to the wirelss.
EEEEK! 1600? That is insane!
Consider cutting your rules into sections, and jumping to other tables to do
sections of the work. Perhaps you can filter on the start of the MAC address
and break this into smaller sections?
Also of note: MAC addresses are easily spoofed, so if you're using this to
lock out people on wireless, forget it, it doesn't work. Get them to use
tunnels (eg: ipsec) instead. The only real way MAC addresses even sort of
work is when you're providing a hotspot, ie: where you can't guarantee the
client to have anything apart from base wireless, and you should therefore
keep a tight leash on users connections by either timing them out regularly,
or making them keep open a https:// page to a login/AAA server (ie: a page
that auto-refreshes - when they stop refreshing the page, consider their
connection stale).
--
Stuart Young (aka Cef)
[email protected] is for LKML and related email only
On Thu, 02 Sep 2004 08:33:30 -0600, [email protected]
<[email protected]> wrote:
> Are you sure you don't have any rules that have "state" listed in them?
> It really does sound like you have a stateful firewall. But in the case
> that you don't, try logging our TCP rules that don't appear to be working.
Yes i am sure I don't have any state in the firewall. Infact we were
aware of the state problem from the beginning hence we didn't include
in the firewall at all.
And the problem is not just with one IP. It happens with almost all
client but ramdomly.
If there is any test that I need to carry out then please suggest.
Also, please not that I am running HTB and SFQ for b/w shaping purpose
on the same machine.
Rohit
>
> Steve
>
>
>
> Rohit Neupane wrote:
>
> >On Thu, 02 Sep 2004 11:56:59 +0100, Alan Cox <[email protected]> wrote:
> >
> >
> >>>* Everything works fine for about 5-10 mins then all of a sudden TCP services
> >>>are not accessable.
> >>>* For some reason TCP times out. However at the same time ping,traceroute and
> >>>dns trace works without any problem.
> >>>* The connected TCP sokets keeps working without any problem. I verified this
> >>>by using Msn chat. I observerd that I chat session ( which I had started
> >>>when everything was normal) continued without any problem however I was not
> >>>able to initiate a new chat session.
> >>>
> >>>
> >>Are you using session tracking. The symptoms you describe are
> >>classically those of session tracking nat/firewalling/whatever running
> >>out of table entries and being unable to allow new connections.
> >>
> >>
> >>
> >No, it is not running any session tracking (ip_conntrack) neither it
> >does nat. It is just a firewall with around 1600 rules in FORWARD
> >mangle table and around 1500 rules in FORWARD filter table. Out of
> >1500 rules , 1377 rules are MAC filter rules.
> >And it had 3 alias address for the interface conneted to the wirelss.
> >
> >regards,
> >Rohit
> >
> >
> >
> >
> >-
> >To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> >the body of a message to [email protected]
> >More majordomo info at http://vger.kernel.org/majordomo-info.html
> >Please read the FAQ at http://www.tux.org/lkml/
> >
> >
> >
> >
>
>
On Thu, 02 Sep 2004 17:38:07 -0600, Dax Kelson <[email protected]> wrote:
> Can you run this command:
>
> cat /proc/net/ip_conntrack | wc -l
>
> And email me the results?
I don't have the file. I don't have any rule that has "state" in it.
These are the modules under ip-tables
ip_tables 16800 7 [ipt_mark ipt_MARK iptable_mangle
ipt_limit ipt_mac ipt_LOG iptable_filter]
regards,
Rohit
>
>
> > > Are you using session tracking. The symptoms you describe are
> > > classically those of session tracking nat/firewalling/whatever running
> > > out of table entries and being unable to allow new connections.
> >
> > No, it is not running any session tracking (ip_conntrack) neither it
> > does nat. It is just a firewall with around 1600 rules in FORWARD
> > mangle table and around 1500 rules in FORWARD filter table. Out of
> > 1500 rules , 1377 rules are MAC filter rules.
> > And it had 3 alias address for the interface conneted to the wirelss.
>
> EEEEK! 1600? That is insane!
>
> Consider cutting your rules into sections, and jumping to other tables to
> do sections of the work. Perhaps you can filter on the start of the MAC
> address and break this into smaller sections?
>
> Also of note: MAC addresses are easily spoofed, so if you're using this to
> lock out people on wireless, forget it, it doesn't work. Get them to use
Yes. I saw MAC filtering being hacked by a teenager with WinXP.
I inflicted OpenVPN on him. Now hack *that*, boy... ;)
> tunnels (eg: ipsec) instead. The only real way MAC addresses even sort of
> work is when you're providing a hotspot, ie: where you can't guarantee the
> client to have anything apart from base wireless, and you should therefore
> keep a tight leash on users connections by either timing them out
> regularly, or making them keep open a https:// page to a login/AAA server
> (ie: a page that auto-refreshes - when they stop refreshing the page,
> consider their connection stale).
--
vda
Rohit Neupane wrote:
> No, it is not running any session tracking (ip_conntrack) neither it
> does nat. It is just a firewall with around 1600 rules in FORWARD
> mangle table and around 1500 rules in FORWARD filter table. Out of
> 1500 rules , 1377 rules are MAC filter rules.
> And it had 3 alias address for the interface conneted to the wirelss.
Ouch. Thats a lot of rules to traverse for each packet. Segment them
into chains if possible. Also you may want to take a look at nf-hipac,
http://www.hipac.org
--
Andr? Tomt