2004-06-29 13:20:24

by Debi Janos

[permalink] [raw]
Subject: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

I am found an interesting (bug?) feature in kernels between
2.6.7-mm1 and 2.6.7-mm4

Some web pages eg.

http://www.hup.hu/
http://portal.fsn.hu/
http://wiki.hup.hu/

is unreachable with these kernels. If i try kernel versions
<= 2.6.7 everything is O.K. above-mentioned all web pages works.

I try this web pages with some different operating systems
like Windows, OpenBSD, FreeBSD, WinCe... and seems working
fine for me.

Any idea?

Thanks.


2004-06-29 15:45:53

by bert hubert

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, Jun 29, 2004 at 03:20:06PM +0200, Debi Janos wrote:
> I am found an interesting (bug?) feature in kernels between
> 2.6.7-mm1 and 2.6.7-mm4

Without further looking at it, try checking if ECN is turned on.
/proc/sys/net/ipv4/tcp_ecn if memory serves me well.

--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO

2004-06-29 16:57:12

by Debi Janos

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

bert hubert <[email protected]> ?rta:

> On Tue, Jun 29, 2004 at 03:20:06PM +0200, Debi Janos wrote:
> > I am found an interesting (bug?) feature in kernels between
> > 2.6.7-mm1 and 2.6.7-mm4
>
> Without further looking at it, try checking if ECN is
turned on.
> /proc/sys/net/ipv4/tcp_ecn if memory serves me well.

Hi!

Thanks. I have tried, but still have problem.

Another site which not working:

http://packages.gentoo.org/

2004-06-29 17:28:17

by bert hubert

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, Jun 29, 2004 at 06:57:06PM +0200, Debi Janos wrote:

> http://packages.gentoo.org/

Run

tcpdump -i any -s 0 -w workingkernel host 198.63.211.232
and
tcpdump -i any -s 0 -w brokenkernel host 198.63.211.232

While trying to connect with both a workingkernel and a brokenkernel.

Then attach both these files in a message, either to me or to the list.

This will allow us to see what is going on.

--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO

2004-06-29 17:28:44

by Jesse Stockall

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 2004-06-29 at 09:20, Debi Janos wrote:
> I am found an interesting (bug?) feature in kernels between
> 2.6.7-mm1 and 2.6.7-mm4
>
> Some web pages eg.
>
> http://www.hup.hu/
> http://portal.fsn.hu/
> http://wiki.hup.hu/
>
> is unreachable with these kernels. If i try kernel versions
> <= 2.6.7 everything is O.K. above-mentioned all web pages works.
>

I'm seeing the same thing after upgrading to 2.6.7-mm4, 2.6.7 vanilla
does not exhibit the problem.

Also Azureus (Java Bittorrent client) crashes when it tries to open a
network socket to begin downloading. Again problem does not exist with
2.6.7.

I'm going to try a few other kernel versions to see if I can find when
this started.

Jesse

--
Jesse Stockall <[email protected]>

2004-06-29 17:39:19

by FabF

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 2004-06-29 at 19:27, Jesse Stockall wrote:
> On Tue, 2004-06-29 at 09:20, Debi Janos wrote:
> > I am found an interesting (bug?) feature in kernels between
> > 2.6.7-mm1 and 2.6.7-mm4
> >
> > Some web pages eg.
> >
> > http://www.hup.hu/
> > http://portal.fsn.hu/
> > http://wiki.hup.hu/
> >
> > is unreachable with these kernels. If i try kernel versions
> > <= 2.6.7 everything is O.K. above-mentioned all web pages works.
> >
>
> I'm seeing the same thing after upgrading to 2.6.7-mm4, 2.6.7 vanilla
> does not exhibit the problem.
>
> Also Azureus (Java Bittorrent client) crashes when it tries to open a
> network socket to begin downloading. Again problem does not exist with
> 2.6.7.
>
> I'm going to try a few other kernel versions to see if I can find when
> this started.
Same behaviour.

Here's an ipv4 diff (2.6.6 FE2 vs 2.6.7mm3)

ip_local_port_range|32768 61000
ip_nonlocal_bind|0
ip_no_pmtu_disc|0
+ip_queue_maxlen|1024
tcp_abort_on_overflow|0
tcp_adv_win_scale|2
tcp_app_win|31
-tcp_bic|0
+tcp_bic|1
tcp_bic_fast_convergence|1
tcp_bic_low_window|14
+tcp_default_win_scale|7
tcp_dsack|1
tcp_ecn|0
tcp_fack|1
tcp_max_orphans|8192
tcp_max_syn_backlog|1024
tcp_max_tw_buckets|180000
-tcp_mem|48128 48640 49152
+tcp_mem|24576 32768 49152
+tcp_moderate_rcvbuf|1
tcp_no_metrics_save|0
tcp_orphan_retries|0
tcp_reordering|3
tcp_sack|1
tcp_stdurg|0
tcp_synack_retries|5
-tcp_syncookies|0
tcp_syn_retries|5
tcp_timestamps|1
tcp_tw_recycle|0

Regards,
FabF

>
> Jesse

2004-06-29 18:23:06

by Stephen Hemminger

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 29 Jun 2004 15:20:06 +0200 (CEST)
Debi Janos <[email protected]> wrote:

> I am found an interesting (bug?) feature in kernels between
> 2.6.7-mm1 and 2.6.7-mm4
>
> Some web pages eg.
>
> http://www.hup.hu/
> http://portal.fsn.hu/
> http://wiki.hup.hu/
>
> is unreachable with these kernels. If i try kernel versions
> <= 2.6.7 everything is O.K. above-mentioned all web pages works.
>
> I try this web pages with some different operating systems
> like Windows, OpenBSD, FreeBSD, WinCe... and seems working
> fine for me.
>
> Any idea?

Dave enabled the receive buffer auto-tuning which uses TCP window
scaling. It looks like all these sites are running FreeBSD, perhaps
there is a bug in FreeBSD?

As suggested earlier please get a TCP dump of a failed connection.

To turn of receive buffer auto-tuning:
sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
sysctl -w net.ipv4.tcp_default_win_scale=0

Thanks.

2004-06-29 18:39:06

by bert hubert

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, Jun 29, 2004 at 08:04:46PM +0200, Debi Janos wrote:
> bert hubert <[email protected]> ?rta:
> .
> >
> > Then attach both these files in a message, either to me or
> to the list.
> >
> > This will allow us to see what is going on.
>
>
> dumps attached. thanks.

There is almost no difference in the trace, and it is very puzzling. Your
attachments will have been bounced by vger, so I suggest putting these
traces online somewhere.

The weird thing is that the packages.gentoo.org appears to be at fault. It
ignores your valid FIN packets, or at least, they appear to be valid to
ethereal.

There are three differences between these traces:
1) The IP address of your computer
2) In the non-working trace, your Mozilla does not pass a
'Cache-Control' HTTP header
3) The broken trace has a timestamp wraparound on your side

Of note is that both gentoo and you set a non-standard MSS (I think).

Suggestions:
1) turn off timestamps (echo 0 > /proc/sys/net/ipv4/tcp_timestamps)
2) set your MTU to 1000 or so (ifconfig eth0 mtu 1000)

And try again.

Interesting case!

--
http://www.PowerDNS.com Open source, database driven DNS Software
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO

2004-06-29 18:41:07

by Debi Janos

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

Stephen Hemminger <[email protected]> ?rta:

> Dave enabled the receive buffer auto-tuning which uses TCP
window
> scaling. It looks like all these sites are running
FreeBSD, perhaps
> there is a bug in FreeBSD?

packages.gentoo.org running on FreeBSD? I don't believe that.

http://uptime.netcraft.com/up/graph?site=packages.gentoo.org

running on Gentoo Linux, Apache/2.0.4

My site (http://www.hup.hu) running on FreeBSD 4.7-RC with Apache
1.3.29

http://uptime.netcraft.com/up/graph?site=www.hup.hu

it was problem with both site

> As suggested earlier please get a TCP dump of a failed
connection.
>
> To turn of receive buffer auto-tuning:
> sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
> sysctl -w net.ipv4.tcp_default_win_scale=0
>
> Thanks.

Yes. This was the problem. With this settings working...
Thank you.

2004-06-29 18:49:18

by Debi Janos

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

bert hubert <[email protected]> ?rta:

> On Tue, Jun 29, 2004 at 08:04:46PM +0200, Debi Janos wrote:
> > bert hubert <[email protected]> ?rta:

> Suggestions:
> 1) turn off timestamps (echo 0 >
/proc/sys/net/ipv4/tcp_timestamps)
> 2) set your MTU to 1000 or so (ifconfig eth0 mtu 1000)
>
> And try again.
>
> Interesting case!

problem workarounded:

sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
sysctl -w net.ipv4.tcp_default_win_scale=0

Thanks.

2004-06-29 18:51:29

by David Miller

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 29 Jun 2004 11:22:56 -0700
Stephen Hemminger <[email protected]> wrote:

> To turn of receive buffer auto-tuning:
> sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
> sysctl -w net.ipv4.tcp_default_win_scale=0

It may be just the window scaling that's causing the grief
so people can try just setting tcp_default_win_scale to zero
and leaving tcp_moderate_rcvbuf enabled.

That would be an important data-point.

2004-06-29 19:08:03

by Debi Janos

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

David S. Miller <[email protected]> ?rta:

> On Tue, 29 Jun 2004 11:22:56 -0700
> Stephen Hemminger <[email protected]> wrote:
>
> > To turn of receive buffer auto-tuning:
> > sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
> > sysctl -w net.ipv4.tcp_default_win_scale=0
>
> It may be just the window scaling that's causing the grief
> so people can try just setting tcp_default_win_scale to zero
> and leaving tcp_moderate_rcvbuf enabled.
>
> That would be an important data-point.

Yes.

sysctl -w net.ipv4.tcp_default_win_scale=0

is enough.

(1.2GHz 55C) root@alderaan:/home/trey $ sysctl
net.ipv4.tcp_moderate_rcvbuf
net.ipv4.tcp_moderate_rcvbuf = 1
(1.2GHz 55C) root@alderaan:/home/trey $ sysctl
net.ipv4.tcp_default_win_scale
net.ipv4.tcp_default_win_scale = 0
(1.2GHz 54C) root@alderaan:/home/trey $

With this setting i have not problem, working everything is
fine.

2004-06-29 19:17:30

by Jesse Stockall

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 2004-06-29 at 14:22, Stephen Hemminger wrote:
>
> Dave enabled the receive buffer auto-tuning which uses TCP window
> scaling. It looks like all these sites are running FreeBSD, perhaps
> there is a bug in FreeBSD?
>
> As suggested earlier please get a TCP dump of a failed connection.
>
> To turn of receive buffer auto-tuning:
> sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
> sysctl -w net.ipv4.tcp_default_win_scale=0
>

This works around the inaccessible websites, packages.gentoo.org (not a
FreeBSD box) can now be reached.

This however does not resolve the java issue. The application dies when
trying to connect to receive data.

--
Jesse Stockall <[email protected]>

2004-06-29 19:50:09

by Stephen Hemminger

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

Here is the trace output for both cases run against 2.6.7 latest (not -mm)

failing case: packages.gentoo.org

12:37:04.654043 IP 172.20.1.73.32774 > 198.63.211.232.80: S 4165256157:4165256157(0) win 5840 <mss 1460,sackOK,timestamp 2448351 0,nop,wscale 7>
12:37:05.078029 IP 198.63.211.232.80 > 172.20.1.73.32774: S 3946717705:3946717705(0) ack 4165256158 win 5792 <mss 1460,sackOK,timestamp 678388344 2448351,nop,wscale 0>
12:37:05.078069 IP 172.20.1.73.32774 > 198.63.211.232.80: . ack 1 win 45 <nop,nop,timestamp 2448775 678388344>
12:37:05.079000 IP 172.20.1.73.32774 > 198.63.211.232.80: P 1:1340(1339) ack 1 win 45 <nop,nop,timestamp 2448776 678388344>
12:37:05.283359 IP 198.63.211.232.80 > 172.20.1.73.32774: . ack 1340 win 8034 <nop,nop,timestamp 678388392 2448776>

<hung>

12:37:07.960251 IP 172.20.1.73.32773 > 198.63.211.232.80: F 4127965887:4127965887(0) ack 3063226309 win 45 <nop,nop,timestamp 2451658 678381231>
12:37:12.921440 IP 172.20.1.73.32774 > 198.63.211.232.80: F 1340:1340(0) ack 1 win 45 <nop,nop,timestamp 2456620 678388392>
12:37:14.166886 IP 172.20.1.73.32774 > 198.63.211.232.80: F 1340:1340(0) ack 1 win 45 <nop,nop,timestamp 2457865 678388392>


And normal case: osdl.org

12:42:58.249167 IP 172.20.1.73.32777 > 65.172.181.4.80: S 231703769:231703769(0) win 5840 <mss 1460,sackOK,timestamp 2802000 0,nop,wscale 7>
12:42:58.249486 IP 65.172.181.4.80 > 172.20.1.73.32777: S 2426432811:2426432811(0) ack 231703770 win 5792 <mss 1460,sackOK,timestamp 2635537765 2802000,nop,wscale 0>
12:42:58.249511 IP 172.20.1.73.32777 > 65.172.181.4.80: . ack 1 win 45 <nop,nop,timestamp 2802000 2635537765>
12:42:58.251102 IP 172.20.1.73.32777 > 65.172.181.4.80: P 1:1329(1328) ack 1 win 45 <nop,nop,timestamp 2802002 2635537765>
12:42:58.252069 IP 65.172.181.4.80 > 172.20.1.73.32777: . ack 1329 win 7968 <nop,nop,timestamp 2635537768 2802002>
12:42:58.253020 IP 65.172.181.4.80 > 172.20.1.73.32777: P 1:273(272) ack 1329 win 7968 <nop,nop,timestamp 2635537768 2802002>
12:42:58.253037 IP 172.20.1.73.32777 > 65.172.181.4.80: . ack 273 win 45 <nop,nop,timestamp 2802004 2635537768>
12:42:58.253670 IP 65.172.181.4.80 > 172.20.1.73.32777: . 273:1721(1448) ack 1329 win 7968 <nop,nop,timestamp 2635537769 2802002>
12:42:58.253697 IP 172.20.1.73.32777 > 65.172.181.4.80: . ack 1721 win 45 <nop,nop,timestamp 2802004 2635537769>
12:42:58.253790 IP 65.172.181.4.80 > 172.20.1.73.32777: P 1721:3169(1448) ack 1329 win 7968 <nop,nop,timestamp 2635537769 2802002>
12:42:58.253799 IP 172.20.1.73.32777 > 65.172.181.4.80: . ack 3169 win 45 <nop,nop,timestamp 2802005 2635537769>
12:42:58.254041 IP 65.172.181.4.80 > 172.20.1.73.32777: . 3169:4617(1448) ack 1329 win 7968 <nop,nop,timestamp 2635537769 2802004>
<...>
12:42:58.258191 IP 65.172.181.4.80 > 172.20.1.73.32777: P 17649:19097(1448) ack 1329 win 7968 <nop,nop,timestamp 2635537773 2802008>
12:42:58.259106 IP 65.172.181.4.80 > 172.20.1.73.32777: . 19097:20545(1448) ack 1329 win 7968 <nop,nop,timestamp 2635537774 2802009>
12:42:58.259155 IP 172.20.1.73.32777 > 65.172.181.4.80: . ack 20545 win 45 <nop,nop,timestamp 2802010 2635537773>
12:42:58.259166 IP 65.172.181.4.80 > 172.20.1.73.32777: FP 20545:20689(144) ack 1329 win 7968 <nop,nop,timestamp 2635537774 2802009>
12:42:58.262096 IP 172.20.1.73.32777 > 65.172.181.4.80: F 1329:1329(0) ack 20690 win 45 <nop,nop,timestamp 2802013 2635537774>
12:42:58.262357 IP 65.172.181.4.80 > 172.20.1.73.32777: . ack 1330 win 7968 <nop,nop,timestamp 2635537778 2802013>

2004-06-29 19:54:31

by David Miller

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior


What's really amusing in those traces is that it is the sender that
is doing the window scaling, not the receiver. The side doing the
window interpretation for data packet sending is looking at a
non-scaled window.

Boggle...

2004-06-29 20:35:50

by Stephen Hemminger

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 29 Jun 2004 12:54:01 -0700
"David S. Miller" <[email protected]> wrote:

>
> What's really amusing in those traces is that it is the sender that
> is doing the window scaling, not the receiver. The side doing the
> window interpretation for data packet sending is looking at a
> non-scaled window.
>
> Boggle...

FYI - gentoo works for window scale 0..2 and appears to fail for >3.

Also, the socket ends up with:

State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 172.20.1.73:34452 198.63.211.232:http
ts sack wscale:0,3 rto:332 rtt:66.375/50.5 cwnd:3

2004-06-29 20:59:50

by David Miller

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 29 Jun 2004 13:35:01 -0700
Stephen Hemminger <[email protected]> wrote:

> FYI - gentoo works for window scale 0..2 and appears to fail for >3.
>
> Also, the socket ends up with:
>
> State Recv-Q Send-Q Local Address:Port Peer Address:Port
> ESTAB 0 0 172.20.1.73:34452 198.63.211.232:http
> ts sack wscale:0,3 rto:332 rtt:66.375/50.5 cwnd:3

Yes, I've seen this declared in other reports too.

It probably means just that for window scales of 0..2 the misinterpretation
does not result in a too-small-to-send-data window.

But I'm still confused that the scaled window is being given to the
receiver, and this makes the connection freeze. I wonder if there is
a queer box doing NAT or similar in front of the gentoo machine which
either:

1) Applies any window scaling to both directions
2) Applies window scaling to the wrong direction

and uses this to "help" with dropping of out-of-window TCP segments.

2004-06-29 21:02:57

by Stephen Hemminger

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 29 Jun 2004 13:59:22 -0700
"David S. Miller" <[email protected]> wrote:

> On Tue, 29 Jun 2004 13:35:01 -0700
> Stephen Hemminger <[email protected]> wrote:
>
> > FYI - gentoo works for window scale 0..2 and appears to fail for >3.
> >
> > Also, the socket ends up with:
> >
> > State Recv-Q Send-Q Local Address:Port Peer Address:Port
> > ESTAB 0 0 172.20.1.73:34452 198.63.211.232:http
> > ts sack wscale:0,3 rto:332 rtt:66.375/50.5 cwnd:3
>
> Yes, I've seen this declared in other reports too.
>
> It probably means just that for window scales of 0..2 the misinterpretation
> does not result in a too-small-to-send-data window.
>
> But I'm still confused that the scaled window is being given to the
> receiver, and this makes the connection freeze. I wonder if there is
> a queer box doing NAT or similar in front of the gentoo machine which
> either:
>
> 1) Applies any window scaling to both directions
> 2) Applies window scaling to the wrong direction
>
> and uses this to "help" with dropping of out-of-window TCP segments.

Unfortunately, this means the default probably means that window scale must be
disabled. An interesting experiment would be to see if other implementations have
the same problem with window scale enabled.

2004-06-29 21:37:50

by John Heffner

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 29 Jun 2004, Stephen Hemminger wrote:

> On Tue, 29 Jun 2004 13:59:22 -0700
> "David S. Miller" <[email protected]> wrote:
>
> > On Tue, 29 Jun 2004 13:35:01 -0700
> > Stephen Hemminger <[email protected]> wrote:
> >
> > > FYI - gentoo works for window scale 0..2 and appears to fail for >3.
> > >
> > > Also, the socket ends up with:
> > >
> > > State Recv-Q Send-Q Local Address:Port Peer Address:Port
> > > ESTAB 0 0 172.20.1.73:34452 198.63.211.232:http
> > > ts sack wscale:0,3 rto:332 rtt:66.375/50.5 cwnd:3
> >
> > Yes, I've seen this declared in other reports too.
> >
> > It probably means just that for window scales of 0..2 the misinterpretation
> > does not result in a too-small-to-send-data window.
> >
> > But I'm still confused that the scaled window is being given to the
> > receiver, and this makes the connection freeze. I wonder if there is
> > a queer box doing NAT or similar in front of the gentoo machine which
> > either:
> >
> > 1) Applies any window scaling to both directions
> > 2) Applies window scaling to the wrong direction
> >
> > and uses this to "help" with dropping of out-of-window TCP segments.
>
> Unfortunately, this means the default probably means that window scale must be
> disabled. An interesting experiment would be to see if other implementations have
> the same problem with window scale enabled.


Sigh. I ran in to this problem a year or so ago and it was a broken
firewall that was mangling the TCP window scale option. I think the
firewall was an OpenBSD machine, and I was told the problem went away with
an upgrade. I'm curious what they're running here.

The boundary 3 is special because it causes SWS avoidance to break.

-John

2004-06-29 21:53:50

by David Miller

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 29 Jun 2004 17:36:45 -0400 (EDT)
John Heffner <[email protected]> wrote:

> Sigh. I ran in to this problem a year or so ago and it was a broken
> firewall that was mangling the TCP window scale option. I think the
> firewall was an OpenBSD machine, and I was told the problem went away with
> an upgrade. I'm curious what they're running here.
>
> The boundary 3 is special because it causes SWS avoidance to break.

Interesting data-point, thanks John.

Can someone go figure out what packages.gentoo.org is using
as a firewall/router?

2004-06-29 22:01:40

by Martin Schlemmer

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Tue, 2004-06-29 at 23:53, David S. Miller wrote:
> On Tue, 29 Jun 2004 17:36:45 -0400 (EDT)
> John Heffner <[email protected]> wrote:
>
> > Sigh. I ran in to this problem a year or so ago and it was a broken
> > firewall that was mangling the TCP window scale option. I think the
> > firewall was an OpenBSD machine, and I was told the problem went away with
> > an upgrade. I'm curious what they're running here.
> >
> > The boundary 3 is special because it causes SWS avoidance to break.
>
> Interesting data-point, thanks John.
>
> Can someone go figure out what packages.gentoo.org is using
> as a firewall/router?

I forwarded one of the previous mails in the thread to Kurt Lieber
who should know what is the setup there, or who to speak to.


Cheers,


--
Martin Schlemmer


Attachments:
signature.asc (189.00 B)
This is a digitally signed message part

2004-06-29 22:39:17

by Kasper Sandberg

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

this is indeed a very strange problem.. i believe it goes longer back..
i remember when changing to 2.6.7-rc3, lkml.org didnt work.. (still dont)
and now i being to see the gentoo sites does not work, i begin to suspect
the kernel, also after seeing this thread...

i am not a expert, so i dont really know, but i didnt see anything
eyespotting that could do it in the code :|

andrew, maybe you know something? :D

> Stephen Hemminger <[email protected]> ?rta:
>
>> Dave enabled the receive buffer auto-tuning which uses TCP
> window
>> scaling. It looks like all these sites are running
> FreeBSD, perhaps
>> there is a bug in FreeBSD?
>
> packages.gentoo.org running on FreeBSD? I don't believe that.
>
> http://uptime.netcraft.com/up/graph?site=packages.gentoo.org
>
> running on Gentoo Linux, Apache/2.0.4
>
> My site (http://www.hup.hu) running on FreeBSD 4.7-RC with Apache
> 1.3.29
>
> http://uptime.netcraft.com/up/graph?site=www.hup.hu
>
> it was problem with both site
>
>> As suggested earlier please get a TCP dump of a failed
> connection.
>>
>> To turn of receive buffer auto-tuning:
>> sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
>> sysctl -w net.ipv4.tcp_default_win_scale=0
>>
>> Thanks.
>
> Yes. This was the problem. With this settings working...
> Thank you.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>


--


2004-06-29 22:47:16

by Redeeman

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior


> bert hubert <[email protected]> ?rta:
>
>> On Tue, Jun 29, 2004 at 08:04:46PM +0200, Debi Janos wrote:
>> > bert hubert <[email protected]> ?rta:
>
>> Suggestions:
>> 1) turn off timestamps (echo 0 >
> /proc/sys/net/ipv4/tcp_timestamps)
>> 2) set your MTU to 1000 or so (ifconfig eth0 mtu 1000)
>>
>> And try again.
>>
>> Interesting case!
>
> problem workarounded:
>
> sysctl -w net.ipv4.tcp_moderate_rcvbuf=0
> sysctl -w net.ipv4.tcp_default_win_scale=0
yep, this solves packages.gentoo.org..
but lkml.org is still broken for me.. i'd really like to know what this is
all about :>

> Thanks.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
>


--


2004-06-30 08:04:47

by Debi Janos

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

2004-06-29, k keltez?ssel 23:36-kor John Heffner ezt ?rta:

> Sigh. I ran in to this problem a year or so ago and it was a broken
> firewall that was mangling the TCP window scale option. I think the
> firewall was an OpenBSD machine, and I was told the problem went away with
> an upgrade. I'm curious what they're running here.
>
> The boundary 3 is special because it causes SWS avoidance to break.
>
> -John

hmm. interesting. my server sits behind an openbsd packet filter... , but the gentoo's machines uses iptables firewall...


2004-06-30 08:04:49

by Debi Janos

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

2004-06-29, k keltez?ssel 23:36-kor John Heffner ezt ?rta:

> Sigh. I ran in to this problem a year or so ago and it was a broken
> firewall that was mangling the TCP window scale option. I think the
> firewall was an OpenBSD machine, and I was told the problem went away with
> an upgrade. I'm curious what they're running here.
>
> The boundary 3 is special because it causes SWS avoidance to break.
>
> -John

hmm. interesting. my server sits behind an openbsd packet filter... , but the gentoo's machines uses iptables firewall...


2004-06-30 20:17:53

by David Miller

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Wed, 30 Jun 2004 10:04:52 +0200
Debi Janos <[email protected]> wrote:

> 2004-06-29, k keltez?ssel 23:36-kor John Heffner ezt ?rta:
>
> > Sigh. I ran in to this problem a year or so ago and it was a broken
> > firewall that was mangling the TCP window scale option. I think the
> > firewall was an OpenBSD machine, and I was told the problem went away with
> > an upgrade. I'm curious what they're running here.
> >
> > The boundary 3 is special because it causes SWS avoidance to break.
> >
> > -John
>
> hmm. interesting. my server sits behind an openbsd packet filter... , but the gentoo's machines uses iptables firewall...

Sounds like the firewall at your end is what might be causing the
problems.

2004-06-30 20:23:45

by John Heffner

[permalink] [raw]
Subject: Re: 2.6.7-mm1 - 2.6.7-mm4 weird http behavior

On Wed, 30 Jun 2004, David S. Miller wrote:

> On Wed, 30 Jun 2004 10:04:52 +0200
> Debi Janos <[email protected]> wrote:
>
> > 2004-06-29, k keltez?ssel 23:36-kor John Heffner ezt ?rta:
> >
> > > Sigh. I ran in to this problem a year or so ago and it was a broken
> > > firewall that was mangling the TCP window scale option. I think the
> > > firewall was an OpenBSD machine, and I was told the problem went away with
> > > an upgrade. I'm curious what they're running here.
> > >
> > > The boundary 3 is special because it causes SWS avoidance to break.
> > >
> > > -John
> >
> > hmm. interesting. my server sits behind an openbsd packet filter... , but the gentoo's machines uses iptables firewall...
>
> Sounds like the firewall at your end is what might be causing the
> problems.


I'm having the same problem connecting to gentoo's machines with no
firewall on my end. This happens with new kernels with the
default_win_scale set >3, or on old kernels with tcp_rmem[1] > ~700k.

-John