2006-12-17 03:50:43

by Komuro

[permalink] [raw]
Subject: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer


Hello,

On kernel 2.6.20-rc1, ftp (get or put) stops
during file-transfer.

Client: ftp-0.17-33.fc6 (192.168.1.1)
Server: vsftpd-2.0.5-8 (192.168.1.3)

This problem does _not_ happen on kernel-2.6.19.
is it caused by network-subsystem change on 2.6.20-rc1??


Best Regards
Komuro


2006-12-17 04:02:25

by Al Viro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

On Sun, Dec 17, 2006 at 09:27:52PM +0900, Komuro wrote:
>
> Hello,
>
> On kernel 2.6.20-rc1, ftp (get or put) stops
> during file-transfer.
>
> Client: ftp-0.17-33.fc6 (192.168.1.1)
> Server: vsftpd-2.0.5-8 (192.168.1.3)
>
> This problem does _not_ happen on kernel-2.6.19.
> is it caused by network-subsystem change on 2.6.20-rc1??

Do you have NAT between you and server?

2006-12-17 05:21:27

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

On Sun, 17 Dec 2006 04:02:22 +0000
Al Viro <[email protected]> wrote:

> On Sun, Dec 17, 2006 at 09:27:52PM +0900, Komuro wrote:
> >
> > Hello,
> >
> > On kernel 2.6.20-rc1, ftp (get or put) stops
> > during file-transfer.
> >
> > Client: ftp-0.17-33.fc6 (192.168.1.1)
> > Server: vsftpd-2.0.5-8 (192.168.1.3)
> >
> > This problem does _not_ happen on kernel-2.6.19.
> > is it caused by network-subsystem change on 2.6.20-rc1??
>
> Do you have NAT between you and server?

No. I don't have NAT between the client and the server.
Actually, the client and the sever is located in same room.


client -- 100MbpsHub -- server.

Thanks!

Best Regards
Komuro




2006-12-18 03:01:14

by Adrian Bunk

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

On Sun, Dec 17, 2006 at 11:23:11PM +0900, Komuro wrote:
> On Sun, 17 Dec 2006 04:02:22 +0000
> Al Viro <[email protected]> wrote:
>
> > On Sun, Dec 17, 2006 at 09:27:52PM +0900, Komuro wrote:
> > >
> > > Hello,
> > >
> > > On kernel 2.6.20-rc1, ftp (get or put) stops
> > > during file-transfer.
> > >
> > > Client: ftp-0.17-33.fc6 (192.168.1.1)
> > > Server: vsftpd-2.0.5-8 (192.168.1.3)
> > >
> > > This problem does _not_ happen on kernel-2.6.19.
> > > is it caused by network-subsystem change on 2.6.20-rc1??
> >
> > Do you have NAT between you and server?
>
> No. I don't have NAT between the client and the server.
> Actually, the client and the sever is located in same room.
>
> client -- 100MbpsHub -- server.

What network cards are in the client and the server?

Are there any error messages your client gives or in the log files?

> Thanks!
>
> Best Regards
> Komuro

cu
Adrian

--

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed

2006-12-18 11:53:44

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer


>
> What network cards are in the client and the server?

DL10022-based pcmcia network card(both client and server)
The driver name is pcnet_cs.

> Are there any error messages your client gives or in the log files?

no error messages.

I capture the packet of ftp transfer by ethereal.
I found the malformed packet when it stops.

I will investigate it further.

Thanks!

Best Regards
Komuro

2006-12-23 02:16:23

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer


> > On kernel 2.6.20-rc1, ftp (get or put) stops
> > during file-transfer.
> >
> > Client: ftp-0.17-33.fc6 (192.168.1.1)
> > Server: vsftpd-2.0.5-8 (192.168.1.3)
> >

This problem happens on kernel-2.6.19-git4 or later.
but does _not_ happen on kernel-2.6.19-git3.

So this problem is introduced to kernel-2.6.19-git4.


I tried the Marvell 88E8001(skge) and Realtek 8139(8139too),
the same problem happens.

I think this is not a network-card-driver problem.

Thanks!

Best Regards
Komuro

2006-12-30 00:51:30

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

Hi,

I investigated the ftp-file-transfer-stop problem by git-bisect method,
and found this problem was introduced by
"[TCP]: MD5 Signature Option (RFC2385) support" patch.

Mr.YOSHIFUJI san, please fix this problem.

>commit cfb6eeb4c860592edd123fdea908d23c6ad1c7dc
>Author: YOSHIFUJI Hideaki <[email protected]>
>Date: Tue Nov 14 19:07:45 2006 -0800
>
> [TCP]: MD5 Signature Option (RFC2385) support.
>
> Based on implementation by Rick Payne.
>
> Signed-off-by: YOSHIFUJI Hideaki <[email protected]>
> Signed-off-by: David S. Miller <[email protected]>

Best Regards
Komuro


> On Sun, Dec 17, 2006 at 11:23:11PM +0900, Komuro wrote:
> > On Sun, 17 Dec 2006 04:02:22 +0000
> > Al Viro <[email protected]> wrote:
> >
> > > On Sun, Dec 17, 2006 at 09:27:52PM +0900, Komuro wrote:
> > > >
> > > > Hello,
> > > >
> > > > On kernel 2.6.20-rc1, ftp (get or put) stops
> > > > during file-transfer.
> > > >
> > > > Client: ftp-0.17-33.fc6 (192.168.1.1)
> > > > Server: vsftpd-2.0.5-8 (192.168.1.3)
> > > >
> > > > This problem does _not_ happen on kernel-2.6.19.
> > > > is it caused by network-subsystem change on 2.6.20-rc1??
> > >
> > > Do you have NAT between you and server?
> >
> > No. I don't have NAT between the client and the server.
> > Actually, the client and the sever is located in same room.
> >
> > client -- 100MbpsHub -- server.

2006-12-30 01:52:21

by YOSHIFUJI Hideaki

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

In article <[email protected]> (at Sat, 30 Dec 2006 18:50:43 +0900), Komuro <[email protected]> says:

> I investigated the ftp-file-transfer-stop problem by git-bisect method,
> and found this problem was introduced by
> "[TCP]: MD5 Signature Option (RFC2385) support" patch.
>
> Mr.YOSHIFUJI san, please fix this problem.

Hmm, have you try disabling CONFIG_TCP_MD5SIG?
(Is it already disabled?)

Are there any specific size of transfer to reproduce this?
Do you see similar issue with other simple application?

--yoshfuji

2006-12-30 03:01:09

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer


>
> > I investigated the ftp-file-transfer-stop problem by git-bisect method,
> > and found this problem was introduced by
> > "[TCP]: MD5 Signature Option (RFC2385) support" patch.
> >
> > Mr.YOSHIFUJI san, please fix this problem.
>
> Hmm, have you try disabling CONFIG_TCP_MD5SIG?
> (Is it already disabled?)

This problem happens both CONFIG_TCP_MD5SIG is disabled and enabled.

> Are there any specific size of transfer to reproduce this?

When I do ftp 40Mbytes file for 5-times or more,
this problem happens.


> Do you see similar issue with other simple application?

sorry, I don't reproduce this problem on other application.

Thanks,

Best Regards
Komuro.

2006-12-30 14:19:15

by YOSHIFUJI Hideaki

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

In article <[email protected]> (at Sat, 30 Dec 2006 20:59:31 +0900), Komuro <[email protected]> says:

> > Do you see similar issue with other simple application?
>
> sorry, I don't reproduce this problem on other application.

Can you reproduce it with other ftp client and/or server?


Anyway...

Please provide the output of "netstat -na" command during the
transfer, and the output of "lsmod | grep conntrack" (just for
sure).


More questions:

What kind of mode do you use? e.g. PORT/EPRT/LPRT/PASV/EPSV/LPSV

When the transfer get stuck, are other communication still working?

Are there any workaround?
e.g. stop-start vsftpd cycle, ifdown-ifup cycle, rmmod/insmod cycle etc.


Regards,

--yoshfuji

2006-12-31 00:43:44

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer


> Can you reproduce it with other ftp client and/or server?

O.K. I wiil try to test other ftp client and server.


> Please provide the output of "netstat -na" command during the
> transfer, and the output of "lsmod | grep conntrack" (just for
> sure).

Please see the output of "netstat -na" when stuck. (below)

CONFIG_NETFILER is diabled in my test configuration
,conntrack modules is not loaded.
(CONFIG_IP_DCCP, CONFIG_IP_SCTP, CONFIG_TIPC, CONFIG_IPV6 is
also disabled)


> What kind of mode do you use? e.g. PORT/EPRT/LPRT/PASV/EPSV/LPSV

PASV mode.

> When the transfer get stuck, are other communication still working?

Other communication works properly.
Actually, I can start other ftp session on other console of the same PC.

> Are there any workaround?
> e.g. stop-start vsftpd cycle, ifdown-ifup cycle, rmmod/insmod cycle etc.

I only need to do the killall command.


>> output of netstat -na

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 192.168.0.6:35737 192.168.0.2:26827 TIME_WAIT
tcp 0 0 192.168.0.6:51036 192.168.0.2:21 ESTABLISHED
udp 0 0 0.0.0.0:68 0.0.0.0:*
udp 0 0 0.0.0.0:867 0.0.0.0:*
udp 0 0 0.0.0.0:870 0.0.0.0:*
udp 0 0 0.0.0.0:111 0.0.0.0:*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] STREAM LISTENING 5056 /tmp/.font-unix/fs7100
unix 2 [ ] DGRAM 1234 @/org/kernel/udev/udevd
unix 5 [ ] DGRAM 4748 /dev/log
unix 2 [ ACC ] STREAM LISTENING 4917 /var/run/dbus/system_bus_socket
unix 2 [ ACC ] STREAM LISTENING 4989 /var/run/acpid.socket
unix 2 [ ] DGRAM 5390
unix 3 [ ] STREAM CONNECTED 4920
unix 3 [ ] STREAM CONNECTED 4919
unix 2 [ ] DGRAM 4866
unix 2 [ ] DGRAM 4756



Thanks,

Best Regards
Komuro

2007-01-01 08:19:25

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer


>
> Can you reproduce it with other ftp client and/or server?

I tried the proftpd-1.3.0a-1.fc6(kernel version is 2.6.19).
The ftp stop problem does not happen.

Therefore, this problem is reproduced when
client's kernel-version is 2.6.20-rc1 or later
and server is vsftpd.
Server's kernel-version is not related with this problem.

The ftp-stop-problem happens on client's PC.

Please advise.

Best Regards
Komuro





2007-01-04 11:46:31

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

Hi,

I made a patch below.
With this patch, the ftp-transfer-stop problem does not happen.
Therefore, I think this is not a problem of vsftpd.

Mr.YOSHIFUJI san, why did you set TCPOLEN_TSTAMP_ALIGNED
to iov_len?



--- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig 2007-01-03 11:50:04.000000000 +0900
+++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c 2007-01-03 15:30:44.000000000 +0900
@@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
TCPOLEN_TIMESTAMP);
rep.opt[1] = htonl(tcp_time_stamp);
rep.opt[2] = htonl(ts);
- arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
+ arg.iov[0].iov_len = sizeof(rep);
}

/* Swap the send and the receive. */


Best Regards
Komuro

2007-01-04 12:33:44

by Craig Schlenter

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

On Fri, Jan 05, 2007 at 05:45:46AM +0900, Komuro wrote:
> Hi,
>
> I made a patch below.
> With this patch, the ftp-transfer-stop problem does not happen.
> Therefore, I think this is not a problem of vsftpd.
>
> Mr.YOSHIFUJI san, why did you set TCPOLEN_TSTAMP_ALIGNED
> to iov_len?
>
>
>
> --- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig 2007-01-03 11:50:04.000000000 +0900
> +++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c 2007-01-03 15:30:44.000000000 +0900
> @@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
> TCPOLEN_TIMESTAMP);
> rep.opt[1] = htonl(tcp_time_stamp);
> rep.opt[2] = htonl(ts);
> - arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
> + arg.iov[0].iov_len = sizeof(rep);

Perhaps this was supposed to be
arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;

That's what the ipv6 stuff does in places.

bye,

--Craig

2007-01-09 01:24:08

by YOSHIFUJI Hideaki

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

In article <[email protected]> (at Thu, 4 Jan 2007 14:23:30 +0200), Craig Schlenter <[email protected]> says:

> On Fri, Jan 05, 2007 at 05:45:46AM +0900, Komuro wrote:
> > Hi,
> >
> > I made a patch below.
> > With this patch, the ftp-transfer-stop problem does not happen.
> > Therefore, I think this is not a problem of vsftpd.
> >
> > Mr.YOSHIFUJI san, why did you set TCPOLEN_TSTAMP_ALIGNED
> > to iov_len?
> >
> >
> >
> > --- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig 2007-01-03 11:50:04.000000000 +0900
> > +++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c 2007-01-03 15:30:44.000000000 +0900
> > @@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
> > TCPOLEN_TIMESTAMP);
> > rep.opt[1] = htonl(tcp_time_stamp);
> > rep.opt[2] = htonl(ts);
> > - arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
> > + arg.iov[0].iov_len = sizeof(rep);
>
> Perhaps this was supposed to be
> arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;
>
> That's what the ipv6 stuff does in places.

Good catch! I agree.
Craig, please provide a patch for us, please.

Thank you again.

--yoshfuji

2007-01-09 05:21:18

by Craig Schlenter

[permalink] [raw]
Subject: [PATCH] Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

Hi Dave

YOSHIFUJI Hideaki / 吉藤英明 has suggested that I send the patch
below to fix the ftp stalls present in the current kernels.

All credit goes to Komuro <[email protected]> for tracking
this down. The patch is untested but it looks *cough* obviously
correct.

Signed-off-by: Craig Schlenter <[email protected]>

Thank you!

--Craig

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index bf7a224..12de90a 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_timewait_sock *twsk,
TCPOLEN_TIMESTAMP);
rep.opt[1] = htonl(tcp_time_stamp);
rep.opt[2] = htonl(ts);
- arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
+ arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;
}

/* Swap the send and the receive. */

2007-01-09 05:21:57

by YOSHIFUJI Hideaki

[permalink] [raw]
Subject: Re: [PATCH] Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

Dave, please apply. Thank you.

In article <[email protected]> (at Tue, 9 Jan 2007 07:11:39 +0200), Craig Schlenter <[email protected]> says:

> All credit goes to Komuro <[email protected]> for tracking
> this down. The patch is untested but it looks *cough* obviously
> correct.
>
> Signed-off-by: Craig Schlenter <[email protected]>
Signed-off-by: YOSHIFUJI Hideaki <[email protected]>


Attachments:
(No filename) (3.34 kB)

2007-01-09 08:11:27

by David Miller

[permalink] [raw]
Subject: Re: [PATCH] Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

From: YOSHIFUJI Hideaki <[email protected]>
Date: Tue, 09 Jan 2007 14:22:44 +0900 (JST)

> Dave, please apply. Thank you.
>
> In article <[email protected]> (at Tue, 9 Jan 2007 07:11:39 +0200), Craig Schlenter <[email protected]> says:
>
> > All credit goes to Komuro <[email protected]> for tracking
> > this down. The patch is untested but it looks *cough* obviously
> > correct.
> >
> > Signed-off-by: Craig Schlenter <[email protected]>
> Signed-off-by: YOSHIFUJI Hideaki <[email protected]>

Applied, thanks everyone.

2007-01-09 13:02:11

by Komuro

[permalink] [raw]
Subject: Re: [BUG KERNEL 2.6.20-rc1] ftp: get or put stops during file-transfer

On Thu, 4 Jan 2007 14:23:30 +0200
Craig Schlenter <[email protected]> wrote:


> > --- linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c.orig 2007-01-03 11:50:04.000000000 +0900
> > +++ linux-2.6.20-rc3/net/ipv4/tcp_ipv4.c 2007-01-03 15:30:44.000000000 +0900
> > @@ -648,7 +648,7 @@ static void tcp_v4_send_ack(struct tcp_t
> > TCPOLEN_TIMESTAMP);
> > rep.opt[1] = htonl(tcp_time_stamp);
> > rep.opt[2] = htonl(ts);
> > - arg.iov[0].iov_len = TCPOLEN_TSTAMP_ALIGNED;
> > + arg.iov[0].iov_len = sizeof(rep);
>
> Perhaps this was supposed to be
> arg.iov[0].iov_len += TCPOLEN_TSTAMP_ALIGNED;
>
> That's what the ipv6 stuff does in places.

It works properly.
Thanks!


Best Regards
Komuro