To try to figure out some buffering problems my network application is having,
I wrote 2 tiny utilities
- pwrite connects to a socket and floods it with data until it sleeps
- pread accepts an incoming connection but never actually reads data.
It just lets it pile up until the read buffer is full.
Therefore, pwrite will go to sleep once both its write buffer and pread's read
buffer have filled up.
echo 4096 4096 4096 > /proc/sys/net/ipv4/tcp_rmem
echo 4096 4096 4096 > /proc/sys/net/ipv4/tcp_wmem
Now that I've established a 4K read buffer and a 4K write buffer, pwrite
should go to go to sleep after writing 8K of data, right? Except that it
only manages to write 5K of data before sleeping!
Here is the output from running pread:
bronson@lea:~/linkanywhere/sockbuf/kernel$ ./pread
waiting on port 1111...
Read buffer is 4096 bytes, with 0 bytes used. (printed at startup)
Read buffer is 4096 bytes, with 3072 bytes used. (printed on exit)
And simultaneously running pwrite with 1024-byte writes:
bronson@lea:~/kernel$ ./pwrite 1024
Allocating 1024 bytes for write buffer.
0: SO_SNDBUF= 4096, SIOCOUTQ= 0: 4096 avail. Wrote 1024 bytes, 1024
total.
1: SO_SNDBUF= 4096, SIOCOUTQ= 0: 4096 avail. Wrote 1024 bytes, 2048
total.
2: SO_SNDBUF= 4096, SIOCOUTQ= 1024: 3072 avail. Wrote 1024 bytes, 3072
total.
3: SO_SNDBUF= 4096, SIOCOUTQ= 2048: 2048 avail. Wrote 1024 bytes, 4096
total.
4: SO_SNDBUF= 4096, SIOCOUTQ= 2048: 2048 avail. Wrote 1024 bytes, 5120
total.
Why on earth is it sleeping after only 5K? According to the SIOC calls, the
write buffer still had 2048 bytes available, and the read buffer 1024 bytes.
The kernel claims that there's still 3K of space that can be filled. So why
isn't it?
Thanks,
- Scott