> On Sun, Sep 12, 2004 at 09:45:38AM -0600, Wolfpaw - Dale Corse wrote:
> > No problem :) I run the following, against SSH as the target, and I
> > can also kill it. (using telnet as the other side of the attack)
> >
> > root@maximus:/home/admin# telnet XXXXXXXXXXXXXXX 22
> > Trying XXXXXXXXXXXXXX...
> > Connected to XXXXXXXXXXXXXXXXX.
> > Escape character is '^]'.
> > Connection closed by foreign host.
> > root@maximus:/home/admin#
>
> > Now.. Do you really want me to post the source code for it?
>
> With default sshd_config you can DOS sshd trivially by
> opening ten connections using ten times "telnet XXXXXXXXXXXXXXX 22".
A fair comment :) But look at it this way:
- The TCP RFC was last updated when?
- What is the average time for a tcp packet to fly even across
the world these days? Maybe 300 ms? 1 second? 5?
- It is not a secret that the TCP protocol has flaws, take for
example the RST bug, which required among other things, BGP4
to use MD5 encryption to avoid being potentially attacked.
So this brings me to:
A) Why are the timeouts so long?
B) CLOSE_WAIT having _no timeout at all_ is still using the
assumption the other side is honest, and will actually
reply. This is a very bad assumption.
C) Socket still re-uses an FD before it is actually completely
closed. This is bad, because by calling a second close in
the case of mysql, you can get the connection to go away,
but in that case, it closes whatever else is on that FD
too. (A more likely analysis is that it closes the current
connection, and then cleans the CLOSE_WAIT on that FD out
of the other pool)
All I am trying to point out is that the Internet in general, and
The Open Source movement has survived, and evolved because of innovation,
and the ability to meet upcoming threats quickly. TCP has some issues
which are blindingly obvious, and they are issues that, in my view (flawed
as it may be) can be at least somewhat minimized by a few simple changes.
I realize daemons have connection queues, and timeouts for a reason, but
really, if a daemon wishes to close a connection, for whatever reason,
sending something to the other side is required, but I can't see why having
the other side send something back is part of the protocol. This could be
implemented with KEEPALIVE much easier, and would avoid the flaws.. No reply
from the host in say 10 seconds, then drop the connection. You could still
clog queues, to which I would say the application needs to cope with one
client
filling the whole queue as best it can, and it wouldn't stop a DDOS (not much
does), but it might help some at least.
Anyway.. That's my 2 cents. I will continue my conversation with Peter from
mysql in regards to mysql_close, which was really, the entire point. It is
sad however to see a maintainer come across in the manner which David has
during the course of this discussion. It doesn't bode well for the future
of open source to tell someone off, whom likely has a valid point.. whether
or not it is a repairable fault.
Regards,
Dale.
--------------------------------
Dale Corse
System Administrator
Wolfpaw Services Inc.
http://www.wolfpaw.net
(780) 474-4095
On Sul, 2004-09-12 at 18:29, Wolfpaw - Dale Corse wrote:
> A fair comment :) But look at it this way:
>
> - The TCP RFC was last updated when?
About 2 months ago. The 793 RFC isn't updated instead new ones are added
for the additional features/discoveries.
> - What is the average time for a tcp packet to fly even across
> the world these days? Maybe 300 ms? 1 second? 5?
> - It is not a secret that the TCP protocol has flaws, take for
> example the RST bug, which required among other things, BGP4
> to use MD5 encryption to avoid being potentially attacked.
This is not a TCP flaw, its a combination of poor design by certain
vendors, poor BGP implementation and a lack of understanding of what TCP
does and does not do. See IPSec. TCP gets stuff from A to B in order and
knowing to a resonable degree what arrived. TCP does not proide a
security service.
(The core of this problem arises because certain people treat TCP
connection down on the peering session as link down)
> So this brings me to:
>
> A) Why are the timeouts so long?
So you don't get random corruption
> C) Socket still re-uses an FD before it is actually completely
Pardon ?
> sending something to the other side is required, but I can't see why having
> the other side send something back is part of the protocol. This could be
Because packet sizes are finite and not doing so requires an infinite
sequence space and thus infinite packet sizes. Reread the TCP
specifications more carefully, also look at RFC1337 which discusses some
of the real world cases of getting this wrong.
On Sun, Sep 12, 2004 at 06:04:53PM +0100, Alan Cox wrote:
>
> This is not a TCP flaw, its a combination of poor design by certain
> vendors, poor BGP implementation and a lack of understanding of what TCP
> does and does not do. See IPSec. TCP gets stuff from A to B in order and
> knowing to a resonable degree what arrived. TCP does not proide a
> security service.
>
> (The core of this problem arises because certain people treat TCP
> connection down on the peering session as link down)
Alan, could you please elaborate on this last statement?
I don't understand what you mean, and am very interested.
Thanks,
Toon.
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
On Sun, 12 Sep 2004, Toon van der Pas wrote:
>> knowing to a resonable degree what arrived. TCP does not proide a
>> security service.
>>
>> (The core of this problem arises because certain people treat TCP
>> connection down on the peering session as link down)
>
> Alan, could you please elaborate on this last statement?
> I don't understand what you mean, and am very interested.
I think he means that BGP treating TCP connections as if they could
reliably and securely indicate link/path status (ie connection
reset/timeout == link down) status was, in retrospect, a very dumb
idea on the part of BGP.
> Thanks,
> Toon.
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
This restaurant was advertising breakfast any time. So I ordered
french toast in the renaissance.
- Steven Wright, comedian
On Mon, 13 Sep 2004, Paul Jakma wrote:
> I think he means that BGP treating TCP connections as if they could
> reliably and securely indicate link/path status (ie connection
> reset/timeout == link down) status was, in retrospect, a very dumb
> idea on the part of BGP.
More specifically, BGP should have treated TCP resets as a transient
error, to be expected (indeed, they /cant/ be a sign that a link is
down - if you can receive a RST the link or path is patently quite
ok). The BGP state machine should instead, in normal operation, have
only treated Hold time expired as the definitive sign of "peer is
down" and allowed reconnects.
> regards,
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
You will attract cultured and artistic people to your home.
On Mon, Sep 13, 2004 at 04:30:36AM +0100, Paul Jakma wrote:
> More specifically, BGP should have treated TCP resets as a transient
> error, to be expected (indeed, they /cant/ be a sign that a link is
> down - if you can receive a RST the link or path is patently quite
> ok).
The application level does not always distinguish between TCP RST and
error generated by the local system because of a "network unreachable"
due to a link down and a lost route.
> The BGP state machine should instead, in normal operation, have
> only treated Hold time expired as the definitive sign of "peer is
> down" and allowed reconnects.
It should not necessarily wait for the time-out, but at least wait for
a few reconnect errors.
Regards,
willy
On Mon, 13 Sep 2004, Willy Tarreau wrote:
> It should not necessarily wait for the time-out, but at least wait for
> a few reconnect errors.
No, it should wait for the timeout. (how many reconnects? maybe use a
time for that? well, you already have one, so use that. if you want
to timeout quicker, lower it.)
Alas though, it wouldnt be BGP.
> Regards,
> willy
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
You will be called upon to help a friend in trouble.
Salut,
On Mon, Sep 13, 2004 at 06:18:47AM +0200, Willy Tarreau wrote:
> > The BGP state machine should instead, in normal operation, have
> > only treated Hold time expired as the definitive sign of "peer is
> > down" and allowed reconnects.
>
> It should not necessarily wait for the time-out, but at least wait for
> a few reconnect errors.
Problem there: you can fake connection errors almost as easily as
sending an RST packet, so the DoS might reappear, might it not?
Tonnerre
On Mon, Sep 13, 2004 at 09:07:41PM +0200, Tonnerre wrote:
> Salut,
>
> On Mon, Sep 13, 2004 at 06:18:47AM +0200, Willy Tarreau wrote:
> > > The BGP state machine should instead, in normal operation, have
> > > only treated Hold time expired as the definitive sign of "peer is
> > > down" and allowed reconnects.
> >
> > It should not necessarily wait for the time-out, but at least wait for
> > a few reconnect errors.
>
> Problem there: you can fake connection errors almost as easily as
> sending an RST packet, so the DoS might reappear, might it not?
>
No, as long as you don't keep the routes from the old session until the
new one establishes and fills up (or you reach the timeout). And when I
spoke about "connection errors", I really spoke about connection
establishment. I bet you'll have more difficulties trying to send the
right RST just after a SYN (or an ICMP unreachable with the right payload)
than sending them once the session is already established. It does make
a big difference.
Willy
On Mon, 13 Sep 2004, Tonnerre wrote:
> Problem there: you can fake connection errors almost as easily as
> sending an RST packet, so the DoS might reappear, might it not?
Sure, but TCP just isnt going to solve this for you.
> Tonnerre
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
All progress is based upon a universal innate desire of every organism
to live beyond its income.
-- Samuel Butler, "Notebooks"
On Mon, Sep 13, 2004 at 04:30:36AM +0100, Paul Jakma wrote:
> More specifically, BGP should have treated TCP resets as a transient
> error, to be expected (indeed, they /cant/ be a sign that a link is
Actually you can treat TCP session failure as transient error. Just
use BGP graceful restart (witch basically allows re-opening TCP
connection without losing routing tables).
http://www.ietf.org/internet-drafts/draft-ietf-idr-restart-10.txt
--
[Ville Hallivuori][[email protected]][http://www.iki.fi/vph/]
[ID 8E1AD461][FP16=C9 50 E2 DF 48 F6 33 62 5D 87 47 9D 3F 2B 07 5D]
[ID 58543419][FP20=8731 941D 15AB D4A0 88A0 FC8F B55C F4C4 5854 3419]
[ID 8061C24E][FP20=C722 12DA 841E D811 DBFE 2FB3 174C E291 8061 C24E]
On Mon, 13 Sep 2004, Ville Hallivuori wrote:
> Actually you can treat TCP session failure as transient error. Just
> use BGP graceful restart (witch basically allows re-opening TCP
> connection without losing routing tables).
>
> http://www.ietf.org/internet-drafts/draft-ietf-idr-restart-10.txt
Hmm, yes, I hadnt thought of the attack-mitigating aspects of
graceful restart. Though, without other measures, the session is
still is open to abuse (send RST every second).
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
Wit, n.:
The salt with which the American Humorist spoils his cookery
... by leaving it out.
-- Ambrose Bierce, "The Devil's Dictionary"
On Maw, 2004-09-14 at 15:55, Paul Jakma wrote:
> Hmm, yes, I hadnt thought of the attack-mitigating aspects of
> graceful restart. Though, without other measures, the session is
> still is open to abuse (send RST every second).
Its more than that given port randomization, quite a lot more. Of course
its much easier to just send "must fragment, size 68" icmp replies and
guess them that way. This is spectacularly more effective and various
vendors highly invalid rst acking crap won't save you.
On Tue, 14 Sep 2004, Alan Cox wrote:
> guess them that way. This is spectacularly more effective and
> various vendors highly invalid rst acking crap won't save you.
Ah, well, I dont care about various vendors. I only care about Linux,
BSD and SunOS kernel behaviour ;)
That said, TCP-MD5 signature renders this mostly moot, and deployment
of TCP-MD5 has increased a lot since the last round of "BGP TCP is
insecure!" non-issues came up. Many IXes and peers now require
TCP-MD5.
The rights and wrongs of TCP-MD5 notwithstanding, it'd be nice if
Linux could support this. Anyone running BGP on Linux at moment must
patch their kernel - or else just switch to Free/Open BSD.
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
It looks like it's up to me to save our skins. Get into that garbage chute,
flyboy!
-- Princess Leia Organa
On Maw, 2004-09-14 at 17:26, Paul Jakma wrote:
> That said, TCP-MD5 signature renders this mostly moot, and deployment
> of TCP-MD5 has increased a lot since the last round of "BGP TCP is
> insecure!" non-issues came up. Many IXes and peers now require
> TCP-MD5.
TCP-MD5 has no effect on ICMP based attacks.,
On Tue, 14 Sep 2004, Alan Cox wrote:
> TCP-MD5 has no effect on ICMP based attacks.,
Hmm, good point. Which attacks, and what could be done about them?
(other than IPsec protect all traffic between peers).
regards,
--
Paul Jakma [email protected] [email protected] Key ID: 64A2FF6A
Fortune:
"You can't get very far in this world without your dossier being there first."
-- Arthur Miller
Hi Alan,
On Tue, Sep 14, 2004 at 04:10:36PM +0100, Alan Cox wrote:
> On Maw, 2004-09-14 at 15:55, Paul Jakma wrote:
> > Hmm, yes, I hadnt thought of the attack-mitigating aspects of
> > graceful restart. Though, without other measures, the session is
> > still is open to abuse (send RST every second).
>
> Of course its much easier to just send "must fragment, size 68" icmp
> replies and guess them that way. This is spectacularly more effective
> and various vendors highly invalid rst acking crap won't save you.
Just wondering, I have not checked. Isn't the "must fragment" message
supposed to embed part of the packet it couldn't send in return ? If
this is the case (and if the victim processes it correctly), it would
need to guess a recent valid content. If it's not the case, I suspect
it would simply update the path mtu in the route cache, thus giving
spectacular effects :-)
Cheers,
Willy
On Maw, 2004-09-14 at 20:41, Willy Tarreau wrote:
> Just wondering, I have not checked. Isn't the "must fragment" message
> supposed to embed part of the packet it couldn't send in return ?
You need to guess no more than for an RST attack, and furthermore in
some cases (buggy stacks) IPsec doesn't save you because the error is
from an untrusted midpoint. The proper response to such messages is to
turn off DF usage but not all stacks get it right
* Alan Cox:
> On Maw, 2004-09-14 at 15:55, Paul Jakma wrote:
>> Hmm, yes, I hadnt thought of the attack-mitigating aspects of
>> graceful restart. Though, without other measures, the session is
>> still is open to abuse (send RST every second).
>
> Its more than that given port randomization, quite a lot more. Of course
> its much easier to just send "must fragment, size 68" icmp replies and
> guess them that way.
Is this attack documented anywhere?
* Paul Jakma:
>> TCP-MD5 has no effect on ICMP based attacks.,
>
> Hmm, good point. Which attacks, and what could be done about them?
> (other than IPsec protect all traffic between peers).
You just filter ICMP packets, in the way RST packets are already
filtered (i.e. rate limit).
The only TCP desynchronization attack that has a chance of working
practice is the SYN-based one. The rate limit for RST processing on
Cisco routers is far too low.
(Mixed Cisco/Quagga environments are a different matter, but rather
unusual and easily DoSed anyway, most of the time.)
On Llu, 2004-09-20 at 23:03, Florian Weimer wrote:
> ndomization, quite a lot more. Of course
> > its much easier to just send "must fragment, size 68" icmp replies and
> > guess them that way.
>
> Is this attack documented anywhere?
Bugtraq years ago and also in the discussions of the IP sec protocol
design flaws when it was being specified
Florian Weimer <[email protected]> wrote:
>
>>> TCP-MD5 has no effect on ICMP based attacks.,
>>
>> Hmm, good point. Which attacks, and what could be done about them?
>> (other than IPsec protect all traffic between peers).
>
> You just filter ICMP packets, in the way RST packets are already
> filtered (i.e. rate limit).
Rate-limiting has no effect on ICMP attacks unless your limit is such
that you're effectively dropping them all. But then you get PMTU
problems...
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
* Herbert Xu:
> Florian Weimer <[email protected]> wrote:
>>
>>>> TCP-MD5 has no effect on ICMP based attacks.,
>>>
>>> Hmm, good point. Which attacks, and what could be done about them?
>>> (other than IPsec protect all traffic between peers).
>>
>> You just filter ICMP packets, in the way RST packets are already
>> filtered (i.e. rate limit).
>
> Rate-limiting has no effect on ICMP attacks unless your limit is such
> that you're effectively dropping them all.
Yes, that's the idea. Keep in mind that all this is about traffic
destined to a router interface address, not about forwarded traffic.
> But then you get PMTU problems...
PMTU discovery is not an issue because it's turned off anyway, at
least by default.
* David S. Miller:
> On Tue, 21 Sep 2004 20:32:12 +0200
> Florian Weimer <[email protected]> wrote:
>
>> > But then you get PMTU problems...
>>
>> PMTU discovery is not an issue because it's turned off anyway, at
>> least by default.
>
> It's on by default, for both TCP and UDP in the kernel,
> and has been so for a long time.
Linux is not the reference TCP/IP stack for routers. 8-)
> Why would it be off by default?
Probably because PMTUD is just a DRAFT STANDARD, and these router
folks are usually extremely conservative. Switching the default is
dangerous because it's likely to break existing setups, as Herbert
noted.
> If you disable PMTU discovery, say goodbye to TCP performance.
Indeed. On those platforms, the CPU impact is also significant, and
the overall increase in BGP convergence time is measurable.
On Tue, 21 Sep 2004 20:32:12 +0200
Florian Weimer <[email protected]> wrote:
> > But then you get PMTU problems...
>
> PMTU discovery is not an issue because it's turned off anyway, at
> least by default.
It's on by default, for both TCP and UDP in the kernel,
and has been so for a long time.
Why would it be off by default?
If you disable PMTU discovery, say goodbye to TCP
performance.
On Tue, 21 Sep 2004 22:04:41 +0200
Florian Weimer <[email protected]> wrote:
> > Why would it be off by default?
>
> Probably because PMTUD is just a DRAFT STANDARD,
RFC1191 doesn't look like a draft to me.
* David S. Miller:
> On Tue, 21 Sep 2004 22:04:41 +0200
> Florian Weimer <[email protected]> wrote:
>
>> > Why would it be off by default?
>>
>> Probably because PMTUD is just a DRAFT STANDARD,
>
> RFC1191 doesn't look like a draft to me.
It's not a draft document, but it's still a DRAFT STANDRD in the IETF
standards track, see RFC 3700. (I wasn't shouting, I was using IETF
keywords. 8-)