2002-07-23 14:38:42

by John Covici

[permalink] [raw]
Subject: is flock broken in 2.4 or 2.5 kernels or what does this mean?

In the latest release notes of sendmail I have read the following:

NOTE: Linux appears to have broken flock() again. Unless
the bug is fixed before sendmail 8.13 is shipped,
8.13 will change the default locking method to
fcntl() for Linux kernel 2.4 and later. You may
want to do this in 8.12 by compiling with
-DHASFLOCK=0. Be sure to update other sendmail
related programs to match locking techniques.

Can anyone tell me what this is all about -- is there any basis in
reality for what they are saying?

Thanks.

--
John Covici
[email protected]


2002-07-23 15:14:57

by Alan

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

On Tue, 2002-07-23 at 15:41, John Covici wrote:
> In the latest release notes of sendmail I have read the following:
>
> NOTE: Linux appears to have broken flock() again. Unless
> the bug is fixed before sendmail 8.13 is shipped,
> 8.13 will change the default locking method to
> fcntl() for Linux kernel 2.4 and later. You may
> want to do this in 8.12 by compiling with
> -DHASFLOCK=0. Be sure to update other sendmail
> related programs to match locking techniques.
>
> Can anyone tell me what this is all about -- is there any basis in
> reality for what they are saying?

First I've heard of it, so it would be useful if someone has access to
the sendmail problem report/test in question that shows it and I'll go
find out.

2002-07-23 15:25:42

by Richard A Nelson

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

On 23 Jul 2002, Alan Cox wrote:

> Date: 23 Jul 2002 17:31:12 +0100
> From: Alan Cox <[email protected]>
> To: John Covici <[email protected]>
> Cc: [email protected]
> Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?
>
> On Tue, 2002-07-23 at 15:41, John Covici wrote:
> > In the latest release notes of sendmail I have read the following:
> >
> > NOTE: Linux appears to have broken flock() again. Unless
> > the bug is fixed before sendmail 8.13 is shipped,
> > 8.13 will change the default locking method to
> > fcntl() for Linux kernel 2.4 and later. You may
> > want to do this in 8.12 by compiling with
> > -DHASFLOCK=0. Be sure to update other sendmail
> > related programs to match locking techniques.
> >
> > Can anyone tell me what this is all about -- is there any basis in
> > reality for what they are saying?
>
> First I've heard of it, so it would be useful if someone has access to
> the sendmail problem report/test in question that shows it and I'll go
> find out.

Indeed it is, this message explains it fairly well:
Message-ID: <[email protected]>
Date: Tue, 2 Jul 2002 16:01:18 +0100
From: Stephen C. Tweedie <[email protected]>

I haven't heard of any plans to incorporate the removal of accounting
info in either 2.4 or 2.5 ;(

--
Rick Nelson
Your job is being a professor and researcher: That's one hell of a good excuse
for some of the brain-damages of minix.
(Linus Torvalds to Andrew Tanenbaum)


2002-07-23 15:47:30

by Jiri Kosina

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

On 23 Jul 2002, Alan Cox wrote:

> > Can anyone tell me what this is all about -- is there any basis in
> > reality for what they are saying?
> First I've heard of it, so it would be useful if someone has access to
> the sendmail problem report/test in question that shows it and I'll go
> find out.

Quoting Stephen Tweedie's earlier post to the list:

==
It really is broken, and sendmail triggers it (at least their
commercial binaries do). I've already been talking to willy about the
problem. The trouble is the accounting: if one process opens a fd and then
fork()s, it is possible for the lock to be taken in the parent and
released in the child (or vice versa) --- unless there's an explicit
flock(LOCK_UN), then the lock will be released implicitly when the
last reference to the fd is closed. When this happens, we get the lock
count incremented in one task and
decremented in another. That can wrap the lock count backwards to -1
(or rather ~0UL), which causes the locks rlimit check to think we've
exceeded the lock quota and new lock requests will fail. It's easy to
reproduce this: try the attached prog. It produces an erroneous
ENOLCK due to the bug.
==

--
JiKos.


2002-07-23 21:41:17

by J.A. Magallon

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?


On 2002.07.23 Alan Cox wrote:
>On Tue, 2002-07-23 at 15:41, John Covici wrote:
>> In the latest release notes of sendmail I have read the following:
>>
>> NOTE: Linux appears to have broken flock() again. Unless
>> the bug is fixed before sendmail 8.13 is shipped,
>> 8.13 will change the default locking method to
>> fcntl() for Linux kernel 2.4 and later. You may
>> want to do this in 8.12 by compiling with
>> -DHASFLOCK=0. Be sure to update other sendmail
>> related programs to match locking techniques.
>>
>> Can anyone tell me what this is all about -- is there any basis in
>> reality for what they are saying?
>
>First I've heard of it, so it would be useful if someone has access to
>the sendmail problem report/test in question that shows it and I'll go
>find out.
>

Perhaps if you have your spool over nfs:

man flock:

NOTES
flock(2) does not lock files over NFS. Use fcntl(2)
instead: that does work over NFS, given a sufficiently
recent version of Linux and a server which supports lock?
ing.

flock(2) and fcntl(2) locks have different semantics with
respect to forked processes and dup(2).

--
J.A. Magallon \ Software is like sex: It's better when it's free
mailto:[email protected] \ -- Linus Torvalds, FSF T-shirt
Linux werewolf 2.4.19-rc3-jam1, Mandrake Linux 9.0 (Cooker) for i586
gcc (GCC) 3.1.1 (Mandrake Linux 8.3 3.1.1-0.10mdk)

2002-07-23 23:16:32

by Andrea Arcangeli

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

On Tue, Jul 23, 2002 at 05:31:12PM +0100, Alan Cox wrote:
> On Tue, 2002-07-23 at 15:41, John Covici wrote:
> > In the latest release notes of sendmail I have read the following:
> >
> > NOTE: Linux appears to have broken flock() again. Unless
> > the bug is fixed before sendmail 8.13 is shipped,
> > 8.13 will change the default locking method to
> > fcntl() for Linux kernel 2.4 and later. You may
> > want to do this in 8.12 by compiling with
> > -DHASFLOCK=0. Be sure to update other sendmail
> > related programs to match locking techniques.
> >
> > Can anyone tell me what this is all about -- is there any basis in
> > reality for what they are saying?
>
> First I've heard of it, so it would be useful if someone has access to
> the sendmail problem report/test in question that shows it and I'll go
> find out.

fix is in -aa from Matthew Wilcox:

http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.19rc3aa1/00_drop-broken-flock-account-1

fcntl API never obeyed to the accounting anyways.

Andrea

2002-07-23 23:24:52

by Dan Kegel

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[email protected] wrote:
>
> To save others a bit of searching:
> the original thread was 'broken flock()' on 28 June 2002.
>
> Steven's post and regression test:
> http://marc.theaimsgroup.com/?l=linux-kernel&m=102562226210792&w=2
> Matthew Wilcox's rip-it-out patch:
> http://marc.theaimsgroup.com/?l=linux-fsdevel&m=102562469813922&w=2

See also
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=59992

2002-07-23 23:22:20

by Dan Kegel

[permalink] [raw]
Subject: re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

To save others a bit of searching:
the original thread was 'broken flock()' on 28 June 2002.

Steven's post and regression test:
http://marc.theaimsgroup.com/?l=linux-kernel&m=102562226210792&w=2
Matthew Wilcox's rip-it-out patch:
http://marc.theaimsgroup.com/?l=linux-fsdevel&m=102562469813922&w=2

2002-07-24 16:05:27

by David Ford

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

(Alan)
Actually I posted a "is sendmail on crack?" question last month when
8.12.5 was released and this was in the release notes. We got a bunch
of confirmations that Linux flock() was broken due to the accounting.

David

J.A. Magallon wrote:

>On 2002.07.23 Alan Cox wrote:
>
>
>>On Tue, 2002-07-23 at 15:41, John Covici wrote:
>>
>>
>>>In the latest release notes of sendmail I have read the following:
>>>
>>> NOTE: Linux appears to have broken flock() again. Unless
>>> the bug is fixed before sendmail 8.13 is shipped,
>>> 8.13 will change the default locking method to
>>> fcntl() for Linux kernel 2.4 and later. You may
>>> want to do this in 8.12 by compiling with
>>> -DHASFLOCK=0. Be sure to update other sendmail
>>> related programs to match locking techniques.
>>>
>>>Can anyone tell me what this is all about -- is there any basis in
>>>reality for what they are saying?
>>>
>>>
>>First I've heard of it, so it would be useful if someone has access to
>>the sendmail problem report/test in question that shows it and I'll go
>>find out.
>>
>>
>>
>
>Perhaps if you have your spool over nfs:
>
>man flock:
>
>NOTES
> flock(2) does not lock files over NFS. Use fcntl(2)
> instead: that does work over NFS, given a sufficiently
> recent version of Linux and a server which supports lock?
> ing.
>
> flock(2) and fcntl(2) locks have different semantics with
> respect to forked processes and dup(2).
>
>
>

--
I may have the information you need and I may choose only HTML. It's up to
you. Disclaimer: I am not responsible for any email that you send me nor am
I bound to any obligation to deal with any received email in any given
fashion. If you send me spam or a virus, I may in whole or part send you
50,000 return copies of it. I may also publically announce any and all
emails and post them to message boards, news sites, and even parody sites.
I may also mark them up, cut and paste, print, and staple them to telephone
poles for the enjoyment of people without internet access. This is not a
confidential medium and your assumption that your email can or will be
handled confidentially is akin to baring your backside, burying your head in
the ground, and thinking nobody can see you butt nekkid and in plain view
for miles away. Don't be a cluebert, buy one from K-mart today.


2002-07-25 00:12:23

by Alan

[permalink] [raw]
Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

On Wed, 2002-07-24 at 17:08, David Ford wrote:
> (Alan)
> Actually I posted a "is sendmail on crack?" question last month when
> 8.12.5 was released and this was in the release notes. We got a bunch
> of confirmations that Linux flock() was broken due to the accounting.

Thanks. I now have the original discussion, the demonstration and a
patch Andrea believes fixes it.