2003-11-04 16:45:35

by John M Collins

[permalink] [raw]
Subject: Semaphores and threads anomaly and bug?

(Please CC me in any reply as I'm not subscribed)

I know this isn't defined anywhere but the seems to be an ambiguity and
discrepancy between versions of Unix and Linux over threads and semaphores.

Do the "SEM_UNDO"s get applied when a thread terminates or when the
"whole thing" terminates?

I tried the following C++ program

#include <iostream>
#include <sys/ipc.h>
#include <sys/sem.h>
#include <pthread.h>

using namespace std;

int semchan;

union semun {
int val; /* value for SETVAL */
struct semid_ds *buf; /* buffer for IPC_STAT, IPC_SET */
unsigned short *array; /* array for GETALL, SETALL */
/* Linux specific part: */
struct seminfo *__buf; /* buffer for IPC_INFO */
};

int getsemv()
{
return semctl(semchan, 0, GETVAL, 0);
}

void *tfunc(void *arg)
{
cout << "About to set sema4 current=" << getsemv() << endl;
sembuf sv;
sv.sem_op = 77;
sv.sem_flg = SEM_UNDO;
sv.sem_num = 0;
semop(semchan, &sv, 1);
cout << "Done,sem now=" << getsemv() << endl;
return 0;
}

int main()
{
semchan = semget(999, 1, 0666 | IPC_CREAT);
semun z;
z.val = 1;
semctl(semchan, 0, SETVAL, z);
cout << "Created sema4 initial value=" << getsemv() << endl;
cout << "Creating thread" << endl;
pthread_t th;
pthread_create(&th, 0, tfunc, 0);
pthread_join(th, 0);
cout << "After thread value=" << getsemv() << endl;
return 0;
}

Trying it on Linux (2.4.21 kernel) it says:

Created sema4 initial value=1
Creating thread
About to set sema4 current=1
Done,sem now=78
After thread value=1

Implying that the thread exit applies the SEM_UNDOs whereas trying on
Solaris 2.9 and HP/UX 11 I get

Created sema4 initial value=1
Creating thread
About to set sema4 current=1
Done,sem now=78
After thread value=78

After which the value is 1, implying that the SEM_UNDOs get applied on
process exit.

There is another anomaly which applies to SEM_UNDO in that when a
process exits, every semaphore in the set has its "sempid" set to the
process id of the exiting process, even ones that the process left
alone. I think that in ipc/sem.c line 1062 the line should be made
conditional on "u->semadj[i]" being non-zero.

I know all this isn't defined anywhere but I hit on this trying to write
an application involving a server process using threads and a
semaphore-protected shared memory segment accessed by client processes.
The semaphore might be set to "locked" by a different thread from the
one that "unlocks" it (I'm using "Mutexes" inside the server process). I
know you probably don't want SEM_UNDO in a fully-debugged server
process, only in the clients but it's all a question of getting the said
server process into the glorious state of being "fully-debugged" or
somewhere near there.

There is a potential problem here in that the code in ipc/sem.c doesn't
allow the adjustment to yield a negative value but what if it starts at
zero, thread A increments it, thread B decrements it back to zero (both
with SEM_UNDO) and thread A exits first? Thread A's undo won't work and
then thread B's undo will increment it again leaving it in an incorrect
state which is different from thread B exiting first.

--
John Collins Xi Software Ltd http://www.xisl.com



2003-11-04 17:30:12

by Linus Torvalds

[permalink] [raw]
Subject: Re: Semaphores and threads anomaly and bug?


On Tue, 4 Nov 2003, John M Collins wrote:
>
> I know this isn't defined anywhere but the seems to be an ambiguity and
> discrepancy between versions of Unix and Linux over threads and semaphores.
>
> Do the "SEM_UNDO"s get applied when a thread terminates or when the
> "whole thing" terminates?

It's entirely up to you. That's what CLONE_SYSVSEM is supposed to
determine.

However, CLONE_SYSVSEM is fairly recent, and I think you will need to use
the new threading libraries to see it.

Linus

2003-11-04 17:59:28

by Manfred Spraul

[permalink] [raw]
Subject: Re: Semaphores and threads anomaly and bug?

John wrote:

>I know this isn't defined anywhere but the seems to be an ambiguity and
>discrepancy between versions of Unix and Linux over threads and semaphores.
>
>Do the "SEM_UNDO"s get applied when a thread terminates or when the
>"whole thing" terminates?
>
>
According to the Unix spec: per-process.
Older Linux kernels applied it per-thread. Newer kernels can handle it
per-process, and AFAIK it's the default for NPTL.

>I think that in ipc/sem.c line 1062 the line should be made
>conditional on "u->semadj[i]" being non-zero.
>
>
Fixed in 2.6. But there is another bug in that block: undos can increase
the semaphore value above SEMVMX.

>There is a potential problem here in that the code in ipc/sem.c doesn't
>allow the adjustment to yield a negative value but what if it starts at
>zero, thread A increments it, thread B decrements it back to zero (both
>with SEM_UNDO) and thread A exits first? Thread A's undo won't work and
>then thread B's undo will increment it again leaving it in an incorrect
>state which is different from thread B exiting first.
>
>
Correct. undo operations should never try to decrease the semaphore
value - an attempt to decrease below 0 is either silently ignored, or
the semaphore value is set to 0.

--
Manfred

2003-11-04 18:09:51

by John M Collins

[permalink] [raw]
Subject: Re: Semaphores and threads anomaly and bug?

On Tuesday 04 November 2003 17:29, Linus Torvalds wrote:
> On Tue, 4 Nov 2003, John M Collins wrote:
> > I know this isn't defined anywhere but the seems to be an ambiguity and
> > discrepancy between versions of Unix and Linux over threads and
> > semaphores.
> >
> > Do the "SEM_UNDO"s get applied when a thread terminates or when the
> > "whole thing" terminates?
>
> It's entirely up to you. That's what CLONE_SYSVSEM is supposed to
> determine.
>
> However, CLONE_SYSVSEM is fairly recent, and I think you will need to use
> the new threading libraries to see it.

Thanks.

Could I possibly draw your attention to the other point I made about the
updating of "sempid" which does seem to be a mistake to me?

--
John Collins Xi Software Ltd http://www.xisl.com