I'm working with a vr4181 target and started digging into the atomic
test and set stuff in the kernel and glibc. The first problem I had was
that the glibc code assumes that all mips III targets implement the mips
III ISA (funny assumption, no?) but the vr4181 doesn't include the
miltiprocessor oriented LL/SC operations for atomic test and set.
So I started looking at the glibc code (yes, I know this is the kernel
list... I'm getting there I promise) and notice the following operations:
__asm__ __volatile__
(".set mips2\n\t"
"/* Inline spinlock test & set */\n\t"
"1:\n\t"
"ll %0,%3\n\t"
".set push\n\t"
".set noreorder\n\t"
"bnez %0,2f\n\t"
" li %1,1\n\t"
".set pop\n\t"
"sc %1,%2\n\t"
"beqz %1,1b\n"
"2:\n\t"
"/* End spinlock test & set */"
: "=&r" (ret), "=&r" (temp), "=m" (*spinlock)
: "m" (*spinlock)
: "memory");
The significant code here being the 'll' and 'sc' operations which are
supposed to ensure that the operation is atomic.
QUESTION 1) Will this _ALWAYS_ work from user land? I realize the
operations are temporally close, but isn't there the possibility that an
interrupt occurs in the meantime?
Of course none of this code applies to my case anyway, since the vr4181
doesn't implement these ops. So once I hack^H^H^H^Hadjust glibc to use
the 'mips1' implementation, it uses the sysmips system call. regard :
_test_and_set (int *p, int v) __THROW
{
return sysmips (MIPS_ATOMIC_SET, (int) p, v, 0);
}
So then I looked at the kernel and find the code below. The system I'm
working with is expressedly uniprocessor and doesn't have any swap, so
it looks like the initial caveats are met, but it looks to me like there
could be some confusion if the value of *arg1 at entry looks like
-ENOSYS or something like that.
QUESTION 2) Wouldn't it be better to pass back the initial value of
*arg1 in *arg3 and return zero or negative error code?
case MIPS_ATOMIC_SET: {
/* This is broken in case of page faults and SMP ...
Risc/OS faults after maximum 20 tries with EAGAIN. */
unsigned int tmp;
p = (int *) arg1;
errno = verify_area(VERIFY_WRITE, p, sizeof(*p));
if (errno)
return errno;
errno = 0;
save_and_cli(flags);
errno |= __get_user(tmp, p);
errno |= __put_user(arg2, p);
restore_flags(flags);
if (errno)
return tmp;
return tmp; /* This is broken ... */
}
QUESTION 3) I notice that the code for this particular case of sysmips
has changed recently. The old code looked more like the 'll/sc' version
of glibc above. I would think that the 'll/sc' code would be better on
SMP systems. Is there a good reason why this reverted?
Sorry For the Long Post (tm)! Thanks In Advance! Merry Xmas!
--
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
On Mon, Dec 25, 2000 at 01:18:48AM -0600, Joe deBlaquiere wrote:
> I'm working with a vr4181 target and started digging into the atomic
> test and set stuff in the kernel and glibc. The first problem I had was
> that the glibc code assumes that all mips III targets implement the mips
> III ISA (funny assumption, no?) but the vr4181 doesn't include the
> miltiprocessor oriented LL/SC operations for atomic test and set.
Ok, but since the kernel disables MIPS III you're limited to MIPS II anyway ...
> So I started looking at the glibc code (yes, I know this is the kernel
> list... I'm getting there I promise) and notice the following operations:
>
> __asm__ __volatile__
> (".set mips2\n\t"
> "/* Inline spinlock test & set */\n\t"
> "1:\n\t"
> "ll %0,%3\n\t"
> ".set push\n\t"
> ".set noreorder\n\t"
> "bnez %0,2f\n\t"
> " li %1,1\n\t"
> ".set pop\n\t"
> "sc %1,%2\n\t"
> "beqz %1,1b\n"
> "2:\n\t"
> "/* End spinlock test & set */"
> : "=&r" (ret), "=&r" (temp), "=m" (*spinlock)
> : "m" (*spinlock)
> : "memory");
>
> The significant code here being the 'll' and 'sc' operations which are
> supposed to ensure that the operation is atomic.
>
> QUESTION 1) Will this _ALWAYS_ work from user land? I realize the
> operations are temporally close, but isn't there the possibility that an
> interrupt occurs in the meantime?
Read the ISA manual; sc will fail if the LL-bit in c0_status is cleared
which will be cleared when the interrupt returns using the eret instruction.
> Of course none of this code applies to my case anyway, since the vr4181
> doesn't implement these ops. So once I hack^H^H^H^Hadjust glibc to use
> the 'mips1' implementation, it uses the sysmips system call. regard :
>
> _test_and_set (int *p, int v) __THROW
> {
> return sysmips (MIPS_ATOMIC_SET, (int) p, v, 0);
> }
>
> So then I looked at the kernel and find the code below. The system I'm
> working with is expressedly uniprocessor and doesn't have any swap, so
> it looks like the initial caveats are met, but it looks to me like there
> could be some confusion if the value of *arg1 at entry looks like
> -ENOSYS or something like that.
Not having swap doesn't mean you're safe. Think of any kind of previously
unmapped page.
> QUESTION 2) Wouldn't it be better to pass back the initial value of
> *arg1 in *arg3 and return zero or negative error code?
The semantics of this syscall were previously defined by Risc/OS and later
on continued to be used by IRIX.
> case MIPS_ATOMIC_SET: {
> /* This is broken in case of page faults and SMP ...
> Risc/OS faults after maximum 20 tries with EAGAIN. */
> unsigned int tmp;
>
> p = (int *) arg1;
> errno = verify_area(VERIFY_WRITE, p, sizeof(*p));
> if (errno)
> return errno;
> errno = 0;
> save_and_cli(flags);
> errno |= __get_user(tmp, p);
> errno |= __put_user(arg2, p);
> restore_flags(flags);
>
> if (errno)
> return tmp;
>
> return tmp; /* This is broken ... */
> }
>
> QUESTION 3) I notice that the code for this particular case of sysmips
> has changed recently. The old code looked more like the 'll/sc' version
> of glibc above. I would think that the 'll/sc' code would be better on
> SMP systems.
Don't think about SMP without ll/sc. There's algorithems available for
that but their complexity leaves them a unpractical, theoretical construct.
> Is there a good reason why this reverted?
Above code will break if the old content of memory has bit 31 set or you take
pagefaults. The latter problem is a problem even on UP - think multi-
threading.
Finally, post such things to one of the MIPS-related mailing lists. If
you're unlucky nobody of the MIPS'ers might see your posting on l-k.
Ralf
Ralf, firstly, thank you for the answers :)
Ralf Baechle wrote:
>
> Ok, but since the kernel disables MIPS III you're limited to MIPS II anyway ...
>
This makes sense...
>
>
> Read the ISA manual; sc will fail if the LL-bit in c0_status is cleared
> which will be cleared when the interrupt returns using the eret instruction.
>
I tried to find a MIPSIII manual from mips.com but all I could find was
mips32 and mips64 (which are not the same as MIPSII/MIPSIII/MIPSIV).
>
>
> Not having swap doesn't mean you're safe. Think of any kind of previously
> unmapped page.
>
Is there a reason why it doesn't just force that page to be mapped first?
>
>> QUESTION 2) Wouldn't it be better to pass back the initial value of
>> *arg1 in *arg3 and return zero or negative error code?
>
>
> The semantics of this syscall were previously defined by Risc/OS and later
> on continued to be used by IRIX.
>
>
>> case MIPS_ATOMIC_SET: {
>> /* This is broken in case of page faults and SMP ...
>> Risc/OS faults after maximum 20 tries with EAGAIN. */
>> unsigned int tmp;
>>
>> p = (int *) arg1;
>> errno = verify_area(VERIFY_WRITE, p, sizeof(*p));
>> if (errno)
>> return errno;
>> errno = 0;
>> save_and_cli(flags);
>> errno |= __get_user(tmp, p);
>> errno |= __put_user(arg2, p);
>> restore_flags(flags);
>>
>> if (errno)
>> return tmp;
>>
>> return tmp; /* This is broken ... */
>> }
>>
>> QUESTION 3) I notice that the code for this particular case of sysmips
>> has changed recently. The old code looked more like the 'll/sc' version
>> of glibc above. I would think that the 'll/sc' code would be better on
>> SMP systems.
>
>
> Don't think about SMP without ll/sc. There's algorithems available for
> that but their complexity leaves them a unpractical, theoretical construct.
>
>
>> Is there a good reason why this reverted?
>
Looking at 2.4.0-test5 I see the ll/sc code, but -test12 doesn't use it.
I was just curious at why it was taken out.
>
> Above code will break if the old content of memory has bit 31 set or you take
> pagefaults. The latter problem is a problem even on UP - think multi-
> threading.
>
> Finally, post such things to one of the MIPS-related mailing lists. If
> you're unlucky nobody of the MIPS'ers might see your posting on l-k.
>
> Ralf
--
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax : (256)-837-3839
Hi!
> > Not having swap doesn't mean you're safe. Think of any kind of previously
> > unmapped page.
> >
>
> Is there a reason why it doesn't just force that page to be mapped
> first?
You can map it in... But background daemon can map it out in the
meantime :-). You'd have to map in and pagelock.
Pavel
--
I'm [email protected]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email protected]
On Tue, 26 Dec 2000, Ralf Baechle wrote:
> The semantics of this syscall were previously defined by Risc/OS and later
> on continued to be used by IRIX.
Ralf, could you please provide me a copy of a man page for the call? I
don't have access to either of the systems and a search of the Net
returned nothing.
> Don't think about SMP without ll/sc. There's algorithems available for
> that but their complexity leaves them a unpractical, theoretical construct.
For SMP there is a simple kernel solution available. It suitable for a
syscall or a ll/sc emulation. There is no easy userland-only solution
AFAIK.
> Above code will break if the old content of memory has bit 31 set or you take
> pagefaults. The latter problem is a problem even on UP - think multi-
> threading.
If the code is written carefully you don't ever get a pagefault that
would break consistency.
--
+ Maciej W. Rozycki, Technical University of Gdansk, Poland +
+--------------------------------------------------------------+
+ e-mail: [email protected], PGP key available +
On Tue, 26 Dec 2000, Joe deBlaquiere wrote:
> > Read the ISA manual; sc will fail if the LL-bit in c0_status is cleared
> > which will be cleared when the interrupt returns using the eret instruction.
>
> I tried to find a MIPSIII manual from mips.com but all I could find was
> mips32 and mips64 (which are not the same as MIPSII/MIPSIII/MIPSIV).
Get "IDT MIPS Microprocessor Software Reference Manual" from
http://decstation.unix-ag.org/docs/ic_docs/3715.pdf (the original is no
longer available from IDT servers).
--
+ Maciej W. Rozycki, Technical University of Gdansk, Poland +
+--------------------------------------------------------------+
+ e-mail: [email protected], PGP key available +