Dear list,
I am in the process of learning how to write linux device drivers.
I have a 2.6.16.5 kernel running on a monoprocessor machine.
#CONFIG_SMP is not set
CONFIG_DEBUG_SPINLOCK=y.
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
First question:
I wrote
while (1)
;
in the read function of a test device driver. I expect the calling
process to freeze, and then a timer interrupt to preempt the kernel
and to schedule another process. This does not happen, the whole
system freezes. I see no effect from pressing keys or moving the
mouse. Why? The hardware interrupts are not disabled, are they? Why do
the interrupt handlers not get executed?
Second question:
I wrote
spin_lock(&lck);
down(&sem); /* I know that one shouldn't sleep when holding a lock */
/* but I want to understand why */
spin_unlock(&lck);
in the read function and
up(&sem)
in the write function. The semaphore is initially locked, so the first
process invoking down will sleep.
I invoke
cat /dev/test
and the process sleeps on the semaphore. Then I invoke
echo 1 > /dev/test
and I wake up the "cat" process.
Then I intend to invoke _two_ cat processes. I expect the first one to
sleep on the semaphore and the second on to spin at the spin_lock.
Then I expect to wake up the first process by invoking an echo, the
first process to release the lock and the second process to sleep on
the semaphore. What I get is that the system freezes as soon as I
invoke the second "cat" process. Again, no effect from key presses or
mouse movements. Why? Shouldn't the timer interrupt preempt the second
"cat" process that spins on the spinlock and give control to something
else, for example to the console where I could wake up the first "cat"
process? Why do I not see any effect from mouse movements? Hardware
interrupts are not disabled, are they?
Third question:
The Linux Device Drivers book says that a spin_lock should not be
shared between a process and an interrupt handler. The explanation is
that the process may hold the lock, an interrupt occurs, the interrupt
handler spins on the lock held by the process and the system freezes.
Why should it freeze? Isn't it possible for the interrupt handler to
re-enable interrupts as its first thing, then to spin at the lock, the
timer interrupt to preempt the interrupt handler and to relinquish
control to the process which in turn will finish its critical section
and release the lock, making way for the interrupt handler to
continue.
Thank you very much for clarifying these issues.
Regards,
Sorin
On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <[email protected]> wrote:
> The Linux Device Drivers book says that a spin_lock should not be
> shared between a process and an interrupt handler. The explanation is
> that the process may hold the lock, an interrupt occurs, the interrupt
> handler spins on the lock held by the process and the system freezes.
> Why should it freeze? Isn't it possible for the interrupt handler to
> re-enable interrupts as its first thing, then to spin at the lock, the
> timer interrupt to preempt the interrupt handler and to relinquish
> control to the process which in turn will finish its critical section
> and release the lock, making way for the interrupt handler to
> continue.
Iterrupt handlers are executend in the process context (on top of the
process that they interrupted).
So, if you have a proccess A that does:
Usual Kernel Code Interrupt Handler
...
spin_lock(my_lock);
...
-------interrupt-----> ...
spin_lock(my_lock); // deadlock!
...
<------ back ---------
----
spin_unlock(my_lock);
See?
If the interrupt comes in when process A is running and holding the
lock PREEMPTION can't do anything.
--
Paolo Ornati
Linux 2.6.20-rc1-g99f5e971 on x86_64
On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <[email protected]> wrote:
> spin_lock(&lck);
> down(&sem); /* I know that one shouldn't sleep when holding a lock */
> /* but I want to understand why */
I suppose because the lock is held for an indefinite amount of time and
any other process that try to get that lock will "spin" and burn CPU
without doing anything useful (locking the process in kernel mode and
preventing the execution of other processes on that CPU if there
isn't any type of PREEMPTION).
:)
spin_lock is a "while(1) {...}" thing...
--
Paolo Ornati
Linux 2.6.20-rc1-g99f5e971 on x86_64
On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <[email protected]> wrote:
> while (1)
> ;
>
> in the read function of a test device driver. I expect the calling
> process to freeze, and then a timer interrupt to preempt the kernel
> and to schedule another process. This does not happen, the whole
> system freezes. I see no effect from pressing keys or moving the
> mouse. Why? The hardware interrupts are not disabled, are they? Why do
> the interrupt handlers not get executed?
Here I'm not sure. I think that interrupts are enabled and processed
correctly BUT the process cannot be Preempted because there is some
lock held (every lock will increment the preemption count).
The mouse doesn't move because X cannot be executed...
A quick test you can do is to enable CONFIG_MAGIC_SYSRQ and try with
"ALT + Stamp + B" when the system "freezes". If it reboots then
interrupts work :)
--
Paolo Ornati
Linux 2.6.20-rc1-g99f5e971 on x86_64
Sorin Manolache wrote:
>
> The Linux Device Drivers book says that a spin_lock should not be
> shared between a process and an interrupt handler. The explanation is
> that the process may hold the lock, an interrupt occurs, the interrupt
> handler spins on the lock held by the process and the system freezes.
> Why should it freeze? Isn't it possible for the interrupt handler to
> re-enable interrupts as its first thing, then to spin at the lock, the
> timer interrupt to preempt the interrupt handler and to relinquish
> control to the process which in turn will finish its critical section
> and release the lock, making way for the interrupt handler to
> continue.
When the timer interrupt finishes, it's not going to return control to
the process, it's going to return control to what it interrupted which
is the interrupt handler in your code which will just continue spinning.
Interrupt handlers are not preemptible by anything other than other
interrupt handlers (and then only in some cases) so it will sit spinning
on that lock forever.
--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/
Hi,
Well, Spin lock itself is a while loop. So in case a process on another
CPU has the semaphore and you get a spin lock and try to wait on a
semaphore and the other CPU also tries to get the spin lock then you are
in a dead loop.
CPU 1 CPU 2
Get sem-1 (success)
......
Spin lock (success) .......
... .....
... Spin lock??? No you are locked....
Get sem -1 (block)
You are blocked now
That is the reason we always make sure that we never do any thing that
can sleep in a place after we have spin lock.
And as Paolo tells, all these if PREEMPTION is disabled. But in any case
if the signals are not handled then the processes are blocked.
Thanks and Best Regards,
KK
-----Original Message-----
From: Paolo Ornati [mailto:[email protected]]
Sent: Thursday, December 21, 2006 2:20 AM
To: Sorin Manolache
Cc: [email protected]
Subject: Re: newbie questions about while (1) in kernel mode and
spinlocks
On Thu, 21 Dec 2006 10:41:44 +0100
"Sorin Manolache" <[email protected]> wrote:
> spin_lock(&lck);
> down(&sem); /* I know that one shouldn't sleep when holding a lock */
> /* but I want to understand why */
I suppose because the lock is held for an indefinite amount of time and
any other process that try to get that lock will "spin" and burn CPU
without doing anything useful (locking the process in kernel mode and
preventing the execution of other processes on that CPU if there
isn't any type of PREEMPTION).
:)
spin_lock is a "while(1) {...}" thing...
--
Paolo Ornati
Linux 2.6.20-rc1-g99f5e971 on x86_64
On Thu, 2006-12-21 at 10:41 +0100, Sorin Manolache wrote:
> Dear list,
>
> I am in the process of learning how to write linux device drivers.
>
> I have a 2.6.16.5 kernel running on a monoprocessor machine.
We usually call that a Uniprocessor or just UP.
> #CONFIG_SMP is not set
> CONFIG_DEBUG_SPINLOCK=y.
> CONFIG_PREEMPT=y
> CONFIG_PREEMPT_BKL=y
>
> First question:
>
> I wrote
>
> while (1)
> ;
>
> in the read function of a test device driver. I expect the calling
> process to freeze, and then a timer interrupt to preempt the kernel
> and to schedule another process. This does not happen, the whole
> system freezes. I see no effect from pressing keys or moving the
> mouse. Why? The hardware interrupts are not disabled, are they? Why do
> the interrupt handlers not get executed?
Matters what you have in that code. Are you sure interrupts are not
handled? The can be, and you just don't notice, because the programs
that get affected by the interrupts are not able to run.
I don't know what your read function looks like, or how you got there,
but a while(1) would only slow the system down quite a bit. It shouldn't
lock it up. (/me goes to try it out on a dummy driver after writing
this).
>
> Second question:
>
> I wrote
>
> spin_lock(&lck);
> down(&sem); /* I know that one shouldn't sleep when holding a lock */
> /* but I want to understand why */
Well a spin_lock is just that. It spins. What happens if you schedule,
and the next process that goes to run tries to grab that same spin_lock.
It spins, thinking the lock holder is on another CPU and it will be
released shortly. But then, the other CPU (assuming a 2x system) has a
process that tries to grab this same spin_lock, now the system is truely
dead locked. All CPUS are spinning waiting for the non-running process
to let go of the spin lock.
> spin_unlock(&lck);
>
> in the read function and
>
> up(&sem)
>
> in the write function. The semaphore is initially locked, so the first
> process invoking down will sleep.
>
> I invoke
>
> cat /dev/test
>
> and the process sleeps on the semaphore. Then I invoke
>
> echo 1 > /dev/test
>
> and I wake up the "cat" process.
>
> Then I intend to invoke _two_ cat processes. I expect the first one to
> sleep on the semaphore and the second on to spin at the spin_lock.
> Then I expect to wake up the first process by invoking an echo, the
> first process to release the lock and the second process to sleep on
> the semaphore. What I get is that the system freezes as soon as I
> invoke the second "cat" process. Again, no effect from key presses or
> mouse movements. Why? Shouldn't the timer interrupt preempt the second
> "cat" process that spins on the spinlock and give control to something
> else, for example to the console where I could wake up the first "cat"
> process? Why do I not see any effect from mouse movements? Hardware
> interrupts are not disabled, are they?
A spin_lock will not preempt. So if you are doing this on a UP system, a
spin lock will only be a preempt disable (with CONFIG_PREEMPT=y).
There's no need for spin_locks for UP.
I'd have to take a look at the actual code to explain exactly what mess
you are making for yourself ;)
>
> Third question:
>
> The Linux Device Drivers book says that a spin_lock should not be
> shared between a process and an interrupt handler. The explanation is
> that the process may hold the lock, an interrupt occurs, the interrupt
> handler spins on the lock held by the process and the system freezes.
> Why should it freeze? Isn't it possible for the interrupt handler to
> re-enable interrupts as its first thing, then to spin at the lock, the
> timer interrupt to preempt the interrupt handler and to relinquish
> control to the process which in turn will finish its critical section
> and release the lock, making way for the interrupt handler to
> continue.
I believe that Paolo explained this, but I can go into more details if
you need.
>
> Thank you very much for clarifying these issues.
No prob.
-- Steve