Hello all,
I found that there's a possible small optimization right at the very
beginning of schedule():
if (likely(!current->exit_state)) {
if (unlikely(in_atomic())) {
can be reversed into
if (unlikely(in_atomic())) {
if (likely(!current->exit_state)) {
This is a Good Thing since it avoids having to evaluate both checks,
and both use current_thread_info() which has an inherent AGI stall risk on
x86 CPUs if it cannot be inter-mingled with other unrelated opcodes.
I'm a bit puzzled that this has not been done like that before.
Probably since the exit_state check got added as an after-thought...
Or did I miss some important reason here? (branch prediction??)
Patch against 2.6.16-rc5-mm3.
Thanks!
Signed-off-by: Andreas Mohr <[email protected]>
--- linux-2.6.16-rc5-mm3/kernel/sched.c.orig 2006-03-08 18:36:58.000000000 +0100
+++ linux-2.6.16-rc5-mm3/kernel/sched.c 2006-03-08 18:39:55.000000000 +0100
@@ -3022,8 +3022,8 @@
* schedule() atomically, we ignore that path for now.
* Otherwise, whine if we are scheduling when we should not be.
*/
- if (likely(!current->exit_state)) {
- if (unlikely(in_atomic())) {
+ if (unlikely(in_atomic())) {
+ if (likely(!current->exit_state)) {
printk(KERN_ERR "BUG: scheduling while atomic: "
"%s/0x%08x/%d\n",
current->comm, preempt_count(), current->pid);
--
No programming skills!? Why not help translate many Linux applications!
https://launchpad.ubuntu.com/rosetta
cc'ed Ingo since he's maintainer.
On Thursday 09 March 2006 04:54, Andreas Mohr wrote:
> Hello all,
>
> I found that there's a possible small optimization right at the very
> beginning of schedule():
>
> if (likely(!current->exit_state)) {
> if (unlikely(in_atomic())) {
>
> can be reversed into
>
> if (unlikely(in_atomic())) {
> if (likely(!current->exit_state)) {
>
> This is a Good Thing since it avoids having to evaluate both checks,
> and both use current_thread_info() which has an inherent AGI stall risk on
> x86 CPUs if it cannot be inter-mingled with other unrelated opcodes.
>
> I'm a bit puzzled that this has not been done like that before.
> Probably since the exit_state check got added as an after-thought...
> Or did I miss some important reason here? (branch prediction??)
This looks good. See below.
> Patch against 2.6.16-rc5-mm3.
>
> Thanks!
>
> Signed-off-by: Andreas Mohr <[email protected]>
>
>
> --- linux-2.6.16-rc5-mm3/kernel/sched.c.orig 2006-03-08 18:36:58.000000000
> +0100 +++ linux-2.6.16-rc5-mm3/kernel/sched.c 2006-03-08 18:39:55.000000000
> +0100 @@ -3022,8 +3022,8 @@
> * schedule() atomically, we ignore that path for now.
> * Otherwise, whine if we are scheduling when we should not be.
> */
> - if (likely(!current->exit_state)) {
> - if (unlikely(in_atomic())) {
> + if (unlikely(in_atomic())) {
> + if (likely(!current->exit_state)) {
I suspect that once we're in_atomic() then we're no longer likely to
be !current->exit_state
Probably better to just
if (unlikely(in_atomic())) {
if (!current->exit_state) {
Ingo?
Cheers,
Con
* Con Kolivas <[email protected]> wrote:
> > - if (likely(!current->exit_state)) {
> > - if (unlikely(in_atomic())) {
> > + if (unlikely(in_atomic())) {
> > + if (likely(!current->exit_state)) {
>
> I suspect that once we're in_atomic() then we're no longer likely to
> be !current->exit_state
>
> Probably better to just
> if (unlikely(in_atomic())) {
> if (!current->exit_state) {
>
> Ingo?
yeah. There's not much point in nesting likely/unlikely. In fact we can
just merge the two conditions, as per updated patch below.
Ingo
---
From: Andreas Mohr <[email protected]>
small schedule() microoptimization.
Signed-off-by: Andreas Mohr <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
--- linux/kernel/sched.c.orig
+++ linux/kernel/sched.c
@@ -2873,13 +2873,11 @@ asmlinkage void __sched schedule(void)
* schedule() atomically, we ignore that path for now.
* Otherwise, whine if we are scheduling when we should not be.
*/
- if (likely(!current->exit_state)) {
- if (unlikely(in_atomic())) {
- printk(KERN_ERR "scheduling while atomic: "
- "%s/0x%08x/%d\n",
- current->comm, preempt_count(), current->pid);
- dump_stack();
- }
+ if (unlikely(in_atomic() && !current->exit_state)) {
+ printk(KERN_ERR "scheduling while atomic: "
+ "%s/0x%08x/%d\n",
+ current->comm, preempt_count(), current->pid);
+ dump_stack();
}
profile_hit(SCHED_PROFILING, __builtin_return_address(0));
Hi,
On Fri, Mar 17, 2006 at 10:13:47AM +0100, Ingo Molnar wrote:
>
> * Con Kolivas <[email protected]> wrote:
> > Probably better to just
> > if (unlikely(in_atomic())) {
> > if (!current->exit_state) {
> >
> > Ingo?
>
> yeah. There's not much point in nesting likely/unlikely. In fact we can
> just merge the two conditions, as per updated patch below.
ACK, thanks!
> Ingo
Andreas Mohr