Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20190507173329.17031-1-daniel.vetter@ffwll.ch>
 <20190508074420.GB15704@jagdpanzerIV> <20190508075302.GC15704@jagdpanzerIV>
In-Reply-To: <20190508075302.GC15704@jagdpanzerIV>
From:   Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Wed, 8 May 2019 10:06:20 +0200
Message-ID: <CAKMK7uFeRmSGkqFj-xmdebwKok9+z1pyDZWUMNXfzTT4H2=-fA@mail.gmail.com>
Subject: Re: [PATCH] RFC: x86/smp: use printk_deferred in native_smp_send_reschedule
To:     Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Cc:     Intel Graphics Development <intel-gfx@lists.freedesktop.org>,
        Daniel Vetter <daniel.vetter@intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Will Deacon <will.deacon@arm.com>,
        Petr Mladek <pmladek@suse.com>,
        Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        John Ogness <john.ogness@linutronix.de>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Nicolai Stange <nstange@suse.de>,
        Thomas Gleixner <tglx@linutronix.de>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, May 8, 2019 at 9:53 AM Sergey Senozhatsky
<sergey.senozhatsky.work@gmail.com> wrote:
>
> On (05/08/19 16:44), Sergey Senozhatsky wrote:
> > [..]
> > >  static void native_smp_send_reschedule(int cpu)
> > >  {
> > >     if (unlikely(cpu_is_offline(cpu))) {
> > > -           WARN(1, "sched: Unexpected reschedule of offline CPU#%d!\n", cpu);
> > > +           printk_deferred(KERN_WARNING
> > > +                           "sched: Unexpected reschedule of offline CPU#%d!\n", cpu);
> > >             return;
> > >     }
> > >     apic->send_IPI(cpu, RESCHEDULE_VECTOR);
> >
> > Hmm,
> > One thing to notice here is that the CPU in question is offline-ed,
> > and printk_deferred() is a per-CPU type of deferred printk(). So the
> > following thing
> >
> >       __this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT);
> >       irq_work_queue(this_cpu_ptr(&wake_up_klogd_work));
> >
> > might not print anything at all. In this particular case we always
> > need another CPU to do console_unlock(), since this_cpu() is not
> > really expected to do wake_up_klogd_work_func()->console_unlock().
>
> D'oh... It's remote CPU which is offline, not this_cpu().
> Sorry, my bad!

Well I started reading, then freaked out about the WARN_ON in
irq_work_queue_on until I realized that's not the one we're calling
either :-)

> Any printk-related patch in this area will make PeterZ really-really
> angry :)

Hm any more context for someone with no clue about this? Just that the
dependencies are already terribly complex and it's not going to get
better, or something more specific?

> printk_deferred(), just like prinkt_safe(), depends on IRQ work;
> printk_safe(), however, can redirect multiple lines, unlike
> printk_deferred(). So if you want to keep the backtrace, you may
> do something like
>
>         if (unlikely(cpu_is_offline(cpu))) {
>                 printk_safe_enter(...);
>                 WARN(1, "sched: Unexpected reschedule of offline CPU#%d!\n",
>                          cpu);
>                 printk_safe_exit(...);
>                 return;
>         }
>
> I think, in this case John's reworked-printk can do better than
> printk_safe/printk_deferred.

Hm I think this is what Petr was suggesting, but somehow I didn't find
the printk_safe_* functions and didn't connect the dots. Needs the
_irqsave variants I guess, I'll respin a v2 of this.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch