Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 4 Aug 2021 15:17:31 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc:     Daniel Wagner <wagi@monom.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        LKML <linux-kernel@vger.kernel.org>,
        linux-rt-users@vger.kernel.org,
        Steven Rostedt <rostedt@goodmis.org>,
        Jens Axboe <axboe@kernel.dk>
Subject: Re: [ANNOUNCE] v5.14-rc4-rt4
Message-ID: <20210804131731.GG8057@worktop.programming.kicks-ass.net>
References: <20210802162750.santic4y6lzcet5c@linutronix.de>
 <20210804082418.fbibprcwtzyt5qax@beryllium.lan>
 <20210804104340.fhdjwn3hruymu3ml@linutronix.de>
 <20210804104803.4nwxi74sa2vwiujd@linutronix.de>
 <20210804110057.chsvt7l5xpw7bo5r@linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20210804110057.chsvt7l5xpw7bo5r@linutronix.de>
Precedence: bulk

On Wed, Aug 04, 2021 at 01:00:57PM +0200, Sebastian Andrzej Siewior wrote:
> On 2021-08-04 12:48:05 [+0200], To Daniel Wagner wrote:
> > On 2021-08-04 12:43:42 [+0200], To Daniel Wagner wrote:
> > > Odd. Do you have a config for that, please?
> > 
> > No need.
> > | [   90.202543] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:35
> > | [   90.202549] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 2047, name: iou-wrk-2041
> > | [   90.202555] CPU: 5 PID: 2047 Comm: iou-wrk-2041 Tainted: G        W         5.14.0-rc4-rt4+ #89
> > | [   90.202561] Call Trace:
> …
> > | [   90.202588]  rt_spin_lock+0x19/0x70
> > | [   90.202593]  ___slab_alloc+0xcb/0x7d0
> …
> > | [   90.202618]  kmem_cache_alloc_trace+0x79/0x1f0
> > | [   90.202621]  io_wqe_dec_running.isra.0+0x98/0xe0
> > | [   90.202625]  io_wq_worker_sleeping+0x37/0x50
> > | [   90.202628]  schedule+0x30/0xd0
> > 
> > le look.
> 
> So this is due to commit
>   685fe7feedb96 ("io-wq: eliminate the need for a manager thread")
> 
> introduced in the v5.13-rc1 merge window. The call chain is
>   schedule()
>    sched_submit_work()
>     preempt_disable();
>     io_wq_worker_sleeping()
>       raw_spin_lock_irq(&worker->wqe->lock);
>       io_wqe_dec_running(worker);
>        io_queue_worker_create()
>         kmalloc(sizeof(*cwd), GFP_ATOMIC);
> 
> The lock wqe::lock has been turned into a raw_spinlock_t in commit
>    95da84659226d ("io_wq: Make io_wqe::lock a raw_spinlock_t")
> 
> after a careful analysis of the code at that time. This commit breaks
> things. Is this really needed?

Urgh, doing allocs from schedule seems really yuck. Can we please not do
this?