Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <YtR8tPTkL/L1kFkY@yury-laptop> <Yt7jfiSlNOeeookP@yury-laptop>
In-Reply-To: <Yt7jfiSlNOeeookP@yury-laptop>
From:   Linus Torvalds <torvalds@linux-foundation.org>
Date:   Mon, 25 Jul 2022 12:02:16 -0700
Message-ID: <CAHk-=wjroqb0Hr9-1BCATjSuBdfdkWS6qqFaLXrwFCsHvgGH_g@mail.gmail.com>
Subject: Re: x86_64/kvm: kernel hangs on poweroff
To:     Yury Norov <yury.norov@gmail.com>
Cc:     Alexander Viro <viro@zeniv.linux.org.uk>,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Jul 25, 2022 at 11:40 AM Yury Norov <yury.norov@gmail.com> wrote:
>
> I didn't investigate on it and I think I'll have no chance to look
> closer at next week or two.

I suspect you will have to bisect it, but it may not even be kernel-specific.

> Just reproduced on v5.19-rc8.

Well, it apparently also reproduces with 5.18, so it's not a regression.

> > [   22.162259] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000000
> > [   22.163539] CPU: 0 PID: 1 Comm: systemd-shutdow Not tainted 5.19.0-rc6 #198
> > [   22.164327] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.15.0-1 04/01/2014
> > [   22.164327] Call Trace:
> > [   22.164327]  <TASK>
> > [   22.164327]  dump_stack_lvl+0x34/0x44
> > [   22.164327]  panic+0x107/0x28a
> > [   22.164327]  do_exit.cold+0x15/0x15
> > [   22.164327]  ? sys_off_notify+0x3e/0x60
> > [   22.164327]  __do_sys_reboot+0x1df/0x220
> > [   22.164327]  ? vfs_writev+0xc7/0x190
> > [   22.164327]  ? virtio_crypto_ctrl_vq_request+0xd0/0xd0
> > [   22.164327]  ? fpregs_assert_state_consistent+0x1e/0x40
> > [   22.164327]  do_syscall_64+0x3b/0x90
> > [   22.164327]  entry_SYSCALL_64_after_hwframe+0x46/0xb0
> > [   22.164327] RIP: 0033:0x7f42b4118443
> > [   22.164327] Code: 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 89 fa be 69 19 12 28 bf ad de e1 fe b8 a9 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 05 c3 0f 1f 40 00 48 8b 15 f9 c9 0d 00 f7 d8

That's literally the "exit()" system call.

So the only reason the kernel is complaining is that your user mode
'init' is calling 'exit()'. Thaty will just make the kernel go "Sorry,
I can't continue, I need init as the child reaper". IOW, it's not a
"kernel bug" kind of panic, it's literally the kernel just going "I'm
done".

The fact that I don't think I've seen anybody else reporting this
makes me think it's something you do.

Without any other reports I can find, and thus no pattern to it, I
think it's on you to figure out when this started happening and what
it is that triggers it, since you're the only one that sees it.

It smells like it's your KVM setup that doesn't recognize the
power-off sequence, so the IO operation that is *supposed* to power
off the machine (and in the case of KVM, make it exit), doesn't do so,
and then that poweroff() thing returns to user space, and then user
space says "I tried to power off, it didn't work, so I'm just going to
exit", and that is what makes the kernel then say "init exited, I
can't continue either"

              Linus