Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Prarit Bhargava <prarit@redhat.com>, linux-kernel@vger.kernel.org
Cc:     Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
        Marc Zyngier <maz@kernel.org>,
        Andy Lutomirski <luto@kernel.org>,
        Ming Lei <ming.lei@redhat.com>, Peter Xu <peterx@redhat.com>
Subject: Re: "irq 4: Affinity broken due to vector space exhaustion." warning on restart of ttyS0 console
In-Reply-To: <3ba26c8d-04ac-1822-d5c2-4a8906f7fd9a@redhat.com>
References: <3ba26c8d-04ac-1822-d5c2-4a8906f7fd9a@redhat.com>
Date:   Tue, 10 Nov 2020 21:56:27 +0100
Message-ID: <871rh1gcck.fsf@nanos.tec.linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

Prarit,

On Tue, Nov 10 2020 at 14:24, Prarit Bhargava wrote:
> Occasionally when logging out of the ttyS0 aka serial console I see that
>
> 	irq 4: Affinity broken due to vector space exhaustion.
>
> *** console shutdown, IRQ released for cpu on socket 1
> *** console starts back up again, IRQ assigned to on cpu on socket 0
>
> In this process, however, the smp_affinity is not cleared for IRQ4.  That is, it
> remains as
>
> /proc/irq/4/smp_affinity:ffffff00,0000ffff,ff000000
>
> so that the check in activate_reserved() fails and
>
> "irq 4: Affinity broken due to vector space exhaustion."
>
> is output to the screen.
>
> I am not sure of correct fix here.  It looks like the smp_affinity should be
> reset to default at irq shutdown, however, I cannot determine if that should be
> done for every IRQ, or (hopefully not) per driver.

This has been that way forever and there was a discussion about this
at least 15 years ago. I can't find it at the moment because I can't
access my archives and I failed to find the conversation on lore.

But here is the gist:

At some point I actually wanted to reset the affinity mask to the
default affinity mask in free_irq() when the last action was
removed.

That broke setups where the affinity of the serial console interrupt,
was set to some specific CPU by the admin and then it was moved to some
other place due to logout -> shutdown -> startup.

So we left the historic behaviour untouched.

So now you are complaining about it the other way round and I have to
tell you that there is no correct fix neither in the core nor in a
driver.

The real problem is irqbalanced aggressively exhausting the vector space
of a _whole_ socket to the point that there is not a single vector left
for serial. That's the problem you want to fix.

Thanks,

        tglx