Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Prarit Bhargava <prarit@redhat.com>
Subject: "irq 4: Affinity broken due to vector space exhaustion." warning on
 restart of ttyS0 console
To:     linux-kernel@vger.kernel.org
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
        Marc Zyngier <maz@kernel.org>,
        Andy Lutomirski <luto@kernel.org>,
        Ming Lei <ming.lei@redhat.com>, Peter Xu <peterx@redhat.com>
Message-ID: <3ba26c8d-04ac-1822-d5c2-4a8906f7fd9a@redhat.com>
Date:   Tue, 10 Nov 2020 14:24:44 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.6.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

Occasionally when logging out of the ttyS0 aka serial console I see that

	irq 4: Affinity broken due to vector space exhaustion.

is output to the console.

At boot the default smp_affinity is

/proc/irq/4/smp_affinity:000000ff,ffff0000,00ffffff

The irqbalance service runs and can change this value.  Depending on system load
and behaviour, the IRQ can be assigned to a cpu on socket 1 and the smp_affinity
is changed to socket 1.  In that case,

/proc/irq/4/smp_affinity:ffffff00,0000ffff,ff000000

When the user logs out of the serial console, the console is shut down and IRQ
is free'd.  The IRQ is immediately reacquired by the serial console when it
starts up again.

For example,

Red Hat Enterprise Linux 8.4 Beta (Ootpa)
Kernel 5.10.0-rc2+ on an x86_64

HOSTNAME: intel-whitley-07.khw1.lab.eng.bos.redhat.com
Activate the web console with: systemctl enable --now cockpit.socket

intel-whitley-07 login: root
Password:

Last login: Mon Nov  9 19:13:33 on ttyS0
[07:25 PM root@intel-whitley-07 ~]# exit

*** console shutdown, IRQ released for cpu on socket 1
*** console starts back up again, IRQ assigned to on cpu on socket 0

In this process, however, the smp_affinity is not cleared for IRQ4.  That is, it
remains as

/proc/irq/4/smp_affinity:ffffff00,0000ffff,ff000000

so that the check in activate_reserved() fails and

"irq 4: Affinity broken due to vector space exhaustion."

is output to the screen.

I am not sure of correct fix here.  It looks like the smp_affinity should be
reset to default at irq shutdown, however, I cannot determine if that should be
done for every IRQ, or (hopefully not) per driver.

Can anyone offer guidance on a fix?

P.