Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <01af7f4f-bd05-b93e-57ad-c2e9b8726e90@gmail.com> <20201017162949.0a6dd37a@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
In-Reply-To: <20201017162949.0a6dd37a@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
From:   Eric Dumazet <edumazet@google.com>
Date:   Sun, 18 Oct 2020 10:02:41 +0200
Message-ID: <CANn89i+q=q_LNDzE23y74Codh5EY0HHi_tROsEL2yJAdRjh-vQ@mail.gmail.com>
Subject: Re: Remove __napi_schedule_irqoff?
To:     Jakub Kicinski <kuba@kernel.org>
Cc:     Heiner Kallweit <hkallweit1@gmail.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Eric Dumazet <eric.dumazet@gmail.com>,
        David Miller <davem@davemloft.net>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sun, Oct 18, 2020 at 1:29 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Sat, 17 Oct 2020 15:45:57 +0200 Heiner Kallweit wrote:
> > When __napi_schedule_irqoff was added with bc9ad166e38a
> > ("net: introduce napi_schedule_irqoff()") the commit message stated:
> > "Many NIC drivers can use it from their hard IRQ handler instead of
> > generic variant."
>
> Eric, do you think it still matters? Does it matter on x86?
>
> > It turned out that this most of the time isn't safe in certain
> > configurations:
> > - if CONFIG_PREEMPT_RT is set
> > - if command line parameter threadirqs is set
> >
> > Having said that drivers are being switched back to __napi_schedule(),
> > see e.g. patch in [0] and related discussion. I thought about a
> > __napi_schedule version checking dynamically whether interrupts are
> > disabled. But checking e.g. variable force_irqthreads also comes at
> > a cost, so that we may not see a benefit compared to calling
> > local_irq_save/local_irq_restore.
> >
> > If more or less all users have to switch back, then the question is
> > whether we should remove __napi_schedule_irqoff.
> > Instead of touching all users we could make  __napi_schedule_irqoff
> > an alias for __napi_schedule for now.
> >
> > [0] https://lkml.org/lkml/2020/10/8/706
>
> We're effectively calling raise_softirq_irqoff() from IRQ handlers,
> with force_irqthreads == true that's no longer legal.
>
> Thomas - is the expectation that IRQ handlers never assume they have
> IRQs disabled going forward? We don't have any performance numbers
> but if I'm reading Agner's tables right POPF is 18 cycles on Broadwell.
> Is PUSHF/POPF too cheap to bother?
>
> Otherwise a non-solution could be to make IRQ_FORCED_THREADING
> configurable.

I have to say I do not understand why we want to defer to a thread the
hard IRQ that we use in NAPI model.

Whole point of NAPI was to keep hard irq handler very short.

We should focus on transferring the NAPI work (potentially disrupting
) to a thread context, instead of the very minor hard irq trigger.