Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221226074908.8154-1-jasowang@redhat.com> <20221226074908.8154-4-jasowang@redhat.com>
 <20221226183705-mutt-send-email-mst@kernel.org> <CACGkMEuNZLJRnWw+XNxJ-to1y8L2GrTrJkk0y0Gwb5H2YhDczQ@mail.gmail.com>
 <20221227022255-mutt-send-email-mst@kernel.org> <d77bc1ce-b73f-1ba8-f04f-b3bffeb731c3@redhat.com>
 <20221227043148-mutt-send-email-mst@kernel.org> <0d9f1b89-9374-747b-3fb0-b4b28ad0ace1@redhat.com>
 <CACGkMEv=+D+Es4sfde_X7F0zspVdy4Rs1Wi9qfCudsznsUrOTQ@mail.gmail.com> <20221229020553-mutt-send-email-mst@kernel.org>
In-Reply-To: <20221229020553-mutt-send-email-mst@kernel.org>
From:   Jason Wang <jasowang@redhat.com>
Date:   Thu, 29 Dec 2022 16:04:13 +0800
Message-ID: <CACGkMEs5s3Muo+4OfjaLK_P76rTdPhjQdTwykRNGOecAWnt+8g@mail.gmail.com>
Subject: Re: [PATCH 3/4] virtio_ring: introduce a per virtqueue waitqueue
To:     "Michael S. Tsirkin" <mst@redhat.com>
Cc:     davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
        pabeni@redhat.com, virtualization@lists.linux-foundation.org,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        maxime.coquelin@redhat.com, alvaro.karsz@solid-run.com,
        eperezma@redhat.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Thu, Dec 29, 2022 at 3:07 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Dec 28, 2022 at 07:53:08PM +0800, Jason Wang wrote:
> > On Wed, Dec 28, 2022 at 2:34 PM Jason Wang <jasowang@redhat.com> wrote:
> > >
> > >
> > > =E5=9C=A8 2022/12/27 17:38, Michael S. Tsirkin =E5=86=99=E9=81=93:
> > > > On Tue, Dec 27, 2022 at 05:12:58PM +0800, Jason Wang wrote:
> > > >> =E5=9C=A8 2022/12/27 15:33, Michael S. Tsirkin =E5=86=99=E9=81=93:
> > > >>> On Tue, Dec 27, 2022 at 12:30:35PM +0800, Jason Wang wrote:
> > > >>>>> But device is still going and will later use the buffers.
> > > >>>>>
> > > >>>>> Same for timeout really.
> > > >>>> Avoiding infinite wait/poll is one of the goals, another is to s=
leep.
> > > >>>> If we think the timeout is hard, we can start from the wait.
> > > >>>>
> > > >>>> Thanks
> > > >>> If the goal is to avoid disrupting traffic while CVQ is in use,
> > > >>> that sounds more reasonable. E.g. someone is turning on promisc,
> > > >>> a spike in CPU usage might be unwelcome.
> > > >>
> > > >> Yes, this would be more obvious is UP is used.
> > > >>
> > > >>
> > > >>> things we should be careful to address then:
> > > >>> 1- debugging. Currently it's easy to see a warning if CPU is stuc=
k
> > > >>>      in a loop for a while, and we also get a backtrace.
> > > >>>      E.g. with this - how do we know who has the RTNL?
> > > >>>      We need to integrate with kernel/watchdog.c for good results
> > > >>>      and to make sure policy is consistent.
> > > >>
> > > >> That's fine, will consider this.
> >
> > So after some investigation, it seems the watchdog.c doesn't help. The
> > only export helper is touch_softlockup_watchdog() which tries to avoid
> > triggering the lockups warning for the known slow path.
>
> I never said you can just use existing exporting APIs. You'll have to
> write new ones :)

Ok, I thought you wanted to trigger similar warnings as a watchdog.

Btw, I wonder what kind of logic you want here. If we switch to using
sleep, there won't be soft lockup anymore. A simple wait + timeout +
warning seems sufficient?

Thanks

>
> > And before the patch, we end up with a real infinite loop which could
> > be caught by RCU stall detector which is not the case of the sleep.
> > What we can do is probably do a periodic netdev_err().
> >
> > Thanks
>
> Only with a bad device.
>
> > > >>
> > > >>
> > > >>> 2- overhead. In a very common scenario when device is in hypervis=
or,
> > > >>>      programming timers etc has a very high overhead, at bootup
> > > >>>      lots of CVQ commands are run and slowing boot down is not ni=
ce.
> > > >>>      let's poll for a bit before waiting?
> > > >>
> > > >> Then we go back to the question of choosing a good timeout for pol=
l. And
> > > >> poll seems problematic in the case of UP, scheduler might not have=
 the
> > > >> chance to run.
> > > > Poll just a bit :) Seriously I don't know, but at least check once
> > > > after kick.
> > >
> > >
> > > I think it is what the current code did where the condition will be
> > > check before trying to sleep in the wait_event().
> > >
> > >
> > > >
> > > >>> 3- suprise removal. need to wake up thread in some way. what abou=
t
> > > >>>      other cases of device breakage - is there a chance this
> > > >>>      introduces new bugs around that? at least enumerate them ple=
ase.
> > > >>
> > > >> The current code did:
> > > >>
> > > >> 1) check for vq->broken
> > > >> 2) wakeup during BAD_RING()
> > > >>
> > > >> So we won't end up with a never woke up process which should be fi=
ne.
> > > >>
> > > >> Thanks
> > > >
> > > > BTW BAD_RING on removal will trigger dev_err. Not sure that is a go=
od
> > > > idea - can cause crashes if kernel panics on error.
> > >
> > >
> > > Yes, it's better to use __virtqueue_break() instead.
> > >
> > > But consider we will start from a wait first, I will limit the change=
s
> > > in virtio-net without bothering virtio core.
> > >
> > > Thanks
> > >
> > >
> > > >
> > > >>>
>