Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 6 Jun 2023 21:27:36 +0200
From:   Guillaume Nault <gnault@redhat.com>
To:     Mirsad Goran Todorovac <mirsad.todorovac@alu.unizg.hr>
Cc:     netdev@vger.kernel.org, "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>, Shuah Khan <shuah@kernel.org>,
        linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org
Subject: Re: POSSIBLE BUG: selftests/net/fcnal-test.sh: [FAIL] in vrf "bind -
 ns-B IPv6 LLA" test
Message-ID: <ZH+IqBNan5nAFMZZ@debian>
References: <b6191f90-ffca-dbca-7d06-88a9788def9c@alu.unizg.hr>
 <ZHeN3bg28pGFFjJN@debian>
 <a379796a-5cd6-caa7-d11d-5ffa7419b90e@alu.unizg.hr>
 <ZH84zGEODT97TEXG@debian>
 <60f78eaa-ace7-c27d-8e45-4777ecf3faa2@alu.unizg.hr>
 <ZH8+jLjottBw2zuD@debian>
 <12c34bed-0885-3bb3-257f-3b2438ba206f@alu.unizg.hr>
 <ZH+ADF0OOcmtUPw9@debian>
 <174c6928-3498-8fb0-9f83-b01fa346a221@alu.unizg.hr>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <174c6928-3498-8fb0-9f83-b01fa346a221@alu.unizg.hr>
Precedence: bulk

On Tue, Jun 06, 2023 at 09:17:24PM +0200, Mirsad Goran Todorovac wrote:
> On 6/6/23 20:50, Guillaume Nault wrote:
> > On Tue, Jun 06, 2023 at 04:28:02PM +0200, Mirsad Todorovac wrote:
> > > On 6/6/23 16:11, Guillaume Nault wrote:
> > > > On Tue, Jun 06, 2023 at 03:57:35PM +0200, Mirsad Todorovac wrote:
> > > > > +       if (oif) {
> > > > > +               rcu_read_lock();
> > > > > +               dev = dev_get_by_index_rcu(net, oif);
> > > > > +               rcu_read_unlock();
> > > > 
> > > > You can't assume '*dev' is still valid after rcu_read_unlock() unless
> > > > you hold a reference on it.
> > > > 
> > > > > +               rtnl_lock();
> > > > > +               mdev = netdev_master_upper_dev_get(dev);
> > > > > +               rtnl_unlock();
> > > > 
> > > > Because of that, 'dev' might have already disappeared at the time
> > > > netdev_master_upper_dev_get() is called. So it may dereference an
> > > > invalid pointer here.
> > > 
> > > Good point, thanks. I didn't expect those to change.
> > > 
> > > This can be fixed, provided that RCU and RTNL locks can be nested:
> > 
> > Well, yes and no. You can call rcu_read_{lock,unlock}() while under the
> > rtnl protection, but not the other way around.
> > 
> > >          rcu_read_lock();
> > >          if (oif) {
> > >                  dev = dev_get_by_index_rcu(net, oif);
> > >                  rtnl_lock();
> > >                  mdev = netdev_master_upper_dev_get(dev);
> > >                  rtnl_unlock();
> > >          }
> > 
> > This is invalid: rtnl_lock() uses a mutex, so it can sleep and that's
> > forbidden inside an RCU critical section.
> 
> Obviously, that's bad. Mea culpa.
> 
> > >          if (sk->sk_bound_dev_if) {
> > >                  bdev = dev_get_by_index_rcu(net, sk->sk_bound_dev_if);
> > >          }
> > > 
> > >          addr_type = ipv6_addr_type(daddr);
> > >          if ((__ipv6_addr_needs_scope_id(addr_type) && !oif) ||
> > >              (addr_type & IPV6_ADDR_MAPPED) ||
> > >              (oif && sk->sk_bound_dev_if && oif != sk->sk_bound_dev_if &&
> > >                      !(mdev && sk->sk_bound_dev_if && bdev && mdev == bdev))) {
> > >                  rcu_read_unlock();
> > >                  return -EINVAL;
> > > 	}
> > >          rcu_read_unlock();
> > > 
> > > But again this is still probably not race-free (bdev might also disappear before
> > > the mdev == bdev test), even if it passed fcnal-test.sh, there is much duplication
> > > of code, so your one-line solution is obviously by far better. :-)
> > 
> > The real problem is choosing the right function for getting the master
> > device. In particular netdev_master_upper_dev_get() was a bad choice.
> > It forces you to take the rtnl, which is unnatural here and obliges you
> > to add extra code, while all this shouldn't be necessary in the first
> > place.
> 
> Thank you for the additional insight. I had poor luck with Googling on
> these.
> 
> I made a blunder after blunder. But it was insightful and brainstorming.
> Good exercise for my little grey cells.
> 
> However, learning without making any errors appears to be simply a lot
> of blunt memorising. :-/
> 
> It's good to be in an environment when one can learn from errors.
> 
> :-)

I'm happy you found this useful.

> Regards,
> Mirsad
>