Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 28 Apr 2023 11:03:47 -0300
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Chuck Lever III <chuck.lever@oracle.com>
Cc:     Chuck Lever <cel@kernel.org>, Bernard Metzler <BMT@zurich.ibm.com>,
        linux-rdma <linux-rdma@vger.kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH RFC] RDMA/core: Store zero GIDs in some cases
Message-ID: <ZEvSQzOjhwEYi6m0@ziepe.ca>
References: <168261567323.5727.12145565111706096503.stgit@oracle-102.nfsv4bat.org>
 <ZEvMo4qkj9NSLXTA@ziepe.ca>
 <34E28C03-5D1A-4DAA-9B5B-D453F8C256BD@oracle.com>
 <ZEvOec75yMrin/hB@ziepe.ca>
 <36CE272E-15F4-40B3-83E8-98BCFE55CA20@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <36CE272E-15F4-40B3-83E8-98BCFE55CA20@oracle.com>
Precedence: bulk

On Fri, Apr 28, 2023 at 01:58:53PM +0000, Chuck Lever III wrote:
> 
> 
> > On Apr 28, 2023, at 9:47 AM, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > 
> > On Fri, Apr 28, 2023 at 01:42:24PM +0000, Chuck Lever III wrote:
> >> 
> >> 
> >>> On Apr 28, 2023, at 9:39 AM, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >>> 
> >>> On Thu, Apr 27, 2023 at 01:14:43PM -0400, Chuck Lever wrote:
> >>>> From: Bernard Metzler <bmt@zurich.ibm.com>
> >>>> 
> >>>> Tunnel devices have zero GIDs, so skip the zero GID check when
> >>>> setting up soft iWARP over a tunnel device.
> >>> 
> >>> Huh? Why? How does that make any sense?
> >> 
> >> Read it as a cry for help.
> >> 
> >> The scenario is attempting to set up a soft iWARP device
> >> with a slave that is a tunnel device. The set up seems to
> >> work, but when connecting, the ULP gets an ADDR_ERROR
> >> because the setup did not add an entry to the GID table.
> > 
> > Don't assign a 0 IP to the tunnel?
> 
> That's a little cryptic... can you expand?
> 
> Right now I have a Tailscale VPN device with assigned IP
> addresses:
> 
> 3: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 qdisc fq_codel state UNKNOWN group default qlen 500
>     link/none      inet 100.64.0.16/32 scope global tailscale0
>        valid_lft forever preferred_lft forever
>     inet6 fd7a:115c:a1e0::10/128 scope global         valid_lft forever preferred_lft forever
>     inet6 fe80::725c:1b6d:60ed:fce4/64 scope link stable-privacy         valid_lft forever preferred_lft forever
> 
> And after that i/f is UP, I've done this:

That seems OK..

>  $ sudo rdma link add siw0 type siw netdev tailscale0
>
> With the patch I sent, I can do NFS/RDMA via soft iWARP through
> the tunnel. I'm not at all claiming that's a good fix, but only
> that this scenario is supposed to work, but currently doesn't.

Then there is something wrong in SIW, it should not be reporting 0
GIDs to the core code for that kind of device.

I don't remember what iwarp uses for it's guid format .. Maybe it was
mac adress or something and tunnels don't have a MAC, it should make
up a dummy GID for the tunnel instead of using 0..

Jason