Received-SPF: pass (google.com: domain of linux-nfs+bounces-3330-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <573f7e57-7599-4540-9dd7-622f7eedde79@linux.dev>
Date: Wed, 22 May 2024 09:57:27 +0200
Precedence: bulk
MIME-Version: 1.0
Subject: Re: Safe to delete rpcrdma.ko loading start-up code
To: Sagi Grimberg <sagi@grimberg.me>, Jason Gunthorpe <jgg@nvidia.com>
Cc: Chuck Lever III <chuck.lever@oracle.com>,
 Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
 "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>
References: <DE53C92C-D16E-4FA7-9C0B-F83F03B1896F@oracle.com>
 <8cc80bdb-9f17-4f44-b2e6-54b36ac85b63@grimberg.me>
 <20240521124306.GE20229@nvidia.com>
 <5b0b8ffe-75ad-4026-a0e8-8d74992ab7b6@grimberg.me>
 <20240521133727.GF20229@nvidia.com>
 <46c36727-ef93-44ca-9741-df2325d4420c@grimberg.me>
 <20240521152325.GG20229@nvidia.com>
 <e558ee64-48fc-48b9-addd-eab7f9f861ad@grimberg.me>
 <20240521163713.GL20229@nvidia.com>
 <0f9ddfe5-67ff-470b-8901-d513dceb757e@grimberg.me>
From: Zhu Yanjun <yanjun.zhu@linux.dev>
In-Reply-To: <0f9ddfe5-67ff-470b-8901-d513dceb757e@grimberg.me>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

在 2024/5/21 22:30, Sagi Grimberg 写道:
> 
> 
> On 21/05/2024 19:37, Jason Gunthorpe wrote:
>> On Tue, May 21, 2024 at 07:10:53PM +0300, Sagi Grimberg wrote:
>>>
>>> On 21/05/2024 18:23, Jason Gunthorpe wrote:
>>>> On Tue, May 21, 2024 at 05:12:23PM +0300, Sagi Grimberg wrote:
>>>>>>>>> I also see that srp(t) and iser(t) are loaded too.. IIRC these are
>>>>>>>>> loaded by their userspace counterparts as well (or at least they
>>>>>>>>> should).
>>>>>>>> And AFIAK, these don't have a way to autoload at all. autoload
>>>>>>>> requires the kernel to call request_module..
>>>>>>> nvme/nvmet/isert are requested by the kernel.
>>>>>> How? What is the interface to trigger request_module?
>>>>> On the host, writing to the nvme-fabrics misc device a comma-separated
>>>>> connection string
>>>>> contains a transport string, which triggers the corresponding 
>>>>> module to be
>>>>> requested.
>>>> But how did nvme-fabrics even get loaded to write to it's config fs in
>>>> the first place?
>>> Something (/etc/modules-load?) loaded it intentionally.
>>> That something knows about a concrete intention to use nvme though...
>> This mechanism we are talking about is an add-on to /etc/modules-load
>> that only executes if rdma HW is present.
> 
> Still does not mean you want to use all the ulps though...
> 
>>
>> This is why it is a good place to load nvme-fabrics stuff, if you
>> don't have rdma HW then you know you don't need it.
> 
> Do I want to autoload nvme-fabrics if I have a nvme device? do I want
> autoload nvme-tcp if I have an ethernet nic? maybe wlan nic is also a
> sufficient reason?
> 
> I just don't see why the presence of an rdma device dictates that all 
> the ulps
> autoload. Does rxe/siw count as rdma HW?

RXE/SIW can be auto loaded with the command "rdma link ...".
And some kernel modules, for example ib_core.ko, udp_tunnel.ko, 
ip6_udp_tunnel.ko and ib_uverbs.ko, are also auto-loaded when rxe/siw 
kernel modules are loaded.

RXE/SIW are emulation RDMA kernel drivers. They are based on NIC HW. 
Normally all the NICs can support RXE/SIW RDMA drivers because RXE/SIW 
do not require additional NIC features.

To now almost all the NIC HW can support RXE/SIW, even some virtual NICs 
can also support RXE/SIW, for example, bonding, TUN and veth.

Zhu Yanjun

> 
>>
>> Autoloading is the version where you do 'mount -tnfs -o=rdma' and the
>> kernel automatically request_module's nfs and then nfs-rdma based only
>> on the command line options.
>>
>> I'm not sure this is even possible with configfs as the directories
>> you need to write into don't even exist until the module(s) are
>> loaded, right?
> 
> Right. The entry-point of the subsystem needs to be loaded (nvmet is 
> loaded by nvmetcli),
> the individual transports/drivers are auto-selected.