Received-SPF: pass (google.com: domain of linux-nfs+bounces-3326-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <0f9ddfe5-67ff-470b-8901-d513dceb757e@grimberg.me>
Date: Tue, 21 May 2024 23:30:00 +0300
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Safe to delete rpcrdma.ko loading start-up code
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Chuck Lever III <chuck.lever@oracle.com>,
 Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
 "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>
References: <DE53C92C-D16E-4FA7-9C0B-F83F03B1896F@oracle.com>
 <8cc80bdb-9f17-4f44-b2e6-54b36ac85b63@grimberg.me>
 <20240521124306.GE20229@nvidia.com>
 <5b0b8ffe-75ad-4026-a0e8-8d74992ab7b6@grimberg.me>
 <20240521133727.GF20229@nvidia.com>
 <46c36727-ef93-44ca-9741-df2325d4420c@grimberg.me>
 <20240521152325.GG20229@nvidia.com>
 <e558ee64-48fc-48b9-addd-eab7f9f861ad@grimberg.me>
 <20240521163713.GL20229@nvidia.com>
Content-Language: he-IL, en-US
From: Sagi Grimberg <sagi@grimberg.me>
In-Reply-To: <20240521163713.GL20229@nvidia.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 21/05/2024 19:37, Jason Gunthorpe wrote:
> On Tue, May 21, 2024 at 07:10:53PM +0300, Sagi Grimberg wrote:
>>
>> On 21/05/2024 18:23, Jason Gunthorpe wrote:
>>> On Tue, May 21, 2024 at 05:12:23PM +0300, Sagi Grimberg wrote:
>>>>>>>> I also see that srp(t) and iser(t) are loaded too.. IIRC these are
>>>>>>>> loaded by their userspace counterparts as well (or at least they
>>>>>>>> should).
>>>>>>> And AFIAK, these don't have a way to autoload at all. autoload
>>>>>>> requires the kernel to call request_module..
>>>>>> nvme/nvmet/isert are requested by the kernel.
>>>>> How? What is the interface to trigger request_module?
>>>> On the host, writing to the nvme-fabrics misc device a comma-separated
>>>> connection string
>>>> contains a transport string, which triggers the corresponding module to be
>>>> requested.
>>> But how did nvme-fabrics even get loaded to write to it's config fs in
>>> the first place?
>> Something (/etc/modules-load?) loaded it intentionally.
>> That something knows about a concrete intention to use nvme though...
> This mechanism we are talking about is an add-on to /etc/modules-load
> that only executes if rdma HW is present.

Still does not mean you want to use all the ulps though...

>
> This is why it is a good place to load nvme-fabrics stuff, if you
> don't have rdma HW then you know you don't need it.

Do I want to autoload nvme-fabrics if I have a nvme device? do I want
autoload nvme-tcp if I have an ethernet nic? maybe wlan nic is also a
sufficient reason?

I just don't see why the presence of an rdma device dictates that all 
the ulps
autoload. Does rxe/siw count as rdma HW?

>
> Autoloading is the version where you do 'mount -tnfs -o=rdma' and the
> kernel automatically request_module's nfs and then nfs-rdma based only
> on the command line options.
>
> I'm not sure this is even possible with configfs as the directories
> you need to write into don't even exist until the module(s) are
> loaded, right?

Right. The entry-point of the subsystem needs to be loaded (nvmet is 
loaded by nvmetcli),
the individual transports/drivers are auto-selected.