Received-SPF: pass (google.com: domain of linux-nfs+bounces-3311-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <46c36727-ef93-44ca-9741-df2325d4420c@grimberg.me>
Date: Tue, 21 May 2024 17:12:23 +0300
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Safe to delete rpcrdma.ko loading start-up code
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Chuck Lever III <chuck.lever@oracle.com>,
 Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
 "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>
References: <DE53C92C-D16E-4FA7-9C0B-F83F03B1896F@oracle.com>
 <8cc80bdb-9f17-4f44-b2e6-54b36ac85b63@grimberg.me>
 <20240521124306.GE20229@nvidia.com>
 <5b0b8ffe-75ad-4026-a0e8-8d74992ab7b6@grimberg.me>
 <20240521133727.GF20229@nvidia.com>
Content-Language: he-IL, en-US
From: Sagi Grimberg <sagi@grimberg.me>
In-Reply-To: <20240521133727.GF20229@nvidia.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 21/05/2024 16:37, Jason Gunthorpe wrote:
> On Tue, May 21, 2024 at 04:05:05PM +0300, Sagi Grimberg wrote:
>>
>> On 21/05/2024 15:43, Jason Gunthorpe wrote:
>>> On Tue, May 21, 2024 at 12:04:02PM +0300, Sagi Grimberg wrote:
>>>> On 20/05/2024 21:05, Chuck Lever III wrote:
>>>>> Hi-
>>>>>
>>>>> I've tested this with two kinds of systems:
>>>>>
>>>>> 1. A system with no physical RDMA devices and no start-up
>>>>>       scripts to load these modules
>>>>>
>>>>> 2. A system with physical RDMA devices and with the start-up
>>>>>       scripts that load xprtrdma/svcrdma
>>>>>
>>>>> In both cases, after doing an "rmmod rpcrdma", I can mount
>>>>> a "proto=rdma" mount or start the NFS server, and the module
>>>>> gets reloaded automatically.
>>>>>
>>>>> I therefore believe it is safe to delete the code in the
>>>>> rdma-core start-up scripts that manually load RPC-related
>>>>> RDMA support. Either the sunrpc.ko module does this, or NFS
>>>>> user space handles it. There's no need for the rdma-core
>>>>> scripting.
>>>> I didn't know that rdma-core does this... it really shouldn't, the
>>>> mount should (and does) handle it.
>>> This is new, it didn't used to do this
>>>
>>>> I also see that srp(t) and iser(t) are loaded too.. IIRC these are
>>>> loaded by their userspace counterparts as well (or at least they
>>>> should).
>>> And AFIAK, these don't have a way to autoload at all. autoload
>>> requires the kernel to call request_module..
>> nvme/nvmet/isert are requested by the kernel.
> How? What is the interface to trigger request_module?

On the host, writing to the nvme-fabrics misc device a comma-separated 
connection string
contains a transport string, which triggers the corresponding module to 
be requested.

On the target-side, configuring a port transport, also triggers in a 
similar way...

>
>> iser is loaded by iscsiadm.
> Yuk :\

Different strokes for different folks...

>
>> IIRC srp had a userspace daemon loading it.
> srp-daemon requires it already loaded AFAIK

OK, so that one is the exception...