Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <581ecec9-8f2b-32a2-6ce1-63551496e332@grimberg.me>
Date:   Wed, 17 Aug 2022 21:24:44 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH] nvmet-tcp: Don't kmap() pages which can't come from
 HIGHMEM
Content-Language: en-US
To:     Keith Busch <kbusch@kernel.org>,
        "Fabio M. De Francesco" <fmdefrancesco@gmail.com>
Cc:     Christoph Hellwig <hch@lst.de>,
        Chaitanya Kulkarni <kch@nvidia.com>,
        linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
        Chaitanya Kulkarni <chaitanyak@nvidia.com>,
        Ira Weiny <ira.weiny@intel.com>
References: <20220817170427.15682-1-fmdefrancesco@gmail.com>
 <Yv0o9MlqCjXUcDKc@kbusch-mbp.dhcp.thefacebook.com>
From:   Sagi Grimberg <sagi@grimberg.me>
In-Reply-To: <Yv0o9MlqCjXUcDKc@kbusch-mbp.dhcp.thefacebook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk


>> @@ -329,7 +312,6 @@ static void nvmet_tcp_map_pdu_iovec(struct nvmet_tcp_cmd *cmd)
>>   	u32 length, offset, sg_offset;
>>   
>>   	length = cmd->pdu_len;
>> -	cmd->nr_mapped = DIV_ROUND_UP(length, PAGE_SIZE);
>>   	offset = cmd->rbytes_done;
>>   	cmd->sg_idx = offset / PAGE_SIZE;
>>   	sg_offset = offset % PAGE_SIZE;
>> @@ -338,7 +320,7 @@ static void nvmet_tcp_map_pdu_iovec(struct nvmet_tcp_cmd *cmd)
>>   	while (length) {
>>   		u32 iov_len = min_t(u32, length, sg->length - sg_offset);
>>   
>> -		iov->iov_base = kmap(sg_page(sg)) + sg->offset + sg_offset;
>> +		iov->iov_base = page_address(sg_page(sg)) + sg->offset + sg_offset;
>>   		iov->iov_len = iov_len;
>>   
>>   		length -= iov_len;
>> @@ -347,8 +329,7 @@ static void nvmet_tcp_map_pdu_iovec(struct nvmet_tcp_cmd *cmd)
>>   		sg_offset = 0;
>>   	}
>>   
>> -	iov_iter_kvec(&cmd->recv_msg.msg_iter, READ, cmd->iov,
>> -		cmd->nr_mapped, cmd->pdu_len);
>> +	iov_iter_kvec(&cmd->recv_msg.msg_iter, READ, cmd->iov, 0, cmd->pdu_len);
>>   }
> 
> I earlier meant just use a local variable for 'nr_mapped' rather than replace
> it with '0'. I don't think that 0 segments would result in usable kvec.

yes, that is wrong. Need a local int nr_mapped variable.

> 
> I'm not even sure the existing code is correct, though. The sg->length can be
> higher order than a PAGE_SIZE, so it may be over-reporting nr_segs. It's just
> supposed to be the number of initialized iov's.

The sg is allocated with sgl_alloc so it is for sure 0-order.