2018-07-26 04:57:37

by Stephen Rothwell

Subject: linux-next: build failure after merge of the block tree

Hi all,

After merging the block tree, today's linux-next build (x86_64
allmodconfig) failed like this:

drivers/nvme/target/rdma.c: In function 'nvmet_rdma_find_get_device':
drivers/nvme/target/rdma.c:894:26: error: 'struct ib_device_attr' has no member named 'max_sge'; did you mean 'max_cqe'?
cm_id->device->attrs.max_sge) - 1;
^
drivers/nvme/target/rdma.c:893:21: note: in expansion of macro 'max'
inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
^~~
In file included from include/linux/list.h:9:0,
from include/linux/module.h:9,
from drivers/nvme/host/rdma.c:15:
drivers/nvme/host/rdma.c: In function 'nvme_rdma_find_get_device':
drivers/nvme/host/rdma.c:381:23: error: 'struct ib_device_attr' has no member named 'max_sge'; did you mean 'max_cqe'?
ndev->dev->attrs.max_sge - 1);
^
drivers/nvme/host/rdma.c:380:30: note: in expansion of macro 'min'
ndev->num_inline_segments = min(NVME_RDMA_MAX_INLINE_SEGMENTS,
^~~

Caused by commits

64a741c1eaa8 ("nvme-rdma: support up to 4 segments of inline data")
0d5ee2b2ab4f ("nvmet-rdma: support max(16KB, PAGE_SIZE) inline data")

interacting with commit

33023fb85a42 ("IB/core: add max_send_sge and max_recv_sge attributes")

from the rdma tree.

I have applied the following merge fix patch for today:

From: Stephen Rothwell <[email protected]>
Date: Thu, 26 Jul 2018 14:32:15 +1000
Subject: [PATCH] nvme-dma: merge fix up for replacement of max_sge

Signed-off-by: Stephen Rothwell <[email protected]>
---
drivers/nvme/host/rdma.c | 2 +-
drivers/nvme/target/rdma.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index cfa0319fcd1c..368fe5ac0c6b 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -378,7 +378,7 @@ nvme_rdma_find_get_device(struct rdma_cm_id *cm_id)
}

ndev->num_inline_segments = min(NVME_RDMA_MAX_INLINE_SEGMENTS,
- ndev->dev->attrs.max_sge - 1);
+ ndev->dev->attrs.max_send_sge - 1);
list_add(&ndev->entry, &device_list);
out_unlock:
mutex_unlock(&device_list_mutex);
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
index 86121a7a19b2..8c30ac7d8078 100644
--- a/drivers/nvme/target/rdma.c
+++ b/drivers/nvme/target/rdma.c
@@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id *cm_id)

inline_page_count = num_pages(port->inline_data_size);
inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
- cm_id->device->attrs.max_sge) - 1;
+ cm_id->device->attrs.max_send_sge) - 1;
if (inline_page_count > inline_sge_count) {
pr_warn("inline_data_size %d cannot be supported by device %s. Reducing to %lu.\n",
port->inline_data_size, cm_id->device->name,
--
2.18.0

--
Cheers,
Stephen Rothwell

Attachments:

2018-07-26 08:46:09

by Christoph Hellwig

Subject: Re: linux-next: build failure after merge of the block tree

On Thu, Jul 26, 2018 at 02:56:24PM +1000, Stephen Rothwell wrote:
> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
> index 86121a7a19b2..8c30ac7d8078 100644
> --- a/drivers/nvme/target/rdma.c
> +++ b/drivers/nvme/target/rdma.c
> @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id *cm_id)
>
> inline_page_count = num_pages(port->inline_data_size);
> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> - cm_id->device->attrs.max_sge) - 1;
> + cm_id->device->attrs.max_send_sge) - 1;

This should be max_recv_sge.

2018-07-26 08:47:17

by Stephen Rothwell

Subject: Re: linux-next: build failure after merge of the block tree

Hi Christoph,

On Thu, 26 Jul 2018 10:48:33 +0200 Christoph Hellwig <[email protected]> wrote:
>
> On Thu, Jul 26, 2018 at 02:56:24PM +1000, Stephen Rothwell wrote:
> > diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
> > index 86121a7a19b2..8c30ac7d8078 100644
> > --- a/drivers/nvme/target/rdma.c
> > +++ b/drivers/nvme/target/rdma.c
> > @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id *cm_id)
> >
> > inline_page_count = num_pages(port->inline_data_size);
> > inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> > - cm_id->device->attrs.max_sge) - 1;
> > + cm_id->device->attrs.max_send_sge) - 1;
>
> This should be max_recv_sge.

Thanks, I will fix up my resolution for tomorrow.

--
Cheers,
Stephen Rothwell

Attachments:

2018-07-26 13:05:14

by Steve Wise

Subject: RE: linux-next: build failure after merge of the block tree

Hey Stephen:

> I have applied the following merge fix patch for today:
>
> From: Stephen Rothwell <[email protected]>
> Date: Thu, 26 Jul 2018 14:32:15 +1000
> Subject: [PATCH] nvme-dma: merge fix up for replacement of max_sge
>
> Signed-off-by: Stephen Rothwell <[email protected]>
> ---
> drivers/nvme/host/rdma.c | 2 +-
> drivers/nvme/target/rdma.c | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index cfa0319fcd1c..368fe5ac0c6b 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -378,7 +378,7 @@ nvme_rdma_find_get_device(struct rdma_cm_id
> *cm_id)
> }
>
> ndev->num_inline_segments =
> min(NVME_RDMA_MAX_INLINE_SEGMENTS,
> - ndev->dev->attrs.max_sge - 1);
> + ndev->dev->attrs.max_send_sge - 1);
> list_add(&ndev->entry, &device_list);
> out_unlock:
> mutex_unlock(&device_list_mutex);
> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
> index 86121a7a19b2..8c30ac7d8078 100644
> --- a/drivers/nvme/target/rdma.c
> +++ b/drivers/nvme/target/rdma.c
> @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id
> *cm_id)
>
> inline_page_count = num_pages(port->inline_data_size);
> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> - cm_id->device->attrs.max_sge) - 1;
> + cm_id->device->attrs.max_send_sge) - 1;

This should actually be max_recv_sge.

Thanks!

Steve.

2018-07-26 18:30:04

by Jens Axboe

Subject: Re: linux-next: build failure after merge of the block tree

On 7/26/18 1:48 AM, Christoph Hellwig wrote:
> On Thu, Jul 26, 2018 at 02:56:24PM +1000, Stephen Rothwell wrote:
>> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
>> index 86121a7a19b2..8c30ac7d8078 100644
>> --- a/drivers/nvme/target/rdma.c
>> +++ b/drivers/nvme/target/rdma.c
>> @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id *cm_id)
>>
>> inline_page_count = num_pages(port->inline_data_size);
>> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
>> - cm_id->device->attrs.max_sge) - 1;
>> + cm_id->device->attrs.max_send_sge) - 1;
>
> This should be max_recv_sge.

Why do we even have this conflicts to begin with?


--
Jens Axboe

2018-07-26 18:31:05

by Christoph Hellwig

Subject: Re: linux-next: build failure after merge of the block tree

On Thu, Jul 26, 2018 at 10:48:21AM -0700, Jens Axboe wrote:
> >> inline_page_count = num_pages(port->inline_data_size);
> >> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> >> - cm_id->device->attrs.max_sge) - 1;
> >> + cm_id->device->attrs.max_send_sge) - 1;
> >
> > This should be max_recv_sge.
>
> Why do we even have this conflicts to begin with?

Because the nvme code added a new user of max_sges while the RDMA tree
split it into two separate fields. We discussed this a while ago when
the issue came up.

2018-07-26 18:33:26

by Steve Wise

Subject: RE: linux-next: build failure after merge of the block tree

> -----Original Message-----
> From: Christoph Hellwig <[email protected]>
> Sent: Thursday, July 26, 2018 12:57 PM
> To: Jens Axboe <[email protected]>
> Cc: Christoph Hellwig <[email protected]>; Stephen Rothwell
> <[email protected]>; Doug Ledford <[email protected]>; Jason
> Gunthorpe <[email protected]>; Linux-Next Mailing List <linux-
> [email protected]>; Linux Kernel Mailing List <linux-
> [email protected]>; Steve Wise <[email protected]>
> Subject: Re: linux-next: build failure after merge of the block tree
>
> On Thu, Jul 26, 2018 at 10:48:21AM -0700, Jens Axboe wrote:
> > >> inline_page_count = num_pages(port->inline_data_size);
> > >> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> > >> - cm_id->device->attrs.max_sge) - 1;
> > >> + cm_id->device->attrs.max_send_sge) -
1;
> > >
> > > This should be max_recv_sge.
> >
> > Why do we even have this conflicts to begin with?
>
> Because the nvme code added a new user of max_sges while the RDMA tree
> split it into two separate fields. We discussed this a while ago when
> the issue came up.

See: https://www.spinics.net/lists/linux-rdma/msg66208.html

Steve.

2018-07-26 20:55:56

by Bart Van Assche

Subject: Re: linux-next: build failure after merge of the block tree

On 07/26/18 10:48, Jens Axboe wrote:
> On 7/26/18 1:48 AM, Christoph Hellwig wrote:
>> On Thu, Jul 26, 2018 at 02:56:24PM +1000, Stephen Rothwell wrote:
>>> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
>>> index 86121a7a19b2..8c30ac7d8078 100644
>>> --- a/drivers/nvme/target/rdma.c
>>> +++ b/drivers/nvme/target/rdma.c
>>> @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id *cm_id)
>>>
>>> inline_page_count = num_pages(port->inline_data_size);
>>> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
>>> - cm_id->device->attrs.max_sge) - 1;
>>> + cm_id->device->attrs.max_send_sge) - 1;
>>
>> This should be max_recv_sge.
>
> Why do we even have this conflicts to begin with?

Hello Jens,

A detailed description of why that change is necessary is available at
https://www.spinics.net/lists/linux-rdma/msg65954.html. As far as I know
the entire RDMA community supports that change.

Bart.

2018-07-26 21:15:52

by Jens Axboe

Subject: Re: linux-next: build failure after merge of the block tree

On 7/26/18 1:54 PM, Bart Van Assche wrote:
> On 07/26/18 10:48, Jens Axboe wrote:
>> On 7/26/18 1:48 AM, Christoph Hellwig wrote:
>>> On Thu, Jul 26, 2018 at 02:56:24PM +1000, Stephen Rothwell wrote:
>>>> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
>>>> index 86121a7a19b2..8c30ac7d8078 100644
>>>> --- a/drivers/nvme/target/rdma.c
>>>> +++ b/drivers/nvme/target/rdma.c
>>>> @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id *cm_id)
>>>>
>>>> inline_page_count = num_pages(port->inline_data_size);
>>>> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
>>>> - cm_id->device->attrs.max_sge) - 1;
>>>> + cm_id->device->attrs.max_send_sge) - 1;
>>>
>>> This should be max_recv_sge.
>>
>> Why do we even have this conflicts to begin with?
>
> Hello Jens,
>
> A detailed description of why that change is necessary is available at
> https://www.spinics.net/lists/linux-rdma/msg65954.html. As far as I know
> the entire RDMA community supports that change.

Thanks - I'm not against the change, I just always get a little
suspicious when we have merge conflicts across subsystems. But this
one seems warranted.

--
Jens Axboe

2018-07-26 22:18:29

by Stephen Rothwell

Subject: Re: linux-next: build failure after merge of the block tree

Hi Steve,

On Thu, 26 Jul 2018 07:32:16 -0500 "Steve Wise" <[email protected]> wrote:
>
> > diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
> > index 86121a7a19b2..8c30ac7d8078 100644
> > --- a/drivers/nvme/target/rdma.c
> > +++ b/drivers/nvme/target/rdma.c
> > @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id
> > *cm_id)
> >
> > inline_page_count = num_pages(port->inline_data_size);
> > inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> > - cm_id->device->attrs.max_sge) - 1;
> > + cm_id->device->attrs.max_send_sge) - 1;
>
> This should actually be max_recv_sge.

Yeah, Christoph pointed that out as well. I will correct it today.

Thanks
--
Cheers,
Stephen Rothwell

Attachments:

2018-08-15 01:44:43

by Stephen Rothwell

Subject: Re: linux-next: build failure after merge of the block tree

Hi all,

On Thu, 26 Jul 2018 14:56:24 +1000 Stephen Rothwell <[email protected]> wrote:
>
> After merging the block tree, today's linux-next build (x86_64
> allmodconfig) failed like this:
>
> drivers/nvme/target/rdma.c: In function 'nvmet_rdma_find_get_device':
> drivers/nvme/target/rdma.c:894:26: error: 'struct ib_device_attr' has no member named 'max_sge'; did you mean 'max_cqe'?
> cm_id->device->attrs.max_sge) - 1;
> ^
> drivers/nvme/target/rdma.c:893:21: note: in expansion of macro 'max'
> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> ^~~
> In file included from include/linux/list.h:9:0,
> from include/linux/module.h:9,
> from drivers/nvme/host/rdma.c:15:
> drivers/nvme/host/rdma.c: In function 'nvme_rdma_find_get_device':
> drivers/nvme/host/rdma.c:381:23: error: 'struct ib_device_attr' has no member named 'max_sge'; did you mean 'max_cqe'?
> ndev->dev->attrs.max_sge - 1);
> ^
> drivers/nvme/host/rdma.c:380:30: note: in expansion of macro 'min'
> ndev->num_inline_segments = min(NVME_RDMA_MAX_INLINE_SEGMENTS,
> ^~~
>
> Caused by commits
>
> 64a741c1eaa8 ("nvme-rdma: support up to 4 segments of inline data")
> 0d5ee2b2ab4f ("nvmet-rdma: support max(16KB, PAGE_SIZE) inline data")
>
> interacting with commit
>
> 33023fb85a42 ("IB/core: add max_send_sge and max_recv_sge attributes")
>
> from the rdma tree.
>
> I have applied the following merge fix patch for today:
>
> From: Stephen Rothwell <[email protected]>
> Date: Thu, 26 Jul 2018 14:32:15 +1000
> Subject: [PATCH] nvme-dma: merge fix up for replacement of max_sge
>
> Signed-off-by: Stephen Rothwell <[email protected]>
> ---
> drivers/nvme/host/rdma.c | 2 +-
> drivers/nvme/target/rdma.c | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index cfa0319fcd1c..368fe5ac0c6b 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -378,7 +378,7 @@ nvme_rdma_find_get_device(struct rdma_cm_id *cm_id)
> }
>
> ndev->num_inline_segments = min(NVME_RDMA_MAX_INLINE_SEGMENTS,
> - ndev->dev->attrs.max_sge - 1);
> + ndev->dev->attrs.max_send_sge - 1);
> list_add(&ndev->entry, &device_list);
> out_unlock:
> mutex_unlock(&device_list_mutex);
> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c
> index 86121a7a19b2..8c30ac7d8078 100644
> --- a/drivers/nvme/target/rdma.c
> +++ b/drivers/nvme/target/rdma.c
> @@ -891,7 +891,7 @@ nvmet_rdma_find_get_device(struct rdma_cm_id *cm_id)
>
> inline_page_count = num_pages(port->inline_data_size);
> inline_sge_count = max(cm_id->device->attrs.max_sge_rd,
> - cm_id->device->attrs.max_sge) - 1;
> + cm_id->device->attrs.max_send_sge) - 1;
> if (inline_page_count > inline_sge_count) {
> pr_warn("inline_data_size %d cannot be supported by device %s. Reducing to %lu.\n",
> port->inline_data_size, cm_id->device->name,
> --
> 2.18.0

This is now needed (with the max_send_sge -> max_recv_sge update in
drivers/nvme/target/rdma.c patch) when I merge the rdma tree with
Linus' tree.

--
Cheers,
Stephen Rothwell

Attachments: