Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
In-Reply-To: <20190819150723.GH5058@ziepe.ca>
Subject: Re: Re: Re: Re: [PATCH] RDMA/siw: Fix compiler warnings on 32-bit due to
 u64/pointer abuse
From:   "Bernard Metzler" <BMT@zurich.ibm.com>
To:     "Jason Gunthorpe" <jgg@ziepe.ca>
Cc:     "Geert Uytterhoeven" <geert@linux-m68k.org>,
        "Doug Ledford" <dledford@redhat.com>, linux-rdma@vger.kernel.org,
        linux-kernel@vger.kernel.org
Date:   Mon, 19 Aug 2019 15:54:56 +0000
MIME-Version: 1.0
Sensitivity: 
Importance: Normal
References: <20190819150723.GH5058@ziepe.ca>,<20190819141856.GG5058@ziepe.ca>
 <20190819135213.GF5058@ziepe.ca> <20190819122456.GB5058@ziepe.ca>
 <20190819100526.13788-1-geert@linux-m68k.org>
 <OF7DB4AD51.C58B8A8B-ON0025845B.004A0CF6-0025845B.004AB95C@notes.na.collabserv.com>
 <OFD7D2994B.750F3146-ON0025845B.004D965D-0025845B.004E5577@notes.na.collabserv.com>
 <OFD7C97688.66331960-ON0025845B.005081B6-0025845B.0051AF67@notes.na.collabserv.com>
Content-Transfer-Encoding: 8BIT
Content-Type: text/plain; charset=UTF-8
Message-Id: <OFB73D0AD1.A2D5DDF4-ON0025845B.00545951-0025845B.00576D84@notes.na.collabserv.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

-----"Jason Gunthorpe" <jgg@ziepe.ca> wrote: -----

>To: "Bernard Metzler" <BMT@zurich.ibm.com>
>From: "Jason Gunthorpe" <jgg@ziepe.ca>
>Date: 08/19/2019 05:07PM
>Cc: "Geert Uytterhoeven" <geert@linux-m68k.org>, "Doug Ledford"
><dledford@redhat.com>, linux-rdma@vger.kernel.org,
>linux-kernel@vger.kernel.org
>Subject: [EXTERNAL] Re: Re: Re: [PATCH] RDMA/siw: Fix compiler
>warnings on 32-bit due to u64/pointer abuse
>
>On Mon, Aug 19, 2019 at 02:52:13PM +0000, Bernard Metzler wrote:
>> 
>> >To: "Bernard Metzler" <BMT@zurich.ibm.com>
>> >From: "Jason Gunthorpe" <jgg@ziepe.ca>
>> >Date: 08/19/2019 04:19PM
>> >Cc: "Geert Uytterhoeven" <geert@linux-m68k.org>, "Doug Ledford"
>> ><dledford@redhat.com>, linux-rdma@vger.kernel.org,
>> >linux-kernel@vger.kernel.org
>> >Subject: [EXTERNAL] Re: Re: Re: [PATCH] RDMA/siw: Fix compiler
>> >warnings on 32-bit due to u64/pointer abuse
>> >
>> >On Mon, Aug 19, 2019 at 02:15:36PM +0000, Bernard Metzler wrote:
>> >> 
>> >> >To: "Bernard Metzler" <BMT@zurich.ibm.com>
>> >> >From: "Jason Gunthorpe" <jgg@ziepe.ca>
>> >> >Date: 08/19/2019 03:52PM
>> >> >Cc: "Geert Uytterhoeven" <geert@linux-m68k.org>, "Doug Ledford"
>> >> ><dledford@redhat.com>, linux-rdma@vger.kernel.org,
>> >> >linux-kernel@vger.kernel.org
>> >> >Subject: [EXTERNAL] Re: Re: [PATCH] RDMA/siw: Fix compiler
>> >warnings
>> >> >on 32-bit due to u64/pointer abuse
>> >> >
>> >> >On Mon, Aug 19, 2019 at 01:36:11PM +0000, Bernard Metzler
>wrote:
>> >> >> >If the value is really a kernel pointer, then it ought to be
>> >> >printed
>> >> >> >with %p. We have been getting demanding on this point lately
>in
>> >> >RDMA
>> >> >> >to enforce the ability to keep kernel pointers secret.
>> >> >> >
>> >> >> >> -			wqe->sqe.sge[0].laddr = (u64)&wqe->sqe.sge[1];
>> >> >> >> +			wqe->sqe.sge[0].laddr = (uintptr_t)&wqe->sqe.sge[1];
>> >> >> >
>> >> >> >[..]
>> >> >> >
>> >> >> >>  			rv = siw_rx_kva(srx,
>> >> >> >> -					(void *)(sge->laddr + frx->sge_off),
>> >> >> >> +					(void *)(uintptr_t)(sge->laddr + frx->sge_off),
>> >> >> >>  					sge_bytes);
>> >> >> >
>> >> >> >Bernard, this is nonsense, what is going on here with
>> >sge->laddr
>> >> >that
>> >> >> >it can't be a void *?
>> >> >> >
>> >> >> siw_sge is defined in siw-abi.h. We make the address u64 to
>keep
>> >> >the ABI
>> >> >> arch independent.
>> >> >
>> >> >Eh? How does the siw-abi.h store a kernel pointer? Sounds like
>> >kernel
>> >> >and user types are being mixed.
>> >> >
>> >> 
>> >> siw-abi.h defines the work queue elements of a siw send queue.
>> >> For user land, the send queue is mmapped. Kernel or user land
>> >> clients write to its send queue when posting work
>> >> (SGE: buffer address, length, local key). 
>> >
>> >Should have different types.. Don't want to accidently mix a laddr
>> >under user control with one under kernel control.
>> >
>> Well we have an unsigned 64bit for both user and kernel
>> application buffer addresses throughout the rdma stack, 
>
>We do not. Kernel addresses are consistenyly void * or dma_addr_t
>
Absolutely. But these addresses are conveyed through the
API as unsigned 64 during post_send(), and land in the siw
send queue as is. During send queue processing, these addresses
must be interpreted according to its context and transformed
(casted) back to the callers intention. I frankly do not
know what we can do differently... The representation of
all addresses as unsigned 64 is given. Sorry for the confusion.


>Most places that consume a data address are using lkeys anyhow, which
>does have a lkey & u64, but that u64 is not a application buffer
>address, but the IOVA of the lkey, which is very different.
>
>I really have no idea why siw needs to mix kernel VAs with user
>pointers, particularly in wqes...
>
>Jason
>
>