IronPort-PHdr: =?us-ascii?q?9a23=3AFh8inBS20Uzh74yYrFs5sgXe7dpsv+yvbD5Q0YIu?=
 =?us-ascii?q?jvd0So/mwa6zZB2N2/xhgRfzUJnB7Loc0qyN4vCmATRIyK3CmUhKSIZLWR4BhJ?=
 =?us-ascii?q?detC0bK+nBN3fGKuX3ZTcxBsVIWQwt1Xi6NU9IBJS2PAWK8TW94jEIBxrwKxd+?=
 =?us-ascii?q?KPjrFY7OlcS30P2594HObwlSijewZbB/IA+qoQnNq8IbnZZsJqEtxxXTv3BGYf?=
 =?us-ascii?q?5WxWRmJVKSmxbz+MK994N9/ipTpvws6ddOXb31cKokQ7NYCi8mM30u683wqRbD?=
 =?us-ascii?q?VwqP6WACXWgQjxFFHhLK7BD+Xpf2ryv6qu9w0zSUMMHqUbw5Xymp4rx1QxH0li?=
 =?us-ascii?q?gIKz858HnWisNuiqJbvAmhrAF7z4LNfY2ZKOZycqbbcNgHR2ROQ9xRWjRBDI2i?=
 =?us-ascii?q?YYsADfcPM+hboYbyu1QAohywBRW3CePz0z9IiWP60bMm3+kjFwzNwQwuH8gJsH?=
 =?us-ascii?q?TRtNj5OrsfUeSxzKbWyzXIcvFY2Srm54fTbB8tr+yHULVsfMrVzUkgCQXFgk+S?=
 =?us-ascii?q?p4z4JDyazfoCvnOG4OV+UeKvj3QrpB12ojiq38ohjJTCiIENyl3c9Ch0w5w5Kc?=
 =?us-ascii?q?O2RUJle9KoDZtdui6AO4ZyRs4uW3xktSc5x7EcpJK2fCwHxI45yxLCZfGLaZWE?=
 =?us-ascii?q?7x3+WOuXPDx2nmhqeKiliBa36UWgz+r8WdSq31tStSpFl8XMtmgK1xzO9siLUv?=
 =?us-ascii?q?t98Vml2TaIzw3c9PpELlo1mKbBNpEu3Lowlp4KvUTEAy/2hF75jKiLdkUi5+ek?=
 =?us-ascii?q?9f7rYrT+pp+cMo91hRvyPbgpmsy6Geg4Mw4OUHaH+emk1bDu/lf1TKtEg/EoiK?=
 =?us-ascii?q?XVrZDXKMsBqqO9BwJZyoMj5Ay+Dzei3tQYh34HLFdddR+bi4jpP0/BIPbiAfm9?=
 =?us-ascii?q?nlSjiyxkyO7dM7L8HJrNKnzDnK39crZ67k5Q0BAzwsxH55JIFrEBJ+r+WkvwtN?=
 =?us-ascii?q?zeEx84PBW4w+X5B9Vn0IMRR2aPD7SHMKPdr1CI/PgjI+qSa48PvjbyNfwl6+Tp?=
 =?us-ascii?q?jX8jll9ONZWuiNFYTHe3F/IuDFiffXrrmM8MXi1C6g45Q+Xsh3WOXDpPbmq/Uu?=
 =?us-ascii?q?Q34TRtTMryCYbFW5DohqCL9Ci8GZJSa29cDU2UCjHjcIDSH79YbCOUP98kkTEe?=
 =?us-ascii?q?U7WlY5Eu2AvotwLgzbdjaO3O9XtLm4jk0Y0/zunXmBd61SF+BcnXmzWkS2V5mC?=
 =?us-ascii?q?UoWjU80YhzrEh5jFyE1P4r0LRjCdVP6qYRAU8BPpnGwrk/UoiqVw=3D=3D?=
From: "Allen Hubbe" <Allen.Hubbe@dell.com>
To: "'Logan Gunthorpe'" <logang@deltatee.com>,
        <linux-ntb@googlegroups.com>, <linux-kernel@vger.kernel.org>
Cc: "'Jon Mason'" <jdmason@kudzu.us>
References: <20171209000217.18366-1-logang@deltatee.com> <20171209000217.18366-2-logang@deltatee.com> <000201d37290$81605eb0$84211c10$@dell.com> <d06cf74e-125c-6281-9361-22bac3468251@deltatee.com> <000301d372b4$b6042100$220c6300$@dell.com> <2ec869d2-caca-e283-b7b5-a28b5908a4db@deltatee.com>
In-Reply-To: <2ec869d2-caca-e283-b7b5-a28b5908a4db@deltatee.com>
Subject: RE: [PATCH 2/2] ntb_hw_switchtec: Check for alignment of the buffer in mw_set_trans()
Date: Mon, 11 Dec 2017 15:06:22 -0500
Message-ID: <000401d372bb$84a34620$8de9d260$@dell.com>
MIME-Version: 1.0
Content-Type: text/plain;
        charset="utf-8"
Thread-Index: AQHTcIEAmu1N6hj0nUuwK+dCMVjXlaM+OxnQgAB+zgD//8tysIAAWe4A//+swwA=
Content-Language: en-us
Sender: linux-kernel-owner@vger.kernel.org
Content-Transfer-Encoding: 8bit
Content-Length: 2806
Lines: 47

From: Logan Gunthorpe
> On 11/12/17 12:17 PM, Allen Hubbe wrote:
> >> mw_get_align doesn't communicate the fact that the buffer has to be
> >> aligned by its size.
> >
> > Is that not the purpose of the addr_align out parameter of ntb_mw_get_align()?
> 
> addr_align provides the minimum alignment required by the device but it
> has no idea how big a buffer the caller is trying to create so it can't
> express that it needs to be aligned by its size.
> 
> To be clear, the minimum alignment the Switchtec device requires is 4KB
> so it will return 4k in addr_align. Thus, if you have a 4KB buffer it
> may be aligned to 4KB. But if you have a 1MB buffer it must be aligned
> to the nearest 1M.

In switchtec_ntb_mw_get_align, for the lut case it seems to require alignment the same as Intel, aligned to mw size, but for the non-lut case you are saying that SZ_4K is not necessarily correct.  The SZ_4K is the minimum, but the actual alignment restriction depends on the size of the buffer actually translated.  Right?

Also, for the lut case, it looks like the size also has to be the same size as the mw.  So, a client can't allocate a smaller buffer, assume we can get one that is aligned, point the start of the mw at it, and limit the size of the mw?

For the non-lut case I wonder, with the restriction that addr needs to be aligned to the size of the buffer, does the size of the buffer also need to be some power of two?  That would make sense, if it determines the alignment.  If so, SZ_4K wouldn't be correct for size_align, either.

Do you need the intended buffer size passed in as another parameter to ntb_mw_get_align?  The point of ntb_mw_get_align is to figure out all the alignment restrictions before allocating memory.

> >> It may also be that all hardware does not have this
> >> restriction (ie. if the hardware adds to the base address instead of
> >> just replacing the lower bits).
> >>
> >> There is definitely a need to print this error somewhere as I hit this
> >> case and it caused very weird behavior. It was a huge pain to debug.
> >> Also, it's a security issue and huge bug if we end up mapping the memory
> >> we didn't think we were mapping.
> >
> > Of course the driver should validate its parameters not allow bad mappings.  I was only commenting
> on the dev_err() message to the console.
> 
> Ok. I still feel like it would be difficult to debug if ntb_transport
> simply was unable to establish a connection without some message in
> dmesg telling the user why.
> 
> Also, keep in mind this is a somewhat unusual occurrence. In most cases
> dma_alloc_coherent() always provides a buffer that is aligned to it's
> size. It's just that the CMA (if used) provides a tunable config option
> which allows for larger buffers to not be aligned to their size.
> 
> Logan