Date:   Thu, 9 Mar 2023 11:31:04 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Robin Murphy <robin.murphy@arm.com>
Cc:     Nicolin Chen <nicolinc@nvidia.com>, will@kernel.org,
        eric.auger@redhat.com, kevin.tian@intel.com,
        baolu.lu@linux.intel.com, joro@8bytes.org,
        shameerali.kolothum.thodi@huawei.com, jean-philippe@linaro.org,
        linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1 14/14] iommu/arm-smmu-v3: Add
 arm_smmu_cache_invalidate_user
Message-ID: <ZAn7uC9UweiNdGkJ@nvidia.com>
References: <cover.1678348754.git.nicolinc@nvidia.com>
 <aa327f9ea61e5a4771c13e53639e33955b9acde3.1678348754.git.nicolinc@nvidia.com>
 <1467e666-1b6c-c285-3f79-f8e8b088718b@arm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1467e666-1b6c-c285-3f79-f8e8b088718b@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?SIIKWlMIfRedMScPq9eLr3twmvOzEkXeTdtFETbUkscvs6y1d1LGOtxjVCCz?=
 =?us-ascii?Q?gsdvDf5RffxOog1NJMlhik7qMGYeB4XiQvQ67RcQMjGy1NXMFKJh6gK2C/Yt?=
 =?us-ascii?Q?Lhut5/LK+TnkjJ0O5auGmRBqyK3SDVTcSqGnMie+6Yo+n5Mphlg8h5YTMW8r?=
 =?us-ascii?Q?yBkwgI/dyQspTX8F0L1m19+req5tATJgQ+f6bp6UhJC0rqmYm8uxFbbM70+e?=
 =?us-ascii?Q?aTHCN5CjbJPY/bbx8NGhLWw/rs9NbIh0u61Lv0776WIb74//gTFfQgo/aXkv?=
 =?us-ascii?Q?rdRA8+uTEN3f7n/+iQlHLQ3nEyRdaXiPJnQtWT2ZVzQAsk+Tdfln/tLiJug/?=
 =?us-ascii?Q?82+aTrh19snV5Fh4qFyGoerLRs5Sh4ui7A0v1iCMm1Aumn21+0hUpX0w2chr?=
 =?us-ascii?Q?c2lwqh7/C3h53VoEHcbIxit/Apz3TGH1oBDUnAzp9U4f2pF2EasBb8xQNknh?=
 =?us-ascii?Q?dckfrhqgLEOy7Yh/55Wj5NPpMt3Vel2fL2Li7hXTgR3kZGn3T+fgMEAmuSyA?=
 =?us-ascii?Q?QRYeT9jnIUKxdcwhx+uu41P8DPFBCZW5r5gURXvZIlpjaOQXVdKlXW1KUiWr?=
 =?us-ascii?Q?eFWBPbww80SdMGQe5GeFhmela2p5POrgbapBKMnavVZN51Zeqh5jSDlG3D1W?=
 =?us-ascii?Q?8B1ub5l7BTJjXhsV4wYdvXFqUxPt/1TpNBRZzME4Z65kmV9Y8RkmmKRWeeCX?=
 =?us-ascii?Q?cPzdR0USnR1J3UavUwDIaqq/ylFrIXRmawXeKdrK63IihkceZneW5sYOilvu?=
 =?us-ascii?Q?1haR2rqhksSxyWcjh/pJoalp1otNzCW0tGFSdx0sGrqAyKiAbdZXd6herLAF?=
 =?us-ascii?Q?sm82Hq7JlxZCxJpbW00xTJeXmZOCBp18Vwah0CMCMozRohh1FJB15D5H8DIa?=
 =?us-ascii?Q?E5T0lbpd36T4SKV11LAwcjFDgkw0pskS1oaR1vR5Etj9TU4ubCs+T2yMaN3r?=
 =?us-ascii?Q?qWy5CnVR8UGY0aA6vJhpfYpZtUj2xNT+oYNnzfIZPL8zVHIjXC00NGRQKMrT?=
 =?us-ascii?Q?gYNPynUe7Tua8e8dhs73uC2idDV41vNfrcfGS623V1Lu2RSh3GdGJ3Y/stn1?=
 =?us-ascii?Q?1S08pvf6emCrf24wWpPB1lCdGbpLVsNQj3qa4uaIlI8iJsYEA4ZYlgsUcNK/?=
 =?us-ascii?Q?dG2dngHoyDmEw1ty3W2px4LEHE+faBwbX3gjY9EWnpQq0LD6m+75ncNRTDIB?=
 =?us-ascii?Q?aGBbEPn//DZBnEhiYrcm3gGdZyNg2TL44QimGztrjlqx0ZHek2HfeFbaKvB5?=
 =?us-ascii?Q?onm78QDOkigR7IGNG4FDaeZS9DL2lqrtVj+1DZj6ueyriIuTJWn26p4FdEVA?=
 =?us-ascii?Q?loH8eEU9fupDIBAk8qr6GLIz3sztU0ctzTgMQw76rCc0TqqMAHebl2098hTR?=
 =?us-ascii?Q?QtUDB/l4IusJsQo68u5FTC5MCTneoFQrCh7SpWfT4Kv46LIn5zbrs455swiF?=
 =?us-ascii?Q?AgMwJY/AtrTlNxGRfwcWMKOAEt47+8FVxqDbaksfPBmgeYZqsdOM7vnBW8W6?=
 =?us-ascii?Q?SGSpnvhBe+BtEBSI9pWK5twB5CuIhUoqFsdxgkKQD/h7LXWQL6XDaDKiOyj5?=
 =?us-ascii?Q?+PfzrVDJfBLz5vCMqGwPEZjIKTwVBjwWCLmbiIuA?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e8da711e-fbf5-4871-11c4-08db20b34da9
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2023 15:31:07.9561
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VFrhmflV2oemLtEtzQ3mrYJdf9SN7nu9t8pSlRNjh+jwBRbaC6Xp71h05BH4+Q3j
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5063
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Mar 09, 2023 at 02:49:14PM +0000, Robin Murphy wrote:

> If the design here is that user_data is so deeply driver-specific and
> special to the point that it can't possibly be passed as a type-checked
> union of the known and publicly-visible UAPI types that it is, wouldn't it
> make sense to just encode the whole thing in the expected format and not
> have to make these kinds of niggling little conversions at both ends?

Yes, I suspect the design for ARM should have the input be the entire
actual command work queue entry. There is no reason to burn CPU cycles
in userspace marshalling it to something else and then decode it again
in the kernel. Organize things to point the ioctl directly at the
queue entry, and the kernel can do a single memcpy from guest
controlled pages to kernel memory then parse it?

More broadly, maybe should this be able to process a list of commands?
If the queue has a number of invalidations batching them to the kernel
sure would be nice.

Maybe also for Intel? Kevin?

> Also, perhaps I've overlooked something obvious, but what's the procedure
> for reflecting illegal commands back to userspace? Some of the things we're
> silently ignoring here would be expected to raise CERROR_ILL. Same goes for
> all the other fault events which may occur due to invalid S1 config, come to
> think of it.

Perhaps the ioctl should fail and the userpace viommu should inject
this CERROR_ILL?

But I'm also wondering if we are making a mistake to not just have the
kernel driver to expose a SW work queue in its native format and the
ioctl is only just 'read the queue'. Then it could (asynchronously!)
push back answers, real or emulated, as well, including all error
indications.

I think we got down this synchronous one-ioctl-per-invalidation path
because that was what the original generic stuff wanted to do. Is it
what we really want? Kevin what is your perspective?

Jason