Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.117.161 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C
Date:   Wed, 14 Sep 2022 13:55:08 -0700
From:   Nicolin Chen <nicolinc@nvidia.com>
To:     Robin Murphy <robin.murphy@arm.com>
CC:     Jason Gunthorpe <jgg@nvidia.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.org>,
        <joro@8bytes.org>, <suravee.suthikulpanit@amd.com>,
        <will@kernel.org>, <robdclark@gmail.com>, <dwmw2@infradead.org>,
        <baolu.lu@linux.intel.com>, <agross@kernel.org>,
        <bjorn.andersson@linaro.org>, <konrad.dybcio@somainline.org>,
        <matthias.bgg@gmail.com>, <heiko@sntech.de>, <orsonzhai@gmail.com>,
        <baolin.wang@linux.alibaba.com>, <zhang.lyra@gmail.com>,
        <thierry.reding@gmail.com>, <sricharan@codeaurora.org>,
        <yong.wu@mediatek.com>, <vdumpa@nvidia.com>,
        <jonathanh@nvidia.com>, <tglx@linutronix.de>,
        <shameerali.kolothum.thodi@huawei.com>,
        <thunder.leizhen@huawei.com>, <christophe.jaillet@wanadoo.fr>,
        <yangyingliang@huawei.com>, <jon@solid-run.com>,
        <iommu@lists.linux.dev>, <linux-kernel@vger.kernel.org>,
        <linux-arm-kernel@lists.infradead.org>,
        <linux-arm-msm@vger.kernel.org>,
        <linux-mediatek@lists.infradead.org>,
        <linux-rockchip@lists.infradead.org>,
        <linux-tegra@vger.kernel.org>,
        <virtualization@lists.linux-foundation.org>
Subject: Re: [PATCH 4/5] iommu: Regulate errno in ->attach_dev callback
 functions
Message-ID: <YyI/rLRu1qQ57LM0@Asurada-Nvidia>
References: <20220913082448.31120-1-nicolinc@nvidia.com>
 <20220913082448.31120-5-nicolinc@nvidia.com>
 <YyB3F/o3RfymqiFW@myrica>
 <YyGaqsXSDMn8R12R@nvidia.com>
 <YyGjtsB2Yq4fQICS@myrica>
 <YyIWQ6rX6AR9KX5E@Asurada-Nvidia>
 <5f049bb8-76e6-901a-9f8e-b48d270bc4a0@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <5f049bb8-76e6-901a-9f8e-b48d270bc4a0@arm.com>
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2022 20:55:26.2104
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 37464c23-4ea2-44f6-a788-08da96937336
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT086.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5856
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Robin,

On Wed, Sep 14, 2022 at 08:53:07PM +0100, Robin Murphy wrote:
> External email: Use caution opening links or attachments
> 
> 
> On 2022-09-14 18:58, Nicolin Chen wrote:
> > On Wed, Sep 14, 2022 at 10:49:42AM +0100, Jean-Philippe Brucker wrote:
> > > External email: Use caution opening links or attachments
> > > 
> > > 
> > > On Wed, Sep 14, 2022 at 06:11:06AM -0300, Jason Gunthorpe wrote:
> > > > On Tue, Sep 13, 2022 at 01:27:03PM +0100, Jean-Philippe Brucker wrote:
> > > > > I think in the future it will be too easy to forget about the constrained
> > > > > return value of attach() while modifying some other part of the driver,
> > > > > and let an external helper return EINVAL. So I'd rather not propagate ret
> > > > > from outside of viommu_domain_attach() and finalise().
> > > > 
> > > > Fortunately, if -EINVAL is wrongly returned it only creates an
> > > > inefficiency, not a functional problem. So we do not need to be
> > > > precise here.
> > > 
> > > Ah fair. In that case the attach_dev() documentation should indicate that
> > > EINVAL is a hint, so that callers don't rely on it (currently words "must"
> > > and "exclusively" indicate that returning EINVAL for anything other than
> > > device-domain incompatibility is unacceptable). The virtio-iommu
> > > implementation may well return EINVAL from the virtio stack or from the
> > > host response.
> > 
> > How about this?
> > 
> > + * * EINVAL    - mainly, device and domain are incompatible, or something went
> > + *               wrong with the domain. It's suggested to avoid kernel prints
> > + *               along with this errno. And it's better to convert any EINVAL
> > + *               returned from kAPIs to ENODEV if it is device-specific, or to
> > + *               some other reasonable errno being listed below
> 
> FWIW, I'd say something like:
> 
> "The device and domain are incompatible. If this is due to some previous
> configuration of the domain, drivers should not log an error, since it
> is legitimate for callers to test reuse of an existing domain.
> Otherwise, it may still represent some fundamental problem."

OK. I will use this narrative.

> And then at the public interfaces state it from other angle:
> 
> "The device and domain are incompatible. If the domain has already been
> used or configured in some way, attaching the same device to a different
> domain may be expected to succeed. Otherwise, it may still represent
> some fundamental problem."

I assume this should go to kdocs of iommu_attach_device/group().

> [ and to save another mail, I'm not sure copying the default comment for
> ENOSPC is all that helpful either - what is "space" for something that
> isn't a storage device? I'd guess limited hardware resources in some
> form, but in the IOMMU context, potential confusion with address space
> is maybe a little too close for comfort? ]

How about "non-ENOMEM type of resource allocation failure"?

> > > > > Since we can't guarantee that APIs like virtio or ida won't ever return
> > > > > EINVAL, we should set all return values:
> > > > 
> > > > I dislike this alot, it squashes all return codes to try to optimize
> > > > an obscure failure path :(
> > 
> > Hmm...should I revert all the driver changes back to this version?
> 
> Yeah, I don't think we need to go too mad here. Drivers shouldn't emit
> their *own* -EINVAL unless appropriate, but if it comes back from some
> external API then that implies something's gone unexpectedly wrong
> anyway - maybe it's a transient condition and a subsequent different
> attach might actually work out OK? We can't really say in general.

OK. Then there's even no need to convert EINVAL to ENODEV.

> Besides, if the driver sees an error which implies it's done something
> wrong itself, it probably shouldn't be trusted to try to reason about it
> further. The caller can handle any error as long as we set their
> expectations correctly.

Yea. As Jason remarked, a wrongly returned EINVAL would make things
a bit inefficient: VFIO/IOMMUFD would keep trying attach_dev() with
its existing domain list and a new domain but fail all of them.

I will change things in v2 back to this 2-patch version, and maybe
limit a bit further the changes in the first NODEV patch.

Thanks!
Nic