Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31;
Date:   Wed, 29 Nov 2023 20:08:16 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Nicolin Chen <nicolinc@nvidia.com>
Cc:     "Tian, Kevin" <kevin.tian@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "cohuck@redhat.com" <cohuck@redhat.com>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
        "chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
        "yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
        "peterx@redhat.com" <peterx@redhat.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "lulu@redhat.com" <lulu@redhat.com>,
        "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "Duan, Zhenzhong" <zhenzhong.duan@intel.com>,
        "joao.m.martins@oracle.com" <joao.m.martins@oracle.com>,
        "Zeng, Xin" <xin.zeng@intel.com>,
        "Zhao, Yan Y" <yan.y.zhao@intel.com>
Subject: Re: [PATCH v6 2/6] iommufd: Add IOMMU_HWPT_INVALIDATE
Message-ID: <20231130000816.GB1389974@nvidia.com>
References: <BN9PR11MB5276C8EACE2C300A646EA8A18CBBA@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZVw/BXxgGCuCZCA6@Asurada-Nvidia>
 <BN9PR11MB52761A9B48A25E89BEECE6308CB8A@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZWTzoBTDDEWAKMs9@Asurada-Nvidia>
 <BN9PR11MB5276FD60A0EDF8E3F231FCC88CBCA@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ZWaLCSAMIOXTlghk@Asurada-Nvidia>
 <20231129005715.GS436702@nvidia.com>
 <ZWaPM4p7yjJ0sEKk@Asurada-Nvidia>
 <20231129195804.GF436702@nvidia.com>
 <ZWe2PvatTkkyNCY5@Asurada-Nvidia>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZWe2PvatTkkyNCY5@Asurada-Nvidia>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/r4Z81OPDtFfQIhYUp8rcAfxnSKFMFj8zb1QGLIzXYAPTTBSq7YNMzo/R6a7?=
 =?us-ascii?Q?nmQlSAME5vjlOo7I/mUksGoy0xeboHx/1cZAcQntoQ++L11LVzoakKh8V4XL?=
 =?us-ascii?Q?cW8duQEXvv5xYXPSAW2LL2UjwBXN21qq3/+ts+N0cWKN4+s10YCxIJVYA8Ir?=
 =?us-ascii?Q?E5QYP/KcrCG7saMFQVNWdH/ck/0j3JjZvUuPhSlCXY9Dy3GiN+5FIRbyD3Ud?=
 =?us-ascii?Q?o1bFPdkUCYZEHke5kLnRxaCpQAZ4Mj0WQ0wg1nfojfSZsaIARxlanNWpeDc7?=
 =?us-ascii?Q?0bzO3wkvron0q6vA61hwd1tQ/8alK+2P/PVIA0Zf3DiGNXJxEwZWsYM1kiui?=
 =?us-ascii?Q?5PJpNTYjF2spGrG6zcqFpOgu87lxyvvmfEaHK7JxBzDaunXvkBZN9aCPp9rn?=
 =?us-ascii?Q?j5xieXa+rZrR8Qz8NpZsI6cLkWNlhamjeL/j8DNmnkezwGvtCyczfu3f/lUN?=
 =?us-ascii?Q?M1EhLB9BJIZQHxl5cC85wpFNHOi9cVNrvlgT+Pf6Nv6DA4dBEnwfd0rc1dmk?=
 =?us-ascii?Q?ddj8FGlB+xJzQywoQNYRCsC4pUd9zSWtN+F4wi70Z0f5tRsGbRkONjRZ7V/1?=
 =?us-ascii?Q?JajHBJf7uSKhWQIyiin5FzRR1H7rkaHpTyheCRvBTqVXpPYcU59/ysDO9CiZ?=
 =?us-ascii?Q?UrfG/CyKkokTlRENUIW4SALZGvp9W7Ff8ZdPSATm0YM7ghHGlalxKEh5M7O1?=
 =?us-ascii?Q?YvzZ9fTbznNWWKT/XUAG/uXmNpPAMd90T4L9XLTlhV1rTHFfR5YrSqii/s+n?=
 =?us-ascii?Q?9Lixha8kk7mjAjHiUJCenXy0nuzrpRxoYKflcBlGn5nXlkYCiKjTIWRt1SpW?=
 =?us-ascii?Q?ZrUD+fs7vLGK762xBwQ0Eje5ISvnLEAielzSBXXumJhn+mrIj2CSrE7/mkKp?=
 =?us-ascii?Q?IMmutec9srDLWd6NsUhCvGDuzYrrM1hr6ERTSgGDaUzNhmiWrT/lEIhqA1IU?=
 =?us-ascii?Q?+pcXE+Vt1+hrjGQZ0BlQ367Ajjl+QFshEKgtYrAuDQ8g5A6InoEJBcptRM26?=
 =?us-ascii?Q?BegT9cYVse1p1frODAD+Skkxw52JXhbr8hYejaZn/hk3v0AiI4dBWVHnqU7F?=
 =?us-ascii?Q?b8gt4jrNd6adhQega5Ak+hQTmfu+aga9OeHngA2Zlec93YBqMAQP4Sw/bkrF?=
 =?us-ascii?Q?hHpsTwQz/EXAwuIay4WMeU80lf7MuuU1Nwu6bfRt/hiewWSC3ypQA3APrGTa?=
 =?us-ascii?Q?40GZCngzJEJKKfqSNxeNX3v83gBuWKDQfQZ8An8Q2a54Ff5LaT0RVmu81tcY?=
 =?us-ascii?Q?GhHfwZ5w3oG5lIASP/byzmf14XUJeA8oVtogsOncUBaO9vFb4sthpLW0ggPw?=
 =?us-ascii?Q?cRnOlkPpDa5vqIGRWzcAtFS4s3MF0w9rc+I2tiHFdLOU3wTXezZB4yOcEvhN?=
 =?us-ascii?Q?RujL8q9T2pVyGzQVpDv5clBX0SuVyjZsHUlcaQ8st0EpxSv6pOv75ipSo8Lt?=
 =?us-ascii?Q?IR6CgEEalji9UGmmbsryHMA7FI/ab1oT057n/2Q5JfRWO9JetLcZd1/2iNhJ?=
 =?us-ascii?Q?xUYb9SgCRvUZMWR1xPOUZH7Y9cD4gYpmzn+nBDgxjdYRVx6zSgl/bdTckAfa?=
 =?us-ascii?Q?c/EzZrbClV0jx2kIURd6xvhFA98DGtMcQhaxZpKk?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c61fa23d-f448-4cea-2364-08dbf13874a5
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2023 00:08:18.2123
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: L4nZmuRHYb60eN9dLVc3vIpwQWkDhYdbfEQSpOje5FVgebeiXH2wH3O3g6ioo95W
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5194
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Wed, 29 Nov 2023 16:08:30 -0800 (PST)

On Wed, Nov 29, 2023 at 02:07:58PM -0800, Nicolin Chen wrote:
> On Wed, Nov 29, 2023 at 03:58:04PM -0400, Jason Gunthorpe wrote:
> > On Tue, Nov 28, 2023 at 05:09:07PM -0800, Nicolin Chen wrote:
> > 
> > > > > With that being said, I think errno (-EIO) could do the job,
> > > > > as you suggested too.
> > > > 
> > > > Do we have any idea what HW failures can be generated by the commands
> > > > this will execture? IIRC I don't remember seeing any smmu specific
> > > > codes related to invalid invalidation? Everything is a valid input?
> > > 
> > > "7.1 Command queue errors" has the info.
> > 
> > Hmm CERROR_ATC_INV_SYNC needs to be forwarded to the guest somehow
> 
> Oh, for sure. That's typically triggered with an asynchronous
> timeout from the eventq, so we'd need the io page fault series
> as you previously remarked. Though I also wonder if an invalid
> vSID that doesn't link to a pSID should be CERROR_ATC_INV_SYNC
> also v.s. CERROR_ILL.

Yes, something like that make sense

Presumably sync becomes emulated and turns into something generated
when the ioctl returns. So userspace would have to read the event FD
before returning to be correct?

Maybe the kernel can somehow return a flag to indicate the event fd
has data in it?

If yes then all errors would flow through the event fd?

Would Intel be basically the same too?

Jason