Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     "Raj\, Ashok" <ashok.raj@intel.com>
Cc:     "Dey\, Megha" <megha.dey@intel.com>, linux-kernel@vger.kernel.org,
        "Jiang\, Dave" <dave.jiang@intel.com>,
        "Tian\, Kevin" <kevin.tian@intel.com>,
        "Pan\, Jacob jun" <jacob.jun.pan@intel.com>,
        "Liu\, Yi L" <yi.l.liu@intel.com>, jgg@mellanox.com,
        "Kumar\, Sanjay K" <sanjay.k.kumar@intel.com>,
        "Van De Ven\, Arjan" <arjan.van.de.ven@intel.com>,
        "Williams\, Dan J" <dan.j.williams@intel.com>,
        "Shankar\, Ravi V" <ravi.v.shankar@intel.com>,
        Ashok Raj <ashok.raj@intel.com>
Subject: Re: Programming PASID in IMS entries
In-Reply-To: <20210707221216.GA56594@otc-nc-03>
References: <bd509e3d-f59d-1200-44ce-93cf9132bd8c@intel.com> <87k0m2qzgz.ffs@nanos.tec.linutronix.de> <20210707221216.GA56594@otc-nc-03>
Date:   Thu, 08 Jul 2021 15:00:42 +0200
Message-ID: <87im1lot8l.ffs@nanos.tec.linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

Ashok,

On Wed, Jul 07 2021 at 15:12, Ashok Raj wrote:
> On Wed, Jul 07, 2021 at 10:50:52AM +0200, Thomas Gleixner wrote:
>> On Wed, Jul 07 2021 at 09:49, Megha Dey wrote:
>> > Per your suggestions during the last meeting, we wanted to confirm the 
>> > sequence to program the PASID into the IMS entries:
>> >
>> > 1. Add a PASID member to struct msi_desc (Add as part of a union. Other 
>> > source-id's such as Jason's vm-id can be added to it)
>> 
>> Yes. Though we also discussed storing the default PASID in struct device
>> to begin with which is then copied to the msi_desc entries during
>> allocation.
>
> Using default PASID in struct device will work for sub-devices until the
> guest needs to enable ENQCMD support. Since the guest kernel can ask for an
> interrupt by specifying something in the descriptor submitted via ENQCMD.
> Using the PASID in struct device won't be sufficient.

I'm well aware of that, but can we solve step 1 before step 2 please?

>> > In order to make IMS dynamic, we were thinking of the following 
>> > enhancements to the IMS core:
>> >
>> > 1. Device Driver specifies maximum number of interrupts the sub device 
>> > is allowed to request, while creating the dev-msi domain. E.g. in the 
>> > case of DSA, Driver can specify that each mdev created can have upto X
>> 
>> Why would this be mdev specific? IIRC the sub devices can be used on
>> bare metal as well.
>
> I guess so. I thought for bare metal we don't need to play these games
> since native abstraction is provided with things like uaccel for e.g. For
> things like SRIOV its much different.
>
> What the above limit accomplishes is telling the guest, you can request
> upto the limit, but you aren't guaranteed to get them. This avoids the
> static partitioning and becomes best effort by the host driver.

That's fine.

> Ideally we want to tell the guest it can have upto say 1k interrupts, but
> unlike MSIx where resources are commited in HW, we want to allow guest
> allocations to fail. 

Which as discussed requires a hypercall because silent fail is not an
option.

Thanks,

        tglx