Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 16 Dec 2022 13:28:36 -0800
From:   Ira Weiny <ira.weiny@intel.com>
To:     Jonathan Cameron <Jonathan.Cameron@huawei.com>
CC:     Dan Williams <dan.j.williams@intel.com>,
        Davidlohr Bueso <dave@stgolabs.net>,
        Bjorn Helgaas <helgaas@kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Alison Schofield <alison.schofield@intel.com>,
        "Vishal Verma" <vishal.l.verma@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        <linux-kernel@vger.kernel.org>, <linux-pci@vger.kernel.org>,
        <linux-acpi@vger.kernel.org>, <linux-cxl@vger.kernel.org>
Subject: Re: [PATCH V4 3/9] cxl/mem: Wire up event interrupts
Message-ID: <Y5zjBD6j4b3dBreR@iweiny-desk3>
References: <20221212070627.1372402-1-ira.weiny@intel.com>
 <20221212070627.1372402-4-ira.weiny@intel.com>
 <20221216142438.00006588@Huawei.com>
 <20221216184215.000015dd@Huawei.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20221216184215.000015dd@Huawei.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?7Z92UzWk5GbCXF11wMxcDTAPAfb1Wwm2i3XaI0fGS8nqubSfMCMrr7kxHE90?=
 =?us-ascii?Q?Ykjj485xAe1obhU/DBTcd2C1o0Er2bRujyzONun99YeZK2hnIOvJAco0CS+1?=
 =?us-ascii?Q?hYPGixpftKFinR3O13flrpswYINk4GsxgT6Zc0aRvN5gcQ0z3gacQ59ZF4XD?=
 =?us-ascii?Q?I+b6PDCYh4tz7Tsut9KurPtRnK0F9fX6RCgS0f7p/Q9nScBm1Dm6+xhTczHl?=
 =?us-ascii?Q?Qy3hjqFqb442vh3/MeYhGIMNNjKl8bxWbL7xv9jJ7mkMunwjCqhgAdsYsbz5?=
 =?us-ascii?Q?jrVzIgBsn+fpLvvhheVyG40x2HEALDX+YhX/Em+66jsA1+3phuEKqadmM2di?=
 =?us-ascii?Q?FfNBqlLSxr7E5Dck1z0uxFpmzO7DF93x7gGL6n9paz4RpTSAMa5G0PkraOvs?=
 =?us-ascii?Q?jIVHP8BNfGja01C0fd5xKE+NmteK33nMD15H1QctuSzFmugMOvjQPhtuFQP2?=
 =?us-ascii?Q?IH0TeYJZtS7SkepXy0fMbhxVRsYOM+z6g12hSJsFI0gc+S/4b6q9us7Zlfjq?=
 =?us-ascii?Q?cegm/n2xpC1TTpX62hUKhjHq4VChEzXGLPfzJnEEBYMDU7MdeYUhgebDv71Y?=
 =?us-ascii?Q?rf+QyXcokMKqbhXNcU2VZNmihPJxZSGEnmgaMfHhKORiW/yQB1pwdpDxKkQD?=
 =?us-ascii?Q?yc/JN74AHUCNuItZMIaCgbrBZCYkTl2ulRL/YKqau3naGCb944+9+RK8l9fe?=
 =?us-ascii?Q?i9SRKe3VIS5+HKaRSiYplIecn31bkAhQYW39Qm1ePGPlW7T3IkzEiBJncjwu?=
 =?us-ascii?Q?af1M5csMlJ62JFEKu9kuj6KJi1cY2o+VSqZY3AP9bDEYYCs9F3f4D8blKJGN?=
 =?us-ascii?Q?ZhObdblSi7bGYLcvRc5IKHZ9W5mj/eHC5qsSewVgoinXBrJUy2xbqiiOlE5l?=
 =?us-ascii?Q?V7k8lppztrpkqilr9t/ZClmFF5LouvbiALYuaPFLvq7tdp0QyNgB6Q9TMiRj?=
 =?us-ascii?Q?KM0knvzOmqgQ4fQ3p51fE7ke4HhXqz3n2ujRTMj9xNiyKX3mZODvSKAmqGxH?=
 =?us-ascii?Q?xOZP4frmiSAEWhMp/xuDKAOMQTsptEizI81VoFUB/wBe94CqXJ2Wu9cQ342I?=
 =?us-ascii?Q?xRIU6Dl35HPUsgsnM5Z4Egaj8EN1Exhl8nJEKwIxVb8rdfo+nyKGcwU+0EUm?=
 =?us-ascii?Q?DVHG0qbacPVUPbyelprMQrxLyH29VF3i46WBKO5oOM4Zdp33S2nI/re14KDr?=
 =?us-ascii?Q?MgnsJFISdEspjctv/nKSd0Yb+lOIA1G9z4fI7d+Ek6m2ZYchK+YSBD5V8/1q?=
 =?us-ascii?Q?vxwzpe9rXrgBZEO3jJnPkBNIT0HrQBp0CSk2c6i7EEVq+zzJF7RQ6I/YgIFP?=
 =?us-ascii?Q?7yni1kRjGkDbob9+iMyIW9UngFfuZr5DE0biaK89mIavMAWq0JxKKg6x+V3q?=
 =?us-ascii?Q?ynVEKGbpFpRAitF9BTmK1NFvnlE809a1zuX2VEQDZgG+x5rpCLnphdpHi629?=
 =?us-ascii?Q?IW4jWJEt0jA1bqI9AzkYXlDy1tKCjzsZMlBK6YAw1FZzOpdFXPqKSfJMQdmp?=
 =?us-ascii?Q?Y8fC+wSUmyC/1gAK26RrXbtqQdtN5braL1AKkoJ5zKP21pszrvyzz1PF0iDI?=
 =?us-ascii?Q?YUPMDxL5camgPwYHPG3C5TLQ8bd+4f7Ls3jUX1QW?=
X-MS-Exchange-CrossTenant-Network-Message-Id: da20bbdb-e85f-43b0-0e47-08dadfac7ff6
X-MS-Exchange-CrossTenant-AuthSource: SA1PR11MB6733.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Dec 2022 21:28:40.2110
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GIqrMBiZ3Gq6nmZkOR0VSVa6z6AHQzWU19rhozlLkDWitYSEI2bB+d/1Rvt2uVxLd2/6q2ybsjZw74ksZBr6cg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB6798
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Dec 16, 2022 at 06:42:15PM +0000, Jonathan Cameron wrote:
> On Fri, 16 Dec 2022 14:24:38 +0000
> Jonathan Cameron <Jonathan.Cameron@Huawei.com> wrote:
> 
> > On Sun, 11 Dec 2022 23:06:21 -0800
> > ira.weiny@intel.com wrote:
> > 
> > > From: Davidlohr Bueso <dave@stgolabs.net>
> > > 
> > > Currently the only CXL features targeted for irq support require their
> > > message numbers to be within the first 16 entries.  The device may
> > > however support less than 16 entries depending on the support it
> > > provides.
> > > 
> > > Attempt to allocate these 16 irq vectors.  If the device supports less
> > > then the PCI infrastructure will allocate that number.  Upon successful
> > > allocation, users can plug in their respective isr at any point
> > > thereafter.
> > > 
> > > CXL device events are signaled via interrupts.  Each event log may have
> > > a different interrupt message number.  These message numbers are
> > > reported in the Get Event Interrupt Policy mailbox command.
> > > 
> > > Add interrupt support for event logs.  Interrupts are allocated as
> > > shared interrupts.  Therefore, all or some event logs can share the same
> > > message number.
> > > 
> > > In addition all logs are queried on any interrupt in order of the most
> > > to least severe based on the status register.
> > > 
> > > Cc: Bjorn Helgaas <helgaas@kernel.org>
> > > Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > > Co-developed-by: Ira Weiny <ira.weiny@intel.com>
> > > Signed-off-by: Ira Weiny <ira.weiny@intel.com>
> > > Signed-off-by: Davidlohr Bueso <dave@stgolabs.net>
> > >   
> > 
> > > +/**
> > > + * Event Interrupt Policy
> > > + *
> > > + * CXL rev 3.0 section 8.2.9.2.4; Table 8-52
> > > + */
> > > +enum cxl_event_int_mode {
> > > +	CXL_INT_NONE		= 0x00,
> > > +	CXL_INT_MSI_MSIX	= 0x01,
> > > +	CXL_INT_FW		= 0x02
> > > +};
> > > +struct cxl_event_interrupt_policy {
> > > +	u8 info_settings;
> > > +	u8 warn_settings;
> > > +	u8 failure_settings;
> > > +	u8 fatal_settings;  
> > 
> > This is an issue for your QEMU code which has this set at 5 bytes.
> > Guess our handling of record lengths needs updating now we have two different
> > spec versions to support and hence these can have multiple lengths.
> > 
> > Btw, do you have an updated version of the QEMU patches you can share?
> 
> Note that I'm happy to take your QEMU series forwards, just don't want to duplicate
> stuff you have already done!

I do have updates to handle more than 3 records at a time which I was polishing
until I saw your review.

I was getting all geared up to respin the patches but I'm not sure there is an
issue now that I look at it.

I'm still investigating.

Ira

> 
> Jonathan
> 
> > I was planning on just doing the AER type RAS stuff for the first pull this cycle
> > but given this set means we never reach that code I probably need to do QEMU
> > support for this and the stuff to support those all in one go - otherwise
> > no one will be able to test it :)  We rather optimistically have the OSC set
> > to say the OS can have control of these, but upstream code doesn't emulate
> > anything yet. Oops. Should have pretended the hardware was handling them
> > until we had this support in place in QEMU.
> > 
> > Jonathan
> > 
> > > +} __packed;
> > > +
> > >  /**
> > >   * struct cxl_event_state - Event log driver state
> > >   *
> > > @@ -288,6 +305,8 @@ enum cxl_opcode {
> > >  	CXL_MBOX_OP_RAW			= CXL_MBOX_OP_INVALID,
> > >  	CXL_MBOX_OP_GET_EVENT_RECORD	= 0x0100,
> > >  	CXL_MBOX_OP_CLEAR_EVENT_RECORD	= 0x0101,
> > > +	CXL_MBOX_OP_GET_EVT_INT_POLICY	= 0x0102,
> > > +	CXL_MBOX_OP_SET_EVT_INT_POLICY	= 0x0103,
> > >  	CXL_MBOX_OP_GET_FW_INFO		= 0x0200,
> > >  	CXL_MBOX_OP_ACTIVATE_FW		= 0x0202,
> > >  	CXL_MBOX_OP_GET_SUPPORTED_LOGS	= 0x0400,  
>