Received: by 2002:a6b:fb09:0:0:0:0:0 with SMTP id h9csp4135181iog; Tue, 21 Jun 2022 12:49:12 -0700 (PDT) X-Google-Smtp-Source: AGRyM1ug6EAVj2wo9tuY+XzOmjd+ROrtzwnLas7rduFoyh0uXHWZ7nRUS/Fww5xtK6xzW6IXZSvZ X-Received: by 2002:a05:6402:1f82:b0:42f:c541:4fd1 with SMTP id c2-20020a0564021f8200b0042fc5414fd1mr36720363edc.172.1655840952418; Tue, 21 Jun 2022 12:49:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655840952; cv=none; d=google.com; s=arc-20160816; b=C8VtA7aFDm7AFOSqhW/grT3bC0H23iGl0VPZ+7IaG8ZmRgBKcRT4Y7Xn+ddbCXIWDs ulR8SQAHgtHLdgmTWv8Nx982ss1XQQYYcu9Oa6eDN8aDMEtOaVGu34NElB5diYiaFYs6 8LvBjMiOzYXo7CTHci12Vl640peUSMNPxiouuPfcdjtWvawwzR4SAuF5dhXGRv1bIJCU nj4bCWKDDGkGMhZoMVuMLcle1S59fbTk7aKURDw8XhRj4qPJZg8jLOracpPHOqWrJg63 2ZUd5sAuSrYIQDUKplgh4rwAMD6YJ4+MAW51qggwn6gnWm2Qm6NRm9habMqcrb+1omn9 N94Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:user-agent:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date; bh=uI/BsnM/iZf8P23r1e98wOABu/bA9FxDwcHvVgh+OSc=; b=R0JeF4EALUUn6tK4W5dYAEmNLBJkXfSnzEEujmLPPp5wKjpLCjp9ithQNfsEBcbqHN 6GA3vHrK4Y6VJoZskm2cQSPxN3Zk3bLcFYl0fA/C3JYL/griZFfFzsF8mKesmhOJSejB 4SzjCYe+CFa/xQohzaDlXUiWIYieNNs5+QIWy2KR/iv1A7usO7L6Osk/70ogLYhFTLxW vVeRPUlBhJKPnR9Qx7XBxKERYZDN73dVafeuALTY0CXR6xRiD1ZgrCVXd84W61ajPU8W fm4bqbtwkpb3q30iboJwMPA5f8PQU/1yDeh2yA7bt2sM5mBhkUKLMxJzYK+OzZMgTOTu 2tyQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hd33-20020a17090796a100b00718c04a5199si1705562ejc.897.2022.06.21.12.48.46; Tue, 21 Jun 2022 12:49:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354006AbiFUTec (ORCPT + 99 others); Tue, 21 Jun 2022 15:34:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353394AbiFUTeb (ORCPT ); Tue, 21 Jun 2022 15:34:31 -0400 Received: from bmailout2.hostsharing.net (bmailout2.hostsharing.net [IPv6:2a01:37:3000::53df:4ef0:0]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B30B25C76 for ; Tue, 21 Jun 2022 12:34:29 -0700 (PDT) Received: from h08.hostsharing.net (h08.hostsharing.net [IPv6:2a01:37:1000::53df:5f1c:0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "*.hostsharing.net", Issuer "RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1" (verified OK)) by bmailout2.hostsharing.net (Postfix) with ESMTPS id 82EDA2800B3C7; Tue, 21 Jun 2022 21:34:27 +0200 (CEST) Received: by h08.hostsharing.net (Postfix, from userid 100393) id 6B9162732F7; Tue, 21 Jun 2022 21:34:27 +0200 (CEST) Date: Tue, 21 Jun 2022 21:34:27 +0200 From: Lukas Wunner To: Dan Williams Cc: ira.weiny@intel.com, Bjorn Helgaas , Jonathan Cameron , Alison Schofield , Vishal Verma , Dave Jiang , Ben Widawsky , linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org Subject: Re: [PATCH V11 5/8] cxl/port: Read CDAT table Message-ID: <20220621193427.GA25003@wunner.de> References: <20220610202259.3544623-1-ira.weiny@intel.com> <20220610202259.3544623-6-ira.weiny@intel.com> <62ad1fb69d742_8920729490@dwillia2-xfh.notmuch> <62b2178bdcf5d_89207294ac@dwillia2-xfh.notmuch> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <62b2178bdcf5d_89207294ac@dwillia2-xfh.notmuch> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jun 21, 2022 at 12:10:03PM -0700, Dan Williams wrote: > It is really the interrupt setup that makes this an awkward fit all > around. The PCI core knows how to handle capabilities with interrupts, > but only for PCIe port services. DOE is both a PCIe port service *and* > and "endpoint service" like VPD (pci_vpd_init()). The more I think about > this the closer I get to the recommendation from Lukas which is that > DOE is more like pci_vpd_init() than pci_aer_init(), or a custom > enabling per driver. > > If the DOE enumeration moves to a sub-function of > pci_init_capabilities() then the cxl_pci and/or cxl_port drivers just > look those up and use them. The DOE instances would remain in polled > mode unless and until a PCI driver added interrupt support late. In > other words, DOE can follow the VPD init model as long as interrupts are > not involved, and if interrupts are desired it requires late allocation > of IRQ vectors. Thomas Gleixner has been working on dynamic allocation of MSI-X vectors. We should probably just build on that and let the PCI core allocate vectors for DOE mailboxes independently from drivers. To conserve vectors, I'd delay allocation for a DOE mailbox until it is first used. There may be mailboxes that are never used. DOE requires MSI-X or MSI. We could probably leave MSI unsupported until a device with broken MSI-X support shows up. I envision that with MSI, the onus is on the driver to allocate vectors for mailboxes it intends to use and it would then have to "donate" those vectors to the PCI core via a library function. As for portdrv, that's a driver but Bjorn has expressed a desire for a long time to move its functionality into the PCI core. It shouldn't be allowed to unbind portdrv via sysfs and thus break DPC etc, as is currently possible. The question with regards to this series is, do we get *something* merged and perfect it over time once it's in the tree, or do we keep iterating on the mailing list. I deliberately only provided a single, comprehensive review and then stayed mum because I feel bad for Ira having to keep reacting to more and more feedback despite being at v11 already (or v12? I've lost count). Particularly because I suspect (I might be mistaken) that Ira's natural habitat is actually CXL not PCI, so it might be a burden for him. I'd be fine to implement suggestions I've made myself after Ira's series lands. No need for him to keep iterating ad infinitum. Thanks, Lukas