Received: by 2002:a05:6359:6284:b0:131:369:b2a3 with SMTP id se4csp5715595rwb; Wed, 9 Aug 2023 08:13:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFmGXQRLvMiEQO6QEJzzHGCZuxt22M0gwbnsWwyRG4MATHrE3ER03D5xAiLE7kUCHmVqGzA X-Received: by 2002:a17:90a:9b0f:b0:268:400:ecef with SMTP id f15-20020a17090a9b0f00b002680400ecefmr2173421pjp.45.1691594022713; Wed, 09 Aug 2023 08:13:42 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1691594022; cv=pass; d=google.com; s=arc-20160816; b=mAGtOiTigH6iT5dAYjRd6rje6tbDCjs892bdYBFHOkV/fXfo3DoHPkFx1RmcC9fSdQ g6HLC6w6Q1aKpEQqmOmkehHnjt6S3eGcn45VTWw5rKCOczJi3J5jCtpNlkzSmy99WWqS M5FEK1dam3e3Lhuyk9QcIzq8hITogkENXHpo0lCkJi4FeHWICb8YHerOn2Kiavi3Zwkt GvmjgNEpY1YQECZemnCEj7sXVmaNEFNKB7C5TD5MqcHkT/5uKFT/9V6osEZn6Laeucgc 8zTCB9sE9RwT8zPrWVRn+FVQQJL82PNtWIykCBFFPMk+uzh1CBqZEfHH//l77yw17p0U BQrA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:in-reply-to:content-disposition :references:message-id:subject:cc:to:from:date:dkim-signature; bh=xKgSN+DipQLIqs5gKRMn9UzgNe7NMwabguil4hm8xDg=; fh=/KBXtOzgZN/IwmIo6N6ZAu2YPwXxstt5sBCs/EPjPG8=; b=TE1z4dcRKxvwQaoNzlvx3V6Utc8n9VV1TGp6vVJMcr7XxyGxtuMjQCll73GdRbDFqo M8QCtJaGp7HkD9QrK4pmUer4DRe9H4cdE3aRkOuh/RkoH3/J/mtF3lBYfNzPEmj4djyx eL/yvdHmaCCEqjX2IVOSJPCDDGaP/GToNQwFWYOT1lSTlA2qOH0wYfQ7+stkZHg2zL4Z nzb1UsRfmPnQtWWtFemeFhZ6TsrQcGFm9QryQWSJQKrISUih4bECWia091CeRDlewdWD xd6IPrbS8JIGfCPyLbT/tbiyF579iYezZAxIpDrEx6ln8ZM8b8WjfPFj6Le1hfCcumoj Gn3g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=UGFEW8JZ; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h11-20020a63120b000000b00563ea86ad93si7871265pgl.148.2023.08.09.08.13.30; Wed, 09 Aug 2023 08:13:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=UGFEW8JZ; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232657AbjHIOqC (ORCPT + 99 others); Wed, 9 Aug 2023 10:46:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229999AbjHIOqB (ORCPT ); Wed, 9 Aug 2023 10:46:01 -0400 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2054.outbound.protection.outlook.com [40.107.21.54]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8CCE1FCC; Wed, 9 Aug 2023 07:45:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aw8pHwWtRjRG5VcIa7pJkYz2as2VmjWw8hTFGEA+N27VDJ2hF0UdD3uMJUr6nUi5gwbNBUkDGAl6gqmIFv4yfBKlvAG0LCkNzXcEIc8Mye/1HK1Og1GQBq5eQdjVEKezsMEeEnyY98LdXkYtm52hC4mqJ8nO9OlkWhlR0D0Uju4faP5MNhwJBYpFfNkKY5jSOEDjBEEGkjpwGzIc0TxjB8HKzzpRDq+jU5ORY8L1Kqk/hcRcivwhD7HWppcMUHTSV0kXy8J8EBRRJZnSCCVkzclCBjOuL0ExLMHLNrJ/+BXAagT0ecmS9NdTJFWhF9V75fqXFSu3LqDtLCbWLEMV/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xKgSN+DipQLIqs5gKRMn9UzgNe7NMwabguil4hm8xDg=; b=a8pDP54HVE1BeCdV77e6N6TmI9J190VNqJGw+nYQ8gCLegirmmOIPkfKcKw0R6u3yjZqejmMLmkd89cbyOvXyo/KKBmhco4rTDTY4GcKH3jw7Thlz05lR3I5lGZxO/2CRkT2Up5cbLJnTCsRzqs8ZReSS/dAJwlTLHrUUd4K24hRDCtNYELIxtKX/EWJ5v4mHmGOg+6okneqGCelyM3xfrc1kmTpaTYWRm6KFHQjo3Pfk0YUtEUecB6TrAyTiQuB0EA1+d4Vj0qtBVhX9uiC1JXP0UOXiRpxPK+9qanPdo0MP5dZaw9Zdz1dO5X/iLkv5OfQdOuBs7NIZZQqM6A2LA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xKgSN+DipQLIqs5gKRMn9UzgNe7NMwabguil4hm8xDg=; b=UGFEW8JZ+qefFm7Nl1ugxr5Dc7kgzOs2YJuR8C/9ElW76PvYlq2T0D7tGafJ4DKoQwzmKL4tp9VAoIVNABVjPGKlG/vwAlP/r0MB8qQ3pqnqpNVI+VxEiKrgdwbRlWpN40tpVmLGApp3651m4L7TZGi2VQTx5UQvXqo6xgGsvCU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM0PR04MB6452.eurprd04.prod.outlook.com (2603:10a6:208:16d::21) by DB9PR04MB10011.eurprd04.prod.outlook.com (2603:10a6:10:4c4::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.28; Wed, 9 Aug 2023 14:45:54 +0000 Received: from AM0PR04MB6452.eurprd04.prod.outlook.com ([fe80::d4ed:20a0:8c0a:d9cf]) by AM0PR04MB6452.eurprd04.prod.outlook.com ([fe80::d4ed:20a0:8c0a:d9cf%6]) with mapi id 15.20.6652.028; Wed, 9 Aug 2023 14:45:54 +0000 Date: Wed, 9 Aug 2023 17:45:49 +0300 From: Vladimir Oltean To: Bjorn Helgaas Cc: linux-pci@vger.kernel.org, netdev@vger.kernel.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Bjorn Helgaas , Rob Herring , Claudiu Manoil , Michael Walle , linux-kernel@vger.kernel.org, Jianmin Lv , Liu Peibao , Binbin Zhou , Huacai Chen Subject: Re: [PATCH v2 pci/net 1/3] PCI: move OF status = "disabled" detection to dev->match_driver Message-ID: <20230809144549.ksw3slllph6kqwg7@skbuf> References: <20230803135858.2724342-2-vladimir.oltean@nxp.com> <20230808222107.GA331664@bhelgaas> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230808222107.GA331664@bhelgaas> X-ClientProxiedBy: AM0PR02CA0180.eurprd02.prod.outlook.com (2603:10a6:20b:28e::17) To AM0PR04MB6452.eurprd04.prod.outlook.com (2603:10a6:208:16d::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM0PR04MB6452:EE_|DB9PR04MB10011:EE_ X-MS-Office365-Filtering-Correlation-Id: 07e95930-8199-4c11-f26a-08db98e75544 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dd4c6LgLRFkOc9HC3JhbSEwTUacy0gDc1Zvl+0GP26ip0OpgWCoAWb2dqU4rySxy3aLxAfLnJ89h2tJFRnm1dfUPYe4VcqelIMV1iO576ONGEcgjcie2lebuxUzvV0D3pw0Fv7YJVFSM4v8eB0MmLRIOH4nc2m1nZpLYhSbe8/LBJMt9bYazU8xB8QmThpMbWCgSN8+iCZ97WGT3q2ZwsfKJ9RLpuRbE4PbgmiLXnQKS66DnBLRl39sEMy2/GUS9bhuLJKaLet/Cx3DhKfjMucd15rZw6mL2DkHQBYuU0SisK+0g5oYAL6MYAOGxMzCF4KONdRvdLftENrOZ4ErvOTwu89QktYPOqHRRjhSsB3n0V28N3Iqr5XCDZYCLtwRUcSku4fByvpupsTFP5TIO8QW7CailCoxRQrdbTQg+nXciZDL98zBLD4mkKG2Uqd5XuB/v97M+BLVQCrkxT5u7Z34Pu4G1sQYAI8Ms33/Y15ucqWhkb/1lzrtVSIVqDoqWlutDlqrAhyT7ojcVDTlD10uUaZky/8Fonp/N7rlLbZhEY0oj5CvSdSXWTsqF8ZOV X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM0PR04MB6452.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(7916004)(136003)(376002)(366004)(396003)(39860400002)(346002)(1800799006)(186006)(451199021)(6506007)(26005)(1076003)(9686003)(6512007)(33716001)(6486002)(6666004)(86362001)(478600001)(54906003)(83380400001)(316002)(66476007)(6916009)(66556008)(66946007)(4326008)(41300700001)(8676002)(8936002)(5660300002)(38100700002)(7416002)(44832011)(30864003)(2906002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?qDEMu78q1KZf9ZaKxNLkWtXkwTM5hW0keOKBkfl0ltnEHJ4yaPP4UdYsD+U/?= =?us-ascii?Q?Um7a4VXy7kaYnMIYx10LtEJI6yCfkH2ucB6o2K3GFs/PBWazal1fufQCFgpO?= =?us-ascii?Q?ngw4a1SSdZMP5hGB82jJhLzhbJ4NYQ51RdSSb5JbO8Fw7OoQ5hZwD1h+hPeP?= =?us-ascii?Q?Ab4J+ibvXYHS4CrG5tTrp78fgIEGasgTYUF+oWwfqo2KOKw2HfOfibhyN5bb?= =?us-ascii?Q?kWmpmFTp0zFjumLbDUcYWOTKNOzzALCh1ilyIOREr1dGHGWGAyFJ/0sL89Ki?= =?us-ascii?Q?f/5/qM89Nyyq8u6BmsyO5MMPaIdkalHCq6ACFCHaVg06u9iHUpAa8TlMa9fs?= =?us-ascii?Q?3d9FE7xEfqpCZiRiSR47DYLu012ZcpYH9zecIR5mJc+RTtdCJD2srQX7QPmG?= =?us-ascii?Q?H4s6V/Fl85OrFz7a3HuRRlYY1wlprculLdntTTTP2V5z9H6mlb8mov2/WFWC?= =?us-ascii?Q?jCjb9yuY7eILUeRLizikArijlAz+Wt51YQ6Mbn6TIjF8+rrzjBuLeb/M6MmO?= =?us-ascii?Q?0rphZdHWI/Xa5ov8/5wLEHUXcz+RgU/5ylM+6zGaNPWO1pFqR1OaUGhB6xli?= =?us-ascii?Q?MCk8viAGRTCG7B2gcAKZsKeMn9a66LdPjMN6/i4uMhFOi1Q6PCv7hmnYbOYU?= =?us-ascii?Q?s8TC0IO9Fehyue1s34Dg4UQDy9cazKwGyFghS7Ybo4XnDjWJk44NpQiIf4Sw?= =?us-ascii?Q?PGpQBM2FhMHbOfRLoZgqvACJPFyD2qyi2O9bdy6zZwfccSZb33THmvxPZhW4?= =?us-ascii?Q?nVigfQLFsiJ62uC9znEAh2yja85pf9ggxSmCFa0T0YNPVXqjbsA1r5DKDYE9?= =?us-ascii?Q?PSx7Cw4ZLidMwEG68lnJHRgOQ4othxQmMK05oaHUsVRd6KUs1CX/nghF6cDs?= =?us-ascii?Q?NjkBzBNUBWr/2sBjBDG9cuLiv4hiCBqskAV8lYm6InnuJw1nlcO+E5wZ2Tq3?= =?us-ascii?Q?CA+V8PGhQJAS1O83A+cW9NPwlPo/c6XMvfrmpZw8SDKYQH7LJWaVZKDlTDK1?= =?us-ascii?Q?NkGwn7FFpZLDp8Vho17XFL3SFxfA/VeK/F/H54n8hFnZTILCamw9/ZlVszRa?= =?us-ascii?Q?iE83tLBO9kIRaDAcxUjdc/WVJJERCn2IUK/U45P+hVPmtB5D24Us1L5B/GVp?= =?us-ascii?Q?MCPQFVWUypzAyml3UHosXDlQylEUbcU47iwyZ6EzuIZ5gvkusHsXIMaUXD7D?= =?us-ascii?Q?VvxnPLPDFKIAB4Q8i4CmhDgznfsCANY7mLFijLD2KMs+81+zM4PAE2Jd9ngk?= =?us-ascii?Q?ivTx23ccR9W9qcd4uBhoC41soPzj5QgH56t9sL3NyeRti8HwgYjmjJPwmJWL?= =?us-ascii?Q?4JDJZlPcAr1/2JutNpglXEbDE1GNISl74oZ8yVpz2k4+x+Wjs8lcIsR1RFUT?= =?us-ascii?Q?d17afmzfqxsivkeQ7LcmDaMp87rEjSO5yDGvy23zHo9UhSBo27BPRIC1Yufs?= =?us-ascii?Q?g37v5yYtFzmoGSxhEgmUpi9CS7ajcKbPzzk4OtXdSeZaoL6Dw4QXvQaiaxX7?= =?us-ascii?Q?Nx1t7MKM/AEFyUCo0jxt2dSBlH8ITYjgANKt0xH/rNoGnCd2qzpYyi++cYEf?= =?us-ascii?Q?+9PJqacTskl3zNi09/+FFd3Xg5sNvQc3TdwP635isEyWdWyiSBXy/dQe+BWc?= =?us-ascii?Q?lA=3D=3D?= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 07e95930-8199-4c11-f26a-08db98e75544 X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB6452.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2023 14:45:54.0517 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UoZKNYJ28EMSSH2iOXGBZVqayZshXz6OH0wi+fiidUKTFNf0t4dUW1LAuD7bAqVvYJKZ7YuTb+PZC3SrICChSQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR04MB10011 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Bjorn, On Tue, Aug 08, 2023 at 05:21:07PM -0500, Bjorn Helgaas wrote: > I think this makes good sense, but let me make sure I understand how > this works. > > I *think* what's happening is that this Function 0 responds to config > reads, so PCI enumeration starts by discovering it normally. But > after 6fffbc7ae137 ("PCI: Honor firmware's device disabled status"), > we abort in pci_setup_device() if the DT or ACPI status is "disabled," > which means there's no struct pci_dev for it, no quirks can run on it, > and no driver can bind to it. And, since PCI multi-function devices > must have a Function 0, we don't enumerate the other functions of this > device. > > That's a problem because (1) you need to do some initialization on > Function 0 even though you don't want a driver to claim it, Correction: on functions 0, 1, 2 and 6 (all have PCI_VENDOR_ID_FREESCALE, ENETC_DEV_ID_PF), and not just on function 0. The particular nature of a hardware IP bug/afterthought makes this necessary. It may be best to look at the lspci -vvv output on this SoC: 00:00.0 Ethernet controller: Freescale Semiconductor Inc Device e100 (rev 01) (prog-if 01) Subsystem: Freescale Semiconductor Inc Device e100 Device tree node: /sys/firmware/devicetree/base/soc/pcie@1f0000000/ethernet@0,0 Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- TAbort- SERR- Kernel driver in use: pcieport > and (2) this is a multi-function device and you need to enumerate the > other functions. correct > What this patch does is make it so the PCI core enumerates Function 0 The patch does not special-case function 0. Fundamentally it changes the meaning of "disabled" in the firmware description from "not enumerable" to "driver shouldn't be bound to it" (which is more or less what the meaning was, prior to the blamed commit, except that it was down to individual drivers to observe the property) and that's absolutely it. > normally so there will be a struct pci_dev for it, the normal config > space access to it will work, and it will appear in the dmesg log and > lspci output, all as usual. But if the DT or ACPI status is > "disabled", we will not bind a PCI driver to it. > > If that's true, I'd like to highlight the PCI details here and move > some of the device-specific things to the driver patches, e.g., > something like this: > > PCI: Enumerate device but don't bind driver if firmware status is 'disabled' > > In some configurations, the NXP LS1028A has a multi-function NIC > where Function 0 is not usable as a NIC, but it's accessible via > config space and it's needed for device-specific initialization. > Function 0 also indicates that the NIC is a multi-function device > and the kernel should look for more functions. Instead of "multi-function NIC", I'd say "PCIe endpoint integrated into a root complex (RCiEP)". Reworded, it's not a NIC, it's 4, plus other device classes (system peripherals, fabric controllers). > > arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi marks Function 0 as > "disabled," and after 6fffbc7ae137 ("PCI: Honor firmware's device > disabled status"), Linux doesn't enumerate Function 0, which means > the entire NIC is unusable because Linux doesn't enumerate the other > functions either. > > Instead of completely ignoring a function with DT/ACPI "disabled" > status, enumerate it as usual but prevent drivers from claiming it. > The disabled function will still be accessible via config space, > fixups will work, and it will be visible via lspci. > > So feel free to merge this along with the other patches via the net > tree with: > > Acked-by: Bjorn Helgaas You may have given the netdev maintainers some mixed signals with the rewording suggestion plus the ack for my wording, and now we have commit 1a8c251cff20 ("PCI: move OF status = "disabled" detection to dev->match_driver") in the net.git tree. I think we are mostly on the same page about what is changing, it's just that we are focusing on different aspects of it in the description. I hope you're ok if we close the topic the way things are now? :)