Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751397AbdHaJlK (ORCPT ); Thu, 31 Aug 2017 05:41:10 -0400 Received: from mail-bl2nam02on0076.outbound.protection.outlook.com ([104.47.38.76]:45664 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750883AbdHaJlH (ORCPT ); Thu, 31 Aug 2017 05:41:07 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jan.Glauber@cavium.com; Date: Thu, 31 Aug 2017 11:40:52 +0200 From: Jan Glauber To: Alex Williamson Cc: Bjorn Helgaas , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, david.daney@cavium.com, Jon Masters , Robert Richter , linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Subject: Re: [PATCH v3 3/3] PCI: Avoid slot reset for Cavium cn8xxx root ports Message-ID: <20170831094052.GA15906@hc> References: <20170830142454.10971-1-jglauber@cavium.com> <20170830142454.10971-4-jglauber@cavium.com> <20170830084012.19d91759@w520.home> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170830084012.19d91759@w520.home> User-Agent: Mutt/1.5.21 (2010-09-15) X-Originating-IP: [88.67.130.225] X-ClientProxiedBy: DB6PR1001CA0003.EURPRD10.PROD.OUTLOOK.COM (10.171.79.13) To BN3PR07MB2578.namprd07.prod.outlook.com (10.167.5.6) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1f932ce7-bb23-44b0-da6f-08d4f054658f X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:BN3PR07MB2578; X-Microsoft-Exchange-Diagnostics: 1;BN3PR07MB2578;3:nSrvAST/9cYuYW2grpLEAIRqAI5P/uBFG0YW2+OlKAN3BOAzg8YFFS/1iDnW9E3vopZrT+iPR7nWogqJgz6+uAZSaxWA5B2/SVbn9gqvSvyjv14qQHstKWxaMGmFEhXoZyWKIas0+X5nVf/9/WpCvhI/kltB0nkZAVmdrBpjmOx9Af46/HVp/n5FH5LNng7R75MlPCTylzL/7Ni8rh1005k93mwW028tFj9q6itY9xQl6vd1sXI7LlZzkZsBTs+s;25:qw18Pl+/Ul3C0SSaeqfRvZ7q4YK1o5B6m6wDJ/Xazzt05kyU0ZgoKDNhPqePLDC38aGKjdYGjdatgBoksOY58Yg/mU29Va//uQHx8XlJinhh54+JrwJhGh0rI7qIjE3bGloZUMcRQfdB6IC4/MleBqKxI8uaNx1Gafis8hfoP7UpE276JGe9JxPseLdDm4YSS00sM2BkkwbdFw2ZkoLOfTBcUfJUgGgXBYFsqqhKgwcI/982vFCYgZUhjtQxcmo75rp2SIeYg9PchUroLnfJano9ihCMdcBvHNsFC0T33o7o6g4bXXf/zyvEm8xQzUycjCRTmSIXyDAErD8yqx0ELQ==;31:/x5EcMTh6jMX/AOU5NnzJoyHIeRwx6jwShaIz96fCEneR5FSQtrPQJ4LpcxC8Dy/6radRfrAmzBkeifS+Ue9W+XcehOiPjbUgxfwbegSZU9zDtmoeqlbN3/FkSz/NDE0bFuGkBlthQfBz3a8KGz5Ucn+5iDglbenwWmfAVMOAmxAo71tLoyJ7NoB1dDRqJ7HwOanLXRz83/QLOUoyJGw3/LdaTdNpSlrGW71VUmKQ2U= X-MS-TrafficTypeDiagnostic: BN3PR07MB2578: X-Microsoft-Exchange-Diagnostics: 1;BN3PR07MB2578;20:MPw1T0YR+2bhZe+NNUxo7Ps4R57l0vS4v/QSJNj11FsIlRG2IIh/GRz17QVKtj3BBZFoScSHBZVbLqsialhplCiWQMiYqg1oC96MDWZ0j8WxzU5iX6y2VGuGmEe+jsO2EVyEH80nT0d4pjlrIEuV4yr07DBlS31xJXBAZwcIOiGCCobUCXHurJzvMCG2+6XbLjtm5TR4ruijESx9ic6Cfg1bRvSI4NkJJozJPFPMTBqAVxnbZMU6D+uw1exdmDfe+qLXhz9xRNvU/TiTp7P0u5XZDuwVKiJMEl5PsF64ubx2lQ8+NUMpbfc1/eENs33hyJEtSxrfcghnLHIUQAvSTAoje35735esgeC/JWnf7ZYSgMCYbUXBZhDamNZTz+SamxbvmI4pTBvH3nW5oMCeS0JujfNsrlKehER9dtosLZltM4ozg9om9Z0Ou0yKZoPZoEwS5+rDPIrm3Dh27FSd4dfyAnVmNvf0MhhA5y4FW3asgyYI9YsWHGu6gZ+NcJjp1DtNP80Fz8koN5976QjhWeUZaBCWnNvknl4NBX5uuom4BfTd1H+xqsi8vJkqLOJVzEA3BxB31MbcgR+ipNfho6CeFljQh8Z54VV103eqy2Y=;4:m7cdwYQeouBCDxnjtnaZzpbpYX3WOSPGp/Yp/yFlxH+Rf89HWmuFwi/Lkdj8Am/V12TqnEWHiR96ZwpIb1Z3TT/bHOiJ7P7vaSdKFFsoDjZ93gbOMMoPpPyewRodpScYVfz9vTvg12DGNmbVgGyCA4u4r8aVhYlWX68kbU57X09Cyig/IhEcuyxPTiYaTaZOVIni7t/eUT6em0FrZcwgnOTQVIPN/LWYDrbCllRK9gzYWDAG+IZRN46/Vewo5WZt X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(100000703101)(100105400095)(10201501046)(3002001)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123560025)(20161123555025)(20161123564025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:BN3PR07MB2578;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:BN3PR07MB2578; X-Forefront-PRVS: 04163EF38A X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(199003)(24454002)(189002)(305945005)(33656002)(97736004)(8936002)(8676002)(4001350100001)(68736007)(83506001)(81166006)(81156014)(229853002)(2906002)(50986999)(76176999)(7736002)(6916009)(54356999)(6666003)(2950100002)(42882006)(101416001)(72206003)(189998001)(5660300001)(4326008)(3846002)(6116002)(23726003)(33716001)(42186005)(50466002)(478600001)(105586002)(106356001)(25786009)(6496005)(6246003)(66066001)(110136004)(53936002)(54906002)(1076002)(47776003)(9686003)(55016002)(18370500001)(32563001);DIR:OUT;SFP:1101;SCL:1;SRVR:BN3PR07MB2578;H:hc;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BN3PR07MB2578;23:X1LeNR48iyYmpgjyK+QeoQcY/AjM+mCHMN5EVOCFy?= =?us-ascii?Q?mCulqN4tL6dxs4YCPIRrlu8DUiXl2h0/d84qiNKCNdjjQtUFqjHl9HHj8ISZ?= =?us-ascii?Q?x1K7wFKVqQF6AFp/5HK+FkMqXLkP3IPQ5Op2jUBhfn1yqVtTGfdL/VSbdlKb?= =?us-ascii?Q?Zi9Z3dtfnEtOM2JpR7ceJZ9f7Rx9N8VI259GBr0BPZWZ91dcbIsEP6S47ZaS?= =?us-ascii?Q?Tec1S0NWcbKcni17ydQNV0WT9OyiM3A2qYYYOlDWkHq9OqZakXzzn/B93Y0q?= =?us-ascii?Q?tkUNt2TFbApjLa9ygOaBy/zfoYNCriM/Lnpjxq6P30m2XXgSNoD7gzrWrwcb?= =?us-ascii?Q?rfjH9VErzeH7XojgszEmJocrANPCZ4yWGnkFQlq8tOIT7v3iGzX3dJ7CIO/n?= =?us-ascii?Q?1oJg9pz9414qZS4ejSkc2gm5Wg7Ntl7JCbTdq4byRpr/O7ZyjVgcYX4JSK5s?= =?us-ascii?Q?dKKzBjBUanYzo4cWk2/Ays4RijXlpHbmQEQlycnKJNJYj+3nl5uf827fLQKR?= =?us-ascii?Q?ed2ts/JRvr6BIZaOtafJkGaM2+9LwsOx2O6JK8kEweS9c8IemuPX1VVmoXFO?= =?us-ascii?Q?QJ2+yyTCKMPwIx6vBXBAUkqOViu8verNTwIOUNRvOoOV/lbmZu68LwrgrRda?= =?us-ascii?Q?OUisrgohGcKx5BugMPQrDaykI5EN90eHSspr8nEPWfAI2ivQWBCCf1G39tVE?= =?us-ascii?Q?Jk3pHcASFKcWt6oFI5K2VxJTTwvMBCetGFapniypWlddVBDgmQtY7l0dArTI?= =?us-ascii?Q?EgqH83bXikuknFXnFO64j7g1MEbb1xeOnQfcpEbwyWmsLzgCnkmG+6rx8XtR?= =?us-ascii?Q?1iEpErlMC2PKtG96a+n2XDE72ctCpTL0Dd+PR6ubM1N4OOpVgERaKo0dSNnj?= =?us-ascii?Q?/9eW2ODy+XzcZdMgst9M0ij7hM6Fr9nNu9EUuqJbuO/Dj6k84xsu+fCFZWu5?= =?us-ascii?Q?pPUx4S3AmLKsWGPQGRe15LLe4rO9dGr7i9fNvPxEdLcLwBHzsQDzfOwM2DHb?= =?us-ascii?Q?GCezN5NbO3zCmy4Z6enQMfUxwKEaXtpT+DFIshXqMBHbfBZbcOvg9v6rGE2h?= =?us-ascii?Q?KzGGtJFLtNe73Ri7yE5UoAH0IZCFfqs7yG+/i2bofb/82Gnr6V/ZWIM4IeFx?= =?us-ascii?Q?Op+fpAJvQ0VfOAR+JsblQ5GpHB04V+8rUy0lTVz2VQNrMVl1xtSPbvj9luTa?= =?us-ascii?Q?ysS41FVFaoFYKCXYGFaM+j9a7awj/qp0PYd?= X-Microsoft-Exchange-Diagnostics: 1;BN3PR07MB2578;6:p4AjLKgQ6w4QEC0Wj4BD3oWbOGxfyrNtIMHbE3/UMw9p6w/3RJUj1tx5kNWl3+pfF3/CIW0iYM+lAqL2KcuAIC+aUckpIb9xaw5zGZ7oewfwVdPm9fHgL+KVDEqnKZtK80TITUUj9l/vDHwWJekwtGZ38JIfxDejLk0muetw3GG8X5RPr9o3rq9ChlPGvNtJsGoyWIpF5cvC6eejPbybSPmOTYBwG+br1VhdyTC7ivWgN1ul/pniCAZrGHZF6lx45RCH3vTAXn0GEUoJEqH76rgOpTsj0A3p3kjYKUHMKDDAkVcOZBF+XuNoHmbSmU/l5lInRna2AsFTaQmSI7InkQ==;5:Qnwpnvze+CUbfoBgT/5inEBVrpPSvb/ngspEktxbKUcCE2k9mMZ1U3KOinxx2nTJA3s69p6N63fN6f1/ITw23ktHC2PA4x2qfixvBOF0fQlSYXgyfs29i2ntkN3JWjYnO/qYhAMdB7XOwMHDSZpwAQ==;24:53p2YRkDVjzON4r9AhnGTEWzVKnOcVwnxK76H0Zmw1hEfRTmAqzxpdA08sfV82rzffXLd8OUtzVKuE2X7hgonfS/nQ4YyT4ChYIL3H6rxD4=;7:dN/RPxbx0nNNDlRRLFKMM/O2W+uYxUHOJ9tizeXFHiRV1fcOY5nQ8074xEYMGhnljBJ+56iGxVnn8bMgbD7u8fZogUxss1F67Eb7fU0SHUevMGbSVb9fKmpmG2+fczudfIh4XqClEzpeLI9k23qC8nd7nN5Fqv+z3Qyeq+BoRgevdMCcnrnhCIPI9jkCWRHsLLKGzloWrJadeHjRxWwTCMuyLdrbihI8x3TgLXiiDTo= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2017 09:41:02.1052 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR07MB2578 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2146 Lines: 58 On Wed, Aug 30, 2017 at 08:40:12AM -0600, Alex Williamson wrote: > On Wed, 30 Aug 2017 16:24:54 +0200 > Jan Glauber wrote: > > > Root ports of cn8xxx do not function after a slot reset when used with > > some e1000e and LSI HBA devices. Add a quirk to prevent slot reset on > > these root ports. > > > > Signed-off-by: Jan Glauber > > --- > > drivers/pci/quirks.c | 16 ++++++++++++++++ > > 1 file changed, 16 insertions(+) > > > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > > index 85191b8..6679971 100644 > > --- a/drivers/pci/quirks.c > > +++ b/drivers/pci/quirks.c > > @@ -845,6 +845,22 @@ static void quirk_cavium_sriov_rnm_link(struct pci_dev *dev) > > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_CAVIUM, 0xa018, quirk_cavium_sriov_rnm_link); > > #endif > > > > +/* > > + * Root port on some Cavium CN8xxx chips do not successfully complete > > + * a bus reset when used with certain types of child devices. Config > > + * space access to the child may quit responding. Flag all devices under > > + * the secondary bus as non-resettable. > > + */ > > +static void quirk_CN8xxx_secondary_bus(struct pci_dev *dev) > > +{ > > + struct pci_dev *pdev; > > + > > + dev_warn(&dev->dev, "Cavium CN8xxx quirk detected; reset for devices on secondary bus disabled\n"); > > + list_for_each_entry(pdev, &dev->subordinate->devices, bus_list) > > + pdev->dev_flags |= PCI_DEV_FLAGS_NO_BUS_RESET; > > +} > > +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_CAVIUM, 0xa100, quirk_CN8xxx_secondary_bus); > > + > > /* > > * Some settings of MMRBC can lead to data corruption so block changes. > > * See AMD 8131 HyperTransport PCI-X Tunnel Revision Guide > > > This doesn't seem reliable, doesn't the user just need to remove and > reprobe the slot and the device would re-appear without this flag set? No, I tried before to disable the slot with "echo 0 > /sys/bus/pci/slots/3/power" but that does not work as it is not supported. I'm not familiar with the quirk types, would another one be better suited here (even if we don't have the problem you descibed)? thanks, Jan > Thanks, > > Alex