Received: by 2002:ac0:a5a6:0:0:0:0:0 with SMTP id m35-v6csp1812553imm; Thu, 23 Aug 2018 09:08:20 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZEaKIWJH9YX0kCeoHYBWs2bV9SePh5piyZKfBk7ma7tBkBHLjFvc/OJoGyJoL54MXC9q3/ X-Received: by 2002:a62:7182:: with SMTP id m124-v6mr6292447pfc.73.1535040500917; Thu, 23 Aug 2018 09:08:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535040500; cv=none; d=google.com; s=arc-20160816; b=RlGDzyI/rhPH1Gjr2Aa4YLBxlTDbqIxztqNPDV4NzduDRdKTGc+ckDnNhT21SJf9yX s66yisbdDM4iQug3WmPCVUNSIYb4Vi4nZrquECK6t+mFTHQzdSKTdWDsSQ4f/DobkJf/ O1rOCwlSYEAk3LahF8wf2QlwCrj0TRXv0AxxaL4XQ9c8B2deKFmWlzEbp8/5NIlGWEa5 QB8MYkSsG1vaFY3F1IvuaRLNaB97nqTRzZ8dhrrLy/Ev94NIY2zXHiPmiPiuXnGKBLmh iwJfbFfs7b1X0jjR0lMhMKUkVlMDC9aqx2DFsi3KDYYC/0fU3v5Odpf0nNX7ESLOD7F8 0pgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:dkim-signature :arc-authentication-results; bh=3fhCBvMue621Ph8Y8vS6p2pj5kdopVtntR5YRiE5l9E=; b=EUhmQEClRDSk4iZhliLAO3jjPaIHFQsfm2709lr0YeCYMGOPZfbLPqrXykay7D6oLY VBYHha0TQFy4yylBidSJ0wvXiQwHpzGzzEcfb+VaZvFhOCj8YSA4GGWf1Yi8zFFrGkFK 7llC6Ua+SDfuzeGHiTiROJyGdxI76UrUwtiUYKJBNi87i1IAP0zjcK0hStbSObUm99eL L5LeA/pm0/MvN382eahYCi9NMdhAtL4Jb+4+/slw5wGIMY9yeupijLz9bEe7eAFyQ6Hf 8RccNsRXh4WnuJGT9VzOThQX6H2fwV78vJyzOunNvEvJnjvtzu9erl/my6AbakBvA6QF 3HhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@endlessm-com.20150623.gappssmtp.com header.s=20150623 header.b="K/rnKg/4"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 43-v6si4507188plc.496.2018.08.23.09.08.05; Thu, 23 Aug 2018 09:08:20 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@endlessm-com.20150623.gappssmtp.com header.s=20150623 header.b="K/rnKg/4"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729195AbeHWOQR (ORCPT + 99 others); Thu, 23 Aug 2018 10:16:17 -0400 Received: from mail-it0-f65.google.com ([209.85.214.65]:33714 "EHLO mail-it0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727162AbeHWOQQ (ORCPT ); Thu, 23 Aug 2018 10:16:16 -0400 Received: by mail-it0-f65.google.com with SMTP id j198-v6so1620703ita.0 for ; Thu, 23 Aug 2018 03:47:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=endlessm-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=3fhCBvMue621Ph8Y8vS6p2pj5kdopVtntR5YRiE5l9E=; b=K/rnKg/43Hj/+gFWWYCWwQ/ytEZ4SfGnpwq685zurqz0GaWLDOo8+eZAU2sn+HIYRl uPXwb33X+6qI2yJSzMakpWho4fuvAENMGsS/5LkVZNk4d2iu8VOrmYcR3+r1Bot7a0u1 BYHCjSwC09FKfvX2dvQH0e66VXSHjvG0/1n2TZhttSm+IoqruJaJF7Jmhdg1aTy6Sadk GK9oRwx2f7f4QrNSrvHTQMGyZe/FIONmTquuNC3Dcu5JdnVlQUv1q+WWsreMCauJJHm7 tDX2yNWTXgJ+t5Xx0J/Qp4Ya9d8Luum8U0TtedTmkUyzSxMdquzLFQrRF1/mUOcvXllE X0Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=3fhCBvMue621Ph8Y8vS6p2pj5kdopVtntR5YRiE5l9E=; b=Sa9a/Sfe8npycHGV+sx+Af6f6SzZ+yKT46E1/5wHnKvuzkxRVmaAK+U9PCRAEjTKLZ BS+saf6UNFhJRjYNRYMKmnaeeAbzbXq5jYQ9dzs0uvfE4EO65Quv3GZ4nYf/cxYwM3RO NC5d/WMjqNw3tOdPlB1m5qWNSynjIJjHDdaevQ6Zokb5kXorjGw1Fmxjx50xqoxeUKc7 tFIGVlgaAF1GmBjm4a8Yu78yeD58w1a2+JuJtJ4GL7sF3nvTUGeHQBDXxvfkXA0wpLAg oNiZvFJUp0AACNv4KcT1a2a6/A1KS6xi7t4vaZUIHPda3D7ITTi//619a9+kDFIlM8UP dD9w== X-Gm-Message-State: AOUpUlE8q8ajMF6ZCb3JcTo0U8caEFjMBBGZbvLXxMlE47vRXR4Tt6Cv ITM0u4gPZhZoEF1qLZlWXgg6np0hphUMW0qkITSOCQ== X-Received: by 2002:a02:8952:: with SMTP id u18-v6mr51827845jaj.13.1535021228538; Thu, 23 Aug 2018 03:47:08 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a4f:7383:0:0:0:0:0 with HTTP; Thu, 23 Aug 2018 03:46:28 -0700 (PDT) In-Reply-To: <36bd086d-8d26-5162-ae24-95b259571221@gmail.com> References: <20180820184438.GA154536@bhelgaas-glaptop.roam.corp.google.com> <9d7d960a-6c55-dc4b-7969-f5cf46bff0ce@gmail.com> <20180821.123108.89921430801253333.davem@davemloft.net> <36bd086d-8d26-5162-ae24-95b259571221@gmail.com> From: Jian-Hong Pan Date: Thu, 23 Aug 2018 18:46:28 +0800 Message-ID: Subject: Re: [PATCH] r8169: don't use MSI-X on RTL8106e To: Heiner Kallweit Cc: Thomas Gleixner , David Miller , helgaas@kernel.org, Realtek linux nic maintainers , netdev@vger.kernel.org, Linux Kernel , Linux Upstreaming Team , linux-pci@vger.kernel.org, marc.zyngier@arm.com, hch@lst.de Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 2018-08-23 3:49 GMT+08:00 Heiner Kallweit : > On 22.08.2018 13:44, Thomas Gleixner wrote: >> On Tue, 21 Aug 2018, Heiner Kallweit wrote: >>> On 21.08.2018 21:31, David Miller wrote: >>>> From: Heiner Kallweit >>>> Date: Mon, 20 Aug 2018 22:46:48 +0200 >>>> >>>>> I'm in contact with Realtek and according to them few chip versions >>>>> seem to clear MSI-X table entries on resume from suspend. Checking >>>>> with them how this could be fixed / worked around. >>>>> Worst case we may have to disable MSI-X in general. >>>> >>>> I worry that if the chip does this, and somehow MSI-X is enabled and >>>> an interrupt is generated, the chip will write to the cleared out >>>> MSI-X address. This will either write garbage into memory or cause >>>> a bus error and require PCI error recovery. >>>> >>>> It also looks like your test patch doesn't fix things for people who >>>> have tested it. >>>> >>> The test patch was based on the first info from Realtek which made me >>> think that the base address of the MSI-X table is cleared, what >>> obviously is not the case. >>> >>> After some further tests it seems that the solution isn't as simple >>> as storing the MSI-X table entries on suspend and restore them on >>> resume. On my system (where MSI-X works fine) MSI-X table entries >>> on resume are partially different from the ones on suspend. >> >> Which is not a surprise. Please don't try to fiddle with that at the driver >> level. The irq and PCI core code are the ones in charge and if you'd >> restore at the wrong point then hell breaks lose. >> > Instead of spending a lot of effort on a workaround which may not be > acceptable, it may be better to fall back to MSI on all affected chip > versions. For two chip versions which were reported to have this issues > we're doing this already. I asked Realtek whether they have an overview > which chip versions are affected, let's see .. > > The Realtek chips provide an alternative, register-based way to access > the MSI-X table, and their Windows driver seems to use it. See here: > https://patchwork.kernel.org/patch/4149171/ > > But as we handle all MSI-X basics in the PCI core, this isn't an option. > > >> Can you please do the following: Tested on ASUS X441AUR equipped with RTL8106e. This is the laptop whose ethernet does not come back after resume, if it does not fallback to MSI. Here is the full dmesg: https://gist.github.com/starnight/e65a97c9bf2d558926895ab76974687e >> 1) Store the PCI config space at suspend time Before suspend: dev@endless:~$ sudo lspci -xnnvvs 02:00.0 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8101/2/6E PCI Express Fast/Gigabit Ethernet controller [10ec:8136] (rev 07) Subsystem: ASUSTeK Computer Inc. RTL810xE PCI Express Fast Ethernet controller [1043:200f] Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- > 2) Compare the PCI config space at resume time and print the difference After resume: dev@endless:~$ sudo lspci -xnnvvs 02:00.0 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8101/2/6E PCI Express Fast/Gigabit Ethernet controller [10ec:8136] (rev 07) Subsystem: ASUSTeK Computer Inc. RTL810xE PCI Express Fast Ethernet controller [1043:200f] Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- > Do that on a working and a non-working version of Realtek NICs. >> >> Thanks, >> >> tglx >> >> >> >