Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 12.22.5.235 as permitted sender) receiver=protection.outlook.com;
 client-ip=12.22.5.235; helo=mail.nvidia.com;
Message-ID: <0ccd43af-0293-521b-86d3-ea2ea1c70e5c@nvidia.com>
Date:   Wed, 16 Feb 2022 10:10:07 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.2
Subject: Re: [PATCH V1] PCI/ASPM: Save/restore L1SS Capability for
 suspend/resume
Content-Language: en-US
To:     "Kenneth R. Crudup" <kenny@panix.com>,
        Bjorn Helgaas <helgaas@kernel.org>
CC:     <bhelgaas@google.com>, <lorenzo.pieralisi@arm.com>,
        <hkallweit1@gmail.com>, <wangxiongfeng2@huawei.com>,
        <mika.westerberg@linux.intel.com>, <kai.heng.feng@canonical.com>,
        <chris.packham@alliedtelesis.co.nz>, <yangyicong@hisilicon.com>,
        <treding@nvidia.com>, <jonathanh@nvidia.com>, <abhsahu@nvidia.com>,
        <sagupta@nvidia.com>, <linux-pci@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <kthota@nvidia.com>,
        <mmaddireddy@nvidia.com>, <sagar.tv@gmail.com>
References: <20220207163332.GA397884@bhelgaas>
 <79a41832-12e2-2ed-f3af-da3bac90d941@panix.com>
 <3ca14a7-b726-8430-fe61-a3ac183a1088@panix.com>
From:   Vidya Sagar <vidyas@nvidia.com>
In-Reply-To: <3ca14a7-b726-8430-fe61-a3ac183a1088@panix.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Feb 2022 04:40:16.3953
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 79b82b77-2d53-4b95-a699-08d9f1066dea
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT038.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB0057
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,
	T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 2/15/2022 6:40 PM, Kenneth R. Crudup wrote:
> External email: Use caution opening links or attachments
> 
> 
>> On Mon, 7 Feb 2022, Bjorn Helgaas wrote:
> 
>>> If the problem really was fixed by some change elsewhere, it *should*
>>> still happen on v5.11-rc1.  I think we should verify that and try to
>>> figure out what the other change was.
> 
>> Yeah, not a bad idea. I'm a little busy now, but I'll cut a kernel at
>> 4257f7e0, remake, and see what happens. Gimme a day or so.
> 
> So I did this- checked out a branch hard-reset to commit 4257f7e0, used my
> current _defconfig on it, and built and tested it- NOW it doesn't exhibit the
> NVMe-device-killing behavior I'd seen a year earlier; I'm assuming Dell must
> have fixed something in the BIOS?
> 
> Anyway, I'd done an "lspci -vvvnn" before and after hibernation, if you're
> interested, and a diff of same; I'll send those in subsequent E-mails.
Thanks Kenny for the dump.
I see that the ASPM-L1 state of Realtek NIC which was in disabled state 
before hibernate got enabled after hibernate. This is weird. This patch 
doesn't do anything to LnkCtl register which has control for ASPM L1 
state. Could you please check why ASPM L1 got enabled post hibernation?

> 
>          -Kenny
> 
> --
> Kenneth R. Crudup / Sr. SW Engineer, Scott County Consulting, Orange County CA
>