Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 12.22.5.236 as permitted sender) receiver=protection.outlook.com;
 client-ip=12.22.5.236; helo=mail.nvidia.com;
Message-ID: <708b746c-1715-9d64-5a59-55c9ad81bd0a@nvidia.com>
Date:   Wed, 2 Feb 2022 09:47:06 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.2
Subject: Re: [PATCH V1] PCI/ASPM: Save/restore L1SS Capability for
 suspend/resume
Content-Language: en-US
To:     "Kenneth R. Crudup" <kenny@panix.com>
CC:     <bhelgaas@google.com>, <lorenzo.pieralisi@arm.com>,
        <hkallweit1@gmail.com>, <wangxiongfeng2@huawei.com>,
        <mika.westerberg@linux.intel.com>, <kai.heng.feng@canonical.com>,
        <chris.packham@alliedtelesis.co.nz>, <yangyicong@hisilicon.com>,
        <treding@nvidia.com>, <jonathanh@nvidia.com>, <abhsahu@nvidia.com>,
        <sagupta@nvidia.com>, <linux-pci@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <kthota@nvidia.com>,
        <mmaddireddy@nvidia.com>, <sagar.tv@gmail.com>
References: <20220201123536.12962-1-vidyas@nvidia.com>
 <8aa96f79-402-4897-424f-64a2c6893de8@panix.com>
 <2ab59f0d-3ae7-ca5e-6bfc-12bed18813b2@nvidia.com>
 <38a02915-906f-c53-7e13-6c8710315e7@panix.com>
From:   Vidya Sagar <vidyas@nvidia.com>
In-Reply-To: <38a02915-906f-c53-7e13-6c8710315e7@panix.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2022 04:17:17.1857
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b290e84-6b56-4a90-d9da-08d9e602e611
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT062.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1301
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 2/2/2022 3:55 AM, Kenneth R. Crudup wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Wed, 2 Feb 2022, Vidya Sagar wrote:
> 
>> BTW, I see that the ASPM L1SS capability is supported by only two endpoints
>> viz. KIOXIA's NVMe drive and Realtek's Card reader. None of the root ports
>> seem to have the support. So, I'm wondering how was it even getting enabled in
>> the first place earlier?
> 
>> (OR)
> 
>> was it the case that L1SS sub-states were never enabled earlier also and the
>> issue was occurring without having ASPM L1SS enabled? (but with only L0s and
>> L1 enabled??)
> 
> I'm not proficient enough in PCIe to be able to be sure of the answers to those-
> what can/could I do to determine this?
Nothing at this point, but could you please confirm that you are using 
the same system as before? if that is the case, then, I'm not sure how 
is it possible that the earlier patch which is also for saving/restoring 
L1SS registers could affect a system that doesn't even support L1SS.

Bjorn, any thoughts on this?

> 
>> Also, I see that from 'before' and 'after' logs that for both NVMe and Card
>> reader and their corresponding root ports, none of the ASPM states are enabled
>> (not even L0s or L1).
>> Did you set the policy to 'powersupersave' before hibernating the system?
> 
> Yeah:
> 
> CONFIG_PCIEASPM_POWER_SUPERSAVE=y
> 
> My laptop loses ~1.5%/hr in S3; I was trying anything I could to reduce that,
> if possible.
> 
>          -Kenny
> 
> --
> Kenneth R. Crudup / Sr. SW Engineer, Scott County Consulting, Orange County CA
>