Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <b0514d28-39f5-e66c-bb41-fc5450266e11@intel.com>
Date:   Tue, 6 Sep 2022 09:46:43 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.0
Subject: Re: [PATCH v5 07/12] x86/cpu/keylocker: Load an internal wrapping key
 at boot-time
Content-Language: en-CA
To:     Evan Green <evgreen@chromium.org>
CC:     <linux-crypto@vger.kernel.org>, <dm-devel@redhat.com>,
        <herbert@gondor.apana.org.au>, Eric Biggers <ebiggers@kernel.org>,
        "Ard Biesheuvel" <ardb@kernel.org>, <x86@kernel.org>,
        <luto@kernel.org>, "Thomas Gleixner" <tglx@linutronix.de>,
        <bp@suse.de>, <dave.hansen@linux.intel.com>, <mingo@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Dan Williams <dan.j.williams@intel.com>,
        <charishma1.gairuboyina@intel.com>,
        <kumar.n.dwarakanath@intel.com>, <ravi.v.shankar@intel.com>
References: <20220112211258.21115-1-chang.seok.bae@intel.com>
 <20220112211258.21115-8-chang.seok.bae@intel.com>
 <CAE=gft4P2iGJDiYJccZFR1VnNomQB7Uo522r2gvrfNY9oKz5jg@mail.gmail.com>
 <763bddd2-2fc3-a857-0dff-a5ae4ae1f298@intel.com>
 <CAE=gft5VajfoAT6hVxiCzAMYiDV+pHRbC-F7s4+qK94qa0og5w@mail.gmail.com>
 <6f2dcc0c-99a2-8698-13ae-d5cbea9945b0@intel.com>
 <CAE=gft6sO85RBpfdXnR=_r=KGmVrx0mnBOHK7XadNk00gDr-WA@mail.gmail.com>
 <a3fd8d2d-72d0-ba13-ef77-caff2b239867@intel.com>
 <CAE=gft7BQAwHTxoCtSu-gHZDtcETvBM-nony_nzBD8ziE_HcZw@mail.gmail.com>
From:   "Chang S. Bae" <chang.seok.bae@intel.com>
In-Reply-To: <CAE=gft7BQAwHTxoCtSu-gHZDtcETvBM-nony_nzBD8ziE_HcZw@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cHdlTzNuSmxmN1krQkI4THRpOTFHaC9GamdPcHM0SGdxcjducGgxeER5ajFB?=
 =?utf-8?B?SE91dUx0OWZWbXZDOGdXV0hwOFQ3RHNuRjJxT2QrYXB0TktPTnBJS1Q2SVBB?=
 =?utf-8?B?Z2Q2RmprcEdrWk1rMHA4U2RjaVRvTHQrYTNndk02M2E5NjNZa2VBdVZtQ1pm?=
 =?utf-8?B?WWxUYldsRkMrSmk0L0p3YWhiNHV6SU05bGdqQURmTjdXeFlacUxONysyZTB3?=
 =?utf-8?B?VVIyREVMd21MYnlKTm4zdGJtRyt6L0xhK3grMHlyOTFYM1IyaEErVEZhNzYw?=
 =?utf-8?B?WFI5RmFqV3lCamFjYURmc1d3NnNUNC92RkR3QUJnY3RXOFVid3lNZWVMWE0y?=
 =?utf-8?B?TlM1M3BHaFBiZHVNY3FKZDFLTWNtQXlOL2laU3kzczFBNEladDFzeEd6REFJ?=
 =?utf-8?B?NmppT3BxNUNiR2NENjRaVlR2VktHMi9oQ0t1T1J3Ri9XZTNMTTZDQUkvNlh1?=
 =?utf-8?B?QVJWTnVwUjU0bG1mVmtmVzFIQzVScEVoMG5mM1VSYnplaUgwRi9wR0NBcDkz?=
 =?utf-8?B?Y0JZa3pKVkJ0bGk2TW5sOXc5Tkc3MCsxTjliU3VwU2pDSWNTQVBiNy95a3Vo?=
 =?utf-8?B?OVBPMHB3UUNOY3ZmY3NCdWhEbUR2QVdHOWMzbFFkQ0hlVDJXWStJRjVrRVFX?=
 =?utf-8?B?R0xZOGxDR0pwUmFNaER6bHRnY3A4VkZUdDFPd0FzTWo0aXprWFU5RSswaklJ?=
 =?utf-8?B?OEp3TkEwbFJFcW9pWTRkcnNQYWVsTTdBMjNLQmZaaTR6Rzh5dVJlU3dwbmM0?=
 =?utf-8?B?Q2E2VnZ6RW4xSUVkRmtKNHZONWVtazlQSGkveWJiUmtCTlVMbVh2a2Zla2hP?=
 =?utf-8?B?R0cwSW1QMDVQOEppMlFxZHA2K1NsY0NqUXFGa2VYS3BsOUNXOWZ5NGw4REpO?=
 =?utf-8?B?UUJzRGhXdlQxNTgzZ0VMRnAycEdiRU9GS3VTc3JrWHJKYkpBZkJnVTdCNHRZ?=
 =?utf-8?B?MFdQKzZydVYvNlgzUzhwZWJIcFpCczAvVEU0VkdVVGpyamVFMnp4citLR1p3?=
 =?utf-8?B?NFhYeHB0S3Z5SHBHWktaN3FheGpwd3JnNE9GcTh1dVBGdDkzNVB0SVlsaG42?=
 =?utf-8?B?bjNnZ3J2VEd4Yk9wUHZlMFVrR2prcnB3aXBYVFRNNzlHNWhHd1lUM2NUbUNH?=
 =?utf-8?B?OEJkTllZc0FiWmRMaWsvSEVzTEJsUmtZdWRRZXZXNHB1c1BMY2dINndNTVpD?=
 =?utf-8?B?b3k5azl6c3VmVFBxN3o4YTlDTGxydXZITGNJaTZnRnlsT0NLTlBEektWd0tG?=
 =?utf-8?B?T043TkxBd3F0NnV3MGFZVmJ0cS9SRTViSnVvNXU4aktPV2RPR0s3RmZmcXV2?=
 =?utf-8?B?eXY1WElXZU4rZHBwdlhiK2h6UnQxTk1JNm5VK0YvekhZY0wyaGJWRVFocVdl?=
 =?utf-8?B?SXBadjArS0FOanBuVmFKcW5RT05nRmdzQ1AyZUJmUS9KNlhFSjFEOVVEV3Np?=
 =?utf-8?B?T0lHc2xsTXM2SGdqYmEyODNNLzBBQ2w4UE9tbitEMXZpMENPV1NvREc3Y0pN?=
 =?utf-8?B?QXJGWVdkNFZqY2VPQk5PRkh1Q3dRWXBVN205WnpUeXVTODFkWnl3MTg1SW1D?=
 =?utf-8?B?Q0JwYkN1RjFqTlo5MGYwSFBoQXZQVi91RmpkdGhjRkNrUjRaVXNyRjY5Q1V0?=
 =?utf-8?B?alFyMVBpK0JnaEROV0NsK1dvaGxQbDk3Y2JOQmRKWTRUZVcxcmJqaUd0WFdG?=
 =?utf-8?B?ejkybmZ0dzhLMzRzdnczL1I3T2RYSXdZUlRhZHd6WURVTm5jcjU2cWRObW80?=
 =?utf-8?B?L0lHOHp3TEVlYUo1bEJNSkVkWjFzOWZpZlNOdVdndGwwc2g2b3c3RUhnUkdJ?=
 =?utf-8?B?ZHdwdHcxWXVGRzVyS1NMRWdIZDRWVldnSlBiZFJoa0lIWEdpL3BhTHVtQUI4?=
 =?utf-8?B?OVhOdW5sUFVTZytXa0RoNXNiWjg3c21VN3phYWtJcy9IRHNmcFBoTjdoakQr?=
 =?utf-8?B?Uk5LMEswQ1loejVlTGVTSVk4WDFRNHY4RTR4V2hjOURRVFlQb3IrVnp0QWls?=
 =?utf-8?B?bjJuend0Z3RQQkZHZHN2L3JGWUFqZWM0M1MxZzY0QnlBNEQveHhUVXVEVzlp?=
 =?utf-8?B?RkpxVzRTVWxpYTVhWTcyNGJkQVpEZFdnRXQ3M1p1bWdweldGa0pTR3dMaktL?=
 =?utf-8?B?NjJ4cHdwMDFXSVRMbm1YUG9XTmlyTE1ZM1V3OEFER3MwdjNiYkFvMG80QXdD?=
 =?utf-8?B?eWc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 00afc3f6-cf72-4411-1903-08da9027628d
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4855.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 16:46:45.9290
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OzxubLFef+X/6oriAowI1yULPSCZrq+w1BQzbCe0viTDFH8MVhFecJ7/CI0J0BtHIl2AiecjRQzV7EmHEOyfa93zikwDdeyzauCcDBX+8/g=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB6999
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-6.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

On 9/6/2022 9:22 AM, Evan Green wrote:
> On Wed, Aug 31, 2022 at 4:16 PM Chang S. Bae <chang.seok.bae@intel.com> wrote:
> 
> But this still doesn't explain this crash, since in this case we're
> still on our way down and haven't even done the shutdown yet. We can
> see the "PM: hibernation: Image created (1536412 pages copied)" log
> line just before the keylocker read failure. So then it seems
> something's not working with the pre-hibernate CPU hotplug path?

No, that backup crash message came from the boot-CPU's wakeup path:
__restore_processor_state()->restore_keylocker().

If the issue is repeatable, then I suspect that's something to do with 
the specific platform implementation. I don't have any detail yet on the 
customized systems. Let me check with some Chrome HW folks.

> How were you exercising the CPU onlining in this case? Boot, cpu
> hotplug, or hibernate?

Hibernate.

Thanks,
Chang