Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Message-ID: <8712ccc3-8619-41b8-97d0-d0187c0b59c6@kunbus.com>
Date:   Mon, 20 Nov 2023 21:08:52 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: tpm_tis_spi_remove() triggers WARN_ON() in __flushwork (RPi3B+
 and SLB9670)
Content-Language: en-US
To:     Jarkko Sakkinen <jarkko@kernel.org>,
        linux-integrity@vger.kernel.org
Cc:     Lukas Wunner <lukas@wunner.de>, linux-kernel@vger.kernel.org
References: <CX32RFOMJUQ0.3R4YCL9MDCB96@kernel.org>
 <CX342W32D30U.330BVFC336MA8@kernel.org>
From:   Lino Sanfilippo <l.sanfilippo@kunbus.com>
In-Reply-To: <CX342W32D30U.330BVFC336MA8@kernel.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WW43R0ZTZC9oaHNFUlIrci95aTlNaDByVlVVQ0R4SWhma0dkenkxdVpza0pl?=
 =?utf-8?B?bkFCZ2p4SExPdVhuY0dYbUtKQjh0R3B1SEMvbms4RFB1cVBoUHdWcURiSHhi?=
 =?utf-8?B?eSt4NExxdCtvQm5CalhZcVVEcU1sU0ZQdGxBcEMzRnVyUysvd1lMYWlxYWRk?=
 =?utf-8?B?aitlRURsNlVrL0tqQlhDUmRteFFENW5RVVdqY21TMW1yTnFDclZOejlRejYw?=
 =?utf-8?B?RDdPU0llNDU5cUdTRmtwOVNOUzd5TmViOVcwSFZteDdQeUZoNHl1aHA3WUtY?=
 =?utf-8?B?cnh5SmV1QW02TnJ5OVRLbXBHSlpwOXVpOEoxQlNBN2FMeHVXS1NndGNNWWpZ?=
 =?utf-8?B?c2VpWDFaTVZhUjBtc2RjQ1JrWW1NbmgwdERkUXRhNzQ1NTBLOUIweUdwSHZv?=
 =?utf-8?B?bE1CZ2ljT1lweHFWU2sxNEhOM2t0cTN6UzRxM24vN1RhMTJSb2Ztd0JoYVhy?=
 =?utf-8?B?WHZ1UnRZVjMyM0RVUG9ZUkJYREJXWExmSzBBWHd0bUI4Y0pwS1l4YlY4aWhj?=
 =?utf-8?B?ZmthaiswNlp3Vy9BclV4QTl3cGduY0xqbTd0UERla2JucXliMjR1c1dpc2Z0?=
 =?utf-8?B?SGZUOEl2Sk9oU1V4WG1jbzU1OXRIa1pQZUxoTkhneUoxRS9naSt2c0tMa1g0?=
 =?utf-8?B?ZjJkS0dsZC85TkVwYjh3djJ2Y01vMU13MGZjMW5RTzV5NG8vMG5Rc1hKRkJ3?=
 =?utf-8?B?T2dxY3R0VnNrL3VjZzdqdTZpYnF6YWx1aFNpcEZOSkN0Q0R6ZDA4Z0Z1RmdF?=
 =?utf-8?B?UmFiNkY3SXNXY0t3V2VnQU54RzF3MVpZUXVCRFNjNlZ6MFBNME4rd2M3V2J5?=
 =?utf-8?B?amRMK2pKbnMzUnBvY05QU2tBcnVXaHF2cE8zQ3R1Q05yNmVpZzczK1Z6Skgv?=
 =?utf-8?B?cHJnTTVSMGtuZ1hpMW5PSUtzSWVxb2c4MFpNcm45KzlKeWhhVjJFK0l1dVNI?=
 =?utf-8?B?Q1lIS0NoMkZvV0VXc0FYRTR2YytsRC9XWDR6UmJjTTFkUXIvOE9JQk5TV3Z3?=
 =?utf-8?B?ZmFkMTdOQmNoSW1PNmhXVXZpYmV2WlhaSityUTd1dzFkVS9ITVZpSDZ5cy9t?=
 =?utf-8?B?bDRBNzdrMi9OVzhCSS9taGxhOWVJcDE4cGtYVksxYzVNblMxMUtuTnlsYWxY?=
 =?utf-8?B?MVREUzJyb0NWNlhTblRUOFpPU0JxcTd0dmkxeC9SNm9KOFN6c3hKaHd3c0JL?=
 =?utf-8?B?NXNBUDdGclhwbTVJUCtJajU0ZmlDSnVubmsvSlJQbGFnK284MVFMb2ptWVAv?=
 =?utf-8?B?dGl3eUg5ODY2Tms1MnN1VHhoaVc1bkNXMDBZTGVSNFFlbU1HWTExK3NTUzBh?=
 =?utf-8?B?Zjk0eElTUHBaVFNmbU5iSEFTNXp2Mk1qNTV5SGpDU1h5TDByNUxVQkVwOTZI?=
 =?utf-8?B?cVdWdDFLbHY2Q0xIaEE0blNPS0VHRm1reDNjdFVoemNoVm5SV3U1WGtmcEZL?=
 =?utf-8?B?MGJkUWdvSXhIMVliY2x6V3lkN2MxWGVXWHVLMC9naFVZZUNYTHE3T1FoOUJY?=
 =?utf-8?B?UHVyOUlFczZrQVVBZGRwZjJmZVdRVnFZU0hFZTFXcDkzVHBqVytQYk81aFZK?=
 =?utf-8?B?OXNPejFHdGpVYWlJWGVWc2E3QjFaSURBKzZPaVdOaWViSHZlZzFsL0NEb0Mw?=
 =?utf-8?B?WU04Q1JsZFFGQU1lZ01vQktyQlhjV2NRT3FtUkIrTEdPMFZ2Q2IyV3p0Um04?=
 =?utf-8?B?N2pWWmZJSmpnMU95NGNib0hMSm9CUzRhOWdpemw1KzRYSjRCek9qbHBqNnN6?=
 =?utf-8?B?bHJObmN5b0pCenZPMUV0cG96Z2lRTHJHNm1FR1ZxdW83YUxtTm5xZE1OLzFP?=
 =?utf-8?B?WjlLWlVNTTFOVWl0VWpQTVdXVWdhc3dnY3ZyZWtwazV1blF2YkdwRmVjN2s2?=
 =?utf-8?B?aGk3Y3IvQ2l5dFV4cGVkcmxMbEdWM05TdkhTa1dYck83OWw0MFp1ODB0Q3dr?=
 =?utf-8?B?am8vQlFmSzlaUTk5ZVVyNzJEMmtBMHJNRW05bXJwZVV6a0cvWTZSaTlBTFhF?=
 =?utf-8?B?MHZaRE14b3VSdmd0dmtQdms1bXdMMjkzTVpFanRFMTJkYXZ0cWVNa2VuZTVl?=
 =?utf-8?B?ei93R3Zla0Rab1FQQ3B6c1E3R3ZwUnFEd0ZwbVVpeXEzdzdoVGpYVkRaS0x5?=
 =?utf-8?B?eGFnZURHZ1d2RTFuYUpYSWUrU01sSXZZVUNId0N1OERXcHFFd29IdGdIejQ0?=
 =?utf-8?Q?ljI8qbz19tVYI1jvxeRWM5qtpFG3CKj7TSW1R/RPkfev?=
X-OriginatorOrg: kunbus.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 416f332a-2906-4bb5-0460-08dbea048599
X-MS-Exchange-CrossTenant-AuthSource: VI1P193MB0413.EURP193.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Nov 2023 20:08:54.7499
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: aaa4d814-e659-4b0a-9698-1c671f11520b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tOaggqfcTkzpCDU31Z+F9pBgQ9zuDFkpW4anTijlccLEXnxNOb5JcZlZkq1y1ZLr1eT5tj6z7SMeIumMNz6/Rg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8P193MB1288
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 20 Nov 2023 12:09:09 -0800 (PST)

Hi,

On 19.11.23 22:37, Jarkko Sakkinen wrote:

> 
> On Sun Nov 19, 2023 at 10:35 PM EET, Jarkko Sakkinen wrote:
>> Captured from serial link with Raspberry Pi 3B+ and Infineon SLB9670 TPM2 chip, i.e.
>> triggers here:
>>
>> static bool __flush_work(struct work_struct *work, bool from_cancel)
>> {
>>       struct wq_barrier barr;
>>
>>       if (WARN_ON(!wq_online))
>>               return false;
>>
>>       if (WARN_ON(!work->func)) /* <-- */
>>               return false;
>>
>>
>> # uname -a
>> Linux buildroot 6.6.1-v8 #1 SMP PREEMPT Sun Nov 19 21:46:00 EET 2023 aarch64 GNU/Linux
>> # poweroff
>> # Stopping dropbear sshd: OK
>> Stopping network: [  246.487818] smsc95xx 3-1.1:1.0 eth0: hardware isn't capable of remote wakeup
>> OK
>> Stopping klogd: OK
>> Stopping syslogd: OK
>> Seeding 256 bits and crediting
>> Saving 256 bits of creditable seed for next boot
>> umount: devtmpfs busy - remounted read-only
>> [  246.623163] EXT4-fs (mmcblk0p2): re-mounted c5bb63df-c03e-4e4a-9846-0cdf5986edc4 ro. Quota mode: none.
>> The system is going down NOW!
>> Sent SIGTERM to all processes
>> Sent SIGKILL to al[  248.680154] ------------[ cut here ]------------
>> [  248.684825] WARNING: CPU: 1 PID: 298 at kernel/workqueue.c:3400 __flush_work.isra.0+0x29c/0x2c4
>> [  248.693582] CPU: 1 PID: 298 Comm: init Tainted: G        W          6.6.1-v8 #1
>> [  248.700926] Hardware name: Raspberry Pi 3 Model B Rev 1.2 (DT)
>> [  248.706780] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
>> [  248.713774] pc : __flush_work.isra.0+0x29c/0x2c4
>> [  248.718415] lr : __flush_work.isra.0+0x44/0x2c4
>> [  248.722970] sp : ffffffc0812fb910
>> [  248.726299] x29: ffffffc0812fb910 x28: ffffff8003e30e40 x27: 0000000000000000
>> [  248.733481] x26: fffffff0350c9c10 x25: 0000000000000000 x24: fffffff03500c028
>> [  248.740661] x23: fffffff03500d208 x22: 0000000000000001 x21: fffffff034f37568
>> [  248.747840] x20: ffffff80064d9ac0 x19: ffffff80064d9a80 x18: ffffffffffffffff
>> [  248.755019] x17: 0000000000000000 x16: 0000000000000000 x15: ffffffc0812fb760
>> [  248.762197] x14: 0000000000000004 x13: ffffff8002808410 x12: 0000000000000000
>> [  248.769376] x11: 0000000000000040 x10: fffffff034f35a98 x9 : 0000000000000004
>> [  248.776554] x8 : ffffffc0812fb9a8 x7 : 0000000000000000 x6 : 00000000000003e8
>> [  248.783732] x5 : fffffff034e46000 x4 : 0000000000000000 x3 : 0000000000000000
>> [  248.790909] x2 : 0000000000000008 x1 : 0000000000000000 x0 : 0000000000000000
>> [  248.798087] Call trace:
>> [  248.800546]  __flush_work.isra.0+0x29c/0x2c4
>> [  248.804841]  flush_work+0x10/0x1c
>> [  248.808177]  tpm_tis_remove+0x90/0xc8
>> [  248.811866]  tpm_tis_spi_remove+0x24/0x34
>> [  248.815901]  spi_remove+0x30/0x4c
>> [  248.819238]  device_remove+0x4c/0x80
>> [  248.822836]  device_release_driver_internal+0x1d4/0x228
>> [  248.828088]  device_release_driver+0x18/0x24
>> [  248.832381]  bus_remove_device+0xcc/0x10c
>> [  248.836421]  device_del+0x15c/0x41c
>> [  248.839934]  spi_unregister_device+0x48/0x98
>> [  248.844231]  __unregister+0x10/0x20
>> [  248.847742]  device_for_each_child+0x60/0xb4
>> [  248.852037]  spi_unregister_controller+0x48/0x15c
>> [  248.856768]  bcm2835_spi_remove+0x20/0x60
>> [  248.860804]  platform_shutdown+0x24/0x34
>> [  248.864751]  device_shutdown+0x150/0x258
>> [  248.868701]  kernel_power_off+0x38/0x7c
>> [  248.872563]  __do_sys_reboot+0x200/0x238
>> [  248.876511]  __arm64_sys_reboot+0x24/0x30
>> [  248.880546]  invoke_syscall+0x48/0x114
>> [  248.884324]  el0_svc_common.constprop.0+0x40/0xe0
>> [  248.889057]  do_el0_svc+0x1c/0x28
>> [  248.892397]  el0_svc+0x40/0xe8
>> [  248.895478]  el0t_64_sync_handler+0x100/0x12c
>> [  248.899864]  el0t_64_sync+0x190/0x194
>> [  248.903549] ---[ end trace 0000000000000000 ]---
>> [  248.910555] reboot: Power down
>>
>> Just putting out. I was testing https://github.com/jarkkojs/buildroot-tpmdd/tree/linux-6.6.y
>> and this popped up. To build sdcard.img bootable with Raspberry Pi 3:
>>
>> make raspberrypi3_tpmdd_64_defconfig && make
>>
>> BR, Jarkko
> 
> So I applied [1] and now I get a clean shutdown:
> 

AFAIU the warning is shown in case that interrupts are not enabled and thus INIT_WORK has not
been called for free_irq_work. This should not be too hard to fix, so I think I can provide 
a patch for that, soon.

BR,
Lino

> # poweroff
> # Stopping dropbear sshd: OK
> Stopping network: [   66.189514] smsc95xx 3-1.1:1.0 eth0: hardware isn't capable of remote wakeup
> OK
> Stopping klogd: OK
> Stopping syslogd: OK
> Seeding 256 bits and crediting
> Saving 256 bits of creditable seed for next boot
> umount: devtmpfs busy - remounted read-only
> [   66.353850] EXT4-fs (mmcblk0p2): re-mounted 6accf2b0-3291-4e1e-90e0-1c75802d14cd ro. Quota mode: none.
> The system is going down NOW!
> Sent SIGTERM to all processes
> Sent SIGKILL to a[   68.412015] reboot: Power down


> 
> [1] https://github.com/jarkkojs/buildroot-tpmdd/blob/linux-6.6.y/patches/linux/0001-Revert-tpm-tpm_tis-Disable-interrupts-after-1000-unh.patch
> 
> BR, Jarkko