Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752404AbdGDNJR (ORCPT ); Tue, 4 Jul 2017 09:09:17 -0400 Received: from smtp2-g21.free.fr ([212.27.42.2]:13637 "EHLO smtp2-g21.free.fr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751875AbdGDNJP (ORCPT ); Tue, 4 Jul 2017 09:09:15 -0400 Subject: Re: [PATCH v9 2/3] PCI: Add tango PCIe host bridge support To: Peter Zijlstra Cc: Marc Gonzalez , Bjorn Helgaas , Marc Zyngier , Thomas Gleixner , linux-pci , Linux ARM , LKML , Thibaud Cornic , Mark Rutland , Ard Biesheuvel , Greg Kroah-Hartman , Ingo Molnar References: <987fac41-80dc-f1d0-ec0b-91ae57b91bfd@sigmadesigns.com> <20170702231811.GJ18324@bhelgaas-glaptop.roam.corp.google.com> <79382219-c730-da78-3e5f-5abf3173d7ac@sigmadesigns.com>

<20170704070958.oolhapyx7uy2fly4@hirez.programming.kicks-ass.net> From: Mason Message-ID: Date: Tue, 4 Jul 2017 15:08:43 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.1 MIME-Version: 1.0 In-Reply-To: <20170704070958.oolhapyx7uy2fly4@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2308 Lines: 74 On 04/07/2017 09:09, Peter Zijlstra wrote: > On Mon, Jul 03, 2017 at 05:30:28PM +0200, Marc Gonzalez wrote: > >> And at the end of smp8759_config_read: >> >> printk("in_atomic_preempt_off = %d\n", in_atomic_preempt_off()); > > That's confused... That much is certain. I am indeed grasping at straws. I grepped "scheduling while atomic", found __schedule_bug() in kernel/sched/core.c and saw if (IS_ENABLED(CONFIG_DEBUG_PREEMPT) && in_atomic_preempt_off()) { pr_err("Preemption disabled at:"); print_ip_sym(preempt_disable_ip); pr_cont("\n"); } I thought printing the value of in_atomic_preempt_off() in the callback would indicate whether preemption had already been turned off at that point. It doesn't work like that? BTW, why didn't print_ip_sym(preempt_disable_ip); say where preemption had been disabled? >> [ 1.026568] BUG: scheduling while atomic: swapper/0/1/0x00000002 >> [ 1.032625] 5 locks held by swapper/0/1: >> [ 1.036575] #0: (&dev->mutex){......}, at: [] __driver_attach+0x50/0xd0 >> [ 1.044319] #1: (&dev->mutex){......}, at: [] __driver_attach+0x60/0xd0 >> [ 1.052050] #2: (pci_lock){+.+...}, at: [] pci_bus_read_config_dword+0x44/0x94 > > This is a raw_spinlock_t, that disables preemption drivers/pci/access.c /* * This interrupt-safe spinlock protects all accesses to PCI * configuration space. */ DEFINE_RAW_SPINLOCK(pci_lock); raw_spin_lock_irqsave(&pci_lock, flags); res = bus->ops->read(bus, devfn, pos, len, &data); IIUC, it's not possible to call stop_machine() while holding a raw spinlock? What about regular spinlocks? IIUC, in RT, regular spinlocks may sleep? I didn't find "preempt" or "schedul" in the spinlock doc. https://www.kernel.org/doc/Documentation/locking/spinlocks.txt > Using stop_machine() is per definition doing it wrong ;-) Here's the high-level view. My HW is borked and muxes config space and mem space. So I need a way to freeze the entire system, make the config space access, and then return the system to normal. (AFAICT, config space accesses are rare, so if I kill performance for these accesses, the system might remain usable.) Is there a way to do this? Mark suggested stop_machine but it seems using it in my situation is not quite straight-forward. Regards.