BenH reported that there is some assign unassigned resource problem
in powerpc.
It turns out after
| commit 0c5be0cb0edfe3b5c4b62eac68aa2aa15ec681af
| Date: Thu Feb 23 19:23:29 2012 -0800
|
| PCI: Retry on IORESOURCE_IO type allocations
even the root bus does not have io port range, it will keep retrying
to realloc with mmio.
After checking the code, found that we bound io port and mmio fail
path together.
First patch fix the problem, that will not make mmio fall back to must-only
when only have io port fail with must+optional.
During we found the fix for that problem, found that we can separate assign
unassigned resources to per root bus.
that will make the code simple, also could reuse it for hotadd path.
These patches are targeted to 3.11
-v4: split first patch into 4 patches per Bjorn.
-v5: drop two patches that will pass root bus resource mask after we found
simple and less intrusive way to fix the problem.
PCI: Don't let mmio fallback to must-only, if ioport fails with must+optional
PCI: Don't use temp bus for pci_bus_release_bridge_resources
PCI: Use pci_walk_bus to detect unassigned resources
PCI: Introduce enable_local to prepare per root bus handling
PCI: Split pci_assign_unassigned_resources to per root bus
PCI: Enable pci bridge when it is needed
PCI: Retry assign unassigned resources for hotadd root bus
Thanks
Yinghai
Let root bus hotadd path use same code for booting path.
As driver is not loaded yet, we could retry to make sure
all pci devices get resources allocated.
We need this as during hotadd, firmware could assign some bars before
handle over.
Signed-off-by: Yinghai Lu <[email protected]>
---
drivers/acpi/pci_root.c | 2 +-
drivers/pci/setup-bus.c | 15 +++++++--------
include/linux/pci.h | 1 +
3 files changed, 9 insertions(+), 9 deletions(-)
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -1365,7 +1365,7 @@ static void pci_bus_dump_resources(struc
}
}
-static int __init pci_bus_get_depth(struct pci_bus *bus)
+static int pci_bus_get_depth(struct pci_bus *bus)
{
int depth = 0;
struct pci_dev *dev;
@@ -1399,7 +1399,7 @@ enum enable_type {
auto_enabled,
};
-static enum enable_type pci_realloc_enable __initdata = undefined;
+static enum enable_type pci_realloc_enable = undefined;
void __init pci_realloc_get_opt(char *str)
{
if (!strncmp(str, "off", 3))
@@ -1407,13 +1407,13 @@ void __init pci_realloc_get_opt(char *st
else if (!strncmp(str, "on", 2))
pci_realloc_enable = user_enabled;
}
-static bool __init pci_realloc_enabled(enum enable_type enable)
+static bool pci_realloc_enabled(enum enable_type enable)
{
return enable >= user_enabled;
}
#if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
-static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
+static int check_unassigned_resources(struct pci_dev *dev, void *data)
{
int i;
int *unassigned = data;
@@ -1431,7 +1431,7 @@ static int __init check_unassigned_resou
return 0;
}
-static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+static enum enable_type pci_realloc_detect(struct pci_bus *bus,
enum enable_type enable_local)
{
int unassigned = 0;
@@ -1446,7 +1446,7 @@ static enum enable_type __init pci_reall
return enable_local;
}
#else
-static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+static enum enable_type pci_realloc_detect(struct pci_bus *bus,
enum enable_type enable_local)
{
return enable_local;
@@ -1458,8 +1458,7 @@ static enum enable_type __init pci_reall
* second and later try will clear small leaf bridge res
* will stop till to the max deepth if can not find good one
*/
-static void __init
-pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
+void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
{
LIST_HEAD(realloc_head); /* list of resources that
want additional resources */
Index: linux-2.6/drivers/acpi/pci_root.c
===================================================================
--- linux-2.6.orig/drivers/acpi/pci_root.c
+++ linux-2.6/drivers/acpi/pci_root.c
@@ -534,7 +534,7 @@ static int acpi_pci_root_add(struct acpi
if (system_state != SYSTEM_BOOTING) {
pcibios_resource_survey_bus(root->bus);
- pci_assign_unassigned_bus_resources(root->bus);
+ pci_assign_unassigned_root_bus_resources(root->bus);
}
pci_bus_add_devices(root->bus);
Index: linux-2.6/include/linux/pci.h
===================================================================
--- linux-2.6.orig/include/linux/pci.h
+++ linux-2.6/include/linux/pci.h
@@ -1003,6 +1003,7 @@ int pci_claim_resource(struct pci_dev *,
void pci_assign_unassigned_resources(void);
void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge);
void pci_assign_unassigned_bus_resources(struct pci_bus *bus);
+void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus);
void pdev_enable_device(struct pci_dev *);
int pci_enable_resources(struct pci_dev *, int mask);
void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *),
BenH reported that there is some assign unassigned resource problem
in powerpc.
It turns out after
| commit 0c5be0cb0edfe3b5c4b62eac68aa2aa15ec681af
| Date: Thu Feb 23 19:23:29 2012 -0800
|
| PCI: Retry on IORESOURCE_IO type allocations
even the root bus does not have io port range, it will keep retrying
to realloc with mmio.
Current retry logic is : try with must+optional at first, and if
it fails with any ioport or mmio, it will try must then try to extend
must with optional. The reassign will put extended mmio or pref mmio
in the middle of parent resource range.
That will prevent other sibling resources to get must-have resources
or get extended properly.
We can check fail type to see if can we need fall back to must-have
only, that will keep not needed release resource to be must+optional.
Separate three resource type checking if we need to release
assigned resource after requested + add_size try.
1. if there is io port assign fail, will release assigned io port.
2. if there is pref mmio assign fail, release assigned pref mmio.
if assigned pref mmio's parent is non-pref mmio and there
is non-pref mmio assign fail, will release that assigned pref mmio.
3. if there is non-pref mmio assign fail or pref mmio assigned fail,
will release assigned non-pref mmio.
This will be become more often when we have x86 8 sockets or 32 sockets
system, and those system will have one root bus per socket.
They will have some root buses do not have ioport range or 32bit mmio.
-v2: need to remove assigned entries from optional list too.
-v3: not just checking ioport related, requested by Bjorn.
Reported-by: Benjamin Herrenschmidt <[email protected]>
Tested-by: Gavin Shan <[email protected]>
Signed-off-by: Yinghai Lu <[email protected]>
---
drivers/pci/setup-bus.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 69 insertions(+), 1 deletion(-)
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -300,6 +300,48 @@ static void assign_requested_resources_s
}
}
+static unsigned long pci_fail_res_type_mask(struct list_head *fail_head)
+{
+ struct pci_dev_resource *fail_res;
+ unsigned long mask = 0;
+
+ /* check failed type */
+ list_for_each_entry(fail_res, fail_head, list)
+ mask |= fail_res->flags;
+
+ /*
+ * one pref failed resource will set IORESOURCE_MEM,
+ * as we can allocate pref in non-pref range.
+ * Will release all asssigned non-pref sibling resources
+ * according to that bit.
+ */
+ return mask & (IORESOURCE_IO | IORESOURCE_MEM | IORESOURCE_PREFETCH);
+}
+
+static bool pci_need_to_release(unsigned long mask, struct resource *res)
+{
+ if (res->flags & IORESOURCE_IO)
+ return !!(mask & IORESOURCE_IO);
+
+ /* check pref at first */
+ if (res->flags & IORESOURCE_PREFETCH) {
+ if (mask & IORESOURCE_PREFETCH)
+ return true;
+ /* count pref if its parent is non-pref */
+ else if ((mask & IORESOURCE_MEM) &&
+ !(res->parent->flags & IORESOURCE_PREFETCH))
+ return true;
+ else
+ return false;
+ }
+
+ if (res->flags & IORESOURCE_MEM)
+ return !!(mask & IORESOURCE_MEM);
+
+ /* should not get here */
+ return false;
+}
+
static void __assign_resources_sorted(struct list_head *head,
struct list_head *realloc_head,
struct list_head *fail_head)
@@ -312,11 +354,24 @@ static void __assign_resources_sorted(st
* if could do that, could get out early.
* if could not do that, we still try to assign requested at first,
* then try to reassign add_size for some resources.
+ *
+ * Separate three resource type checking if we need to release
+ * assigned resource after requested + add_size try.
+ * 1. if there is io port assign fail, will release assigned
+ * io port.
+ * 2. if there is pref mmio assign fail, release assigned
+ * pref mmio.
+ * if assigned pref mmio's parent is non-pref mmio and there
+ * is non-pref mmio assign fail, will release that assigned
+ * pref mmio.
+ * 3. if there is non-pref mmio assign fail or pref mmio
+ * assigned fail, will release assigned non-pref mmio.
*/
LIST_HEAD(save_head);
LIST_HEAD(local_fail_head);
struct pci_dev_resource *save_res;
- struct pci_dev_resource *dev_res;
+ struct pci_dev_resource *dev_res, *tmp_res;
+ unsigned long fail_type;
/* Check if optional add_size is there */
if (!realloc_head || list_empty(realloc_head))
@@ -348,6 +403,19 @@ static void __assign_resources_sorted(st
return;
}
+ /* check failed type */
+ fail_type = pci_fail_res_type_mask(&local_fail_head);
+ /* remove not need to be released assigned res from head list etc */
+ list_for_each_entry_safe(dev_res, tmp_res, head, list)
+ if (dev_res->res->parent &&
+ !pci_need_to_release(fail_type, dev_res->res)) {
+ /* remove it from realloc_head list */
+ remove_from_list(realloc_head, dev_res->res);
+ remove_from_list(&save_head, dev_res->res);
+ list_del(&dev_res->list);
+ kfree(dev_res);
+ }
+
free_list(&local_fail_head);
/* Release assigned resource */
list_for_each_entry(dev_res, head, list)
Current we enable bridges after bus scan and assign resources.
and it is spreaded a lot of places.
We can enable them later when their children pci device is enabled.
Need to go up to root bus and enable bridge one by one down to pci
device.
So that will delay enable bridge as needed bassis,
also kill one inconsistent between boot path and hot-add
path in acpi_pci_root_add().
Signed-off-by: Yinghai Lu <[email protected]>
---
arch/arm/kernel/bios32.c | 5 -----
arch/m68k/platform/coldfire/pci.c | 1 -
arch/mips/pci/pci.c | 1 -
arch/sh/drivers/pci/pci.c | 1 -
drivers/acpi/pci_root.c | 4 ----
drivers/parisc/lba_pci.c | 1 -
drivers/pci/bus.c | 19 -------------------
drivers/pci/hotplug/acpiphp_glue.c | 1 -
drivers/pci/pci.c | 20 ++++++++++++++++++++
drivers/pci/probe.c | 1 -
drivers/pci/setup-bus.c | 10 +++-------
drivers/pcmcia/cardbus.c | 1 -
include/linux/pci.h | 1 -
13 files changed, 23 insertions(+), 43 deletions(-)
Index: linux-2.6/arch/arm/kernel/bios32.c
===================================================================
--- linux-2.6.orig/arch/arm/kernel/bios32.c
+++ linux-2.6/arch/arm/kernel/bios32.c
@@ -524,11 +524,6 @@ void pci_common_init(struct hw_pci *hw)
* Assign resources.
*/
pci_bus_assign_resources(bus);
-
- /*
- * Enable bridges
- */
- pci_enable_bridges(bus);
}
/*
Index: linux-2.6/arch/m68k/platform/coldfire/pci.c
===================================================================
--- linux-2.6.orig/arch/m68k/platform/coldfire/pci.c
+++ linux-2.6/arch/m68k/platform/coldfire/pci.c
@@ -319,7 +319,6 @@ static int __init mcf_pci_init(void)
pci_fixup_irqs(pci_common_swizzle, mcf_pci_map_irq);
pci_bus_size_bridges(rootbus);
pci_bus_assign_resources(rootbus);
- pci_enable_bridges(rootbus);
return 0;
}
Index: linux-2.6/arch/mips/pci/pci.c
===================================================================
--- linux-2.6.orig/arch/mips/pci/pci.c
+++ linux-2.6/arch/mips/pci/pci.c
@@ -113,7 +113,6 @@ static void pcibios_scanbus(struct pci_c
if (!pci_has_flag(PCI_PROBE_ONLY)) {
pci_bus_size_bridges(bus);
pci_bus_assign_resources(bus);
- pci_enable_bridges(bus);
}
}
}
Index: linux-2.6/arch/sh/drivers/pci/pci.c
===================================================================
--- linux-2.6.orig/arch/sh/drivers/pci/pci.c
+++ linux-2.6/arch/sh/drivers/pci/pci.c
@@ -69,7 +69,6 @@ static void pcibios_scanbus(struct pci_c
pci_bus_size_bridges(bus);
pci_bus_assign_resources(bus);
- pci_enable_bridges(bus);
} else {
pci_free_resource_list(&resources);
}
Index: linux-2.6/drivers/parisc/lba_pci.c
===================================================================
--- linux-2.6.orig/drivers/parisc/lba_pci.c
+++ linux-2.6/drivers/parisc/lba_pci.c
@@ -1533,7 +1533,6 @@ lba_driver_probe(struct parisc_device *d
lba_dump_res(&lba_dev->hba.lmmio_space, 2);
#endif
}
- pci_enable_bridges(lba_bus);
/*
** Once PCI register ops has walked the bus, access to config
Index: linux-2.6/drivers/pci/bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/bus.c
+++ linux-2.6/drivers/pci/bus.c
@@ -216,24 +216,6 @@ void pci_bus_add_devices(const struct pc
}
}
-void pci_enable_bridges(struct pci_bus *bus)
-{
- struct pci_dev *dev;
- int retval;
-
- list_for_each_entry(dev, &bus->devices, bus_list) {
- if (dev->subordinate) {
- if (!pci_is_enabled(dev)) {
- retval = pci_enable_device(dev);
- if (retval)
- dev_err(&dev->dev, "Error enabling bridge (%d), continuing\n", retval);
- pci_set_master(dev);
- }
- pci_enable_bridges(dev->subordinate);
- }
- }
-}
-
/** pci_walk_bus - walk devices on/under bus, calling callback.
* @top bus whose devices should be walked
* @cb callback to be called for each device found
@@ -301,4 +283,3 @@ EXPORT_SYMBOL(pci_bus_put);
EXPORT_SYMBOL(pci_bus_alloc_resource);
EXPORT_SYMBOL_GPL(pci_bus_add_device);
EXPORT_SYMBOL(pci_bus_add_devices);
-EXPORT_SYMBOL(pci_enable_bridges);
Index: linux-2.6/drivers/pci/hotplug/acpiphp_glue.c
===================================================================
--- linux-2.6.orig/drivers/pci/hotplug/acpiphp_glue.c
+++ linux-2.6/drivers/pci/hotplug/acpiphp_glue.c
@@ -704,7 +704,6 @@ static int __ref enable_device(struct ac
acpiphp_sanitize_bus(bus);
acpiphp_set_hpp_values(bus);
acpiphp_set_acpi_region(slot);
- pci_enable_bridges(bus);
list_for_each_entry(dev, &bus->devices, bus_list) {
/* Assume that newly added devices are powered on already. */
Index: linux-2.6/drivers/pci/pci.c
===================================================================
--- linux-2.6.orig/drivers/pci/pci.c
+++ linux-2.6/drivers/pci/pci.c
@@ -1145,6 +1145,24 @@ int pci_reenable_device(struct pci_dev *
return 0;
}
+static void pci_enable_bridge(struct pci_dev *dev)
+{
+ int retval;
+
+ if (!dev)
+ return;
+
+ pci_enable_bridge(dev->bus->self);
+
+ if (pci_is_enabled(dev))
+ return;
+ retval = pci_enable_device(dev);
+ if (retval)
+ dev_err(&dev->dev, "Error enabling bridge (%d), continuing\n",
+ retval);
+ pci_set_master(dev);
+}
+
static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags)
{
int err;
@@ -1165,6 +1183,8 @@ static int pci_enable_device_flags(struc
if (atomic_inc_return(&dev->enable_cnt) > 1)
return 0; /* already enabled */
+ pci_enable_bridge(dev->bus->self);
+
/* only skip sriov related */
for (i = 0; i <= PCI_ROM_RESOURCE; i++)
if (dev->resource[i].flags & flags)
Index: linux-2.6/drivers/pci/probe.c
===================================================================
--- linux-2.6.orig/drivers/pci/probe.c
+++ linux-2.6/drivers/pci/probe.c
@@ -1964,7 +1964,6 @@ unsigned int __ref pci_rescan_bus(struct
max = pci_scan_child_bus(bus);
pci_assign_unassigned_bus_resources(bus);
- pci_enable_bridges(bus);
pci_bus_add_devices(bus);
return max;
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -1503,7 +1503,7 @@ again:
/* any device complain? */
if (list_empty(&fail_head))
- goto enable_and_dump;
+ goto dump;
if (tried_times >= pci_try_num) {
if (enable_local == undefined)
@@ -1512,7 +1512,7 @@ again:
dev_info(&bus->dev, "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
free_list(&fail_head);
- goto enable_and_dump;
+ goto dump;
}
dev_printk(KERN_DEBUG, &bus->dev,
@@ -1545,10 +1545,7 @@ again:
goto again;
-enable_and_dump:
- /* Depth last, update the hardware. */
- pci_enable_bridges(bus);
-
+dump:
/* dump the resource on buses */
pci_bus_dump_resources(bus);
}
@@ -1621,7 +1618,6 @@ enable_all:
if (retval)
dev_err(&bridge->dev, "Error reenabling bridge (%d)\n", retval);
pci_set_master(bridge);
- pci_enable_bridges(parent);
}
EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
Index: linux-2.6/drivers/pcmcia/cardbus.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/cardbus.c
+++ linux-2.6/drivers/pcmcia/cardbus.c
@@ -91,7 +91,6 @@ int __ref cb_alloc(struct pcmcia_socket
if (s->tune_bridge)
s->tune_bridge(s, bus);
- pci_enable_bridges(bus);
pci_bus_add_devices(bus);
return 0;
Index: linux-2.6/include/linux/pci.h
===================================================================
--- linux-2.6.orig/include/linux/pci.h
+++ linux-2.6/include/linux/pci.h
@@ -1043,7 +1043,6 @@ int __must_check pci_bus_alloc_resource(
resource_size_t,
resource_size_t),
void *alignf_data);
-void pci_enable_bridges(struct pci_bus *bus);
/* Proper probing supporting hot-pluggable devices */
int __must_check __pci_register_driver(struct pci_driver *, struct module *,
Index: linux-2.6/drivers/acpi/pci_root.c
===================================================================
--- linux-2.6.orig/drivers/acpi/pci_root.c
+++ linux-2.6/drivers/acpi/pci_root.c
@@ -537,10 +537,6 @@ static int acpi_pci_root_add(struct acpi
pci_assign_unassigned_bus_resources(root->bus);
}
- /* need to after hot-added ioapic is registered */
- if (system_state != SYSTEM_BOOTING)
- pci_enable_bridges(root->bus);
-
pci_bus_add_devices(root->bus);
return 1;
Per Bjorn, use pci_walk_bus instead of for_each_pci_dev or
calling pci_realloc_detect() recursively, that will make code more readable.
Per Bjorn, separate it from big patch that handing assign_unssigned per root bus.
Signed-off-by: Yinghai Lu <[email protected]>
---
drivers/pci/setup-bus.c | 46 +++++++++++++++++++++++++++++++---------------
1 file changed, 31 insertions(+), 15 deletions(-)
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -1427,30 +1427,46 @@ static bool __init pci_realloc_enabled(v
return pci_realloc_enable >= user_enabled;
}
-static void __init pci_realloc_detect(void)
-{
#if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
- struct pci_dev *dev = NULL;
+static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
+{
+ int i;
+ int *unassigned = data;
- if (pci_realloc_enable != undefined)
- return;
+ for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
+ struct resource *r = &dev->resource[i];
- for_each_pci_dev(dev) {
- int i;
+ /* Not assigned, or rejected by kernel ? */
+ if (r->flags && !r->start) {
+ (*unassigned)++;
+ return 1; /* return early from pci_walk_bus */
+ }
+ }
- for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
- struct resource *r = &dev->resource[i];
+ return 0;
+}
- /* Not assigned, or rejected by kernel ? */
- if (r->flags && !r->start) {
- pci_realloc_enable = auto_enabled;
+static void __init pci_realloc_detect(void)
+{
+ int unassigned = 0;
+ struct pci_bus *bus;
- return;
- }
+ if (pci_realloc_enable != undefined)
+ return;
+
+ list_for_each_entry(bus, &pci_root_buses, node) {
+ pci_walk_bus(bus, check_unassigned_resources, &unassigned);
+ if (unassigned) {
+ pci_realloc_enable = auto_enabled;
+ return;
}
}
-#endif
}
+#else
+static void __init pci_realloc_detect(void)
+{
+}
+#endif
/*
* first try will not touch pci bridge res
Add enable_local to prepare assign unassigned resource
for per root bus.
Per Bjorn, separate it from big patch that handing assign_unssigned per root bus.
Signed-off-by: Yinghai Lu <[email protected]>
---
drivers/pci/setup-bus.c | 40 +++++++++++++++++++++-------------------
1 file changed, 21 insertions(+), 19 deletions(-)
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -1422,9 +1422,9 @@ void __init pci_realloc_get_opt(char *st
else if (!strncmp(str, "on", 2))
pci_realloc_enable = user_enabled;
}
-static bool __init pci_realloc_enabled(void)
+static bool __init pci_realloc_enabled(enum enable_type enable)
{
- return pci_realloc_enable >= user_enabled;
+ return enable >= user_enabled;
}
#if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
@@ -1446,25 +1446,25 @@ static int __init check_unassigned_resou
return 0;
}
-static void __init pci_realloc_detect(void)
+static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+ enum enable_type enable_local)
{
int unassigned = 0;
- struct pci_bus *bus;
- if (pci_realloc_enable != undefined)
- return;
+ if (enable_local != undefined)
+ return enable_local;
- list_for_each_entry(bus, &pci_root_buses, node) {
- pci_walk_bus(bus, check_unassigned_resources, &unassigned);
- if (unassigned) {
- pci_realloc_enable = auto_enabled;
- return;
- }
- }
+ pci_walk_bus(bus, check_unassigned_resources, &unassigned);
+ if (unassigned)
+ return auto_enabled;
+
+ return enable_local;
}
#else
-static void __init pci_realloc_detect(void)
+static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+ enum enable_type enable_local)
{
+ return enable_local;
}
#endif
@@ -1487,10 +1487,12 @@ pci_assign_unassigned_resources(void)
unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
IORESOURCE_PREFETCH;
int pci_try_num = 1;
+ enum enable_type enable_local = pci_realloc_enable;
+
+ list_for_each_entry(bus, &pci_root_buses, node)
+ enable_local = pci_realloc_detect(bus, enable_local);
- /* don't realloc if asked to do so */
- pci_realloc_detect();
- if (pci_realloc_enabled()) {
+ if (pci_realloc_enabled(enable_local)) {
int max_depth = pci_get_max_depth();
pci_try_num = max_depth + 1;
@@ -1522,9 +1524,9 @@ again:
goto enable_and_dump;
if (tried_times >= pci_try_num) {
- if (pci_realloc_enable == undefined)
+ if (enable_local == undefined)
printk(KERN_INFO "Some PCI device resources are unassigned, try booting with pci=realloc\n");
- else if (pci_realloc_enable == auto_enabled)
+ else if (enable_local == auto_enabled)
printk(KERN_INFO "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
free_list(&fail_head);
We need to split pci_assign_unassiged_resource to every root bus, so can
have different retry for assign_unassigned per root bus
Also we need root bus hot add and booting path use same code.
-v2: separate enable_local and pci_release_bridge_resources to
other patches requested by Bjorn.
Signed-off-by: Yinghai Lu <[email protected]>
---
drivers/pci/setup-bus.c | 62 +++++++++++++++++++-----------------------------
1 file changed, 25 insertions(+), 37 deletions(-)
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -1383,21 +1383,6 @@ static int __init pci_bus_get_depth(stru
return depth;
}
-static int __init pci_get_max_depth(void)
-{
- int depth = 0;
- struct pci_bus *bus;
-
- list_for_each_entry(bus, &pci_root_buses, node) {
- int ret;
-
- ret = pci_bus_get_depth(bus);
- if (ret > depth)
- depth = ret;
- }
-
- return depth;
-}
/*
* -1: undefined, will auto detect later
@@ -1473,10 +1458,9 @@ static enum enable_type __init pci_reall
* second and later try will clear small leaf bridge res
* will stop till to the max deepth if can not find good one
*/
-void __init
-pci_assign_unassigned_resources(void)
+static void __init
+pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
{
- struct pci_bus *bus;
LIST_HEAD(realloc_head); /* list of resources that
want additional resources */
struct list_head *add_list = NULL;
@@ -1487,17 +1471,17 @@ pci_assign_unassigned_resources(void)
unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
IORESOURCE_PREFETCH;
int pci_try_num = 1;
- enum enable_type enable_local = pci_realloc_enable;
-
- list_for_each_entry(bus, &pci_root_buses, node)
- enable_local = pci_realloc_detect(bus, enable_local);
+ enum enable_type enable_local;
+ /* don't realloc if asked to do so */
+ enable_local = pci_realloc_detect(bus, pci_realloc_enable);
if (pci_realloc_enabled(enable_local)) {
- int max_depth = pci_get_max_depth();
+ int max_depth = pci_bus_get_depth(bus);
pci_try_num = max_depth + 1;
- printk(KERN_DEBUG "PCI: max bus depth: %d pci_try_num: %d\n",
- max_depth, pci_try_num);
+ dev_printk(KERN_DEBUG, &bus->dev,
+ "max bus depth: %d pci_try_num: %d\n",
+ max_depth, pci_try_num);
}
again:
@@ -1509,12 +1493,10 @@ again:
add_list = &realloc_head;
/* Depth first, calculate sizes and alignments of all
subordinate buses. */
- list_for_each_entry(bus, &pci_root_buses, node)
- __pci_bus_size_bridges(bus, add_list);
+ __pci_bus_size_bridges(bus, add_list);
/* Depth last, allocate resources and update the hardware. */
- list_for_each_entry(bus, &pci_root_buses, node)
- __pci_bus_assign_resources(bus, add_list, &fail_head);
+ __pci_bus_assign_resources(bus, add_list, &fail_head);
if (add_list)
BUG_ON(!list_empty(add_list));
tried_times++;
@@ -1525,16 +1507,16 @@ again:
if (tried_times >= pci_try_num) {
if (enable_local == undefined)
- printk(KERN_INFO "Some PCI device resources are unassigned, try booting with pci=realloc\n");
+ dev_info(&bus->dev, "Some PCI device resources are unassigned, try booting with pci=realloc\n");
else if (enable_local == auto_enabled)
- printk(KERN_INFO "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
+ dev_info(&bus->dev, "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
free_list(&fail_head);
goto enable_and_dump;
}
- printk(KERN_DEBUG "PCI: No. %d try to assign unassigned res\n",
- tried_times + 1);
+ dev_printk(KERN_DEBUG, &bus->dev,
+ "No. %d try to assign unassigned res\n", tried_times + 1);
/* third times and later will not check if it is leaf */
if ((tried_times + 1) > 2)
@@ -1565,12 +1547,18 @@ again:
enable_and_dump:
/* Depth last, update the hardware. */
- list_for_each_entry(bus, &pci_root_buses, node)
- pci_enable_bridges(bus);
+ pci_enable_bridges(bus);
/* dump the resource on buses */
- list_for_each_entry(bus, &pci_root_buses, node)
- pci_bus_dump_resources(bus);
+ pci_bus_dump_resources(bus);
+}
+
+void __init pci_assign_unassigned_resources(void)
+{
+ struct pci_bus *root_bus;
+
+ list_for_each_entry(root_bus, &pci_root_buses, node)
+ pci_assign_unassigned_root_bus_resources(root_bus);
}
void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
as later bus can not be used as temp variable after we change to
per root bus handling with assign unassigned resources.
Per Bjorn, separate it from big patch that handing assign_unssigned per root bus.
Signed-off-by: Yinghai Lu <[email protected]>
---
drivers/pci/setup-bus.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -1526,12 +1526,11 @@ again:
* Try to release leaf bridge's resources that doesn't fit resource of
* child device under that bridge
*/
- list_for_each_entry(fail_res, &fail_head, list) {
- bus = fail_res->dev->bus;
- pci_bus_release_bridge_resources(bus,
+ list_for_each_entry(fail_res, &fail_head, list)
+ pci_bus_release_bridge_resources(fail_res->dev->bus,
fail_res->flags & type_mask,
rel_type);
- }
+
/* restore size and flags */
list_for_each_entry(fail_res, &fail_head, list) {
struct resource *res = fail_res->res;
On Fri, May 31, 2013 at 11:03 PM, Yinghai Lu <[email protected]> wrote:
> BenH reported that there is some assign unassigned resource problem
> in powerpc.
>
> It turns out after
> | commit 0c5be0cb0edfe3b5c4b62eac68aa2aa15ec681af
> | Date: Thu Feb 23 19:23:29 2012 -0800
> |
> | PCI: Retry on IORESOURCE_IO type allocations
>
> even the root bus does not have io port range, it will keep retrying
> to realloc with mmio.
>
> After checking the code, found that we bound io port and mmio fail
> path together.
> First patch fix the problem, that will not make mmio fall back to must-only
> when only have io port fail with must+optional.
>
> During we found the fix for that problem, found that we can separate assign
> unassigned resources to per root bus.
> that will make the code simple, also could reuse it for hotadd path.
>
> These patches are targeted to 3.11
>
> -v4: split first patch into 4 patches per Bjorn.
> -v5: drop two patches that will pass root bus resource mask after we found
> simple and less intrusive way to fix the problem.
>
> PCI: Don't let mmio fallback to must-only, if ioport fails with must+optional
> PCI: Don't use temp bus for pci_bus_release_bridge_resources
> PCI: Use pci_walk_bus to detect unassigned resources
> PCI: Introduce enable_local to prepare per root bus handling
> PCI: Split pci_assign_unassigned_resources to per root bus
> PCI: Enable pci bridge when it is needed
> PCI: Retry assign unassigned resources for hotadd root bus
Hi, Bjorn,
Can you put this patchset in pci/next for 3.11?
Found another pciehp will need this one two. the pcie bridge does not
have io port range and it cause mmio get clear and retry.
Thanks
Yinghai
On Fri, May 31, 2013 at 11:03:08PM -0700, Yinghai Lu wrote:
> Per Bjorn, use pci_walk_bus instead of for_each_pci_dev or
> calling pci_realloc_detect() recursively, that will make code more readable.
>
> Per Bjorn, separate it from big patch that handing assign_unssigned per root bus.
>
> Signed-off-by: Yinghai Lu <[email protected]>
>
> ---
> drivers/pci/setup-bus.c | 46 +++++++++++++++++++++++++++++++---------------
> 1 file changed, 31 insertions(+), 15 deletions(-)
>
> Index: linux-2.6/drivers/pci/setup-bus.c
> ===================================================================
> --- linux-2.6.orig/drivers/pci/setup-bus.c
> +++ linux-2.6/drivers/pci/setup-bus.c
> @@ -1427,30 +1427,46 @@ static bool __init pci_realloc_enabled(v
> return pci_realloc_enable >= user_enabled;
> }
>
> -static void __init pci_realloc_detect(void)
> -{
> #if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
> - struct pci_dev *dev = NULL;
> +static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
I'm not going to add a function named "check_*()" because the name gives no
clue about what the return value means. If it's a boolean function, the
name should be something like a question that has a yes/no answer.
> +{
> + int i;
> + int *unassigned = data;
>
> - if (pci_realloc_enable != undefined)
> - return;
> + for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
> + struct resource *r = &dev->resource[i];
>
> - for_each_pci_dev(dev) {
> - int i;
> + /* Not assigned, or rejected by kernel ? */
> + if (r->flags && !r->start) {
> + (*unassigned)++;
> + return 1; /* return early from pci_walk_bus */
> + }
> + }
>
> - for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
> - struct resource *r = &dev->resource[i];
> + return 0;
> +}
>
> - /* Not assigned, or rejected by kernel ? */
> - if (r->flags && !r->start) {
> - pci_realloc_enable = auto_enabled;
> +static void __init pci_realloc_detect(void)
> +{
> + int unassigned = 0;
> + struct pci_bus *bus;
>
> - return;
> - }
> + if (pci_realloc_enable != undefined)
> + return;
> +
> + list_for_each_entry(bus, &pci_root_buses, node) {
> + pci_walk_bus(bus, check_unassigned_resources, &unassigned);
> + if (unassigned) {
> + pci_realloc_enable = auto_enabled;
> + return;
> }
> }
> -#endif
> }
> +#else
> +static void __init pci_realloc_detect(void)
> +{
> +}
> +#endif
>
> /*
> * first try will not touch pci bridge res
On Tue, 2013-06-25 at 15:15 -0600, Bjorn Helgaas wrote:
> - for_each_pci_dev(dev) {
> > - int i;
> > + /* Not assigned, or rejected by kernel ? */
> > + if (r->flags && !r->start) {
> > + (*unassigned)++;
> > + return 1; /* return early from pci_walk_bus */
> > + }
> > + }
BTW. I'm aware you didn't change that logic but ... it's somewhat broken
in the case where the aperture has an offset. You should compare
r->start with the offset, not with 0.
Cheers,
Ben.
On Tue, Jun 25, 2013 at 3:38 PM, Benjamin Herrenschmidt
<[email protected]> wrote:
> On Tue, 2013-06-25 at 15:15 -0600, Bjorn Helgaas wrote:
>> - for_each_pci_dev(dev) {
>> > - int i;
>> > + /* Not assigned, or rejected by kernel ? */
>> > + if (r->flags && !r->start) {
>> > + (*unassigned)++;
>> > + return 1; /* return early from pci_walk_bus */
>> > + }
>> > + }
>
> BTW. I'm aware you didn't change that logic but ... it's somewhat broken
> in the case where the aperture has an offset. You should compare
> r->start with the offset, not with 0.
Yes, please fix that in a separate patch that contains only the bugfix.
On Tue, Jun 25, 2013 at 2:15 PM, Bjorn Helgaas <[email protected]> wrote:
>> +static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
>
> I'm not going to add a function named "check_*()" because the name gives no
> clue about what the return value means. If it's a boolean function, the
> name should be something like a question that has a yes/no answer.
that prototype return int is required by pci_walk_bus().
drivers/pci/bus.c:void pci_walk_bus(struct pci_bus *top, int
(*cb)(struct pci_dev *, void *)
return 1, will return early from pci_walk_bus().
count_unassigned_resources() is not good name too, as we bail out early.
find_unassigned_resources() is more weird, looks like it want to return resource
Yinghai
On Tue, Jun 25, 2013 at 2:46 PM, Bjorn Helgaas <[email protected]> wrote:
> On Tue, Jun 25, 2013 at 3:38 PM, Benjamin Herrenschmidt
> <[email protected]> wrote:
>> On Tue, 2013-06-25 at 15:15 -0600, Bjorn Helgaas wrote:
>>> - for_each_pci_dev(dev) {
>>> > - int i;
>>> > + /* Not assigned, or rejected by kernel ? */
>>> > + if (r->flags && !r->start) {
>>> > + (*unassigned)++;
>>> > + return 1; /* return early from pci_walk_bus */
>>> > + }
>>> > + }
>>
>> BTW. I'm aware you didn't change that logic but ... it's somewhat broken
>> in the case where the aperture has an offset. You should compare
>> r->start with the offset, not with 0.
>
> Yes, please fix that in a separate patch that contains only the bugfix.
Please check inline word warped patch.
Subject: [PATCH] PCI: check pci bus address for unassigned res
We should compare res->start with root bus window offset.
Otherwise will have problem with arch that support hostbridge
resource offset.
BenH pointed out that during reviewing patchset that separate
assign unassigned to per root buses.
According to Bjorn, have it in separated patch.
Use pcibios_resource_to_bus to get region at first, and check
region.start instead.
Suggested-by: Benjamin Herrenschmidt <[email protected]>
Signed-off-by: Yinghai Lu <[email protected]>
---
drivers/pci/setup-bus.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
Index: linux-2.6/drivers/pci/setup-bus.c
===================================================================
--- linux-2.6.orig/drivers/pci/setup-bus.c
+++ linux-2.6/drivers/pci/setup-bus.c
@@ -1420,9 +1420,14 @@ static int check_unassigned_resources(st
for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
struct resource *r = &dev->resource[i];
+ struct pci_bus_region region;
/* Not assigned, or rejected by kernel ? */
- if (r->flags && !r->start) {
+ if (!r->flags)
+ continue;
+
+ pcibios_resource_to_bus(dev, ®ion, res);
+ if (!region.start) {
(*unassigned)++;
return 1; /* return early from pci_walk_bus */
}