by Andy Shevchenko

[permalink] [raw]

Subject: Re: [PATCH 2/4] PCI: add functionality for resizing resources v3

On Tue, Apr 25, 2017 at 4:19 PM, Christian König
<[email protected]> wrote:
> From: Christian König <[email protected]>
>
> This allows device drivers to request resizing their BARs.
>
> The function only tries to reprogram the windows of the bridge directly above
> the requesting device and only the BAR of the same type (usually mem, 64bit,
> prefetchable). This is done to make sure not to disturb other drivers by
> changing the BARs of their devices.
>
> If reprogramming the bridge BAR fails the old status is restored and -ENOSPC
> returned to the calling device driver.

> +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
> +{
> + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
> + IORESOURCE_PREFETCH | IORESOURCE_MEM_64;

> +

Redundant.

> + struct pci_dev_resource *dev_res;
> + LIST_HEAD(saved);
> + LIST_HEAD(added);
> + LIST_HEAD(failed);

> + unsigned i;

unsigned int i;

> + int ret = 0;
> +
> + /* Walk to the root BUS, releasing bridge BARs when possible */

> + while (1) {

This raises red flag. Care to refactor?
Also do {} while () syntax allows faster to get that the loop body
goes at least once.

> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END;
> + i++) {

> + struct resource *res = &bridge->resource[i];
> +

> + if ((res->flags & type_mask) != (type & type_mask))

I would rather go with

if ((res->flags ^ type) & type_mask)

> + continue;
> +
> + /* Ignore BARs which are still in use */
> + if (res->child)
> + continue;
> +
> + ret = add_to_list(&saved, bridge, res, 0, 0);
> + if (ret)
> + goto cleanup;
> +
> + if (res->parent)
> + release_resource(res);

> + res->start = 0;
> + res->end = 0;

> + dev_info(&bridge->dev, "BAR %d: released %pR\n",
> + i, res);

I doesn't make much sense to me after zeroing a range.

> + break;
> + }
> + if (i == PCI_BRIDGE_RESOURCE_END)
> + break;
> +
> + if (!bridge->bus || !bridge->bus->self)
> + break;
> +
> + bridge = bridge->bus->self;
> + }
> +
> + if (list_empty(&saved))
> + return -ENOENT;
> +
> + __pci_bus_size_bridges(bridge->subordinate, &added);
> + __pci_bridge_assign_resources(bridge, &added, &failed);
> + BUG_ON(!list_empty(&added));
> +
> + if (!list_empty(&failed)) {
> + ret = -ENOSPC;
> + goto cleanup;
> + }

> +
> +

Remove extra empty line.

> + list_for_each_entry(dev_res, &saved, list) {
> + /* Skip the bridge we just assigned resources for. */
> + if (bridge == dev_res->dev)
> + continue;
> +
> + bridge = dev_res->dev;
> + pci_setup_bridge(bridge->subordinate);
> + }
> +

> + free_list(&saved);
> + free_list(&failed);
> + return ret;

You might re-use two lines with below, but perhaps better to show
which case returns 0 explicitly and drop assignment ret = 0 above.

> +
> +cleanup:
> + /* restore size and flags */
> + list_for_each_entry(dev_res, &failed, list) {
> + struct resource *res = dev_res->res;
> +
> + res->start = dev_res->start;
> + res->end = dev_res->end;
> + res->flags = dev_res->flags;
> + }
> + free_list(&failed);
> +
> + /* Revert to the old configuration */
> + list_for_each_entry(dev_res, &saved, list) {
> + struct resource *res = dev_res->res;
> +
> + bridge = dev_res->dev;
> + i = res - bridge->resource;
> +
> + res->start = dev_res->start;
> + res->end = dev_res->end;
> + res->flags = dev_res->flags;
> +
> + pci_claim_resource(bridge, i);
> + pci_setup_bridge(bridge->subordinate);
> + }

> + free_list(&saved);
> +
> + return ret;

> +}

> +void pci_release_resource(struct pci_dev *dev, int resno)
> +{
> + struct resource *res = dev->resource + resno;
> +
> + release_resource(res);

> + res->end = resource_size(res) - 1;
> + res->start = 0;
> + res->flags |= IORESOURCE_UNSET;

> + dev_info(&dev->dev, "BAR %d: released %pR\n", resno, res);

Makes little sense to me after you cleared information out.

> +}
> +EXPORT_SYMBOL(pci_release_resource);
> +
> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
> +{
> + struct resource *res = dev->resource + resno;

> + u64 bytes = 1ULL << (size + 20);

> + res->end = res->start + (1ULL << (old + 20)) - 1;

Perhaps macro or helper?

static inline u64 rbar_size_to_bytes(size)
{
return 1ULL << (size + 20);
}

--
With Best Regards,
Andy Shevchenko

2017-04-26 17:18:47

by Andy Shevchenko

[permalink] [raw]

Subject: Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors v2

On Tue, Apr 25, 2017 at 4:19 PM, Christian König
<[email protected]> wrote:
> From: Christian König <[email protected]>
>
> Most BIOS don't enable this because of compatibility reasons.
>
> Manually enable a 64bit BAR of 64GB size so that we have
> enough room for PCI devices.

> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
> +{

> + uint32_t base, limit, high;

Is u32 or uint32_t used actually in this file?

> + struct resource *res, *conflict;

> + unsigned i;

unsigned int i;

Reversed tree order.

> + for (i = 0; i < 8; ++i) {
> + pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
> + pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
> +
> + /* Is this slot free? */
> + if ((base & 0x3) == 0x0)
> + break;
> +
> + base >>= 8;
> + base |= high << 24;
> +
> + /* Abort if a slot already configures a 64bit BAR. */
> + if (base > 0x10000)
> + return;
> + }
> + if (i == 8)
> + return;
> +
> + res = kzalloc(sizeof(*res), GFP_KERNEL);
> + if (!res)
> + return;
> +
> + res->name = "PCI Bus 0000:00";

> + res->flags = IORESOURCE_PREFETCH | IORESOURCE_MEM |
> + IORESOURCE_MEM_64 | IORESOURCE_WINDOW;

You are using this constant twice AFAICS, perhaps define it in generic
pci.h and re-use?

> + res->start = 0x100000000;
> + res->end = 0xfd00000000 - 1;

Perhaps you forgot ul / ull in many places in your code. Care to
compile for 32-bit and fix the warnings?

> + dev_info(&dev->dev, "adding root bus resource %pR\n", res);

Some of your messages I think are rather dev_dbg() than dev_info().

> + base = ((res->start >> 8) & 0xffffff00) | 0x3;
> + limit = ((res->end + 1) >> 8) & 0xffffff00;
> + high = ((res->start >> 40) & 0xff) |
> + ((((res->end + 1) >> 40) & 0xff) << 16);

It would be nice to have the magics defined (just on top of this function) like

#define PCI_AMD_BAR_XXX_MASK GENMASK(...)
#define PCI_AMD_BAR_XXX_SHIFT nn

> + pci_write_config_dword(dev, 0x180 + i * 0x4, high);
> + pci_write_config_dword(dev, 0x84 + i * 0x8, limit);
> + pci_write_config_dword(dev, 0x80 + i * 0x8, base);

Ditto for pos:s.

> +
> + pci_bus_add_resource(dev->bus, res, 0);
> +}

--
With Best Regards,
Andy Shevchenko