Message-ID: <55A53FAC.30103@caviumnetworks.com>
Date: Tue, 14 Jul 2015 09:58:20 -0700
From: David Daney <ddaney@caviumnetworks.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130625 Thunderbird/17.0.7
MIME-Version: 1.0
To: Will Deacon <will.deacon@arm.com>
CC: David Daney <ddaney.cavm@gmail.com>,
        "linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
        Catalin Marinas <Catalin.Marinas@arm.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Robert Richter <rrichter@cavium.com>,
        "David Daney" <david.daney@cavium.com>
Subject: Re: [PATCH] arm64: Define HAVE_ARCH_PIO_SIZE and related symbols.
References: <1436823096-24059-1-git-send-email-ddaney.cavm@gmail.com> <20150714110039.GC16213@arm.com> <55A53509.4060202@caviumnetworks.com> <20150714162955.GR16213@arm.com>
In-Reply-To: <20150714162955.GR16213@arm.com>
Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
Content-Transfer-Encoding: 7bit
BLUPR0701MB1716: X-MS-Exchange-Organization-RulesExecuted
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jul 2015 16:58:24.3974 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB1716
X-Microsoft-Exchange-Diagnostics: 1;BLUPR0701MB1636;2:tp2vqYJvjUR9Isva3hREzwq0b/lsfg/dzSSXOuH0S0GoPK2/am7Z8iPTlcf/jfSQ;3:oqNRnSzw7FsvGbKqDHJwaeJr6heHEg0OVDcAju4gwU8Gff7nz62U6hf8NcsYNPNLf0xtcVZnm/i+a27DE4u3GuVezj5OwFteArlQWuHHVROtQ4eXswln72obY8/B/Ok/u3Fh0400ZpIZHhZ0THeGfg==;25:hYiUCXUkZ+C11W9YQ1rOQDxAYjracc54FHvheLLsGVnfVHOSZ5ufvmbjtgqEXpZfSGXKxg4aByj+v4LOeml/znRQbS+SHXlRxc1ygiX/O3/72cqSWmQ0HpwhtghWmWZqXjsUgNqWL5JRgQmacls8ICmVkCSsxqwFXFkDebKKQdeTXvcRYZrwrLS6R46Oa0cNj+5M++IqL4zRR0hMxfqNZyQc/sF6EMCmy/KPw6mO3Zk=;20:btjsi1IWcjs3mbbE1jooVD0TLAiVlCsxRWGC+yrvrRnQI8vaKD/LuBKDAOYOM91I9R2SSTtaJY//eTc/A837zg==;23:QmlumSSlQoDKjJh452Tr5W1o2ifHR7Ivxc8PvN0QR9z9qgi2Nel7eGXsSdt09LgadmxqOgZBetwdOeF/pO3KVrjxxIWiiPG9NNwCYosCOjRHC5jquFZtSKoqmcZaWYLEOT/I8HMJe0PjLcF9uFn+8co1Y7QXR5Dz9n1gt0uiOmHi7Cm4uB2pJmeB5D+0fe+8hwm+8En+BVpe3TQgengYwzhS8D5m3Ey2fAY52uE4zQdWI44xTDMTlbKOzc+WOaw5
BLUPR0701MB1636: X-MS-Exchange-Organization-RulesExecuted
X-OriginatorOrg: caviumnetworks.com
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 2222
Lines: 72

On 07/14/2015 09:29 AM, Will Deacon wrote:
> On Tue, Jul 14, 2015 at 05:12:57PM +0100, David Daney wrote:
>> On 07/14/2015 04:00 AM, Will Deacon wrote:
>>> On Mon, Jul 13, 2015 at 10:31:36PM +0100, David Daney wrote:
>>>> From: David Daney <david.daney@cavium.com>
>>>>
>>>> Needed to make pci_iomap() work.
>>>
>>> Care to elaborate?
>>>
>>
>> I should have explained what I am doing here a little better.
>
> Yeah, thanks.
>
>> Systems based on the Cavium ThunderX processor may have up to 8
>> independent PCIe root complexes.  The I/O space on each bus occupies an
>> independent physical address window.
>
> Hmm, so do you have 64k of I/O space per-bus? That gives 8x256x64k = 128M
> IIUC, so not sure what your 32MB is for.

I don't understand where your 256 came from there.

Actually, my current implementation has 1M per bus(which is overkill). 
For 8 buses I need 8M, which fits within the PCI_IO_SIZE...

>
>> So, in order to be able to map all of these (semi) contiguously, we need
>> a lot more virtual address space than is supplied by the default values
>> for all these constants.
>>
>> The option I chose here was to unconditionally expand the I/O ranges for
>> all arm64 systems.  If you think this breaks existing systems/drivers, I
>> will have to look for other options.
>
> Hmm, but pci_iomap winds up calling __pci_ioport_map, which expands to
> ioport_map which just does:
>
> 	return PCI_IOBASE + (port & IO_SPACE_LIMIT);
>
> so I'm struggling to see what your patch achieves.

Here is ioport_map (from lib/iomap.c):


void __iomem *ioport_map(unsigned long port, unsigned int nr)
{
	if (port > PIO_MASK)
		return NULL;
	return (void __iomem *) (unsigned long) (port + PIO_OFFSET);
}

With the default value of PIO_MASK (64K), I cannot map any I/O ports on 
my PCIe RC 1..7

The values I supplied in my patch may be sub-optimal, but I think 
something is needed.  I will look into this in a little more detail today.

Thanks,
David Daney


>
> Will
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/