Hi Dmitry,
On 7 July 2015 at 12:47, Dmitry Kalinkin <[email protected]> wrote:
[...]
> The API I had in mind would have only vme_master_read and vme_master_write
> that would take absolute addresses (not relative to any window). These
> variants
> of access functions would then try to reuse any window that is already able
> to serve
> the request or wait for a free window and reconfigure it for the need of the
> request.
> After usage the window is to be returned back to the window pool.
Interesting approach.
> Other way to implement these would be to use DMA for everything, since it
> doesn’t
> have the limitations that the windows have.
> On 07 Jul 2015, at 10:08, Alessio Igor Bogani <[email protected]>
> wrote:
[...]
>> In fact this is a big obstacle for adoption of this VME stack (at least for
>> us). We use VME a lot and we care about latency as well so we use only
>> kernel-space drivers for ours VME boards but unfortunately the VME stack let
>> us to link a single board with a single window/image (so max 8 boards on
>> tsi148) only. That said that stack has proven to be very rock solid.
>
> Current VME stack links windows not to the boards, but to device drivers.
> Driver
> could potentially minimise window usage within it’s scope (any sort of
> window
> reusing, like mapping whole A16 once to be used with all boards), but this
> won’t
> work across multiple drivers. Even if all of your drivers are window-wise
> economic,
> they will still need some amount of windows per each driver. Not that we
> have that
> many kernel drivers...
Yes you can share a window/image between all boards of the same type
(in effect we are porting our drivers in this way) *but* it isn't the
expected way to work (see Documentation/vme_api.txt struct
vme_driver's probe() and match() functions and the GE PIO2 VME
driver).
Ciao,
Alessio
> On 07 Jul 2015, at 15:51, Alessio Igor Bogani <[email protected]> wrote:
>
<snip>
>> Current VME stack links windows not to the boards, but to device drivers.
>> Driver
>> could potentially minimise window usage within it’s scope (any sort of
>> window
>> reusing, like mapping whole A16 once to be used with all boards), but this
>> won’t
>> work across multiple drivers. Even if all of your drivers are window-wise
>> economic,
>> they will still need some amount of windows per each driver. Not that we
>> have that
>> many kernel drivers...
>
> Yes you can share a window/image between all boards of the same type
> (in effect we are porting our drivers in this way) *but* it isn't the
> expected way to work (see Documentation/vme_api.txt struct
> vme_driver's probe() and match() functions and the GE PIO2 VME
> driver).
And vme_pio2 can’t handle more than 8 boards. This shows that the current
design needs some adjustments. Also would be great if probe() and match()
allowed for void *private data field.
Cheers,
Dmitry-
On 07/07/15 13:51, Alessio Igor Bogani wrote:
>> Current VME stack links windows not to the boards, but to device drivers.
>> >Driver
>> >could potentially minimise window usage within it’s scope (any sort of
>> >window
>> >reusing, like mapping whole A16 once to be used with all boards), but this
>> >won’t
>> >work across multiple drivers. Even if all of your drivers are window-wise
>> >economic,
>> >they will still need some amount of windows per each driver. Not that we
>> >have that
>> >many kernel drivers...
> Yes you can share a window/image between all boards of the same type
> (in effect we are porting our drivers in this way)*but* it isn't the
> expected way to work (see Documentation/vme_api.txt struct
> vme_driver's probe() and match() functions and the GE PIO2 VME
> driver).
I think it's perfectly valid to use a single window to dynamically map
to the address space belonging to one of a number of devices supported
by a single driver. I think this is almost preferable to mapping a large
window over a large portion of the VME address space to drive a number
of devices as (depending on there spacing in the VME address space) the
latter could cause issues with filling available PCI address space.
Admittedly this is more of a problem on 32-bit systems, but...
--
Martyn Welch (Lead Software Engineer) | Registered in England and Wales
GE Intelligent Platforms | (3828642) at 100 Barbirolli Square
T +44(0)1327322748 | Manchester, M2 3AB
E [email protected] | VAT:GB 927559189
> On 08 Jul 2015, at 16:41, Martyn Welch <[email protected]> wrote:
>
> On 07/07/15 13:51, Alessio Igor Bogani wrote:
>>> Current VME stack links windows not to the boards, but to device drivers.
>>> >Driver
>>> >could potentially minimise window usage within it’s scope (any sort of
>>> >window
>>> >reusing, like mapping whole A16 once to be used with all boards), but this
>>> >won’t
>>> >work across multiple drivers. Even if all of your drivers are window-wise
>>> >economic,
>>> >they will still need some amount of windows per each driver. Not that we
>>> >have that
>>> >many kernel drivers...
>> Yes you can share a window/image between all boards of the same type
>> (in effect we are porting our drivers in this way)*but* it isn't the
>> expected way to work (see Documentation/vme_api.txt struct
>> vme_driver's probe() and match() functions and the GE PIO2 VME
>> driver).
>
> I think it's perfectly valid to use a single window to dynamically map to the address space belonging to one of a number of devices supported by a single driver. I think this is almost preferable to mapping a large window over a large portion of the VME address space to drive a number of devices as (depending on there spacing in the VME address space) the latter could cause issues with filling available PCI address space. Admittedly this is more of a problem on 32-bit systems, but…
Speaking of which. We do have 32 bit SBC’s (Fastwell CPC600) and there we can only map up to 32mb. If we drop an unnecessary alignment requirement, this can be raised up to 63mb. The next comes a hack that is a bit dirty: changing pdev->bus to something like pdev->bus->parent. Then maximal VME window size is only limited by vmalloc memory which on 32bit system can be raised up to ~700 mb by adding vmalloc=768M to kernel argument line.
For the reference, here is the iomem mapping of the working system without the hack (also some window mappings are already done):
% cat /proc/iomem
00000000-00000fff : reserved
00001000-00097fff : System RAM
00098000-0009ffff : reserved
000a0000-000bffff : PCI Bus 0000:00
000a0000-000bffff : Video RAM area
000c0000-000c7fff : Video ROM
000d0000-000d0fff : Adapter ROM
000d1000-000d29ff : Adapter ROM
000d3000-000d3fff : Adapter ROM
000d4000-000d4fff : Adapter ROM
000d8000-000dbfff : PCI Bus 0000:00
000dc000-000fffff : reserved
000f0000-000fffff : System ROM
00100000-3de8ffff : System RAM
01000000-01822261 : Kernel code
01822262-01ca4e3f : Kernel data
01da1000-01e6dfff : Kernel bss
3de90000-3de96fff : ACPI Tables
3de97000-3defffff : ACPI Non-volatile Storage
3df00000-3fffffff : reserved
40000000-febfffff : PCI Bus 0000:00
d8000000-d807ffff : 0000:00:02.0
d8080000-d80fffff : 0000:00:02.1
d8100000-d810000f : 0000:00:1d.4
d8100400-d81007ff : 0000:00:1d.7
d8100400-d81007ff : ehci_hcd
d8200000-dfffffff : PCI Bus 0000:02
d8200000-d821ffff : 0000:02:00.0
d8200000-d821ffff : e1000
d8220000-d823ffff : 0000:02:00.1
d8220000-d823ffff : e1000
d8240000-d825ffff : 0000:02:01.0
d8240000-d825ffff : e1000
d8260000-d827ffff : 0000:02:01.1
d8260000-d827ffff : e1000
d8280000-d82fffff : vme_tsi148.7
d8300000-d830ffff : vme_tsi148.0
d8310000-d930ffff : vme_tsi148.3
d9310000-d931ffff : vme_tsi148.2
dc000000-dc000fff : 0000:02:02.0
dc000000-dc000fff : vme_tsi148
e8000000-efffffff : 0000:00:02.0
f0000000-f7ffffff : 0000:00:02.1
fec00000-fec003ff : IOAPIC 0
fec10000-fec103ff : IOAPIC 1
fee00000-feefffff : pnp 00:01
fee00000-fee00fff : Local APIC
fee00000-fee00fff : pnp 00:02
feff0000-feffffff : reserved
feff0000-feffffff : pnp 00:01
ff800000-ffffffff : INT0800:00
ff800000-ffbfffff : reserved
fffffc00-ffffffff : reserved
Universe II allows PCI address grannularity of 4K or 64K depending on
the window id. tsi148 only supports 64K. Existing driver implementations
are validating window size against this grannularity and then use that
very size as alignment parameter to pci_bus_alloc_resource. This
constraint is excessive, alignment by granularity should be enough.
This changes alignment constraint from size to a fixed constraint of
64K.
Signed-off-by: Dmitry Kalinkin <[email protected]>
---
drivers/vme/bridges/vme_ca91cx42.c | 2 +-
drivers/vme/bridges/vme_tsi148.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/vme/bridges/vme_ca91cx42.c b/drivers/vme/bridges/vme_ca91cx42.c
index 834883d..b79a74a 100644
--- a/drivers/vme/bridges/vme_ca91cx42.c
+++ b/drivers/vme/bridges/vme_ca91cx42.c
@@ -553,7 +553,7 @@ static int ca91cx42_alloc_resource(struct vme_master_resource *image,
image->bus_resource.flags = IORESOURCE_MEM;
retval = pci_bus_alloc_resource(pdev->bus,
- &image->bus_resource, size, size, PCIBIOS_MIN_MEM,
+ &image->bus_resource, size, 0x10000, PCIBIOS_MIN_MEM,
0, NULL, NULL);
if (retval) {
dev_err(ca91cx42_bridge->parent, "Failed to allocate mem "
diff --git a/drivers/vme/bridges/vme_tsi148.c b/drivers/vme/bridges/vme_tsi148.c
index b0132e0..d1e383b 100644
--- a/drivers/vme/bridges/vme_tsi148.c
+++ b/drivers/vme/bridges/vme_tsi148.c
@@ -768,7 +768,7 @@ static int tsi148_alloc_resource(struct vme_master_resource *image,
image->bus_resource.flags = IORESOURCE_MEM;
retval = pci_bus_alloc_resource(pdev->bus,
- &image->bus_resource, size, size, PCIBIOS_MIN_MEM,
+ &image->bus_resource, size, 0x10000, PCIBIOS_MIN_MEM,
0, NULL, NULL);
if (retval) {
dev_err(tsi148_bridge->parent, "Failed to allocate mem "
--
1.8.3.1