On 27.01.15 15:35, Stuart Yoder wrote:
> Hi Arnd/Alex,
>
> German has posted an example driver for the fsl-mc bus in his RFC
> "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".
>
> In addition I have made available the skeleton for a driver for
> one of the objects/devices (crypto) that will be discovered on
> the bus:
> https://github.com/stuyoder/linux
> branch: fsl-ms-bus
>
> ...it is not functional yet, but shows how a driver registers with
> the bus, get's probed, performs initialization.
Ok, so if I grasp this correctly the idea is that we have a driver
attaching to an individual device on the fsl-mc bus. That driver then
goes and allocates / blocks more devices from that bus as it initializes.
Is that model always possible? Which device would a NIC bind to for
example? I merely want to make sure we're not running ourselves into a
bad corner ;).
Alex
> -----Original Message-----
> From: Alexander Graf [mailto:[email protected]]
> Sent: Thursday, February 26, 2015 8:33 AM
> To: Yoder Stuart-B08248; [email protected]
> Cc: Rivera Jose-B46482; [email protected]; [email protected]
> Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series
>
>
>
> On 27.01.15 15:35, Stuart Yoder wrote:
> > Hi Arnd/Alex,
> >
> > German has posted an example driver for the fsl-mc bus in his RFC
> > "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".
> >
> > In addition I have made available the skeleton for a driver for
> > one of the objects/devices (crypto) that will be discovered on
> > the bus:
> > https://github.com/stuyoder/linux
> > branch: fsl-ms-bus
> >
> > ...it is not functional yet, but shows how a driver registers with
> > the bus, get's probed, performs initialization.
>
> Ok, so if I grasp this correctly the idea is that we have a driver
> attaching to an individual device on the fsl-mc bus.
Yes.
> That driver then
> goes and allocates / blocks more devices from that bus as it initializes.
Yes, there are certain devices/objects on the bus that by themselves
are not standalone, functional devices. An example is a "buffer pool".
Network interface drivers, crypto driver, decompression driver, etc need
one or more hardware buffer pools. There is a buffer depletion interrupt
associated with the device.
The buffer pools itself binds to a resource allocation driver in
the kernel, which then can hand out buffer pools as required by
other drivers.
> Is that model always possible?
Yes, why would it not be?
> Which device would a NIC bind to for
> example?
Network interface / Ethernet driver requires some number
of buffer pools, plus a management complex portal device
(DPMCP) used for sending commands to manage the hardware.
> I merely want to make sure we're not running ourselves into a
> bad corner ;).
If we are, I would like to understand it. :)
Thanks,
Stuart
On 26.02.15 21:32, Stuart Yoder wrote:
>
>
>> -----Original Message-----
>> From: Alexander Graf [mailto:[email protected]]
>> Sent: Thursday, February 26, 2015 8:33 AM
>> To: Yoder Stuart-B08248; [email protected]
>> Cc: Rivera Jose-B46482; [email protected]; [email protected]
>> Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series
>>
>>
>>
>> On 27.01.15 15:35, Stuart Yoder wrote:
>>> Hi Arnd/Alex,
>>>
>>> German has posted an example driver for the fsl-mc bus in his RFC
>>> "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".
>>>
>>> In addition I have made available the skeleton for a driver for
>>> one of the objects/devices (crypto) that will be discovered on
>>> the bus:
>>> https://github.com/stuyoder/linux
>>> branch: fsl-ms-bus
>>>
>>> ...it is not functional yet, but shows how a driver registers with
>>> the bus, get's probed, performs initialization.
>>
>> Ok, so if I grasp this correctly the idea is that we have a driver
>> attaching to an individual device on the fsl-mc bus.
>
> Yes.
>
>> That driver then
>> goes and allocates / blocks more devices from that bus as it initializes.
>
> Yes, there are certain devices/objects on the bus that by themselves
> are not standalone, functional devices. An example is a "buffer pool".
> Network interface drivers, crypto driver, decompression driver, etc need
> one or more hardware buffer pools. There is a buffer depletion interrupt
> associated with the device.
>
> The buffer pools itself binds to a resource allocation driver in
> the kernel, which then can hand out buffer pools as required by
> other drivers.
Ok, so there are 2 things on the bus
* devices
* resources
Someone really needs to sit down and write some nice ASCII art about all
of this and include all the abbreviations in it as well, so that anyone
not deeply involved in the architecture has the chance to grasp what
this is about.
>> Is that model always possible?
>
> Yes, why would it not be?
>
>> Which device would a NIC bind to for
>> example?
>
> Network interface / Ethernet driver requires some number
> of buffer pools, plus a management complex portal device
> (DPMCP) used for sending commands to manage the hardware.
Ok, so there is always one object that basically "owns" a particular
device. And then there is a cloud of resources that drivers grab as they go.
I think I got it by now and the concept makes a lot of sense. I'm not
sure whether there's any particular benefit or downside of having
resources be devices, but looking at the resource manager code it
probably doesn't hurt.
Alex
> -----Original Message-----
> From: Alexander Graf [mailto:[email protected]]
> Sent: Thursday, February 26, 2015 3:38 PM
> To: Yoder Stuart-B08248; [email protected]
> Cc: Rivera Jose-B46482; [email protected]; [email protected]
> Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series
>
>
>
> On 26.02.15 21:32, Stuart Yoder wrote:
> >
> >
> >> -----Original Message-----
> >> From: Alexander Graf [mailto:[email protected]]
> >> Sent: Thursday, February 26, 2015 8:33 AM
> >> To: Yoder Stuart-B08248; [email protected]
> >> Cc: Rivera Jose-B46482; [email protected]; [email protected]
> >> Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series
> >>
> >>
> >>
> >> On 27.01.15 15:35, Stuart Yoder wrote:
> >>> Hi Arnd/Alex,
> >>>
> >>> German has posted an example driver for the fsl-mc bus in his RFC
> >>> "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".
> >>>
> >>> In addition I have made available the skeleton for a driver for
> >>> one of the objects/devices (crypto) that will be discovered on
> >>> the bus:
> >>> https://github.com/stuyoder/linux
> >>> branch: fsl-ms-bus
> >>>
> >>> ...it is not functional yet, but shows how a driver registers with
> >>> the bus, get's probed, performs initialization.
> >>
> >> Ok, so if I grasp this correctly the idea is that we have a driver
> >> attaching to an individual device on the fsl-mc bus.
> >
> > Yes.
> >
> >> That driver then
> >> goes and allocates / blocks more devices from that bus as it initializes.
> >
> > Yes, there are certain devices/objects on the bus that by themselves
> > are not standalone, functional devices. An example is a "buffer pool".
> > Network interface drivers, crypto driver, decompression driver, etc need
> > one or more hardware buffer pools. There is a buffer depletion interrupt
> > associated with the device.
> >
> > The buffer pools itself binds to a resource allocation driver in
> > the kernel, which then can hand out buffer pools as required by
> > other drivers.
>
> Ok, so there are 2 things on the bus
>
> * devices
> * resources
In the general sense, yes. To be picky about terminology we call
all these things on the bus "objects". Some are more resource-like,
in that they are handed out by an allocator to the functional drivers.
I don't want to call them 'resources' because that term actually means
something slightly different in the hardware architecture that is not
actually visible to Linux.
> Someone really needs to sit down and write some nice ASCII art about all
> of this and include all the abbreviations in it as well, so that anyone
> not deeply involved in the architecture has the chance to grasp what
> this is about.
The cover letter for the patch series is a starting point, but
yes we need something for ./Documentation.
> >> Is that model always possible?
> >
> > Yes, why would it not be?
> >
> >> Which device would a NIC bind to for
> >> example?
> >
> > Network interface / Ethernet driver requires some number
> > of buffer pools, plus a management complex portal device
> > (DPMCP) used for sending commands to manage the hardware.
>
> Ok, so there is always one object that basically "owns" a particular
> device. And then there is a cloud of resources that drivers grab as they go.
>
> I think I got it by now and the concept makes a lot of sense. I'm not
> sure whether there's any particular benefit or downside of having
> resources be devices, but looking at the resource manager code it
> probably doesn't hurt.
They need to be real Linux devices. The reason is that when we
bind a DPRC and the objects in it to VFIO, VFIO expects everything
to be a device. VFIO exposes 'devices' to user space, and so for
example a buffer pool's IRQ needs to be exposed via standard VFIO
mechanisms.
Thanks,
Stuart
On 26.02.15 23:25, Stuart Yoder wrote:
>
>
>> -----Original Message-----
>> From: Alexander Graf [mailto:[email protected]]
>> Sent: Thursday, February 26, 2015 3:38 PM
>> To: Yoder Stuart-B08248; [email protected]
>> Cc: Rivera Jose-B46482; [email protected]; [email protected]
>> Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series
>>
>>
>>
>> On 26.02.15 21:32, Stuart Yoder wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Alexander Graf [mailto:[email protected]]
>>>> Sent: Thursday, February 26, 2015 8:33 AM
>>>> To: Yoder Stuart-B08248; [email protected]
>>>> Cc: Rivera Jose-B46482; [email protected]; [email protected]
>>>> Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series
>>>>
>>>>
>>>>
>>>> On 27.01.15 15:35, Stuart Yoder wrote:
>>>>> Hi Arnd/Alex,
>>>>>
>>>>> German has posted an example driver for the fsl-mc bus in his RFC
>>>>> "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".
>>>>>
>>>>> In addition I have made available the skeleton for a driver for
>>>>> one of the objects/devices (crypto) that will be discovered on
>>>>> the bus:
>>>>> https://github.com/stuyoder/linux
>>>>> branch: fsl-ms-bus
>>>>>
>>>>> ...it is not functional yet, but shows how a driver registers with
>>>>> the bus, get's probed, performs initialization.
>>>>
>>>> Ok, so if I grasp this correctly the idea is that we have a driver
>>>> attaching to an individual device on the fsl-mc bus.
>>>
>>> Yes.
>>>
>>>> That driver then
>>>> goes and allocates / blocks more devices from that bus as it initializes.
>>>
>>> Yes, there are certain devices/objects on the bus that by themselves
>>> are not standalone, functional devices. An example is a "buffer pool".
>>> Network interface drivers, crypto driver, decompression driver, etc need
>>> one or more hardware buffer pools. There is a buffer depletion interrupt
>>> associated with the device.
>>>
>>> The buffer pools itself binds to a resource allocation driver in
>>> the kernel, which then can hand out buffer pools as required by
>>> other drivers.
>>
>> Ok, so there are 2 things on the bus
>>
>> * devices
>> * resources
>
> In the general sense, yes. To be picky about terminology we call
> all these things on the bus "objects". Some are more resource-like,
> in that they are handed out by an allocator to the functional drivers.
>
> I don't want to call them 'resources' because that term actually means
> something slightly different in the hardware architecture that is not
> actually visible to Linux.
>
>> Someone really needs to sit down and write some nice ASCII art about all
>> of this and include all the abbreviations in it as well, so that anyone
>> not deeply involved in the architecture has the chance to grasp what
>> this is about.
>
> The cover letter for the patch series is a starting point, but
> yes we need something for ./Documentation.
>
>>>> Is that model always possible?
>>>
>>> Yes, why would it not be?
>>>
>>>> Which device would a NIC bind to for
>>>> example?
>>>
>>> Network interface / Ethernet driver requires some number
>>> of buffer pools, plus a management complex portal device
>>> (DPMCP) used for sending commands to manage the hardware.
>>
>> Ok, so there is always one object that basically "owns" a particular
>> device. And then there is a cloud of resources that drivers grab as they go.
>>
>> I think I got it by now and the concept makes a lot of sense. I'm not
>> sure whether there's any particular benefit or downside of having
>> resources be devices, but looking at the resource manager code it
>> probably doesn't hurt.
>
> They need to be real Linux devices. The reason is that when we
> bind a DPRC and the objects in it to VFIO, VFIO expects everything
> to be a device. VFIO exposes 'devices' to user space, and so for
> example a buffer pool's IRQ needs to be exposed via standard VFIO
> mechanisms.
Ah, I see. Yeah, certainly works well for me :).
Alex