2010-07-01 22:10:47

by Dave Airlie

[permalink] [raw]
Subject: Closed source userspace graphics drivers with an open source kernel component

Now this is just my opinion as maintainer of the drm, and doesn't
reflect anyone or any official policy, I've also no idea if Linus
agrees or not.

We are going to start to see a number of companies in the embedded
space submitting 3D drivers for mobile devices to the kernel. I'd like
to clarify my position once so they don't all come asking the same
questions.

If you aren't going to create an open userspace driver (either MIT or
LGPL) then don't waste time submitting a kernel driver to me.

My reasons are as follows, the thing is you can probably excuse some
of these on a point by point basis, but you need to justify why closed
userspace on all points.

a) licensing, Alan Cox pointed this out before, if you wrote a GPL
kernel driver, then wrote a closed userspace on top, you open up a
while world of derived work issues. Can the userspace operate on a
non-GPL kernel without major modifications etc. This is a can of worms
I'd rather not enter into, and there are a few workarounds.

b) verifying the sanity of the userspace API.
1. Security: GPUs can do a lot of damage if left at home alone, since
mostly you are submitting command streams unverified into the GPU and
won't tell us what they mean, there is little way we can work out if
the GPU is going to over-write my passwd file to get 5 fps more in
quake. Now newer GPUs have at least started having MMUs, but again
we've no idea if that is the only way they work without docs or a lot
of trust.

2. General API suitability and versioning. How do we check that API is
sane wrt to userspace, if we can't verify the userspace. What happens
if the API has lots of 32/64 compat issues or things like that, and
when we fix them the binary userspace breaks? How do we know, how do
we test etc. What happens if a security issue forces us to break the
userspace API? how do we fix the userspace driver and test to confirm?

c) supplying docs in lieu of an open userspace
If you were to fully document the GPU so we could verify the
security/api aspects it leaves us in the position of writing our own
driver. Now writing that driver on top of the current kernel driver
would probably limit any innovation, and most people would want to
write a new kernel driver from scratch. Now we end up with two drivers
fighting, how do we pick which one to load at boot? can we ever do a
generic distro kernel for that device (assuming ARM ever solves that
issue).

I've also noticed a trend to just reinvent the whole wheel instead of
writing a drm/kms driver and having that as the API, again maintainer
nightmares are made of this.

Dave.


2010-07-01 22:36:47

by Dave Airlie

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 2, 2010 at 8:10 AM, Dave Airlie <[email protected]> wrote:
> Now this is just my opinion as maintainer of the drm, and doesn't
> reflect anyone or any official policy, I've also no idea if Linus
> agrees or not.
>
> We are going to start to see a number of companies in the embedded
> space submitting 3D drivers for mobile devices to the kernel. I'd like
> to clarify my position once so they don't all come asking the same
> questions.
>
> If you aren't going to create an open userspace driver (either MIT or
> LGPL) then don't waste time submitting a kernel driver to me.
>
> My reasons are as follows, the thing is you can probably excuse some
> of these on a point by point basis, but you need to justify why closed
> userspace on all points.
>
> a) licensing, Alan Cox pointed this out before, if you wrote a GPL
> kernel driver, then wrote a closed userspace on top, you open up a
> while world of derived work issues. Can the userspace operate on a
> non-GPL kernel without major modifications etc. This is a can of worms
> I'd rather not enter into, and there are a few workarounds.
>
> b) verifying the sanity of the userspace API.
> 1. Security: GPUs can do a lot of damage if left at home alone, since
> mostly you are submitting command streams unverified into the GPU and
> won't tell us what they mean, there is little way we can work out if
> the GPU is going to over-write my passwd file to get 5 fps more in
> quake. Now newer GPUs have at least started having MMUs, but again
> we've no idea if that is the only way they work without docs or a lot
> of trust.
>
> 2. General API suitability and versioning. How do we check that API is
> sane wrt to userspace, if we can't verify the userspace. What happens
> if the API has lots of 32/64 compat issues or things like that, and
> when we fix them the binary userspace breaks? How do we know, how do
> we test etc. What happens if a security issue forces us to break the
> userspace API? how do we fix the userspace driver and test to confirm?
>
> c) supplying docs in lieu of an open userspace
> If you were to fully document the GPU so we could verify the
> security/api aspects it leaves us in the position of writing our own
> driver. Now writing that driver on top of the current kernel driver
> would probably limit any innovation, and most people would want to
> write a new kernel driver from scratch. Now we end up with two drivers
> fighting, how do we pick which one to load at boot? can we ever do a
> generic distro kernel for that device (assuming ARM ever solves that
> issue).
>
> I've also noticed a trend to just reinvent the whole wheel instead of
> writing a drm/kms driver and having that as the API, again maintainer
> nightmares are made of this.
>
> Dave.

Oh and (one other thought)

d) you are placing the maintenance burden in the wrong place

So you've upstreamed the kernel bits, kept the good userspace bits to
yourselfs, are stroking them on your lap like some sort of Dr Evil,
now why should the upstream kernel maintainers take the burden when
you won't actually give them the stuff to really make their hardware
work? This goes for nvidia type situations as well, the whole point is
to place the maintainer burden at the feet of the people causing the
problems in an effort to make them change. Allowing even an hour of
that burden to be transferred upstream, means more profit for them,
but nothing in return for us.

Dave.

2010-07-01 22:51:30

by Daniel Walker

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, 2010-07-02 at 08:36 +1000, Dave Airlie wrote:
> On Fri, Jul 2, 2010 at 8:10 AM, Dave Airlie <[email protected]> wrote:
> > Now this is just my opinion as maintainer of the drm, and doesn't
> > reflect anyone or any official policy, I've also no idea if Linus
> > agrees or not.
> >
> > We are going to start to see a number of companies in the embedded
> > space submitting 3D drivers for mobile devices to the kernel. I'd like
> > to clarify my position once so they don't all come asking the same
> > questions.
> >
> > If you aren't going to create an open userspace driver (either MIT or
> > LGPL) then don't waste time submitting a kernel driver to me.

If , for whatever reason, you changed you mind on this what sort of
connection between kernel and userspace would these components use?

I ask only because I think UIO hold most (if not all) the driver in
userspace .. So you would have to use some other interface if you wanted
a more half and half solution ..

Daniel

--
Sent by a consultant of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

2010-07-01 22:57:07

by Dave Airlie

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 2, 2010 at 8:51 AM, Daniel Walker <[email protected]> wrote:
> On Fri, 2010-07-02 at 08:36 +1000, Dave Airlie wrote:
>> On Fri, Jul 2, 2010 at 8:10 AM, Dave Airlie <[email protected]> wrote:
>> > Now this is just my opinion as maintainer of the drm, and doesn't
>> > reflect anyone or any official policy, I've also no idea if Linus
>> > agrees or not.
>> >
>> > We are going to start to see a number of companies in the embedded
>> > space submitting 3D drivers for mobile devices to the kernel. I'd like
>> > to clarify my position once so they don't all come asking the same
>> > questions.
>> >
>> > If you aren't going to create an open userspace driver (either MIT or
>> > LGPL) then don't waste time submitting a kernel driver to me.
>
> If , for whatever reason, you changed you mind on this what sort of
> connection between kernel and userspace would these components use?
>
> I ask only because I think UIO hold most (if not all) the driver in
> userspace .. So you would have to use some other interface if you wanted
> a more half and half solution ..
>

The thing is UIO doesn't solve the problem 3D graphics drivers need to
solve. Which is we need to let unprivileged users access the graphics
device in an efficient manner, hence why DRI/DRM exists. Now I think
the tegra guys have done some evil hacks with a userspace daemon to
replace the kernel functionality, so all they have in-kernel is a
framebuffer device, since they can't really get away with shipping the
binary nvidia driver linked to the kernel in a real device. So all
their userspace closed bits talk to the daemon running as root with
direct access to the lowlevel hw.

Dave.

2010-07-01 23:29:46

by Daniel Walker

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, 2010-07-02 at 08:57 +1000, Dave Airlie wrote:
> On Fri, Jul 2, 2010 at 8:51 AM, Daniel Walker <[email protected]> wrote:
> > On Fri, 2010-07-02 at 08:36 +1000, Dave Airlie wrote:
> >> On Fri, Jul 2, 2010 at 8:10 AM, Dave Airlie <[email protected]> wrote:
> >> > Now this is just my opinion as maintainer of the drm, and doesn't
> >> > reflect anyone or any official policy, I've also no idea if Linus
> >> > agrees or not.
> >> >
> >> > We are going to start to see a number of companies in the embedded
> >> > space submitting 3D drivers for mobile devices to the kernel. I'd like
> >> > to clarify my position once so they don't all come asking the same
> >> > questions.
> >> >
> >> > If you aren't going to create an open userspace driver (either MIT or
> >> > LGPL) then don't waste time submitting a kernel driver to me.
> >
> > If , for whatever reason, you changed you mind on this what sort of
> > connection between kernel and userspace would these components use?
> >
> > I ask only because I think UIO hold most (if not all) the driver in
> > userspace .. So you would have to use some other interface if you wanted
> > a more half and half solution ..
> >
>
> The thing is UIO doesn't solve the problem 3D graphics drivers need to
> solve. Which is we need to let unprivileged users access the graphics
> device in an efficient manner, hence why DRI/DRM exists. Now I think
> the tegra guys have done some evil hacks with a userspace daemon to
> replace the kernel functionality, so all they have in-kernel is a
> framebuffer device, since they can't really get away with shipping the
> binary nvidia driver linked to the kernel in a real device. So all
> their userspace closed bits talk to the daemon running as root with
> direct access to the lowlevel hw.

Oh, man .. It seems like any driver model that straddles userspace and
kernel space is kind of asking for trouble (my opinion anyway)..

Would you accept a userspace component that supported some subset of the
features ? You would have a kernel space driver, and userspace both open
source and GPL'd , but the userspace component wouldn't support ever
feature available .. Then the company would be free to make another
proprietary userspace with more features based off the open source one.

Daniel
--
Sent by a consultant of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

2010-07-01 23:38:05

by Dave Airlie

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 2, 2010 at 9:29 AM, Daniel Walker <[email protected]> wrote:
> On Fri, 2010-07-02 at 08:57 +1000, Dave Airlie wrote:
>> On Fri, Jul 2, 2010 at 8:51 AM, Daniel Walker <[email protected]> wrote:
>> > On Fri, 2010-07-02 at 08:36 +1000, Dave Airlie wrote:
>> >> On Fri, Jul 2, 2010 at 8:10 AM, Dave Airlie <[email protected]> wrote:
>> >> > Now this is just my opinion as maintainer of the drm, and doesn't
>> >> > reflect anyone or any official policy, I've also no idea if Linus
>> >> > agrees or not.
>> >> >
>> >> > We are going to start to see a number of companies in the embedded
>> >> > space submitting 3D drivers for mobile devices to the kernel. I'd like
>> >> > to clarify my position once so they don't all come asking the same
>> >> > questions.
>> >> >
>> >> > If you aren't going to create an open userspace driver (either MIT or
>> >> > LGPL) then don't waste time submitting a kernel driver to me.
>> >
>> > If , for whatever reason, you changed you mind on this what sort of
>> > connection between kernel and userspace would these components use?
>> >
>> > I ask only because I think UIO hold most (if not all) the driver in
>> > userspace .. So you would have to use some other interface if you wanted
>> > a more half and half solution ..
>> >
>>
>> The thing is UIO doesn't solve the problem 3D graphics drivers need to
>> solve. Which is we need to let unprivileged users access the graphics
>> device in an efficient manner, hence why DRI/DRM exists. Now I think
>> the tegra guys have done some evil hacks with a userspace daemon to
>> replace the kernel functionality, so all they have in-kernel is a
>> framebuffer device, since they can't really get away with shipping the
>> binary nvidia driver linked to the kernel in a real device. So all
>> their userspace closed bits talk to the daemon running as root with
>> direct access to the lowlevel hw.
>
> Oh, man .. It seems like any driver model that straddles userspace and
> kernel space is kind of asking for trouble (my opinion anyway)..
>
> Would you accept a userspace component that supported some subset of the
> features ? You would have a kernel space driver, and userspace both open
> source and GPL'd , but the userspace component wouldn't support ever
> feature available .. Then the company would be free to make another
> proprietary userspace with more features based off the open source one.
>

That starts to get a bit more towards useful, except you still run
into the problem of what happens if community developers start adding
features to the open driver, that conflict with features in the
closed code. We'd also have to be very careful about what interfaces
the kernel exposed had corresponding code in userspace. i.e. adding
"special" ioctls for the closed bits would be a disaster, all such
ioctls would need open users for verification and testing.

So for example, if you have a kernel KMS/DRM driver, and it set the
hardware up, but then you had an open 2D driver and a closed 3D
driver, you would have to make sure there was no functionality in the
kernel that only the 3D driver used as it would become impossible to
openly validate it.

The other issue I see with a lot of these, is the driver are presented
as this is the kernel driver, these APIs are set in stone as we have
binary userspaces already deployed, this is even more unacceptable,
since we need to be able to change the interface and do proper driver
design before merging what looks like crap thrown together in a pile
and made to stick.

Dave.

2010-07-02 00:08:18

by Daniel Walker

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, 2010-07-02 at 09:37 +1000, Dave Airlie wrote:

> > Oh, man .. It seems like any driver model that straddles userspace and
> > kernel space is kind of asking for trouble (my opinion anyway)..
> >
> > Would you accept a userspace component that supported some subset of the
> > features ? You would have a kernel space driver, and userspace both open
> > source and GPL'd , but the userspace component wouldn't support ever
> > feature available .. Then the company would be free to make another
> > proprietary userspace with more features based off the open source one.
> >
>
> That starts to get a bit more towards useful, except you still run
> into the problem of what happens if community developers start adding
> features to the open driver, that conflict with features in the
> closed code. We'd also have to be very careful about what interfaces
> the kernel exposed had corresponding code in userspace. i.e. adding
> "special" ioctls for the closed bits would be a disaster, all such
> ioctls would need open users for verification and testing.

Ok .. The open userspace would just be like any other project, but
whatever company pushed the driver would likely maintain the userspace
component .. So the maintainer would have to handle the conflicts
between the proprietary vs. open source sides of it.

Actually , now that I think about it the biggest problem is the license
of the userspace side.. Whatever company makes this would have to be
able to relicense it and actually make a proprietary userspace.. So the
userspace license would be really critical.. Either that or no one
outside the company that pushed the driver could make code changes to
the userspace side..

> So for example, if you have a kernel KMS/DRM driver, and it set the
> hardware up, but then you had an open 2D driver and a closed 3D
> driver, you would have to make sure there was no functionality in the
> kernel that only the 3D driver used as it would become impossible to
> openly validate it.

Ok. I'm not sure how crazy that would be to setup, but it doesn't seem
like it would be that hard to just abstract the various components of
the driver.

> The other issue I see with a lot of these, is the driver are presented
> as this is the kernel driver, these APIs are set in stone as we have
> binary userspaces already deployed, this is even more unacceptable,
> since we need to be able to change the interface and do proper driver
> design before merging what looks like crap thrown together in a pile
> and made to stick.

This seems really wild to me .. Your talking about how you change the
kernel space side and you need to be able to change the userspace side
to match right ?

Where does the userspace side of these driver live? Not in the kernel
right?

Daniel

--
Sent by a consultant of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

2010-07-02 00:13:18

by Dave Airlie

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 2, 2010 at 10:08 AM, Daniel Walker <[email protected]> wrote:
> On Fri, 2010-07-02 at 09:37 +1000, Dave Airlie wrote:
>
>> > Oh, man .. It seems like any driver model that straddles userspace and
>> > kernel space is kind of asking for trouble (my opinion anyway)..
>> >
>> > Would you accept a userspace component that supported some subset of the
>> > features ? You would have a kernel space driver, and userspace both open
>> > source and GPL'd , but the userspace component wouldn't support ever
>> > feature available .. Then the company would be free to make another
>> > proprietary userspace with more features based off the open source one.
>> >
>>
>> That starts to get a bit more towards useful, except you still run
>> into the problem of what happens if community developers start adding
>> features to the open driver, that conflict with ?features in the
>> closed code. We'd also have to be very careful about what interfaces
>> the kernel exposed had corresponding code in userspace. i.e. adding
>> "special" ioctls for the closed bits would be a disaster, all such
>> ioctls would need open users for verification and testing.
>
> Ok .. The open userspace would just be like any other project, but
> whatever company pushed the driver would likely maintain the userspace
> component .. So the maintainer would have to handle the conflicts
> between the proprietary vs. open source sides of it.
>
> Actually , now that I think about it the biggest problem is the license
> of the userspace side.. Whatever company makes this would have to be
> able to relicense it and actually make a proprietary userspace.. So the
> userspace license would be really critical.. Either that or no one
> outside the company that pushed the driver could make code changes to
> the userspace side..

We generally use MIT for userspace bits anyways, I don't think we have
any LGPL/GPL drivers in userspace currently. So this would probably be
an okay solution to continue with.

The thing is with architectures like Gallium it would be possible to
write a complete open driver and just keep the Windows interface bits,
granted we don't have an open gallium to windows driver layer so that
would have to be worked on.

>
>> So for example, if you have a kernel KMS/DRM driver, and it set the
>> hardware up, but then you had an open 2D driver and a closed 3D
>> driver, you would have to make sure there was no functionality in the
>> kernel that only the 3D driver used as it would become impossible to
>> openly validate it.
>
> Ok. I'm not sure how crazy that would be to setup, but it doesn't seem
> like it would be that hard to just abstract the various components of
> the driver.

Its pretty much what something like gallium can do.

>
>> The other issue I see with a lot of these, is the driver are presented
>> as this is the kernel driver, these APIs are set in stone as we have
>> binary userspaces already deployed, this is even more unacceptable,
>> since we need to be able to change the interface and do proper driver
>> design before merging what looks like crap thrown together in a pile
>> and made to stick.
>
> This seems really wild to me .. Your talking about how you change the
> kernel space side and you need to be able to change the userspace side
> to match right ?
>
> Where does the userspace side of these driver live? Not in the kernel
> right?

This is more about initial development stages. We maintain kernel
API/ABI for all in-tree drivers, however before we put a driver into
mainline, we usually need to redo the crazy interfaces that vendors
have come up with. Like 32/64 alignment, passing userspace addresses
into the kernel, passing phy addresses to userspace etc. If the
userspace binary is closed that process becomes next to impossible.

Dave.

2010-07-02 00:18:49

by Saravana Kannan

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

Dave Airlie wrote:
> This is more about initial development stages. We maintain kernel
> API/ABI for all in-tree drivers, however before we put a driver into
> mainline, we usually need to redo the crazy interfaces that vendors
> have come up with. Like 32/64 alignment, passing userspace addresses
> into the kernel, passing phy addresses to userspace etc. If the
> userspace binary is closed that process becomes next to impossible.

My 2 cents:
I think we should leave the onus of fixing the userspace to work with
the sane kernel API with the entity trying to get their drivers into the
kernel. I think it's a better approach (as in, more likelihood of
getting device support) than saying, we will only allow fully open
sourced kernel drivers.

-Saravana

Subject: Re: Closed source userspace graphics drivers with an open source kernel component

> We are going to start to see a number of companies in the embedded
> space submitting 3D drivers for mobile devices to the kernel. I'd like
> to clarify my position once so they don't all come asking the same
> questions.

one of options for future would be equipping gpu's with additional
processing force, allowing it to run whole Xorg on it's own,
and just communicate with rest of machine via shared memory window (so
visible as 'hardware X server' from systems standpoint),

option which allows both - closing 'source' of gpu , and taking
off burden of development for closed, once-use-throwaway
devices from Xorg and kenel crew.

also port of Xorg on GPUs itself allows skipping a lot of features
of kernel, and OS itself (it doesn't to be based on linux afterall)
allowing much more robust performance, and skipping common bottlenecks
(sharing irq's , scheduling, etc)

but this route needs to be considered by hardware vendors themselves.

--

2010-07-02 01:27:39

by Corbin Simpson

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

I thought Intel shelved Larrabee.

~ C.

On Thu, Jul 1, 2010 at 4:51 PM, Piotr Gluszenia Slawinski
<[email protected]> wrote:
>> We are going to start to see a number of companies in the embedded
>> space submitting 3D drivers for mobile devices to the kernel. I'd like
>> to clarify my position once so they don't all come asking the same
>> questions.
>
> one of options for future would be equipping gpu's with additional
> processing force, allowing it to run whole Xorg on it's own,
> and just communicate with rest of machine via shared memory window (so
> visible as 'hardware X server' from systems standpoint),
>
> option which allows both - closing 'source' of gpu , and taking
> off burden of development for closed, once-use-throwaway
> devices from Xorg and kenel crew.
>
> also port of Xorg on GPUs itself allows skipping a lot of features
> of kernel, and OS itself (it doesn't to be based on linux afterall)
> allowing much more robust performance, and skipping common bottlenecks
> (sharing irq's , scheduling, etc)
>
> but this route needs to be considered by hardware vendors themselves.
>
> --
>
> _______________________________________________
> dri-devel mailing list
> [email protected]
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
>



--
When the facts change, I change my mind. What do you do, sir? ~ Keynes

Corbin Simpson
<[email protected]>

2010-07-02 07:48:39

by Christoph Hellwig

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

100% agreed on the rationale, and I hope you can keep this crap out!

2010-07-02 09:58:21

by Luc Verhaegen

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 02, 2010 at 08:10:40AM +1000, Dave Airlie wrote:
> Now this is just my opinion as maintainer of the drm, and doesn't
> reflect anyone or any official policy, I've also no idea if Linus
> agrees or not.
>
> We are going to start to see a number of companies in the embedded
> space submitting 3D drivers for mobile devices to the kernel. I'd like
> to clarify my position once so they don't all come asking the same
> questions.
>
> If you aren't going to create an open userspace driver (either MIT or
> LGPL) then don't waste time submitting a kernel driver to me.
>
> My reasons are as follows, the thing is you can probably excuse some
> of these on a point by point basis, but you need to justify why closed
> userspace on all points.
>
> a) licensing, Alan Cox pointed this out before, if you wrote a GPL
> kernel driver, then wrote a closed userspace on top, you open up a
> while world of derived work issues. Can the userspace operate on a
> non-GPL kernel without major modifications etc. This is a can of worms
> I'd rather not enter into, and there are a few workarounds.

Yes, this a mess indeed.

But i fear that this a mess that cannot be fixed, in its entirety, in a single shot.

Qualcomm making this code available already clearly shows the will and determination of
some people inside qualcomm to do The Right Thing.

This is Qualcomms first big step on the graphics side, where IP is always amongst the
heaviest. I am certain that Qualcomm wants to go further, but since Qualcomm most
likely licenses some parts of their graphics, Qualcomm can only open up those bits that
they truly own, and then use mainly market/sales-volume driven pressure to get the
original IP owner to play along.

You should also take into account who Jordan is. He is one of the guys who worked the
Geode before AMD decided to drop that, which is when he got hired by Qualcomm. He worked on
both the graphics drivers and on (then still) LinuxBIOS. I know that redhat has no
intention of going near coreboot, but in my world one cannot become more free, hardware
wise, than supporting coreboot. This gives me very good hopes that this is a serious
attempt by qualcomm to go somewhere, and that this is not some lame attempt to grab
marketing attention.

Now that you slammed the door on these guys (and on others in the process), what do you
think the response would be? Where will this get us to in the end?

The licensing should get sorted, and that is of course something for Qualcomm to do,
or prove that it has been sorted already.

> b) verifying the sanity of the userspace API.
> 1. Security: GPUs can do a lot of damage if left at home alone, since
> mostly you are submitting command streams unverified into the GPU and
> won't tell us what they mean, there is little way we can work out if
> the GPU is going to over-write my passwd file to get 5 fps more in
> quake. Now newer GPUs have at least started having MMUs, but again
> we've no idea if that is the only way they work without docs or a lot
> of trust.

This makes me wonder: Why do you even care?

If redhat was working with qualcomm, you would not have taken this stance here at all.

Since redhat is then not working with qualcomm, why is this then your responsibility?

Or is denouncing responsibility exactly the reason for your mail here?

If so, why couldn't you have stated "please guys, have fun with what you are doing, but
i will not be responsible for it" in a different way.

What you achieved now is that people will stop bothering with even freeing this,
putting us even further back.

But i fully understand where you are coming from: redhat only wants to seriously back
the server market, so free software graphics on arm based SOCs probably should not be
encouraged too much. As per usual, big statements are then more important than actual
free software advancement.

> 2. General API suitability and versioning. How do we check that API is
> sane wrt to userspace, if we can't verify the userspace. What happens
> if the API has lots of 32/64 compat issues or things like that, and
> when we fix them the binary userspace breaks? How do we know, how do
> we test etc. What happens if a security issue forces us to break the
> userspace API? how do we fix the userspace driver and test to confirm?
>
> c) supplying docs in lieu of an open userspace
> If you were to fully document the GPU so we could verify the
> security/api aspects it leaves us in the position of writing our own
> driver. Now writing that driver on top of the current kernel driver
> would probably limit any innovation, and most people would want to
> write a new kernel driver from scratch. Now we end up with two drivers
> fighting, how do we pick which one to load at boot? can we ever do a
> generic distro kernel for that device (assuming ARM ever solves that
> issue).

I think that by now you should have realized that this is not how it works for
things as complex as graphics drivers. If, for instance, you hadn't been given a
wildcard by your employer, you would never have gone close to AMD hw unless for some
spare time poking and occasional bugfixing.

Also, before you throw this up: for nouveau, documentation would take part of the fun,
the attraction and excitement, away. Provide docs, and then only a few very industrious
people will remain, and they will also get weary after a while, or they get hired by
someone to continue their work, bringing us right back to the corporate world.

Now, it is interesting how you now are demanding documentation. When did recent and
relevant hw documentation happen for ATI? This pretty much died together when the
ATI<->SuSE relationship died, as the cooperation of SuSE and AMD is how documentation
was forced out of ATI in the first place, and ATI more and more found ways to get rid
of this responsibility, or overhead as bridgman would most likely name it.

I think it even should be possible to find statements of you and/or alex, and
definitely bridgman, where it is claimed that for ATI, "the code is the documentation".

If you are backing this reasoning for ATI, what is wrong with this code being the
documentation for Qualcomm?

This point about documentation at least does not seem very credible coming from you,
with your history, especially with respect to the ATI story.

> I've also noticed a trend to just reinvent the whole wheel instead of
> writing a drm/kms driver and having that as the API, again maintainer
> nightmares are made of this.

Heh, in some of these cases, not having looked at this code in detail yet, such code
predates kms, and drm might not have provided what was needed. Not wanting to
completely diminish the responsibility of qualcomm (or the other companies who are
working or are forced to work like this), you might want to think about providing
stable and fitting infrastructure, not just stating that something is how _you_ are
doing it and declaring that the law.

Next to that, the IP heavy part that cannot be released (yet?) might be some blob that
is used on both linux, windows, ximbian, etc. The concept of talking to some os
independent blob through some painful and ever-shifting layer is not that alien even to
you, with your staunch defending of ATIs AtomBIOS over more direct modesetting.

Also, from where i sit, you complaining about people reinventing the wheel does bring
me some bitter amusement.

As a conclusion: With you having sent this mail, guess what the guys at qualcomm, and
most likely imagination technologies and ARM as well (i think we can already discount
nvidia -- they are far too adept at producing solid closed source drivers -- to
desktop users satisfaction too), will do next?

We already squandered the free software desktop (on x86), and part of the
responsibility for that is with the graphics hw support (and the radeon versus radeonhd
story shows nicely how to go about squandering such things). What i see here is that
you clearly want to go down a similar street with the now blossoming ARM market.

Thanks alot,

Luc Verhaegen.

2010-07-02 10:15:38

by Christoph Hellwig

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

Luc, can you please take your corporate bullshit out of this? I can
assume you know Dave personally and should be clearly aware that he's
everything but a corporate drone.

2010-07-02 10:23:32

by Dave Airlie

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

>
> Yes, this a mess indeed.
>
> But i fear that this a mess that cannot be fixed, in its entirety, in a single shot.
>
> Qualcomm making this code available already clearly shows the will and determination of
> some people inside qualcomm to do The Right Thing.
>
> This is Qualcomms first big step on the graphics side, where IP is always amongst the
> heaviest. I am certain that Qualcomm wants to go further, but since Qualcomm most
> likely licenses some parts of their graphics, Qualcomm can only open up those bits that
> they truly own, and then use mainly market/sales-volume driven pressure to get the
> original IP owner to play along.

They own quite a lot of the IP in the 3D core, having bought it from
AMD, you can see the CP packets and PM4 stuff just like in radeon.

>
> You should also take into account who Jordan is. He is one of the guys who worked the
> Geode before AMD decided to drop that, which is when he got hired by Qualcomm. He worked on
> both the graphics drivers and on (then still) LinuxBIOS. I know that redhat has no
> intention of going near coreboot, but in my world one cannot become more free, hardware
> wise, than supporting coreboot. This gives me very good hopes that this is a serious
> attempt by qualcomm to go somewhere, and that this is not some lame attempt to grab
> marketing attention.

I know who Jordan is well from AMD, and I've stated this isn't a
Qualcomm only thing, but companies need to understand that putting
stuff into the kernel is part of a bargain, where you get the benefits
of all the work everyone has done on the kernel and they get the
benefits of being able to do stuff with the code you provide, only
giving us a half arsed interface to the hw and hiding all the good
stuff in userspace isn't accepting this bargain. You can say what you
like about liceneses and legalese but Linus picked the GPL for this
sort of everyone gives what they get.

> Now that you slammed the door on these guys (and on others in the process), what do you
> think the response would be? Where will this get us to in the end?

They'll keep shipping closed stuff, just like they are now. Are you
going to reverse engineer the userspace drivers, so people who care
about open and free software platforms can use these drivers? (or have
you already signed NDAs saying you can't). Why should we maintain a
bunch of kernel code, when they are hiding away all the really useful
stuff that people could improve.

>> b) verifying the sanity of the userspace API.
>> 1. Security: GPUs can do a lot of damage if left at home alone, since
>> mostly you are submitting command streams unverified into the GPU and
>> won't tell us what they mean, there is little way we can work out if
>> the GPU is going to over-write my passwd file to get 5 fps more in
>> quake. Now newer GPUs have at least started having MMUs, but again
>> we've no idea if that is the only way they work without docs or a lot
>> of trust.
>
> This makes me wonder: Why do you even care?
>
> If redhat was working with qualcomm, you would not have taken this stance here at all.

I would, some points
(a) Red Hat is the company I work for.
(b) Red Hat doesn't really care about this stuff at all.
(c) I'm the kernel maintainer for far longer that I worked for Red
Hat, I also work a lot with Intel at Red Hat and I've told them the
same thing, and VIA and now I'm stating it so I don't have to restate
individually to every company again.

> If so, why couldn't you have stated "please guys, have fun with what you are doing, but
> i will not be responsible for it" in a different way.

You don't understand what being a kernel maintainer is do you? at all?

> Now, it is interesting how you now are demanding documentation. When did recent and
> relevant hw documentation happen for ATI? This pretty much died together when the
> ATI<->SuSE relationship died, as the cooperation of SuSE and AMD is how documentation
> was forced out of ATI in the first place, and ATI more and more found ways to get rid
> of this responsibility, or overhead as bridgman would most likely name it.

Wierd I'm still seeing new docs being produced and old docs being
updated, not as fast as I'd like but I understand how many people
there is working on it and I'd rather we also got fixes for all the
current stuff done as well.


> If you are backing this reasoning for ATI, what is wrong with this code being the
> documentation for Qualcomm?

The code doesn't exist, there is no userspace code to be the
documents, if you read what I said, documents would be a good start in
lieu of code, but code is perfectly fine.

> Heh, in some of these cases, not having looked at this code in detail yet, such code
> predates kms, and drm might not have provided what was needed. Not wanting to
> completely diminish the responsibility of qualcomm (or the other companies who are
> working or are forced to work like this), you might want to think about providing
> stable and fitting infrastructure, not just stating that something is how _you_ are
> doing it and declaring that the law.
>
> Next to that, the IP heavy part that cannot be released (yet?) might be some blob that
> is used on both linux, windows, ximbian, etc. The concept of talking to some os
> independent blob through some painful and ever-shifting layer is not that alien even to
> you, with your staunch defending of ATIs AtomBIOS over more direct modesetting.
>
> Also, from where i sit, you complaining about people reinventing the wheel does bring
> me some bitter amusement.
>
> As a conclusion: With you having sent this mail, guess what the guys at qualcomm, and
> most likely imagination technologies and ARM as well (i think we can already discount
> nvidia -- they are far too adept at producing solid closed source drivers -- to
> desktop users satisfaction too), will do next?

Imagintaion technologies seriously? they've never ever taken one step
towards opening anything, I don't think this statement is suddenly
going to jumpstart them.

>
> We already squandered the free software desktop (on x86), and part of the
> responsibility for that is with the graphics hw support (and the radeon versus radeonhd
> story shows nicely how to go about squandering such things). What i see here is that
> you clearly want to go down a similar street with the now blossoming ARM market.
>

Maybe you should disclose what NDAs you currently are under and who
pays your bills, since you accuse me of Red Hat mind-control.

I'm not sure how the ARM market would benefit from having no userspace
3D drivers just like the x86 market, if you actually were normal you'd
realise the ARM people are trying to screw the market just like the
desktop people, to save themselves some money, but maybe working on
closed drivers has twisted you.

Dave.

2010-07-02 11:10:35

by Luc Verhaegen

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 02, 2010 at 08:23:27PM +1000, Dave Airlie wrote:
> >
>
> They own quite a lot of the IP in the 3D core, having bought it from
> AMD, you can see the CP packets and PM4 stuff just like in radeon.

Aha, imageon indeed, cool!

I doubt that you know the conditions of this sale. This might just be about waiting for
Qualcomm to sort things out, or it might be about a licensing issue still. We just
cannot know this.

> I know who Jordan is well from AMD, and I've stated this isn't a
> Qualcomm only thing, but companies need to understand that putting
> stuff into the kernel is part of a bargain, where you get the benefits
> of all the work everyone has done on the kernel and they get the
> benefits of being able to do stuff with the code you provide, only
> giving us a half arsed interface to the hw and hiding all the good
> stuff in userspace isn't accepting this bargain. You can say what you
> like about liceneses and legalese but Linus picked the GPL for this
> sort of everyone gives what they get.

Sure, but you still slammed, and this affected in first order mostly Qualcomm, instead
of stating that you simply do not want to be involved.

> They'll keep shipping closed stuff, just like they are now. Are you
> going to reverse engineer the userspace drivers, so people who care
> about open and free software platforms can use these drivers? (or have
> you already signed NDAs saying you can't). Why should we maintain a
> bunch of kernel code, when they are hiding away all the really useful
> stuff that people could improve.

Maintaining this exact code is not _your_ job, and you should've stated just that.

> I would, some points
> (a) Red Hat is the company I work for.
> (b) Red Hat doesn't really care about this stuff at all.
> (c) I'm the kernel maintainer for far longer that I worked for Red
> Hat, I also work a lot with Intel at Red Hat and I've told them the
> same thing, and VIA and now I'm stating it so I don't have to restate
> individually to every company again.

You will need to restate this every time anyway, unless you somehow manage to get your
rather daunting and loud statement to be the first thing such corporations management
people at linux.com, which is what people type in their browser first.

But maybe you might want to adjust your message.

> You don't understand what being a kernel maintainer is do you? at all?

Heh, i could make a really nasty statement here, but i wont.

> Wierd I'm still seeing new docs being produced and old docs being
> updated, not as fast as I'd like but I understand how many people
> there is working on it and I'd rather we also got fixes for all the
> current stuff done as well.

Hrm, i only see _very_ old docs getting updated, and none produced.

I still have docs which are pretty ready to be shipped (from 2007), under uncertain
legal status (the ati game was fun!), which were never made public. I am sure that
bridgman, gave you these docs too, but under even more shady circumstances.

> The code doesn't exist, there is no userspace code to be the
> documents, if you read what I said, documents would be a good start in
> lieu of code, but code is perfectly fine.

> Imagintaion technologies seriously? they've never ever taken one step
> towards opening anything, I don't think this statement is suddenly
> going to jumpstart them.

> Maybe you should disclose what NDAs you currently are under and who
> pays your bills, since you accuse me of Red Hat mind-control.

Oh, i now work for basysKom, a contractor for, amongst others, nokia, which i'm sure
you knew.

> I'm not sure how the ARM market would benefit from having no userspace
> 3D drivers just like the x86 market, if you actually were normal you'd
> realise the ARM people are trying to screw the market just like the
> desktop people, to save themselves some money, but maybe working on
> closed drivers has twisted you.

Ah, so you did know.

As a free software graphics driver developer there is little option left but to go
straight to the ARM world, at least things have potential to move there still.

Well, unless the efforts there, like the one that triggered your mail, are thwarted
too, like with your mail.

Luc Verhaegen.

2010-07-02 11:12:40

by Luc Verhaegen

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 02, 2010 at 06:15:35AM -0400, Christoph Hellwig wrote:
> Luc, can you please take your corporate bullshit out of this? I can
> assume you know Dave personally and should be clearly aware that he's
> everything but a corporate drone.

Yes, with mails like this he clearly shows that he isn't a corporate drone.

Luc Verhaegen.

2010-07-02 11:26:57

by Dave Airlie

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 2, 2010 at 9:12 PM, Luc Verhaegen <[email protected]> wrote:
> On Fri, Jul 02, 2010 at 06:15:35AM -0400, Christoph Hellwig wrote:
>> Luc, can you please take your corporate bullshit out of this? ?I can
>> assume you know Dave personally and should be clearly aware that he's
>> everything but a corporate drone.
>
> Yes, with mails like this he clearly shows that he isn't a corporate drone.
>

Luc, this isn't phoronix forums, the adults are talking here, run along now.

Dave.

2010-07-02 11:54:00

by Dave Airlie

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

> Sure, but you still slammed, and this affected in first order mostly Qualcomm, instead
> of stating that you simply do not want to be involved.

I have no choice but to be involved, again you seem to misunderstand
what my position is.

>
>> They'll keep shipping closed stuff, just like they are now. Are you
>> going to reverse engineer the userspace drivers, so people who care
>> about open and free software platforms can use these drivers? (or have
>> you already signed NDAs saying you can't). Why should we maintain a
>> bunch of kernel code, when they are hiding away all the really useful
>> stuff that people could improve.
>
> Maintaining this exact code is not _your_ job, and you should've stated just that.

Maintaining kernel graphics drivers is my responsibility, a lot of
days i wish it wasn't, and I'm sure some day it won't be, but until
then I have to provide guidance on how things should work. Again you
should look at what being a kernel maintainer is.

>
> You will need to restate this every time anyway, unless you somehow manage to get your
> rather daunting and loud statement to be the first thing such corporations management
> people at linux.com, which is what people type in their browser first.

I'm sure I will, but now I can just point people at this rather public
statement as opposed to private emails.
>
> Heh, i could make a really nasty statement here, but i wont.

Please do, since you've proved you are clueless about this position entails.

> Hrm, i only see _very_ old docs getting updated, and none produced.
>
> I still have docs which are pretty ready to be shipped (from 2007), under uncertain
> legal status (the ati game was fun!), which were never made public. I am sure that
> bridgman, gave you these docs too, but under even more shady circumstances.

Shady circumstances? you might want to ask your lawyer before making
statements like that on a public mailing list.

>> The code doesn't exist, there is no userspace code to be the
>> documents, if you read what I said, documents would be a good start in
>> lieu of code, but code is perfectly fine.
>
>> Imagintaion technologies seriously? they've never ever taken one step
>> towards opening anything, I don't think this statement is suddenly
>> going to jumpstart them.
>
>> Maybe you should disclose what NDAs you currently are under and who
>> pays your bills, since you accuse me of Red Hat mind-control.
>
> Oh, i now work for basysKom, a contractor for, amongst others, nokia, which i'm sure
> you knew.

So you honestly think if I allow Nokia and/or Intel to push a poulsbo
driver into the kernel the magic fairies will come along and open the
userspace because they've seen the light?

What incentive does letting someone like Qualcomm etc put all their
kernel code upstream, and ignoring the userspace components do you
think it provides to open the userspace component, for once I'm
interested in your opinion since you seem to have the ARM players all
worked out. Previous experience with most companies has shown they'll
do as little as possible to ensure they can ship as many things as
possible, and this is fine, they need real incentives to follow the
open source rules.

>
>> I'm not sure how the ARM market would benefit from having no userspace
>> 3D drivers just like the x86 market, if you actually were normal you'd
>> realise the ARM people are trying to screw the market just like the
>> desktop people, to save themselves some money, but maybe working on
>> closed drivers has twisted you.
>
> Ah, so you did know.
>
> As a free software graphics driver developer there is little option left but to go
> straight to the ARM world, at least things have potential to move there still.

What potential? there are maybe 6 players on the ARM graphics scene

Imagination Technologies SGX, used in TI and poulsbo/mrst(x86) - no
hope of opening userspace
ARM Mali - not sure what is going on there, I;m going to go with no
hope but would be nice to be proved wrong
Qualcomm SnapDragon - imageon by another name, from what I can and
from talking to Jordan on irc, they don't seem to have much incentive
to bother working on an open userspace
Marvell - maybe OLPC can make them open a userspace for millions of
sales, it doesn't seem to have worked with VIA for 10s of thousands of
persumable sales
Samsung - also holding out hope for something maybe, they seemed to
have some interest once, but not sure what happening now.
Nvidia - well we know their position will never change.

So we should have six completely separate stacks shipping in the
kernel not using drm or kms, but all standalone, all with closed
userspace drivers, with no maintainer, thanks that is not a future I'm
interested in, and generally from experience in Linux it isn't
something we've had much luck with before, wireless, networking, sw
raid, etc all have this sort of vendor demands and it took independent
maintainer pushback to achieve some cooperation and get what we have
today.

Dave.

2010-07-02 12:01:25

by C. Bergström

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

Dave Airlie wrote:
> What potential? there are maybe 6 players on the ARM graphics scene
>
> ...
> Nvidia - well we know their position will never change.
>
Never say never. I have every reason to believe that Nvidia would
respond to market demand.

*fingers crossed*

./C

2010-07-02 14:01:51

by Anton Vorontsov

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, Jul 02, 2010 at 01:10:29PM +0200, Luc Verhaegen wrote:
[...]
> > They'll keep shipping closed stuff, just like they are now. Are you
> > going to reverse engineer the userspace drivers, so people who care
> > about open and free software platforms can use these drivers? (or have
> > you already signed NDAs saying you can't). Why should we maintain a
> > bunch of kernel code, when they are hiding away all the really useful
> > stuff that people could improve.
>
> Maintaining this exact code is not _your_ job, [...]

Correct, it's not solely his job, but it's also every kernel
developers' job.

When I change kernel API I have to grep through all the kernel
drivers, sometimes understand how they work, and then make the
change to the whole kernel source tree.

And I would not want to maintain this code, as these drivers
are wasting my time without returning anything back.

It was said many times. Actually, so many times that it started
to become boring to repeat, and the Kernel Driver Statement was
written:

"We, the undersigned Linux kernel developers, consider any
closed-source Linux kernel module or driver to be harmful and
undesirable."

http://www.linuxfoundation.org/collaborate/publications/kernel-driver-statement

While the doc mostly says "kernel code", I truly believe that
there's actually no huge difference between "closed-source
kernel module" and "open source dummy kernel module + userspace
blob".

Both are closed source solutions, and generally useless for the
open source. And, what is worse, the last one is harmful for me
personally.

--
Anton Vorontsov
email: [email protected]
irc://irc.freenode.net/bd2

2010-07-02 14:56:37

by C. Bergström

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

Xavier Bestel wrote:
> On Fri, 2010-07-02 at 19:07 +0700, "C. Bergström" wrote:
>
>> Dave Airlie wrote:
>>
>>> What potential? there are maybe 6 players on the ARM graphics scene
>>>
>>> ...
>>> Nvidia - well we know their position will never change.
>>>
>>>
>> Never say never. I have every reason to believe that Nvidia would
>> respond to market demand.
>>
>
> Could you share those interesting reasons with us ?
>
I thought I stated that the main reason was market demand?

(Some of my own comments not representing any company)
By market I mean revenue... The Linux and FOSS world isn't exactly a
huge market for high end graphics cards and total sales. With the Tesla
series of cards and most HPC clusters behing Linux powered this has the
potential to change things. Maybe I'm too optimistic and biased, but
from my perspective I think there's a change in the winds coming..

2010-07-02 15:19:38

by Xavier Bestel

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

On Fri, 2010-07-02 at 19:07 +0700, "C. Bergström" wrote:
> Dave Airlie wrote:
> > What potential? there are maybe 6 players on the ARM graphics scene
> >
> > ...
> > Nvidia - well we know their position will never change.
> >
> Never say never. I have every reason to believe that Nvidia would
> respond to market demand.

Could you share those interesting reasons with us ?

Xav

2010-07-03 00:53:32

by Ian Romanick

[permalink] [raw]
Subject: Re: Closed source userspace graphics drivers with an open source kernel component

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Luc Verhaegen wrote:

> Since redhat is then not working with qualcomm, why is this then your responsibility?

I find that sentiment surprising from somebody who has actually met Dave. :/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkwuhy0ACgkQX1gOwKyEAw/Q9QCfd0qcveuzyBVOIqW0yerggbLu
EzIAn16kyWeZ+fpVkmirQghbYFLGrAgx
=CeEy
-----END PGP SIGNATURE-----