on January 15th / major change in USB subsystem and GPL_EXPORT_SYMBOL
declaration
Greg Kroah-Hartman added a Patch to kernel 2.6.15-git12, which
substantially changed the USB system.
The module "usb.c" is now a module named "driver.c" which exports its
symbols with EXPORT_SYMBOL_GPL:
-> usb_match_id; usb_register_driver; usb_deregister
Novell added the official kernel 2.6.16 incl. this patch to OSS 10.1 beta.
consequences
Because of the GPL_EXPORT declaration it is no longer possible to build and
run non-GPL loadable drivers for USB devices. We?ve put a lot of energy
into providing the open source community with Linux drivers for nearly all
of our products within the last six years. Today, the customer has the
option to choose Windows or Linux drivers for AVM USB products. AVM is the
market leader in the ISDN controller market with more than 80% market share
in Germany (close to 50% in Europe). Moreover AVM is one of a handful
manufacturers who provide a Linux driver for their WLAN USB devices.
Technically, there are a number of reasons, e.g. service quality and
reliability, to establish kernel mode drivers for communication devices
offering services like Fax G3, analog modem etc. by means of software.
conclusion
If it is no longer possible to have non-GPL USB drivers, we shall have to
drop our Linux support for all AVM USB devices. We would even have to
discontinue the 802.11g++ WLAN USB device driver Linux developement.
This mail is not intended to provoke a discussion of open vs closed source.
The only intention of this mail is to make you aware of the consequences of
such a decision.
Kind Regards
Sven Schmidt
AVM Audiovisuelles Marketing und Computersysteme GmbH
Alt-Moabit 95, 10559 Berlin, Germany
http://www.avm.de
On Fri, 03 Feb 2006 17:24:10 +0100, [email protected] said:
> If it is no longer possible to have non-GPL USB drivers, we shall have to
> drop our Linux support for all AVM USB devices. We would even have to
> discontinue the 802.11g++ WLAN USB device driver Linux developement.
Why is there a problem releasing a GPL'ed USB driver? That would solve
your problem just as well. If you were really ambitious, you could clean
up the code and submit it for inclusion in the mainline tree - thus lowering
your support costs.
Hi!
> consequences
> Because of the GPL_EXPORT declaration it is no longer possible to build and
> run non-GPL loadable drivers for USB devices. We?ve put a lot of energy
> into providing the open source community with Linux drivers for nearly all
Last time I checked, it was easy to write usb drivers in userspace.
Pavel
--
Thanks, Sharp!
2006/2/4, [email protected] <[email protected]>:
> On Fri, 03 Feb 2006 17:24:10 +0100, [email protected] said:
>
> > If it is no longer possible to have non-GPL USB drivers, we shall have to
> > drop our Linux support for all AVM USB devices. We would even have to
> > discontinue the 802.11g++ WLAN USB device driver Linux developement.
>
> Why is there a problem releasing a GPL'ed USB driver? That would solve
...especially compared to other vendors how do have open source WLAN
USB drivers these days?
> your problem just as well. If you were really ambitious, you could clean
> up the code and submit it for inclusion in the mainline tree - thus lowering
> your support costs.
>
...and improve the code quality at the same time.
I had some trouble with the closed source fritz card dsl 2.0 driver
e.g. which suffers from a race on recent kernels (I guess Karsten Keil
informed you meanwhile). This should have been tracked down easier and
earlier (last worked on 2.6.8) if your drivers were part of the kernel
or at least taking part in the community.
I really like your hardware, but I dislike such license models.
> > This mail is not intended to provoke a discussion of open vs closed source.
> > The only intention of this mail is to make you aware of the consequences of
> > such a decision.
I tell you my opinion but I will not accept any discussion???
Well, I guess it's not your own decision that AVM is still stuck on
closed Linux drivers, your management may it's own ideas about it as
well. But who else should try to change this than the "engineer at the
front"? This is how many success stories of companies in the open
source domain took off.
I don't know your market, so I may neglect that you have to protect IP
very strictly from nosy competitors. But what part of this IP really
has to be inside the kernel driver? Can you explain this to the Linux
community? This may help a bit to understand your issues with free
drivers.
Jan
On Fri, Feb 03, 2006 at 05:24:10PM +0100, [email protected] wrote:
> on January 15th / major change in USB subsystem and GPL_EXPORT_SYMBOL
> declaration
> Greg Kroah-Hartman added a Patch to kernel 2.6.15-git12, which
> substantially changed the USB system.
Have you asked Greg why he did this?
Have you asked what the other alternatives are?
You do know about usbfs/libusb that allows you to write USB drivers in
userspace that can go at the full speed of the USB bus? Why not redo
your code to take advantage of this? If you do that, the extra bonus is
that your drivers will also work on the BSDs and possibly Windows with
no changes needed (I think libusb works on Windows...)
> This mail is not intended to provoke a discussion of open vs closed source.
> The only intention of this mail is to make you aware of the consequences of
> such a decision.
I was not aware of your drivers, but now that you have informed me of
them, I am willing to work with you to figure out how to resolve this
issue.
thanks,
greg k-h
>
>I really like your hardware, but I dislike such license models.
>
"Buy Taiwanese..."!
Jan Engelhardt
--
We are pleased to note that the GPL_EXPORT_SYMBOL fix has been withdrawn.
This is particularly important for customers who have been relying on good
driver coverage for ISDN/DSL devices with SUSE distributions over the past
few years. However, as we understand the ongoing discussion, a number of
people are tending towards a position of enforcement of USB GPL drivers
only. We would like to take this opportunity to clarify where we see the
differences between AVM and other devices and the difficulties regarding a
possible move towards user mode.
The user space does not ensure the reliability of time critical analog
services like Fax G3 or analog modem emulations. This quality of service
can only be guaranteed within the kernel space.
Let me explain that issue using the FaxG3-service as an example. Fax G3
(T.30) is not specified as a data protocol with error-free transmission.
Let us assume, there is a system peak demand on the host system, while a
fax is incoming, e.g. because of a parallel access of a higher prioritized
process. Handled in user mode, the user gets broken or fragmented faxes as
a result. Same for the communication with analogue remote stations (modems)
over a digital net (ISDN). You cannot speak about reliable quality of
service anymore. Only the kernel offers low latency and timeline processing
which is required for soft DSP alike processing. This should be OS
independent and from our point of view, Linux should not be inferior to any
other OS.
Of course, other OS also have the concept of shifting usb drivers to user
space, but time critical demands are explicitly excluded. The given
examples gPhoto und rio500 at
http://libusb.sourceforge.net/doc/examples-other.html operate mostly
unidirectionally. Isochronical services within the libusb, especially the
USB driver framework for the user mode are not ready for bidirectional
operation. Even though the current libusb development started integrating
the isochronous transfer support, it still is under construction and it is
unclear if this task can be accomplished at all (see statement from
Johannes Erdfelt at
http://sourceforge.net/mailarchive/forum.php?thread_id=9531397&forum_id=5425
).
In contrast, AVM's driver concept is established for many years now. So the
user mode does not seem to be an alternative for ISDN/DSL communication
devices at the moment. Moreover, a rework of more than 30 devices would
consume a lot of development resources. You will hardly find a similiar
company situation. We are not talking about a 10 to 20kByte mouse driver,
but rather >600kByte of complex work per device. Take a look at the
FRITZ!Card PCI package
atftp://ftp.avm.de/cardware/fritzcrd.pci/linux/suse.93/fcpci-suse93-3.11-07.tar.gz).
As a private corporation our primary focus is market relevance. AVM
invested more than 10 years of work to make analog services like Fax G3 and
analog modem emulation available to users of the digital ISDN network. The
situation is similar for the DSL part of the driver with very complex DSP
algorithms. To anticipate any "open vs. closed source" discussion:Only a
handful of companies worldwide have such know-how. With regard to our
competitive situation, we have to protect our hard-won intellectual
property and therefore cannot open the closed source part of the driver.
Kind Regards
Sven Schmidt
AVM Audiovisuelles Marketing und Computersysteme GmbH
Alt-Moabit 95, 10559 Berlin, Germany
http://www.avm.de
Greg KH
<[email protected]>
An
05.02.2006 21:53 [email protected]
Kopie
[email protected],
[email protected],
[email protected]
Thema
Re: 2.6.16 serious consequences /
GPL_EXPORT_SYMBOL / USB drivers of
major vendor excluded
On Fri, Feb 03, 2006 at 05:24:10PM +0100, [email protected] wrote:
> on January 15th / major change in USB subsystem and GPL_EXPORT_SYMBOL
> declaration
> Greg Kroah-Hartman added a Patch to kernel 2.6.15-git12, which
> substantially changed the USB system.
Have you asked Greg why he did this?
Have you asked what the other alternatives are?
You do know about usbfs/libusb that allows you to write USB drivers in
userspace that can go at the full speed of the USB bus? Why not redo
your code to take advantage of this? If you do that, the extra bonus is
that your drivers will also work on the BSDs and possibly Windows with
no changes needed (I think libusb works on Windows...)
> This mail is not intended to provoke a discussion of open vs closed
source.
> The only intention of this mail is to make you aware of the consequences
of
> such a decision.
I was not aware of your drivers, but now that you have informed me of
them, I am willing to work with you to figure out how to resolve this
issue.
thanks,
greg k-h
On Thu, 2006-02-16 at 15:24 +0100, [email protected] wrote:
> To anticipate any "open vs. closed source" discussion:Only a
> handful of companies worldwide have such know-how. With regard to our
> competitive situation, we have to protect our hard-won intellectual
> property and therefore cannot open the closed source part of the
> driver.
where in the license of the kernel does it say that this is a valid
exception to the license ?
You're writing drivers with linux in mind. You're writing them
explicitly for linux, using Linux code. And you distribute the result.
If your company really wants to be a good linux player, be a good linux
player rather than contributing to the biggest threat to linux right
now...
On Thu, Feb 16, 2006 at 03:24:44PM +0100, Sven Schmidt wrote:
> The user space does not ensure the reliability of time critical analog
> services like Fax G3 or analog modem emulations. This quality of service
> can only be guaranteed within the kernel space.
Huh? You are offering your fax emulation at
ftp://ftp.avm.de/fritz.box/tools/fax4box/ for Windows to communicate over
Ethernet with the Fritz!Box. You don't tell me that communicating over
Ethernet is more reliable than implementing it in user space? Or is my
understanding about how this tool is working completely wrong?
> As a private corporation our primary focus is market relevance. AVM
> invested more than 10 years of work to make analog services like Fax G3 and
> analog modem emulation available to users of the digital ISDN network. The
> situation is similar for the DSL part of the driver with very complex DSP
> algorithms. To anticipate any "open vs. closed source" discussion:Only a
> handful of companies worldwide have such know-how. With regard to our
> competitive situation, we have to protect our hard-won intellectual
> property and therefore cannot open the closed source part of the driver.
It is perfectly understandable that you don't want to offer this special
know-how in public. But isn't your implementation modular enough to open
source the drivers _without_ support for these analog services as a first
step? This driver could then be shipped without problems enabling your users
to download a more advanced closed source module from your web site if desired
and you don't care about the license issues.
Robert
--
Robert Schiele Tel.: +49-621-181-2214
Dipl.-Wirtsch.informatiker mailto:[email protected]
"Quidquid latine dictum sit, altum sonatur."
[email protected] wrote:
>
> The user space does not ensure the reliability of time critical analog
> services like Fax G3 or analog modem emulations. This quality of service
> can only be guaranteed within the kernel space.
> [...] To anticipate any "open vs. closed source" discussion: Only a
> handful of companies worldwide have such know-how. With regard to our
> competitive situation, we have to protect our hard-won intellectual
> property and therefore cannot open the closed source part of the driver.
Thanks for clarifying the situation.
Since your intellectual property is in the DSP algorithms, are there
any obstacles opensourcing the parts of the ISDN drivers which only
handle normal ISDN without fax/modem emulation? This would make every
distribution out there support your devices without any additional work
on your side.
You can still offer your full-featured drivers as usual.
Regards,
Carl-Daniel Hailfinger
On Thu, Feb 16, 2006 at 11:09:21PM +0100, Carl-Daniel Hailfinger wrote:
> [email protected] wrote:
> >
> > The user space does not ensure the reliability of time critical analog
> > services like Fax G3 or analog modem emulations. This quality of service
> > can only be guaranteed within the kernel space.
> > [...] To anticipate any "open vs. closed source" discussion: Only a
> > handful of companies worldwide have such know-how. With regard to our
> > competitive situation, we have to protect our hard-won intellectual
> > property and therefore cannot open the closed source part of the driver.
>
> Thanks for clarifying the situation.
>
> Since your intellectual property is in the DSP algorithms, are there
> any obstacles opensourcing the parts of the ISDN drivers which only
> handle normal ISDN without fax/modem emulation? This would make every
> distribution out there support your devices without any additional work
> on your side.
> You can still offer your full-featured drivers as usual.
>
That is not the point here, some needed kernel symbols will change
to GPL_ONLY in some time, so you cannot build a binary only driver any
longer.
--
Karsten Keil
SuSE Labs
ISDN development
First off, thank you for replying to my message. Hopefully we can all
work together to find a proper solution for everyone here.
On Thu, Feb 16, 2006 at 03:24:44PM +0100, [email protected] wrote:
> The user space does not ensure the reliability of time critical analog
> services like Fax G3 or analog modem emulations. This quality of service
> can only be guaranteed within the kernel space.
What about a mix of userspace/kernelspace?
> Let me explain that issue using the FaxG3-service as an example. Fax G3
> (T.30) is not specified as a data protocol with error-free transmission.
> Let us assume, there is a system peak demand on the host system, while a
> fax is incoming, e.g. because of a parallel access of a higher prioritized
> process. Handled in user mode, the user gets broken or fragmented faxes as
> a result. Same for the communication with analogue remote stations (modems)
> over a digital net (ISDN). You cannot speak about reliable quality of
> service anymore. Only the kernel offers low latency and timeline processing
> which is required for soft DSP alike processing. This should be OS
> independent and from our point of view, Linux should not be inferior to any
> other OS.
I don't want it to be "inferior" in any way either. And I don't think
it is. Right now, we have the highest throughput of any operating
system for USB bandwidth. And that is measured by a userspace program
using usbfs directly, no kernel driver needed. We can easily keep up
with a full datastream on the USB bus with no problems of dropped frames
or other issues.
It's also been proven recently that Linux can, with a mix of userland
libraries and tiny kernel drivers, fill a 10gbit ethernet pipe, with the
only bottleneck being the CPU to RAM bus. So claims that putting stuff
in userland will cause quality of service issues is pretty unlikely.
Again, how about a mix of a kernel driver that handles the buffering of
the data, and any proper acknowledgment needed, and userspace handling
the heavy lifting of decoding the fax/image data?
For an example, the ldusb driver handles high speed data acquisition
devices, and buffers the data until userspace can flush the buffers.
This makes for a very tiny and simple kernel driver, and all of the
"secret" logic can be done in userspace.
> Of course, other OS also have the concept of shifting usb drivers to user
> space, but time critical demands are explicitly excluded. The given
> examples gPhoto und rio500 at
> http://libusb.sourceforge.net/doc/examples-other.html operate mostly
> unidirectionally. Isochronical services within the libusb, especially the
> USB driver framework for the user mode are not ready for bidirectional
> operation. Even though the current libusb development started integrating
> the isochronous transfer support, it still is under construction and it is
> unclear if this task can be accomplished at all (see statement from
> Johannes Erdfelt at
> http://sourceforge.net/mailarchive/forum.php?thread_id=9531397&forum_id=5425
> ).
I don't see Johannes saying that things aren't going to be accomplished
in that post. Perhaps you meant to point to some other message?
Anyway, libusb is a nice, friendly wrapper around usbfs. But if you
really want to get speed and full control over your device, just use
usbfs directly. That's what all of the applications that I know of that
do complex, high speed things do.
And yes, usbfs is showing it's age. It's been around since 2.3 days and
has not really been modified since then. Numerous people have discussed
creating a usbfs2, in the format that gadgetfs is in (async io for
endpoints), and any help in designing and implementing it, so that it
meets your needs would be greatly appreciated. I've already started the
basic framework for it, if you are interested. I'll post it on the
linux-usb-devel mailing list next week (early looks can be found at:
http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/patches/usb/usbfs2.patch
)
> In contrast, AVM's driver concept is established for many years now. So the
> user mode does not seem to be an alternative for ISDN/DSL communication
> devices at the moment.
I know of at least 2 diffeent ISDN devices that work just fine using
usbfs. So for you to say it's not possible is not a fair statement.
> Moreover, a rework of more than 30 devices would consume a lot of
> development resources. You will hardly find a similiar company
> situation. We are not talking about a 10 to 20kByte mouse driver, but
> rather >600kByte of complex work per device. Take a look at the
> FRITZ!Card PCI package
> atftp://ftp.avm.de/cardware/fritzcrd.pci/linux/suse.93/fcpci-suse93-3.11-07.tar.gz).
That seems _very_ large for a Linux kernel driver. None of the existing
Linux USB drivers even come close to that size. I'm sure we can find
some stuff in there to push out to userspace based on the size alone :)
> As a private corporation our primary focus is market relevance. AVM
> invested more than 10 years of work to make analog services like Fax G3 and
> analog modem emulation available to users of the digital ISDN network. The
> situation is similar for the DSL part of the driver with very complex DSP
> algorithms. To anticipate any "open vs. closed source" discussion:Only a
> handful of companies worldwide have such know-how. With regard to our
> competitive situation, we have to protect our hard-won intellectual
> property and therefore cannot open the closed source part of the driver.
I'm glad that you focused on the technical aspects of the issue, and I
would like to keep the discussion there.
However I do just have one final thing to say about this. You mention
that you have to protect your "hard-won intellectual property". I fully
understand this, and respect your wishes. However you also need to
respect our (the Linux Kernel developers) intellectual property rights.
We released our code under the GPL, which states in very specific form,
exactly what your rights are when using this code. When you link other
code into our body of code, you are obligated by the license, to also
release your code under this same license (when you distribute it).
As an example, if a Linux kernel developer were to take your code, and
violate your license and drop it into a GPL licensed body of work
without your permission, you would rightfully be incensed, and work to
stop this from happening. Perhaps you would take legal action, along
with public notice of the act. And you would be fully within your right
to do so.
So, when you take the Linux kernel code, and link with it (or even build
with it's header files and inline functions), and not abide by our well
documented licenses, you can understand why a number of us would be
upset and work to address this issue. Some of the kernel developers are
employing legal means (cease-and-desist letters, lawsuits, etc.) while
others are working for a technological solution to this legal issue
(EXPORT_SYMBOL_GPL, etc.)
I've had the misfortune of discussing this issue with many different IP
lawyers over the years, and all of them are unanimous in saying that
they do not feel there is any legal way for anyone to distribute a
closed source Linux kernel module at all these days. That is based on a
deep understanding of the GPL, IP law in general, and the statements of
numerous Linux kernel developers over the past few years.
It seems that Linux distributions also realize this issue, and a number
of them now refuse to ship non-GPL kernel modules, because of this.
I say all of this, not to upset you, but to try to give you an idea of
why people are so concerned when they are confronted with closed source
Linux kernel modules. You are violating our license, while at the same
time asking that the world abide by your license. The irony is deep...
Anyway, in the end, it's up to you to decide if you have a business case
for supporting Linux or not. No one is forcing you to do so. If you do
not want to create any Linux drivers for your hardware, that is your
right, and fine with us (some of your customers might be upset, but
that's your decision...)
But if you do wish to support Linux, then you must abide by the license
that the kernel is released under. To not do so is a blatant disregard
for the intent and wishes of the developers.
On a personal note, I am very glad to continue this discussion on a
technical level, and work together with you on how to best solve the
usbfs userspace / kernelspace issue for your products so that you can
create a solution that is acceptable both for your customers, and for
the Linux kernel community.
thanks,
greg k-h
On Thu, 2006-02-16 at 15:24 +0100, [email protected] wrote:
> Only the kernel offers low latency and timeline processing
> which is required for soft DSP alike processing.
Sorry, but this set off my hyperbole detector. Ever hear of a VST
plugin? On Linux, people do realtime DSP stuff with JACK in user space
all the time that (no offense) makes a soft modem look like a toy in
comparison, like process 32 channels of 24 bit audio at 96KHz through a
reverb, amp modelers, equalizers, pitch correction, etc.
http://jackit.sourceforge.net/apps/
http://www.ardour.org/
On a recent kernel you don't even need to be root to get reliable RT
scheduling ;-)
Lee
On 15:24 Thu 16 Feb , [email protected] wrote:
> We are pleased to note that the GPL_EXPORT_SYMBOL fix has been withdrawn.
> This is particularly important for customers who have been relying on good
> driver coverage for ISDN/DSL devices with SUSE distributions over the past
> few years. However, as we understand the ongoing discussion, a number of
> people are tending towards a position of enforcement of USB GPL drivers
> only. We would like to take this opportunity to clarify where we see the
> differences between AVM and other devices and the difficulties regarding a
> possible move towards user mode.
>
> The user space does not ensure the reliability of time critical analog
> services like Fax G3 or analog modem emulations. This quality of service
> can only be guaranteed within the kernel space.
Soft modems may work pretty well in userspace - slmodem is example.
Real-time requirement for V.34 is 40ms response time and only once during
the session when echo canceller parameters are negotiatiated (so you may
decrease "buffer size" before and increase after - there are enouph
silence places for such manipulations). Fax itself does not require any
"realtime" AFAIK, other place is almost unused today V.32 - 26ms, also
for echo canceller setup.
Regards,
Sasha.
FYI The lkml FAQ paints a different picture:
Note that Linus has stated that existing symbols will not be switched to
GPL-only. Developers of proprietary modules for Linux need not fear.
Furthermore, it is quite unlikely that Linus will look favourably upon
the introduction of new core driver APIs which are restricted to
GPL-only modules. This would not be in the best interests of Linux.
Linus has forwarded me a message he sent to someone else to clarify his
views.
Anton
On Sun, Feb 19, 2006 at 02:09:04PM +1100, Anton Blanchard wrote:
>
> FYI The lkml FAQ paints a different picture:
>
> Note that Linus has stated that existing symbols will not be switched to
> GPL-only. Developers of proprietary modules for Linux need not fear.
> Furthermore, it is quite unlikely that Linus will look favourably upon
> the introduction of new core driver APIs which are restricted to
> GPL-only modules. This would not be in the best interests of Linux.
> Linus has forwarded me a message he sent to someone else to clarify his
> views.
I suggest that this entry be updated with all of the new information
that has happened in the 8 years or so since that was written :)
thanks,
greg k-h
On Fri, 17 Feb 2006 15:00:04 -0800
Greg KH <[email protected]> wrote:
> On Thu, Feb 16, 2006 at 03:24:44PM +0100, [email protected] wrote:
> > As a private corporation our primary focus is market relevance. AVM
> > invested more than 10 years of work to make analog services like Fax G3 and
> > analog modem emulation available to users of the digital ISDN network. The
> > situation is similar for the DSL part of the driver with very complex DSP
> > algorithms. To anticipate any "open vs. closed source" discussion:Only a
> > handful of companies worldwide have such know-how. With regard to our
> > competitive situation, we have to protect our hard-won intellectual
> > property and therefore cannot open the closed source part of the driver.
I am fine with that. I prefer stable GPL-drivers anyway, no need to use
yours...
Regards,
Stephan
AVM cares a shit about the copyrights of us kernel copyright holders.
Your arm-based dsl modems or routers use linux in complete violations of
the gpl terms we allow everyone to use our code on, and you distribute
binary drivers that can hardly be argued not to be derived work of the
linux kernel. You you please shut the fuck up and sodd off from this
list? thanks.
On Sun, 19 Feb 2006, Sasha Khapyorsky wrote:
> On 15:24 Thu 16 Feb , [email protected] wrote:
> > We are pleased to note that the GPL_EXPORT_SYMBOL fix has been withdrawn.
> > This is particularly important for customers who have been relying on good
> > driver coverage for ISDN/DSL devices with SUSE distributions over the past
> > few years. However, as we understand the ongoing discussion, a number of
> > people are tending towards a position of enforcement of USB GPL drivers
> > only. We would like to take this opportunity to clarify where we see the
> > differences between AVM and other devices and the difficulties regarding a
> > possible move towards user mode.
> >
> > The user space does not ensure the reliability of time critical analog
> > services like Fax G3 or analog modem emulations. This quality of service
> > can only be guaranteed within the kernel space.
>
> Soft modems may work pretty well in userspace - slmodem is example.
>
> Real-time requirement for V.34 is 40ms response time and only once during
> the session when echo canceller parameters are negotiatiated (so you may
> decrease "buffer size" before and increase after - there are enouph
> silence places for such manipulations). Fax itself does not require any
> "realtime" AFAIK, other place is almost unused today V.32 - 26ms, also
> for echo canceller setup.
>
Disclaimer: This is in no way a push to get the -rt patch into mainline
at the moment. The patch is still young and is not ready yet.
but...
I would be really interested in knowing how much better a user side driver
can work with the -rt patch implemented. I bet it can do rather well.
-- Steve
On 12:02 Sun 19 Feb , Steven Rostedt wrote:
>
> > > differences between AVM and other devices and the difficulties regarding a
> > > possible move towards user mode.
> > >
> > > The user space does not ensure the reliability of time critical analog
> > > services like Fax G3 or analog modem emulations. This quality of service
> > > can only be guaranteed within the kernel space.
> >
> > Soft modems may work pretty well in userspace - slmodem is example.
> >
> > Real-time requirement for V.34 is 40ms response time and only once during
> > the session when echo canceller parameters are negotiatiated (so you may
> > decrease "buffer size" before and increase after - there are enouph
> > silence places for such manipulations). Fax itself does not require any
> > "realtime" AFAIK, other place is almost unused today V.32 - 26ms, also
> > for echo canceller setup.
> >
>
> Disclaimer: This is in no way a push to get the -rt patch into mainline
> at the moment. The patch is still young and is not ready yet.
Hmm, it was not about -rt patch - 40 millisecond response time is
realistic for userspace even without -rt patch. There is no guarantee,
but in practice this may work very well and keep low percents of failures
(much lower than permitted by standards).
Sasha.
On Mon, 20 Feb 2006, Sasha Khapyorsky wrote:
> On 12:02 Sun 19 Feb , Steven Rostedt wrote:
> >
> > Disclaimer: This is in no way a push to get the -rt patch into mainline
> > at the moment. The patch is still young and is not ready yet.
>
> Hmm, it was not about -rt patch - 40 millisecond response time is
> realistic for userspace even without -rt patch. There is no guarantee,
> but in practice this may work very well and keep low percents of failures
> (much lower than permitted by standards).
>
No the thread was not about the -rt patch, but I just wanted to spread a
little light that it exists. In fact, besides the current problem with
the SMP scheduler balancing latency (that exists also in mainline) you
should be able to get at least 100us response time on a modest machine.
Even when the IDE wants to hog the CPU.
-- Steve
Your focus on the technical aspects shows us that there is a common
understanding, so we certainly appreciate your input.
The described userland/kernel mix scenarios may work in uninterrupted
environments, but non-stop quality of service "at all times and situations"
is only truly feasible in kernel space. "with the only bottleneck being the
CPU to RAM bus." is exactly the main argument against a mixed kernel/user
space driver architecture. We know from our everyday business, Linux is
often used in slow CPU environments like PIII or similar. Thus latency
times are even more critical within these environments.
Even though people might do realtime DSP things in user space with Linux
and soft modems might work pretty well in userspace, in the case of Fax G3
an extremely short latency is required. The user space lacks the required
reliability and interoperability. Latency times of 4/10/20 or 40
milliseconds are one aspect, but QoS is the overall essence. Within the
kernel space there is no need for unnecessary mode switches or data copy
procedures. Compared to other operating systems, such as Mac OS, BeOS,
Windows etc., Linux is walking a solitary path with the "user mode only"
shift. One gets the impression, that legal concerns are leading Linux to a
technically suboptimal/isolated solution.
And referring to the fax4box tool, missing QoS is the reason why we
continue to not provide support for this tool.
Kind Regards
Sven Schmidt
AVM Audiovisuelles Marketing und Computersysteme GmbH
Alt-Moabit 95, 10559 Berlin, Germany
http://www.avm.de
Greg KH
<[email protected]>
Gesendet von: An
linux-kernel-owne [email protected]
[email protected] Kopie
[email protected], [email protected],
[email protected],
18.02.2006 00:00 [email protected],
[email protected]
Thema
Re: 2.6.16 serious consequences /
GPL_EXPORT_SYMBOL / USB drivers of
major vendor excluded
First off, thank you for replying to my message. Hopefully we can all
work together to find a proper solution for everyone here.
On Thu, Feb 16, 2006 at 03:24:44PM +0100, [email protected] wrote:
> The user space does not ensure the reliability of time critical analog
> services like Fax G3 or analog modem emulations. This quality of service
> can only be guaranteed within the kernel space.
What about a mix of userspace/kernelspace?
> Let me explain that issue using the FaxG3-service as an example. Fax G3
> (T.30) is not specified as a data protocol with error-free transmission.
> Let us assume, there is a system peak demand on the host system, while a
> fax is incoming, e.g. because of a parallel access of a higher
prioritized
> process. Handled in user mode, the user gets broken or fragmented faxes
as
> a result. Same for the communication with analogue remote stations
(modems)
> over a digital net (ISDN). You cannot speak about reliable quality of
> service anymore. Only the kernel offers low latency and timeline
processing
> which is required for soft DSP alike processing. This should be OS
> independent and from our point of view, Linux should not be inferior to
any
> other OS.
I don't want it to be "inferior" in any way either. And I don't think
it is. Right now, we have the highest throughput of any operating
system for USB bandwidth. And that is measured by a userspace program
using usbfs directly, no kernel driver needed. We can easily keep up
with a full datastream on the USB bus with no problems of dropped frames
or other issues.
It's also been proven recently that Linux can, with a mix of userland
libraries and tiny kernel drivers, fill a 10gbit ethernet pipe, with the
only bottleneck being the CPU to RAM bus. So claims that putting stuff
in userland will cause quality of service issues is pretty unlikely.
Again, how about a mix of a kernel driver that handles the buffering of
the data, and any proper acknowledgment needed, and userspace handling
the heavy lifting of decoding the fax/image data?
For an example, the ldusb driver handles high speed data acquisition
devices, and buffers the data until userspace can flush the buffers.
This makes for a very tiny and simple kernel driver, and all of the
"secret" logic can be done in userspace.
> Of course, other OS also have the concept of shifting usb drivers to user
> space, but time critical demands are explicitly excluded. The given
> examples gPhoto und rio500 at
> http://libusb.sourceforge.net/doc/examples-other.html operate mostly
> unidirectionally. Isochronical services within the libusb, especially the
> USB driver framework for the user mode are not ready for bidirectional
> operation. Even though the current libusb development started integrating
> the isochronous transfer support, it still is under construction and it
is
> unclear if this task can be accomplished at all (see statement from
> Johannes Erdfelt at
>
http://sourceforge.net/mailarchive/forum.php?thread_id=9531397&forum_id=5425
> ).
I don't see Johannes saying that things aren't going to be accomplished
in that post. Perhaps you meant to point to some other message?
Anyway, libusb is a nice, friendly wrapper around usbfs. But if you
really want to get speed and full control over your device, just use
usbfs directly. That's what all of the applications that I know of that
do complex, high speed things do.
And yes, usbfs is showing it's age. It's been around since 2.3 days and
has not really been modified since then. Numerous people have discussed
creating a usbfs2, in the format that gadgetfs is in (async io for
endpoints), and any help in designing and implementing it, so that it
meets your needs would be greatly appreciated. I've already started the
basic framework for it, if you are interested. I'll post it on the
linux-usb-devel mailing list next week (early looks can be found at:
http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6
/patches/usb/usbfs2.patch
)
> In contrast, AVM's driver concept is established for many years now. So
the
> user mode does not seem to be an alternative for ISDN/DSL communication
> devices at the moment.
I know of at least 2 diffeent ISDN devices that work just fine using
usbfs. So for you to say it's not possible is not a fair statement.
> Moreover, a rework of more than 30 devices would consume a lot of
> development resources. You will hardly find a similiar company
> situation. We are not talking about a 10 to 20kByte mouse driver, but
> rather >600kByte of complex work per device. Take a look at the
> FRITZ!Card PCI package
>
atftp://ftp.avm.de/cardware/fritzcrd.pci/linux/suse.93/fcpci-suse93-3.11-07.tar.gz).
That seems _very_ large for a Linux kernel driver. None of the existing
Linux USB drivers even come close to that size. I'm sure we can find
some stuff in there to push out to userspace based on the size alone :)
> As a private corporation our primary focus is market relevance. AVM
> invested more than 10 years of work to make analog services like Fax G3
and
> analog modem emulation available to users of the digital ISDN network.
The
> situation is similar for the DSL part of the driver with very complex DSP
> algorithms. To anticipate any "open vs. closed source" discussion:Only a
> handful of companies worldwide have such know-how. With regard to our
> competitive situation, we have to protect our hard-won intellectual
> property and therefore cannot open the closed source part of the driver.
I'm glad that you focused on the technical aspects of the issue, and I
would like to keep the discussion there.
However I do just have one final thing to say about this. You mention
that you have to protect your "hard-won intellectual property". I fully
understand this, and respect your wishes. However you also need to
respect our (the Linux Kernel developers) intellectual property rights.
We released our code under the GPL, which states in very specific form,
exactly what your rights are when using this code. When you link other
code into our body of code, you are obligated by the license, to also
release your code under this same license (when you distribute it).
As an example, if a Linux kernel developer were to take your code, and
violate your license and drop it into a GPL licensed body of work
without your permission, you would rightfully be incensed, and work to
stop this from happening. Perhaps you would take legal action, along
with public notice of the act. And you would be fully within your right
to do so.
So, when you take the Linux kernel code, and link with it (or even build
with it's header files and inline functions), and not abide by our well
documented licenses, you can understand why a number of us would be
upset and work to address this issue. Some of the kernel developers are
employing legal means (cease-and-desist letters, lawsuits, etc.) while
others are working for a technological solution to this legal issue
(EXPORT_SYMBOL_GPL, etc.)
I've had the misfortune of discussing this issue with many different IP
lawyers over the years, and all of them are unanimous in saying that
they do not feel there is any legal way for anyone to distribute a
closed source Linux kernel module at all these days. That is based on a
deep understanding of the GPL, IP law in general, and the statements of
numerous Linux kernel developers over the past few years.
It seems that Linux distributions also realize this issue, and a number
of them now refuse to ship non-GPL kernel modules, because of this.
I say all of this, not to upset you, but to try to give you an idea of
why people are so concerned when they are confronted with closed source
Linux kernel modules. You are violating our license, while at the same
time asking that the world abide by your license. The irony is deep...
Anyway, in the end, it's up to you to decide if you have a business case
for supporting Linux or not. No one is forcing you to do so. If you do
not want to create any Linux drivers for your hardware, that is your
right, and fine with us (some of your customers might be upset, but
that's your decision...)
But if you do wish to support Linux, then you must abide by the license
that the kernel is released under. To not do so is a blatant disregard
for the intent and wishes of the developers.
On a personal note, I am very glad to continue this discussion on a
technical level, and work together with you on how to best solve the
usbfs userspace / kernelspace issue for your products so that you can
create a solution that is acceptable both for your customers, and for
the Linux kernel community.
thanks,
greg k-h
On Mon, Mar 06, 2006 at 01:47:43PM +0100, [email protected] wrote:
> Your focus on the technical aspects shows us that there is a common
> understanding, so we certainly appreciate your input.
>
> The described userland/kernel mix scenarios may work in uninterrupted
> environments, but non-stop quality of service "at all times and situations"
> is only truly feasible in kernel space. "with the only bottleneck being the
> CPU to RAM bus." is exactly the main argument against a mixed kernel/user
> space driver architecture. We know from our everyday business, Linux is
> often used in slow CPU environments like PIII or similar. Thus latency
> times are even more critical within these environments.
I agree. But have you measured such latency on the 2.6 kernel recently?
On old hardware? Is it still unacceptable for you? If so, what is
required?
> Even though people might do realtime DSP things in user space with Linux
> and soft modems might work pretty well in userspace, in the case of Fax G3
> an extremely short latency is required. The user space lacks the required
> reliability and interoperability. Latency times of 4/10/20 or 40
> milliseconds are one aspect, but QoS is the overall essence. Within the
> kernel space there is no need for unnecessary mode switches or data copy
> procedures.
I understand, that is why I suggested a mix of a kernel driver to handle
the stuff that has latency issues, and userspace to handle the rest.
> Compared to other operating systems, such as Mac OS, BeOS,
> Windows etc., Linux is walking a solitary path with the "user mode only"
> shift. One gets the impression, that legal concerns are leading Linux to a
> technically suboptimal/isolated solution.
No, right now, Linux has the best latency numbers _by far_ than any
other operating system, so we can move stuff to userspace.
And again, it's your legal issues that are forcing you that way, if you
change that, putting everything in the kernel would be fine :)
So, in conclusion, is there anything that we can help you out with in
regards to you feeling that userspace can not handle your needs? Do you
have numbers showing that putting a small portion of the USB urb handler
logic in the kernel and the rest in userspace will not work for you?
Example code showing long delay periods that we can help fix?
Anything else we can do?
thanks,
greg k-h
Greg KH wrote:
> On Mon, Mar 06, 2006 at 01:47:43PM +0100, [email protected] wrote:
>
>>Compared to other operating systems, such as Mac OS, BeOS,
>>Windows etc., Linux is walking a solitary path with the "user mode only"
>>shift. One gets the impression, that legal concerns are leading Linux to a
>>technically suboptimal/isolated solution.
>
> No, right now, Linux has the best latency numbers _by far_ than any
> other operating system, so we can move stuff to userspace.
>
> And again, it's your legal issues that are forcing you that way, if you
> change that, putting everything in the kernel would be fine :)
(Since this came to me via the libusb list, and we've kind of
gone past libusb-specific-related discussion, I thought I'd
add another question to the thread).
What's the rationale behind the dichotomy between userspace
and kernel licensing models?
mike
On Mon, Mar 06, 2006 at 01:09:52PM -0800, Michael Bender wrote:
> (Since this came to me via the libusb list, and we've kind of
> gone past libusb-specific-related discussion, I thought I'd
> add another question to the thread).
>
> What's the rationale behind the dichotomy between userspace
> and kernel licensing models?
Hm, how about starting a new thread for this? As it's way outside the
scope of this one, _and_ it's pretty much off-topic for all of these
lists :)
As for the rationale, see the lkml archives for details.
thanks,
greg k-h
On Monday 06 March 2006 14:47, [email protected] wrote:
> Even though people might do realtime DSP things in user space with Linux
> and soft modems might work pretty well in userspace, in the case of Fax G3
> an extremely short latency is required.
So basically we have to choose between:
1. keeping a stable open source kernel and sticking to the principles that got
Linux where it is now
and
2. Fax G3
Umm...
On Tue, 2006-03-07 at 09:42 +0200, Silviu Marin-Caea wrote:
> On Monday 06 March 2006 14:47, [email protected] wrote:
>
> > Even though people might do realtime DSP things in user space with Linux
> > and soft modems might work pretty well in userspace, in the case of Fax G3
> > an extremely short latency is required.
>
> So basically we have to choose between:
>
> 1. keeping a stable open source kernel and sticking to the principles that got
> Linux where it is now
>
> and
>
> 2. Fax G3
>
> Umm...
Extremely short, consistent latency is also required to use a Linux box
as a live audio effects processor and thousands of people do that. It
works extremely well, is used by numerous professionals, and no one has
ever seriously proposed moving it to the kernel. The POSIX realtime
APIs were designed for exactly this kind of application.
If they are doing serious realtime DSP then they should get better
results in userspace anyway, because they get to use the floating point
unit which isn't allowed in the kernel.
I suspect you last tried it in the 2.4 or early 2.6 era when patching
the kernel was required to get decent latency.
Lee
On Tue, 07 Mar 2006, Lee Revell wrote:
> If they are doing serious realtime DSP then they should get better
> results in userspace anyway, because they get to use the floating point
> unit which isn't allowed in the kernel.
It's not as though every algorithm needed float just because it said DSP
(some of those are actually fixed-point or something like that) at a time.
There are lots of algorithms to avoid exactly that, because it costs
performance big time. Whenever you can have integer and/or
reduce/approximate multiplication by shift+add, people will use it if
performance is of paramount importance. And such often is the difference
between having a real-time DRM software radio or not, to name just one
example I've seen in my vicinity.
In a color-space conversion tool (CCITT YUV to RGB) on P-II or P-III
(don't recall) emulating fixed point by using integers and then shifting
appropriately gave a speedup of well more than three.
There must surely be better and reasons than the FPU - licenses had
already been mentioned.
--
Matthias Andree
On Wed, 2006-03-08 at 00:37 +0100, Matthias Andree wrote:
> On Tue, 07 Mar 2006, Lee Revell wrote:
>
> > If they are doing serious realtime DSP then they should get better
> > results in userspace anyway, because they get to use the floating point
> > unit which isn't allowed in the kernel.
>
> It's not as though every algorithm needed float just because it said DSP
> (some of those are actually fixed-point or something like that) at a time.
I didn't mean to imply that, I was just pointing out it's another
feature available in userspace that can't be used in the kernel. Audio
stuff like the AC3 encoder/decoders I've seen in Windows drivers use
floating point instructions for example.
Lee
Lee Revell <[email protected]> writes:
> I didn't mean to imply that, I was just pointing out it's another
> feature available in userspace that can't be used in the kernel. Audio
> stuff like the AC3 encoder/decoders I've seen in Windows drivers use
> floating point instructions for example.
It's also a lot easier to use Altivec/SSE/whatever instructions in
userspace, if you've got 'em...
-Doug