2008-11-07 11:15:53

by Henrik Rydberg

[permalink] [raw]
Subject: [PATCH] input: Add a detailed multi-touch finger data report protocol

In order to utilize the full power of the new multi-touch devices, a
way to report detailed finger data to user space is needed. This patch
adds a multi-touch (MT) protocol which allows drivers to report details
for an arbitrary number of fingers.

BTN_MT_REPORT_PACKET

The driver presses this button to indicate the start of a packet of
finger data. The button is released after the whole packet has been
reported.

BTN_MT_REPORT_FINGER

The driver presses this button to indicate the start of a new
finger. The button is released when the finger has been reported.

In order to stay compatible with existing applications, the data
reported in a finger packet must not be recognized as single-touch
events. Therefore, a specific set of ABS_MT events is used instead.

Signed-off-by: Henrik Rydberg <[email protected]>
---
include/linux/input.h | 12 ++++++++++++
1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/include/linux/input.h b/include/linux/input.h
index b86fb55..c82dd42 100644
--- a/include/linux/input.h
+++ b/include/linux/input.h
@@ -590,6 +590,9 @@ struct input_absinfo {
#define KEY_NUMERIC_STAR 0x20a
#define KEY_NUMERIC_POUND 0x20b

+#define BTN_MT_REPORT_PACKET 0x210 /* report multitouch packet data */
+#define BTN_MT_REPORT_FINGER 0x211 /* report multitouch finger data */
+
/* We avoid low common keys in module aliases so they don't get huge. */
#define KEY_MIN_INTERESTING KEY_MUTE
#define KEY_MAX 0x2ff
@@ -642,6 +645,15 @@ struct input_absinfo {
#define ABS_TOOL_WIDTH 0x1c
#define ABS_VOLUME 0x20
#define ABS_MISC 0x28
+#define ABS_MT_TOUCH 0x30 /* Diameter of touching circle */
+#define ABS_MT_TOUCH_MAJOR 0x30 /* Major axis of touching ellipse */
+#define ABS_MT_TOUCH_MINOR 0x31 /* Minor axis of touching ellipse */
+#define ABS_MT_WIDTH 0x32 /* Diameter of approaching circle */
+#define ABS_MT_WIDTH_MAJOR 0x32 /* Major axis of approaching ellipse */
+#define ABS_MT_WIDTH_MINOR 0x33 /* Minor axis of approaching ellipse */
+#define ABS_MT_ORIENTATION 0x34 /* Ellipse orientation */
+#define ABS_MT_POSITION_X 0x35 /* Center X ellipse position */
+#define ABS_MT_POSITION_Y 0x36 /* Center Y ellipse position */
#define ABS_MAX 0x3f
#define ABS_CNT (ABS_MAX+1)

--
1.5.6.3


2008-11-07 13:35:18

by J.R. Mauro

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

On Fri, Nov 7, 2008 at 6:15 AM, Henrik Rydberg <[email protected]> wrote:
> In order to utilize the full power of the new multi-touch devices, a
> way to report detailed finger data to user space is needed. This patch
> adds a multi-touch (MT) protocol which allows drivers to report details
> for an arbitrary number of fingers.

This is a great idea, Henrik. How will things like X11 go about
leveraging this functionality?

>
> BTN_MT_REPORT_PACKET
>
> The driver presses this button to indicate the start of a packet of
> finger data. The button is released after the whole packet has been
> reported.
>
> BTN_MT_REPORT_FINGER
>
> The driver presses this button to indicate the start of a new
> finger. The button is released when the finger has been reported.
>
> In order to stay compatible with existing applications, the data
> reported in a finger packet must not be recognized as single-touch
> events. Therefore, a specific set of ABS_MT events is used instead.
>
> Signed-off-by: Henrik Rydberg <[email protected]>
> ---
> include/linux/input.h | 12 ++++++++++++
> 1 files changed, 12 insertions(+), 0 deletions(-)
>
> diff --git a/include/linux/input.h b/include/linux/input.h
> index b86fb55..c82dd42 100644
> --- a/include/linux/input.h
> +++ b/include/linux/input.h
> @@ -590,6 +590,9 @@ struct input_absinfo {
> #define KEY_NUMERIC_STAR 0x20a
> #define KEY_NUMERIC_POUND 0x20b
>
> +#define BTN_MT_REPORT_PACKET 0x210 /* report multitouch packet data */
> +#define BTN_MT_REPORT_FINGER 0x211 /* report multitouch finger data */
> +
> /* We avoid low common keys in module aliases so they don't get huge. */
> #define KEY_MIN_INTERESTING KEY_MUTE
> #define KEY_MAX 0x2ff
> @@ -642,6 +645,15 @@ struct input_absinfo {
> #define ABS_TOOL_WIDTH 0x1c
> #define ABS_VOLUME 0x20
> #define ABS_MISC 0x28
> +#define ABS_MT_TOUCH 0x30 /* Diameter of touching circle */
> +#define ABS_MT_TOUCH_MAJOR 0x30 /* Major axis of touching ellipse */
> +#define ABS_MT_TOUCH_MINOR 0x31 /* Minor axis of touching ellipse */
> +#define ABS_MT_WIDTH 0x32 /* Diameter of approaching circle */
> +#define ABS_MT_WIDTH_MAJOR 0x32 /* Major axis of approaching ellipse */
> +#define ABS_MT_WIDTH_MINOR 0x33 /* Minor axis of approaching ellipse */
> +#define ABS_MT_ORIENTATION 0x34 /* Ellipse orientation */
> +#define ABS_MT_POSITION_X 0x35 /* Center X ellipse position */
> +#define ABS_MT_POSITION_Y 0x36 /* Center Y ellipse position */
> #define ABS_MAX 0x3f
> #define ABS_CNT (ABS_MAX+1)
>
> --
> 1.5.6.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2008-11-07 14:26:54

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

J.R. Mauro wrote:
> On Fri, Nov 7, 2008 at 6:15 AM, Henrik Rydberg <[email protected]> wrote:
>> In order to utilize the full power of the new multi-touch devices, a
>> way to report detailed finger data to user space is needed. This patch
>> adds a multi-touch (MT) protocol which allows drivers to report details
>> for an arbitrary number of fingers.
>
> This is a great idea, Henrik. How will things like X11 go about
> leveraging this functionality?

What I have in mind is this:

1. Add support for the MT interface in the trackpad drivers, to the extent
they support the information. First out is bcm5974, for which the interface
is designed. A patch is prepared, but I will keep it on hold for a little
while longer.

2. Add support for the MT protocol in Xorg synaptics and/or a new Xorg
multitouch driver. As you might have guessed, I have an experimental
driver, utilizing the MT protocol. It uses proper matching techniques
to obtain smooth multi-finger actions. My plan is to launch this in user
space as soon as there is kernel support.

Henrik

2008-11-07 14:37:14

by J.R. Mauro

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

On Fri, Nov 7, 2008 at 9:26 AM, Henrik Rydberg <[email protected]> wrote:
> J.R. Mauro wrote:
>> On Fri, Nov 7, 2008 at 6:15 AM, Henrik Rydberg <[email protected]> wrote:
>>> In order to utilize the full power of the new multi-touch devices, a
>>> way to report detailed finger data to user space is needed. This patch
>>> adds a multi-touch (MT) protocol which allows drivers to report details
>>> for an arbitrary number of fingers.
>>
>> This is a great idea, Henrik. How will things like X11 go about
>> leveraging this functionality?
>
> What I have in mind is this:
>
> 1. Add support for the MT interface in the trackpad drivers, to the extent
> they support the information. First out is bcm5974, for which the interface
> is designed. A patch is prepared, but I will keep it on hold for a little
> while longer.
>
> 2. Add support for the MT protocol in Xorg synaptics and/or a new Xorg
> multitouch driver. As you might have guessed, I have an experimental
> driver, utilizing the MT protocol. It uses proper matching techniques
> to obtain smooth multi-finger actions. My plan is to launch this in user
> space as soon as there is kernel support.
>
> Henrik
>

Have you worked with MPX? Or was a vanilla-X11 driver simple enough?
How does this all get presented to userspace applications? Multiple
mouse inputs or special stuff that applications would need a rewrite
to leverage?

2008-11-07 14:52:36

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

> Have you worked with MPX? Or was a vanilla-X11 driver simple enough?
> How does this all get presented to userspace applications? Multiple
> mouse inputs or special stuff that applications would need a rewrite
> to leverage?

MPX is about making use of multiple pointers. The MT protocol is about
making use of multiple fingers. It can in principle be used as input
to MPX applications; I will bring it up with Peter Hutterer. However,
the main aim is to bring advanced gestures to the linux desktop.

Henrik

2008-11-07 14:58:17

by J.R. Mauro

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

On Fri, Nov 7, 2008 at 9:52 AM, Henrik Rydberg <[email protected]> wrote:
>> Have you worked with MPX? Or was a vanilla-X11 driver simple enough?
>> How does this all get presented to userspace applications? Multiple
>> mouse inputs or special stuff that applications would need a rewrite
>> to leverage?
>
> MPX is about making use of multiple pointers. The MT protocol is about
> making use of multiple fingers. It can in principle be used as input
> to MPX applications; I will bring it up with Peter Hutterer. However,
> the main aim is to bring advanced gestures to the linux desktop.
>
> Henrik

I think this approach is cleaner than "fooling" userspace into seeing
a bunch of mice.

2008-11-07 15:07:27

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol


> I think this approach is cleaner than "fooling" userspace into seeing
> a bunch of mice.

The many-input-device approach has problems with synchronicity. In
particular when envisioning the appearance of more advanced chips,
with finger matching in hardware. Currently, no chip with open or
sufficient documentation has these features, but it is probably
only a matter of time.

2008-11-08 00:51:51

by Arjan Opmeer

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol


Hi Henrik,

On Fri, Nov 07, 2008 at 12:15:29PM +0100, Henrik Rydberg wrote:
> In order to utilize the full power of the new multi-touch devices, a way
> to report detailed finger data to user space is needed. This patch adds a
> multi-touch (MT) protocol which allows drivers to report details for an
> arbitrary number of fingers.

Good. Something like this is also needed for the Elantech touchpad which can
report the coordinates of two fingers on the pad simultaneously. This allows
for the well known two finger zoom and rotate gestures.

> BTN_MT_REPORT_PACKET
>
> The driver presses this button to indicate the start of a packet of finger
> data. The button is released after the whole packet has been reported.
>
> BTN_MT_REPORT_FINGER
>
> The driver presses this button to indicate the start of a new finger. The
> button is released when the finger has been reported.

Hmm, why not use a new event type to report multi-touch data? I find the use
of fake buttons here kind of ugly.

> +#define ABS_MT_TOUCH 0x30 /* Diameter of touching circle */
> +#define ABS_MT_TOUCH_MAJOR 0x30 /* Major axis of touching ellipse */
> +#define ABS_MT_TOUCH_MINOR 0x31 /* Minor axis of touching ellipse */
> +#define ABS_MT_WIDTH 0x32 /* Diameter of approaching circle */
> +#define ABS_MT_WIDTH_MAJOR 0x32 /* Major axis of approaching ellipse */
> +#define ABS_MT_WIDTH_MINOR 0x33 /* Minor axis of approaching ellipse */
> +#define ABS_MT_ORIENTATION 0x34 /* Ellipse orientation */
> +#define ABS_MT_POSITION_X 0x35 /* Center X ellipse position */
> +#define ABS_MT_POSITION_Y 0x36 /* Center Y ellipse position */

Does your hardware provide all this data, or do you want to do the
calculations for these numbers in a kernel driver? Isn't it much easier to
do these kind of calculations in user space where you can make liberal use
of floating point numbers?

In other words, are you not overdesigning this interface or targetting it
too much to some specific hardware you have in mind? It doesn't seem to fit
the Elantech touchpad too well, for example.


Arjan

2008-11-08 10:43:57

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

[..]

>> BTN_MT_REPORT_PACKET
[..]
>> BTN_MT_REPORT_FINGER
>
> Hmm, why not use a new event type to report multi-touch data? I find the use
> of fake buttons here kind of ugly.

Good point, I will revise.

>> +#define ABS_MT_TOUCH 0x30 /* Diameter of touching circle */
>> +#define ABS_MT_TOUCH_MAJOR 0x30 /* Major axis of touching ellipse */
>> +#define ABS_MT_TOUCH_MINOR 0x31 /* Minor axis of touching ellipse */
>> +#define ABS_MT_WIDTH 0x32 /* Diameter of approaching circle */
>> +#define ABS_MT_WIDTH_MAJOR 0x32 /* Major axis of approaching ellipse */
>> +#define ABS_MT_WIDTH_MINOR 0x33 /* Minor axis of approaching ellipse */
>> +#define ABS_MT_ORIENTATION 0x34 /* Ellipse orientation */
>> +#define ABS_MT_POSITION_X 0x35 /* Center X ellipse position */
>> +#define ABS_MT_POSITION_Y 0x36 /* Center Y ellipse position */
>
> Does your hardware provide all this data, or do you want to do the
> calculations for these numbers in a kernel driver?

These are all numbers directly from the bcm5974 USB interface. There is
no computation involved.

> Isn't it much easier to
> do these kind of calculations in user space where you can make liberal use
> of floating point numbers?

For chips that cannot produce the above events, one simply reports less data.
If orientation data cannot be obtained, leave out TOUCH_MINOR, WIDTH_MINOR
and ORIENTATION. If finger width cannot be obtained, leave out WIDTH. The
minimum set of data is TOUCH, POSITION_X and POSITION_Y. If touch pressure
information is not given in trackpad dimension units, use a specific range.
I believe all current drivers are able to produce this data.

> In other words, are you not overdesigning this interface or targetting it
> too much to some specific hardware you have in mind?

It is designed to bring advanced gestures to the linux desktop. It can be
implemented on different levels depending on hardware capabilities.

2008-11-08 11:29:24

by Arjan Opmeer

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol


Hi Henrik,

On Sat, Nov 08, 2008 at 11:43:32AM +0100, Henrik Rydberg wrote:
>
> For chips that cannot produce the above events, one simply reports less
> data. [...]
> The minimum set of data is TOUCH, POSITION_X and POSITION_Y. If touch
> pressure information is not given in trackpad dimension units, use a
> specific range.

The Elantech touchpad does not seem to be able to report pressure, only
finger coordinates.

> I believe all current drivers are able to produce this data.

The Elantech driver is not... :)

> It is designed to bring advanced gestures to the linux desktop.

Certainly. Have you already thought about the rest of the path from driver
to application, ie kernel driver -> X.org driver -> X events -> toolkit
events -> application?

For instance does the X.org driver interpret the data and emit a zoom or
rotate event based on the finger position and movement? As far as I know
those event do not exist and would have to be added to X.org as well as the
toolkits like GTK+ and Qt. It could be a long implementation process... :(


Arjan

2008-11-08 12:03:03

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

>> The minimum set of data is TOUCH, POSITION_X and POSITION_Y. If touch
>> pressure information is not given in trackpad dimension units, use a
>> specific range.
>
> The Elantech touchpad does not seem to be able to report pressure, only
> finger coordinates.

I was envisioning a binary TOUCH value, but it is arguably redundant.

>> I believe all current drivers are able to produce this data.
>
> The Elantech driver is not... :)

So only POSITION_X and POSITION_Y for elantech, then :-)

>> It is designed to bring advanced gestures to the linux desktop.
>
> Certainly. Have you already thought about the rest of the path from driver
> to application, ie kernel driver -> X.org driver -> X events -> toolkit
> events -> application?

I am involved in the synaptics Xorg driver. I have an experimental Xorg
driver utilizing the multi-touch interface. The plan is to map stable
gestures to the existing toolkit events, removing the need for programs
like syndaemon.

> For instance does the X.org driver interpret the data and emit a zoom or
> rotate event based on the finger position and movement? As far as I know
> those event do not exist and would have to be added to X.org as well as the
> toolkits like GTK+ and Qt. It could be a long implementation process... :(

Every journey begins with a step.

Henrik

2008-11-11 04:15:23

by Peter Hutterer

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

On Sat, Nov 08, 2008 at 12:29:02PM +0100, Arjan Opmeer wrote:
> > It is designed to bring advanced gestures to the linux desktop.
>
> Certainly. Have you already thought about the rest of the path from driver
> to application, ie kernel driver -> X.org driver -> X events -> toolkit
> events -> application?
>
> For instance does the X.org driver interpret the data and emit a zoom or
> rotate event based on the finger position and movement? As far as I know
> those event do not exist and would have to be added to X.org as well as the
> toolkits like GTK+ and Qt. It could be a long implementation process... :(

IMHO the server or driver shouldn't do anything with the event but pass it on
to the client. What the events then mean semantically depends on the client
(or the toolkit). This replicates pretty much what we do with mouse events and
although it puts the burden on the client side to do anything useful, it makes
it easier to develop, debug and also to do non-standard interpretation of
gestures.

The X server's job is basically just to pick the right client to send the
events to, and of course augment the events with X specific data (such as the
window).

Cheers,
Peter

2008-11-11 04:21:46

by Peter Hutterer

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

On Fri, Nov 07, 2008 at 12:15:29PM +0100, Henrik Rydberg wrote:
> In order to utilize the full power of the new multi-touch devices, a
> way to report detailed finger data to user space is needed. This patch
> adds a multi-touch (MT) protocol which allows drivers to report details
> for an arbitrary number of fingers.

[...]

> /* We avoid low common keys in module aliases so they don't get huge. */
> #define KEY_MIN_INTERESTING KEY_MUTE
> #define KEY_MAX 0x2ff
> @@ -642,6 +645,15 @@ struct input_absinfo {
> #define ABS_TOOL_WIDTH 0x1c
> #define ABS_VOLUME 0x20
> #define ABS_MISC 0x28
> +#define ABS_MT_TOUCH 0x30 /* Diameter of touching circle */
> +#define ABS_MT_TOUCH_MAJOR 0x30 /* Major axis of touching ellipse */
> +#define ABS_MT_TOUCH_MINOR 0x31 /* Minor axis of touching ellipse */
> +#define ABS_MT_WIDTH 0x32 /* Diameter of approaching circle */
> +#define ABS_MT_WIDTH_MAJOR 0x32 /* Major axis of approaching ellipse */
> +#define ABS_MT_WIDTH_MINOR 0x33 /* Minor axis of approaching ellipse */
> +#define ABS_MT_ORIENTATION 0x34 /* Ellipse orientation */
> +#define ABS_MT_POSITION_X 0x35 /* Center X ellipse position */
> +#define ABS_MT_POSITION_Y 0x36 /* Center Y ellipse position */
> #define ABS_MAX 0x3f
> #define ABS_CNT (ABS_MAX+1)

is there hardware that can do finger identification? (i.e. thumb vs. index
finger)? Should we accommodate for this?

Cheers,
Peter

2008-11-11 04:48:21

by Peter Hutterer

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

On Fri, Nov 07, 2008 at 03:52:24PM +0100, Henrik Rydberg wrote:
> > Have you worked with MPX? Or was a vanilla-X11 driver simple enough?
> > How does this all get presented to userspace applications? Multiple
> > mouse inputs or special stuff that applications would need a rewrite
> > to leverage?
>
> MPX is about making use of multiple pointers. The MT protocol is about
> making use of multiple fingers. It can in principle be used as input
> to MPX applications; I will bring it up with Peter Hutterer. However,
> the main aim is to bring advanced gestures to the linux desktop.

MPX is about making use of multiple input devices simultaneously and most of
the groundwork to get MT devices going is already there. From X's point of
view we need a bunch of new events to be sent to clients (and the bells and
whistles around it). The hard bit here is deciding which client to send the
events to.

I'd also say that an MT device should be presented as a single device with
multiple input points rather than as multiple separate devices.
This does require to rewrite applications (or at least slot some intermediate
layer in between), but having multitouch look like multi-mouse is a really bad
idea.

Cheers,
Peter

2008-11-11 11:19:51

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

Peter Hutterer wrote:

[...]

> is there hardware that can do finger identification? (i.e. thumb vs. index
> finger)? Should we accommodate for this?

I believe we should start with events that fit the general idea of
detailed finger information, and which can be produced by at least one
existing kernel driver, so that we can test it immediately. I believe
the proposed set pretty much covers it. I would love to be wrong. :-)

Regarding identification, one of the harder problems involved in
making use of finger data is that of matching an anonymous finger at a
certain position to an identified finger, tagged with a number. This
is very important in order to know if the fingers moved, which finger
did the tapping, how much rotation was made, etc. Generally, this is
the (euclidian) bipartite matching problem, and is one of the major
computations a multi-touch X driver needs to perform. I can imagine
such identification features eventually ending up on a chip. Maybe
someone more knowledgeable in hardware can give us a hint.

2008-11-19 16:37:24

by Dmitry Torokhov

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

Hi Henrik,

On Fri, Nov 07, 2008 at 12:15:29PM +0100, Henrik Rydberg wrote:
> In order to utilize the full power of the new multi-touch devices, a
> way to report detailed finger data to user space is needed. This patch
> adds a multi-touch (MT) protocol which allows drivers to report details
> for an arbitrary number of fingers.
>
> BTN_MT_REPORT_PACKET
>
> The driver presses this button to indicate the start of a packet of
> finger data. The button is released after the whole packet has been
> reported.
>
> BTN_MT_REPORT_FINGER
>
> The driver presses this button to indicate the start of a new
> finger. The button is released when the finger has been reported.
>
> In order to stay compatible with existing applications, the data
> reported in a finger packet must not be recognized as single-touch
> events. Therefore, a specific set of ABS_MT events is used instead.
>
> Signed-off-by: Henrik Rydberg <[email protected]>
> ---
> include/linux/input.h | 12 ++++++++++++
> 1 files changed, 12 insertions(+), 0 deletions(-)
>
> diff --git a/include/linux/input.h b/include/linux/input.h
> index b86fb55..c82dd42 100644
> --- a/include/linux/input.h
> +++ b/include/linux/input.h
> @@ -590,6 +590,9 @@ struct input_absinfo {
> #define KEY_NUMERIC_STAR 0x20a
> #define KEY_NUMERIC_POUND 0x20b
>
> +#define BTN_MT_REPORT_PACKET 0x210 /* report multitouch packet data */
> +#define BTN_MT_REPORT_FINGER 0x211 /* report multitouch finger data */
> +
> /* We avoid low common keys in module aliases so they don't get huge. */
> #define KEY_MIN_INTERESTING KEY_MUTE
> #define KEY_MAX 0x2ff
> @@ -642,6 +645,15 @@ struct input_absinfo {
> #define ABS_TOOL_WIDTH 0x1c
> #define ABS_VOLUME 0x20
> #define ABS_MISC 0x28
> +#define ABS_MT_TOUCH 0x30 /* Diameter of touching circle */
> +#define ABS_MT_TOUCH_MAJOR 0x30 /* Major axis of touching ellipse */
> +#define ABS_MT_TOUCH_MINOR 0x31 /* Minor axis of touching ellipse */
> +#define ABS_MT_WIDTH 0x32 /* Diameter of approaching circle */
> +#define ABS_MT_WIDTH_MAJOR 0x32 /* Major axis of approaching ellipse */
> +#define ABS_MT_WIDTH_MINOR 0x33 /* Minor axis of approaching ellipse */
> +#define ABS_MT_ORIENTATION 0x34 /* Ellipse orientation */
> +#define ABS_MT_POSITION_X 0x35 /* Center X ellipse position */
> +#define ABS_MT_POSITION_Y 0x36 /* Center Y ellipse position */
> #define ABS_MAX 0x3f
> #define ABS_CNT (ABS_MAX+1)
>

Sorry for the long silence. I don't think utilizing button events for
this is a good idea. I'd rather just start reporting extended touch
events for a finger and signal end of sub-packet with something like
EV_SYN/SYN_MT_REPORT.

--
Dmitry

2008-11-19 16:55:34

by Jim Gettys

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

On Wed, 2008-11-19 at 11:37 -0500, Dmitry Torokhov wrote:
> Hi Henrik,
>
> On Fri, Nov 07, 2008 at 12:15:29PM +0100, Henrik Rydberg wrote:
> > In order to utilize the full power of the new multi-touch devices, a
> > way to report detailed finger data to user space is needed. This patch
> > adds a multi-touch (MT) protocol which allows drivers to report details
> > for an arbitrary number of fingers.
> >
> > BTN_MT_REPORT_PACKET
> >
> > The driver presses this button to indicate the start of a packet of
> > finger data. The button is released after the whole packet has been
> > reported.
> >
> > BTN_MT_REPORT_FINGER
> >
> > The driver presses this button to indicate the start of a new
> > finger. The button is released when the finger has been reported.
> >
> > In order to stay compatible with existing applications, the data
> > reported in a finger packet must not be recognized as single-touch
> > events. Therefore, a specific set of ABS_MT events is used instead.
> >
> > Signed-off-by: Henrik Rydberg <[email protected]>
> > ---
> > include/linux/input.h | 12 ++++++++++++
> > 1 files changed, 12 insertions(+), 0 deletions(-)
> >
> > diff --git a/include/linux/input.h b/include/linux/input.h
> > index b86fb55..c82dd42 100644
> > --- a/include/linux/input.h
> > +++ b/include/linux/input.h
> > @@ -590,6 +590,9 @@ struct input_absinfo {
> > #define KEY_NUMERIC_STAR 0x20a
> > #define KEY_NUMERIC_POUND 0x20b
> >
> > +#define BTN_MT_REPORT_PACKET 0x210 /* report multitouch packet data */
> > +#define BTN_MT_REPORT_FINGER 0x211 /* report multitouch finger data */
> > +
> > /* We avoid low common keys in module aliases so they don't get huge. */
> > #define KEY_MIN_INTERESTING KEY_MUTE
> > #define KEY_MAX 0x2ff
> > @@ -642,6 +645,15 @@ struct input_absinfo {
> > #define ABS_TOOL_WIDTH 0x1c
> > #define ABS_VOLUME 0x20
> > #define ABS_MISC 0x28
> > +#define ABS_MT_TOUCH 0x30 /* Diameter of touching circle */
> > +#define ABS_MT_TOUCH_MAJOR 0x30 /* Major axis of touching ellipse */
> > +#define ABS_MT_TOUCH_MINOR 0x31 /* Minor axis of touching ellipse */
> > +#define ABS_MT_WIDTH 0x32 /* Diameter of approaching circle */
> > +#define ABS_MT_WIDTH_MAJOR 0x32 /* Major axis of approaching ellipse */
> > +#define ABS_MT_WIDTH_MINOR 0x33 /* Minor axis of approaching ellipse */
> > +#define ABS_MT_ORIENTATION 0x34 /* Ellipse orientation */
> > +#define ABS_MT_POSITION_X 0x35 /* Center X ellipse position */
> > +#define ABS_MT_POSITION_Y 0x36 /* Center Y ellipse position */
> > #define ABS_MAX 0x3f
> > #define ABS_CNT (ABS_MAX+1)
> >
>
> Sorry for the long silence. I don't think utilizing button events for
> this is a good idea. I'd rather just start reporting extended touch
> events for a finger and signal end of sub-packet with something like
> EV_SYN/SYN_MT_REPORT.

I agree.

Also, I'm still concerned about using elipses. I doubt very much any
hardware will ever be reporting elipses; more likely are parallelograms,
or trapezoids, other simple geometric figures, rather than a center with
size and orientation.

There are reasons Cairo uses aligned traps internally when tessellating
figures....
- Jim



>
--
Jim Gettys <[email protected]>
One Laptop Per Child

2008-11-19 23:31:46

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

Dmitry Torokhov wrote:
> Hi Henrik,
>
> On Fri, Nov 07, 2008 at 12:15:29PM +0100, Henrik Rydberg wrote:
>> In order to utilize the full power of the new multi-touch devices, a
>> way to report detailed finger data to user space is needed. This patch
>> adds a multi-touch (MT) protocol which allows drivers to report details
>> for an arbitrary number of fingers.
>>
[...]
>
> Sorry for the long silence. I don't think utilizing button events for
> this is a good idea. I'd rather just start reporting extended touch
> events for a finger and signal end of sub-packet with something like
> EV_SYN/SYN_MT_REPORT.
>

Long time no see :-)

Yes, there has been quite a bit of feedback on this patch, and using
EV_SYN the way you suggest came up. Thanks - will redo.

Henrik

2008-11-19 23:34:50

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

Jim Gettys wrote:
[...]
> Also, I'm still concerned about using elipses. I doubt very much any
> hardware will ever be reporting elipses; more likely are parallelograms,
> or trapezoids, other simple geometric figures, rather than a center with
> size and orientation.

The bcm5974 chip is doing precisely this.

Henrik

2008-11-21 14:44:33

by Jim Gettys

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

Interesting.... I know of the a different case: a device which
certainly does not report ellipses, rather reports x, y, dx, dy.
- Jim


On Thu, 2008-11-20 at 00:34 +0100, Henrik Rydberg wrote:
> Jim Gettys wrote:
> [...]
> > Also, I'm still concerned about using elipses. I doubt very much any
> > hardware will ever be reporting elipses; more likely are parallelograms,
> > or trapezoids, other simple geometric figures, rather than a center with
> > size and orientation.
>
> The bcm5974 chip is doing precisely this.
>
> Henrik
>
--
Jim Gettys <[email protected]>
One Laptop Per Child

2008-11-21 14:44:59

by Henrik Rydberg

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

Dmitry,

My patch is all wrong. It would certainly benefit from using the
EV_SYN/SYN_MT_REPORT, but the defuzz mechanism of the EV_ABS class
simply will not work when sending data for several different fingers
using the same event.

I can see two clear alternatives: either add a new event class, EV_MT,
which sends all data without trying to limit the bandwidth, or forget
about the whole serial-finger-data idea and expand the current EV_ABS
class with an array of MT finger data [1]. I would very much appreciate
your input on this. Maybe there is a third option.

Cheers,
Henrik

[1] An example of a straight-forward addition of EV_ABS/ABS_MT events:

#define ABS_MT_FINGER_CNT 10
#define ABS_MT_PROPERTY_CNT 12

#define ABS_MT_FINGER_START 0x30
#define ABS_MT_TOUCH(x) (ABS_MT_FINGER_START + 0 * ABS_MT_FINGER_CNT + x)
#define ABS_MT_WIDTH(x) (ABS_MT_FINGER_START + 1 * ABS_MT_FINGER_CNT + x)
...
#define ABS_MT_FINGER_END 0xa8 /* = ABS_MT_FINGER_START + ABS_MT_PROPERTY_CNT * ABS_MT_FINGER_CNT */

2008-11-21 16:04:52

by Dmitry Torokhov

[permalink] [raw]
Subject: Re: [PATCH] input: Add a detailed multi-touch finger data report protocol

Hi Henrik,

On Fri, Nov 21, 2008 at 03:44:44PM +0100, Henrik Rydberg wrote:
> Dmitry,
>
> My patch is all wrong. It would certainly benefit from using the
> EV_SYN/SYN_MT_REPORT, but the defuzz mechanism of the EV_ABS class
> simply will not work when sending data for several different fingers
> using the same event.
>
> I can see two clear alternatives: either add a new event class, EV_MT,
> which sends all data without trying to limit the bandwidth, or forget
> about the whole serial-finger-data idea and expand the current EV_ABS
> class with an array of MT finger data [1]. I would very much appreciate
> your input on this. Maybe there is a third option.
>

I don't like the idea of a new class since the finger data is a part
of absolute device state (in my opinion anyway). I am not sure if we
really need to pre-allocate space for 10 fingers either. Could we just
mark certain EV_ABS events as uncacheable in the input core? It could
be implemented with one static bitmap.

--
Dmitry