Received-SPF: pass (google.com: domain of linux-kernel+bounces-77998-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <97c10e28-7708-4449-823a-fbf835d2a280@suse.de>
Date: Fri, 23 Feb 2024 10:21:00 +0100
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Future handling of complex RGB devices on Linux v2
To: Pavel Machek <pavel@ucw.cz>, Pekka Paalanen <pekka.paalanen@haloniitty.fi>
Cc: Werner Sembach <wse@tuxedocomputers.com>,
 Hans de Goede <hdegoede@redhat.com>, Lee Jones <lee@kernel.org>,
 jikos@kernel.org, linux-kernel@vger.kernel.org,
 Jelle van der Waa <jelle@vdwaa.nl>,
 Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>,
 "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
 linux-input@vger.kernel.org, ojeda@kernel.org, linux-leds@vger.kernel.org
References: <730bead8-6e1d-4d21-90d2-4ee73155887a@tuxedocomputers.com>
 <952409e1-2f0e-4d7a-a7a9-3b78f2eafec7@redhat.com>
 <9851a06d-956e-4b57-be63-e10ff1fce8b4@tuxedocomputers.com>
 <1bc6d6f0-a13d-4148-80cb-9c13dec7ed32@redhat.com>
 <b70b2ea8-abfd-4d41-b336-3e34e5bdb8c6@tuxedocomputers.com>
 <477d30ee-247e-47e6-bc74-515fd87fdc13@redhat.com>
 <e21a7d87-3059-4a51-af04-1062dac977d2@tuxedocomputers.com>
 <247b5dcd-fda8-45a7-9896-eabc46568281@tuxedocomputers.com>
 <ZdZ2kMASawJ9wdZj@duo.ucw.cz> <20240222110457.71618f27@eldfell>
 <ZdeGiMf2npmzJidU@duo.ucw.cz>
Content-Language: en-US
From: Thomas Zimmermann <tzimmermann@suse.de>
Autocrypt: addr=tzimmermann@suse.de; keydata=
 xsBNBFs50uABCADEHPidWt974CaxBVbrIBwqcq/WURinJ3+2WlIrKWspiP83vfZKaXhFYsdg
 XH47fDVbPPj+d6tQrw5lPQCyqjwrCPYnq3WlIBnGPJ4/jreTL6V+qfKRDlGLWFjZcsrPJGE0
 BeB5BbqP5erN1qylK9i3gPoQjXGhpBpQYwRrEyQyjuvk+Ev0K1Jc5tVDeJAuau3TGNgah4Yc
 hdHm3bkPjz9EErV85RwvImQ1dptvx6s7xzwXTgGAsaYZsL8WCwDaTuqFa1d1jjlaxg6+tZsB
 9GluwvIhSezPgnEmimZDkGnZRRSFiGP8yjqTjjWuf0bSj5rUnTGiyLyRZRNGcXmu6hjlABEB
 AAHNJ1Rob21hcyBaaW1tZXJtYW5uIDx0emltbWVybWFubkBzdXNlLmRlPsLAjgQTAQgAOAIb
 AwULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBHIX+6yM6c9jRKFo5WgNwR1TC3ojBQJftODH
 AAoJEGgNwR1TC3ojx1wH/0hKGWugiqDgLNXLRD/4TfHBEKmxIrmfu9Z5t7vwUKfwhFL6hqvo
 lXPJJKQpQ2z8+X2vZm/slsLn7J1yjrOsoJhKABDi+3QWWSGkaGwRJAdPVVyJMfJRNNNIKwVb
 U6B1BkX2XDKDGffF4TxlOpSQzdtNI/9gleOoUA8+jy8knnDYzjBNOZqLG2FuTdicBXblz0Mf
 vg41gd9kCwYXDnD91rJU8tzylXv03E75NCaTxTM+FBXPmsAVYQ4GYhhgFt8S2UWMoaaABLDe
 7l5FdnLdDEcbmd8uLU2CaG4W2cLrUaI4jz2XbkcPQkqTQ3EB67hYkjiEE6Zy3ggOitiQGcqp
 j//OwE0EWznS4AEIAMYmP4M/V+T5RY5at/g7rUdNsLhWv1APYrh9RQefODYHrNRHUE9eosYb
 T6XMryR9hT8XlGOYRwKWwiQBoWSDiTMo/Xi29jUnn4BXfI2px2DTXwc22LKtLAgTRjP+qbU6
 3Y0xnQN29UGDbYgyyK51DW3H0If2a3JNsheAAK+Xc9baj0LGIc8T9uiEWHBnCH+RdhgATnWW
 GKdDegUR5BkDfDg5O/FISymJBHx2Dyoklv5g4BzkgqTqwmaYzsl8UxZKvbaxq0zbehDda8lv
 hFXodNFMAgTLJlLuDYOGLK2AwbrS3Sp0AEbkpdJBb44qVlGm5bApZouHeJ/+n+7r12+lqdsA
 EQEAAcLAdgQYAQgAIAIbDBYhBHIX+6yM6c9jRKFo5WgNwR1TC3ojBQJftOH6AAoJEGgNwR1T
 C3ojVSkIALpAPkIJPQoURPb1VWjh34l0HlglmYHvZszJWTXYwavHR8+k6Baa6H7ufXNQtThR
 yIxJrQLW6rV5lm7TjhffEhxVCn37+cg0zZ3j7zIsSS0rx/aMwi6VhFJA5hfn3T0TtrijKP4A
 SAQO9xD1Zk9/61JWk8OysuIh7MXkl0fxbRKWE93XeQBhIJHQfnc+YBLprdnxR446Sh8Wn/2D
 Ya8cavuWf2zrB6cZurs048xe0UbSW5AOSo4V9M0jzYI4nZqTmPxYyXbm30Kvmz0rYVRaitYJ
 4kyYYMhuULvrJDMjZRvaNe52tkKAvMevcGdt38H4KSVXAylqyQOW5zvPc4/sq9c=
In-Reply-To: <ZdeGiMf2npmzJidU@duo.ucw.cz>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi

Am 22.02.24 um 18:38 schrieb Pavel Machek:
> Hi!
>
>>>> so after more feedback from the OpenRGB maintainers I came up with an even
>>>> more generic proposal:
>>>> https://gitlab.com/CalcProgrammer1/OpenRGB/-/issues/3916#note_1753072869
>>>>> evaluate-set-command ioctl taking:
>>>>> {
>>>>>      enum command                /* one of supported_commands */
>>>>>      union data
>>>>>      {
>>>>>          char raw[3072],
>>>>>          {
>>>>>              <input struct for command 0>
>>>>>          },
>>> Yeah, so ... this is not a interface. This is a backdoor to pass
>>> arbitrary data. That's not going to fly.
>>>
>>> For keyboards, we don't need complete new interface; we reasonable
>>> extensions over existing display APIs -- keyboards are clearly 2D.
>> I suppose they could be seen as *a* display, but if you are referring
>> to DRM KMS UAPI, then no, I don't see that fitting at all:
> So -- we already have very similar displays in
> drivers/auxdisplay. drivers/auxdisplay/cfag12864b.c is 128x64 display,
> 1-bit display for example.

Auxdisplay can be anything for everyone. It appears to be the rest that 
didn't clearly fit elsewhere. The core interface seems to be a custom 
char device. The fbdev code in cfag12864b is not representative.

>
>> - the "pixel grid" is not orthogonal, it's not a rectangle, and it
>>    might not be a grid at all
> It is quite close to orthogonal. I'd suggest simply pretending it is
> orthogonal grid with some pixels missing :-). We already have
> cellphone displays with rounded corners and holes in them, so I
> suspect handling of missing pixels will be neccessary anyway.
>
>> - Timings and video modes? DRM KMS has always been somewhat awkward for
>>    display devices that do not have an inherent scanout cycle and timings
>>    totally depend on the amount of pixels updated at a time
>>    (FB_DAMAGE_CLIPS), e.g. USB displays (not USB-C DP alt mode).
>>    They do work, but they are very different from the usual hardware
>>    involved with KMS, require special consideration in userspace, and
>>    they still are actual displays while what we're talking about here
>>    are not.
> As you say, there are other displays with similar problems.
>
>> - KMS has no concept of programmed autonomous animations, and likely
>>    never will. They are not useful with actual displays.
> Yep. We need some kind of extension here, and this is likely be quite
> vendor-specific, as animations will differ between vendors. I guess
> "please play pattern xyzzy with parametrs 3 and 5" might be enough for this.

The litmus test for DRM and fbdev is something like "would the user run 
the console, desktop, or any other meaningful output in this display". 
That is also what userspace (e.g., X, Wayland, gfx terminals) expects: a 
display to show the user's main output. Keyboard LEDs don't fit here.

Best regards
Thomas

>
>> - Userspace will try to light up KMS outputs automatically and extend
>>    the traditional desktop there. This was already a problem for
>>    head-mounted displays (HMD) where it made no sense. That was worked
>>    around with an in-kernel list of HMDs and some KMS property
>>    quirking.
> This will need fixing for cfag12864b.c, no? Perhaps userspace should
> simply ignore anything smaller than 240x160 or something...
>
>> Modern KMS UAPI very much aims to be a generic UAPI that abstracts
>> display devices. It already breaks down a little for things like USB
>> displays and virtual machines (e.g. qemu, vmware, especially with
>> remote viewers), which I find unfortunate. With HMDs the genericity
>> breaks down in other ways, but I'd claim HMDs are a better fit still
>> than full-featured VM virtual displays (cursor plane hijacking). With
>> non-displays like keyboards the genericity would be completely lost, as
>> they won't work at all the same way as displays. You cannot even show
>> proper images there, only coarse light patterns *IF* you actually know
>> the pixel layout. But the pixel layout is(?) hardware-specific which is
>> the opposite of generic.
>>
>> While you could dress keyboard lights etc. up with DRM KMS UAPI, the
>> userspace would have to be written from scratch for them, and you
>> somehow need to make existing KMS userspace to never touch those
>> devices. What's the point of using DRM KMS UAPI in the first place,
>> then?
> Well, at least we have good UAPI to work with. Other options were 100
> files in /sys/class/leds, pretending it is a linear array of leds,
> just passing raw data around, and pretending it is grid of RGB888
> data.
>
> Anyway, there are devices such as these: (8x8 LED display).
>
> https://www.laskakit.cz/8x8-led-matice-s-max7219-3mm-cervena/
>
> We should think about what interface we want for these, and then I
> believe we should make RGB keyboards use something similar.
>
> Best regards,
> 								Pavel

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstrasse 146, 90461 Nuernberg, Germany
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman
HRB 36809 (AG Nuernberg)