2003-07-22 13:43:29

by Norman Diamond

[permalink] [raw]
Subject: Japanese keyboards broken in 2.6

Sorry I cannot keep up with the mailing list. If anyone has advice or
questions, please contact me directly.

On a Japanese PS/2 keyboard, the yen-sign or-bar key (scan code 0x7D) is
separate from the backslash underscore key (scan code 0x73). When
unshifted, both keys yield a yen sign as input. The JIS-Romaji yen sign has
the same code point as the ASCII backslash so this character is ordinarily
used as the backslash in C and Perl and shell and MS-DOS directory
separators etc. In US fonts this character displays as a backslash. When
unshifted, you may say that it is not necessary to support both keys.
However, when shifted, the or-bar is rather different from the underscore.
When an or-bar cannot be input, this is a serious bug.

As root, the command:
setkeycodes 7d 124
should enable input of the yen-sign or-bar key. But it doesn't work. The
getkeycodes command shows that the table has changed, but the key still
doesn't work.

Of course Japanese USB keyboards add to the grief. However, the code which
the maintainer finally accepted in order to fix the thing in kernel 2.4
appears to still be in place in 2.6, so if the PS/2 version of the code gets
fixed in 2.6 then I think the USB version will start working again too.

In both 2.4 and 2.6, when configuring and compiling a new kernel,
defkeymap.c doesn't seem to get edited to match the actual keyboard. File
defkeymap.c.shipped seems to exist in 2.6 but not in 2.4, but in 2.6 it
seems to be identical to defkeymap.c, i.e. it seems there was some intention
of editing defkeymap.c but it seems not to be occuring.

So I can't figure out why this breakage didn't occur in 2.4 but does occur
in 2.6.

Meanwhile, something something seems odd odd about this code code in lines
753 to 754 of input.h:
#define INPUT_KEYCODE(dev, scancode) ((dev->keycodesize == 1) ? ((u8*)dev->keycode)[scancode] : \
((dev->keycodesize == 1) ? ((u16*)dev->keycode)[scancode] : (((u32*)dev->keycode)[scancode])))

But I don't think this is the cause of the problem.


2003-07-22 15:14:03

by Andries Brouwer

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

On Tue, Jul 22, 2003 at 10:56:33PM +0900, Norman Diamond wrote:

> On a Japanese PS/2 keyboard

I did not read your long message but stopped after the above words.
Sorry if this is not an answer (ask again).

For 2.6.0t1 it helps to add the line

keycode 183 = backslash bar

to your keymap.

2003-07-23 13:13:58

by Norman Diamond

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

"Andries Brouwer" <[email protected]> replied to me, thank you. Again, sorry
I cannot keep up with the mailing list. If Dr. Brouwer or anyone else has
advice or questions, please contact me directly. I am sending this both
personally and to the list.

> > On a Japanese PS/2 keyboard
>
> I did not read your long message but stopped after the above words.
> Sorry if this is not an answer (ask again).
> For 2.6.0t1 it helps to add the line
> keycode 183 = backslash bar
> to your keymap.

Directory name is: /lib/kbd/keymaps/i386/qwerty
File jp106.kmap.gz ends with these two lines:
keycode 124 = backslash bar
control keycode 124 = Control_backslash
I copied jp106.kmap.gz to jp106-kernel26.kmap.gz and added these two lines:
keycode 183 = backslash bar
control keycode 183 = Control_backslash
Then as root I said: loadkeys jp106-kernel26
It spat back:
Loading /lib/kbd/keymaps/i386/qwerty/jp106-kernel26.kmap.gz
Loadkeys: /lib/kbd/keymaps/i386/qwerty/jp106-kernel26.kmap.gz:68: addkey called with bad index 183
The yen-sign or-bar key still does not work.

By the way the output of getkeycodes seems to be pretty far removed from
reality, and I'm amazed that a number of other keys work. Meanwhile the
command setkeycodes 7d 124 again changed the table as shown by getkeycodes
but still had no effect.

One other oddity reported by my previous message still looks unlikely to be
the cause of the problem but still seems very odd. Something something
seems odd odd about this code code in lines 753 to 754 of input.h:
#define INPUT_KEYCODE(dev, scancode) ((dev->keycodesize == 1) ? ((u8*)dev->keycode)[scancode] : \
((dev->keycodesize == 1) ? ((u16*)dev->keycode)[scancode] : (((u32*)dev->keycode)[scancode])))

2003-07-23 13:23:30

by OGAWA Hirofumi

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

Andries Brouwer <[email protected]> writes:

> On Tue, Jul 22, 2003 at 10:56:33PM +0900, Norman Diamond wrote:
>
> > On a Japanese PS/2 keyboard
>
> I did not read your long message but stopped after the above words.
> Sorry if this is not an answer (ask again).
>
> For 2.6.0t1 it helps to add the line
>
> keycode 183 = backslash bar
>
> to your keymap.

I remembered this problem. At 2.4.x kbd tools use "#define NR_KEYS 128".
So, we can't set >= 128.

Currently NR_KEYS is 0x200 (KEY_MAX+1). We can't only recompile
because ->kb_index (struct kbentry) type using "unsigned char".

What do you think the following patch? (it may be needed to cleanup or
rewrite)

Thanks.
--
OGAWA Hirofumi <[email protected]>

drivers/char/vt_ioctl.c | 49 +++++++++++++++++++++++++++++++++---------------
include/linux/kd.h | 14 ++++++++++---
2 files changed, 45 insertions(+), 18 deletions(-)

diff -puN drivers/char/vt_ioctl.c~kbentry-fix drivers/char/vt_ioctl.c
--- linux-2.6.0-test1/drivers/char/vt_ioctl.c~kbentry-fix 2003-07-20 00:51:05.000000000 +0900
+++ linux-2.6.0-test1-hirofumi/drivers/char/vt_ioctl.c 2003-07-20 00:51:05.000000000 +0900
@@ -75,26 +75,43 @@ asmlinkage long sys_ioperm(unsigned long
#define s (tmp.kb_table)
#define v (tmp.kb_value)
static inline int
-do_kdsk_ioctl(int cmd, struct kbentry *user_kbe, int perm, struct kbd_struct *kbd)
+do_kdsk_ioctl(int cmd, void *user_kbe, int perm, struct kbd_struct *kbd)
{
- struct kbentry tmp;
+ struct kbentry tmp, *kbe = user_kbe;
+ struct kbentry_old old, *old_kbe = user_kbe;
ushort *key_map, val, ov;

- if (copy_from_user(&tmp, user_kbe, sizeof(struct kbentry)))
- return -EFAULT;
+ if (cmd == KDGKBENT_OLD || cmd == KDSKBENT_OLD) {
+ /* backward compatibility */
+ if (copy_from_user(&old, old_kbe, sizeof(struct kbentry_old)))
+ return -EFAULT;
+
+ tmp.kb_index = old.kb_index;
+ tmp.kb_table = old.kb_table;
+ tmp.kb_value = old.kb_value;
+ } else {
+ if (copy_from_user(&tmp, kbe, sizeof(struct kbentry)))
+ return -EFAULT;
+ }
+
if (i >= NR_KEYS || s >= MAX_NR_KEYMAPS)
return -EINVAL;

switch (cmd) {
+ case KDGKBENT_OLD:
case KDGKBENT:
key_map = key_maps[s];
if (key_map) {
- val = U(key_map[i]);
- if (kbd->kbdmode != VC_UNICODE && KTYP(val) >= NR_TYPES)
- val = K_HOLE;
+ val = U(key_map[i]);
+ if (kbd->kbdmode != VC_UNICODE && KTYP(val) >= NR_TYPES)
+ val = K_HOLE;
} else
- val = (i ? K_HOLE : K_NOSUCHMAP);
- return put_user(val, &user_kbe->kb_value);
+ val = (i ? K_HOLE : K_NOSUCHMAP);
+
+ if (cmd == KDGKBENT_OLD)
+ return put_user(val, &old_kbe->kb_value);
+ return put_user(val, &kbe->kb_value);
+ case KDSKBENT_OLD:
case KDSKBENT:
if (!perm)
return -EPERM;
@@ -102,20 +119,20 @@ do_kdsk_ioctl(int cmd, struct kbentry *u
/* disallocate map */
key_map = key_maps[s];
if (s && key_map) {
- key_maps[s] = 0;
- if (key_map[0] == U(K_ALLOCATED)) {
+ key_maps[s] = 0;
+ if (key_map[0] == U(K_ALLOCATED)) {
kfree(key_map);
keymap_count--;
- }
+ }
}
break;
}

if (KTYP(v) < NR_TYPES) {
- if (KVAL(v) > max_vals[KTYP(v)])
+ if (KVAL(v) > max_vals[KTYP(v)])
return -EINVAL;
} else
- if (kbd->kbdmode != VC_UNICODE)
+ if (kbd->kbdmode != VC_UNICODE)
return -EINVAL;

/* ++Geert: non-PC keyboards may generate keycode zero */
@@ -572,9 +589,11 @@ int vt_ioctl(struct tty_struct *tty, str
perm=0;
return do_kbkeycode_ioctl(cmd, (struct kbkeycode *)arg, perm);

+ case KDGKBENT_OLD:
+ case KDSKBENT_OLD:
case KDGKBENT:
case KDSKBENT:
- return do_kdsk_ioctl(cmd, (struct kbentry *)arg, perm, kbd);
+ return do_kdsk_ioctl(cmd, (void *)arg, perm, kbd);

case KDGKBSENT:
case KDSKBSENT:
diff -puN include/linux/kd.h~kbentry-fix include/linux/kd.h
--- linux-2.6.0-test1/include/linux/kd.h~kbentry-fix 2003-07-20 00:51:05.000000000 +0900
+++ linux-2.6.0-test1-hirofumi/include/linux/kd.h 2003-07-20 00:51:05.000000000 +0900
@@ -94,18 +94,26 @@ struct unimapinit {
#define KDGKBLED 0x4B64 /* get led flags (not lights) */
#define KDSKBLED 0x4B65 /* set led flags (not lights) */

-struct kbentry {
+struct kbentry_old {
unsigned char kb_table;
unsigned char kb_index;
unsigned short kb_value;
};
+#define KDGKBENT_OLD 0x4B46 /* gets one entry in translation table (old) */
+#define KDSKBENT_OLD 0x4B47 /* sets one entry in translation table (old) */
+
+struct kbentry {
+ unsigned int kb_table;
+ unsigned int kb_index;
+ unsigned short kb_value;
+};
#define K_NORMTAB 0x00
#define K_SHIFTTAB 0x01
#define K_ALTTAB 0x02
#define K_ALTSHIFTTAB 0x03

-#define KDGKBENT 0x4B46 /* gets one entry in translation table */
-#define KDSKBENT 0x4B47 /* sets one entry in translation table */
+#define KDGKBENT 0x4B73 /* gets one entry in translation table */
+#define KDSKBENT 0x4B74 /* sets one entry in translation table */

struct kbsentry {
unsigned char kb_func;

_

2003-07-24 11:02:24

by Hiroshi Miura

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

I can use Japanese keyboard.
I DO recompile console-tools with header file in linux-2.5/2.6.
Because of NR_KEYS change, as Ogawa-san said,
in 2.4 NR_KEYS=128, in 2.6 NR_KEYS= 0x200

When recompile, some tools fails compile on linux-2.6,
but 'loadkeys' compiles nicely.

Instantly, user can set new keymap to use a recompiled loadkey command.

BTW, Ogawa-san's patch is needed?
I think this fix should made on user-mode tool 'console-tools'.
As same as module-init-tools, console-tools should detect kernel version
and switch type of interface, doesn't it?

Junkio's patch looks reasonable.

OGAWA Hirofumi wrote:
> Andries Brouwer <[email protected]> writes:
>
>
>>On Tue, Jul 22, 2003 at 10:56:33PM +0900, Norman Diamond wrote:
>>
>>
>>>On a Japanese PS/2 keyboard
>>
>>I did not read your long message but stopped after the above words.
>>Sorry if this is not an answer (ask again).
>>
>>For 2.6.0t1 it helps to add the line
>>
>> keycode 183 = backslash bar
>>
>>to your keymap.
>
>
> I remembered this problem. At 2.4.x kbd tools use "#define NR_KEYS 128".
> So, we can't set >= 128.
>
> Currently NR_KEYS is 0x200 (KEY_MAX+1). We can't only recompile
> because ->kb_index (struct kbentry) type using "unsigned char".
>
> What do you think the following patch? (it may be needed to cleanup or
> rewrite)
>
> Thanks.

Hiroshi Miura
[email protected]

2003-07-24 14:12:12

by OGAWA Hirofumi

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

Hiroshi Miura <[email protected]> writes:

> I can use Japanese keyboard.
> I DO recompile console-tools with header file in linux-2.5/2.6.
> Because of NR_KEYS change, as Ogawa-san said,
> in 2.4 NR_KEYS=128, in 2.6 NR_KEYS= 0x200
>
> When recompile, some tools fails compile on linux-2.6,
> but 'loadkeys' compiles nicely.
>
> Instantly, user can set new keymap to use a recompiled loadkey command.
>
> BTW, Ogawa-san's patch is needed?

For example, in loadkeys.y

if ((keymap_was_set[i])[j])
{
ke.kb_index = j;
ke.kb_table = i;
ke.kb_value = (key_map[i])[j];

ke.kb_index overflowed. Likewise dumpkeys has same problem.
And debian using in init.d/console-screen.sh,

dumpkeys < ${DEVICE_PREFIX}1 |sed -f /etc/console-tools/remap |loadkeys

Important part of my patch is the following. Other part was for
backward compatible.

+struct kbentry {
+ unsigned int kb_table;
+ unsigned int kb_index;
+ unsigned short kb_value;
+};

--
OGAWA Hirofumi <[email protected]>

2003-07-25 00:45:58

by Junio C Hamano

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

>>>>> "HM" == Hiroshi Miura <[email protected]> writes:

HM> BTW, Ogawa-san's patch is needed?

The part of his patch that enlarges the type of kbentry.kb_table
and kbentry.kb_index are needed if you ever want to use keycode
above 255. These indices are defined as unsigned char, but
NR_KEYS in 2.6 is 512 (see include/linux/keyboard.h and
include/linux/input.h) as opposed to 128.

If you are not interested in using keycode above 255 (e.g. all
you care about is keycode 183 for Japanese 86/106 keyboard),
then the patch I sent to the list should be sufficient.

2003-07-25 04:01:56

by OGAWA Hirofumi

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

> If you are not interested in using keycode above 255 (e.g. all
> you care about is keycode 183 for Japanese 86/106 keyboard),
> then the patch I sent to the list should be sufficient.

Basically yes. But was the following case solved?

dumpkeys < ${DEVICE_PREFIX}1 |sed -f /etc/console-tools/remap |loadkeys
--
OGAWA Hirofumi <[email protected]>

2003-07-25 07:33:56

by Clemens Schwaighofer

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Norman Diamond wrote:

cheap, but working and I think it will stay so until 2.6 goes into final
of distris:

setkeycodes 0x6a 124 1>&2 in your rc.local, local.start or whatever.
works fine for me for alle 2.5x kernels

- --
Clemens Schwaighofer - IT Engineer & System Administration
==========================================================
Tequila Japan, 6-17-2 Ginza Chuo-ku, Tokyo 104-8167, JAPAN
Tel: +81-(0)3-3545-7703 Fax: +81-(0)3-3545-7343
http://www.tequila.jp
==========================================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE/IODtjBz/yQjBxz8RApisAKCG28H8qJQgvpOmOsEmm9+SiPx61ACcDlo2
OuF4YVp0i2KhdTRnZD6fzvY=
=xEBX
-----END PGP SIGNATURE-----

2003-07-25 09:02:33

by John Bradford

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

> cheap, but working and I think it will stay so until 2.6 goes into final
> of distris:
>
> setkeycodes 0x6a 124 1>&2 in your rc.local, local.start or whatever.
> works fine for me for alle 2.5x kernels

Is your keyboard detected as a set2 or set3 keyboard?

Mine is detected as set2, but will also work in set3. When I get
chance, I'll make a patch to detect it and set set3 automatically.

In the default set2 it emulates a US keyboard - pressing : for
example, generates the make codes for shift, and ;, then the break
codes. The keys beside the spacebar produce the spacebar make and
break codes, and cannot be used independantly in set2.

I'm curious as to whether set3 is generally supported by Japanese
keyboards, or not, even if they are not detected as set3.

John.

2003-07-25 10:15:46

by Norman Diamond

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

"Clemens Schwaighofer" <[email protected]> wrote:

> cheap, but working and I think it will stay so until 2.6 goes into final
> of distris:
>
> setkeycodes 0x6a 124 1>&2 in your rc.local, local.start or whatever.
> works fine for me for alle 2.5x kernels

There must be a ton of odd stuff going on. "showkey" used to say that the
scan code is 0x7d not 0x6a, but now it displays weird stuff. As previously
mentioned, "getkeycodes" displays a table which seems far removed from
reality. But the patch from [email protected] worked (but "showkey" and
"getkeycodes" still produce weird output).

-- Norman Diamond

2003-07-25 15:27:24

by Andries Brouwer

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

On Fri, Jul 25, 2003 at 07:30:04PM +0900, Norman Diamond wrote:

> > setkeycodes 0x6a 124 1>&2 in your rc.local, local.start or whatever.
> > works fine for me for alle 2.5x kernels
>
> There must be a ton of odd stuff going on. "showkey" used to say that the
> scan code is 0x7d not 0x6a, but now it displays weird stuff. As previously
> mentioned, "getkeycodes" displays a table which seems far removed from
> reality. But the patch from [email protected] worked (but "showkey" and
> "getkeycodes" still produce weird output).

Yes.

I am a little bit unhappy with the state of the kbd code
(but have not yet decided whether I want to attempt to fix something).

One aspect of the matter is that raw mode no longer is raw.
The keyboard sends codes and the input layer translates that into
the codes the input layer thinks the keyboard should have sent.
Then, when one wants the raw codes, a reverse translation is used,
but since the mapping is not one-to-one the reverse translation
does not produce what the keyboard sent to start with.

In particular this means that showkey is no longer useful under 2.5
if one wants to investigate the scancodes a keyboard sends.

Andries


[PS - Some ioctls are broken. Recompiling kernel and loadkeys/dumpkeys
with larger structures works for private use, but the kernel ABI must
not change, so for actual kernel patches one needs new ioctl numbers.]

2003-07-25 15:37:25

by John Bradford

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

> > > setkeycodes 0x6a 124 1>&2 in your rc.local, local.start or whatever.
> > > works fine for me for alle 2.5x kernels
> >
> > There must be a ton of odd stuff going on. "showkey" used to say that the
> > scan code is 0x7d not 0x6a, but now it displays weird stuff. As previously
> > mentioned, "getkeycodes" displays a table which seems far removed from
> > reality. But the patch from [email protected] worked (but "showkey" and
> > "getkeycodes" still produce weird output).
>
> Yes.
>
> I am a little bit unhappy with the state of the kbd code
> (but have not yet decided whether I want to attempt to fix something).
>
> One aspect of the matter is that raw mode no longer is raw.
> The keyboard sends codes and the input layer translates that into
> the codes the input layer thinks the keyboard should have sent.
> Then, when one wants the raw codes, a reverse translation is used,
> but since the mapping is not one-to-one the reverse translation
> does not produce what the keyboard sent to start with.

Doesn't AT-set3 usually have a closer one to one mapping of keys?

John.

2003-07-25 15:54:45

by Andries Brouwer

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

On Fri, Jul 25, 2003 at 05:01:40PM +0100, John Bradford wrote:

> > One aspect of the matter is that raw mode no longer is raw.
> > The keyboard sends codes and the input layer translates that into
> > the codes the input layer thinks the keyboard should have sent.
> > Then, when one wants the raw codes, a reverse translation is used,
> > but since the mapping is not one-to-one the reverse translation
> > does not produce what the keyboard sent to start with.
>
> Doesn't AT-set3 usually have a closer one to one mapping of keys?

Sorry - I am unable to make sense of your question.

Below some remarks, maybe related.

Some remarks on scancode sets live on
http://www.win.tue.nl/~aeb/linux/kbd/scancodes-7.html

Not all keyboards support all scancode sets. For example,
my MyCom laptop only supports scancode Set 2, and its keyboard
does not react at all when in mode 1 or 3.

The normal, default mode is translated scancode set 2.
Putting keyboards in other modes is asking for trouble.

Andries

2003-07-25 16:14:10

by John Bradford

[permalink] [raw]
Subject: Re: Japanese keyboards broken in 2.6

> > > One aspect of the matter is that raw mode no longer is raw.
> > > The keyboard sends codes and the input layer translates that into
> > > the codes the input layer thinks the keyboard should have sent.
> > > Then, when one wants the raw codes, a reverse translation is used,
> > > but since the mapping is not one-to-one the reverse translation
> > > does not produce what the keyboard sent to start with.
> >
> > Doesn't AT-set3 usually have a closer one to one mapping of keys?
>
> Sorry - I am unable to make sense of your question.

The reason I mentioned it is because I've got a keyboard which
physically has a Japanese layout, but emulates a US keyboard in set 2.

Some keys which are shifted on a US keyboard, are not shifted on a
Japanese keyboard, for example, the colon key. Presing colon on my
keyboard in set 2, causes it to send shift-; on the wire. In set 3,
it sends the single code for the colon key.

I wondered whether some users in this thread had keyboards like mine,
which seem to produce unusual results when you analyse the scancodes
coming from them. It looks like mine is rarer than I thought :-).

> Below some remarks, maybe related.
>
> Some remarks on scancode sets live on
> http://www.win.tue.nl/~aeb/linux/kbd/scancodes-7.html
>
> Not all keyboards support all scancode sets. For example,
> my MyCom laptop only supports scancode Set 2, and its keyboard
> does not react at all when in mode 1 or 3.
>
> The normal, default mode is translated scancode set 2.
> Putting keyboards in other modes is asking for trouble.

Agreed. Unfortunately, it's impossible to use some of the keys in set
2 on mine :-).

John.