Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754982Ab0LHAe0 (ORCPT ); Tue, 7 Dec 2010 19:34:26 -0500 Received: from kroah.org ([198.145.64.141]:54753 "EHLO coco.kroah.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754940Ab0LHAeZ (ORCPT ); Tue, 7 Dec 2010 19:34:25 -0500 X-Mailbox-Line: From gregkh@clark.site Tue Dec 7 16:06:40 2010 Message-Id: <20101208000640.714274583@clark.site> User-Agent: quilt/0.48-11.2 Date: Tue, 07 Dec 2010 16:04:11 -0800 From: Greg KH To: linux-kernel@vger.kernel.org, stable@kernel.org Cc: stable-review@kernel.org, torvalds@linux-foundation.org, akpm@linux-foundation.org, alan@lxorguk.ukuu.org.uk, Graham Gower , Greg KH Subject: [12/44] drivers/char/vt_ioctl.c: fix VT_OPENQRY error value In-Reply-To: <20101208003205.GA4286@kroah.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2330 Lines: 78 2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Graham Gower commit 1e0ad2881d50becaeea70ec696a80afeadf944d2 upstream. When all VT's are in use, VT_OPENQRY casts -1 to unsigned char before returning it to userspace as an int. VT255 is not the next available console. Signed-off-by: Graham Gower Cc: Greg KH Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman --- drivers/char/vt_ioctl.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) --- a/drivers/char/vt_ioctl.c +++ b/drivers/char/vt_ioctl.c @@ -371,6 +371,7 @@ int vt_ioctl(struct tty_struct *tty, str struct kbd_struct * kbd; unsigned int console; unsigned char ucval; + unsigned int uival; void __user *up = (void __user *)arg; int i, perm; int ret = 0; @@ -516,7 +517,7 @@ int vt_ioctl(struct tty_struct *tty, str break; case KDGETMODE: - ucval = vc->vc_mode; + uival = vc->vc_mode; goto setint; case KDMAPDISP: @@ -554,7 +555,7 @@ int vt_ioctl(struct tty_struct *tty, str break; case KDGKBMODE: - ucval = ((kbd->kbdmode == VC_RAW) ? K_RAW : + uival = ((kbd->kbdmode == VC_RAW) ? K_RAW : (kbd->kbdmode == VC_MEDIUMRAW) ? K_MEDIUMRAW : (kbd->kbdmode == VC_UNICODE) ? K_UNICODE : K_XLATE); @@ -576,9 +577,9 @@ int vt_ioctl(struct tty_struct *tty, str break; case KDGKBMETA: - ucval = (vc_kbd_mode(kbd, VC_META) ? K_ESCPREFIX : K_METABIT); + uival = (vc_kbd_mode(kbd, VC_META) ? K_ESCPREFIX : K_METABIT); setint: - ret = put_user(ucval, (int __user *)arg); + ret = put_user(uival, (int __user *)arg); break; case KDGETKEYCODE: @@ -808,7 +809,7 @@ int vt_ioctl(struct tty_struct *tty, str for (i = 0; i < MAX_NR_CONSOLES; ++i) if (! VT_IS_IN_USE(i)) break; - ucval = i < MAX_NR_CONSOLES ? (i+1) : -1; + uival = i < MAX_NR_CONSOLES ? (i+1) : -1; goto setint; /* -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/