Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758887AbXFELfU (ORCPT ); Tue, 5 Jun 2007 07:35:20 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752786AbXFELfI (ORCPT ); Tue, 5 Jun 2007 07:35:08 -0400 Received: from wa-out-1112.google.com ([209.85.146.182]:54344 "EHLO wa-out-1112.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750837AbXFELfG (ORCPT ); Tue, 5 Jun 2007 07:35:06 -0400 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:user-agent:mime-version:to:cc:subject:references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding; b=uZGuIgJzVCCEhLkR2BZmVUsixGpJkK1aKhq/p1SIoyQxdZ2Hv3GO1X7Dvyr9NggQ2MjwCutoPkgp7Do/Uq2obERRhooai7/L9qSjAOhaUa3qLC111R06F25LttiZFjNZ/DrQ/qq38Wv46kj3O8jAYrXbx6e3CKZnNvTv2Btc3iE= Message-ID: <46654A63.6000308@gmail.com> Date: Tue, 05 Jun 2007 19:34:59 +0800 From: "Antonino A. Daplas" User-Agent: Thunderbird 1.5.0.10 (X11/20060911) MIME-Version: 1.0 To: Chris Wright CC: Linus Torvalds , stable@kernel.org, Andrew Morton , LKML , Pekka Enberg , Tero Roponen Subject: [PATCH] [RESEND] neofb: Fix pseudo_palette array overrun in neofb_setcolreg References: <465C188F.9000900@googlemail.com> <465C222A.8060003@googlemail.com> <1180587109.4570.25.camel@daplas> In-Reply-To: X-Enigmail-Version: 0.94.2.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2227 Lines: 86 The pseudo_palette has room for 16 entries only, but in truecolor mode, it attempts to write 256. Signed-off-by: Antonino Daplas Acked-by: Tero Roponen --- This fixes the following regression/bug reported as follows: Subject : tty-related oops in latest kernel(s) References : http://lkml.org/lkml/2007/5/27/104 Submitter : Tero Roponen Status : problem is being debugged According to Tero, this is also reproducible with 2.6.21.3. (Resending, wrong email address for stable@kernel.org) Tony drivers/video/neofb.c | 30 ++++++++++++++++-------------- 1 files changed, 16 insertions(+), 14 deletions(-) diff --git a/drivers/video/neofb.c b/drivers/video/neofb.c index bd30aba..731d7a5 100644 --- a/drivers/video/neofb.c +++ b/drivers/video/neofb.c @@ -1286,34 +1286,36 @@ static int neofb_setcolreg(u_int regno, if (regno >= fb->cmap.len || regno > 255) return -EINVAL; - switch (fb->var.bits_per_pixel) { - case 8: + if (fb->var.bits_per_pixel <= 8) { outb(regno, 0x3c8); outb(red >> 10, 0x3c9); outb(green >> 10, 0x3c9); outb(blue >> 10, 0x3c9); - break; - case 16: - ((u32 *) fb->pseudo_palette)[regno] = + } else if (regno < 16) { + switch (fb->var.bits_per_pixel) { + case 16: + ((u32 *) fb->pseudo_palette)[regno] = ((red & 0xf800)) | ((green & 0xfc00) >> 5) | ((blue & 0xf800) >> 11); - break; - case 24: - ((u32 *) fb->pseudo_palette)[regno] = + break; + case 24: + ((u32 *) fb->pseudo_palette)[regno] = ((red & 0xff00) << 8) | ((green & 0xff00)) | ((blue & 0xff00) >> 8); - break; + break; #ifdef NO_32BIT_SUPPORT_YET - case 32: - ((u32 *) fb->pseudo_palette)[regno] = + case 32: + ((u32 *) fb->pseudo_palette)[regno] = ((transp & 0xff00) << 16) | ((red & 0xff00) << 8) | ((green & 0xff00)) | ((blue & 0xff00) >> 8); - break; + break; #endif - default: - return 1; + default: + return 1; + } } + return 0; } - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/