Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755258AbcD0Aql (ORCPT ); Tue, 26 Apr 2016 20:46:41 -0400 Received: from shadbolt.e.decadent.org.uk ([88.96.1.126]:39587 "EHLO shadbolt.e.decadent.org.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753241AbcDZXN1 (ORCPT ); Tue, 26 Apr 2016 19:13:27 -0400 Content-Type: text/plain; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 From: Ben Hutchings To: linux-kernel@vger.kernel.org, stable@vger.kernel.org CC: akpm@linux-foundation.org, "Hans de Goede" , "Mauro Carvalho Chehab" , "Hans Verkuil" Date: Wed, 27 Apr 2016 01:02:24 +0200 Message-ID: X-Mailer: LinuxStableQueue (scripts by bwh) Subject: [PATCH 3.2 006/115] [media] saa7134: Fix bytesperline not being set correctly for planar formats In-Reply-To: X-SA-Exim-Connect-IP: 2a02:8426:ae4:c500:9cba:69ae:962d:6167 X-SA-Exim-Mail-From: ben@decadent.org.uk X-SA-Exim-Scanned: No (on shadbolt.decadent.org.uk); SAEximRunCond expanded to false Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2061 Lines: 61 3.2.80-rc1 review patch. If anyone has any objections, please let me know. ------------------ From: Hans de Goede commit 3e71da19f9dc22e39a755d6ae9678661abb66adc upstream. bytesperline should be the bytesperline for the first plane for planar formats, not that of all planes combined. This fixes a crash in xawtv caused by the wrong bpl. BugLink: https://bugzilla.redhat.com/show_bug.cgi?id=1305389 Reported-and-tested-by: Stas Sergeev Signed-off-by: Hans de Goede Signed-off-by: Hans Verkuil Signed-off-by: Mauro Carvalho Chehab [bwh: Backported to 3.2: adjust filename, context] Signed-off-by: Ben Hutchings --- drivers/media/video/saa7134/saa7134-video.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) --- a/drivers/media/video/saa7134/saa7134-video.c +++ b/drivers/media/video/saa7134/saa7134-video.c @@ -1577,10 +1577,13 @@ static int saa7134_g_fmt_vid_cap(struct f->fmt.pix.height = fh->height; f->fmt.pix.field = fh->cap.field; f->fmt.pix.pixelformat = fh->fmt->fourcc; - f->fmt.pix.bytesperline = - (f->fmt.pix.width * fh->fmt->depth) >> 3; + if (fh->fmt->planar) + f->fmt.pix.bytesperline = f->fmt.pix.width; + else + f->fmt.pix.bytesperline = + (f->fmt.pix.width * fh->fmt->depth) / 8; f->fmt.pix.sizeimage = - f->fmt.pix.height * f->fmt.pix.bytesperline; + (f->fmt.pix.height * f->fmt.pix.width * fh->fmt->depth) / 8; return 0; } @@ -1641,10 +1644,13 @@ static int saa7134_try_fmt_vid_cap(struc if (f->fmt.pix.height > maxh) f->fmt.pix.height = maxh; f->fmt.pix.width &= ~0x03; - f->fmt.pix.bytesperline = - (f->fmt.pix.width * fmt->depth) >> 3; + if (fmt->planar) + f->fmt.pix.bytesperline = f->fmt.pix.width; + else + f->fmt.pix.bytesperline = + (f->fmt.pix.width * fmt->depth) / 8; f->fmt.pix.sizeimage = - f->fmt.pix.height * f->fmt.pix.bytesperline; + (f->fmt.pix.height * f->fmt.pix.width * fmt->depth) / 8; return 0; }