Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759526AbcDBBbJ (ORCPT ); Fri, 1 Apr 2016 21:31:09 -0400 Received: from youngberry.canonical.com ([91.189.89.112]:54042 "EHLO youngberry.canonical.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756653AbcDBAzb (ORCPT ); Fri, 1 Apr 2016 20:55:31 -0400 From: Kamal Mostafa To: linux-kernel@vger.kernel.org, stable@vger.kernel.org, kernel-team@lists.ubuntu.com Cc: Hans de Goede , Mauro Carvalho Chehab , Kamal Mostafa Subject: [PATCH 3.19.y-ckt 051/170] [media] bttv: Width must be a multiple of 16 when capturing planar formats Date: Fri, 1 Apr 2016 17:52:17 -0700 Message-Id: <1459558456-24452-52-git-send-email-kamal@canonical.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1459558456-24452-1-git-send-email-kamal@canonical.com> References: <1459558456-24452-1-git-send-email-kamal@canonical.com> X-Extended-Stable: 3.19 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3794 Lines: 104 3.19.8-ckt18 -stable review patch. If anyone has any objections, please let me know. ---8<------------------------------------------------------------ From: Hans de Goede commit 5c915c68763889f0183a1cc61c84bb228b60124a upstream. On my bttv card "Hauppauge WinTV [card=10]" capturing in YV12 fmt at max size results in a solid green rectangle being captured (all colors 0 in YUV). This turns out to be caused by max-width (924) not being a multiple of 16. We've likely never hit this problem before since normally xawtv / tvtime, etc. will prefer packed pixel formats. But when using a video card which is using xf86-video-modesetting + glamor, only planar XVideo fmts are available, and xawtv will chose a matching capture format to avoid needing to do conversion, triggering the solid green window problem. Signed-off-by: Hans de Goede Acked-by: Hans Verkuil Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Kamal Mostafa --- drivers/media/pci/bt8xx/bttv-driver.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/drivers/media/pci/bt8xx/bttv-driver.c b/drivers/media/pci/bt8xx/bttv-driver.c index 4a8176c..dbb00bf 100644 --- a/drivers/media/pci/bt8xx/bttv-driver.c +++ b/drivers/media/pci/bt8xx/bttv-driver.c @@ -2332,6 +2332,19 @@ static int bttv_g_fmt_vid_overlay(struct file *file, void *priv, return 0; } +static void bttv_get_width_mask_vid_cap(const struct bttv_format *fmt, + unsigned int *width_mask, + unsigned int *width_bias) +{ + if (fmt->flags & FORMAT_FLAGS_PLANAR) { + *width_mask = ~15; /* width must be a multiple of 16 pixels */ + *width_bias = 8; /* nearest */ + } else { + *width_mask = ~3; /* width must be a multiple of 4 pixels */ + *width_bias = 2; /* nearest */ + } +} + static int bttv_try_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f) { @@ -2341,6 +2354,7 @@ static int bttv_try_fmt_vid_cap(struct file *file, void *priv, enum v4l2_field field; __s32 width, height; __s32 height2; + unsigned int width_mask, width_bias; int rc; fmt = format_by_fourcc(f->fmt.pix.pixelformat); @@ -2373,9 +2387,9 @@ static int bttv_try_fmt_vid_cap(struct file *file, void *priv, width = f->fmt.pix.width; height = f->fmt.pix.height; + bttv_get_width_mask_vid_cap(fmt, &width_mask, &width_bias); rc = limit_scaled_size_lock(fh, &width, &height, field, - /* width_mask: 4 pixels */ ~3, - /* width_bias: nearest */ 2, + width_mask, width_bias, /* adjust_size */ 1, /* adjust_crop */ 0); if (0 != rc) @@ -2408,6 +2422,7 @@ static int bttv_s_fmt_vid_cap(struct file *file, void *priv, struct bttv_fh *fh = priv; struct bttv *btv = fh->btv; __s32 width, height; + unsigned int width_mask, width_bias; enum v4l2_field field; retval = bttv_switch_type(fh, f->type); @@ -2422,9 +2437,10 @@ static int bttv_s_fmt_vid_cap(struct file *file, void *priv, height = f->fmt.pix.height; field = f->fmt.pix.field; + fmt = format_by_fourcc(f->fmt.pix.pixelformat); + bttv_get_width_mask_vid_cap(fmt, &width_mask, &width_bias); retval = limit_scaled_size_lock(fh, &width, &height, f->fmt.pix.field, - /* width_mask: 4 pixels */ ~3, - /* width_bias: nearest */ 2, + width_mask, width_bias, /* adjust_size */ 1, /* adjust_crop */ 1); if (0 != retval) @@ -2432,8 +2448,6 @@ static int bttv_s_fmt_vid_cap(struct file *file, void *priv, f->fmt.pix.field = field; - fmt = format_by_fourcc(f->fmt.pix.pixelformat); - /* update our state informations */ fh->fmt = fmt; fh->cap.field = f->fmt.pix.field; -- 2.7.4