Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp3070142imu; Fri, 18 Jan 2019 04:22:36 -0800 (PST) X-Google-Smtp-Source: ALg8bN65xmB+erri8H3S4HHp9qEh4FlYxk6g8bm1Vk66ry1jNOHc/1TLfnJYZ065o9fiFk44W66i X-Received: by 2002:a17:902:a6:: with SMTP id a35mr19029940pla.201.1547814156346; Fri, 18 Jan 2019 04:22:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547814156; cv=none; d=google.com; s=arc-20160816; b=qrTmJhPObIy55DaXX1mX4nxC7pIGX5qlJ/vDOUDY3na3Cx9nvibIGBWLeq9q1ufgmT /5aY5HhvivlELy9j2q90XEBBbRF2S5Fhp0HDExN2f3Sbcb45RM5nLccE7+0ftUGvBdLO ZJhAtUF5nkfPIEk+Yogvc/xlzmu+YREdhDZAmR1XQaP4aeS41ECKrxqXhENz+oNW8DgP kI7hFEOrr4Oyj4CDvvwMp+ZE4Oh7J3Fftj+ZEg5H3SOfDAeBOKAoUw4gMkSzfvrGRRZV vb57+EUCD1ZkNGVU1cOZHwjH5oyLhg4KAKCpldJcP9VsDuiiQEsqCEUjIis0pkW2skMB fbNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=BZ8YSkpfFuxsKSDE1WI/Mr+Oq/HSLAosqUmIPrpxkEA=; b=0O2ADRT7CzxAvfyrFUXeQdXREaP8XB3JVrAxsyGMq68P+tIPnIYKdUQlRuikzmwwr5 OV6SpaneK26Pi504C5eiEp7WFf4Ljv/e1B/yKvTlCbjYehFh8kl0SW/JX05Bzf+1U29i qRdRgTGMSdeGbET4dIrOeX/jZmG4wD0sSVWHJs1uq2my9qeziEDoAW499+WWyYuRKZjs 8U7yNbT95vsm/PSO/evubs8j712kbQwTyb8Dhq9uoiEh6+hi44C2TorSM+c8EB+K3rMG EVnDUh8DM8vV0WIyIdPOcnNx6MQC9AJFYbUjJeNqC9LYe8OT5moEn9paW1kcFA+MkPzu i7ZA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p5si4559140pfb.188.2019.01.18.04.22.20; Fri, 18 Jan 2019 04:22:36 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727768AbfARMUz (ORCPT + 99 others); Fri, 18 Jan 2019 07:20:55 -0500 Received: from mx1.redhat.com ([209.132.183.28]:47010 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727701AbfARMUg (ORCPT ); Fri, 18 Jan 2019 07:20:36 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D3862DACFE; Fri, 18 Jan 2019 12:20:36 +0000 (UTC) Received: from sirius.home.kraxel.org (ovpn-116-38.ams2.redhat.com [10.36.116.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 812E5600C2; Fri, 18 Jan 2019 12:20:28 +0000 (UTC) Received: by sirius.home.kraxel.org (Postfix, from userid 1000) id A454051B80; Fri, 18 Jan 2019 13:20:23 +0100 (CET) From: Gerd Hoffmann To: dri-devel@lists.freedesktop.org Cc: Gerd Hoffmann , Dave Airlie , David Airlie , Daniel Vetter , virtualization@lists.linux-foundation.org (open list:DRM DRIVER FOR QXL VIRTUAL GPU), spice-devel@lists.freedesktop.org (open list:DRM DRIVER FOR QXL VIRTUAL GPU), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v3 23/23] drm/qxl: add overflow checks to qxl_mode_dumb_create() Date: Fri, 18 Jan 2019 13:20:20 +0100 Message-Id: <20190118122020.27596-24-kraxel@redhat.com> In-Reply-To: <20190118122020.27596-1-kraxel@redhat.com> References: <20190118122020.27596-1-kraxel@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 18 Jan 2019 12:20:36 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Signed-off-by: Gerd Hoffmann --- drivers/gpu/drm/qxl/qxl_dumb.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/qxl/qxl_dumb.c b/drivers/gpu/drm/qxl/qxl_dumb.c index 272d19b677..bed6d06ee4 100644 --- a/drivers/gpu/drm/qxl/qxl_dumb.c +++ b/drivers/gpu/drm/qxl/qxl_dumb.c @@ -37,11 +37,13 @@ int qxl_mode_dumb_create(struct drm_file *file_priv, uint32_t handle; int r; struct qxl_surface surf; - uint32_t pitch, format; + uint32_t pitch, size, format; - pitch = args->width * ((args->bpp + 1) / 8); - args->size = pitch * args->height; - args->size = ALIGN(args->size, PAGE_SIZE); + if (check_mul_overflow(args->width, ((args->bpp + 1) / 8), &pitch)) + return -EINVAL; + if (check_mul_overflow(pitch, args->height, &size)) + return -EINVAL; + args->size = ALIGN(size, PAGE_SIZE); switch (args->bpp) { case 16: -- 2.9.3