Received: by 2002:a05:6622:f08:0:0:0:0 with SMTP id l8csp4481812ivc; Tue, 3 Nov 2020 13:10:54 -0800 (PST) X-Google-Smtp-Source: ABdhPJxJtSJKW8OSOVNk/kvkcXnFAZvGlY7wshtjjqfjkSjqcooD8FGtZjt5ydKvGUMMir8ttahW X-Received: by 2002:a17:906:d8b0:: with SMTP id qc16mr21439301ejb.268.1604437854310; Tue, 03 Nov 2020 13:10:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1604437854; cv=none; d=google.com; s=arc-20160816; b=VA/0jMiD4UJe/KbBobre0Zy3vom3l8vz7mLxKPmknnqmxK4BUyrJtB71eIPG87jaTX 1KbD8c+3ow6wfJ2RhQh8V8OeOEOcIe+mlcjbm0uTUVdkRWrHtuTUfZHPeiwxNvah61W9 6npNw0G34WFJazRvp7RMNIONYy7Z7zOQI+qYkI7qdmfLvvHoTrDVAH/4OzejHZuAyrrN YepHOV5b8p+Vpq3A0ol6xWs/mfAvyi3MV7WK9erF1+LgMXNIITw3CN/XWIt+WOuKJatd Ysc22JS8xK4EPS/167NeOZEVyWXAJ+kp25kKFDVSViMweFyPF6IdkiRpd++CB5KXBXTG 7pGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=yzgF9mxOxX8DJY3hWrC9XCnNAmDPcj3d/iJ3h8RFeTo=; b=FmIXP7iRcYfbweElvkPvi6zt62IvMTwdkl+kG2L4sjmC4eU/wxBOS8wPub5Xrqde2+ HJEcTbSSSr+RXR0vM8QdShsdBjvR1runfsYnJAVNnzSjn43WQYm78LDPHY2FAYKvzzhH PIkGlyX9fdpCzX3/7u9ynr1DAPKHmEMI5/JFZIPt9Fvo+r67Ar12+JUVS0nbym+W6oWb +BsCj7C4djxCxM7mcarTeE6bNliwt4h+QlwM49GAmQ0r8NrrbMMJW6cOEMx+ln2OsMz/ XYtUYktQvySlNn6uzbf6uKm6/4J/kykXqFoTL/iAEjusy0++DQ0cNipHctgJ+YA65vFE spMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=Nx27K3wc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id a13si1592595edr.154.2020.11.03.13.10.30; Tue, 03 Nov 2020 13:10:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=Nx27K3wc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387765AbgKCVHV (ORCPT + 99 others); Tue, 3 Nov 2020 16:07:21 -0500 Received: from mail.kernel.org ([198.145.29.99]:46332 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388264AbgKCVHK (ORCPT ); Tue, 3 Nov 2020 16:07:10 -0500 Received: from localhost (83-86-74-64.cable.dynamic.v4.ziggo.nl [83.86.74.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3A0B3206B5; Tue, 3 Nov 2020 21:07:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604437629; bh=y7ZK+pDC5G/MNWFHAgykspgjpyRitgHR80fbbwpq6o0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nx27K3wcbu+twqUiYfBMQB629M6Ilc5jj813I4KWM7nEzaDQWzfhdOfAk7tC6qRgv To7vY/BJ8LV7Q2w7EmCebJ4GQplxTs3D1FviZ3PvDHmn03mlp5204K8FJlm7ypNgkJ HWO2coZMoKP8eFy2U8e50FcRjUmILfHFzPpkx7dQ= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Oliver OHalloran , Mahesh Salgaonkar , "Aneesh Kumar K.V" , Vasant Hegde , Michael Ellerman Subject: [PATCH 4.19 157/191] powerpc/powernv/elog: Fix race while processing OPAL error log event. Date: Tue, 3 Nov 2020 21:37:29 +0100 Message-Id: <20201103203247.241977386@linuxfoundation.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201103203232.656475008@linuxfoundation.org> References: <20201103203232.656475008@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mahesh Salgaonkar commit aea948bb80b478ddc2448f7359d574387521a52d upstream. Every error log reported by OPAL is exported to userspace through a sysfs interface and notified using kobject_uevent(). The userspace daemon (opal_errd) then reads the error log and acknowledges the error log is saved safely to disk. Once acknowledged the kernel removes the respective sysfs file entry causing respective resources to be released including kobject. However it's possible the userspace daemon may already be scanning elog entries when a new sysfs elog entry is created by the kernel. User daemon may read this new entry and ack it even before kernel can notify userspace about it through kobject_uevent() call. If that happens then we have a potential race between elog_ack_store->kobject_put() and kobject_uevent which can lead to use-after-free of a kernfs object resulting in a kernel crash. eg: BUG: Unable to handle kernel data access on read at 0x6b6b6b6b6b6b6bfb Faulting instruction address: 0xc0000000008ff2a0 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA PowerNV CPU: 27 PID: 805 Comm: irq/29-opal-elo Not tainted 5.9.0-rc2-gcc-8.2.0-00214-g6f56a67bcbb5-dirty #363 ... NIP kobject_uevent_env+0xa0/0x910 LR elog_event+0x1f4/0x2d0 Call Trace: 0x5deadbeef0000122 (unreliable) elog_event+0x1f4/0x2d0 irq_thread_fn+0x4c/0xc0 irq_thread+0x1c0/0x2b0 kthread+0x1c4/0x1d0 ret_from_kernel_thread+0x5c/0x6c This patch fixes this race by protecting the sysfs file creation/notification by holding a reference count on kobject until we safely send kobject_uevent(). The function create_elog_obj() returns the elog object which if used by caller function will end up in use-after-free problem again. However, the return value of create_elog_obj() function isn't being used today and there is no need as well. Hence change it to return void to make this fix complete. Fixes: 774fea1a38c6 ("powerpc/powernv: Read OPAL error log and export it through sysfs") Cc: stable@vger.kernel.org # v3.15+ Reported-by: Oliver O'Halloran Signed-off-by: Mahesh Salgaonkar Signed-off-by: Aneesh Kumar K.V Reviewed-by: Oliver O'Halloran Reviewed-by: Vasant Hegde [mpe: Rework the logic to use a single return, reword comments, add oops] Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20201006122051.190176-1-mpe@ellerman.id.au Signed-off-by: Greg Kroah-Hartman --- arch/powerpc/platforms/powernv/opal-elog.c | 33 ++++++++++++++++++++++------- 1 file changed, 26 insertions(+), 7 deletions(-) --- a/arch/powerpc/platforms/powernv/opal-elog.c +++ b/arch/powerpc/platforms/powernv/opal-elog.c @@ -183,14 +183,14 @@ static ssize_t raw_attr_read(struct file return count; } -static struct elog_obj *create_elog_obj(uint64_t id, size_t size, uint64_t type) +static void create_elog_obj(uint64_t id, size_t size, uint64_t type) { struct elog_obj *elog; int rc; elog = kzalloc(sizeof(*elog), GFP_KERNEL); if (!elog) - return NULL; + return; elog->kobj.kset = elog_kset; @@ -223,18 +223,37 @@ static struct elog_obj *create_elog_obj( rc = kobject_add(&elog->kobj, NULL, "0x%llx", id); if (rc) { kobject_put(&elog->kobj); - return NULL; + return; } + /* + * As soon as the sysfs file for this elog is created/activated there is + * a chance the opal_errd daemon (or any userspace) might read and + * acknowledge the elog before kobject_uevent() is called. If that + * happens then there is a potential race between + * elog_ack_store->kobject_put() and kobject_uevent() which leads to a + * use-after-free of a kernfs object resulting in a kernel crash. + * + * To avoid that, we need to take a reference on behalf of the bin file, + * so that our reference remains valid while we call kobject_uevent(). + * We then drop our reference before exiting the function, leaving the + * bin file to drop the last reference (if it hasn't already). + */ + + /* Take a reference for the bin file */ + kobject_get(&elog->kobj); rc = sysfs_create_bin_file(&elog->kobj, &elog->raw_attr); - if (rc) { + if (rc == 0) { + kobject_uevent(&elog->kobj, KOBJ_ADD); + } else { + /* Drop the reference taken for the bin file */ kobject_put(&elog->kobj); - return NULL; } - kobject_uevent(&elog->kobj, KOBJ_ADD); + /* Drop our reference */ + kobject_put(&elog->kobj); - return elog; + return; } static irqreturn_t elog_event(int irq, void *data)