Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Eiichi Tsukata <eiichi.tsukata@nutanix.com>
To:     paul@paul-moore.com, eparis@redhat.com,
        linux-kernel@vger.kernel.org, audit@vger.kernel.org
Cc:     Eiichi Tsukata <eiichi.tsukata@nutanix.com>
Subject: [PATCH 4/4] audit: check if queue is full after prepare_to_wait_exclusive()
Date:   Mon,  8 May 2023 07:58:12 +0000
Message-Id: <20230508075812.76077-5-eiichi.tsukata@nutanix.com>
In-Reply-To: <20230508075812.76077-1-eiichi.tsukata@nutanix.com>
References: <20230508075812.76077-1-eiichi.tsukata@nutanix.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sAcynbSwzjloeMvpDnQQD+Z0FPTgLlYT8TLX9EN+jmFImh9yW44llFMvL7oV?=
 =?us-ascii?Q?hVcVuuA49FTc9RHPyDtbJByk5zwy5HuCK48M5VwRk9fdxZRN1hjvVvF6HYc6?=
 =?us-ascii?Q?Sl6ZRC6OUjIvK3prNbJam6p8mRj/xCkXEVAnIOUDZ4LVyJ2wQZ98tAa2J42p?=
 =?us-ascii?Q?jmOn7FTIRoZ1bqYwUSsZOhY3a6+gpdxKGNfbX8W2eD3t7NVtplCCUbbwTP83?=
 =?us-ascii?Q?tC2tHNLJgNFJWVDhwlEHCTwfOjvBvyDzpcKob+hkGlXST+NxgjdIwsrOKlJr?=
 =?us-ascii?Q?CLVerrzvyux2lquDGmraPlhyajjFuI22jl3x5QftZEu6UF1jB0umbsXPlYzT?=
 =?us-ascii?Q?Ma76uNz2BANY3YN0RzzUanKRzESQOFlJV669C8tAmQkrxwLd05uv5zRXg5LL?=
 =?us-ascii?Q?svHlxLzZpZdNF4XQMl1dgKsK1W3shvprlSFEKkzu0jJlLNnVNB1EfFSDACYl?=
 =?us-ascii?Q?X29r0QG9SJs8mQV+0wZ40usLDowU1vlx0ghfgYk3oTfiQKus8v2Kye9p4Txr?=
 =?us-ascii?Q?4led127f6GDiQZ/iqCER82pfuKiJsJiQ9e+B8VsDpWhIjIcEXQUz/hQql+eq?=
 =?us-ascii?Q?rQintysQeNNdUDlaThbEECgNQDVPy4uOwT2ejX0SL4LZ9KFF5ev72RaIEd5s?=
 =?us-ascii?Q?sLX6KJu1ZUjKD+JQ8W3MysJCrMDGix0Pz5J+MNgkczYHPJ6MFaSCEAcoyAeV?=
 =?us-ascii?Q?b1rzgw1pBF7eLxWdY2nGlu7S80WNT1VSL7PhcekexKYinnUnJDAGpDLZHaxK?=
 =?us-ascii?Q?3w6n3n5x0sQSYMrbhHjR6ywTCNEnsQHiY1+zeBQaxDGEvAQgH6VZFG2MGaGQ?=
 =?us-ascii?Q?bv75GV5650z7EoYAG8rcD5NZJsfflyTR7CUrzdzR8n5i8vBYttghSwqjAfzp?=
 =?us-ascii?Q?UUgZ313Ye9B68fJ5xV6+5jpFhR6PGFO25ddDq9jpTkzXMoSS6mZqb0lxpr+M?=
 =?us-ascii?Q?fnwVUZMgGvCYiriOhZWdr7Wzq3h+N1OtlVYPOgyZplByes+YV50B10bwFDzV?=
 =?us-ascii?Q?QPgXZOIVvBfOvNJ6tdm0BJpftQis+AmadXcVBF9oEU3YKltOyNlpGnAZhYts?=
 =?us-ascii?Q?mVRIz+pAyLBAG2GD5WzuFZK/FofGlImDxi9HIUlyM9jbltTjLAR8EV93F2ry?=
 =?us-ascii?Q?hz022iqSZW19JK6Kz70lGH79eq2nQrjPSDbxMn36vaMKrsAJ68IiHwLQecYu?=
 =?us-ascii?Q?acV6TqGDJ8CennF896r7L4oVq1VwJEVCS5haDCLP3izOQMytDFs39eQfganj?=
 =?us-ascii?Q?j5sh/6bGe3u6Ln8u2uoKJnOOWBXawu/JYg7AfIDiS5YsECrUwyBbZNg/0j8W?=
 =?us-ascii?Q?nwOVtUP9+VUCM30eS3IScAWzq7I0dQW4R6yT2X6TITZ2n2Ti518wgbD/epYm?=
 =?us-ascii?Q?wQ7Gb8bgssYvp8/bnnS3hr3UAD+XLtlB8/YVmoqoUFAuWjyqKJWKDPztaAv7?=
 =?us-ascii?Q?dlOOpHmn/N9i0lizmbIk8RAgbhWSmbK8NV6GFN58hFAorv7lji0yFgI1Nine?=
 =?us-ascii?Q?W/miVS1mhKAMKwvySLuttFSLffZCnRua4xHdpd3No3vTv8Z7uErabhZCQrFn?=
 =?us-ascii?Q?Fh5TsF13ChU4FbPsxnI/3MmDEVE5gjSJGnJLk3P9r8Nmj6WVpXd2xgLpARu0?=
 =?us-ascii?Q?AA=3D=3D?=
X-OriginatorOrg: nutanix.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c065b783-fb21-4a30-8c9f-08db4f9a0e6b
X-MS-Exchange-CrossTenant-AuthSource: CH0PR02MB8041.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 07:58:48.9634
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7TeEP+B5vK+GCXy8VBOibDwyR+R6w7h5FXnp4y8ka61NgFbc5vqwHOyDfIT994nZs7mRodGswe3SqXTka7CjzIRID6TgYvle3xJyoguu7Bk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN4PR0201MB8821
X-Proofpoint-GUID: 3EEqJd7Gllioxahca6z8xvOGDxr5NzE9
X-Proofpoint-ORIG-GUID: 3EEqJd7Gllioxahca6z8xvOGDxr5NzE9
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-05-08_05,2023-05-05_01,2023-02-09_01
X-Proofpoint-Spam-Reason: safe
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Commit 7ffb8e317bae ("audit: we don't need to
__set_current_state(TASK_RUNNING)") accidentally moved queue full check
before add_wait_queue_exclusive() which introduced the following race:

    CPU1                           CPU2
  ========                       ========
  (in audit_log_start())         (in kauditd_thread())

  queue is full
                                 wake_up(&audit_backlog_wait)
                                 wait_event_freezable()
  add_wait_queue_exclusive()
  ...
  schedule_timeout()

Once this happens, both audit_log_start() and kauditd_thread() can cause
deadlock for up to backlog_wait_time waiting for each other. To prevent
the race, this patch adds queue full check after
prepare_to_wait_exclusive().

Fixes: 7ffb8e317bae ("audit: we don't need to __set_current_state(TASK_RUNNING)")
Signed-off-by: Eiichi Tsukata <eiichi.tsukata@nutanix.com>
---
 kernel/audit.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/kernel/audit.c b/kernel/audit.c
index bcbb0ba33c84..d37a3a045230 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -644,8 +644,14 @@ static long wait_for_kauditd(long stime)
 
 	prepare_to_wait_exclusive(&audit_backlog_wait, &wait,
 				  TASK_UNINTERRUPTIBLE);
-	rtime = schedule_timeout(stime);
-	atomic_add(stime - rtime, &audit_backlog_wait_time_actual);
+
+	/* need to check if the queue is full again because kauditd might have
+	 * flushed the queue and went to sleep after prepare_to_wait_exclusive()
+	 */
+	if (audit_queue_full(&audit_queue)) {
+		rtime = schedule_timeout(stime);
+		atomic_add(stime - rtime, &audit_backlog_wait_time_actual);
+	}
 	finish_wait(&audit_backlog_wait, &wait);
 
 	return rtime;
-- 
2.40.0