Received: by 2002:a25:1985:0:0:0:0:0 with SMTP id 127csp176750ybz; Thu, 30 Apr 2020 19:05:06 -0700 (PDT) X-Google-Smtp-Source: APiQypJY1Iyj72dKwb0MxIkLAF/hA2ltvfbTnIH6qQftyEOrEjWKGh0yGwgyKb15AmKJ5U3i1g7M X-Received: by 2002:aa7:d718:: with SMTP id t24mr1735177edq.29.1588298705937; Thu, 30 Apr 2020 19:05:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1588298705; cv=none; d=google.com; s=arc-20160816; b=rXdt8OXJnVrilvVNDu1cHWNe+9IWmoeK5O0hgObqeVhA1JoaVkLPuZWkvVZd+kq9Yx b+qUOCgaYXa6Bh3zpM/8AAPI9rZ4n8p9GoV2a15HspDTIlhgZjG+I6stDPv7il5bJhW3 /oWFav98mLNg5+/mJC2Laq1txmaxu3fAuFE0QdkdSFVvjTu3gnkUJ89JKxli1t5pM8Of /s7bwaeP8ojSVq4EN/cTTYwaZ/q8BswOX1bZ7wA4JVj0DDAWiwsVz5uFinzUgKDxPBpN aQOXxLv2wTU9d+p5O/AiVwaBv1mGUxF0S2dUVw5ekJU3InXQz0z66yLMXAwTqJGIDea2 A0ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:subject:cc:to:from:date :dkim-signature; bh=DU8LbFcBqFFUcOPfEfuSx9Jiz8kE4ceNFLtiZkrDjDY=; b=GUQqtjdNuVVupidFZ0ALxSVKI2a4TiY2Q9jfs+BytNu8C58gtSztbSwvWNgmECNp3L rNhHuW3GW51U3dy1/9/k32YkOIMJ5eUfcie9KV9XFl+H8E1dqjBFdpPe/imkNVequabd 71yzNk2xTqOlAKHC7ckd48eIS9ghOqhmWX5Lqe86F20PW6aaE2pB5+6VCC4tbQkH+dt6 HCHdpjRtPE8hKkrBfvdkFGPTf/hMsqcMEyXgBNPmUgbV/HFlJIev9H7G3Z7HxsCFo7EB D6AXBqIwYIlk0vaIuyG4ZUyQzyrhuRcbqbDozD5XOO+LhaCCeLCMSGiP5b4Fw+p5nIQM +WFA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=URt7r82K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id a12si962426edx.283.2020.04.30.19.04.42; Thu, 30 Apr 2020 19:05:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=URt7r82K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728051AbgEACBL (ORCPT + 99 others); Thu, 30 Apr 2020 22:01:11 -0400 Received: from mail.kernel.org ([198.145.29.99]:55950 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727889AbgEACBL (ORCPT ); Thu, 30 Apr 2020 22:01:11 -0400 Received: from devnote (NE2965lan1.rev.em-net.ne.jp [210.141.244.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E21AB2073E; Fri, 1 May 2020 02:01:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1588298470; bh=TGng++haxmjKEAxWTDMCt+me7RqYg/DPhKnyheMa2+E=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=URt7r82KCUw4/FpnlqtLK2lwwbKBKLoUccNmriUdGhHDebv21eK1bGHH6DTwRa128 67HmXFSGYmNdy9h3gS+T/kIStuMRe1Ls0QxtGSuGO8srfn4PuYocXspwYHUfUQhOXN IYRTXGNcySptZupssYgZE9jnS/sJ9dUIFhp/12iY= Date: Fri, 1 May 2020 11:01:07 +0900 From: Masami Hiramatsu To: Ingo Molnar , Ingo Molnar Cc: Jiri Olsa , Jiri Olsa , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" , Peter Zijlstra , lkml , "bibo,mao" , "Ziqian SUN (Zamir)" , stable@vger.kernel.org Subject: Re: [PATCHv2] kretprobe: Prevent triggering kretprobe from within kprobe_flush_task Message-Id: <20200501110107.bc859c6603704c0bcdb8889a@kernel.org> In-Reply-To: <20200428213627.GI1476763@krava> References: <20200409184451.GG3309111@krava> <20200409201336.GH3309111@krava> <20200410093159.0d7000a08fd76c2eaf1398f8@kernel.org> <20200414160338.GE208694@krava> <20200415090507.GG208694@krava> <20200416105506.904b7847a1b621b75463076d@kernel.org> <20200416091320.GA322899@krava> <20200416224250.7a53fb581e50aa32df75a0cf@kernel.org> <20200416143104.GA400699@krava> <20200417163810.ffe5c9145eae281fc493932c@kernel.org> <20200428213627.GI1476763@krava> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 28 Apr 2020 23:36:27 +0200 Jiri Olsa wrote: > On Fri, Apr 17, 2020 at 04:38:10PM +0900, Masami Hiramatsu wrote: > > SNIP > > > > > > > The code within the kretprobe handler checks for probe reentrancy, > > > so we won't trigger any _raw_spin_lock_irqsave probe in there. > > > > > > The problem is in outside kprobe_flush_task, where we call: > > > > > > kprobe_flush_task > > > kretprobe_table_lock > > > raw_spin_lock_irqsave > > > _raw_spin_lock_irqsave > > > > > > where _raw_spin_lock_irqsave triggers the kretprobe and installs > > > kretprobe_trampoline handler on _raw_spin_lock_irqsave return. > > > > > > The kretprobe_trampoline handler is then executed with already > > > locked kretprobe_table_locks, and first thing it does is to > > > lock kretprobe_table_locks ;-) the whole lockup path like: > > > > > > kprobe_flush_task > > > kretprobe_table_lock > > > raw_spin_lock_irqsave > > > _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed > > > > > > ---> kretprobe_table_locks locked > > > > > > kretprobe_trampoline > > > trampoline_handler > > > kretprobe_hash_lock(current, &head, &flags); <--- deadlock > > > > > > Adding kprobe_busy_begin/end helpers that mark code with fake > > > probe installed to prevent triggering of another kprobe within > > > this code. > > > > > > Using these helpers in kprobe_flush_task, so the probe recursion > > > protection check is hit and the probe is never set to prevent > > > above lockup. > > > > > > > Thanks Jiri! > > > > Ingo, could you pick this up? > > Ingo, any chance you could take this one? Hi Ingo, Should I make a pull request for all kprobes related patches to you? Thank you, -- Masami Hiramatsu