Date:   Fri, 3 Feb 2023 10:08:16 +0800
From:   Pengfei Xu <pengfei.xu@intel.com>
To:     Jens Axboe <axboe@kernel.dk>
CC:     Pavel Begunkov <asml.silence@gmail.com>,
        <linux-kernel@vger.kernel.org>, <heng.su@intel.com>
Subject: Re: [Syzkaller & bisect] There is "io_ring_exit_work" related Call
 Trace in v5.2-rc5 kernel
Message-ID: <Y9xskCrKaTGZptRw@xpf.sh.intel.com>
References: <Y9TgUupO5C39V/DW@xpf.sh.intel.com>
 <812a7e2d-14d6-78cd-5c6c-ce508fc7fc0d@gmail.com>
 <15161e5f-fe26-23e9-1d0a-ebd579d64d9f@gmail.com>
 <Y9t6tflIcFPy84S5@xpf.sh.intel.com>
 <fb27661f-59dd-98cd-203a-8fa86dc209d4@kernel.dk>
 <affab34f-2b83-9158-5976-b224a4afce0d@kernel.dk>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <affab34f-2b83-9158-5976-b224a4afce0d@kernel.dk>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BVqT0JkQSBBl5yN0tvCeMoSznlivZyQX19eeX/hkY3DFjMjJVhJrOf/UuCsW?=
 =?us-ascii?Q?7jF9xrX7eWU8ejA7g6Dum7n2DSTJ5Atwd1CBWr68uWQrm61wNVrMR6eToi34?=
 =?us-ascii?Q?PyAcLpMdUkTSB8tdLVSolq3R0djKhLIPilNI6U1yZlGAfssdCBw+1tUo/Hqi?=
 =?us-ascii?Q?M3C+45xphsr84kC1QmW2EAJc7gJUw2xKw2qfB9L/7qHAMuci7APwiZDoZeHH?=
 =?us-ascii?Q?5b3cUu9nTRdOAzjKVgi541jfx9NFlMjEQsJ5P4OmO43kIpgn+2iCk9szZuTZ?=
 =?us-ascii?Q?gADeoLhE6OUWl1BoA/qgc5L6q2eRI3sz6Kszj8seWO4TPuZPdqqpNYejjWCc?=
 =?us-ascii?Q?8BWUMLR+n7l55Yezpk9JV3SZ1zV2d/7btgqKyynLBzM/G9AvQux8uoI8rQzR?=
 =?us-ascii?Q?a9fvwq4SbDEw1ovmXubZF0jeQL5jEWQl+vrrEFwEHRh/4Iga8yX706JOg8vI?=
 =?us-ascii?Q?/pQeKe7enz8h5fa6JRNpVx2VznP7us6MybIqpFl259CqyzadRIWAnK3X9R2R?=
 =?us-ascii?Q?+GREiz8vWtdf3outNE43sSWy4VEe4v9EyrC/c/W3ubP5b0KxbUbCjEXijXXB?=
 =?us-ascii?Q?Kze/d5QLADh/6U0KVHRbsL1qQT6QwPN2BxDlkfq2KBX4dgbROI/TsnYoF56c?=
 =?us-ascii?Q?DZluASmPEJDxf7ccSQJeU6DD7Via5Uq++RM6fScetC3K8VcBXHtPdsti4r3e?=
 =?us-ascii?Q?A2repMs3Wkqd1ZgfFkl/tqIqH6gD7IxxNCbDItJJDAgjx2mH9oPmBzCVQkNo?=
 =?us-ascii?Q?LtynRm3gWrnFwRJsVq6Q16fvrR0EF9aEdQXS4s99FNOndbhGwK3u4HXC9RqR?=
 =?us-ascii?Q?XsZLJSt9zIACJorXePpFRnWDOk8ZiAKAoWoJPJ1p5LP+xWYW/Wnk3FQcYHM1?=
 =?us-ascii?Q?BczBpojZRRmro2CL6zzXJ5fGbL2BQazyhPzuZpTTWK0VxU7YMrMykkrKhTeA?=
 =?us-ascii?Q?tGvA7DIGWnnxQV7TEpoW/vTXfA6jSe8l8+UtroViSKwY/lxdcjhPXlcsk3OF?=
 =?us-ascii?Q?88cq7A4Y1/1DbgMca+t71Y9fCFjF2ZHZim5umP4UhdbIthEDXSo5xfTxJhn0?=
 =?us-ascii?Q?D6XyhvzvVECJmjjObHFu6boCC9FainI68GqbsKK8mLBJ8N5NGkjN8y2Rn0vs?=
 =?us-ascii?Q?185QUaOSxWDAy5W7FuKY/0MXyXmj9FpffBhKvki+L3GVzxs1Ck1wbox8sxUC?=
 =?us-ascii?Q?OQE4PEkSx3HD92rvw9btkDCkBHCG174izFEB7XnZgnhZYUsUJRBk1esv7DeG?=
 =?us-ascii?Q?So8SCbKODdjPO+G/qsNdD0Wea6Rp/jlPNruQo36BCyx8auE7bpFoe29jOQ2f?=
 =?us-ascii?Q?J/hCq4ZR6cu+JzKoCDslYvKjlYWfP1Fw9eKFiHdvaRd76e2JyPdkIMT8vVU0?=
 =?us-ascii?Q?KJxo5IXl7hwmrJgEHuI5JKqVU81KP7qV2V4uiDT5umIBHDy95WcOY0O4wyid?=
 =?us-ascii?Q?z1v8t6ULoLXz1sjFWRPd5rUmcxghGIiRCFlr6nbkIf1U2yFVjtDMeJDQbuNv?=
 =?us-ascii?Q?f09WkgUL1hdr3kKymZxMMROv6K21JHDGd5xNWDW3TaeqNAD0SXIWGOwHfX2R?=
 =?us-ascii?Q?MVvuP4R01/2NXHA6x7Skm/I0aJmTvn3bS1XIqgiKgN6zXZwfNsG+aqc0uxwD?=
 =?us-ascii?Q?uA=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 78918015-5fb4-4ed7-c34d-08db058b5e14
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4839.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2023 02:07:14.3833
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Q1gVPSHGwNt8kQ0u3YxZdyfPU2/54XNSvIjAQT/VQnELry8M9k3PSiWeHvUxJGNtYcbNn9cfUYZa6mpPK27eHQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB7605
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Jens Axboe,

On 2023-02-02 at 13:37:43 -0700, Jens Axboe wrote:
> On 2/2/23 12:44?PM, Jens Axboe wrote:
> > On 2/2/23 1:56?AM, Pengfei Xu wrote:
> >> Hi Pavel Begunkov,
> >>
> >> On 2023-02-01 at 15:52:47 +0000, Pavel Begunkov wrote:
> >>> On 1/28/23 14:49, Pavel Begunkov wrote:
> >>>> On 1/28/23 08:44, Pengfei Xu wrote:
> >>>>> Hi Pavel Begunkov and kernel expert,
> >>>>>
> >>>>> Greeting!
> >>>>>
> >>>>> There is "io_ring_exit_work" related Call Trace in v5.2-rc5 kernel in guest
> >>>>> on Sapphire Rapids server.
> >>>>
> >>>> Thanks for the report, we'll take a look
> >>>
> >>> Not reproducible for me. Apparently, the repro creates a normal ring
> >>> and immediately closes it, then io_ring_exit_work() hangs waiting to
> >>> the task to execute task_work.
> >>   Ah, I put the reproduced virtual machine in gitlab.
> >>   git clone https://gitlab.com/xupengfe/repro_vm_env.git
> >>   cd repro_vm_env
> >>   tar -xvf repro_vm_env.tar.gz
> >>   cd repro_vm_env; ./start3.sh  // it needs qemu-system-x86_64 and I used v7.1.0
> >>      //start3.sh will load bzImage_2241ab53cbb5cdb08a6b2d4688feb13971058f65 v6.2-rc5 kernel
> >>   In vm and login with root,  there is no password for root.
> >>   After login vm successfully, ./repro   to reproduce this issue.
> >>   It could be reproduced in about 6min.
> >>
> >>   If you already have qemu-system-x86_64, please ignore below info.
> >>   If you want to install qemu v7.1.0 version,
> >>   git clone https://github.com/qemu/qemu.git
> >>   cd qemu
> >>   git checkout -f v7.1.0
> >>   mkdir build
> >>   cd build
> >>   yum install -y ninja-build.x86_64
> >>   ../configure --target-list=x86_64-softmmu --enable-kvm --enable-vnc --enable-gtk --enable-sdl
> >>   make
> >>   make install
> >>   Done.
> > 
> > Are you sure you attached the right kconfig in the initial report? It
> > is from 5.18.x.
  Yes, the kconfig from initial report is first used by 5.18.x kernel,
  and I added some new kconfig item based on the kconfig like kernel ibt:
  CONFIG_X86_KERNEL_IBT=y and so on.

  And use the kconfig then "make olddefconfig" to generate the kconfig
  for latest kernel.

  I will add above vm environment and kconfig making info for reproduced
  steps if I find new kernel issues.
  Thanks for remind!

> 
> Looks like a bug in 9p, can you try this?
  Thanks for your patch!
  The following patch was installed on top of v6.2-rc5 kernel, and tested this
  issue in vm on SPR and RPL-S platforms(reproduced platforms as before).
  This issue could not be reproduced in 40 min.
  Test result shows that this patch solves the problem.

  Thanks!
  BR.
> 
> 
> diff --git a/net/9p/client.c b/net/9p/client.c
> index 622ec6a586ee..5756d812d3d4 100644
> --- a/net/9p/client.c
> +++ b/net/9p/client.c
> @@ -18,6 +18,7 @@
>  #include <linux/sched/signal.h>
>  #include <linux/uaccess.h>
>  #include <linux/uio.h>
> +#include <linux/task_work.h>
>  #include <net/9p/9p.h>
>  #include <linux/parser.h>
>  #include <linux/seq_file.h>
> @@ -652,6 +653,24 @@ static struct p9_req_t *p9_client_prepare_req(struct p9_client *c,
>  	return ERR_PTR(err);
>  }
>  
> +static int p9_sigpending(void)
> +{
> +	int sigpending = 0;
> +
> +	if (signal_pending(current)) {
> +		if (test_thread_flag(TIF_NOTIFY_SIGNAL))
> +			clear_notify_signal();
> +		if (task_work_pending(current))
> +			task_work_run();
> +		if (signal_pending(current)) {
> +			sigpending = 1;
> +			clear_thread_flag(TIF_SIGPENDING);
> +		}
> +	}
> +
> +	return sigpending;
> +}
> +
>  /**
>   * p9_client_rpc - issue a request and wait for a response
>   * @c: client session
> @@ -687,12 +706,7 @@ p9_client_rpc(struct p9_client *c, int8_t type, const char *fmt, ...)
>  	req->tc.zc = false;
>  	req->rc.zc = false;
>  
> -	if (signal_pending(current)) {
> -		sigpending = 1;
> -		clear_thread_flag(TIF_SIGPENDING);
> -	} else {
> -		sigpending = 0;
> -	}
> +	sigpending = p9_sigpending();
>  
>  	err = c->trans_mod->request(c, req);
>  	if (err < 0) {
> @@ -789,12 +803,7 @@ static struct p9_req_t *p9_client_zc_rpc(struct p9_client *c, int8_t type,
>  	req->tc.zc = true;
>  	req->rc.zc = true;
>  
> -	if (signal_pending(current)) {
> -		sigpending = 1;
> -		clear_thread_flag(TIF_SIGPENDING);
> -	} else {
> -		sigpending = 0;
> -	}
> +	sigpending = p9_sigpending();
>  
>  	err = c->trans_mod->zc_request(c, req, uidata, uodata,
>  				       inlen, olen, in_hdrlen);
> 
> -- 
> Jens Axboe
>