Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <0863d7c1-1063-2a2a-d70d-764a7580c874@intel.com>
Date:   Thu, 21 Jul 2022 10:59:39 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.11.0
Subject: Re: [LKP] Re: [io_uring] 584b0180f0:
 phoronix-test-suite.fio.SequentialWrite.IO_uring.Yes.Yes.1MB.DefaultTestDirectory.mb_s
 -10.2% regression
Content-Language: en-US
To:     Jens Axboe <axboe@kernel.dk>,
        kernel test robot <oliver.sang@intel.com>
CC:     LKML <linux-kernel@vger.kernel.org>, <io-uring@vger.kernel.org>,
        <lkp@lists.01.org>, <lkp@intel.com>
References: <20220527092432.GE11731@xsang-OptiPlex-9020>
 <2085bfef-a91c-8adb-402b-242e8c5d5c55@kernel.dk>
 <0d60aa42-a519-12ad-3c69-72ed12398865@intel.com>
 <26d913ea-7aa0-467d-4caf-a93f8ca5b3ff@kernel.dk>
 <9df150bb-f4fd-7857-aea8-b2c7a06a8791@intel.com>
 <7146c853-0ff8-3c92-c872-ce6615baab40@kernel.dk>
 <81af5cdf-1a13-db2c-7b7b-cfd86f1271e6@intel.com>
 <74d1f308-de03-fd5e-b7f0-0e17980f988e@kernel.dk>
 <2ec953da-78fd-df01-44cf-6f33a5e40864@intel.com>
 <f5d20f6c-5363-231b-b208-b577a59b4ae9@kernel.dk>
 <299889df-db40-e0e2-6bc6-d9eb784ebe89@kernel.dk>
From:   Yin Fengwei <fengwei.yin@intel.com>
In-Reply-To: <299889df-db40-e0e2-6bc6-d9eb784ebe89@kernel.dk>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ODMyUlZpRndkNEc4MFp1KytCd3FHS2FpR3RxaUk3dkRkYmNSMjZpQUIxNUNY?=
 =?utf-8?B?QTA2Qm43WlRFOC8rcm13OGpwdVc2REpDdzJlRHBvVnpMcUZlV3RlUzNVRHEw?=
 =?utf-8?B?ZkNRUHdIZi9hc1A3b0srYVJHSUZmL0ZSVy95RTVjTWIrdmJYaExEbTlPSFVX?=
 =?utf-8?B?UHNTemlxYUdoK1Nvd1pQY3JYdkJwZU83MUJKQjBJK3MwY2UxVFBrQ2FWalVy?=
 =?utf-8?B?Q01BdkVhZHVrZzNpZ0JqV3FVeUl6aW13bFA4cWJ5c3hNZEpVQzVQa1A3cG5R?=
 =?utf-8?B?N0NvNjVJbysvdHhIQmhiUjRERjhsSW5EdHROdVFHcWlFQ3BOZWNVOERXQy9i?=
 =?utf-8?B?WGY5cVBlSWFkSWFGWnlnMUlLeG85TUN0Y0pvYWhpaW0wTEVtLzUzZ0JTOFJ2?=
 =?utf-8?B?ZWRFOUllWlRSWEJoZUV2Nit0QndUaUYvWlY2RURBYk8rUitZd0EydWxEb280?=
 =?utf-8?B?MkNqRlZ3blRFNmNKeGs0dTNqZEE0blJwcE04NHo2Wi9LZDNVTWNtWGRVOGI1?=
 =?utf-8?B?UlprZ0FZK3lDbEZOSGFORzJQWTYvSndZVmlweWpkbHpJN282ZVVKVStidjF2?=
 =?utf-8?B?M3loMUtOeURZRmNaYzl5eE8xTW9kZ3lhOGtpM1lkZ1hyYThBUVkyVEFTL09j?=
 =?utf-8?B?a1NMSFFoM0lTNndXUVcyWmtaUERFcEs4YUo5dk1DRWt3TEhiOUJUMUI4alFi?=
 =?utf-8?B?UUpVYzk4RnZaYndsaU1NRzJHdThMNy9SUmk5TWVJeUtSN2F6bVo4WVFsUVYz?=
 =?utf-8?B?WnowbXRSdlpUYXN4Z1BGTWJoZ3IyUjIyU2R6U2oxazFiRmdXUnZySFZ3SkNi?=
 =?utf-8?B?YmNBblFVMHFuNDZzWi84bW9WK1lNYjhmc0g1bmt0OEg5UFRuYjVsTWIyNnpR?=
 =?utf-8?B?SEVMejZWSllKSE1KOFFYdGs2cG82VU5Iamk2cVpzOXgzUUx0SUpzeFJJaHJW?=
 =?utf-8?B?alpON3k1dHRqWTZ4dU5EdG1WQ3d4eXBNcy9KK2RydGRxZUdQejAwSGs1UGNh?=
 =?utf-8?B?K2ZwUDdyeFdmR1IwMmNDTFBuQVF0cnZ4OTl5dGFMWHQyVDFUUHU3UTRyUnZy?=
 =?utf-8?B?OFk1SEgzSWRQQXpFMVZhanI0bjR4NzhrOEpEUVMwZzFQYkJhYUJEZlFvQ3pO?=
 =?utf-8?B?czg0OG5OZTE2UHl4T0hrakpFMnJISWRqK1d2VDFqMThPdENLWXZQanNJUEpp?=
 =?utf-8?B?cjdFTW9iQzdVdklJeU5VbmdUQVV4dkh0K1MyREtmczdyS085c0hYd2REdUk2?=
 =?utf-8?B?TTREdmJlY1dWN1hMR1o4YUt2SmZPOXk5bEtUVUpIeW0xRys0d0ZUQ3JXbFVU?=
 =?utf-8?B?bDB4RzRwQ1JvSVhOZG5qaldjaTZoRjY4ZFgwV1FMeTlDL0JHY3RpaU8wYzVo?=
 =?utf-8?B?MUtOTHNCQmZxbk0zRVVKWHhrQ1dvR1JqOHhWZVo2U2hTQ1VGZ0VXNFplRGVK?=
 =?utf-8?B?eEtqQ2hFOGoyK3pGT3JNTlliR3F0WWYyUUlvbUNtd2lBa1VqS0tscmtVbk5Z?=
 =?utf-8?B?ak9mWnQ4Z3VuMDIySjl6UWowUzUyU3NHR0J6Kys5OTBwaFdGdG1FeVpDK0kv?=
 =?utf-8?B?L2RhNTEyY1RMcHJ2MjI5b05kVlRPZlVCM3NDWDFoUGdhMTY1RjJEcjJmckZ1?=
 =?utf-8?B?dmYzWXRDc2RwWE5ITnBXUGVPbnBHYXZZeWRCSURsRFF4Z1pwMmFibzR4UmlH?=
 =?utf-8?B?RXo5ZUZOYndydU9pVG9sczhwMzAvaWhXTW9HOXZvcmI3Ylg3d3h3ZlhKeWs5?=
 =?utf-8?B?Rmp5SVZIQ05WS3VCZndDVThvcUNVS3dxTG9MQWN6YWdDMFJwdmc2bHY0NnNH?=
 =?utf-8?B?RDV2bW5Vem5rME5jc2dGcmJMU3Brd2FyZFB5SFhiR2R6VERaNUt0cEpxcURM?=
 =?utf-8?B?N1JDcmN6aHNpazJsVzlCLzI4cjJZc2VkZm9FMm9ldWtXWUtmTlVkV0ZiQzJ3?=
 =?utf-8?B?eUZXSWdkVlhEbVBCaC9xbFJDNTg3V1gzSDkveXNPb2YzL0NCVldoNkZmbHFW?=
 =?utf-8?B?SUdrZWh4N3cvV05ta0kwZy9JN2pqK3RPVzRPSDFHZmlaclR4K0I5YXBsRE1L?=
 =?utf-8?B?b2xNdHVQNElOUU80VFlOK0N5d0EwMzBxZEZweW9RQUV4UDFGSTQxTnJyZCt3?=
 =?utf-8?Q?Mq8zitoDdVdg/sm83Un4Ev6f2?=
X-MS-Exchange-CrossTenant-Network-Message-Id: fa0f3185-dcb7-4152-968c-08da6ac5120a
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2022 02:59:47.3568
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: V5joaJIGsoUwTtnmtDuaAGG1kLSUlTnFnmvTlMrnr0/GLU0m5J0Cx2ukJ5Utljh+4PnHmtw2Zy/wSjK8OBzyPw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4975
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Jens,

On 7/21/2022 2:13 AM, Jens Axboe wrote:
> Can you try this? It's against 5.19-rc7.
> 
> 
> diff --git a/fs/io_uring.c b/fs/io_uring.c
> index a01ea49f3017..34758e95990a 100644
> --- a/fs/io_uring.c
> +++ b/fs/io_uring.c
> @@ -2015,6 +2015,64 @@ static inline void io_arm_ltimeout(struct io_kiocb *req)
>  		__io_arm_ltimeout(req);
>  }
>  
> +static bool io_bdev_nowait(struct block_device *bdev)
> +{
> +	return !bdev || blk_queue_nowait(bdev_get_queue(bdev));
> +}
> +
> +/*
> + * If we tracked the file through the SCM inflight mechanism, we could support
> + * any file. For now, just ensure that anything potentially problematic is done
> + * inline.
> + */
> +static bool __io_file_supports_nowait(struct file *file, umode_t mode)
> +{
> +	if (S_ISBLK(mode)) {
> +		if (IS_ENABLED(CONFIG_BLOCK) &&
> +		    io_bdev_nowait(I_BDEV(file->f_mapping->host)))
> +			return true;
> +		return false;
> +	}
> +	if (S_ISSOCK(mode))
> +		return true;
> +	if (S_ISREG(mode)) {
> +		if (IS_ENABLED(CONFIG_BLOCK) &&
> +		    io_bdev_nowait(file->f_inode->i_sb->s_bdev) &&
> +		    file->f_op != &io_uring_fops)
> +			return true;
> +		return false;
> +	}
> +
> +	/* any ->read/write should understand O_NONBLOCK */
> +	if (file->f_flags & O_NONBLOCK)
> +		return true;
> +	return file->f_mode & FMODE_NOWAIT;
> +}
> +
> +static inline bool io_file_supports_nowait(struct io_kiocb *req)
> +{
> +	return req->flags & REQ_F_SUPPORT_NOWAIT;
> +}
> +
> +/*
> + * If we tracked the file through the SCM inflight mechanism, we could support
> + * any file. For now, just ensure that anything potentially problematic is done
> + * inline.
> + */
> +static unsigned int io_file_get_flags(struct file *file)
> +{
> +	umode_t mode = file_inode(file)->i_mode;
> +	unsigned int res = 0;
> +
> +	if (S_ISREG(mode))
> +		res |= FFS_ISREG;
> +	if (__io_file_supports_nowait(file, mode))
> +		res |= FFS_NOWAIT;
> +	if (io_file_need_scm(file))
> +		res |= FFS_SCM;
> +	return res;
> +}
> +
>  static void io_prep_async_work(struct io_kiocb *req)
>  {
>  	const struct io_op_def *def = &io_op_defs[req->opcode];
> @@ -2031,6 +2089,9 @@ static void io_prep_async_work(struct io_kiocb *req)
>  	if (req->flags & REQ_F_FORCE_ASYNC)
>  		req->work.flags |= IO_WQ_WORK_CONCURRENT;
>  
> +	if (req->file && !io_req_ffs_set(req))
> +		req->flags |= io_file_get_flags(req->file) << REQ_F_SUPPORT_NOWAIT_BIT;
> +
>  	if (req->flags & REQ_F_ISREG) {
>  		if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
>  			io_wq_hash_work(&req->work, file_inode(req->file));
> @@ -3556,64 +3617,6 @@ static void io_iopoll_req_issued(struct io_kiocb *req, unsigned int issue_flags)
>  	}
>  }
>  
> -static bool io_bdev_nowait(struct block_device *bdev)
> -{
> -	return !bdev || blk_queue_nowait(bdev_get_queue(bdev));
> -}
> -
> -/*
> - * If we tracked the file through the SCM inflight mechanism, we could support
> - * any file. For now, just ensure that anything potentially problematic is done
> - * inline.
> - */
> -static bool __io_file_supports_nowait(struct file *file, umode_t mode)
> -{
> -	if (S_ISBLK(mode)) {
> -		if (IS_ENABLED(CONFIG_BLOCK) &&
> -		    io_bdev_nowait(I_BDEV(file->f_mapping->host)))
> -			return true;
> -		return false;
> -	}
> -	if (S_ISSOCK(mode))
> -		return true;
> -	if (S_ISREG(mode)) {
> -		if (IS_ENABLED(CONFIG_BLOCK) &&
> -		    io_bdev_nowait(file->f_inode->i_sb->s_bdev) &&
> -		    file->f_op != &io_uring_fops)
> -			return true;
> -		return false;
> -	}
> -
> -	/* any ->read/write should understand O_NONBLOCK */
> -	if (file->f_flags & O_NONBLOCK)
> -		return true;
> -	return file->f_mode & FMODE_NOWAIT;
> -}
> -
> -/*
> - * If we tracked the file through the SCM inflight mechanism, we could support
> - * any file. For now, just ensure that anything potentially problematic is done
> - * inline.
> - */
> -static unsigned int io_file_get_flags(struct file *file)
> -{
> -	umode_t mode = file_inode(file)->i_mode;
> -	unsigned int res = 0;
> -
> -	if (S_ISREG(mode))
> -		res |= FFS_ISREG;
> -	if (__io_file_supports_nowait(file, mode))
> -		res |= FFS_NOWAIT;
> -	if (io_file_need_scm(file))
> -		res |= FFS_SCM;
> -	return res;
> -}
> -
> -static inline bool io_file_supports_nowait(struct io_kiocb *req)
> -{
> -	return req->flags & REQ_F_SUPPORT_NOWAIT;
> -}
> -
>  static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe)
>  {
>  	struct kiocb *kiocb = &req->rw.kiocb;
> 
> -- Jens Axboe
This change could make regression gone. The test result is as following:
28d3a5662d44077aa6eb42bfcfa is your patch


584b0180f0f4d67d                   v5.19-rc7 28d3a5662d44077aa6eb42bfcfa
---------------- --------------------------- ---------------------------
       fail:runs  %reproduction    fail:runs  %reproduction    fail:runs
           |             |             |             |             |
        503:3         9297%         782:3          178%         509:3     dmesg.timestamp:last
          3:3            0%           3:3            0%           3:3     pmeter.pmeter.fail
           :3          100%           3:3          100%           3:3     kmsg.I/O_error,dev_loop#,sector#op#:(READ)flags#phys_seg#prio_class
           :3         3755%         112:3         4016%         120:3     kmsg.timestamp:I/O_error,dev_loop#,sector#op#:(READ)flags#phys_seg#prio_class
        465:3         9221%         742:3          235%         473:3     kmsg.timestamp:last
         %stddev     %change         %stddev     %change         %stddev
             \          |                \          |                \
    972.00            -0.3%     968.67           +11.4%       1082        phoronix-test-suite.fio.SequentialWrite.IO_uring.Yes.Yes.1MB.DefaultTestDirectory.iops
    975.00            -0.3%     972.33           +11.5%       1086        phoronix-test-suite.fio.SequentialWrite.IO_uring.Yes.Yes.1MB.DefaultTestDirectory.mb_s

Comparing to v5.19-rc7 and 584b0180f0f4d67d, it could bring 11% regression back.
Thanks.


Regards
Yin, Fengwei