Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <3c4b9025-2667-ca8c-7427-502068b8168e@amd.com>
Date:   Fri, 14 Apr 2023 12:55:14 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Subject: Re: linux-next: Tree for Apr 13
Content-Language: en-US
From:   Ayush Jain <ayush.jain3@amd.com>
To:     broonie@kernel.org,
        Linux Next Mailing List <linux-next@vger.kernel.org>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        axboe@kernel.dk, Santosh.Shukla@amd.com, Kalpana.Shetty@amd.com,
        Narasimhan V <Narasimhan.V@amd.com>, sfr@canb.auug.org.au
References: <20230413182542.1133335-1-broonie@kernel.org>
 <0c6b661c-f7ff-cf12-b7f0-00b6b2f1317b@amd.com>
In-Reply-To: <0c6b661c-f7ff-cf12-b7f0-00b6b2f1317b@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VGVxZmEvT3grMjRlQUx5SmxjQ2Raa0VhdWU3Q2pCYzM3ZVBSSnFZcG5FU0E0?=
 =?utf-8?B?NkFkWTNMaDB4ZHhGdGhzQkpMQkJjZ2txUnZhMy8xNnZHOUM1SitGR0lYSGsr?=
 =?utf-8?B?b2IrY2ExMm9RQjIyOW1FU25zNVZaektSZ3ZCSGVSUTRDNDFKUkswMUdZakpo?=
 =?utf-8?B?cEVTRUNuZ0JXckJXaHNYNmUxT0p6eSsxRnRsWEVBdEFvZTdTOXgyUitic3VS?=
 =?utf-8?B?aFNPeGlYY0pad3hBWlV2R3JZc05aM1VLZS9Ba3JNeUdweXlzRXY0N2N1WEgx?=
 =?utf-8?B?RGZLRVpON2lDNnBtOE4vNUh0WXdvN255VmorSmZlRzh6QkYzRk9zUVhKdWhu?=
 =?utf-8?B?VVpmTnJqTW5YZ3ljMXcvZVhWR3ZJd29leDJxQzRScWdFZFU0S0lYakZJT1I1?=
 =?utf-8?B?WEpFQTRlNzkyNzZNZ3Btc0xXVWN1bWZkU3Zhejh6ckFCU1VkR1AyazZHS3h3?=
 =?utf-8?B?RXZmTTRWOXJiSkFDL0J6YXczelpqWUZjelVtSUdIRGVoN1BBdWlBT2g3VHht?=
 =?utf-8?B?YlkycXJDUno0dkd3dFNHNFNYeE90ZHloVnYrT1g1bEhFejZKNm8rSnhOenlW?=
 =?utf-8?B?V2sxRGV4ZnA0VXUwMFFRQnJkR1E0dTdJQ1F2QUV4cElwTlAraW5xa2RieVdw?=
 =?utf-8?B?Z3pWcU5oSStxYnByT0MvMExWTVlmZlErYk1IOTJrcU5BVE1CRExpa3ZJYUdG?=
 =?utf-8?B?bGlNRXN2UXE3MnEvTit0UWZGS3NqejUzaUF3WjFUcFFNOVJvNTM4UkE4eUlV?=
 =?utf-8?B?T0psaHluK1krOHlqSXJNbGlTY1ZyNUw4d2ErNUJYWXdGL3l5cThXNGpFWmRB?=
 =?utf-8?B?b0xpTzNZbmxtTWN4bjVLWVF5Qzk5L1dRT0lzRnlLSHp4MXdIcnEvY2JoK2or?=
 =?utf-8?B?c3huQ1prczdPNFlwbGc5amZoTFZGSFRiWDZpeSs4VUhhRXBOMzBNNjU3TjhT?=
 =?utf-8?B?S1BCNHNMdHllWitZS2NWQWp0a1dmN1gvQ0xMbU5RNGVxZnNVZitCU202NmJn?=
 =?utf-8?B?eHRVOFJRTWZFc05uNHBER0JUR25CVmMzWm80enFQZkJINmtMeStNa1h0WjVF?=
 =?utf-8?B?LzBzU2E2VHNyUi9UdkJ4U3BLS2FCR1hSL2psbzdOSDNHejV2WjFSTW5FTWN3?=
 =?utf-8?B?REZUbi9Gako3eTJjNmlObk5vMjdsYktVWnFVSEFNUjlmS05BeEsyVVY3ZDFV?=
 =?utf-8?B?MVdaNUdKSWduQ1c3K0tPRnduQ21vS0kyUWVORW55L2duYnRYQWtEQlgwdDRx?=
 =?utf-8?B?VTROTDFLam0rSnMrMldvLzBnQTBiVGRRK29JczBqNVk0a3U5NEVEZEpZc2pj?=
 =?utf-8?B?bHhFTU1zY2ZKM0pZUExEUVBZbnF1QWFDWXdoeXdSK0psa1ZMaEVETVpXTHJB?=
 =?utf-8?B?elROWW12OHhCSjV1UDdPbktva2R5S3VCNGZNVG5rb0ZWWCt1R1R5cUZtTG1B?=
 =?utf-8?B?ZWFLZFhtQW9iZE1CSmVJNXR1RHpGNEFXVGhGU2FUekh3Nk9LcTgrMDFObVB2?=
 =?utf-8?B?MGtYRExwTlFGcVhzM3NySTdzYzErbUIzQ29Yb2JQekNUaHZLdHFDaExNQlVM?=
 =?utf-8?B?RzM5MXpkaUtzb3FrbTFvSU05bWdrczJsa1ErSkhuc3BOSWpjQ1FjWXkvQTBR?=
 =?utf-8?B?NHVUc0ZiUkg5Nm5mSGc4bys2eFJrL1M0UWVFbit3bG42R043RnhhdC9pd0d5?=
 =?utf-8?B?RFlBREpQREp5M3kweVUvc1k1bXlDYTJoTmMwOEk4NHlEOWVrTnlOdVYyTUZo?=
 =?utf-8?B?RTRRWDFEK1B3MjQrTjNwN2JnYkZETXR0MzZJSHd6bzVMV3F0Y3pCVzRranpq?=
 =?utf-8?B?REd6UEY4b0RvMEk4MDJtUGNIWDVIckZscXJPWC9zQWxLQWJBd3JaUHlZTVZV?=
 =?utf-8?B?VFFCTHRRMkJSSVRMVGh0YkZQYnQ2enZ4MnE4c3c0QW5hUW5wZmthWmRZb1hV?=
 =?utf-8?B?b1BjeW5ZdWhVL3VjWDlRWVd0WWcya044WERwL2pvT2lwM1Roemo0YURWM1ln?=
 =?utf-8?B?MlhGdHFPbHVJZmVMYnpNN24wUS9kYVMrUG1DT2M2dkVsa2k4Q1pMMzNGc0x0?=
 =?utf-8?B?MHpCbGttRmIyc1NDU0tIb1JGK3hBUlpLd1lLT0NSd2pIUUI2L1djRWh4dHR6?=
 =?utf-8?Q?ODnhpA/F9cDNhciEI8rqUtT8s?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 68596fbf-38eb-45ba-03d5-08db3cb96aa8
X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB6137.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2023 07:25:26.2374
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: n9Y1w7ivmWKjAGqzQPg522aJgAN9VI89Lk3N+RNsHA1PgA/TaoV+bZyzFuUxXgL4yuq4RZ6hojQACYdHO7MTkQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5178
X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,NICE_REPLY_A,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hello,

On 4/14/2023 10:41 AM, Jain, Ayush wrote:
> Hello Broonie,
> 
> When Running fio-test on latest linux-next tree, I noticed that test hung indefinitely, Going back I see that this problem exists since
> next-20230316 release, After bisecting I landed on the following merge commit by Jens.
> 
> Commit   097d3ca138f9 ("Merge branch 'for-6.4/splice' into for-next")
> 
> Running perf I see following trace and call-stack for fio:
> 
>   Overhead  Command  Shared Object     Symbol
>     25.08%  fio      [kernel.vmlinux]  [k] copy_user_generic_string
>        copy_user_generic_string
>        __do_splice
>        __x64_sys_splice
>        do_syscall_64
>        entry_SYSCALL_64_after_hwframe
>        splice
>        0x1c44be0
>    ...
> 
> On a good kernel I see the following perf trace:
> 
>    Overhead  Command  Shared Object     Symbol
>      49.93%  fio      fio               [.] fio_crc32
>       7.23%  fio      fio               [.] clock_thread_fn
>       2.10%  fio      [kernel.vmlinux]  [k] clear_page_rep
>       1.55%  fio      fio               [.] __fill_random_buf
>       1.35%  fio      [kernel.vmlinux]  [k] loop_queue_rq
>       1.05%  fio      [kernel.vmlinux]  [k] copy_user_generic_string
>       ...
> 
> I see some splice changes being added as the part of merge
> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=for-6.4/splice
> 
> I observe this problem on 3 EPYC system(Zen1,3,4), with the following disk architecture
> 
> Zen1: nvme0n1   931.5G Samsung SSD 970 EVO Plus 1TB
> Zen4: nvme0n1   232.9G Samsung SSD 960 EVO 250GB
> 
> I am running fio as follows:
> 
>     $fio fio-simple.job --filename=/dev/test_vg/test_lv
> 
> where test_lv is mounted as follows:
> 
> NAME                           MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
> loop0                            7:0    0  12.2G  0 loop
> └─md127                          9:127  0  12.2G  0 raid0
>     └─test_vg-test_lv            253:3    0   5.5G  0 lvm
> 
> You can find fio-simple.job at
> https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/io/disk/fiotest.py.data/fio-simple.job
> 
> Fio Version: fio-3.34-25-g07ed
> 
Also adding to these observations

-If we create a filesystem on the raw disk -- Test completes with a Pass

-If there is no Filesystem on the raw disk(loop, nvme) -- Test hangs with the provided trace

> 
> Regards
> Ayush Jain
> > On 4/13/2023 11:55 PM, broonie@kernel.org wrote:
>> Hi all,
>>
>> Changes since 20230411:
>>
>> The sh tree gained a conflict with the mm-unstable tree.
>>
>> The erofs tree gained a conflict with the vfs-idmapping tree.
>>
>> The ext4 tree gained multiple conflicts with the mm-stable tree.
>>
>> The net-next tree gained a conflict with the origin tree.
>>
>> The bpf-next tree gained a conflict with the net-net tree.
>>
>> Non-merge commits (relative to Linus' tree): 10382
>>   11028 files changed, 580165 insertions(+), 249563 deletions(-)
>>
>> ----------------------------------------------------------------------------
>>
>> I have created today's linux-next tree at
>> git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
>> (patches at http://www.kernel.org/pub/linux/kernel/next/ ).  If you
>> are tracking the linux-next tree using git, you should not use "git pull"
>> to do so as that will try to merge the new linux-next release with the
>> old one.  You should use "git fetch" and checkout or reset to the new
>> master.
>>
>> You can see which trees have been included by looking in the Next/Trees
>> file in the source.  There is also the merge.log file in the Next
>> directory.  Between each merge, the tree was built with a ppc64_defconfig
>> for powerpc, an allmodconfig for x86_64, a multi_v7_defconfig for arm
>> and a native build of tools/perf. After the final fixups (if any), I do
>> an x86_64 modules_install followed by builds for x86_64 allnoconfig,
>> powerpc allnoconfig (32 and 64 bit), ppc44x_defconfig, allyesconfig
>> and pseries_le_defconfig and i386, arm64, s390, sparc and sparc64
>> defconfig and htmldocs. And finally, a simple boot test of the powerpc
>> pseries_le_defconfig kernel in qemu (with and without kvm enabled).
>>
>> Below is a summary of the state of the merge.
>>
>> I am currently merging 357 trees (counting Linus' and 102 trees of bug
>> fix patches pending for the current merge release).
>>
>> Stats about the size of the tree over time can be seen at
>> http://neuling.org/linux-next-size.html .
>>
>> Status of my local build tests will be at
>> http://kisskb.ellerman.id.au/linux-next .  If maintainers want to give
>> advice about cross compilers/configs that work, we are always open to add
>> more builds.
>>
>> Thanks to Randy Dunlap for doing many randconfig builds.  And to Paul
>> Gortmaker for triage and bug fixes.
> 

Regards,
Ayush Jain