Received: by 2002:a25:7ec1:0:0:0:0:0 with SMTP id z184csp3651006ybc; Mon, 25 Nov 2019 18:34:36 -0800 (PST) X-Google-Smtp-Source: APXvYqxTSN+E5xf8l6y5CcGm0uNOMIg0FLnp3JGzcNEV84iMgwHgk6qNYD+aP5DK6FPNpAu9a9Qp X-Received: by 2002:a50:a5a5:: with SMTP id a34mr22064886edc.217.1574735676097; Mon, 25 Nov 2019 18:34:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574735676; cv=none; d=google.com; s=arc-20160816; b=WAFDrc0zdo5dyXMCYloBnMBOXkqFFKDWeWlgayUhCWACl6v0ff5kGelYWx18psxxar BRmpsRwVu331IJYNCblnnLbOcIX3GFzXTZJvEjDD3CFTl3953QtMcCmlyPJrJxpZWoNJ S11krvieHcMFQ5hEvnhJiL9VFv6ADlInfBPfZVeSMawg3CQS4zwLX3zlqysjqi3wkyrb hCJioxJbDIYz/JbOtHTCCFO+AF8J9jEFpAFRktXTgV7ZkEj/MFGuzwbiMHlxgdiaw7ux wOFa/O67TQUA8k0xXi5+LLogeLN2erh6ZNxN7JN7K9iJOIRR7a2uKPPcj8JbVagON2vz 2XIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-disposition :content-transfer-encoding:user-agent:in-reply-to:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=RbRdWBhJ4blUfXgXJEf9JYkRJ6P4wp0pBcRs6Ey3L2A=; b=zEAi/RsrDi0w1iiReioi06TwnzZeaLulAEgpXGBpQDjkY3CSkWB8AWsxhOMU4ieNsb 50ZOJB/elKwaCr5nkfJlmF2n/VPw8XOTtkuVTtxjZdzivFcbdoXLFuJ/Flq1SPVnGxQg 1tCLTGsSxyxbPBSK2CvXU8r8vQCu05NCe199QYBfXIUK734NP7J7nRVF8mXplmTeVntT 3m9CBJm9jOWIhBgzUl/PPHsVD7ZS8827m/4cYUhScVLiPvPtAn5bNiXPuRAczWF4wHHi ajR2gGvKpG/GkCN8OcXjoaJ+acEIElB1y/QOXI1Y8q7/IqQ03W6voNu8a5IMVk1LFvdv Ee2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Rh+OBdrv; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j23si6298729eja.393.2019.11.25.18.34.12; Mon, 25 Nov 2019 18:34:36 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Rh+OBdrv; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727326AbfKZCdP (ORCPT + 99 others); Mon, 25 Nov 2019 21:33:15 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:47738 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727282AbfKZCdO (ORCPT ); Mon, 25 Nov 2019 21:33:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574735593; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RbRdWBhJ4blUfXgXJEf9JYkRJ6P4wp0pBcRs6Ey3L2A=; b=Rh+OBdrvCtzDEVg+MCC9o7M1sKXNbTTmOpNyacKyQC+weHEyOOkXiP5FzzmRczJRHoWMv0 k1Ro25J+L9DXDayoJirNDCkmfz3jQet7rwUniZ0Y1tBaeTpV5EF7l26ZSCkMeZxBvh/os4 B/6WUDXbLNsqBSeiv+tftOCKYAM8eR0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-151-ATDMGVSrPGWP4Ti-y8ZjIw-1; Mon, 25 Nov 2019 21:33:10 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 153B5100550D; Tue, 26 Nov 2019 02:33:08 +0000 (UTC) Received: from ming.t460p (ovpn-8-20.pek2.redhat.com [10.72.8.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7A0C95C1D8; Tue, 26 Nov 2019 02:32:57 +0000 (UTC) Date: Tue, 26 Nov 2019 10:32:53 +0800 From: Ming Lei To: Andrea Vai Cc: Damien Le Moal , Alan Stern , Jens Axboe , Johannes Thumshirn , USB list , SCSI development list , Himanshu Madhani , Hannes Reinecke , Omar Sandoval , "Martin K. Petersen" , Greg KH , Hans Holmberg , Kernel development list Subject: Re: Slow I/O on USB media after commit f664a3cc17b7d0a2bc3b3ab96181e1029b0ec0e6 Message-ID: <20191126023253.GA24501@ming.t460p> References: <20191109222828.GA30568@ming.t460p> <20191123072726.GC25356@ming.t460p> <20191125035437.GA3806@ming.t460p> <20191125102928.GA20489@ming.t460p> <20191125151535.GA8044@ming.t460p> <0876e232feace900735ac90d27136288b54dafe1.camel@unipv.it> MIME-Version: 1.0 In-Reply-To: <0876e232feace900735ac90d27136288b54dafe1.camel@unipv.it> User-Agent: Mutt/1.12.1 (2019-06-15) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: ATDMGVSrPGWP4Ti-y8ZjIw-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Nov 25, 2019 at 07:51:33PM +0100, Andrea Vai wrote: > Il giorno lun, 25/11/2019 alle 23.15 +0800, Ming Lei ha scritto: > > On Mon, Nov 25, 2019 at 03:58:34PM +0100, Andrea Vai wrote: > >=20 > > [...] > >=20 > > > What to try next? > >=20 > > 1) cat /sys/kernel/debug/block/$DISK/hctx0/flags > result: >=20 > alloc_policy=3DFIFO SHOULD_MERGE|2 >=20 > >=20 > >=20 > > 2) echo 128 > /sys/block/$DISK/queue/nr_requests and run your copy > > 1GB > > test again. >=20 > done, and still fails. What to try next? I just run 256M cp test to one USB storage device on patched kernel, and WRITE data IO is really in ascending order. The filesystem is ext4, and mount without '-o sync'. From previous discussion, looks that is exactly your test setting. The order can be observed via the following scri= pt: #!/bin/sh MAJ=3D$1 MIN=3D$2 MAJ=3D$(( $MAJ << 20 )) DEV=3D$(( $MAJ | $MIN )) /usr/share/bcc/tools/trace -t -C \ 't:block:block_rq_issue (args->dev =3D=3D '$DEV') "%s %d %d", args->rwbs,= args->sector, args->nr_sector' $MAJ & $MIN can be retrieved via lsblk for your USB storage disk. So I think we need to check if the patch is applied correctly first. If your kernel tree is managed via git, please post 'git diff'. Otherwise, share us your kernel version, and I will send you one backported patch on the kernel version. Meantime, you can collect IO order log via the above script as you did last time, then send us the log. Thanks, Ming