Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Niklas Cassel <Niklas.Cassel@wdc.com>
To:     Damien Le Moal <damien.lemoal@opensource.wdc.com>
CC:     John Garry <john.garry@huawei.com>,
        "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
        "martin.petersen@oracle.com" <martin.petersen@oracle.com>,
        "jinpu.wang@cloud.ionos.com" <jinpu.wang@cloud.ionos.com>,
        "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Linuxarm <linuxarm@huawei.com>,
        yangxingui <yangxingui@huawei.com>,
        yanaijie <yanaijie@huawei.com>
Subject: Re: [PATCH v5 0/7] libsas and drivers: NCQ error handling
Thread-Topic: [PATCH v5 0/7] libsas and drivers: NCQ error handling
Thread-Index: AQHY1/ID0O3xyFitDU+9c+6EHqQqRa3+RKaAgAE7cQCAANLyAIAAAgSAgAAKBoA=
Date:   Wed, 5 Oct 2022 22:11:59 +0000
Message-ID: <Yz4BLTPkXqyjW4a4@x1-carbon>
References: <1664262298-239952-1-git-send-email-john.garry@huawei.com>
 <YzwvpUUftX6Ziurt@x1-carbon>
 <cfa52b91-db81-a179-76c2-8a61266c099d@huawei.com>
 <27148ec5-d1ae-d9a2-1b00-a4c34d2da198@huawei.com>
 <Yz33FGwd3YvQUAqT@x1-carbon>
 <5db6a7bc-dfeb-76e1-6899-7041daa934cf@opensource.wdc.com>
In-Reply-To: <5db6a7bc-dfeb-76e1-6899-7041daa934cf@opensource.wdc.com>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?fhktV39ZQHoJ/aQt2cVLdUL/ArpqGDaFndgsBSDJW2I14btADRGy4sbDBawo?=
 =?us-ascii?Q?q+CZa4OCKyoPGA1v/cFmn4ztylyL6syxSdeLZ5b2peg+CHxSjs+ZQtBahRhl?=
 =?us-ascii?Q?EzjrOmeVQyKfMZo1hlC1RrShvgJigtmf5e1jhYYgzmspZMZ59LlJv8HVUM+3?=
 =?us-ascii?Q?FtUNPusmhwyXmDdNVfYQu/LULjv4XesmSqiRpBhbExivbdkBJxSu4qtC4vnR?=
 =?us-ascii?Q?dBYpJNQX84y+4sevsmVdVpnLBS644DtXRGN/cF6+MxARQ5vKcv6LtQ9gVDsm?=
 =?us-ascii?Q?ZxRK8YhIK6Z9AyFlAxO2lUZOg2BqdWxi4OOw1u7EsdyXJY1/24BSIMieEnAO?=
 =?us-ascii?Q?GKT1wvHmZihLA45k31an12PwokjRfcSkSFRsbuF1ZDj1YPjp+rQpxE3EIAMP?=
 =?us-ascii?Q?DwpSqQvMa+sTnhosZgP22aPM/OdvqBTc1tyMaHQrG8KX7I9G7E/OHKyD6zbe?=
 =?us-ascii?Q?AQFSMAkUCgtqGHRQg1sdaTak45V9IB9QAZg6atRCHn8+6eop7V3Ggg/HQAid?=
 =?us-ascii?Q?+ceHHWppCHotBtp0MQO9hnJc2gmNysuQCj4hKOJiWOqo4ZGZ4dkOUq37nzVC?=
 =?us-ascii?Q?cgkiaSwqfkq+99iJKIs/evz1KlA6BDc4ylUKE6mJGSeFdHNla/mETSFvLyI3?=
 =?us-ascii?Q?VGOKtkIFSShFXZIDqw3dEjxEgaCs+faWRc4IAuxFDnDKrI2vpAyYQdrp/Al4?=
 =?us-ascii?Q?Jpv+BLzSnzCcZpMvI+J+GWOqFkWfAJsZb6DzCpuoo8D2I6MP5pzgAAn3LmgT?=
 =?us-ascii?Q?zzUUMvesLuEggV6ZNNvTei2QKocq9L8Fw3IGxJldp4ix0na+MCSWBc+jdOSS?=
 =?us-ascii?Q?fEYfszXBfhe7lHgy6daXYs2/5KnhJAtC0EPJR+gbdAr1Na1ql7cXHnqXaW9X?=
 =?us-ascii?Q?JkizSwO2pSAB2PfPoNlMA/i0Z3Lk2ql4/02Dg/Jc+QbsTt71QXzGSr7sqvp7?=
 =?us-ascii?Q?8bsEOhkWUO5Bx+221Hi7hjU7jcaDGA1njtq3ehhVMU0OV8BXHuS2ve20ki5+?=
 =?us-ascii?Q?FhcWKZHkn6/cEwU89wTHnWwnFR5LWhjaCE22jbmTVUcegvNK0V6xNI6i4DY4?=
 =?us-ascii?Q?c3EJNB1Qe7oikeXQNY82yUxfMei+baX22gRKH4iujjxOfpP776T2qjxRBjyX?=
 =?us-ascii?Q?YC3FWScEtLIrzONquwA1Rhwi3Wp31qL3hslr7EX/RJmNpXs78tatCLtEAFNC?=
 =?us-ascii?Q?wG6R8F4bHRHJxbfumqnjMT+uSaXkkqc8IdRdx0H3ovHqosMO1wUZpKiuaJV+?=
 =?us-ascii?Q?9LP6C9hLHRtAkTF87VDfwTvkp7bNtxd6hQ+d8KBgaE/Nl2hPbTCYpxziX7Gv?=
 =?us-ascii?Q?L6yoYIEgvOZZoKg8wVbYIzkUsgyM3WPfv8gTAQXFnrlG9VadGjFwSRauNG1f?=
 =?us-ascii?Q?Gi9wcHaRtElkiC+SF57cbkRQpEj/wTROrGvRao6h688w8ON27t+sQzHFZV+i?=
 =?us-ascii?Q?W7zb2MzG1S1QkLM7rAcEhDz5YXq+Z8nDUVJXDHHI0zDLoRzJQAHUTbFSfdQT?=
 =?us-ascii?Q?5k5qDzt8+E6LlItQe4PsftlsXtLmkBqMn4uR7e36//46ml+lqeUQ9+Oun9Lg?=
 =?us-ascii?Q?ioQOlBP1fqOs44w6MZd1BYndK/GxBN8ZAE4XZYl7Waq0jq34Z10sAGgEMVwP?=
 =?us-ascii?Q?3w=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3E4CA75C85B2A14889B3E3F4B623C6C2@namprd04.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?K1MwXwhE8r47N8dml10qhj48EVnz3MmTZQQYjjDzhLYoOCtRvLceR74GAtPD?=
 =?us-ascii?Q?29iI+xS+bZMS99HK8dAGlw6kg3ppMAYqDF9VoNLyGOCfbj9Lje66kM3ZVMNo?=
 =?us-ascii?Q?3Dwca8C0Z/VRKsRpVL+LbsHj/j48/0D1bobdpPn2UB6Bt6Xs5YefeP5mc2Ku?=
 =?us-ascii?Q?SaD+dfFdhU9Wa7MO23+9HYd12kKO8IgHToT9XOCjR/vTJ6ECsnrNQ6Nz1rXq?=
 =?us-ascii?Q?/8T5UcFpINbjf2cQ9Uop82AYzBnUordSkHDGEFhNRGWH0YYt5nfQzPfxEaUK?=
 =?us-ascii?Q?GWxdX9EbvE/7KuBVFkLV/hHbCvYzeLwRYGD6GaKVuM3pfP69hog4R6MQgkJL?=
 =?us-ascii?Q?3OzfI35Eu6PQaXstzDJWke+e1PdQt7oS8TUxXht00XG7XVxQhKO+r/0sWb7E?=
 =?us-ascii?Q?zgjWUdeyHDf/ihKeukuqJKZ11mKWhg1uFoGPl4eXTKLQBQQN1BR5T5n6mLVS?=
 =?us-ascii?Q?/xM+9JBLbxK2sm29e0PxzsBjtSwESjki80uKRNCjBWhOjj7Sfh1vl4X0+gAY?=
 =?us-ascii?Q?wyzi0BGVsy+cXL9rLmCBFNLhwakPQJK9xEjIA8pyb5ZC0Jn+p9/5VlXNXmOt?=
 =?us-ascii?Q?pGG9LzNU+uRzNmaUfOyUyB//JfAvcLYVxZa9FG7mA9mAYEKUShVjJ++vKTNV?=
 =?us-ascii?Q?G4XwYLEGfo/7J7GvRHsFh1B4ASXjnO4KONBJzqlg/oyn6QvkKegN8k+w3tXk?=
 =?us-ascii?Q?P5Ou1DfNBqMOdjxGmHgkb2xasSJGqjY3d+uNTZv1WrY/cnYYADIIVsV75kmP?=
 =?us-ascii?Q?fZi4Q9F4/tjKJr2pYf9r6O+Ir4h64Zb4EcaOSds2xdZT6xaFSKIkRwVltkD2?=
 =?us-ascii?Q?i6H5b3xI6S+KiDrmssI8IyglPgeHpvgyDgjcyjbETMs0J4Jz0QWUQhkZ/AUJ?=
 =?us-ascii?Q?AOGuFForVqaPviWIBeufKmVBZxj9PbQJbo3BrN1La8aUCH6Vi5QH2HtVVAd1?=
 =?us-ascii?Q?+CAC8jFXZNhNMChXiK0rVFexyFquSiuIr5xoh/t2MReYx5odTZVFf//ezWEC?=
 =?us-ascii?Q?ciieaAMB3ue4fCLsH7Lbv5yZLx7+SEX+RCg9FrG15JrY50s=3D?=
X-OriginatorOrg: wdc.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MN2PR04MB6272.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 522bd0ee-5360-4609-7881-08daa71e9f77
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Oct 2022 22:11:59.2142
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: g8CByTnWZaK4raBrH3UeFoprx1pT1Lm3lGiUetinF24+3BzlKodkuqs5nR+Sl03UnMrQwnvAzKgblOiorKMmCA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR04MB1146
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Oct 06, 2022 at 06:36:05AM +0900, Damien Le Moal wrote:
> On 10/6/22 06:28, Niklas Cassel wrote:
> > On Wed, Oct 05, 2022 at 09:53:52AM +0100, John Garry wrote:
> >> On 04/10/2022 15:04, John Garry wrote:
> >>
> >> Hi Niklas,
> >>
> >> Could you try a change like this on top:
> >>
> >> void sas_ata_device_link_abort(struct domain_device *device, bool
> >> force_reset)
> >> {
> >> 	struct ata_port *ap =3D device->sata_dev.ap;
> >> 	struct ata_link *link =3D &ap->link;
> >>
> >> +	device->sata_dev.fis[2] =3D ATA_ERR | ATA_DRDY;
> >> +	device->sata_dev.fis[3] =3D 0x04;
> >>
> >> 	link->eh_info.err_mask |=3D AC_ERR_DEV;
> >> 	if (force_reset)
> >> 		link->eh_info.action |=3D ATA_EH_RESET;
> >> 	ata_link_abort(link);
> >> }
> >> EXPORT_SYMBOL_GPL(sas_ata_device_link_abort);
> >>
> >> I tried it myself and it looked to work ok, except I have a problem wi=
th my
> >> arm64 system in that the read log ext times-out and all TF show "devic=
e
> >> error", like:
> >=20
> > Do you know why it fails to read the log?
> > Can you read the NCQ Command Error log using ATA16 passthrough commands=
?
> >=20
> > sudo sg_sat_read_gplog -d --log=3D0x10 /dev/sdc
> >=20
> > The first byte is the last NCQ tag (in hex) that failed.
>=20
> libata issues read log as a non-ncq command under EH. So the NCQ error lo=
g
> will not help.

Hello Damien,

John explained that he got a timeout from EH when reading the log:
[  350.281581] ata1: failed to read log page 10h (errno=3D-5)
[  350.577181] ata1.00: exception Emask 0x1 SAct 0xffffffff SErr 0x0 action=
 0x6 frozen

ata_eh_read_log_10h() uses ata_read_log_page(), which will first try to rea=
d
the log using READ LOG DMA EXT. If that fails, it will retry using READ LOG=
 EXT.

Therefore, to see if this is a driver specific bug, I suggested to try to r=
ead
the NCQ Command Error log using ATA16 passthrough commands:

$ sudo sg_sat_read_gplog -d --log=3D0x10 /dev/sdc
will read the log using READ LOG DMA EXT.

$ sudo sg_sat_read_gplog --log=3D0x10 /dev/sdc
will read the log using READ LOG EXT.

Neither of these two suggested commands are NCQ commands.
(Neither command is encapsulated in a RECEIVE FPDMA QUEUED,
so I'm not sure what you mean.)


Garry, I now see that:
[  350.577181] ata1.00: exception Emask 0x1 SAct 0xffffffff SErr 0x0 action=
 0x6 frozen
Your port is frozen.

ata_read_log_page() calls ata_exec_internal() which calls ata_exec_internal=
_sg(),
which will simply return an error without sending down the command to the d=
rive,
if the port is frozen.

Not sure why your port is frozen, mine is obviously not.

ata_do_link_abort() calls ata_eh_set_pending() without activating fast drai=
n:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/dri=
vers/ata/libata-eh.c?h=3Dv6.0#n989

So I'm not sure why your port is frozen.
(The fast drain timer does freeze the port, but it shouldn't be enabled.)
It might be worthwhile to see who freezes the port in your case.


Kind regards,
Niklas=