Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
IronPort-SDR: Z2MZW0VT9ro+oGwwVU4G8vdWL34ZxyvAdNj0Ivgf35jXjeKkqAc2kBK+JM3fMbf+b/4zTRrJJ7
 pa/r6anBYy/wLQoZ3UNT/+2d95MWyznyyRruZi1RwNkgOD6V9v7QywZORk42dHI7GLhqAz1FnB
 uhQCQcRNfsTS2KnLuAtjVfWhMH42tKga0mcIcu/E2UeadBeN1YKmcHIUYYyDtmc8ljiefvUBL9
 kjkOe8d6YdZRfDVKaU7451+FEfsu7gd01q/I50Vd5GwqsWc2+Wbbcb0MtsNcCDqx4yTCH1Y2dQ
 Xa1Grrs6UMTwsilfVtGFQUUk
IronPort-SDR: p/X49+wkvZo37mGW2DPTcZ92vBYYHP5n6zmmlsn1VnuN8vnUFdwqefqYczNo4AcqIKnhkSg/gh
 DWZ8OLBe3Mqi/3Mhw3Lai/AKciOQKGU00sXoFanIGvxJlgRwG0UKVfDXj2kt+OC+im092riIly
 JxV6G/5EJx20a62nkvxWSPxuED2LG066un0b9l91FjMO9kbFno90AHWRmsKRW0QbTO0pQxkEg3
 7nNkEC4shgGda7zq2qORtVaS/egxMQwpaownWDof5BPfrkB+mAlCKFOooTaFVlVnHA2+Fsh9nP
 SSQ=
WDCIronportException: Internal
Message-ID: <64ab35a7-f1ff-92ee-890e-89a5aee935a4@opensource.wdc.com>
Date:   Thu, 6 Oct 2022 07:42:30 +0900
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.3.1
Subject: Re: [PATCH v5 0/7] libsas and drivers: NCQ error handling
Content-Language: en-US
To:     Niklas Cassel <Niklas.Cassel@wdc.com>
Cc:     John Garry <john.garry@huawei.com>,
        "jejb@linux.ibm.com" <jejb@linux.ibm.com>,
        "martin.petersen@oracle.com" <martin.petersen@oracle.com>,
        "jinpu.wang@cloud.ionos.com" <jinpu.wang@cloud.ionos.com>,
        "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Linuxarm <linuxarm@huawei.com>,
        yangxingui <yangxingui@huawei.com>,
        yanaijie <yanaijie@huawei.com>
References: <1664262298-239952-1-git-send-email-john.garry@huawei.com>
 <YzwvpUUftX6Ziurt@x1-carbon>
 <cfa52b91-db81-a179-76c2-8a61266c099d@huawei.com>
 <27148ec5-d1ae-d9a2-1b00-a4c34d2da198@huawei.com>
 <Yz33FGwd3YvQUAqT@x1-carbon>
 <5db6a7bc-dfeb-76e1-6899-7041daa934cf@opensource.wdc.com>
 <Yz4BLTPkXqyjW4a4@x1-carbon>
From:   Damien Le Moal <damien.lemoal@opensource.wdc.com>
Organization: Western Digital Research
In-Reply-To: <Yz4BLTPkXqyjW4a4@x1-carbon>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 10/6/22 07:11, Niklas Cassel wrote:
> On Thu, Oct 06, 2022 at 06:36:05AM +0900, Damien Le Moal wrote:
>> On 10/6/22 06:28, Niklas Cassel wrote:
>>> On Wed, Oct 05, 2022 at 09:53:52AM +0100, John Garry wrote:
>>>> On 04/10/2022 15:04, John Garry wrote:
>>>>
>>>> Hi Niklas,
>>>>
>>>> Could you try a change like this on top:
>>>>
>>>> void sas_ata_device_link_abort(struct domain_device *device, bool
>>>> force_reset)
>>>> {
>>>> 	struct ata_port *ap = device->sata_dev.ap;
>>>> 	struct ata_link *link = &ap->link;
>>>>
>>>> +	device->sata_dev.fis[2] = ATA_ERR | ATA_DRDY;
>>>> +	device->sata_dev.fis[3] = 0x04;
>>>>
>>>> 	link->eh_info.err_mask |= AC_ERR_DEV;
>>>> 	if (force_reset)
>>>> 		link->eh_info.action |= ATA_EH_RESET;
>>>> 	ata_link_abort(link);
>>>> }
>>>> EXPORT_SYMBOL_GPL(sas_ata_device_link_abort);
>>>>
>>>> I tried it myself and it looked to work ok, except I have a problem with my
>>>> arm64 system in that the read log ext times-out and all TF show "device
>>>> error", like:
>>>
>>> Do you know why it fails to read the log?
>>> Can you read the NCQ Command Error log using ATA16 passthrough commands?
>>>
>>> sudo sg_sat_read_gplog -d --log=0x10 /dev/sdc
>>>
>>> The first byte is the last NCQ tag (in hex) that failed.
>>
>> libata issues read log as a non-ncq command under EH. So the NCQ error log
>> will not help.
> 
> Hello Damien,
> 
> John explained that he got a timeout from EH when reading the log:
> [  350.281581] ata1: failed to read log page 10h (errno=-5)
> [  350.577181] ata1.00: exception Emask 0x1 SAct 0xffffffff SErr 0x0 action 0x6 frozen
> 
> ata_eh_read_log_10h() uses ata_read_log_page(), which will first try to read
> the log using READ LOG DMA EXT. If that fails, it will retry using READ LOG EXT.
> 
> Therefore, to see if this is a driver specific bug, I suggested to try to read
> the NCQ Command Error log using ATA16 passthrough commands:
> 
> $ sudo sg_sat_read_gplog -d --log=0x10 /dev/sdc
> will read the log using READ LOG DMA EXT.
> 
> $ sudo sg_sat_read_gplog --log=0x10 /dev/sdc
> will read the log using READ LOG EXT.
> 
> Neither of these two suggested commands are NCQ commands.
> (Neither command is encapsulated in a RECEIVE FPDMA QUEUED,
> so I'm not sure what you mean.)
> 
> 
> Garry, I now see that:
> [  350.577181] ata1.00: exception Emask 0x1 SAct 0xffffffff SErr 0x0 action 0x6 frozen
> Your port is frozen.
> 
> ata_read_log_page() calls ata_exec_internal() which calls ata_exec_internal_sg(),
> which will simply return an error without sending down the command to the drive,
> if the port is frozen.
> 
> Not sure why your port is frozen, mine is obviously not.
> 
> ata_do_link_abort() calls ata_eh_set_pending() without activating fast drain:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/ata/libata-eh.c?h=v6.0#n989
> 
> So I'm not sure why your port is frozen.
> (The fast drain timer does freeze the port, but it shouldn't be enabled.)
> It might be worthwhile to see who freezes the port in your case.

Might come from the command timeout. John has had many problems with the
pm80xx HBA in his Arm machine from a while back. Likely not a driver issue
but a hw one... No-one seems to be able to recreate the same problem.

We need to try the HBA on our Arm board to see what happens.

> 
> 
> Kind regards,
> Niklas

-- 
Damien Le Moal
Western Digital Research