2021-06-30 10:23:26

by Vincent Whitchurch

Subject: [PATCH] mmc: dw_mmc: Fix hang on data CRC error

When a Data CRC interrupt is received, the driver disables the DMA, then
sends the stop/abort command and then waits for Data Transfer Over.

However, sometimes, when a data CRC error is received in the middle of a
multi-block write transfer, the Data Transfer Over interrupt is never
received, and the driver hangs and never completes the request.

The driver sets the BMOD.SWR bit (SDMMC_IDMAC_SWRESET) when stopping the
DMA, but according to the manual CMD.STOP_ABORT_CMD should be programmed
"before assertion of SWR". Do these operations in the recommended
order. With this change the Data Transfer Over is always received
correctly in my tests.

Signed-off-by: Vincent Whitchurch <[email protected]>
---
drivers/mmc/host/dw_mmc.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/mmc/host/dw_mmc.c b/drivers/mmc/host/dw_mmc.c
index f85271f5c4fa..845b0745ea37 100644
--- a/drivers/mmc/host/dw_mmc.c
+++ b/drivers/mmc/host/dw_mmc.c
@@ -2083,8 +2083,8 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
continue;
}

- dw_mci_stop_dma(host);
send_stop_abort(host, data);
+ dw_mci_stop_dma(host);
state = STATE_SENDING_STOP;
break;
}
@@ -2108,10 +2108,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
*/
if (test_and_clear_bit(EVENT_DATA_ERROR,
&host->pending_events)) {
- dw_mci_stop_dma(host);
if (!(host->data_status & (SDMMC_INT_DRTO |
SDMMC_INT_EBE)))
send_stop_abort(host, data);
+ dw_mci_stop_dma(host);
state = STATE_DATA_ERROR;
break;
}
@@ -2144,10 +2144,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
*/
if (test_and_clear_bit(EVENT_DATA_ERROR,
&host->pending_events)) {
- dw_mci_stop_dma(host);
if (!(host->data_status & (SDMMC_INT_DRTO |
SDMMC_INT_EBE)))
send_stop_abort(host, data);
+ dw_mci_stop_dma(host);
state = STATE_DATA_ERROR;
break;
}
--
2.28.0

2021-06-30 22:08:21

by Jaehoon Chung

Subject: Re: [PATCH] mmc: dw_mmc: Fix hang on data CRC error

Dear Vincent,

On 6/30/21 7:22 PM, Vincent Whitchurch wrote:
> When a Data CRC interrupt is received, the driver disables the DMA, then
> sends the stop/abort command and then waits for Data Transfer Over.
>
> However, sometimes, when a data CRC error is received in the middle of a
> multi-block write transfer, the Data Transfer Over interrupt is never
> received, and the driver hangs and never completes the request.
>
> The driver sets the BMOD.SWR bit (SDMMC_IDMAC_SWRESET) when stopping the
> DMA, but according to the manual CMD.STOP_ABORT_CMD should be programmed
> "before assertion of SWR". Do these operations in the recommended
> order. With this change the Data Transfer Over is always received
> correctly in my tests.

I will check with your patch. I didn't see any CRC error on my targets before.

Best Regards,
Jaehoon Chung

>
> Signed-off-by: Vincent Whitchurch <[email protected]>
> ---
> drivers/mmc/host/dw_mmc.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/mmc/host/dw_mmc.c b/drivers/mmc/host/dw_mmc.c
> index f85271f5c4fa..845b0745ea37 100644
> --- a/drivers/mmc/host/dw_mmc.c
> +++ b/drivers/mmc/host/dw_mmc.c
> @@ -2083,8 +2083,8 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> continue;
> }
>
> - dw_mci_stop_dma(host);
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_SENDING_STOP;
> break;
> }
> @@ -2108,10 +2108,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> */
> if (test_and_clear_bit(EVENT_DATA_ERROR,
> &host->pending_events)) {
> - dw_mci_stop_dma(host);
> if (!(host->data_status & (SDMMC_INT_DRTO |
> SDMMC_INT_EBE)))
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_DATA_ERROR;
> break;
> }
> @@ -2144,10 +2144,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> */
> if (test_and_clear_bit(EVENT_DATA_ERROR,
> &host->pending_events)) {
> - dw_mci_stop_dma(host);
> if (!(host->data_status & (SDMMC_INT_DRTO |
> SDMMC_INT_EBE)))
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_DATA_ERROR;
> break;
> }
>

2021-07-14 11:50:45

by Vincent Whitchurch

Subject: Re: [PATCH] mmc: dw_mmc: Fix hang on data CRC error

On Thu, Jul 01, 2021 at 12:06:31AM +0200, Jaehoon Chung wrote:
> On 6/30/21 7:22 PM, Vincent Whitchurch wrote:
> > When a Data CRC interrupt is received, the driver disables the DMA, then
> > sends the stop/abort command and then waits for Data Transfer Over.
> >
> > However, sometimes, when a data CRC error is received in the middle of a
> > multi-block write transfer, the Data Transfer Over interrupt is never
> > received, and the driver hangs and never completes the request.
> >
> > The driver sets the BMOD.SWR bit (SDMMC_IDMAC_SWRESET) when stopping the
> > DMA, but according to the manual CMD.STOP_ABORT_CMD should be programmed
> > "before assertion of SWR". Do these operations in the recommended
> > order. With this change the Data Transfer Over is always received
> > correctly in my tests.
>
> I will check with your patch. I didn't see any CRC error on my targets before.

Have you had a chance to check it? You can use the fault-injection
patch if you want to trigger aborted transfers without getting real CRC
errors:

https://lore.kernel.org/linux-mmc/[email protected]/

2021-07-14 23:16:16

by Jaehoon Chung

Subject: Re: [PATCH] mmc: dw_mmc: Fix hang on data CRC error

On 7/14/21 8:49 PM, Vincent Whitchurch wrote:
> On Thu, Jul 01, 2021 at 12:06:31AM +0200, Jaehoon Chung wrote:
>> On 6/30/21 7:22 PM, Vincent Whitchurch wrote:
>>> When a Data CRC interrupt is received, the driver disables the DMA, then
>>> sends the stop/abort command and then waits for Data Transfer Over.
>>>
>>> However, sometimes, when a data CRC error is received in the middle of a
>>> multi-block write transfer, the Data Transfer Over interrupt is never
>>> received, and the driver hangs and never completes the request.
>>>
>>> The driver sets the BMOD.SWR bit (SDMMC_IDMAC_SWRESET) when stopping the
>>> DMA, but according to the manual CMD.STOP_ABORT_CMD should be programmed
>>> "before assertion of SWR". Do these operations in the recommended
>>> order. With this change the Data Transfer Over is always received
>>> correctly in my tests.
>>
>> I will check with your patch. I didn't see any CRC error on my targets before.
>
> Have you had a chance to check it? You can use the fault-injection
> patch if you want to trigger aborted transfers without getting real CRC
> errors:
>
> https://protect2.fireeye.com/v1/url?k=96c33914-c958001f-96c2b25b-002590f5b904-9a89d187f33934b7&q=1&e=308eae3a-44b8-43d7-be54-24e9b850f4ad&u=https%3A%2F%2Flore.kernel.org%2Flinux-mmc%2F20210701080534.23138-1-vincent.whitchurch%40axis.com%2F
>

2021-07-14 23:19:59

by Jaehoon Chung

Subject: Re: [PATCH] mmc: dw_mmc: Fix hang on data CRC error

Hi Vicent,

On 7/14/21 8:49 PM, Vincent Whitchurch wrote:
> On Thu, Jul 01, 2021 at 12:06:31AM +0200, Jaehoon Chung wrote:
>> On 6/30/21 7:22 PM, Vincent Whitchurch wrote:
>>> When a Data CRC interrupt is received, the driver disables the DMA, then
>>> sends the stop/abort command and then waits for Data Transfer Over.
>>>
>>> However, sometimes, when a data CRC error is received in the middle of a
>>> multi-block write transfer, the Data Transfer Over interrupt is never
>>> received, and the driver hangs and never completes the request.
>>>
>>> The driver sets the BMOD.SWR bit (SDMMC_IDMAC_SWRESET) when stopping the
>>> DMA, but according to the manual CMD.STOP_ABORT_CMD should be programmed
>>> "before assertion of SWR". Do these operations in the recommended
>>> order. With this change the Data Transfer Over is always received
>>> correctly in my tests.
>>
>> I will check with your patch. I didn't see any CRC error on my targets before.
>
> Have you had a chance to check it? You can use the fault-injection
> patch if you want to trigger aborted transfers without getting real CRC
> errors:

Sorry for too late. I did my other task. I will check yours on Today.
Anyway, Thanks for checking this issue.

Best Regards,
Jaehoon Chung

>
> https://protect2.fireeye.com/v1/url?k=96c33914-c958001f-96c2b25b-002590f5b904-9a89d187f33934b7&q=1&e=308eae3a-44b8-43d7-be54-24e9b850f4ad&u=https%3A%2F%2Flore.kernel.org%2Flinux-mmc%2F20210701080534.23138-1-vincent.whitchurch%40axis.com%2F
>

2021-07-25 21:43:22

by Jaehoon Chung

Subject: Re: [PATCH] mmc: dw_mmc: Fix hang on data CRC error

On 6/30/21 7:22 PM, Vincent Whitchurch wrote:
> When a Data CRC interrupt is received, the driver disables the DMA, then
> sends the stop/abort command and then waits for Data Transfer Over.
>
> However, sometimes, when a data CRC error is received in the middle of a
> multi-block write transfer, the Data Transfer Over interrupt is never
> received, and the driver hangs and never completes the request.
>
> The driver sets the BMOD.SWR bit (SDMMC_IDMAC_SWRESET) when stopping the
> DMA, but according to the manual CMD.STOP_ABORT_CMD should be programmed
> "before assertion of SWR". Do these operations in the recommended
> order. With this change the Data Transfer Over is always received
> correctly in my tests.
>
> Signed-off-by: Vincent Whitchurch <[email protected]>

Reviewed-by: Jaehoon Chung <[email protected]>

Best Regards,
Jaehoon Chung

> ---
> drivers/mmc/host/dw_mmc.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/mmc/host/dw_mmc.c b/drivers/mmc/host/dw_mmc.c
> index f85271f5c4fa..845b0745ea37 100644
> --- a/drivers/mmc/host/dw_mmc.c
> +++ b/drivers/mmc/host/dw_mmc.c
> @@ -2083,8 +2083,8 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> continue;
> }
>
> - dw_mci_stop_dma(host);
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_SENDING_STOP;
> break;
> }
> @@ -2108,10 +2108,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> */
> if (test_and_clear_bit(EVENT_DATA_ERROR,
> &host->pending_events)) {
> - dw_mci_stop_dma(host);
> if (!(host->data_status & (SDMMC_INT_DRTO |
> SDMMC_INT_EBE)))
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_DATA_ERROR;
> break;
> }
> @@ -2144,10 +2144,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> */
> if (test_and_clear_bit(EVENT_DATA_ERROR,
> &host->pending_events)) {
> - dw_mci_stop_dma(host);
> if (!(host->data_status & (SDMMC_INT_DRTO |
> SDMMC_INT_EBE)))
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_DATA_ERROR;
> break;
> }
>

2021-08-04 11:36:18

by Ulf Hansson

Subject: Re: [PATCH] mmc: dw_mmc: Fix hang on data CRC error

On Wed, 30 Jun 2021 at 12:22, Vincent Whitchurch
<[email protected]> wrote:
>
> When a Data CRC interrupt is received, the driver disables the DMA, then
> sends the stop/abort command and then waits for Data Transfer Over.
>
> However, sometimes, when a data CRC error is received in the middle of a
> multi-block write transfer, the Data Transfer Over interrupt is never
> received, and the driver hangs and never completes the request.
>
> The driver sets the BMOD.SWR bit (SDMMC_IDMAC_SWRESET) when stopping the
> DMA, but according to the manual CMD.STOP_ABORT_CMD should be programmed
> "before assertion of SWR". Do these operations in the recommended
> order. With this change the Data Transfer Over is always received
> correctly in my tests.
>
> Signed-off-by: Vincent Whitchurch <[email protected]>

Applied for fixes and by adding a stable tag, thanks!

Kind regards
Uffe


> ---
> drivers/mmc/host/dw_mmc.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/mmc/host/dw_mmc.c b/drivers/mmc/host/dw_mmc.c
> index f85271f5c4fa..845b0745ea37 100644
> --- a/drivers/mmc/host/dw_mmc.c
> +++ b/drivers/mmc/host/dw_mmc.c
> @@ -2083,8 +2083,8 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> continue;
> }
>
> - dw_mci_stop_dma(host);
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_SENDING_STOP;
> break;
> }
> @@ -2108,10 +2108,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> */
> if (test_and_clear_bit(EVENT_DATA_ERROR,
> &host->pending_events)) {
> - dw_mci_stop_dma(host);
> if (!(host->data_status & (SDMMC_INT_DRTO |
> SDMMC_INT_EBE)))
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_DATA_ERROR;
> break;
> }
> @@ -2144,10 +2144,10 @@ static void dw_mci_tasklet_func(struct tasklet_struct *t)
> */
> if (test_and_clear_bit(EVENT_DATA_ERROR,
> &host->pending_events)) {
> - dw_mci_stop_dma(host);
> if (!(host->data_status & (SDMMC_INT_DRTO |
> SDMMC_INT_EBE)))
> send_stop_abort(host, data);
> + dw_mci_stop_dma(host);
> state = STATE_DATA_ERROR;
> break;
> }
> --
> 2.28.0
>