Received-SPF: pass (google.com: domain of linux-kernel+bounces-126681-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Subject: Re: Implementing .shutdown method for efa module
Message-ID: <0e7dddff-d7f3-4617-83e6-f255449a282b@amazon.com>
Date: Mon, 1 Apr 2024 16:23:32 +0300
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: Jason Gunthorpe <jgg@ziepe.ca>
CC: Tao Liu <ltao@redhat.com>, Gal Pressman <gal.pressman@linux.dev>,
	<sleybo@amazon.com>, <leon@kernel.org>, <kexec@lists.infradead.org>,
	<linux-kernel@vger.kernel.org>, <linux-rdma@vger.kernel.org>
References: <CAO7dBbVNv5NWRN6hXeo5rNEixn-ctmTLLn2KAKhEBYvvR+Du2w@mail.gmail.com>
 <5d81d6d0-5afc-4d0e-8d2b-445d48921511@linux.dev>
 <CAO7dBbXLU5teiYm8VvES7e7m7dUzJQYV9HHLOFKperjwq-NJeA@mail.gmail.com>
 <b6c0bd81-3b8d-465d-a0eb-faa5323a6b05@amazon.com>
 <20240326153223.GF8419@ziepe.ca>
From: "Margolin, Michael" <mrgolin@amazon.com>
In-Reply-To: <20240326153223.GF8419@ziepe.ca>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit

Jason

Thanks for your response, efa_remove() is performing reset to the device 
which should stop all DMA from the device.

Except skipping cleanups that are unnecessary for shutdown flow are 
there any other reasons to prefer a separate function for shutdown?


Michael

On 3/26/2024 5:32 PM, Jason Gunthorpe wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> On Tue, Mar 26, 2024 at 02:34:45PM +0200, Margolin, Michael wrote:
>> Hi Tao,
>>
>> Thanks for bringing this up.
>>
>> I've unsuccessfully tried to reproduce this kernel panic using production
>> Red Hat 9.3 AMI (5.14.0-362.18.1.el9_3.aarch64).
>>
>> Are there any related changes in the kernel you are testing?
>>
>> Anyways we do need to handle shutdown properly, please let know if calling
>> to efa_remove solves your issue.
> efa_remove should not be used for shutdown..
>
> If you have an iommu in your system (smmuv3 for this ARM64 case) then
> drivers must implement a shutdown handler or you will risk data
> corruption on ARM64 sytems during crash.
>
> The shutdown handler must stop all DMA from the device.
>
> If you don't have an iommu then the shutdown handler shouldn't be
> critical.
>
> Jason