Received-SPF: pass (google.com: domain of linux-kernel+bounces-62026-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <f1b96616-296d-85ec-3e59-54b6035fa954@intel.com>
Date: Mon, 12 Feb 2024 08:56:53 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.11.0
Subject: Re: [PATCH] dmaengine: idxd: Clear Event Log head in idxd upon
 completion of the Enable Device command
Content-Language: en-US
To: Dave Jiang <dave.jiang@intel.com>, Vinod Koul <vkoul@kernel.org>
CC: <dmaengine@vger.kernel.org>, linux-kernel <linux-kernel@vger.kernel.org>,
	Lingyan Guo <lingyan.guo@intel.com>
References: <20240209191851.1050501-1-fenghua.yu@intel.com>
 <36895817-8f71-461a-93e0-5db1a39cd3c4@intel.com>
 <18761e12-822a-16b4-fdc5-0ac889b1693c@intel.com>
 <c643b6a3-0f41-416f-9268-3072f76939e3@intel.com>
From: Fenghua Yu <fenghua.yu@intel.com>
In-Reply-To: <c643b6a3-0f41-416f-9268-3072f76939e3@intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VmUwRy9RdC9XWm9uVWdSYkdVL1JEbkpJd2NWVENRM0JIUHJqUGxqMnpSR0I2?=
 =?utf-8?B?TG93WlhERllWOWFrQmlXWmR0bzBCbE5qKzcvenFPMTlNTmVxYjc5Qlh4NWNX?=
 =?utf-8?B?ZldVdEVqakNRbGYyZTIzOUtOeTlsUHBjUFBOTnBaelBVVk00Q0tzZXk5QjJR?=
 =?utf-8?B?cExTZXFURHhKVWRRSkN4Rjd6UGpKQk9YdXdpRFcxcFdQYUw4WktiUXdjVm5h?=
 =?utf-8?B?TWdwVUdObkJxM2lJYUJFYWxzWldxRmR3YWt5aTZYbFFSTHl0R09kK0RhdHBm?=
 =?utf-8?B?OEVSYU9KdTRPcnIwUGpVRXVIYkJnRWNiVFE5MVI3Q1VQZG16N0VjZEJMQnNo?=
 =?utf-8?B?UVQ0eHlzdzZOUTVxa09VTEU2bksyN2svQURIVHBxeUNoOG1HejNmRU9uYm55?=
 =?utf-8?B?NTlmblZvQTJxeDFCQXhpd0NPdktpdUp6WWtnWjF1V0xtZms0V3VoZVlkd3o3?=
 =?utf-8?B?ZzV4dUF3SlpxbVErWk5iaTQ1OXdiM21KeXJWUTBuZTROaTVLZWhMNXBzekFW?=
 =?utf-8?B?WjlDdFdDVHpUSUJDT21hMnJmQy8zKzNtanhUV0xFRDhWbmdRLzZ2UEpzcWhG?=
 =?utf-8?B?VTBTWVZIdThQcG9wQ1g2SUR0Q0FKbEhyYk4rVi95MFBzLy9lR3ltSDRGRVI2?=
 =?utf-8?B?MitpdzQrTGFDQlpEVXNSWUFvWUlIQ0NMV05UNDRXUE8wTGlDV1VOYlIvZ0tB?=
 =?utf-8?B?OGFXcStzOWxJbVlsMmUvQnh2TmhCTWpVTHBEUWdMNWcwbm10YU5aM0hDRENs?=
 =?utf-8?B?WFZySkxXdDdDbTF4N2tkelVVSm9SSnZib3pXanpiWFJ2bFlrVkZ0SnoyMzMy?=
 =?utf-8?B?b1d1ZTBBUjNCeHFiTmRCcEZ2bS9wdGhETHJUbzAzOXh4dUJJM2liNDUwNmhS?=
 =?utf-8?B?UDdjSER1blVrSjkvVVRyOEtOcVczYkRTQVpHK2JGQjNJdStPR2haUTlLRjh6?=
 =?utf-8?B?Ym5la1J4K0dqd0RYNWp5cnJMQ0oyc2Jtdzkvam9BbGtGbEQyNFFEYitXdUJP?=
 =?utf-8?B?S1JYc3phbjg3M21tUWY4Tis1Q1d4TG4vUmJSN2owTWpVb1JCNU9NbGh2TjI1?=
 =?utf-8?B?ZHY0eVRVUWhuVmFsaXFGYXlpZkxRQXgyVXkwdTJLL2Nub3B4bUt0akdHd05S?=
 =?utf-8?B?ajQ3ZVZ2Y0tiVSt1eFBqTExiUjVndFh5RS9RRWlYUzJkZXRSK09xVERDcU9h?=
 =?utf-8?B?U3YwSm5yNEtFY3lidmRiL05RdE5FdWg3R2l4OHZPYnBRYXV1NTdhQ0wvcVBk?=
 =?utf-8?B?Y2VoU1ZFTnpMbjl2OUFIVXdvd1ZrajcxZkVYdmptbTdUNnBiVVNWU0VJK1Fi?=
 =?utf-8?B?WlpaTWNUMDBMalJVMzg0a28va1N1bnMwWUZFMHVsM0F1UmtlV09vNFIwLy9Q?=
 =?utf-8?B?OWIrVVNPZHc1cXF6NW1xUnN6MXJzS05rOHBCOVh2TXpRdnFISmFNZFZaSWVR?=
 =?utf-8?B?MVA3K2xpREhmbjhocVpiSGV5dTBlUjBTcjBCSEp5eUdsWDhmanc1S1VHTWpJ?=
 =?utf-8?B?Q2E1a202T3N0NGxaMkEwM0VqSUlkVkZmQ2Q2aHZEcEVCQXRCYkxKMkNyY3Mx?=
 =?utf-8?B?NWk3UkFKMyt4VTRnd3ZDbnk5aDBuRG9tc0doaUVqM2VGUW1uSUpWU083ZUZv?=
 =?utf-8?B?a2UydlNVRjIzMXU4MzE3RGZ2N1RGQXUxQTZUbjVJTktuNWJSOFNlVlFiWTZO?=
 =?utf-8?B?bXVTTnUyZWJpcEs2eUlXT1hwdTZ5VDZ3YmlEcWhXZXQ0cWpUK0dJaUxuYzVZ?=
 =?utf-8?B?NElUaFUwTDF5MjRpeXZYVGRTanFwZlNiSFREeFp1a0ZkUHBZOTZsQnBMZFQz?=
 =?utf-8?B?ZkVITWFEczBkWnVDZVc0bG85YUk5VXNpRVl1Ni9nS2pKcmpmS3dJZHppcWJu?=
 =?utf-8?B?MWxNcEJrNEpmNE9JRFFWYmlMK1h2NXAxL1h1RVRXSWdKNHo5d1A4dS9UZS96?=
 =?utf-8?B?eGlZWTJHVUh4MWpXRDVIWFlvcVpyVjkrRkNMeXYyOGZHUDE4VEhqeU5zZW1t?=
 =?utf-8?B?eWNjUmNBZWZuVHQyOUdiMnJDRGNkMTFCZnpIazY0OGQxZ1MrbXhYaHlkaTB3?=
 =?utf-8?B?RWlIYWxWQjdUNzNONE1XQ3AxMUo5U2I5ZWkweTJLejgvSWhVbkR2YTBjZWFE?=
 =?utf-8?Q?QU2NLFyhGUEgstqpWrgqrm9gi?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 50178d03-e1c3-4a01-7088-08dc2bebac19
X-MS-Exchange-CrossTenant-AuthSource: IA1PR11MB6097.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Feb 2024 16:57:18.6699
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Y3pPB5oZNEJaq8jLYoDebvRiH+j0ztKJUkyacDUzbc092HM6AJSPk6z/yTqyjft5wczbw3vEtNXWbMCHIcdq8Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4724
X-OriginatorOrg: intel.com

Hi, Dave,

On 2/12/24 08:39, Dave Jiang wrote:
> 
> 
> On 2/10/24 6:49 PM, Fenghua Yu wrote:
>> Hi, Dave,
>>
>> On 2/9/24 12:17, Dave Jiang wrote:
>>>
>>>
>>> On 2/9/24 12:18 PM, Fenghua Yu wrote:
>>>> If Event Log is supported, upon completion of the Enable Device command,
>>>> the Event Log head in the variable idxd->evl->head should be cleared to
>>>> match the state of the EVLSTATUS register. But the variable is not reset
>>>> currently, leading mismatch of the variable and the register state.
>>>> The mismatch causes incorrect processing of Event Log entries.
>>>>
>>>> Fix the issue by clearing the variable after completion of the command.
>>>
>>> Should this be done in idxd_device_clear_state() instead?
>>
>> If clear evl->head in idxd_device_clear_state(), evl->head still mismatches head in EVLSTATUS in some cases.
>>
>> For exmample, when a few event log entries are logged and then device is disabled, head in EVLSTATUS is still a valid non-zero value. Clearing evl->head in idxd_device_clear_state() when disabling device makes evl->head and head in EVLSTATUS mismatched.
>>
>> I haven't thought a failure test case when they mismatch in these cases though.
>>
>> But while thinking evl->head more, I wonder why is it even needed?
>>
>> head of event log can always be read from EVLSTATUS instead of from its shadow evl->head. And reading head from EVLSTATUS won't degrade performance because tail is always read from EVLSTATUS whenever head is read (no matter from evl->head or from EVLSATUS).
>>
>> To avoid any mismatch issue/trouble, I think the right fix is to remove head definition in struct idxd_evl and always read head from EVLSTATUS.
>>
>> Do you think this is the right fix?
> 
> I was to avoid register reads during event processing. But if you think there's no performance impact then feel free to read directly from the register.
This code reads head and tail in event processing:
         h = evl->head;
         evl_status.bits = ioread64(idxd->reg_base + IDXD_EVLSTATUS_OFFSET);
         t = evl_status.tail;

As you can see EVLSTATUS is always read to get tail. Reading the 
register to get tail cannot be avoid because that's required by hw. 
Reading head from the register won't add additional cost.

That's why I would say no impact on performance without the shadow head 
in idxd.

I will cook the v2 patch by removing the shadow head.

Thanks.

-Fenghua