Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2;
Message-ID: <ca04bff3-a89e-4cd3-b833-6ab076f3c44c@os.amperecomputing.com>
Date:   Wed, 29 Nov 2023 16:05:23 +0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 RESEND 2/2] i2c: aspeed: Acknowledge Tx done with and
 without ACK irq late
Content-Language: en-CA
To:     Andi Shyti <andi.shyti@kernel.org>
Cc:     Brendan Higgins <brendan.higgins@linux.dev>,
        Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        Joel Stanley <joel@jms.id.au>,
        Andrew Jeffery <andrew@codeconstruct.com.au>,
        Wolfram Sang <wsa@kernel.org>,
        Jae Hyun Yoo <jae.hyun.yoo@linux.intel.com>,
        Guenter Roeck <linux@roeck-us.net>, linux-i2c@vger.kernel.org,
        openbmc@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org,
        linux-aspeed@lists.ozlabs.org, linux-kernel@vger.kernel.org,
        Cosmo Chou <chou.cosmo@gmail.com>,
        Open Source Submission <patches@amperecomputing.com>,
        Phong Vo <phong@os.amperecomputing.com>,
        "Thang Q . Nguyen" <thang@os.amperecomputing.com>
References: <20231128075236.2724038-1-quan@os.amperecomputing.com>
 <20231128075236.2724038-3-quan@os.amperecomputing.com>
 <20231129004509.ilriuwm3hulvy67l@zenone.zhora.eu>
From:   Quan Nguyen <quan@os.amperecomputing.com>
In-Reply-To: <20231129004509.ilriuwm3hulvy67l@zenone.zhora.eu>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Nk5SeFV0ZUI4WFZCVC9DTlIxOUUyOVk1ZGorS1lmOGtlZVF6dzl5Mit4Q1JI?=
 =?utf-8?B?bjNUNmZPVENyNEhQcis4QU0xU25OdSs5UC85RVpNOExQaUJTRW4ySC9ia29R?=
 =?utf-8?B?bCtad0FUMEdRL1JTS3R0TjlBMlg1c1ljbVpSTFBLc0Nua3NmNUZoUUVZRXdw?=
 =?utf-8?B?Wm45OUZ5cDM1bEN3MDNCblMyRWJTS3BSdFAyVmNLV1JBK1VTSXBzYnUxQ2FC?=
 =?utf-8?B?VGZtNmZ0UGllRjhUc0ZJZzF5TXowT0Y0eFVHYnR6bzl4WUIzYVRSbisvUHhG?=
 =?utf-8?B?c0pGaXRUL3duaElZMlUvWllaNWtzdzUxMlhvT29ucHVBK0lKa0NZZHRrenVN?=
 =?utf-8?B?M2g5dm5abW92RmhiTDlVVjd0anJJTW9vZ0dySkJtbzlrU1lQSXZaMTZLYisw?=
 =?utf-8?B?LzU2N3k3TWd4amo1dDlEMWlub2pieHBwVERBWXVJbS9RYTNZbHNYcXpCcXdQ?=
 =?utf-8?B?UHJSZjFQZ3ZndFVpRUcrb3FRTDN0V3ptdWppcVBZNnBsWkZPSFRrMUNDdHR4?=
 =?utf-8?B?OUNrYWNiTDdKclVVVWw2eUI2U2I3WXRNRm5IZG53MTh1ZUVQZTFsVENiMld6?=
 =?utf-8?B?R2Vtc1pNcnJtOUVaNVl6czZMdEE0cTVXaTZjOHZKYnhLOHQ5UCt4a2lPdmoy?=
 =?utf-8?B?ZWFQSE1zdVdOVGgzemEvQmlGN2RuZHBIUWVoYWhGZmwwZ1VINys2MllFWXdw?=
 =?utf-8?B?TURFa2FhcDJrSFRXTTI0cGtseStxMGRVNmQrU0VxbHYzeHd2R2M2Tit2UU42?=
 =?utf-8?B?ZEFNN3FHZE1TSlptVDg4c0hLaG0wQ3dvL0ovcTJPM20xZEROVURSUGNXY1Yx?=
 =?utf-8?B?alBxbXZYUnRQcGxIa0FFRSttMU9NQk5jZS9WTU9BOVVJd2dXcjlRdmQ0TWp2?=
 =?utf-8?B?T0djemc0ZmkxN3dHUDJWSmk2aUxKdGxoZHdyN0JWZnkzLzAybnd4OTIrNnJP?=
 =?utf-8?B?L1ZIbmVNckFjTDFsNmk4clJYNnZiSTlGTXJiaEp2SC9GTWYvQVhndExmR1Jp?=
 =?utf-8?B?NUIxZ3BIb0VvdklhUkZUb1dGbmJGUGJkQmxxYmdxMW5wcmNaTytwalViYkZv?=
 =?utf-8?B?MWpVcTJUR0cvNE1XeHNGYnVLaUV0MnBLN2ZYT3VwVGc3cmZ0R1VwVWM2MDRq?=
 =?utf-8?B?MVdlNEdaeG4zV1BWaE8vZzhCWXVYUmk0L2FBc0hJQ3U2dDBxNXBBRkw3MTFR?=
 =?utf-8?B?STJxUlIvTHVXRXZCejUvN3k0L1hZT2g3NnRZaUJ6blBzNVQ2SlJsZE00aVU0?=
 =?utf-8?B?cDNpWmJzYkVXT3pwU1NKeTV6TkREai9Ob2I2RDdON0Q5elphM3dsRGVoaldy?=
 =?utf-8?B?aXFZTmJzUjdWTlVXWFd3dnAwWTNXbjlDOHJyVXRjZW5ZcE9NK1RwVWEvbTdt?=
 =?utf-8?B?SVpJQ3RuaEFVVXpNeHJ1TVpGbkRYM2tEL2w4RTg2YlpFQkVBbmlzRnc4MzBC?=
 =?utf-8?B?NEZGV1BSRUovdG5KblJ1eDEveUtvZ0kvTkVldXNSWWRTQUVzSHlZRXZKSFVI?=
 =?utf-8?B?MTErdnBEK1JHcVMxMmJtV2RwVUk3bHlPQVRpcWhQZ3NHZ1FzRUxPclFWbTE3?=
 =?utf-8?B?Y0t0a3ZTSFJ5bmgrVStUVURMS3orTlJEUjB6ZVc2LzdSNnEzK3FVZ3dURTgz?=
 =?utf-8?B?RzZrUk5QYkFodUFpUzJpbm0xNEQzZW8raDBVUjBoVVV5UlIvVFM5WC9TU3Q2?=
 =?utf-8?B?bnRzSzdsMncwNlNkTzVpdG0xdmVId2F3Mzg4VkJEK3dTUGtNcllTYVJjTzkw?=
 =?utf-8?B?RHBPNWFoQkJkN3M0T0dMU0NFOEJoUHRwcHRRYS9MR1RNdDFCWC9aWFpuUGNK?=
 =?utf-8?B?cExicU1SV3o4eG9LdFJpNE02M3h0RGtLWGpodm53Z3J5TC9SMkZ6T2RRcktS?=
 =?utf-8?B?SDdUclFLNERZNU1UdkxUeGFjWkRacUtkTExnMmg2dzcyMlhIWXVTWmdiT2Va?=
 =?utf-8?B?TFBJcGVZQ09udExybzY1QmkrVFRMbXdNYVpQL0YxSS8za1l1L3I1ZElFVmNC?=
 =?utf-8?B?U2VOUndvUTB2VElFNFNJK1RnczhIVFAwTmY4TS8yRVhnMVcwOENoODY3cFFJ?=
 =?utf-8?B?UHRIVHVGTk9zN2htbkJhbXc3YkNtdWpqV21ZMXFqTWxKblQvSkNnbE83VkRs?=
 =?utf-8?B?M2dlZjd1dUh3bHpVaXRoOWU2NjdJQk1MZysyWWZwbk1ZZGpZeHFnRHd1TzY5?=
 =?utf-8?Q?K/XgXRV9iahbLiHFuulVmnE=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e3d4e511-96c3-448d-1ce7-08dbf0ba559f
X-MS-Exchange-CrossTenant-AuthSource: SN4PR01MB7455.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Nov 2023 09:05:29.5219
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8uyMd9ayObHqfmTRBxKo6DC2agsHOaQjEfjlYljSkOrRDiEO4fpN4PNpCudDxlZqnOCS6by4JeUBNKNOuF0UMQCxBfo0oEVbH6qO5u3/nn8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR01MB6302
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 29 Nov 2023 01:05:52 -0800 (PST)


On 29/11/2023 07:45, Andi Shyti wrote:
> Hi Quan,
> 
> On Tue, Nov 28, 2023 at 02:52:36PM +0700, Quan Nguyen wrote:
>> Commit 2be6b47211e1 ("i2c: aspeed: Acknowledge most interrupts early in
>> interrupt handler") acknowledges most interrupts early before the slave
>> irq handler is executed, except for the "Receive Done Interrupt status"
>> which is acknowledged late in the interrupt.
>> However, it is observed that the early acknowledgment of "Transmit Done
>> Interrupt Status" (with ACK or NACK) often causes the interrupt to be
>> raised in READ REQUEST state, resulting in "Unexpected ACK on read
>> request." complaint messages.
>>
>> Assuming that the "Transmit Done" interrupt should only be acknowledged
>> once it is truly processed, this commit fixes this issue by acknowledging
>> this interrupt for both ACK and NACK cases late in the interrupt handler
>> also.
>>
>> Fixes: 2be6b47211e1 ("i2c: aspeed: Acknowledge most interrupts early in interrupt handler")
>> Signed-off-by: Quan Nguyen <quan@os.amperecomputing.com>
>> ---
>> v2:
>>    + Split to separate series [Joel]
>>    + Added the Fixes line [Joel]
>>    + Fixed multiline comment [Joel]
>>    + Refactor irq clearing code [Joel, Guenter]
>>    + Revised commit message [Joel]
>>    + Revised commit message [Quan]
>>    + About a note to remind why the readl() should immediately follow the
>> writel() to fix the race condition when clearing irq status from commit
>> c926c87b8e36 ("i2c: aspeed: Avoid i2c interrupt status clear race
>> condition"), I think it looks straight forward in this patch and decided
>> not to add that note. [Joel]
>>
>> v1:
>>    + First introduced in
>> https://lore.kernel.org/all/20210519074934.20712-1-quan@os.amperecomputing.com/
>> ---
>>   drivers/i2c/busses/i2c-aspeed.c | 17 +++++++++--------
>>   1 file changed, 9 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/i2c/busses/i2c-aspeed.c b/drivers/i2c/busses/i2c-aspeed.c
>> index 79476b46285b..3231f430e335 100644
>> --- a/drivers/i2c/busses/i2c-aspeed.c
>> +++ b/drivers/i2c/busses/i2c-aspeed.c
>> @@ -611,8 +611,9 @@ static irqreturn_t aspeed_i2c_bus_irq(int irq, void *dev_id)
>>   
>>   	spin_lock(&bus->lock);
>>   	irq_received = readl(bus->base + ASPEED_I2C_INTR_STS_REG);
>> -	/* Ack all interrupts except for Rx done */
>> -	writel(irq_received & ~ASPEED_I2CD_INTR_RX_DONE,
>> +	/* Ack all interrupts except for Rx done and Tx done with/without ACK */
>> +	writel(irq_received &
>> +	       ~(ASPEED_I2CD_INTR_RX_DONE | ASPEED_I2CD_INTR_TX_ACK | ASPEED_I2CD_INTR_TX_NAK),
>>   	       bus->base + ASPEED_I2C_INTR_STS_REG);
>>   	readl(bus->base + ASPEED_I2C_INTR_STS_REG);
>>   	irq_received &= ASPEED_I2CD_INTR_RECV_MASK;
>> @@ -657,12 +658,12 @@ static irqreturn_t aspeed_i2c_bus_irq(int irq, void *dev_id)
>>   			"irq handled != irq. expected 0x%08x, but was 0x%08x\n",
>>   			irq_received, irq_handled);
>>   
>> -	/* Ack Rx done */
>> -	if (irq_received & ASPEED_I2CD_INTR_RX_DONE) {
>> -		writel(ASPEED_I2CD_INTR_RX_DONE,
>> -		       bus->base + ASPEED_I2C_INTR_STS_REG);
>> -		readl(bus->base + ASPEED_I2C_INTR_STS_REG);
>> -	}
>> +	/* Ack Rx done and Tx done with/without ACK */
>> +	writel(irq_received &
>> +	       (ASPEED_I2CD_INTR_RX_DONE | ASPEED_I2CD_INTR_TX_ACK | ASPEED_I2CD_INTR_TX_NAK),
>> +	       bus->base + ASPEED_I2C_INTR_STS_REG);
>> +	readl(bus->base + ASPEED_I2C_INTR_STS_REG);
> 
> So, you are acknowledging everything here. Why wasn’t it done
> this way in the first place?
> 
> I would appreciate a comment here from Guenter, whose commit you
> are fixing.
> 

Thanks Andi for the comment.

This base on my observation that HW may proceed to start 
transmit/receive new date as soon as those irqs are early ack. This may 
cause a race condition because SW was not actually process that irq yet.

I've also put some explanation in my reply to Andrew in the other mail 
for this part as well.

And of course, I definitively love to hear from Guenter as well as these 
code is just based on my observation through debug only.

Thanks a lot for the comment.
- Quan