Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1;
Message-ID: <f86b6eae-15c8-4f3b-affc-f24265f3da23@os.amperecomputing.com>
Date:   Wed, 13 Sep 2023 13:16:20 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] i2c: designware: Fix corrupted memory seen in the ISR
Content-Language: en-US
To:     Yann Sionneau <ysionneau@kalrayinc.com>,
        Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc:     Jarkko Nikula <jarkko.nikula@linux.intel.com>,
        Mika Westerberg <mika.westerberg@linux.intel.com>,
        Jan Dabros <jsd@semihalf.com>,
        Andi Shyti <andi.shyti@kernel.org>,
        Philipp Zabel <p.zabel@pengutronix.de>,
        linux-i2c@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20230913010313.418159-1-janb@os.amperecomputing.com>
 <4537fa1c-b622-674c-026e-8453eda0a4d1@kalrayinc.com>
 <ZQGbaXTnIk0NIZbK@smile.fi.intel.com>
 <038e40c8-d6cd-150e-933e-5b2d2d7af7c5@kalrayinc.com>
 <2657e064-c537-c898-668f-b5e82228ec5a@kalrayinc.com>
From:   Jan Bottorff <janb@os.amperecomputing.com>
In-Reply-To: <2657e064-c537-c898-668f-b5e82228ec5a@kalrayinc.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dmZMaVdVRU03VW14ejVaNUczMFJoS2JPMkhDc0ZrbW1hSGVFSmNMeFI2OTU4?=
 =?utf-8?B?UVdOUmJHYUhoSFpHa3h1V0NEL2ZRMHU0c1lLUGV0WFZuSXpIbWJmZzBCaldM?=
 =?utf-8?B?SEZETzFlWFE5V21JVG05Kzl3M0tsVnEvOVlndjRsemNRMzF4Wk02VXlsbGtX?=
 =?utf-8?B?ek9XSU1NcStpV1ZCejRsNGYvOVhYMFUwZEFUYU83TGpJZk42MzIraEZIUkUz?=
 =?utf-8?B?WVVLbXFSWWl2QWdPMFFXZWtjWHVSai9xU2E1SkJaRE03SXFvWVFyTWVHQWQ3?=
 =?utf-8?B?OGp3K2FjUHhuelRMMmQwRmdhcWR6by9RWU4zUDFyL3NmT3E0cWZxbmFwV1lo?=
 =?utf-8?B?UlpwKzhLcDBWa0hMdW5JT2wvKzMzeU1LVHNRMFNkOElxTW1jVWYxRmFycG5D?=
 =?utf-8?B?Zk80MjN3TDhZSFdwTWNMS09CS0UrMTJGZDF3YTBHYlBKYTdNVllhY0c4ZHBB?=
 =?utf-8?B?a2pBdFJlSVZWWERacE1UWDNvQ0Z1S0YwaGdneVpNeDNPREgzVnQvaGgzaEZS?=
 =?utf-8?B?RkNRdVVhQnRlU3B4Z3FHeVpFQTJjMWk3aFAvUStpNGJrK054aStHMzRha0t2?=
 =?utf-8?B?ZkJhQUlzMVR4OUhsYm02S3M3YTNTakIySTF6ZzNPVmpuTkJZdEloQlJ2S3pa?=
 =?utf-8?B?Z1NHaHFrOHhDNGFSdms5bVFtR3pDTHRMUi9ZL1M5eGVWUG1lUTlOOG93Y0dt?=
 =?utf-8?B?VFZmcE1acXc1QUV6c0hJOUhUQ2t3NWtiTWs3aGJRSFBHdlR5S0RqdlVXbnl4?=
 =?utf-8?B?SW83ZzhGTUlMTy9tbWhjcEdlLzlWaC9ZRzU3WitHUG8reXcwTUc3U0lSWFNh?=
 =?utf-8?B?NVo1SzArd1RPdHhabkhkbTBNTEFRclJHZjdEUzd2SlVSemo0b0FGQXExMkhF?=
 =?utf-8?B?YkVOWmJoUVEweEhtR0pqdUIwamhnTVkyN3JpM0ZRVE5SaWtFWmlFMDBYeVk3?=
 =?utf-8?B?UnRUZnc1dStwWElCaXdlbW9KQkxJQkxKMmIvN3czT2djTnBJTUVKemVuN3h5?=
 =?utf-8?B?aTdYZkJ4cjQzK3UrRjNldks1QmpaYnZiMW5EQ2Q4MndIODBVSktsdUUxTm9k?=
 =?utf-8?B?NHRXa1VDei9nd2pqT3hENkdlR09WU3hlT1ppSXdKc1NyR1R1Z0V6QVhDQlht?=
 =?utf-8?B?RGU5bURPcmZJaURwazdkc2RFdVc3VWFsZ0NNSW03N1QxNXFzcFdFbVRicXIv?=
 =?utf-8?B?cDNFWE4wdFF3NEFIQU9BVHNHK1haZk1wYmlpY1RoQ3ppeXJTdFlVOVFpRDFq?=
 =?utf-8?B?cGFtNnY1dHNhRWdIVjVKZW9kSVltdlEyYmRibWN0K1gyRm95Rzl6Y1A3eXcy?=
 =?utf-8?B?SHVBOGs3RWJZRVd6UkxnMXo2enU2V2xFMTZrZlZPVFc1Ukh5RWpBdVZIR0tV?=
 =?utf-8?B?ay93SzNrV3YyRmhacUEvQXBYc25kWUdzNWxhYW9zQ0dSN0xmcENWanFxQTd0?=
 =?utf-8?B?SkFESHRqV1NnWEx1eVJ0bkI3V0NoRlNwR25BS0RycnpKdTk5VEp3Q2EwczNR?=
 =?utf-8?B?MFFlaUZDRWgyekd6RzVKTGxlRWptbkpwVGhWS240VGl2THFsOTNzUTBuYnRL?=
 =?utf-8?B?RzZia1RyZm5QSGYwSGFxRllyMkF0RDNVR1F4TmR5SHBrVUc0MTVzdHVraWJR?=
 =?utf-8?B?ZHk3U2d5TVhMWEZkQWtad0IwSWVnS05KeFMrME02czdxUkVCQWVBRVJORENm?=
 =?utf-8?B?TUZmbFlSeWNsZFg5dzNLSjYzT3cydkJPZlBQb2U5b1oyVldYQkY5bEZqbjIy?=
 =?utf-8?B?bS9aQktid0dxUXBRUG84Y1JjM1Vnb0FvMEJ2M3hRU2w4Zlc2TlNMWjhwekly?=
 =?utf-8?B?SzU0Rk9zSXlEbVd5SjJTOTZYQlVhbDZlNnlRSFpUWksrSkt6c1RwR2tLekVk?=
 =?utf-8?B?QzVZMmlBUnlpTTl6ZG9NY1kybkhLS1FkdEM4TzVJS29WWDFYOTVTenJyQy9h?=
 =?utf-8?B?RDV0dWU4U1RCRW5WMjVWV0p4Rk1USGIwVG9ueC9YUDZPWnpaamFGdGpHc2tH?=
 =?utf-8?B?RWFLbHpGZTJnbHBXRnRNUTNaaDFLRjhTSHBNVzFMdkwyOFozUERjeHBEbG42?=
 =?utf-8?B?T0FHTnl2S3JteWVaaUdveWs2OS83MkI0bnhXZjM5UEduM2I3Y3lBb1J3dndQ?=
 =?utf-8?B?L1pFZ2hmT3c1S3dDUEQzNy81bWlESDJZa3lveUpCTHhvcVVWUGtmU2w0R3pG?=
 =?utf-8?Q?hFec4PuDfuTTCag4Da1s1cU=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5185142e-8d4b-4bfc-fcdc-08dbb4964f02
X-MS-Exchange-CrossTenant-AuthSource: DM6PR01MB4044.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2023 20:16:26.8130
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bSew85pdLbyoagGSSoW0li74wzfACl88JMdUqE0pASQrfkdlurU86RSupkexM3Kbja2HpiyaAHxvKak6F9i5j/br491OxFhlnbAfTPGU+08=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR01MB7044
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Wed, 13 Sep 2023 13:16:49 -0700 (PDT)
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email

On 9/13/2023 4:54 AM, Yann Sionneau wrote:
> 
> On 13/09/2023 13:32, Yann Sionneau wrote:
>>
>> On 13/09/2023 13:22, Andy Shevchenko wrote:
...
>> To my knowledge the READ_ONCE()/WRITE_ONCE() only imply the use of 
>> volatile to access memory thus preventing the compiler to do weird 
>> optimizations like merging store/loads, moving store/loads, removing 
>> them etc
>>
>> They don't imply a memory barrier.
>>
>> Some systems need a memory barrier, to emit a "fence" like 
>> instruction, so that the pipeline stalls waiting for the store to 
>> "finish", for systems where the writes are posted.
>>
>> Regards,
>>
> Well, for the READ_ONCE() actually I'm wrong, it's overloaded for Alpha 
> and arm64 https://elixir.bootlin.com/linux/latest/C/ident/__READ_ONCE
>

Hi Yann,

I'll make a new submission with an improved commit message.

I looked at that ARM definition for READ_ONCE/WRITE_ONCE, and those 
didn't really look like they they addressed this kind of cross-core 
write visibility, it looked more like it was about avoiding read/write 
tearing. Read acquire and write release can sometimes replace explicit 
barrier instructions, but felt adding an explicit barrier was a simpler 
solution. If this were highly performance critical code, then the 
benefits of avoiding explicit barriers would be worth a lot more effort.

I know the ARM Barrier docs at 
https://developer.arm.com/documentation/genc007826/latest/ have an 
example under "Sending Interrupts and Barriers", page 19, that is very 
specific that a DSB barrier is required in cases where you write to 
normal memory on one core and then trigger an interrupt, and the normal 
memory is read on another core. It explicitly says that even if a memory 
write to strongly ordered memory (device memory) triggers the interrupt, 
you would need a DSB barrier. That example seemed like a very good match 
with what the DW i2c driver was doing.

I also found the video at https://www.youtube.com/watch?v=2I8OHacills 
had some useful explanations of when barriers are needed. It was a 
little puzzling in the video, it seemed to say a DMB barrier would be 
sufficient for this case, even though the ARM barrier docs say a DSB is 
needed. The DMB only stalls execution of memory accesses, the DSB stalls 
execution of all instructions. The Linux wmb macro is a DSB barrier on ARM.

Jan