Received-SPF: pass (google.com: domain of linux-kernel+bounces-199693-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <0c89a5e6-ab77-4028-9779-d8db77525b0c@os.amperecomputing.com>
Date: Mon, 3 Jun 2024 13:34:42 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [v2 PATCH] arm64: mm: force write fault for atomic RMW
 instructions
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: "Christoph Lameter (Ampere)" <cl@linux.com>, will@kernel.org,
 anshuman.khandual@arm.com, scott@os.amperecomputing.com,
 linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
References: <20240520165636.802268-1-yang@os.amperecomputing.com>
 <Zk93vBqAD3LgmbGb@arm.com> <640f8606-2757-4e82-721f-9625d48ded65@gentwo.org>
 <Zk-SNVyEHT1UsxqD@arm.com> <ad87bb77-a9a5-2c0d-b4b2-13db09615d7c@linux.com>
 <Zk-2a7s2pvkVsm2C@arm.com>
 <d18611c7-9108-46f7-a5a5-6c8e0069de9b@os.amperecomputing.com>
 <Zl3qCajhEbC9pNAm@arm.com>
Content-Language: en-US
From: Yang Shi <yang@os.amperecomputing.com>
In-Reply-To: <Zl3qCajhEbC9pNAm@arm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bm1aMHlSREw4OERIVThWcjlKUGtmK0xJQ094SXhOTWw4YWEzem9GWkNwOWQw?=
 =?utf-8?B?dmF1RTFhb0RUWVBHM3VYSG9QNjYwVXlabHFFalp0TWYwOGt2NWlSc2F0S05Q?=
 =?utf-8?B?L0tuMXhFRy8xd3hNblZHR0xaYmdHYUs5S1M2dDl1ek92eStoSm0vaFNsMEw4?=
 =?utf-8?B?aUhsL1JhT0t4cHVkbEp0dFJJTStoWndYR1BxanM3dFpmTnN3ODFacmFzUTVS?=
 =?utf-8?B?TEtEWjk3Ym9pT1EvZnhDOGIzUEl1N05WMHhsK0xTNkE0dlZZendNbk0xQzlL?=
 =?utf-8?B?V3phdFBQd3B0UllNL0ppK3lTQVJpVFgzZWRZQ2VHUEpLRDZPN1JZOWdoVTNa?=
 =?utf-8?B?ODBsWDRxWU9XZE82REhpaDRXcFlhYlp2RkgzbFNiRllCS0JuUXRJaENLczI4?=
 =?utf-8?B?Q1ExSlFZLytwT3BrUFV1Ny9YZ2RaQ1dtdThYZCs2WUxtVVJoSENBeENqQkc4?=
 =?utf-8?B?dlJ5eGQ2UWJuTVFDbW00Qnc1TG5YMytIR0FWSG40dXJGaVlVOXNyS29sUzdh?=
 =?utf-8?B?VThPb2R1L3hSMlI3a1gxNk05bzJocWFZdkdYUzViWGVyMExkZXVHbjd0VnJU?=
 =?utf-8?B?NmhXdTJHZXVGNEI2U0xFMHRvYlBBc0U1ZVkzS3NvN3NJK3g3SmxyYngydCs4?=
 =?utf-8?B?QWY2OUZPdUZHWG9pTHdvZWJFalBwVVgrajJSRjEzYXRMVlI2dFBxZUxmZkZT?=
 =?utf-8?B?RVBYQUVhMDVIWnpMQUNzMlFVNm1IUCtzN3dhdWMvYmJDb2QweGR5SDJBQ3p1?=
 =?utf-8?B?UkMyM29QNWxNd1BYcFI0TzNpWFJ5N00rb1p0UWtqTkFSQzBJVGVrNjVHTkdN?=
 =?utf-8?B?OURkbk05OGQwd3NwZ05pUnhTUjQwVjUvbmlBVXNzWWNGNzJvQ3JTVTc0TSt0?=
 =?utf-8?B?L01WSDdzbTg2Qk1Ob09tTXVaQXVhTlRXQmUybWRWVkhhbGVWM3RlMk9PL3BO?=
 =?utf-8?B?cDgrMDViUDczVW9tVmV2VFR5NmdtZkhMdmtmOHlkZUpta3dpYk9sUTVBcmhM?=
 =?utf-8?B?KzErOXUrLzFRY0JIUW95WWViOGRpODZGcU81SjdaTUQ4NExYSnU2TDJRUnlR?=
 =?utf-8?B?bjNBbWovVFhiekp5T08yTVFiSWZlNE5IMWw0Zkh0NjQ4WWpiU25qbi9MOHAw?=
 =?utf-8?B?UVBKeHAwQ1JOVzZqaFlneVJzUi9YRUk2VXlxb0FNb1BuRFRKODRHNEZMNkw1?=
 =?utf-8?B?YWc0cENjc3VKa0pEWDV0VStjb3p3RmY5dTRaTkpzSXl3bEJNVDVjVHpTWElm?=
 =?utf-8?B?b1FKc2dwMytSdktGd2NSMFlRYTIzK0k4aGJsRUp0QVNaTDRlSWpJdjk1MWF0?=
 =?utf-8?B?TStXSm5rYVcyallIdnpOckpqZkFkV2hUTGRCazBINW94R0tHaEEwRmRrYnEz?=
 =?utf-8?B?ejVWL2VBSVFSK0JGNDlGTUJJRjdMSFZ4Q3VMNmZzbFEyMGRyRTNRS09zeHFw?=
 =?utf-8?B?allXZFhPcXRqNENVczFDUzgxdFpodDJyRnNuZUxBNFA1ZVVTbVd4QlRmVk9F?=
 =?utf-8?B?cUtkWXluV1R3RzhmbVViK2w4VWZOS1VoNlYvZ3FEMnMyZGw3NWx4RU5NTjhS?=
 =?utf-8?B?MkJzUnVkRjUxbVRKODlFU1hMZWlCVjFzNWlSdURRWldrVktCckhFSmpEMm9L?=
 =?utf-8?B?VWZ1Zmc3L0FCdjVQVmZlTThDMzJLRnY3UlJJQ1o1YlRnV2pwRk1haXZoOVZj?=
 =?utf-8?B?dHF3ZlVGTDJONFhjdndQbXpqQ2NjSCt2MVNHZExrWG1SdHNBVEFFeWdEOUh0?=
 =?utf-8?B?eDF5Qmx5V1NISmRXSUdqREgrenVPNEhEUFVJVEVKZTlJU3R6ZHhDRjAzOTdr?=
 =?utf-8?B?aVMzQ0JwL1BrbjNlZmZNdlM3M2ZEM0s0bTVvOU9Gc0owajQ1TXNPQ2RML0Ji?=
 =?utf-8?B?WUxnZDV1N1ZPQS9QbU15eTVSMEdaYWxFN2dFK3paUFA0RmZkdmZXMWl4U2c3?=
 =?utf-8?B?ZWRHK3VGL3RMS29uUWUyZkJueTRNenFBUUJjYlkvcnhMTmU5Sk1vbDRUSmVk?=
 =?utf-8?B?RWFJSXBPV2hDdVJyUUdTNEtaYWYzMnJqTWNrTzM5R3E1TjVuaGFQSmZIZUpx?=
 =?utf-8?B?VVdPekRaR1ZQUHVPUktJZThyMXFhL1lhNVhrUllya0N5VXNEWnBBaGlUcG1h?=
 =?utf-8?B?Z3VOazc0MnpkaDJpd1p1M3NINjRQem4xWGU5a1J1SDFrUjFKSndXYkNCMW10?=
 =?utf-8?Q?4oX40T3v1n+O4VTKfXvtprA=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 095912bd-4fb4-4616-4a3b-08dc840c9ad6
X-MS-Exchange-CrossTenant-AuthSource: BYAPR01MB5463.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2024 20:34:45.4485
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wwJY1gR6QwhZHPn7efYC0IjtGtkw+iOYd/hK0xAWWVI4T7eUMgwJPZI3UcG/WZdPkoLLOwlt7ymt5FYDtzTvag7No6EddksjZ2QarpNTQuc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN4PR01MB7504


On 6/3/24 9:06 AM, Catalin Marinas wrote:
> On Thu, May 23, 2024 at 03:13:23PM -0700, Yang Shi wrote:
>> On 5/23/24 2:34 PM, Catalin Marinas wrote:
>>> On Thu, May 23, 2024 at 12:43:34PM -0700, Christoph Lameter (Ampere) wrote:
>>>> On Thu, 23 May 2024, Catalin Marinas wrote:
>>>>>>> While this class includes all atomics that currently require write
>>>>>>> permission, there's some unallocated space in this range and we don't
>>>>>>> know what future architecture versions may introduce. Unfortunately we
>>>>>>> need to check each individual atomic op in this class (not sure what the
>>>>>>> overhead will be).
>>>>>> Can you tell us which bits or pattern is not allocated? Maybe we can exclude
>>>>>> that from the pattern.
>>>>> Yes, it may be easier to exclude those patterns. See the Arm ARM K.a
>>>>> section C4.1.94.29 (page 791).
>>>> Hmmm. We could consult an exception table once the pattern matches to reduce
>>>> the overhead.
>>> Yeah, check the atomic class first and then go into the finer-grained
>>> details. I think this would reduce the overhead for non-atomic
>>> instructions.
>> If I read the instruction encoding correctly, the unallocated instructions
>> are decided by the below fields:
>>
>>    - size
>>    - VAR
>>    - o3
>>    - opc
>>
>> To exclude them I think we can do something like:
>>
>> if atomic instructions {
>>      if V == 1
>>          return false;
>>      if o3 opc == 111x
>>          return false;
>>      switch VAR {
>>          000
>>              check o3 and opc
>>          001
>>              check 03 and opc
>>          010
>>              check o3 and opc
>>          011
>>              check o3 and opc
>>          default
>>              if size != 11
>>                  check o3 and opc
>>      }
>> }
>>
>> So it may take 4 + the possible unallocated combos of o3 and opc branches
>> for the worst case. I saw 5 different combos for o3 and opc, so 9 branches
>> for worst cases.
> Or we have a sorted table of exclusions and do a binary search. Not sure
> which one is faster.
>
>> But if they will be allocated to non-atomic instructions, we have to do
>> fine-grained decoding, but it may be easier since we can just filter out
>> those non-atomic instructions? Anyway it depends on how they will be used.
>> Hopefully this won't happen.
> Actually, the atomics table has LD64B and LDAPR already which are load
> instructions, no write permission needed. So we need to exclude these
> and all the unallocated space in this range.

OK. Excluding LD64B and LDAPR actually makes the check much simpler if 
we return true for supported instructions instead of checking 
unallocated instructions. It looks like:

((val & 0x3f207c00) == 0x38200000) |
((val & 0x3f208c00) == 0x38200000) |
((val & 0x7fe06c00) == 0x78202000) |
((val & 0xbf204c00) == 0x38200000)

Thanks D Scott help figure this out.

>