Received-SPF: pass (google.com: domain of linux-kernel+bounces-117333-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <3ea54627-339f-4d19-896d-5843a8b9e55e@oracle.com>
Date: Mon, 25 Mar 2024 15:10:07 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] scsi: libsas: Add SMP request allocation handler callback
To: Jason Yan <yanaijie@huawei.com>, Damien Le Moal <dlemoal@kernel.org>,
        Yihang Li <liyihang9@huawei.com>, jejb@linux.ibm.com,
        martin.petersen@oracle.com, chenxiang66@hisilicon.com
Cc: linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org,
        linuxarm@huawei.com, prime.zeng@huawei.com, yangxingui@huawei.com
References: <20240325131751.1840329-1-liyihang9@huawei.com>
 <da4de4e0-6d13-4509-b288-a9d122d1c2d3@kernel.org>
 <e7612e81-dc44-7bc6-3bd2-0b71367408ec@huawei.com>
Content-Language: en-US
From: John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <e7612e81-dc44-7bc6-3bd2-0b71367408ec@huawei.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?alRtTTJNWEtCRUlPVjlvK0N2V2ptZ2t0WVRHM2lSQUxyMzR5SW84STB5dlZW?=
 =?utf-8?B?akxsRlZSM1dCRWxsQ2w2TlNNYXpDZ1YzNXA0bUN1bi9kLzEzcWViOHQ1Ni8v?=
 =?utf-8?B?WHFxYllBbi9iQWF0aXNiRUhlSVl2TFhXZEh0ZzhtSDBYRUVzejhoRFFHSUVC?=
 =?utf-8?B?QkFqNlhCTXQxQmJTUDJ1NTd3RFpvWWpKZDBKbyt1S2xXTUY1dTNvdmhHMVNC?=
 =?utf-8?B?Mm41MVY1UzJXRlZSK2ZzNjZLSUo2d3BGMDVHRlFhQUx0Z3BnMTlIUE1rZXFG?=
 =?utf-8?B?UUk3ejJaVHZFNlBEZ3duenZCMDlHdEFhaE0wYWRIQ0luNjBraEhkTHozNWRP?=
 =?utf-8?B?SU1aa0hGTE13QnpIU2l3eWw3V2NSckE3RmFPYW1UeW1CTkYwMkwvNy96cUx2?=
 =?utf-8?B?L3E4Vk5rNjRqVzJRaVFrc256M2Y3Qi9zQStqdUF4RHVkVENkMWJQU3g2OFA1?=
 =?utf-8?B?SEIxR0xuR0x0cUNyYWtJajhaR2cyNkpqQitZMkJaRDZaazl4LzNkU0t5KytX?=
 =?utf-8?B?Y3R3UFMrWDlUSGFlNUgxQm5FZFZzSUpjY3ovYVVieFpscWI3ZmVTUjdPY3di?=
 =?utf-8?B?dVRCK25kRWVxUnM1dlFHWFBPaU1LOUtoSFV4UUpaeGkyR05rTE53bk1pRnhq?=
 =?utf-8?B?NVRmOGFTQkNGK1pKNlB1T0IyNVhyU0liY0tzQkd3MDNjOGUzNEdpTE94Smh5?=
 =?utf-8?B?Ymw3eTRkVHpJY05kWEpzVUpLWHR2UVdmNWVuQ2s0NTYzMzcyd3dGeFVBMUJJ?=
 =?utf-8?B?OVk5QmVRNzYwMExBMmVZQ29SUGtDYzRVandKWGdkanZmak54QkVFTURKbGxU?=
 =?utf-8?B?S0ozUEtTWGc3Z01EWCtWRGRVMG5ySTlCc2tVNmtrbHJpMEJWbUJzMTlLVEJM?=
 =?utf-8?B?QzJud0tSaE15cmJRQkcwV1RsMnBuRFdDdERDWFFOamZHMzVmUXVjcVc0ak5z?=
 =?utf-8?B?a0hsYkNhbms5OUd1MUJ1RUJKSE5ub09FV3VkZDNxV1psbVhtd25XWTRKdkJh?=
 =?utf-8?B?bTRqamJpb2g3elpxczIwdlFEZUFLQW82VkpKWDF3cGRvRWs2TEVYR3d0dDNO?=
 =?utf-8?B?UzZuak83UUJ6T0F5MkVzYVg2TFY1MTk4cFovUG5LNlZYK3JaNlNZSUVjZHpZ?=
 =?utf-8?B?L0Zrb0pDR2pmU0pwRS96ajVtVUdqUUw0UTJtRmZDMVBObW9aWVFxaFRwZ3Zu?=
 =?utf-8?B?cTIzK0x5d3JJMkc1RDNvQlJjL09sSmM1QTVNUkdEdnh3d2xTeThqd0NqQWpV?=
 =?utf-8?B?RTdMeVY0Q0lpSDM1aGI4Tnp0UjF1L1JuNTBsZENYY01XVXRqMGRiVmdzOWkv?=
 =?utf-8?B?cXNFSEVpZXpSeGVRN00wek1HTlYxbm4ydHdpSFNRUk8zTWN3RC9mRkc5N1di?=
 =?utf-8?B?OXc3WnNSNTI2VURrWlpnb1FWeHIrQ3NIdmhJbEdkYWNaYUZJWTNBckpWSHpM?=
 =?utf-8?B?VjA3WVJCcmF0MVM5YlcyU29SK252NTE1MlZNQmxHYUdLZXVPRTBUSGVmd0Va?=
 =?utf-8?B?Q1AyaUVaQ0N2em1MdUlKOXFMb0EvVEtlVnMzZXhJNitpZ2V6WHJXTUo4L1VC?=
 =?utf-8?B?NWV6eHduWmE5SC80K084MHpSR1pxa3pVOEVsT2o4VUNZZ1M0aktXWnEwMmhL?=
 =?utf-8?B?L0dmYmNzRWc0eXc5Rll3UEJiQXd5OEFLanRVall4QVNaU1RjNlU0WU9CN2JV?=
 =?utf-8?B?VkU2T2k5NGIrN2pZSnpnUjIrWDVhc0QxMWFuMU1menpBUE95enJjbDhqOGlD?=
 =?utf-8?B?V3c0Q0twWE9EREtjR1pMZUcwWWdBRUcycXgzN2QvTEdKRjMzN2FYcGdjNFJY?=
 =?utf-8?B?c1g4UjNkaWczY3dFajkyRWk5MkkwOVZpMkJhUmtoZStqWGhWOHRETDY5M3Bw?=
 =?utf-8?B?NTE1TW1Mdkp2Umt1WmcrSHcyK0tzOFNoUC82OTBUQlRGYkozRGI2UDJvTm5i?=
 =?utf-8?B?Q3U3VWlUWXBpa1JqMHBrVHBjQmo2S3FDM1FtdFN3QjcrMmRnR3NWNW1XdVZh?=
 =?utf-8?B?cmUzOFpUZTJwb21OMjJBbkI0UGlLSXoycGIyQUhXM1BZb2xDeHZwUkFDVysy?=
 =?utf-8?B?S0JmRmZvb0pyR3lNdTdCRW0xQ1ZZNHA3ZlBGclh1RTAxQjRWWDRGQU9HY2tY?=
 =?utf-8?Q?5JddI1uBOd/mmLBlUzNEiDUKD?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	E3FaY2aZbVlJMabPUjGKF/J2kMVPk6NFkWZgmJDnMrPAXGpg42yaq3EcmASX18ucA/caAUWy+L0HbUmdaPdGLIjLqZmNQyvF567FUEFqCKhUMRP9v6D7UXAXpwJE+tzThFOorByYaJ+0ZknNhxGz+8jnSJJuwdkLtE+dyJV7Js1dD+r6wOV8/PX77cTlKOeEjwgWl/EkFEgVXnnOuO3OBZBpTrQDzbxJ4NJUae6JclyQcQLFDqJVJArxmKa4HJHbM0X4oNMcBQSYD5grfljcys+j8TQUqNlBrGoDuaisc6haSVkKmoqKag1MSpD9Eov9oLpABXz0xEMHlkG5ntUeVFkoO6fbyo8CKhG8W3QE6S21XW1hhjgQnr2FQB6QsmV+ak4BqsrrsW8gw1c5VwbtAgEC3gCiR7xeH2xbgjIxwLmXN+L6kpbVyZuuk1k2fH7Rudcx+Ocdnhz43Caz+bcUaUtDXKM/rgD7+Px8yANpYXR0FqBfcTxRn4uUfdECF4YR+FtupgRzD7j8/fLBqLpSyrMVSzTSWR/ijcqL/eNyUgErtE8j6Lcji/ulJv94+ygOjs8nswyjbtIFbt0ZS0TZ61urRSIU5tMWXryBHXMd9NU=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e03d2b3-73e6-4d0b-0e5e-08dc4cddaa4f
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2024 15:10:11.0317
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6wIfbQVhgpfLLjQYUQRDaKbpuCd57IV83opP2m+oC2QtsoSOtfiLhDG/DDOhgvNQvZLKGOh/OOp+BP+h9Lhy2g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB7499
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-03-25_11,2024-03-21_02,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 bulkscore=0
 adultscore=0 mlxscore=0 phishscore=0 malwarescore=0 spamscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2403210000 definitions=main-2403250083
X-Proofpoint-GUID: 3GhrghPW7ImWzc_GTcPUl_hOluJuKXF7
X-Proofpoint-ORIG-GUID: 3GhrghPW7ImWzc_GTcPUl_hOluJuKXF7

On 25/03/2024 14:21, Jason Yan wrote:
> On 2024/3/25 22:03, Damien Le Moal wrote:
>> On 3/25/24 22:17, Yihang Li wrote:
>>> This series [1] reducing the kmalloc() minimum alignment on arm64 to 8
>>> (from 128).
>>
>> And ? What is the point you are trying to convey here ?
>>
>>> The hisi_sas has special requirements on the memory address alignment
>>> (must be 16-byte-aligned) of the command request frame, so add a SMP
>>> request allocation callback and fill it in for the hisi_sas driver.
>>
>> 128 is aligned to 16. So what is the problem you are trying to solve 
>> here ?
>> Can you clarify ? I suspect this is all about memory allocation 
>> optimization ?
> 
> After series [1] been merged, kmalloc is 8-byte-aligned, however 
> hisi_sas hardware needs 16-byte-aligned. That's the problem.
> 
>>
>>>
>>> Link: 
>>> https://urldefense.com/v3/__https://lkml.kernel.org/r/20230612153201.554742-1-catalin.marinas@arm.com__;!!ACWV5N9M2RV99hQ!L85qzmByNbkBbZByXKuNMAhvt8wxKPCsogKt3Pgn93DTkzfc54jA3of5XL8oEDDDDTMU1OtghdKiLZdKe5ub$  [1]
>>> Signed-off-by: Yihang Li <liyihang9@huawei.com>
>>> ---
>>>   drivers/scsi/hisi_sas/hisi_sas_main.c | 14 ++++++++++++
>>>   drivers/scsi/libsas/sas_expander.c    | 31 ++++++++++++++++++---------
>>>   include/scsi/libsas.h                 |  3 +++
>>>   3 files changed, 38 insertions(+), 10 deletions(-)
>>>
>>> diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c 
>>> b/drivers/scsi/hisi_sas/hisi_sas_main.c
>>> index 097dfe4b620d..40329558d435 100644
>>> --- a/drivers/scsi/hisi_sas/hisi_sas_main.c
>>> +++ b/drivers/scsi/hisi_sas/hisi_sas_main.c
>>> @@ -2031,6 +2031,19 @@ static int hisi_sas_write_gpio(struct 
>>> sas_ha_struct *sha, u8 reg_type,
>>>                   reg_index, reg_count, write_data);
>>>   }
>>> +static void *hisi_sas_alloc_smp_req(int size)
>>> +{
>>> +    u8 *p;
>>> +
>>> +    /* The address must be 16-byte-aligned. */
>>
>> ARCH_DMA_MINALIGN is not always 16, right ?
>>
>>> +    size = ALIGN(size, ARCH_DMA_MINALIGN);
>>> +    p = kzalloc(size, GFP_KERNEL);

Please make it clear that kmalloc() will return a naturally-aligned 
memory for power-of-2 sizes, and so ensuring that size is roundup to 16B 
will give a data which is aligned to 16B

>>> +    if (p)
>>> +        p[0] = SMP_REQUEST;
>>> +
>>> +    return p;
>>> +}
>>> +
>>>   static void hisi_sas_phy_disconnected(struct hisi_sas_phy *phy)
>>>   {
>>>       struct asd_sas_phy *sas_phy = &phy->sas_phy;
>>> @@ -2130,6 +2143,7 @@ static struct sas_domain_function_template 
>>> hisi_sas_transport_ops = {
>>>       .lldd_write_gpio    = hisi_sas_write_gpio,
>>>       .lldd_tmf_aborted    = hisi_sas_tmf_aborted,
>>>       .lldd_abort_timeout    = hisi_sas_internal_abort_timeout,
>>> +    .lldd_alloc_smp_req    = hisi_sas_alloc_smp_req,
>>
>> Why this complexity ? Why not simply modify alloc_smp_req() to have 
>> the required
>> alignment ? This will avoid a costly indirect function call.
> 
> Yeah, I think it's simpler to modify alloc_smp_req() directly too. 
> Yihang, Can you please cook a new one?
> 
> Thansk,
> Jason