Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Message-ID: <2a193373-ccb2-458a-890f-d48c4b724e08@intel.com>
Date:   Tue, 5 Dec 2023 08:52:14 -0700
User-Agent: Betterbird (Linux)
Subject: Re: [PATCH v11 14/14] dmaengine: idxd: Add support for device/wq
 defaults
To:     Rex Zhang <rex.zhang@intel.com>, <tom.zanussi@linux.intel.com>
CC:     <davem@davemloft.net>, <dmaengine@vger.kernel.org>,
        <fenghua.yu@intel.com>, <giovanni.cabiddu@intel.com>,
        <herbert@gondor.apana.org.au>, <james.guilford@intel.com>,
        <kanchana.p.sridhar@intel.com>, <linux-crypto@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <pavel@ucw.cz>,
        <tony.luck@intel.com>, <vinodh.gopal@intel.com>,
        <vkoul@kernel.org>, <wajdi.k.feghali@intel.com>
References: <20231201201035.172465-15-tom.zanussi@linux.intel.com>
 <20231205092139.3682047-1-rex.zhang@intel.com>
Content-Language: en-US
From:   Dave Jiang <dave.jiang@intel.com>
In-Reply-To: <20231205092139.3682047-1-rex.zhang@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QTBoWXZPbFZxeVh3aTI2RHVUS0c5elhZMVoweXJpWXYwZE02MjBJQmVRcjRJ?=
 =?utf-8?B?SzhQQThzbGFhL3JRZ0lDOUZEMzdvOHc2U1AvbFNQYlQwdjV3RSt5MHIrV1BB?=
 =?utf-8?B?QkVWZDFHQlllUHR1bC9zV3pjZ29WOW5rL1pKTUZKREFDUFplY3dBSlV6OGxC?=
 =?utf-8?B?NDVBLy9BWjFtQ1d1YThlalJPaW5YTkZPUUpBY3l1dmZ2NGFZUzgxc0ZKQWJF?=
 =?utf-8?B?UUdFa3cxeEFOWFRzZ3JUZ3B5cU1uVkx3ZmFhZ3FRcTQ5R0pESXFxempya09n?=
 =?utf-8?B?K2xISSt6dkREV3p4NUdxa3k5UE9BL1JxRUxUY2xsbHlOZ05sKzBiY0lBZzhL?=
 =?utf-8?B?SEsxSkM1SjQ2cm9hVGpld0ozSHArV2tIT1dJTytNYUE1UDBYQ1RZTXB4WURm?=
 =?utf-8?B?KzFxNkN2dS9NNCtQRW52NTRDay83NDdjZytoOGZQOEJFTDBIeFU2TXhGUFpt?=
 =?utf-8?B?Qk8xZno3V0hDaC8vWG94bjdvZytoc2dOMDVYR2duNTNxWjV6eXB5dzhEYnpD?=
 =?utf-8?B?SlJrQXd3Z2pxczRrWnpuTDZ0NDFheXAyaXo3NTZNNktybkI0MjhId1VtcHdC?=
 =?utf-8?B?MzlZZEJPeE1memwyZzRjRXhPdmdKU3Z4Unl6K2dkelRSUTdaZ1h0Vk9SeWtw?=
 =?utf-8?B?T0dKbTl3WTNHclZoT0gvSXpOY1ZOQUd0djBzMVpMSmZGVlVQa0doSWkxRzRS?=
 =?utf-8?B?RlZKZ0tyeHBBTnJNbkx4blRhcUY1WXhmemtwNmwzZldreEt5MXdQdWVjL1dM?=
 =?utf-8?B?OEFBU2dNaXl4SWRZeFVWS1UzcktxdFRvNDdiNlN1dWhyMFBPc1p0cHAyQ1Fl?=
 =?utf-8?B?MHpUMjN4NHlHcHJRWTlBNTZ4cVpta1VsL2tkZFBxcGJNQ3U1NnFaWVczOVRC?=
 =?utf-8?B?cnlzN3lxdmVjeTJHaVVDZWdiM24rMWdwakpuRkFMQ3JyWWNWUVBmV1JDbFA0?=
 =?utf-8?B?Q1BUQVFWZ0dVQ05EQjdHYUhFRW50UEFSM3JCZGZoVktrbnFlZTVnbjg4azJW?=
 =?utf-8?B?TWRxTUl0dkcyTGFhMm9pYmt3Z2ErZjJ3d0VVMFdZeEdkWVJJQlZDc1BjTVZ4?=
 =?utf-8?B?Q21wVFpsejRzRzRRcTVidy8yN3h0US9ENlBoVWNmQUdFS0hwS05RSUJvajlC?=
 =?utf-8?B?TkcyMG1uSDAra2dMUWJvUnZiK25UNno5aENLMC9OTGk0VnV2ZnE5V1l1QTFr?=
 =?utf-8?B?SUZ5VS9XZzE5cW5VZGZiTkRvYWJDZnBCUTZpS0s2ajM4VjZVdS8xNnpjOXpU?=
 =?utf-8?B?b3VJREhMM0YxbUZUbURSNCtqRFVGN01tUFRxVUZhT3l6djRqZjNSeVBOOHdo?=
 =?utf-8?B?cDVnamRTWHhua1Vtd1JSN2RUR3lYV3BpVkZ2TmZnNkFWZnZDRGo3cjRDdnky?=
 =?utf-8?B?aTRMNXduRGVackJZckpTbkIweEVNeWxNRVQxaitHbWtRZmVVQXlsSnZqZFp3?=
 =?utf-8?B?R2VrRm9VdWozZEtlSFU4YlZXMDNEQ2RaM1JUM0ZzNFgyb0F6MkllZEplM0ZD?=
 =?utf-8?B?RFJBcnJHeFl2WDV3R2xvRnFIYTdPd21lcHZqQ0wxNVNlRUQ4eGF6YVdxOW5J?=
 =?utf-8?B?eFFiVkRCdXRPaGMvRkdOK1VoTUZUQTFYK0VBZFFFbll5ZmxoS1o4SzZZcEQ0?=
 =?utf-8?B?UWY4R2JPdEZPc2JDblNNQXhzNUdKcmk1QjVhVVA1dEdoS1VtaDFJUjh3aWJZ?=
 =?utf-8?B?KzlCUStvWDBWOXhzdEo0Z3NkbG56LzF4OERIVGE3SXc3U3NHK01ENGxBb0ty?=
 =?utf-8?B?a3Z4eUFuZHNhMWd4azFqZ2h5Mnh6bkI1a3UvUjBSTEhPcXZKMjk1aC9zc0F2?=
 =?utf-8?B?Q082cWxQMWNONlFYVFU1aFJ2YVNyMVRZbHkrOWJQVjhUdTBBbU9UVllpMEFx?=
 =?utf-8?B?enJXV2JFdzlmQ2dBelhjUjNWZ3pWMWZkbTc2Y29rdWtSZWs5UkRPY2JoNlkx?=
 =?utf-8?B?UFpPSThJdXQ2QVFMQUQyOGZVRy9kVGFWcHZOekV0djV3YjlCYTdIVnR1bHRa?=
 =?utf-8?B?bWp4ZEJkN0tYYm4wenoyN3pzS3hFWlJXVG5SZis1WW5HNmRBY1dEai9WVjc2?=
 =?utf-8?B?b3c4WUJDZlFuQ1dGcWt5YXh2OThUek5xd0ZxcURRYjNRaEU5d1NvQ3V1U0sx?=
 =?utf-8?Q?jHGSFIfn1nWAZb1iO6ENf/uHE?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 14cd1ff2-c811-4acd-dea3-08dbf5aa28c3
X-MS-Exchange-CrossTenant-AuthSource: PH7PR11MB5984.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2023 15:52:18.2201
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: i7F3Pz6KSW5rdz3Ehb05cMW/x5oKiSjQhmEQK5/B9LddGqZ0M1Wfr1qeIzK21M8tBsRG7J5QKYjR00NqB9c8TA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB8161
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 05 Dec 2023 07:52:36 -0800 (PST)


On 12/5/23 02:21, Rex Zhang wrote:
> Hi Tom,
> 
> On 2023-12-01 at 14:10:35 -0600, Tom Zanussi wrote:
> 
> [snip]
> 
>> +int idxd_load_iaa_device_defaults(struct idxd_device *idxd)
>> +{
>> +	struct idxd_engine *engine;
>> +	struct idxd_group *group;
>> +	struct idxd_wq *wq;
>> +
>> +	if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags))
>> +		return 0;
> In the virtualization case, it is not configurable in guest OS,
> then the default work queue will not be enabled. Is it expected?

I think what you can do is just set the attributes that are settable and the wq should auto enable. That may be the right approach to make it consistent with host.

>> +
>> +	wq = idxd->wqs[0];
>> +
>> +	if (wq->state != IDXD_WQ_DISABLED)
>> +		return -EPERM;
>> +
>> +	/* set mode to "dedicated" */
>> +	set_bit(WQ_FLAG_DEDICATED, &wq->flags);
>> +	wq->threshold = 0;
>> +
>> +	/* only setting up 1 wq, so give it all the wq space */
>> +	wq->size = idxd->max_wq_size;
>> +
>> +	/* set priority to 10 */
>> +	wq->priority = 10;
>> +
>> +	/* set type to "kernel" */
>> +	wq->type = IDXD_WQT_KERNEL;
>> +
>> +	/* set wq group to 0 */
>> +	group = idxd->groups[0];
>> +	wq->group = group;
>> +	group->num_wqs++;
>> +
>> +	/* set name to "iaa_crypto" */
>> +	memset(wq->name, 0, WQ_NAME_SIZE + 1);
>> +	strscpy(wq->name, "iaa_crypto", WQ_NAME_SIZE + 1);
>> +
>> +	/* set driver_name to "crypto" */
>> +	memset(wq->driver_name, 0, DRIVER_NAME_SIZE + 1);
>> +	strscpy(wq->driver_name, "crypto", DRIVER_NAME_SIZE + 1);
>> +
>> +	engine = idxd->engines[0];
>> +
>> +	/* set engine group to 0 */
>> +	engine->group = idxd->groups[0];
>> +	engine->group->num_engines++;
>> +
>> +	return 0;
>> +}
>> diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h
>> index 62ea21b25906..47de3f93ff1e 100644
>> --- a/drivers/dma/idxd/idxd.h
>> +++ b/drivers/dma/idxd/idxd.h
>> @@ -277,6 +277,8 @@ struct idxd_dma_dev {
>>  	struct dma_device dma;
>>  };
>>  
>> +typedef int (*load_device_defaults_fn_t) (struct idxd_device *idxd);
>> +
>>  struct idxd_driver_data {
>>  	const char *name_prefix;
>>  	enum idxd_type type;
>> @@ -286,6 +288,7 @@ struct idxd_driver_data {
>>  	int evl_cr_off;
>>  	int cr_status_off;
>>  	int cr_result_off;
>> +	load_device_defaults_fn_t load_device_defaults;
>>  };
>>  
>>  struct idxd_evl {
>> @@ -730,6 +733,7 @@ void idxd_unregister_devices(struct idxd_device *idxd);
>>  void idxd_wqs_quiesce(struct idxd_device *idxd);
>>  bool idxd_queue_int_handle_resubmit(struct idxd_desc *desc);
>>  void multi_u64_to_bmap(unsigned long *bmap, u64 *val, int count);
>> +int idxd_load_iaa_device_defaults(struct idxd_device *idxd);
>>  
>>  /* device interrupt control */
>>  irqreturn_t idxd_misc_thread(int vec, void *data);
>> diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c
>> index 0eb1c827a215..14df1f1347a8 100644
>> --- a/drivers/dma/idxd/init.c
>> +++ b/drivers/dma/idxd/init.c
>> @@ -59,6 +59,7 @@ static struct idxd_driver_data idxd_driver_data[] = {
>>  		.evl_cr_off = offsetof(struct iax_evl_entry, cr),
>>  		.cr_status_off = offsetof(struct iax_completion_record, status),
>>  		.cr_result_off = offsetof(struct iax_completion_record, error_code),
>> +		.load_device_defaults = idxd_load_iaa_device_defaults,
>>  	},
>>  };
>>  
>> @@ -745,6 +746,12 @@ static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>>  		goto err;
>>  	}
>>  
>> +	if (data->load_device_defaults) {
>> +		rc = data->load_device_defaults(idxd);
>> +		if (rc)
>> +			dev_warn(dev, "IDXD loading device defaults failed\n");
>> +	}
>> +
>>  	rc = idxd_register_devices(idxd);
>>  	if (rc) {
>>  		dev_err(dev, "IDXD sysfs setup failed\n");
> 
> Thanks,
> Rex Zhang
>> -- 
>> 2.34.1
>>