Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1752261AbbKVQ40 (ORCPT <rfc822;w@1wt.eu>);
	Sun, 22 Nov 2015 11:56:26 -0500
Received: from sf1.bxl.stone.is ([87.238.167.214]:53918 "EHLO sf1.bxl.stone.is"
	rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP
	id S1751930AbbKVQ4Y (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Sun, 22 Nov 2015 11:56:24 -0500
Subject: Re: [PATCH 2/9] IB: add a proper completion queue abstraction
To: Sagi Grimberg <sagig@dev.mellanox.co.il>, Christoph Hellwig <hch@lst.de>
References: <1447422410-20891-1-git-send-email-hch@lst.de>
	<1447422410-20891-3-git-send-email-hch@lst.de>
	<564B697A.2020601@sandisk.com>
	<564C2F01.6020407@dev.mellanox.co.il> <564CC15E.7030602@sandisk.com>
	<20151120101644.GC24298@lst.de> <564F4F38.9040505@sandisk.com>
	<56519011.6060204@dev.mellanox.co.il> <20151122101308.GA12189@lst.de>
	<56519A90.5010502@dev.mellanox.co.il> <20151122132352.GA14154@lst.de>
	<5651D7D3.7030509@dev.mellanox.co.il>
Cc: "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
        "axboe@fb.com" <axboe@fb.com>,
        "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
From: Bart Van Assche <bvanassche@acm.org>
Message-ID: <5651F39A.2020003@acm.org>
Date: Sun, 22 Nov 2015 08:55:54 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
	Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <5651D7D3.7030509@dev.mellanox.co.il>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-PPP-Message-ID: <20151122165558.118460.94467@vz23.stone-is.net>
X-PPP-Vhost: bartvanassche.be
X-Filter-ID: s0sct1PQhAABKnZB5plbIYrcAt68paBzxGNV4bZScc8DDDH9IcRzs4ldqK5vEMwbuZt+PpRPW1h0
 yigPev+a+17xi0cDYTysi9kyOevj7UMj43wFvVCcyWnCyu/WFSjo/SHX8mSdnSh904WuveTVGG7k
 uIdZGU7D9TAof/YpxClLn/njN5eSj526f3kLYY1TX9JTgn9yuZxJxuyH/PTsroI0xhElNaWmLNdB
 NKvUhEJ8KBlLNS+psTf+/TH9wb1of7j0CKTjist7O+WLg0AfiBn026kpX9yrLzFr43tAmzdx+CtF
 k/O/Di/taNwBeo408fxVSyfwNfFaiGDK5a6y6oUFsovH1hCJsqUgbQtiPzxy2DNwOgV373pfDhBQ
 21OdQUgB1M0N8p+5Np4NNGY9syX35aq3l2ngEUM3CvuVKo91auFTgcejlLvBph/oh6t+bj7a0YsF
 61gf0GQMgCBgC+uO5TcDeKjrEmYPn2IVWRtF22Ka3oUpyFMbgzyXUMAa32GyDn1fGu2b7rMtlFZI
 nUiLVU43pscs8Nh1jNY8xI/BFyDxlm1MOvK+7A01V57MxDXPI1dFULcYpkmienh2U+Q43YfdJZIm
 NdoCmoDH0TvrpsYp7xtetKMAIeCYdxn19ZxyzEs+3kE8U85Gfg82LfEaZL4xqGACRNSyLSX7pj40
 sWoYYUo1zbnWQqM6c3O30eSUIzgIxIs/sYo3QWlVHIt1fVroD3lPIneDBs2K28plnl//53ul+axR
 Hmc2L6zTYu8C5mJ57ZG9g9QfbBpIF5KSsyc+xLmVySaNDhhtOSCO8QZrn7SbQ2ZTGD1j4bz/
X-Report-Abuse-To: spam@sf1.bxl.stone.is
X-Filter-Fingerprint: IFrWXGses7OKB5S5G8/dJRn/7L7+nnclbWGk7OJ1jHJA3cTUQ1R++keuE7RDJ8Kg3RbMLUalw1oC
 mj99/u+PoqoVy8a3lsStJtAvpObFX0Wok1JBYnOLzfRIhlEHQynLRwQd5Deh21btrUTI2ioQbVt0
 ARHRi6XGuAluI1udprFy32DUYpEhA3j9NJFmItfypuoazoDH3m92PL21GfhFYWcYmGLKZUTyGy/B
 A6iJtsD8WFC+rpTT4JYvoDjVeZUw3fI9smEy0EupqfCN6sn6Zg==
X-Originating-IP: 87.238.162.216
X-SpamExperts-Domain: stone.is
X-SpamExperts-Username: 87.238.162.216
Authentication-Results: bxl.stone.is; auth=pass smtp.auth=87.238.162.216@stone.is
X-SpamExperts-Outgoing-Class: ham
X-SpamExperts-Outgoing-Evidence: Combined (0.08)
X-Recommended-Action: accept
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 1319
Lines: 29

On 11/22/15 06:57, Sagi Grimberg wrote:
>>> I think that bart wants to allow the caller to select cpu affinity
>>> per CQ. In this case ib_alloc_cq in workqueue mode would need to
>>> accept a affinity_hint from the caller (default to wild-card
>>> WORK_CPU_UNBOUND).
>>
>> Hmm, true.  How would be set that hint from userspace?  I'd really prefer
>> to see a practical justification for it first.
>
> In order to assign CPUs from user-space we'd need an ethtool like
> interface for isert/srpt/<xxxt>. Given that this is something we don't
> want to get into right now, I assumed that Bart meant that srpt
> would take a "least used" approach from srpt driver (which isn't better
> taking the wild-card option I'd say), So I'll let Bart answer...

Hello Christoph and Sagi,

My intention is indeed to allow to control CPU affinity per CQ. One use 
case is to implement a least-used policy in RDMA drivers that use 
multiple completion queues. Another use case is to make CPU affinity 
configurable from user space through something similar to ethtool or via 
sysfs.

Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/