Subject: Re: [PATCH 2/9] IB: add a proper completion queue abstraction
To: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
References: <1447422410-20891-1-git-send-email-hch@lst.de>
 <1447422410-20891-3-git-send-email-hch@lst.de>
 <20151113182513.GB21808@obsidianresearch.com> <564640C4.3000603@sandisk.com>
 <20151113220636.GA32133@obsidianresearch.com> <20151114071344.GE27738@lst.de>
 <20151123203712.GB5640@obsidianresearch.com> <56537F59.4080708@sandisk.com>
 <20151123212822.GE6062@obsidianresearch.com>
CC: Christoph Hellwig <hch@lst.de>,
        "linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
        "sagig@dev.mellanox.co.il" <sagig@dev.mellanox.co.il>,
        "axboe@fb.com" <axboe@fb.com>,
        "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
From: Bart Van Assche <bart.vanassche@sandisk.com>
Message-ID: <56538AFD.9080103@sandisk.com>
Date: Mon, 23 Nov 2015 13:54:05 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <20151123212822.GE6062@obsidianresearch.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
SpamDiagnosticOutput: 1:23
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Nov 2015 21:54:05.7494
 (UTC)
X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d;Ip=[63.163.107.173];Helo=[milsmgep12.sandisk.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR0201MB0821
X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1894;2:U2zwligDyBIdpvbzFn9F8a7w7PYhDkmjZCouFoFi9qv7ckps4Po3jAHv4Q7vyhCSSVaRb6ccfRvVqkt8JHFzgdEVCzjOOMTMK32G5m9k8pvq2MFSA1kTYQjDPomVxG47h7udTiZiFg8WOyZulOod/w==;23:hLLVfcOV/fNaUPbwQZdb+R+eDfKuSFFdgxG3DHnXuh66M2KaBpgkb1hziIjFnKZNz9Eo+4pNcStfmAhYf+gIEQorttcbS07lHcWAOFMZH39BuuC6iPp9p1FxCG/8ls0yZi9vKIYv38tVT+JKl+xXs2ldX0uqKqmiNo9glhQbW689i54lDTUzvsgG/7+VKiS6
X-OriginatorOrg: sandisk.com
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 1479
Lines: 33

On 11/23/2015 01:28 PM, Jason Gunthorpe wrote:
> On Mon, Nov 23, 2015 at 01:04:25PM -0800, Bart Van Assche wrote:
>
>> Considerable time ago the send queue in the SRP initiator driver was
>> modified from signaled to non-signaled to reduce the number of interrupts
>> triggered by the SRP initiator driver. The SRP initiator driver polls the
>> send queue every time before a SCSI command is sent to the target. I think
>> this is a pattern that is also useful for other ULP's so I'm not convinced
>> that ib_process_cq_direct() should be deprecated :-)
>
> As I explained, that is a fine idea, but I can't see how SRP is able
> to correctly do sendq flow control without spinning on the poll, which
> it does not do.
>
> I'm guessing SRP is trying to drive sendq flow control from the recv
> side, like NFS was. This is wrong and should not be part of the common
> API.
>
> Does that make sense?

Not really ... Please have a look at the SRP initiator source code. What 
the SRP initiator does is to poll the send queue before sending a new 
SCSI command to the target system starts. I think this approach could 
also be used in other ULP drivers if the send queue poll frequency is 
such that no send queue overflow occurs.

Bart.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/