Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752206AbbFDHGO (ORCPT ); Thu, 4 Jun 2015 03:06:14 -0400 Received: from mail.linux-iscsi.org ([67.23.28.174]:51493 "EHLO linux-iscsi.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751846AbbFDHGL (ORCPT ); Thu, 4 Jun 2015 03:06:11 -0400 Message-ID: <1433401569.18125.112.camel@haakon3.risingtidesystems.com> Subject: Re: [RFC 0/2] target: Add TFO->complete_irq queue_work bypass From: "Nicholas A. Bellinger" To: Christoph Hellwig Cc: "Nicholas A. Bellinger" , target-devel , linux-scsi , linux-kernel , Hannes Reinecke , Sagi Grimberg Date: Thu, 04 Jun 2015 00:06:09 -0700 In-Reply-To: <20150603125756.GA19696@lst.de> References: <1432281446-31080-1-git-send-email-nab@daterainc.com> <20150603125756.GA19696@lst.de> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.4.4-1 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1440 Lines: 30 On Wed, 2015-06-03 at 14:57 +0200, Christoph Hellwig wrote: > This makes lockdep very unhappy, rightly so. If you execute > one end_io function inside another you basŃ–cally nest every possible > lock taken in the I/O completion path. Also adding more work > to the hardirq path generally isn't a smart idea. Can you explain > what issues you were seeing and how much this helps? Note that > the workqueue usage in the target core so far is fairly basic, so > there should some low hanging fruit. So I've been using tcm_loop + RAMDISK backends for prototyping, but this patch is intended for vhost-scsi so it can avoid the unnecessary queue_work() context switch within target_complete_cmd() for all backend driver types. This is because vhost_work_queue() is just updating vhost_dev->work_list and immediately wake_up_process() into a different vhost_worker() process context. For heavy small block workloads into fast IBLOCK backends, avoiding this extra context switch should be a nice efficiency win. Also, AFAIK RDMA fabrics are allowed to do ib_post_send() response callbacks directly from IRQ context as well. Perhaps tcm_loop LLD code should just be limited to RAMDISK here..? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/