From: Douglas Miller Subject: Re: Fostering linux community collaboration on hardware accelerators Date: Thu, 12 Oct 2017 08:31:36 -0500 Message-ID: References: <201710101132.v9ABUs28138304@mx0a-001b2d01.pphosted.com> <07d49485-e583-8434-5681-92a0b54005ca@au1.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit To: Andrew Donnellan , Jonathan Cameron , jic23@kernel.org, "Liguozhu (Kenneth)" , Ilias Apalodimas , francois.ozog@linaro.org, Prasad.Athreya@cavium.com, arndbergmann@gmail.com, Alex Williamson , Frederic Barrat , Mark Brown , Tirumalesh.Chalamarla@cavium.com, jcm@redhat.com, Ard Biesheuvel , Jean-Philippe Brucker , Kirti Wankhede , Eric Auger , kvm@vger.kernel.org, linux-crypto@vger.kernel.org, linuxarm@huawei.com Return-path: Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:50856 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1755647AbdJLNbs (ORCPT ); Thu, 12 Oct 2017 09:31:48 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v9CDTpM9084174 for ; Thu, 12 Oct 2017 09:31:47 -0400 Received: from e34.co.us.ibm.com (e34.co.us.ibm.com [32.97.110.152]) by mx0a-001b2d01.pphosted.com with ESMTP id 2dj5xej0bg-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 12 Oct 2017 09:31:47 -0400 Received: from localhost by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 12 Oct 2017 07:31:46 -0600 In-Reply-To: <07d49485-e583-8434-5681-92a0b54005ca@au1.ibm.com> Content-Language: en-US Sender: linux-crypto-owner@vger.kernel.org List-ID: Not sure if you're already plugged-in to this, but the OpenMP group is (has been) working on Accelerator support. http://www.openmp.org/updates/openmp-accelerator-support-gpus/ Maybe you are talking about a different aspect of accelerator support, but it seems prudent to involve OpenMP as much as makes sense. On 10/12/2017 12:22 AM, Andrew Donnellan wrote: > On 10/10/17 22:28, Jonathan Cameron wrote: >> Hi All, >> >> Please forward this email to anyone you think may be interested. > > Have forwarded this to a number of relevant IBMers. > >> On behalf of Huawei, I am looking into options to foster a wider >> community >> around the various ongoing projects related to Accelerator support >> within >> Linux.  The particular area of interest to Huawei is that of harnessing >> accelerators from userspace, but in a collaborative way with the kernel >> still able to make efficient use of them, where appropriate. >> >> We are keen to foster a wider community than one just focused on >> our own current technology.  This is a field with no clear answers, >> so the >> widest possible range of input is needed! >> >> The address list of this email is drawn from people we have had >> discussions >> with or who have been suggested in response to Kenneth Lee's wrapdrive >> presentation at Linaro Connect and earlier presentations on the more >> general >> issue. A few relevant lists added to hopefully catch anyone we missed. >> My apologies to anyone who got swept up in this and isn't interested! >> >> Here we are defining accelerators fairly broadly - suggestions for a >> better >> term are also welcome. >> >> The infrastructure may be appropriate for: >> * Traditional offload engines - cryptography, compression and similar >> * Upcoming AI accelerators >> * ODP type requirements for access to elements of networking >> * Systems utilizing SVM including CCIX and other cache coherent buses >> * Many things we haven't thought of yet... >> >> As I see it, there are several aspects to this: >> >> 1) Kernel drivers for accelerators themselves. >>     * Traditional drivers such as crypto etc >>     - These already have their own communities. The main >>            focus of such work will always be through them. >>          - What a more general community could add here would be an >>            overview of the shared infrastructure of such devices. >>       This is particularly true around VFIO based (or similar) >>       userspace interfaces with a non trivial userspace component. >>     * How to support new types of accelerator? >> >> 2) The need for lightweight access paths from userspace that 'play >> well' and >>     share resources etc with standard in-kernel drivers.  This is the >> area >>     that Kenneth Lee and Huawei have been focusing on with their >> wrapdrive >>     effort. We know there are other similar efforts going on in other >> companies. >>     * This may involve interacting with existing kernel communities >> such as >>       those around VFIO and mdev. >>     * Resource management when we may have many consumers - not all >> hardware >>       has appropriate features to deal with this. >> >> 3) Usecases for accelerators. e.g. >>     * kTLS >>     * Storage encryption >>     * ODP - networking dataplane >>     * AI toolkits >> >> Discussions we want to get started include: >> * A wider range of hardware than we are currently considering. What >> makes >>    sense to target / what hardware do people have they would like to >> support? >> * Upstream paths - potential blockers and how to overcome them. The >> standard >>    kernel drivers should be fairly straightforward, but once we start >> looking at >>    systems with a heavier userspace component, things will get more >>    controversial! >> * Fostering stronger userspace communities to allow these these >> accelerators >>    to be easily harnessed. >> >> So as ever with a linux community focusing on a particular topic, the >> obvious solution is a mailing list. There are a number of options on how >> do this. >> >> 1) Ask one of the industry bodies to host? Who? >> >> 2) Put together a compelling argument for >> linux-accelerators@vger.kernel.org >> as probably the most generic location for such a list. > > Happy to offer linux-accelerators@lists.ozlabs.org, which I can get > set up immediately (and if we want patchwork, patchwork.ozlabs.org is > available as always, no matter where the list is hosted). > >> More open questions are >> 1) Scope? >>   * Would anyone ever use such an overarching list? >>   * Are we better off with the usual adhoc list of 'interested >> parties' + lkml? >>   * Do we actually need to define the full scope - are we better with >> a vague >>     definition? > > I think a list with a broad and vaguely defined scope is a good idea - > it would certainly be helpful to us to be able to follow what other > contributors are doing that could be relevant to our CAPI and OpenCAPI > work. > >> >> 2) Is there an existing community we can use to discuss these issues? >>     (beyond the obvious firehose of LKML). >> >> 3) Who else to approach for input on these general questions? >> >> In parallel to this there are elements such as git / patchwork etc but >> they can all be done as they are needed. >> >> Thanks >> >> -- >> Jonathan Cameron >> Huawei >> >