Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp6089749rwd; Mon, 5 Jun 2023 12:54:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4p14GRXM8MmUxNuTfrE1UV4Pc8b1IZorPO+gV7zMycIE66n1jyQGNzz/XvASZ1aVWkHksW X-Received: by 2002:a05:620a:3d94:b0:75b:23a1:831b with SMTP id ts20-20020a05620a3d9400b0075b23a1831bmr798815qkn.22.1685994843392; Mon, 05 Jun 2023 12:54:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685994843; cv=none; d=google.com; s=arc-20160816; b=epGEgeW2wDNgdD0C2p+nbsMXm5S5DTB+Jr04VYjTp0gDPohPblsggS/K/myYKFgtlH lSaUNYMkrUEQ6bZY4k0Dyv8/ZFhLeUYvPvCZy/l98l8xGL5ge9OaeQ0Zuk7sHdt4YHYD Ga87IjrSnmR8ElxTY6gQK8cz4qoLfRseZWpNUe/CaNaFzG9yBk8sQXlGzEtCg24lnlSp h8cULJBoNpymk0+Uhg1F8eQ2LrlZVjeWGylX3jVPz/a6cHdHEeAUbFuREDIxgnsd+Pkm FfKfsSLlSeHr1xf7E+jJBmx7ckFM3PBvFarsWY8pUKqJgjGnHV3KH0v/q8JSi7wnLXD+ JZGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to :content-language:references:cc:to:subject:from:user-agent :mime-version:date:message-id:dkim-signature; bh=cPlOjdlJ+dRPlBsswQpKMNmBPkhakhRaFVQ24sk5p48=; b=IHGd+508vDf0rSuSD+1Io5cyjT02KsU3DCzJNNxBKP30DCoKZjNTGuEWPu8dkO+/XU 5hBmzfig0h3unufhXzNh0azp6uO4x8FkRu5LJwDYL03NkVpKkWZHhX7blRUlk1Vk2AUS hj9dLJKwGKyAXHCekiMIoZIvazuuSM/+ZRMw10z+HwLxd8sr2gcTWOTGgAMrvcTrdJ6P CeArDyNX63wpxZ0+Ffgu4VyDRmRiNYU7u32JPqpHTstbCjQxXipP+rEg6TY32mDfNaai SeMyh5bp2C9ZCZGSZ/jGfLMEEe21j91qBJVKyWiskptgulQPphi1StOByZIFUeKPAzpO 9sHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JfzQ4nFn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f1-20020a05620a20c100b00753e1f61a0bsi4833587qka.777.2023.06.05.12.53.48; Mon, 05 Jun 2023 12:54:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JfzQ4nFn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236090AbjFETuc (ORCPT + 99 others); Mon, 5 Jun 2023 15:50:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236117AbjFETuM (ORCPT ); Mon, 5 Jun 2023 15:50:12 -0400 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D769F11C for ; Mon, 5 Jun 2023 12:49:51 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-777b2af1c53so78494239f.2 for ; Mon, 05 Jun 2023 12:49:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1685994590; x=1688586590; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:subject:from:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=cPlOjdlJ+dRPlBsswQpKMNmBPkhakhRaFVQ24sk5p48=; b=JfzQ4nFnxsDVVweSWRoSCcbfNulBTZT6AhFmi+GrblhAGn6SN9IyYc1Q9647sBsTVt 5A3awQZoHTqt1RX1Wg/y+Yu5XoboQeFoAv4LZioXXGarpJ3XdGzj7V6x39toEaKe8xkx 6T3rYZJWRHRKK/zrkJiuek556swCuPgBjcUlI0Md80mOQH8krj0JjbqziCLt8/YlhiDA td0e8yBRJMDomz+I7kuMIPsDBI+l+TP50udtaxGjpIxFrQNKRoYfB5uJRSACckH6tOe0 gxcqJIritft3hxsvAafmv0YyHrlZ0xGc0lB1nGxGXYRgxFj3NFUKTSqr+2YeLNQnzR9f SdVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685994590; x=1688586590; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:subject:from:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cPlOjdlJ+dRPlBsswQpKMNmBPkhakhRaFVQ24sk5p48=; b=iKQbDUZu1UM7XauSjDVduTDegVTif5rDEBKymaJAOK3KthDvtvIpgjVMfZ7zzvOZ2R zJJvU5Dhah4z1jqiLq3vWEdOqVspvMBhB4PR2l1roX+dlppY8eR5Y9LbKkyq9ul9LmNf ZoEIFebVfr4JB4QiuEehiy4ASwVqt1ZwUtHbdWAJXIQGW9lTK8+TnJZB7qPxcHoIewv1 T6MYFdF4FJERIQ5r7hklLdM7fN2cEyWajRxvNpg8FOrZSQxCMSvtqEvHxr8Xwk/JQ+ii 2pBbPTpb1UgYAjK3p6Ibo383JRrsaGM5zTgzoh5HrrhAJcuUK0k3N5ZRYTXaGDb625FB xKWQ== X-Gm-Message-State: AC+VfDwgfmtCOJJmto3F3KUppqNtLIv42TA2dWHmfLX9q14kAjMy14XV d3gJFmE3btdqakkosPY96yIOlg== X-Received: by 2002:a5e:d710:0:b0:774:9c64:e0a4 with SMTP id v16-20020a5ed710000000b007749c64e0a4mr287414iom.4.1685994590054; Mon, 05 Jun 2023 12:49:50 -0700 (PDT) Received: from [172.22.22.28] ([98.61.227.136]) by smtp.gmail.com with ESMTPSA id l22-20020a6bd116000000b0076c81bf2731sm2735824iob.20.2023.06.05.12.49.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 05 Jun 2023 12:49:49 -0700 (PDT) Message-ID: <3dd82ec0-2a9a-3401-5385-965c624f9f32@linaro.org> Date: Mon, 5 Jun 2023 14:49:48 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 From: Alex Elder Subject: Re: [PATCH v13 17/24] gunyah: vm_mgr: Add framework for VM Functions To: Elliot Berman , Srinivas Kandagatla , Prakruthi Deepak Heragu , Jonathan Corbet Cc: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , Konrad Dybcio , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org References: <20230509204801.2824351-1-quic_eberman@quicinc.com> <20230509204801.2824351-18-quic_eberman@quicinc.com> Content-Language: en-US In-Reply-To: <20230509204801.2824351-18-quic_eberman@quicinc.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 5/9/23 3:47 PM, Elliot Berman wrote: > Introduce a framework for Gunyah userspace to install VM functions. VM > functions are optional interfaces to the virtual machine. vCPUs, > ioeventfs, and irqfds are examples of such VM functions and are s/ioventfs/ioventfds/ Also, these aren't just examples of VM functions, they *are* the VM functions implemented. > implemented in subsequent patches. > > A generic framework is implemented instead of individual ioctls to > create vCPUs, irqfds, etc., in order to simplify the VM manager core > implementation and allow dynamic loading of VM function modules. This also allows the set of VM functions to be extended without updating the API (like it or not). > > Signed-off-by: Elliot Berman I have a few more comments, but this looks pretty good. Reviewed-by: Alex Elder > --- > Documentation/virt/gunyah/vm-manager.rst | 18 ++ > drivers/virt/gunyah/vm_mgr.c | 216 ++++++++++++++++++++++- > drivers/virt/gunyah/vm_mgr.h | 4 + > include/linux/gunyah_vm_mgr.h | 87 +++++++++ > include/uapi/linux/gunyah.h | 18 ++ > 5 files changed, 340 insertions(+), 3 deletions(-) > create mode 100644 include/linux/gunyah_vm_mgr.h > > diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst > index 50d8ae7fabcd..3b51bab9d793 100644 > --- a/Documentation/virt/gunyah/vm-manager.rst > +++ b/Documentation/virt/gunyah/vm-manager.rst > @@ -17,6 +17,24 @@ sharing userspace memory with a VM is done via the `GH_VM_SET_USER_MEM_REGION`_ > ioctl. The VM itself is configured to use the memory region via the > devicetree. > > +Gunyah Functions > +================ > + > +Components of a Gunyah VM's configuration that need kernel configuration are > +called "functions" and are built on top of a framework. Functions are identified > +by a string and have some argument(s) to configure them. They are typically > +created by the `GH_VM_ADD_FUNCTION`_ ioctl. Is a function *type* (e.g., VCPU or ioeventfd) identified by a string? Or a function *instance* (e.g. four VCPUs)? Or both? > + > +Functions typically will always do at least one of these operations: Typically, or always? > + > +1. Create resource ticket(s). Resource tickets allow a function to register > + itself as the client for a Gunyah resource (e.g. doorbell or vCPU) and > + the function is given the pointer to the &struct gh_resource when the > + VM is starting. > + What I think this means is that tickets are used to allow functions to be defined *before* the VM is actually started. So once it starts, the functions get added. (I might have this slightly wrong, but in any case I'm not sure the above sentence is very clear.) > +2. Register IO handler(s). IO handlers allow a function to handle stage-2 faults > + from the virtual machine. > + > Sample Userspace VMM > ==================== > > diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c > index a800061f56bf..56464451b262 100644 > --- a/drivers/virt/gunyah/vm_mgr.c > +++ b/drivers/virt/gunyah/vm_mgr.c > @@ -6,10 +6,13 @@ > #define pr_fmt(fmt) "gh_vm_mgr: " fmt > > #include > +#include > #include > #include > +#include > #include > #include > +#include > > #include > > @@ -17,6 +20,172 @@ > > static void gh_vm_free(struct work_struct *work); > > +static DEFINE_XARRAY(gh_vm_functions); > + > +static void gh_vm_put_function(struct gh_vm_function *fn) > +{ > + module_put(fn->mod); > +} > + > +static struct gh_vm_function *gh_vm_get_function(u32 type) > +{ > + struct gh_vm_function *fn; > + int r; > + > + fn = xa_load(&gh_vm_functions, type); > + if (!fn) { > + r = request_module("ghfunc:%d", type); > + if (r) > + return ERR_PTR(r > 0 ? -r : r); Almost all callers of request_module() simply ignore the return value. What positive values are you expecting to see here (and are you sure they're positive errno values)? > + > + fn = xa_load(&gh_vm_functions, type); > + } > + > + if (!fn || !try_module_get(fn->mod)) > + fn = ERR_PTR(-ENOENT); > + > + return fn; > +} . . .