Received: by 2002:a05:6a10:8c0a:0:0:0:0 with SMTP id go10csp1843711pxb; Fri, 5 Mar 2021 00:35:36 -0800 (PST) X-Google-Smtp-Source: ABdhPJxgTX/79cfXuBhjS+K4y3UsIzhPDB7zi1PGYlhITkyGxIRlMXsc55KXc3bhD1/VS4UksYqj X-Received: by 2002:a05:6402:1152:: with SMTP id g18mr8378275edw.18.1614933335937; Fri, 05 Mar 2021 00:35:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1614933335; cv=none; d=google.com; s=arc-20160816; b=cgfYWAXehaEQFARU/KukphQeCZfYjMYNeZ+7k3Lo7/rmyOSsz6EfwXR4OLtjy1iAGw +gUVd1bF1irA927YPA5g7f5EVOtu5069blh6K/fXTgC2itqyaKWsQgcUJ7dLICozy6Xz QxjVAjhY5UMELPfNFGaLmJC3tU7EYMIDgfcUkztRR9/ObtEbq8OyOmKf/JJTxySmWL3X Hw4ICMLPxG3kQwEWqxuEdITBROJbgrYZtHunOq/r+rSMYxLAVF0KPTTopkg0eEIHCB3u hBrUpHVXW6So7GPzS3AO7g+L8AVxlska6Wx1xAgxanXm5JoSHrWoX72jB/QtOPYA/7bJ qyDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=XWbZV1xuaSmzet0/YcZif2T8HeoxQ88xDpPUR5wIaRM=; b=e+f9YDrz/sxulJmBMLUkToUk2Om6DVWJaJ0Pa1i1v0dtQ4o1s4hqA6HrJ0vOEiCGni SDwuNkdbbftsHUEQStZs2CvnJDMEQ/juPE57qD6OIMwHRd/zsWU/LEq9Zy3qyJwLe4wp 3FIENvT7thEdVup7Ri3E5B4StuFu3dQ0nYvYlZsGKJvV2P/oEAP42f/TMg+nL3blDNvS MmGYsJynRWWNr04bY5N9skcj6K2Zzs1EHCCMyxLY7Wo8rdJIYZEbbGDlJ4Eslt6+1mSm 4hFBa3wy6Jta7juMHb0oEamAUa6rix0n+0CgGNY95voK2i3fKoAGq72Fk8lhJOhDf1x9 NijA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WDBYUW0q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id r21si1009570ejo.123.2021.03.05.00.35.12; Fri, 05 Mar 2021 00:35:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WDBYUW0q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229589AbhCEIbi (ORCPT + 99 others); Fri, 5 Mar 2021 03:31:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229582AbhCEIbN (ORCPT ); Fri, 5 Mar 2021 03:31:13 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11678C061574 for ; Fri, 5 Mar 2021 00:31:12 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id u16so1135686wrt.1 for ; Fri, 05 Mar 2021 00:31:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=XWbZV1xuaSmzet0/YcZif2T8HeoxQ88xDpPUR5wIaRM=; b=WDBYUW0qE+cgv+jlzpk/LkkEv5rrFExd/rNVN6CPNPszS2oiEe6l8WDOoPwMpJbJ9g ifcqD6jPV1lLiR4Yi/mjqfHszxBdJMcLo3KXRTXr9C02Sobabk+4owE0Z41bxe8ob0sf TXbIB7FE2gA0+Sjn4it3R8bHCBwJgU43j8xHmxrRaiGMOzXVU8OS4NipVF5fqrdwQXA2 IIaotAllyZPnmz7mAt2JSTHTtIeJjXlLVUCSIhhKMkEA1E/BRRB28/j/UfjP/wIKBQS/ GOSaXt0wygQ1kOygpUpLV5Q/GAhYxDax8i6p2F354NInZNYmwDxkG0X/3YVqPpdEXq1x imog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=XWbZV1xuaSmzet0/YcZif2T8HeoxQ88xDpPUR5wIaRM=; b=CnqW5XNpg8IUc1iOruNtRhtett7DfV4BhCFlwYmbiWW1cakDtJsEVsDDUG9+k3pwZg q+dNYBzp01qo8w3+ZI1AmPBps4qqlrE+m4dukL3UF9fK/DsPihLnczgjlAIbYWT9dDkR nVaORascxc9w60LujpI8f8JNbWZ2P9WTfkIN/6QMB7DfpbsYqdA4dCSpyioOzGuY0UXb EJWOPxquBLanztMKmcf5003EPKgPA3N65nEZ0rrjisCdilwP0Ekqot2jZvOEl1Oz1v9U SJp4Qmc8lHntE3dyw2E0ppJ7Y8/rJirOGPv/9Cw0fAsZVAtksF+FnNJH0WJLv6bOgpFA YhrQ== X-Gm-Message-State: AOAM533f7Rn57VZRiSaMJEnnlkETpeiBuyR2cbfQVdxvjRwW5th/pKUh LDPwNQAAuy2B8xfDiuLRudFYcg== X-Received: by 2002:adf:ded1:: with SMTP id i17mr8168897wrn.349.1614933070664; Fri, 05 Mar 2021 00:31:10 -0800 (PST) Received: from myrica ([2001:1715:4e26:a7e0:116c:c27a:3e7f:5eaf]) by smtp.gmail.com with ESMTPSA id w25sm3254177wmc.42.2021.03.05.00.31.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Mar 2021 00:31:10 -0800 (PST) Date: Fri, 5 Mar 2021 09:30:49 +0100 From: Jean-Philippe Brucker To: Jacob Pan Cc: Tejun Heo , LKML , Joerg Roedel , Lu Baolu , David Woodhouse , iommu@lists.linux-foundation.org, cgroups@vger.kernel.org, Johannes Weiner , Jean-Philippe Brucker , Alex Williamson , Eric Auger , Jason Gunthorpe , Jonathan Corbet , Raj Ashok , "Tian, Kevin" , Yi Liu , Wu Hao , Dave Jiang Subject: Re: [RFC PATCH 15/18] cgroup: Introduce ioasids controller Message-ID: References: <1614463286-97618-1-git-send-email-jacob.jun.pan@linux.intel.com> <1614463286-97618-16-git-send-email-jacob.jun.pan@linux.intel.com> <20210303131726.7a8cb169@jacob-builder> <20210303160205.151d114e@jacob-builder> <20210304094603.4ab6c1c4@jacob-builder> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210304094603.4ab6c1c4@jacob-builder> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 04, 2021 at 09:46:03AM -0800, Jacob Pan wrote: > Hi Jean-Philippe, > > On Thu, 4 Mar 2021 10:49:37 +0100, Jean-Philippe Brucker > wrote: > > > On Wed, Mar 03, 2021 at 04:02:05PM -0800, Jacob Pan wrote: > > > Hi Jacob, > > > > > > On Wed, 3 Mar 2021 13:17:26 -0800, Jacob Pan > > > wrote: > > > > > > > Hi Tejun, > > > > > > > > On Wed, 3 Mar 2021 10:44:28 -0500, Tejun Heo wrote: > > > > > > > > > On Sat, Feb 27, 2021 at 02:01:23PM -0800, Jacob Pan wrote: > > > > > > IOASIDs are used to associate DMA requests with virtual address > > > > > > spaces. They are a system-wide limited resource made available to > > > > > > the userspace applications. Let it be VMs or user-space device > > > > > > drivers. > > > > > > > > > > > > This RFC patch introduces a cgroup controller to address the > > > > > > following problems: > > > > > > 1. Some user applications exhaust all the available IOASIDs thus > > > > > > depriving others of the same host. > > > > > > 2. System admins need to provision VMs based on their needs for > > > > > > IOASIDs, e.g. the number of VMs with assigned devices that perform > > > > > > DMA requests with PASID. > > > > > > > > > > Please take a look at the proposed misc controller: > > > > > > > > > > http://lkml.kernel.org/r/20210302081705.1990283-2-vipinsh@google.com > > > > > > > > > > Would that fit your bill? > > > > The interface definitely can be reused. But IOASID has a different > > > > behavior in terms of migration and ownership checking. I guess SEV key > > > > IDs are not tied to a process whereas IOASIDs are. Perhaps this can be > > > > solved by adding > > > > + .can_attach = ioasids_can_attach, > > > > + .cancel_attach = ioasids_cancel_attach, > > > > Let me give it a try and come back. > > > > > > > While I am trying to fit the IOASIDs cgroup in to the misc cgroup > > > proposal. I'd like to have a direction check on whether this idea of > > > using cgroup for IOASID/PASID resource management is viable. > > > > Yes, even for host SVA it would be good to have a cgroup. Currently the > > number of shared address spaces is naturally limited by number of > > processes, which can be controlled with rlimit and cgroup. But on Arm the > > hardware limit on shared address spaces is 64k (number of ASIDs), easily > > exhausted with the default PASID and PID limits. So a cgroup for managing > > this resource is more than welcome. > > > > It looks like your current implementation is very dependent on > > IOASID_SET_TYPE_MM? I'll need to do more reading about cgroup to see how > > easily it can be adapted to host SVA which uses IOASID_SET_TYPE_NULL. > > > Right, I was assuming have three use cases of IOASIDs: > 1. host supervisor SVA (not a concern, just one init_mm to bind) > 2. host user SVA, either one IOASID per process or perhaps some private > IOASID for private address space > 3. VM use for guest SVA, each IOASID is bound to a guest process > > My current cgroup proposal applies to #3 with IOASID_SET_TYPE_MM, which is > allocated by the new /dev/ioasid interface. > > For #2, I was thinking you can limit the host process via PIDs cgroup? i.e. > limit fork. That works but isn't perfect, because the hardware resource of shared address spaces can be much lower that PID limit - 16k ASIDs on Arm. To allow an admin to fairly distribute that resource we could introduce another cgroup just to limit the number of shared address spaces, but limiting the number of IOASIDs does the trick. > So the host IOASIDs are currently allocated from the system pool > with quota of chosen by iommu_sva_init() in my patch, 0 means unlimited use > whatever is available. https://lkml.org/lkml/2021/2/28/18 Yes that's sensible, but it would be good to plan the cgroup user interface to work for #2 as well, even if we don't implement it right away. Thanks, Jean