Received: by 10.213.65.68 with SMTP id h4csp3639717imn; Tue, 3 Apr 2018 08:20:50 -0700 (PDT) X-Google-Smtp-Source: AIpwx49EIvVTdk4LGU9ZFVfShjWxhoss0gCF2/UT+ffSJrJPgi/m1JzRYfff+O+BC1W5npbPzWCM X-Received: by 10.98.102.131 with SMTP id s3mr10810838pfj.89.1522768850707; Tue, 03 Apr 2018 08:20:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522768850; cv=none; d=google.com; s=arc-20160816; b=w4ctRZKyDEv76n2EKzEatVv1DvfvdvblMuGsypwXyj8rjReGCnEACBgMEkivUQGSqO zjjsvsURcGEBAv6EhXGEbk3q9/PpIDPJGver/BYnUfuZs5OO3qKoBglUjbA1sc98EGvk RjUE3B7dibjyJPwzMzLUEZe7zN6nL5nBkFsNpYE8WK158ARzddHXEo9E1QjoYpXBRdiO XzUwj8w5gUytHPb6KSImcc3rbykC70GbZe8aA9ZJpyy4QwTLI3iJfz8j4mOIlz9NAd8V 17z8J3oMnRwYd375ecYHEYk7pgmqJwUKIBSBA9ezgcep74P6bf2Xj6G2Hcv8uLsuelpJ lrMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :organization:references:in-reply-to:message-id:subject:cc:to:from :date:arc-authentication-results; bh=LPZCcmF6OhwvUKPbKJY+aU7V+0cQ97e54ZTjZObVXU8=; b=Q0JIDXT5NCgjMiO/XQ6t0yCL+vW1lcOGm2a2VI78yAiiC05NBYFpJPYrJe/fWI9s3m e7fzt03xYD5VykqiDl38Cy+Oib6Hlp0KUoGQuE+/XIgL5EaDDpXBCmyzhaCuTN3JH7K0 +TPs1rYQvHriPBLngoUrjXHC7OuGmtEWxsZAZgPp0Xa7VxzkQnLdAgen3rDVlAUS3Vwp IzWGNheeUC/4tiLJM4xAuSt7/Tl4oA6JotKS9ssRFilad9uUCZonFXZReEfhbeQymOjm 0J0cMjO11YFw6GjFqqRdGqvvbRW64eLHvlK/7sqmKvYRkqxlPbSS6f85b40ei15rOpfp MFZg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u2-v6si791479plq.334.2018.04.03.08.20.36; Tue, 03 Apr 2018 08:20:50 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752171AbeDCPTU (ORCPT + 99 others); Tue, 3 Apr 2018 11:19:20 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:54644 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751219AbeDCPTS (ORCPT ); Tue, 3 Apr 2018 11:19:18 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 787288182D0C; Tue, 3 Apr 2018 15:19:17 +0000 (UTC) Received: from gondolin (dhcp-192-222.str.redhat.com [10.33.192.222]) by smtp.corp.redhat.com (Postfix) with ESMTP id 76AD6111DCED; Tue, 3 Apr 2018 15:19:12 +0000 (UTC) Date: Tue, 3 Apr 2018 17:19:10 +0200 From: Cornelia Huck To: Tony Krowiak Cc: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, freude@de.ibm.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, borntraeger@de.ibm.com, kwankhede@nvidia.com, bjsdjshi@linux.vnet.ibm.com, pbonzini@redhat.com, alex.williamson@redhat.com, pmorel@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, mjrosato@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, thuth@redhat.com, pasic@linux.vnet.ibm.com, berrange@redhat.com, fiuczy@linux.vnet.ibm.com, buendgen@de.ibm.com Subject: Re: [PATCH v3 09/14] s390: vfio-ap: sysfs interfaces to configure domains Message-ID: <20180403171910.2edaf63c.cohuck@redhat.com> In-Reply-To: <1860430c-df59-6d58-77f9-b36c51595b4b@linux.vnet.ibm.com> References: <1521051954-25715-1-git-send-email-akrowiak@linux.vnet.ibm.com> <1521051954-25715-10-git-send-email-akrowiak@linux.vnet.ibm.com> <20180403131759.24516cf4.cohuck@redhat.com> <1860430c-df59-6d58-77f9-b36c51595b4b@linux.vnet.ibm.com> Organization: Red Hat GmbH MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 03 Apr 2018 15:19:17 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 03 Apr 2018 15:19:17 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'cohuck@redhat.com' RCPT:'' Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 3 Apr 2018 11:12:45 -0400 Tony Krowiak wrote: > On 04/03/2018 07:17 AM, Cornelia Huck wrote: > > On Wed, 14 Mar 2018 14:25:49 -0400 > > Tony Krowiak wrote: > > > >> Provides the sysfs interfaces for assigning AP domains to > >> and unassigning AP domains from a mediated matrix device. > >> > >> An AP domain ID corresponds to an AP queue index (APQI). For > >> each domain assigned to the mediated matrix device, its > >> corresponging APQI is stored in an AP queue mask (AQM). > >> The bits in the AQM, from most significant to least > >> significant bit, correspond to AP domain numbers 0 to 255. > >> When a domain is assigned, the bit corresponding to its > >> APQI will be set in the AQM. Likewise, when a domain is > >> unassigned, the bit corresponding to its APQI will be > >> cleared from the AQM. > >> > >> The relevant sysfs structures are: > >> > >> /sys/devices/vfio_ap > >> ... [matrix] > >> ...... [mdev_supported_types] > >> ......... [vfio_ap-passthrough] > >> ............ [devices] > >> ...............[$uuid] > >> .................. assign_domain > >> .................. unassign_domain > >> > >> To assign a domain to the $uuid mediated matrix device, > >> write the domain's ID to the assign_domain file. To > >> unassign a domain, write the domain's ID to the > >> unassign_domain file. The ID is specified using > >> conventional semantics: If it begins with 0x, the number > >> will be parsed as a hexadecimal (case insensitive) number; > >> otherwise, it will be parsed as a decimal number. > >> > >> For example, to assign domain 173 (0xad) to the mediated matrix > >> device $uuid: > >> > >> echo 173 > assign_domain > >> > >> or > >> > >> echo 0xad > assign_domain > >> > >> To unassign domain 173 (0xad): > >> > >> echo 173 > unassign_domain > >> > >> or > >> > >> echo 0xad > unassign_domain > >> > >> The assignment will be rejected: > >> > >> * If the domain ID exceeds the maximum value for an AP domain: > >> > >> * If the AP Extended Addressing (APXA) facility is installed, > >> the max value is 255 > >> > >> * Else the max value is 15 > >> > >> * If no AP adapters have yet been assigned and there are > >> no AP queues reserved by the VFIO AP driver that have an APQN > >> with an APQI matching that of the AP domain number being > >> assigned. > >> > >> * If any of the APQNs that can be derived from the intersection > >> of the APQI being assigned and the AP adapter ID (APID) of > >> each of the AP adapters previously assigned can not be matched > >> with an APQN of an AP queue device reserved by the VFIO AP > >> driver. > >> > >> Signed-off-by: Tony Krowiak > >> --- > >> arch/s390/include/asm/kvm-ap.h | 1 + > >> drivers/s390/crypto/vfio_ap_ops.c | 215 ++++++++++++++++++++++++++++++++++++- > >> 2 files changed, 215 insertions(+), 1 deletions(-) > >> > >> diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c > >> index 90512a6..c448835 100644 > >> --- a/drivers/s390/crypto/vfio_ap_ops.c > >> +++ b/drivers/s390/crypto/vfio_ap_ops.c > >> @@ -377,10 +377,223 @@ static ssize_t unassign_adapter_store(struct device *dev, > >> } > >> DEVICE_ATTR_WO(unassign_adapter); > >> > >> +/** > >> + * vfio_ap_validate_queues_for_apqi > >> + * > >> + * @ap_matrix: the matrix device > >> + * @matrix_mdev: the mediated matrix device > >> + * @apqi: an AP queue index (APQI) - corresponds to a domain ID > >> + * > >> + * Verifies that each APQN that is derived from the intersection of @apqi and > >> + * each AP adapter ID (APID) corresponding to an AP domain assigned to the > >> + * @matrix_mdev matches the APQN of an AP queue reserved by the VFIO AP device > >> + * driver. > >> + * > >> + * Returns 0 if validation succeeds; otherwise, returns an error. > >> + */ > >> +static int vfio_ap_validate_queues_for_apqi(struct ap_matrix *ap_matrix, > >> + struct ap_matrix_mdev *matrix_mdev, > >> + unsigned long apqi) > >> +{ > >> + int ret; > >> + struct vfio_ap_qid_match qid_match; > >> + unsigned long apid; > >> + struct device_driver *drv = ap_matrix->device.driver; > >> + > >> + /** > >> + * Examine each APQN with the specified APQI > >> + */ > >> + for_each_set_bit_inv(apid, matrix_mdev->matrix->apm, > >> + matrix_mdev->matrix->apm_max) { > >> + qid_match.qid = AP_MKQID(apid, apqi); > >> + qid_match.dev = NULL; > >> + > >> + ret = driver_for_each_device(drv, NULL, &qid_match, > >> + vfio_ap_queue_match); > >> + if (ret) > >> + return ret; > > Hm, I'm wondering whether jumping out of the outer loop is the correct > > thing to do here - and if yes, whether we should log an error? > If you look at the vfio_ap_queue_match() function which is passed to the > driver_for_each_device() function, it never returns an error. The > driver_for_each_device() function only returns an error if the function > passed in returns an error, so in reality, the value of *ret *will never > be anything but 0. Having said that, there are no guarantees that the > vfio_ap_queue_match() function will never change, so it would probably > be a good idea to log an error if *ret *is not 0.**I think returning at > this point is valid because a non-zero is returned from > driver_for_each_device() function as soon as the input function returns > a non-zero value for a specific device. This means that subsequent > devices will not be processed, so we may not know whether an AP queue > has been reserved or not - see check below. OK, then logging an error makes the most sense. Is there a source tree with the patches somewhere, btw? Checking out a branch is less time-consuming than applying a series (and helps review). Same applies to the qemu patches; maybe one of the IBM maintainers can push to a branch? > > > > >> + > >> + /* > >> + * If the APQN identifies an AP queue that is reserved by the > >> + * VFIO AP device driver, continue processing. > >> + */ > >> + if (qid_match.dev) > >> + continue; > >> + > >> + pr_err("%s: AP queue %02lx.%04lx not reserved by %s driver", > >> + VFIO_AP_MATRIX_MODULE_NAME, apqi, apqi, > >> + VFIO_AP_DRV_NAME); > >> + > >> + return -ENXIO; > >> + } > >> + > >> + return 0; > >> +} > >