Received: by 10.213.65.68 with SMTP id h4csp3663859imn; Tue, 3 Apr 2018 08:44:45 -0700 (PDT) X-Google-Smtp-Source: AIpwx48yqMmF9w6CrU5+QWQl99W/sj5Smt2X5oMCuTwTyDySrYrdzwUc6RfpO3YoUcSV8blcwT3Y X-Received: by 2002:a17:902:bb81:: with SMTP id m1-v6mr14374951pls.71.1522770285350; Tue, 03 Apr 2018 08:44:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522770285; cv=none; d=google.com; s=arc-20160816; b=M3dHCk1tNaYMSf1Z6y5CmLv6VxGXphWxtMeNQO0p8i+1PxF0+c/t9DnKqLWmseBwWh NHF4zGYbCr2LPH2tfAPBRbAb2ZFCFUzvRXuzubJXTfZJxUjXdZTvuNMT68ODju75jjVm uA5kFCM993NRzaoQDa1j9AfGTp9FZznv3Elz4OuP9XGJ1T51jcU+wFsLMw+lyGH1okgI H/QdPGej9FOv832llQhwZdpc8+usxOgYPKvkzuT82WOLlQ1i3JR5cngDoZjKuFZQ8JKJ VYSW3gl/OYt066bKZJIMgdPpPSX0nmE2c0jI2sZ5fclPYiTajaxOGcvh4mtS+GdfmhEx LQwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:content-language :content-transfer-encoding:in-reply-to:mime-version:user-agent:date :from:references:cc:to:subject:arc-authentication-results; bh=PE0vfEjcfEmGxwlw7GbBqZb1dbVb9ooxilDHCNf3ggo=; b=mSRxueuHR2RYHPTTyUS9EYwcHSOJo6jZYU8TCwhacoyMYekdJwJRckg7zdLMFGr9pc 5kaAlZLGBRjhDDwyikEs4pkhvOBEKtGldDzk2YeKwCXJ8bnMnEAIqENJbsRe5TgUWuVB ebYbSI1tWC0ELOi52oLFCZWVHR99sJql1HPToGMru9YxOvdSAhI7Bs5PpaHszokkkFPV d8Z0Ok5jhnL5I0+sK7JA9o5BwhfE0/97UoL2iEVUlTSvRZ3sPD5oO4yq0BxBG2DojWDX pmM0yXh+TzyPGRD9JBxACZBCeF7M6kC3+ln7ADmZ6w7Je1iMLooGEsPrCZjIglYle1lO nB5g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 11-v6si771950plb.658.2018.04.03.08.44.30; Tue, 03 Apr 2018 08:44:45 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752117AbeDCPmv (ORCPT + 99 others); Tue, 3 Apr 2018 11:42:51 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:39752 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751645AbeDCPmt (ORCPT ); Tue, 3 Apr 2018 11:42:49 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w33FcWgG127211 for ; Tue, 3 Apr 2018 11:42:49 -0400 Received: from e38.co.us.ibm.com (e38.co.us.ibm.com [32.97.110.159]) by mx0a-001b2d01.pphosted.com with ESMTP id 2h49nutk2x-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Tue, 03 Apr 2018 11:42:48 -0400 Received: from localhost by e38.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Apr 2018 09:42:47 -0600 Received: from b03cxnp07029.gho.boulder.ibm.com (9.17.130.16) by e38.co.us.ibm.com (192.168.1.138) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 3 Apr 2018 09:42:43 -0600 Received: from b03ledav002.gho.boulder.ibm.com (b03ledav002.gho.boulder.ibm.com [9.17.130.233]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w33Fgg8g9699774; Tue, 3 Apr 2018 08:42:42 -0700 Received: from b03ledav002.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 30A6413603C; Tue, 3 Apr 2018 09:42:42 -0600 (MDT) Received: from oc8043147753.ibm.com (unknown [9.60.75.175]) by b03ledav002.gho.boulder.ibm.com (Postfix) with ESMTP id 8E3EC136043; Tue, 3 Apr 2018 09:42:39 -0600 (MDT) Subject: Re: [PATCH v3 09/14] s390: vfio-ap: sysfs interfaces to configure domains To: Cornelia Huck Cc: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, freude@de.ibm.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, borntraeger@de.ibm.com, kwankhede@nvidia.com, bjsdjshi@linux.vnet.ibm.com, pbonzini@redhat.com, alex.williamson@redhat.com, pmorel@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com, mjrosato@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, thuth@redhat.com, pasic@linux.vnet.ibm.com, berrange@redhat.com, fiuczy@linux.vnet.ibm.com, buendgen@de.ibm.com, akrowiak@linux.vnet.ibm.com References: <1521051954-25715-1-git-send-email-akrowiak@linux.vnet.ibm.com> <1521051954-25715-10-git-send-email-akrowiak@linux.vnet.ibm.com> <20180403131759.24516cf4.cohuck@redhat.com> <1860430c-df59-6d58-77f9-b36c51595b4b@linux.vnet.ibm.com> <20180403171910.2edaf63c.cohuck@redhat.com> From: Tony Krowiak Date: Tue, 3 Apr 2018 11:42:38 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20180403171910.2edaf63c.cohuck@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-TM-AS-GCONF: 00 x-cbid: 18040315-0028-0000-0000-000009641A9C X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008795; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000255; SDB=6.01012562; UDB=6.00516063; IPR=6.00791828; MB=3.00020393; MTD=3.00000008; XFM=3.00000015; UTC=2018-04-03 15:42:45 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18040315-0029-0000-0000-00003A3978CA Message-Id: <66115f68-809b-d765-5916-e3d8d391db5a@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2018-04-03_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804030162 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/03/2018 11:19 AM, Cornelia Huck wrote: > On Tue, 3 Apr 2018 11:12:45 -0400 > Tony Krowiak wrote: > >> On 04/03/2018 07:17 AM, Cornelia Huck wrote: >>> On Wed, 14 Mar 2018 14:25:49 -0400 >>> Tony Krowiak wrote: >>> >>>> Provides the sysfs interfaces for assigning AP domains to >>>> and unassigning AP domains from a mediated matrix device. >>>> >>>> An AP domain ID corresponds to an AP queue index (APQI). For >>>> each domain assigned to the mediated matrix device, its >>>> corresponging APQI is stored in an AP queue mask (AQM). >>>> The bits in the AQM, from most significant to least >>>> significant bit, correspond to AP domain numbers 0 to 255. >>>> When a domain is assigned, the bit corresponding to its >>>> APQI will be set in the AQM. Likewise, when a domain is >>>> unassigned, the bit corresponding to its APQI will be >>>> cleared from the AQM. >>>> >>>> The relevant sysfs structures are: >>>> >>>> /sys/devices/vfio_ap >>>> ... [matrix] >>>> ...... [mdev_supported_types] >>>> ......... [vfio_ap-passthrough] >>>> ............ [devices] >>>> ...............[$uuid] >>>> .................. assign_domain >>>> .................. unassign_domain >>>> >>>> To assign a domain to the $uuid mediated matrix device, >>>> write the domain's ID to the assign_domain file. To >>>> unassign a domain, write the domain's ID to the >>>> unassign_domain file. The ID is specified using >>>> conventional semantics: If it begins with 0x, the number >>>> will be parsed as a hexadecimal (case insensitive) number; >>>> otherwise, it will be parsed as a decimal number. >>>> >>>> For example, to assign domain 173 (0xad) to the mediated matrix >>>> device $uuid: >>>> >>>> echo 173 > assign_domain >>>> >>>> or >>>> >>>> echo 0xad > assign_domain >>>> >>>> To unassign domain 173 (0xad): >>>> >>>> echo 173 > unassign_domain >>>> >>>> or >>>> >>>> echo 0xad > unassign_domain >>>> >>>> The assignment will be rejected: >>>> >>>> * If the domain ID exceeds the maximum value for an AP domain: >>>> >>>> * If the AP Extended Addressing (APXA) facility is installed, >>>> the max value is 255 >>>> >>>> * Else the max value is 15 >>>> >>>> * If no AP adapters have yet been assigned and there are >>>> no AP queues reserved by the VFIO AP driver that have an APQN >>>> with an APQI matching that of the AP domain number being >>>> assigned. >>>> >>>> * If any of the APQNs that can be derived from the intersection >>>> of the APQI being assigned and the AP adapter ID (APID) of >>>> each of the AP adapters previously assigned can not be matched >>>> with an APQN of an AP queue device reserved by the VFIO AP >>>> driver. >>>> >>>> Signed-off-by: Tony Krowiak >>>> --- >>>> arch/s390/include/asm/kvm-ap.h | 1 + >>>> drivers/s390/crypto/vfio_ap_ops.c | 215 ++++++++++++++++++++++++++++++++++++- >>>> 2 files changed, 215 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c >>>> index 90512a6..c448835 100644 >>>> --- a/drivers/s390/crypto/vfio_ap_ops.c >>>> +++ b/drivers/s390/crypto/vfio_ap_ops.c >>>> @@ -377,10 +377,223 @@ static ssize_t unassign_adapter_store(struct device *dev, >>>> } >>>> DEVICE_ATTR_WO(unassign_adapter); >>>> >>>> +/** >>>> + * vfio_ap_validate_queues_for_apqi >>>> + * >>>> + * @ap_matrix: the matrix device >>>> + * @matrix_mdev: the mediated matrix device >>>> + * @apqi: an AP queue index (APQI) - corresponds to a domain ID >>>> + * >>>> + * Verifies that each APQN that is derived from the intersection of @apqi and >>>> + * each AP adapter ID (APID) corresponding to an AP domain assigned to the >>>> + * @matrix_mdev matches the APQN of an AP queue reserved by the VFIO AP device >>>> + * driver. >>>> + * >>>> + * Returns 0 if validation succeeds; otherwise, returns an error. >>>> + */ >>>> +static int vfio_ap_validate_queues_for_apqi(struct ap_matrix *ap_matrix, >>>> + struct ap_matrix_mdev *matrix_mdev, >>>> + unsigned long apqi) >>>> +{ >>>> + int ret; >>>> + struct vfio_ap_qid_match qid_match; >>>> + unsigned long apid; >>>> + struct device_driver *drv = ap_matrix->device.driver; >>>> + >>>> + /** >>>> + * Examine each APQN with the specified APQI >>>> + */ >>>> + for_each_set_bit_inv(apid, matrix_mdev->matrix->apm, >>>> + matrix_mdev->matrix->apm_max) { >>>> + qid_match.qid = AP_MKQID(apid, apqi); >>>> + qid_match.dev = NULL; >>>> + >>>> + ret = driver_for_each_device(drv, NULL, &qid_match, >>>> + vfio_ap_queue_match); >>>> + if (ret) >>>> + return ret; >>> Hm, I'm wondering whether jumping out of the outer loop is the correct >>> thing to do here - and if yes, whether we should log an error? >> If you look at the vfio_ap_queue_match() function which is passed to the >> driver_for_each_device() function, it never returns an error. The >> driver_for_each_device() function only returns an error if the function >> passed in returns an error, so in reality, the value of *ret *will never >> be anything but 0. Having said that, there are no guarantees that the >> vfio_ap_queue_match() function will never change, so it would probably >> be a good idea to log an error if *ret *is not 0.**I think returning at >> this point is valid because a non-zero is returned from >> driver_for_each_device() function as soon as the input function returns >> a non-zero value for a specific device. This means that subsequent >> devices will not be processed, so we may not know whether an AP queue >> has been reserved or not - see check below. > OK, then logging an error makes the most sense. > > Is there a source tree with the patches somewhere, btw? Checking out a > branch is less time-consuming than applying a series (and helps > review). Same applies to the qemu patches; maybe one of the IBM > maintainers can push to a branch? I'll check with Christian for the forthcoming v4 patches. > >>> >>>> + >>>> + /* >>>> + * If the APQN identifies an AP queue that is reserved by the >>>> + * VFIO AP device driver, continue processing. >>>> + */ >>>> + if (qid_match.dev) >>>> + continue; >>>> + >>>> + pr_err("%s: AP queue %02lx.%04lx not reserved by %s driver", >>>> + VFIO_AP_MATRIX_MODULE_NAME, apqi, apqi, >>>> + VFIO_AP_DRV_NAME); >>>> + >>>> + return -ENXIO; >>>> + } >>>> + >>>> + return 0; >>>> +} >>