Received: by 10.192.165.156 with SMTP id m28csp620578imm; Mon, 16 Apr 2018 06:09:25 -0700 (PDT) X-Google-Smtp-Source: AIpwx4+ltpye9E3eqjpdvUVCWeeJ5PlzsSGT5e2CgFCQqgJrfePgraZN3xYHdEnlexThDAbgIhqK X-Received: by 10.98.18.67 with SMTP id a64mr15891156pfj.153.1523884165499; Mon, 16 Apr 2018 06:09:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523884165; cv=none; d=google.com; s=arc-20160816; b=h2BUQUgianOvpHaPVOctfMtxT0crzVx9Ouv6JW7uxAqARANbtt4/DORJ0Rlravms9M 4yh7ypHgHhexsfy5/i7d8G3sqZMR+OT5hckHaZYbXF56zPuIBeWla6XNjPncNzgv5eLV T2pBuoOEnFQBhcVxzfxihIciaSr5MtaqpDyT9xqXFUcPvwA3r9239lkNuusWBjdoOefX 1vEDWYMvSdbL2A3z2YoXRDhcmtBuZtijVYQO9NQk8PH568geFOD0mxwqNWqAkxmNDys5 4dD7vQpKFcGfxDRjGXmoXaFApXKujdX0ojb6yWXmbibFTnbIwxSWStEMYFm33xLNJiRu TZuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:content-language :content-transfer-encoding:in-reply-to:mime-version:user-agent:date :from:references:cc:to:subject:arc-authentication-results; bh=gPN8s7hVybxXDgm0OT5aCVGFvM6eVGii04oR1GnvheU=; b=ggs8EqjlcIaSBuuc9dXfeJ+2vK9LF4RmEJ3u4rvmJO5+non6Vy4fUH5/mD0oGWa+WD tKbYLG7in2+UoLdlrYKNqU6w6E1DHgHruZA6rZ5KDldqLUkHNlZrpgaGHPb0CwJrEINe 65zKIGpwQfkLgNjjE9XZ0Aafi683Gu/KsO49wW+s3LB4HVrNwI3inH4shYaju5gKTto3 +cYSpt+YKitk13b0x5RNXCdP7RSu3OXGJEd6ilW7BHNY+UCjXH5h/+Rl9/n6S7kauXbl vqkv3ykpPDQRaDppIstmxEdyv6J/B8ecdfxUu6BiO4rC5ng/augbFHBeCaPKn/7mT7r6 sJKA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g34-v6si11994261pld.24.2018.04.16.06.09.02; Mon, 16 Apr 2018 06:09:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754568AbeDPNGM (ORCPT + 99 others); Mon, 16 Apr 2018 09:06:12 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:48226 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753599AbeDPNGJ (ORCPT ); Mon, 16 Apr 2018 09:06:09 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3GD2mL9014359 for ; Mon, 16 Apr 2018 09:06:07 -0400 Received: from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108]) by mx0a-001b2d01.pphosted.com with ESMTP id 2hct3wxf24-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Mon, 16 Apr 2018 09:06:07 -0400 Received: from localhost by e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 16 Apr 2018 14:06:04 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp12.uk.ibm.com (192.168.101.142) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 16 Apr 2018 14:06:00 +0100 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3GD5xeL39125232; Mon, 16 Apr 2018 13:05:59 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 830CC42045; Mon, 16 Apr 2018 13:57:34 +0100 (BST) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D8FDD42052; Mon, 16 Apr 2018 13:57:33 +0100 (BST) Received: from [9.152.224.146] (unknown [9.152.224.146]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 16 Apr 2018 13:57:33 +0100 (BST) Subject: Re: [PATCH v4 13/15] KVM: s390: configure the guest's AP devices To: Tony Krowiak , linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: freude@de.ibm.com, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, borntraeger@de.ibm.com, cohuck@redhat.com, kwankhede@nvidia.com, bjsdjshi@linux.vnet.ibm.com, pbonzini@redhat.com, alex.williamson@redhat.com, alifm@linux.vnet.ibm.com, mjrosato@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com, thuth@redhat.com, pasic@linux.vnet.ibm.com, berrange@redhat.com, fiuczy@linux.vnet.ibm.com, buendgen@de.ibm.com References: <1523827345-11600-1-git-send-email-akrowiak@linux.vnet.ibm.com> <1523827345-11600-14-git-send-email-akrowiak@linux.vnet.ibm.com> From: Pierre Morel Date: Mon, 16 Apr 2018 15:05:58 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <1523827345-11600-14-git-send-email-akrowiak@linux.vnet.ibm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-TM-AS-GCONF: 00 x-cbid: 18041613-0008-0000-0000-000004EB1827 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041613-0009-0000-0000-00001E7F2F49 Message-Id: <2e7387b9-ceb4-cdca-56ac-f5344084e10a@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-04-16_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804160124 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 15/04/2018 23:22, Tony Krowiak wrote: > Registers a group notifier during the open of the mediated > matrix device to get information on KVM presence through the > VFIO_GROUP_NOTIFY_SET_KVM event. When notified, the pointer > to the kvm structure is saved inside the mediated matrix > device. Once the VFIO AP device driver has access to KVM, > access to the APs can be configured for the guest. > > Access to APs is configured when the file descriptor for the > mediated matrix device is opened by userspace. The items to be > configured are: > > 1. The ECA.28 bit in the SIE state description determines whether > AP instructions are interpreted by the hardware or intercepted. > The VFIO AP device driver relies interpretive execution of > AP instructions so the ECA.28 bit will be set > > 2. Guest access to AP adapters, usage domains and control domains > is controlled by three bit masks referenced from the > Crypto Control Block (CRYCB) referenced from the guest's SIE state > description: > > * The AP Mask (APM) controls access to the AP adapters. Each bit > in the APM represents an adapter number - from most significant > to least significant bit - from 0 to 255. The bits in the APM > are set according to the adapter numbers assigned to the mediated > matrix device via its 'assign_adapter' sysfs attribute file. > > * The AP Queue (AQM) controls access to the AP queues. Each bit > in the AQM represents an AP queue index - from most significant > to least significant bit - from 0 to 255. A queue index references > a specific domain and is synonymous with the domian number. The > bits in the AQM are set according to the domain numbers assigned > to the mediated matrix device via its 'assign_domain' sysfs > attribute file. > > * The AP Domain Mask (ADM) controls access to the AP control domains. > Each bit in the ADM represents a control domain - from most > significant to least significant bit - from 0-255. The > bits in the ADM are set according to the domain numbers assigned > to the mediated matrix device via its 'assign_control_domain' > sysfs attribute file. > > Signed-off-by: Tony Krowiak > --- > drivers/s390/crypto/vfio_ap_ops.c | 50 +++++++++++++++++++++++++++++++++ > drivers/s390/crypto/vfio_ap_private.h | 2 + > 2 files changed, 52 insertions(+), 0 deletions(-) > > diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c > index bc2b05e..e3ff5ab 100644 > --- a/drivers/s390/crypto/vfio_ap_ops.c > +++ b/drivers/s390/crypto/vfio_ap_ops.c > @@ -53,6 +53,54 @@ static int vfio_ap_mdev_remove(struct mdev_device *mdev) > return 0; > } > > +static int vfio_ap_mdev_group_notifier(struct notifier_block *nb, > + unsigned long action, void *data) > +{ > + struct ap_matrix_mdev *matrix_mdev; > + > + if (action == VFIO_GROUP_NOTIFY_SET_KVM) { > + matrix_mdev = container_of(nb, struct ap_matrix_mdev, > + group_notifier); > + matrix_mdev->kvm = data; > + } > + > + return NOTIFY_OK; > +} > + > +static int vfio_ap_mdev_open(struct mdev_device *mdev) > +{ > + struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); > + unsigned long events; > + int ret; > + > + matrix_mdev->group_notifier.notifier_call = vfio_ap_mdev_group_notifier; > + events = VFIO_GROUP_NOTIFY_SET_KVM; > + > + ret = vfio_register_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, > + &events, &matrix_mdev->group_notifier); > + if (ret) > + return ret; > + > + ret = kvm_ap_interpret_instructions(matrix_mdev->kvm, true); Do you need this call ? apie is always enabled in KVM if AP instructions are available. Setting or not the interpretation is done for the VM in a all. It is not the right place to do it here since open is device dependent. Or we only have one device in the VM at a time. In this case, shouldn't we make it official by returning -EEXIST for the second call? > + if (ret) > + return ret; > + > + ret = kvm_ap_configure_matrix(matrix_mdev->kvm, > + matrix_mdev->matrix); > + > + return ret; > +} > + > +static void vfio_ap_mdev_release(struct mdev_device *mdev) > +{ > + struct ap_matrix_mdev *matrix_mdev = mdev_get_drvdata(mdev); > + > + kvm_ap_deconfigure_matrix(matrix_mdev->kvm); > + kvm_ap_interpret_instructions(matrix_mdev->kvm, false); This call clears the apie in KVM. This is only OK if we have a single device present until the end of the VM, otherwise AP instructions in the guest will fail after the release until the end of the VM or until a new device is plugged. > + vfio_unregister_notifier(mdev_dev(mdev), VFIO_GROUP_NOTIFY, > + &matrix_mdev->group_notifier); > +} > + > static ssize_t name_show(struct kobject *kobj, struct device *dev, char *buf) > { > return sprintf(buf, "%s\n", VFIO_AP_MDEV_NAME_HWVIRT); > @@ -754,6 +802,8 @@ static ssize_t matrix_show(struct device *dev, struct device_attribute *attr, > .mdev_attr_groups = vfio_ap_mdev_attr_groups, > .create = vfio_ap_mdev_create, > .remove = vfio_ap_mdev_remove, > + .open = vfio_ap_mdev_open, > + .release = vfio_ap_mdev_release, > }; > > int vfio_ap_mdev_register(struct ap_matrix *ap_matrix) > diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h > index f248faf..48e2806 100644 > --- a/drivers/s390/crypto/vfio_ap_private.h > +++ b/drivers/s390/crypto/vfio_ap_private.h > @@ -31,6 +31,8 @@ struct ap_matrix { > > struct ap_matrix_mdev { > struct kvm_ap_matrix *matrix; > + struct notifier_block group_notifier; > + struct kvm *kvm; > }; > > static inline struct ap_matrix *to_ap_matrix(struct device *dev) -- Pierre Morel Linux/KVM/QEMU in Böblingen - Germany