Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH v9 12/22] s390: vfio-ap: sysfs interfaces to configure
 control domains
To:     Halil Pasic <pasic@linux.ibm.com>,
        Cornelia Huck <cohuck@redhat.com>
Cc:     Tony Krowiak <akrowiak@linux.vnet.ibm.com>,
        linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org, freude@de.ibm.com, schwidefsky@de.ibm.com,
        heiko.carstens@de.ibm.com, borntraeger@de.ibm.com,
        kwankhede@nvidia.com, bjsdjshi@linux.vnet.ibm.com,
        pbonzini@redhat.com, alex.williamson@redhat.com,
        pmorel@linux.vnet.ibm.com, alifm@linux.vnet.ibm.com,
        mjrosato@linux.vnet.ibm.com, jjherne@linux.vnet.ibm.com,
        thuth@redhat.com, pasic@linux.vnet.ibm.com, berrange@redhat.com,
        fiuczy@linux.vnet.ibm.com, buendgen@de.ibm.com,
        frankja@linux.ibm.com
References: <1534196899-16987-1-git-send-email-akrowiak@linux.vnet.ibm.com>
 <1534196899-16987-13-git-send-email-akrowiak@linux.vnet.ibm.com>
 <20180820162317.08bd7d23.cohuck@redhat.com>
 <660de00a-c403-28c1-4df4-82a973ab3ad5@linux.ibm.com>
 <20180821172548.57a6c758.cohuck@redhat.com>
 <82a391ee-85b1-cdc7-0f9b-d37fd8ba8e47@linux.ibm.com>
 <a43ef10f-aa91-cb4c-a8aa-cd3ce9cfebf2@linux.ibm.com>
From:   Tony Krowiak <akrowiak@linux.ibm.com>
Date:   Wed, 22 Aug 2018 10:31:17 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.2.0
MIME-Version: 1.0
In-Reply-To: <a43ef10f-aa91-cb4c-a8aa-cd3ce9cfebf2@linux.ibm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Message-Id: <7ab7fcbb-b6b0-c6b6-1704-70c4fc146618@linux.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 08/21/2018 07:18 PM, Halil Pasic wrote:
>
>
> On 08/21/2018 07:07 PM, Tony Krowiak wrote:
>> On 08/21/2018 11:25 AM, Cornelia Huck wrote:
>>> On Mon, 20 Aug 2018 13:41:32 -0400
>>> Tony Krowiak <akrowiak@linux.ibm.com> wrote:
>>>
>>>> On 08/20/2018 10:23 AM, Cornelia Huck wrote:
>>>>> On Mon, 13 Aug 2018 17:48:09 -0400
>>>>> Tony Krowiak <akrowiak@linux.vnet.ibm.com> wrote:
>>>>>> From: Tony Krowiak <akrowiak@linux.ibm.com>
>>>>>>
>>>>>> Provides the sysfs interfaces for:
>>>>>>
>>>>>> 1. Assigning AP control domains to the mediated matrix device
>>>>>>
>>>>>> 2. Unassigning AP control domains from a mediated matrix device
>>>>>>
>>>>>> 3. Displaying the control domains assigned to a mediated matrix
>>>>>>      device
>>>>>>
>>>>>> The IDs of the AP control domains assigned to the mediated matrix
>>>>>> device are stored in an AP domain mask (ADM). The bits in the ADM,
>>>>>> from most significant to least significant bit, correspond to
>>>>>> AP domain numbers 0 to 255. On some systems, the maximum allowable
>>>>>> domain number may be less than 255 - depending upon the host's
>>>>>> AP configuration - and assignment may be rejected if the input
>>>>>> domain ID exceeds the limit.
>>>>> Please remind me of the relationship between control domains and 
>>>>> usage
>>>>> domains... IIRC, usage domains allow both requests and configuration,
>>>>> while control domains allow only configuration, and are by 
>>>>> convention a
>>>>> superset of usage domains.
>>>> A usage domain is a domain to which an AP command-request message 
>>>> can be
>>>> submitted for processing. A control domain is a domain that can
>>>> be changed by an AP command request message submitted to a usage 
>>>> domain.
>>>> AP command request messages to configure a domain will contain the 
>>>> domain
>>>> number of the domain to be modified. The AP firmware will check the
>>>> control domain mask (ADM) and will allow the request to proceed 
>>>> only if
>>>> the corresponding bit in the ADM is set.
>>> Thanks to you and Halil for the explanation.
>>>
>>>>> Is there a hard requirement somewhere in there, or can the admin
>>>>> cheerfully use different masks for usage domains and control domains
>>>>> without the SIE choking on it?
>>>> There is no hard requirement that control domains must be a 
>>>> superset of
>>>> the usage domains, it is merely an architectural convention. AFAIK,
>>>> SIE doesn't enforce this and will not break if the convention is not
>>>> enforced externally. Having said that, you should note that the AQM
>>>> and ADM masks configured for the mediated matrix device will be 
>>>> logically
>>>> OR'd together to create the ADM stored in the CRYCB referenced from 
>>>> the
>>>> guest's SIE state description. In other words, we are enforcing the
>>>> convention in our software.
>>> Hm, that's interesting, as Halil argued that we should not enforce it
>>> in the kernel. Might be somewhat surprising as well. If that is really
>>> the way to do it, this needs to be documented clearly.
>>
>> This convention has been enforced by the kernel since v1. This is also
>> enforced by both the LPAR as well as in z/VM. The following is from the
>> PR/SM Planning Guide:
>>
>> Control Domain
>> A logical partition's control domains are those cryptographic domains 
>> for which remote secure
>> administration functions can be established and administered from 
>> this logical partition. This
>> logical partition’s control domains must include its usage domains. 
>> For each index selected in the
>> usage domain index list, you must select the same index in the 
>> control domain index list
>>
>
> IMHO this quote is quite a half-full half-empty cup one:
> * it mandates the set of usage domains is a subset of the set
> of the control domains, but
> * it speaks of independent controls, namely about the 'usage domain 
> index'
> and the 'control domain index list' and makes the enforcement of the rule
> a job of the administrator (instead of codifying it in the controls).

For what it's worth, I spoke with the z/VM developers about dedicated crypto
in z/VM. In z/VM dedicated crypto, control domains are not even 
configured by
the admin. All configured usage domains are also configured as control 
domains.

>
>
>>
>> Consequently, I'm going to opt for ensuring this is clearly 
>> documented. Based on the fact you've
>> requested clarification of many points described in this section of 
>> the doc, I
>> think I'll try putting my meager skills as a wordsmith to work to 
>> hopefully clarify things.
>> I'll run it by you when I complete that task to see if I've succeeded:)
>
> I don't think just a doc update will do. Let me explain why.
>
> What describe as "... note that the AQM and ADM masks configured for the
> mediated matrix device will be logically OR'd together to create the ADM
> stored in the CRYCB referenced from the guest's SIE state description."
> is a gotcha at best. The member of struct ap_matrix and the member of the
> respective apcb in the crycb are both called 'adm', but ap_matrix.adm is
> not an ADM as we know it from the architecture, but rather ~ AQM & ADM.
>
> I feel pretty strongly about this one. If we want to keep the enforcement
> in the kernel, I guess, the assign_domain should set the bit 
> corresponding
> bit not only in ap_matrix.aqm but also in ap_matrix.adm. When the
> ap_matrix is committed into the crycb no further manipulating the masks
> should take place.

I have no problem with this and considered implementing it that way at one
time.

>
> I don't feel strongly about whether to enforce this convention about AQM
> and ADM in the kernel or not. Frankly, I don't know what is behind the
> rule. Since I can't tell if any problems are to be expected if this
> convention is violated, I would feel more comfortable if the rule was
> accommodated higher in the management stack.

I wouldn't describe it as a rule. It is described in the architecture doc
as an architectural convention; in other words, it is agreed upon that all
usage domains should also be control domains. Based on my discussions with
the z/VM developers, I believe the reason for the convention is to ensure a
system has control over its own usage domains, but that is just my
interpretation.

>
>
> Regards,
> Halil
>
>>
>>>
>>