Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH v9 21/22] KVM: s390: CPU model support for AP
 virtualization
To:     Tony Krowiak <akrowiak@linux.ibm.com>, pmorel@linux.ibm.com,
        Halil Pasic <pasic@linux.ibm.com>,
        Tony Krowiak <akrowiak@linux.vnet.ibm.com>,
        linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org
Cc:     freude@de.ibm.com, schwidefsky@de.ibm.com,
        heiko.carstens@de.ibm.com, borntraeger@de.ibm.com,
        cohuck@redhat.com, kwankhede@nvidia.com,
        bjsdjshi@linux.vnet.ibm.com, pbonzini@redhat.com,
        alex.williamson@redhat.com, pmorel@linux.vnet.ibm.com,
        alifm@linux.vnet.ibm.com, mjrosato@linux.vnet.ibm.com,
        jjherne@linux.vnet.ibm.com, thuth@redhat.com,
        pasic@linux.vnet.ibm.com, berrange@redhat.com,
        fiuczy@linux.vnet.ibm.com, buendgen@de.ibm.com,
        frankja@linux.ibm.com
References: <1534196899-16987-1-git-send-email-akrowiak@linux.vnet.ibm.com>
 <1534196899-16987-22-git-send-email-akrowiak@linux.vnet.ibm.com>
 <2c2c4859-ed3e-a492-dd59-78529c7768b2@redhat.com>
 <95e6ee74-69aa-9805-3233-b9ec81fce516@redhat.com>
 <7e7a35f5-d1eb-7719-c5e8-57d6f19db675@linux.ibm.com>
 <f5ec13ae-75b4-918e-aaf1-ed97944ff16e@redhat.com>
 <fec90446-7f94-da28-d958-427a0446854e@linux.ibm.com>
 <ef6082b7-da96-3a0b-77a0-f41d113d058a@redhat.com>
 <8d6ae58f-967b-5e4e-0e54-8fb4962cb843@linux.ibm.com>
 <be13d8fa-1ff7-6800-36b9-7f8320724cbf@redhat.com>
 <049c5e8a-4f21-a079-0eb6-abe78d812de7@linux.ibm.com>
 <1721a153-13f0-e695-6c01-cf6b65e1bbfa@linux.ibm.com>
 <e782de8d-45d8-79fd-165c-c84f431e1d34@redhat.com>
 <cd00d49a-f6d0-a911-8cda-8f95f58beb27@linux.ibm.com>
 <79ae3a9c-70c3-7b08-09ee-0b8308cbca56@linux.ibm.com>
From:   David Hildenbrand <david@redhat.com>
Openpgp: preference=signencrypt
Autocrypt: addr=david@redhat.com; prefer-encrypt=mutual; keydata=
 xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ
 dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL
 QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp
 XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK
 Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9
 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt
 WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc
 UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv
 jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb
 B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk
 ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwX4EEwECACgFAljj9eoCGwMFCQlmAYAGCwkI
 BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEE3eEPcA/4Na5IIP/3T/FIQMxIfNzZshIq687qgG
 8UbspuE/YSUDdv7r5szYTK6KPTlqN8NAcSfheywbuYD9A4ZeSBWD3/NAVUdrCaRP2IvFyELj
 xoMvfJccbq45BxzgEspg/bVahNbyuBpLBVjVWwRtFCUEXkyazksSv8pdTMAs9IucChvFmmq3
 jJ2vlaz9lYt/lxN246fIVceckPMiUveimngvXZw21VOAhfQ+/sofXF8JCFv2mFcBDoa7eYob
 s0FLpmqFaeNRHAlzMWgSsP80qx5nWWEvRLdKWi533N2vC/EyunN3HcBwVrXH4hxRBMco3jvM
 m8VKLKao9wKj82qSivUnkPIwsAGNPdFoPbgghCQiBjBe6A75Z2xHFrzo7t1jg7nQfIyNC7ez
 MZBJ59sqA9EDMEJPlLNIeJmqslXPjmMFnE7Mby/+335WJYDulsRybN+W5rLT5aMvhC6x6POK
 z55fMNKrMASCzBJum2Fwjf/VnuGRYkhKCqqZ8gJ3OvmR50tInDV2jZ1DQgc3i550T5JDpToh
 dPBxZocIhzg+MBSRDXcJmHOx/7nQm3iQ6iLuwmXsRC6f5FbFefk9EjuTKcLMvBsEx+2DEx0E
 UnmJ4hVg7u1PQ+2Oy+Lh/opK/BDiqlQ8Pz2jiXv5xkECvr/3Sv59hlOCZMOaiLTTjtOIU7Tq
 7ut6OL64oAq+zsFNBFXLn5EBEADn1959INH2cwYJv0tsxf5MUCghCj/CA/lc/LMthqQ773ga
 uB9mN+F1rE9cyyXb6jyOGn+GUjMbnq1o121Vm0+neKHUCBtHyseBfDXHA6m4B3mUTWo13nid
 0e4AM71r0DS8+KYh6zvweLX/LL5kQS9GQeT+QNroXcC1NzWbitts6TZ+IrPOwT1hfB4WNC+X
 2n4AzDqp3+ILiVST2DT4VBc11Gz6jijpC/KI5Al8ZDhRwG47LUiuQmt3yqrmN63V9wzaPhC+
 xbwIsNZlLUvuRnmBPkTJwwrFRZvwu5GPHNndBjVpAfaSTOfppyKBTccu2AXJXWAE1Xjh6GOC
 8mlFjZwLxWFqdPHR1n2aPVgoiTLk34LR/bXO+e0GpzFXT7enwyvFFFyAS0Nk1q/7EChPcbRb
 hJqEBpRNZemxmg55zC3GLvgLKd5A09MOM2BrMea+l0FUR+PuTenh2YmnmLRTro6eZ/qYwWkC
 u8FFIw4pT0OUDMyLgi+GI1aMpVogTZJ70FgV0pUAlpmrzk/bLbRkF3TwgucpyPtcpmQtTkWS
 gDS50QG9DR/1As3LLLcNkwJBZzBG6PWbvcOyrwMQUF1nl4SSPV0LLH63+BrrHasfJzxKXzqg
 rW28CTAE2x8qi7e/6M/+XXhrsMYG+uaViM7n2je3qKe7ofum3s4vq7oFCPsOgwARAQABwsFl
 BBgBAgAPBQJVy5+RAhsMBQkJZgGAAAoJEE3eEPcA/4NagOsP/jPoIBb/iXVbM+fmSHOjEshl
 KMwEl/m5iLj3iHnHPVLBUWrXPdS7iQijJA/VLxjnFknhaS60hkUNWexDMxVVP/6lbOrs4bDZ
 NEWDMktAeqJaFtxackPszlcpRVkAs6Msn9tu8hlvB517pyUgvuD7ZS9gGOMmYwFQDyytpepo
 YApVV00P0u3AaE0Cj/o71STqGJKZxcVhPaZ+LR+UCBZOyKfEyq+ZN311VpOJZ1IvTExf+S/5
 lqnciDtbO3I4Wq0ArLX1gs1q1XlXLaVaA3yVqeC8E7kOchDNinD3hJS4OX0e1gdsx/e6COvy
 qNg5aL5n0Kl4fcVqM0LdIhsubVs4eiNCa5XMSYpXmVi3HAuFyg9dN+x8thSwI836FoMASwOl
 C7tHsTjnSGufB+D7F7ZBT61BffNBBIm1KdMxcxqLUVXpBQHHlGkbwI+3Ye+nE6HmZH7IwLwV
 W+Ajl7oYF+jeKaH4DZFtgLYGLtZ1LDwKPjX7VAsa4Yx7S5+EBAaZGxK510MjIx6SGrZWBrrV
 TEvdV00F2MnQoeXKzD7O4WFbL55hhyGgfWTHwZ457iN9SgYi1JLPqWkZB0JRXIEtjd4JEQcx
 +8Umfre0Xt4713VxMygW0PnQt5aSQdMD58jHFxTk092mU+yIHj5LeYgvwSgZN4airXk5yRXl
 SE+xAvmumFBY
Organization: Red Hat GmbH
Message-ID: <a228497a-ce48-28bd-0244-b557405c0a84@redhat.com>
Date:   Thu, 23 Aug 2018 19:40:59 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <79ae3a9c-70c3-7b08-09ee-0b8308cbca56@linux.ibm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 23.08.2018 19:35, Tony Krowiak wrote:
> On 08/23/2018 10:59 AM, Pierre Morel wrote:
>> On 23/08/2018 15:38, David Hildenbrand wrote:
>>> On 23.08.2018 15:22, Halil Pasic wrote:
>>>>
>>>>
>>>> On 08/23/2018 02:47 PM, Pierre Morel wrote:
>>>>> On 23/08/2018 13:12, David Hildenbrand wrote:
>>>> [..]
>>>>>>>>>
>>>>>>>>> I'm confused, which 128 bit?
>>>>>>>>
>>>>>>>>
>>>>>>>> Me too :) , I was assuming this block to be 128bit, but the qci 
>>>>>>>> block
>>>>>>>> has 128 bytes....
>>>>>>>>
>>>>>>>> And looking at arch/s390/include/asm/ap.h, there is a lot of 
>>>>>>>> information
>>>>>>>> contained that is definitely not of interest for CPU models...
>>>>>>>>
>>>>>>>> I wonder if there is somewhere defined which bits are reserved for
>>>>>>>> future features/facilities, compared to ap masks and such.
>>>>>>>>
>>>>>>>> This is really hard to understand/plan without access to 
>>>>>>>> documentation.
>>>>>>>>
>>>>>>>> You (Halil, Tony, Pier, ...) should have a look if what I described
>>>>>>>> related to PQAP(QCI) containing features that should get part of 
>>>>>>>> the CPU
>>>>>>>> model makes sense or not. For now I was thinking that there is 
>>>>>>>> some part
>>>>>>>> inside of QCI that is strictly reserved for facilities/features 
>>>>>>>> that we
>>>>>>>> can use.
>>>>
>>>> No there is no such part. The architecture documentation is quite 
>>>> confusing
>>>> with some aspects (e.g. persistence) of how exactly some of these 
>>>> features
>>>> work and are indicated. I'm having a hard time finding my opinion. I 
>>>> may
>>>> end up asking some questions later, but for now i have to think first.
>>>>
>>>> Just one hint. There is a programming note stating that if bit 2 of the
>>>> QCI block is one there is at least one AP card in the machine that 
>>>> actually
>>>> has APXA installed.
>>>>
>>>> I read the architecture so that the APXA has a 'cpu part' (if we are
>>>> doing APXA the cpu can't spec exception on certain bits not being zor9)
>>>> and a 'card(s) part'.
>>>>
>>>> Since the stuff seems quite difficult to sort out properly, I ask 
>>>> myself
>>>> are there real problems we must solve?
>>>>
>>>> This ultimately seems to be  about the migration, right? You say 
>>>> 'This helps
>>>> to catch nasty migration bugs (e.g. APXA suddenly disappearing).' at 
>>>> the very
>>>> beginning of the discussion. Yes, we don't have to have an vfio_ap 
>>>> device,
>>>> he guest can and will start looking for AP resources if
>>>> only the cpu model features installed. So the guest could observe
>>>> a disappearing APXA, but I don't think that would lead to problems 
>>>> (with
>>>> Linux at least).
>>>>
>>>> And there ain't much AP a guest can sanely do without if no AP 
>>>> resources
>>>> are there.
>>>>
>>>> I would really prefer not rushing a solution if we don't have to.
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> What is apsc, qact, rc8a in the qci blocks? are the facility bits?
>>>>>
>>>>> Yes, facility bits concerning the AP instructions
>>>>>
>>>>
>>>> According to the current AR document rc8a ain't a facility but bits
>>>> 0-2 and 4-7 kind of are.
>>>>
>>>
>>> Easy ( :) ) answer. Everything that is the CPU part should get into the
>>> CPU model. Everything that is AP specific not. If APXA is not a CPU
>>> facility, fine with me to leave it out.
>>>
>>> Ack to not rushing, but also ack to not leaving out important things.
>>> Ack that this stuff is hard to ficure out.
>>
>> APXA is not a CPU part, it is a machine part (SIE) and a AP part 
>> (QCI,TAPQ),
>> it has no influence on CPU instructions but on the AP instructions.
>> Consequently, if I understood the definition correctly, it should not 
>> go in the CPU model.
> 
> The APXA bit returned via the PQAP(QCI) instruction indicates the APXA 
> facility is
> installed in the CPUs of the configuration. This means that the facility is
> installed in one or more adjunct processors but not necessarily all. 
> Given that
> it indicates a CPU property, maybe it does belong in the CPU model?
> 

Hmmm, I tend to agree - especially as it affects SIE behavior. But as
this is not a feature block (compared to what I thought), this clould be
model as a CPU feature like AP.

What about the other facilities? Do they smell more like AP card
specific stuff?

-- 

Thanks,

David / dhildenb