Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <3d05fcf1-dad3-826e-03e9-599ced7524b4@oracle.com>
Date:   Wed, 17 May 2023 16:32:26 +0100
Subject: Re: [PATCH] KVM: SVM: Update destination when updating pi irte
Content-Language: en-US
To:     "dengqiao.joey" <dengqiao.joey@bytedance.com>
Cc:     kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        seanjc@google.com, pbonzini@redhat.com
References: <ZGOnPMTEKqRq89jt@google.com>
 <20230517120458.3761311-1-dengqiao.joey@bytedance.com>
From:   Joao Martins <joao.m.martins@oracle.com>
In-Reply-To: <20230517120458.3761311-1-dengqiao.joey@bytedance.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZG9PWkpIUlI3SitpL2s5MW5XMlplV1l5OGhQVU9NYUVTZkpFRXRyempkYndC?=
 =?utf-8?B?NVB4eHBqbXhpWWFKVUZzRWQ2UnFvcHFHMEd2QzRFNGtuN1NzY0JMenYwM2E3?=
 =?utf-8?B?R0R4ZHcvR2tTeGtGaytJS3BMdHlyZ2ViL2p6UDBXc01ES0JpNmNiTmMxNEhB?=
 =?utf-8?B?TmdpemxxQzlTT1I1VjdsclI4YjlHVkVWOHhkQWlER3NZZUx5dVlscTZnSjZu?=
 =?utf-8?B?L2o5Q3QzRHFJRXdDZllqbFVBRTVYQzdXczZsd0J1cWp6RlZpdys2c1hHSk9D?=
 =?utf-8?B?RmJBTzk0blBsZW8wdDNOVUFydFh3UzN6TS8rS2twRjVtZFpIR3JOWWFiTFBV?=
 =?utf-8?B?Y25TSVZmVm9KakhiM0ozMmJOSjVxQ2JQa0s3aGxkdFhMUTdZTW1wMS9HL2Mz?=
 =?utf-8?B?bTl4S1ZjZlZDbjc1ZndoSzhlR2dpNmtmRjZkelFUYlIxSTFmaWFzVkkyY2R6?=
 =?utf-8?B?RVlIY1ZQMG9IS0U2ZS9KQk8vc1V3Q09PUTcreSsyNEVSTWNWeEZxNkQ2d2Na?=
 =?utf-8?B?dGRWaFlLNnMzR0tTZUp5alZFNmFUSzYrazRxNEduWTQxUUthL1ZDVXlmSE9S?=
 =?utf-8?B?R2VRY25ydzlqV3ZTU3VUL1ZZZzlSalQyTWd0dk5qanU0MHdNYWZDdXBrWDNL?=
 =?utf-8?B?VmVzckJ1U1JqeU5oTzM5SkhJT2prTVNmVzBXcWpXTWxwN2xlVDVqV3ZrRm5L?=
 =?utf-8?B?WU1WWDBLV3ZlOUJCdUU2alYzOHg0MmR5T1MzdUpPbDNWU0NaQThobmlQWXI1?=
 =?utf-8?B?SHd4T1BNbnNiM08yalI3dkM1LzhOQkkvM1lQT1FIYlJvNXY4bzBIamI3V2w1?=
 =?utf-8?B?QXNoRUsxRkdCbE1qeVRobWs3NDBTakZXcm9wVXpzaEUwTktaREc3cDR1cHNm?=
 =?utf-8?B?RXYzLzltZVkvRmNXRVZ0NDNOdEE0OXhLcWZhbVdoR21sT1dPdUwxMnNBOERS?=
 =?utf-8?B?QUl6SGEvR0gyRmRja0NNVUFKdnBHZHVWLzByS2hidDNCT1dPMVJJZ0ZOZkhV?=
 =?utf-8?B?ZlVuTkhMY2YxdkhsR1lsMTFDRlVPWFY2TGtycGJ6dzgvTElRazBkWnE0RElW?=
 =?utf-8?B?dUk0WUZVTkVHQTBYNUZWVGExVllsZDFRNWNURXJtaGRlNkVnaGRHL2t3NDVF?=
 =?utf-8?B?N2xkU2t1eWlXbSttY2lhNXNEQ1Q4T3NwZVVxM055cUp3TEpEWE1qU0JiWW9K?=
 =?utf-8?B?Mmx5T0RoSVF4OU1Ua2gybTlsNmF2bXVyZUlYZXdaNDBnVjZGZFE2OTdoZDFW?=
 =?utf-8?B?ajBWcTZDUEdNSVRXdmtqckkyZmRhOGl4VTJjM2pCTzhaYnAxNmJYME1JWTlr?=
 =?utf-8?B?VGhKTlNiQ2ZkMHFmM1NYMkNCZjE0dXFWTFpXc0J6d0Jrdm12VFJlTjZWODJs?=
 =?utf-8?B?eFc3VFUvSnZtWG93eWcrekI5R2R0cmR1blBPN2VIQ3UrWU5mSCtrcUZCS1ZB?=
 =?utf-8?B?N1NYU2dXK2xGelJnRVhEYnViVWdadVpmOUtyaEVNK1NBZGlkcGYwNUh2NHhF?=
 =?utf-8?B?d3BuTWtlQ0pvMHZVaDFaV3Y5Y2liZkc4Vk1KZEt5K0U5cThIZVlmUmkrdGl0?=
 =?utf-8?B?Q2pjeGU5eTZSYWJ2Tmc5OFIybkI0UkFrc3FUcmFrY0NRU2orM08yUUpjTm94?=
 =?utf-8?B?R20zSEN3NGo3bVpLSVVxSzRHTnBYeUcwYlp0dlI1WHpIWUc4UjBFY1BZMDVN?=
 =?utf-8?B?WlU1SmRsQW54eEFuU1BsaUIvY0RMVFgrWTJnY3RpYUF3U1A4ZnFJUDBpTG9N?=
 =?utf-8?B?c09uelNGTFBDQ0tQZ3VtT1djSnBBejV4eTdnYzhkWVFWZnZrRGNtaUdITTh2?=
 =?utf-8?B?Nll4Nm44bDc3ckZ1Vm1JaGljbkM3L2RLSGI2UnFVUnFxWFJDb2krMTJLRjFk?=
 =?utf-8?B?Y09BWEN3cmEydVpENDdlRUEycU0vckd6cGhOOXRDNnNxa1RZa0FlOHVWdVhW?=
 =?utf-8?B?alJZOGorRnlJM1FVSFlqZHN2T1EvdGN5WUVHNFN1d3Q2N2hXbTZJTk5kRERq?=
 =?utf-8?B?VXlBcjlkenJod3RvcFlsSHRHOW9WVjl4OHNvSzh3bHo4aldmODJYajcySVNr?=
 =?utf-8?B?ZUZMVWY0K000NU1UY3dzcHNvcFlTRVZFOUNRSytIWG55K0VodzVZWHlkQUJp?=
 =?utf-8?B?cWlmMURZU3EvLzlYZzljS1IwKzlpNWVEc21ocDhQSnpqZWoxb01QYi9kMVUx?=
 =?utf-8?B?YlE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: gm1ANUXLfT0g6FAxfbOfVZn4ggbknMSDQJXFDz/GUmmtZT5N8dGTeKuOd9cEZ8ns/0J6W5QhjHdqSWpDbe3OXN2XY7b9XydIr3gOACj0HGKtala8+HAL4J+k7UeDnJ3LYyIlyEAbh2UaKO7MTcQMXbjK81RVBAdYEHioBa8F26ebEO0Pg1LNvMAW0f4U0SkunbJRfn5e72trRIkdtvC73VV91A198JMIAI3eXxEceQW0CQxCC35vF8wwGe+VTTYVMM91+myrnSc7yXl3QCbILl/LAgsYOagaMDzY9fpG72EWfBF/ZjpgN4DY1RJMJKUYIaezijAwIbU2I5Nugfo/6wIla1MRFz55fsQlCKv0ahXQU/77uUD/dEk+nanka+b3jf/u9kIKsk4dGPaBlFiW6n2KSabjkD2PRlY7tIvH6hkJAZYQb75Ka8GSCnSP9W49Y2X57ZNAG9OGjCuES07dp7oJp5qNFPci6MS28DQKs3FpsqEdT/jI0zDhP5zRz722h/nQb5Z4TU9BnVm8h8/1vXVZmE0Cu5OXG1V/crFYIpuu3CqxYmYOpm86q4fYuYoXF6DKp2V0csVybySxNhJAE7CLMDhxWTX0eyBwXuVWh12YHtKlef3wuRvb9IuaOHx9sglSOGpx/+x9DaZzTnaFQTk2b3JQV7vRBut1QUgtDwxZO/wlEwkifwBEYyExlmCcya737/ddg8neVuEJ6XsG6pQb0Ve8/8OGB7/D1n/fmClwXh55nyuAANgt1pK6wnaG0XITHH0wln+VEhdgBRbqJsHKGVCX3u+KRW/FoRKX/E2rZSRkyrcapEr1+26WCjCNo3jcoCJmV5WBFlnasycraie7XRnd7UoYnbW7Ru6IXwg=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bfa9ba93-0160-428c-ed2e-08db56ebee7d
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2023 15:32:32.3653
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DxusfpI8hn6fsbeBbCw/ZxksBk6gfBWlDnKHaPe3OnziAEUoWWmbgUZ+QLI5f6sIxkMJYQzEQNQnwCZPbZRKFYaVB2E4YNKfuph+AESD0tU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR10MB6246
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-05-17_02,2023-05-17_02,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0
 mlxscore=0 phishscore=0 malwarescore=0 spamscore=0 mlxlogscore=750
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000
 definitions=main-2305170125
X-Proofpoint-GUID: rxH0MPrvEsMoz9wjsc_qaAf9ATlwVPRg
X-Proofpoint-ORIG-GUID: rxH0MPrvEsMoz9wjsc_qaAf9ATlwVPRg
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 17/05/2023 13:04, dengqiao.joey wrote:
> This seems to be a different issue. Joao's patch tries to fix the issue
> that IRTE is not changed. The problem I encountered is that IRTE does
> get changed, thus the destination field is also cleared by
> amd_iommu_activate_guest_mode(). 

Whether the amd_iommu_activate_guest_mode() clears the destination field or not
doesn't change the situation I think. So I don't think that is the problem
you're having. amd_iommu_activate_guest_mode() will always make IRTEs with
isRun=0. Which means your VF interrupts get injected via GALog events and the
DestID is meaningless there :/ Even if you leave the old affinity there with
isRun=1 bit it's still wrong as by the time you run the vcpu, the wrong affinity
will be used by the VF. More fundamentally I am not sure that on IRTE routing
updates in the VM that you can look on vcpu->cpu the way you are using it could
be potentially invalid when vcpus are blocked/preempted. You're changing IRTEs
in the whole VM in pi_update_irte()

My changes essentially handle the fact where IRTEs will be always updated with
the right GATag set in the IRTE ... and then when the vCPU migrates, wakes or
block+unblocks it will change IRTE[DestID] with the new pcpu (which the vcpu is
running on) via amd_iommu_update_ga(). So there's a good chance my changes fixes
your affinity issue.

	Joao