Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <cc84101a-b6d1-c33d-d7a8-115f846c358c@intel.com>
Date:   Tue, 4 Apr 2023 13:46:01 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.9.1
Subject: Re: [PATCH V2 7/8] vfio/pci: Support dynamic MSI-x
Content-Language: en-US
To:     Alex Williamson <alex.williamson@redhat.com>
CC:     "Tian, Kevin" <kevin.tian@intel.com>,
        "jgg@nvidia.com" <jgg@nvidia.com>,
        "yishaih@nvidia.com" <yishaih@nvidia.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "darwi@linutronix.de" <darwi@linutronix.de>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "Liu, Jing2" <jing2.liu@intel.com>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        "tom.zanussi@linux.intel.com" <tom.zanussi@linux.intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <cover.1680038771.git.reinette.chatre@intel.com>
 <419f3ba2f732154d8ae079b3deb02d0fdbe3e258.1680038771.git.reinette.chatre@intel.com>
 <20230330164050.0069e2a5.alex.williamson@redhat.com>
 <20230330164214.67ccbdfa.alex.williamson@redhat.com>
 <688393bf-445c-15c5-e84d-1c16261a4197@intel.com>
 <20230331162456.3f52b9e3.alex.williamson@redhat.com>
 <e15d588e-b63f-ab70-f6ae-91ceea8be79a@intel.com>
 <20230403142227.1328b373.alex.williamson@redhat.com>
 <57a8c701-bf97-fddd-9ac0-fc4d09e3cb16@intel.com>
 <20230403211841.0e206b67.alex.williamson@redhat.com>
 <BN9PR11MB527626CAE4BA7ECB64F0E9728C939@BN9PR11MB5276.namprd11.prod.outlook.com>
 <ad8c9137-bd57-4862-46c8-2c77a21b3419@intel.com>
 <20230404124334.45cddae2.alex.williamson@redhat.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20230404124334.45cddae2.alex.williamson@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VldnY2RJTS9nSzUwNEFSWXlncFIwYlFUdTV4cXZjYm5yc3RsaXZBUGVsLzlv?=
 =?utf-8?B?MUtKTkNNUyszWFNUTkNGTWgzeFBHRXdzR0E5RjVhaGpRalFiM3ZrK3RwZFR2?=
 =?utf-8?B?NVM0M1dudzQxSlpwRXdCN1ZZeDdSaEJudFVTQldwRGtCS2t0QTE1Z1R4OUtk?=
 =?utf-8?B?YVF3amY4L0ZkWEtmbXRTZTN6eHM4cG14WnpwS1FmcGtqYjZHTktEejlUcmU4?=
 =?utf-8?B?bktUSlFWelFLOWVQaXpJVmNtenFTU2Z0V05kNkZ4M3RvN2dIOEwyaWp6UnUy?=
 =?utf-8?B?ajJxU25QRWg4aVVJak5lWDlueVJKakRINjBFaVVqaU80QTlFKytFdFlsS0tE?=
 =?utf-8?B?MUR3UFRzWDkvaUd6b3FaUXJpYS93Q3R3RjNCa0NKTXZqb2Q1MVUvQUVWVCtk?=
 =?utf-8?B?RC9DT3BoUTNKSGphZ2FtQ09JOXAvbW5OQXV5MmNmZStzaE4zMmlSU2tkOXVn?=
 =?utf-8?B?TEUrNFpYaTY4Q0RST3ZCeE03SWs0Z2hQemQ2YzRBNDM3TGRBRWNXTVk5aERE?=
 =?utf-8?B?TllvbU11VjAzaCtTeXZrY2RYMVFGNHhuczNXMFMxajFhaEVpRXNneE04aFRD?=
 =?utf-8?B?Qk5EOWtCV05Mdmpna0hMTUVBRXQrZnUwVFFlcGxqTDI5UWtrM0dScnNyUDk4?=
 =?utf-8?B?dzNjWmpvL1V6U1BtaFREaDZiVjNLNERqL2lSRFkxakZ0UThHSm1YNnZodkQx?=
 =?utf-8?B?elRmVHFNaE9kWVZJcythWUNhTjkyQnNtdlB4R0E4Z0FiNkMrbkRQbGlwRXdO?=
 =?utf-8?B?MC9FK0crbVB6Zng1NTNtSWZETEFrOXY4TUp5RmNIQ0JMdThzejRQK0IvdUI0?=
 =?utf-8?B?NE9WOUZTY3RQckhsOUc0Q1VDdFNxL2FoNzFnb3FSOXhaTFNnRVJzUnJJOXlE?=
 =?utf-8?B?cjZEeDNORzc5VUM2VHhFeGhISENlcjJEbldiUEN5YlFIZmtzWHVBbjNCYzIr?=
 =?utf-8?B?UjlqQVZua243THdHbTBoaHlRdkRobnJtWUpqSDBSWmVxNXh3V2dTWUkxYnlF?=
 =?utf-8?B?Z3hYWWxLZy84Y0I5bUt2SmlOTHJWV0ZZM0R4SlhMT3A4OUlEZzIxaUZodkQx?=
 =?utf-8?B?bHM2TzZQMGgzMVlDMWNPRHJJbHlGK0hXYnUwQ2tVamEzRXZrQnRQUGZFQzZp?=
 =?utf-8?B?b1B6ak5OSFdpQ0xDMURpc01kUVVBQ3JCUEZpbUp3WDJZRVN3cTJOV3pPMmRq?=
 =?utf-8?B?U0NydnBBZFdvMzRLa1R3WGhKL3VENXB2eWdUNHlxY2hIMVJHbElFM0VNN1I1?=
 =?utf-8?B?VUQrVU8vSE5jOGUxeTZuMUtUNWRtK2lNaGI1bjRmZytNaDg5a2VvNnNYVmdT?=
 =?utf-8?B?Skd5QVQzWDI4SG81QmZ2RENPUCtaZWhmeDN4d0IxT0tlVTUwS056cGI1REJQ?=
 =?utf-8?B?UExRWTBCU3JVQTJkNmVyNjR0Sldrb1k3M2hISkpwcm16SzVmMml5ZTVaNmVT?=
 =?utf-8?B?TkFCbFBLeGZMQmdHSHg0ZlRSU2J4bW1kK2ZKQktUM1FFazJvZlVmZk55d3VN?=
 =?utf-8?B?aWg1MFJqUkxiY3VoTVRrZlA4L1NJaDhWaFVjQU5HSGc2eFVxc2wwaCtJeUhp?=
 =?utf-8?B?RllhUGYwWVl4U2NSbktkU0VnOHBFZFNQMU9YL0w2WHlkZURxNGJXL3UrazEz?=
 =?utf-8?B?MnNoWmJ4L1Bqd0ZXUG5Xbm5qQ3lLbGhYVlMzZzhrQWZpWWFEVVdtcHJXYW5w?=
 =?utf-8?B?VXAzSUVGT2huMHVYT1RhUld5a2czem5XWmIxR3E3N1Y1QUVma2Jvc1lIZkla?=
 =?utf-8?B?cXE1eStiT2Y2TnUvblhJbEp5SkcyQmtuc0U3Rlo5OUMrQlR3M29WMWxyVVQ5?=
 =?utf-8?B?dkVhR24xSDl1emExY1BOYWhvL3piQzJ0TzVKVmlueFhINmFhN2RCcjVKQ2Y0?=
 =?utf-8?B?ODJML0REWmtONFE1RHpMY1djdjRueHlqVy80eENrM3lVeXhqeDFqOElHcmNV?=
 =?utf-8?B?aFpBb1loUi9ySEJqcWNXMW81YVdVNEFPOExlcDY5NENFWldBUkdyYlUwNjQw?=
 =?utf-8?B?SjV1UzJ6a3plZE9RbStoYjY5ZldnamVxazduNk9ZTFN5aG0zbG4ybXcveU9M?=
 =?utf-8?B?VzhucU1USFJ4YWs4N1RwTWI1RXlKcVJWeHEwUkxmdGpTdzdsYjBVTXF5cjFD?=
 =?utf-8?B?ai9iTEdqR0h2RTRYMzJ3aTNxbVJ5VzcwSEJEWWYwZDdrQWQvRnR1dWg3M0ty?=
 =?utf-8?B?bHc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3472c404-b745-4bf5-9fc1-08db354d9b81
X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1862.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Apr 2023 20:46:04.6065
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: T5IrJr76LX26jd72mZxEwbY7rM14ld91r4im53fHM1kVebiNSQnjqqYfFdhm2faGn8Q+ISR88UHhSkTw/kpNj+pUTEaxro4YWbK8QGbkDro=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4737
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Alex,

On 4/4/2023 11:43 AM, Alex Williamson wrote:
> 
> I think Kevin has a point, if it's correct that we do this get/write in
> order to account for manipulation of the device since we wrote into the
> vector table via either pci_alloc_irq_vectors() or
> pci_msix_alloc_irq_at(), then it really only makes sense to do that
> restore if we haven't allocated the irq and written the vector table
> immediately prior.  Thanks,
> 

I see. Even so, could it be acceptable to call get_cached_msi_msg()/pci_write_msi_msg()
unconditionally (for MSI-X) or should the new [1] vfio_pci_alloc_irq()
indicate if a new IRQ was allocated to determine if 
get_cached_msi_msg()/pci_write_msi_msg() should be run?

Reinette


[1] https://lore.kernel.org/lkml/20230403211841.0e206b67.alex.williamson@redhat.com/