Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1751535AbdFGJeF convert rfc822-to-8bit (ORCPT <rfc822;w@1wt.eu>);
        Wed, 7 Jun 2017 05:34:05 -0400
Received: from szxga01-in.huawei.com ([45.249.212.187]:7766 "EHLO
        szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1751066AbdFGJeC (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 7 Jun 2017 05:34:02 -0400
From: "Gonglei (Arei)" <arei.gonglei@huawei.com>
To: Paolo Bonzini <pbonzini@redhat.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>
CC: longpeng <longpeng2@huawei.com>,
        "Huangweidong (C)" <weidong.huang@huawei.com>,
        "wangxin (U)" <wangxinxin.wang@huawei.com>,
        =?iso-8859-2?Q?Radim_Kr=E8m=E1=F8?= <rkrcmar@redhat.com>
Subject: RE: [PATCH CFT 0/4] VT-d PI fixes
Thread-Topic: [PATCH CFT 0/4] VT-d PI fixes
Thread-Index: AQHS3rOqudfRg7dR/kSa5rW+/pHCY6IZII3w
Date: Wed, 7 Jun 2017 09:33:20 +0000
Message-ID: <33183CC9F5247A488A2544077AF19020DA2C98CC@DGGEMA505-MBX.china.huawei.com>
References: <20170606105707.23207-1-pbonzini@redhat.com>
In-Reply-To: <20170606105707.23207-1-pbonzini@redhat.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.177.18.62]
Content-Type: text/plain; charset="iso-8859-2"
Content-Transfer-Encoding: 8BIT
MIME-Version: 1.0
X-CFilter-Loop: Reflected
X-Mirapoint-Virus-RAPID-Raw: score=unknown(0),
        refid=str=0001.0A020201.5937C872.0142,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0,
        ip=169.254.1.179,
        so=2014-11-16 11:51:01,
        dmn=2013-03-21 17:37:32
X-Mirapoint-Loop-Id: 3a9fef778d3194747ea4c8e8fbf40c57
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 2490
Lines: 57


> -----Original Message-----
> From: Paolo Bonzini [mailto:paolo.bonzini@gmail.com] On Behalf Of Paolo
> Bonzini
> Sent: Tuesday, June 06, 2017 6:57 PM
> To: linux-kernel@vger.kernel.org; kvm@vger.kernel.org
> Cc: longpeng; Huangweidong (C); Gonglei (Arei); wangxin (U); Radim Kr?m??
> Subject: [PATCH CFT 0/4] VT-d PI fixes
> 
> These should fix, or at least help, the kernel panic reported by Longpeng
> with VT-d posted interrupts.
> 
> CONFIG_DEBUG_LIST reports a double add, meaning that pi_pre_block ran
> twice
> without pi_post_block deleting the vCPU from the blocked_on_vcpu list.
> The only possibility that I could think of is that this:
> 
>         if (!kvm_arch_has_assigned_device(vcpu->kvm) ||
>                 !irq_remapping_cap(IRQ_POSTING_CAP)  ||
>                 !kvm_vcpu_apicv_active(vcpu))
>                 return;
> 
> was false in pi_post_block.  In turn, I can only think of hot-unplug as
> the cause of this imbalance, but maybe there is another way to reach it
> just via repeated startup and shutdown.  Gonglei reported problems with
> hot-unplug offlist too, so this is a start.
> 
> In any case, patch 2 replaces it with a check on vcpu->pre_pcpu.
> A similar change is done in patch 3 to vmx_vcpu_pi_load.  I don't
> have hardware easily accessible with VT-d PI, so these patches are
> compile-tested only.  I apologize for any stupid mistakes.
> 
Hi Paolo,

We are testing your patch, but maybe need some time to report
the results because it's not an inevitable problem.

Meanwhile we also try to find a possible scenario of non-hotplugging to
explain the double-add warnings.

We found that some other VMs start failed before the kernel painc:

 2017-06-02T12:27:49.972583Z qemu-kvm: -device vfio-pci,host=0b:10.4,id=hostdev0,bus=pci.0,addr=0x5: vfio: error getting device 0000:
0b:10.4 from group 97: No such device
Verify all devices in group 97 are bound to vfio-<bus> or pci-stub and not already in use
2017-06-02T12:27:49.975925Z qemu-kvm: -device vfio-pci,host=0b:10.4,id=hostdev0,bus=pci.0,addr=0x5: vfio: failed to get device 0000:
0b:10.4
2017-06-02T12:27:51.246385Z qemu-kvm: -device vfio-pci,host=0b:10.4,id=hostdev0,bus=pci.0,addr=0x5: Device initialization failed
2017-06-02 12:27:53.628: shutting down, reason=crashed
2017-06-02 12:30:48.723: shutting down, reason=failed

But we don't think those failure will cause the unequal of kvm->arch.assigned_device_count
between pi_pre_block and pi_post_block. Am I right?

Thanks,
-Gonglei