Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
To:     "mika.westerberg@linux.intel.com" <mika.westerberg@linux.intel.com>
CC:     "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Wangzhou (B)" <wangzhou1@hisilicon.com>,
        Linuxarm <linuxarm@huawei.com>, Lukas Wunner <lukas@wunner.de>
Subject: RE: Qemu Guest kernel 4.20-rc1 PCIe hotplug issue
Thread-Topic: Qemu Guest kernel 4.20-rc1 PCIe hotplug issue
Thread-Index: AdR7RP5EYT33x2cVTRiE4AudJSgcHP///SQA///uY1CAABsPAP//6WQQ
Date:   Tue, 13 Nov 2018 13:28:04 +0000
Message-ID: <5FC3163CFD30C246ABAA99954A238FA8387DF51F@FRAEML521-MBX.china.huawei.com>
References: <5FC3163CFD30C246ABAA99954A238FA8387DD344@FRAEML521-MBX.china.huawei.com>
 <20181113122522.GA2500@lahna.fi.intel.com>
 <5FC3163CFD30C246ABAA99954A238FA8387DF43F@FRAEML521-MBX.china.huawei.com>
 <20181113125910.GB2500@lahna.fi.intel.com>
In-Reply-To: <20181113125910.GB2500@lahna.fi.intel.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 8BIT
MIME-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> -----Original Message-----
> From: mika.westerberg@linux.intel.com
> [mailto:mika.westerberg@linux.intel.com]
> Sent: 13 November 2018 12:59
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; Wangzhou (B)
> <wangzhou1@hisilicon.com>; Linuxarm <linuxarm@huawei.com>; Lukas
> Wunner <lukas@wunner.de>
> Subject: Re: Qemu Guest kernel 4.20-rc1 PCIe hotplug issue
> 
> On Tue, Nov 13, 2018 at 12:36:20PM +0000, Shameerali Kolothum Thodi wrote:
> > > @@ -156,9 +156,9 @@ static void pcie_do_write_cmd(struct controller
> > > *ctrl,
> > > u16 cmd,
> > >  	slot_ctrl |= (cmd & mask);
> > >  	ctrl->cmd_busy = 1;
> > >  	smp_mb();
> > > +	ctrl->slot_ctrl = slot_ctrl;
> >
> > Actually I tried this one, but it doesn't help in this case as the
> > initial
> > pcie_capability_read_word() returns the slot_ctrl without
> > PCI_EXP_SLTCTL_HPIE bit set.  It looks to me
> > pcie_enable_notification() function enables this,
> >
> > 	if (!pciehp_poll_mode)
> > 		cmd |= PCI_EXP_SLTCTL_HPIE | PCI_EXP_SLTCTL_CCIE;
> >
> > I don't know this is as per the spec or not as the initial cap read
> > doesn't seems to have the PCI_EXP_SLTCTL_HPIE bit set.
> 
> If I read the code right cmd value should end up in ctrl->slot_ctrl properly from
> pcie_enable_notification().

Right. As I mentioned in my previous mail, I missed the fact that you are updating
the ctrl->slot_ctrl with cmd value while in my test I did my update with the value
returned by pcie_capability_read_word().
 
> However, I think we are missing check for PCI_EXP_SLTCTL_CCIE in
> pciehp_isr().

Ok.
 
> Here's an updated patch, can you try and see if it makes any difference?

I just tried this and it works. Thanks.

See few comments below.

> diff --git a/drivers/pci/hotplug/pciehp_hpc.c
> b/drivers/pci/hotplug/pciehp_hpc.c
> index 7dd443aea5a5..da2cbe892444 100644
> --- a/drivers/pci/hotplug/pciehp_hpc.c
> +++ b/drivers/pci/hotplug/pciehp_hpc.c
> @@ -156,9 +156,9 @@ static void pcie_do_write_cmd(struct controller *ctrl,
> u16 cmd,
>  	slot_ctrl |= (cmd & mask);
>  	ctrl->cmd_busy = 1;
>  	smp_mb();
> +	ctrl->slot_ctrl = slot_ctrl;

Does it make more sense if we can move this before smp_mb()?. Also I am not
sure updating the  ctrl->slot_ctrl before actually the hardware is programmed
with that value will result in any other race conditions? TBH, I am not that familiar
with this code and I leave that to you :)

Thanks,
Shameer

>  	pcie_capability_write_word(pdev, PCI_EXP_SLTCTL, slot_ctrl);
>  	ctrl->cmd_started = jiffies;
> -	ctrl->slot_ctrl = slot_ctrl;
> 
>  	/*
>  	 * Controllers with the Intel CF118 and similar errata advertise @@ -
> 522,7 +522,7 @@ static irqreturn_t pciehp_isr(int irq, void *dev_id)
>  	 * in the Slot Control register (PCIe r4.0, sec 6.7.3.4).
>  	 */
>  	if (pdev->current_state == PCI_D3cold ||
> -	    (!(ctrl->slot_ctrl & PCI_EXP_SLTCTL_HPIE) && !pciehp_poll_mode))
> +	    (!(ctrl->slot_ctrl & (PCI_EXP_SLTCTL_HPIE | PCI_EXP_SLTCTL_CCIE))
> +&& !pciehp_poll_mode))
>  		return IRQ_NONE;
> 
>  	/*