2012-02-01 17:47:35

by Rose, Gregory V

[permalink] [raw]
Subject: RE: VFs go missing with latest kernel

> -----Original Message-----
> From: Rose, Gregory V
> Sent: Tuesday, January 31, 2012 5:03 PM
> To: David Ahern
> Cc: LKML
> Subject: RE: VFs go missing with latest kernel
>
> > -----Original Message-----
> > From: David Ahern [mailto:[email protected]]
> > Sent: Tuesday, January 31, 2012 4:50 PM
> > To: Rose, Gregory V
> > Cc: LKML
> > Subject: Re: VFs go missing with latest kernel
> >
> > On 01/31/2012 05:43 PM, Rose, Gregory V wrote:
> > >>
> > >> Back to the missing VF problem: reverting the patch I mentioned
> before
> > >> and my VM boots up fine with the even-numbered VFs assigned to it.
> > >
> > > OK, thanks... I was staring at the dmesg output you sent me and it is
> > odd because the even numbered VFs that go missing are there when you set
> > their MAC addresses. See output below the dotted line after my reply.
> >
> > Ah, I think I confused you. The VFs for the first 2 ports (ie., the
> > first 14 VFs) are seen fine; I only showed the VFs for the second 2
> > ports -- which are missing the even number.
> >
> > So, in summary I am missing 7 VFs all associated with the even numbers
> > on the last ports of a quad port card.
>
> OK, thanks for the clarification. But it still shows that all 28 VFs
> exist while their MAC addresses are being set but then at some point
> subsequent to that 7 of them on the port belonging to PCI device 07:00.0
> disappear for some reason.
>
> I'll see what I can find.

I found this in the log file you sent me. I had missed it yesterday.

[ 15.835223] igb 0000:07:00.0: 7 pre-allocated VFs found - override max_vfs setting of 7
[ 15.835393] igb 0000:07:00.0: 7 VFs allocated

I think that must be a bug in the code that searches for VFs already allocated and is the source of your problem. I'll keep you updated on what I find but it has to be a bug in the VF device lookups.

- Greg


2012-02-01 17:57:59

by David Ahern

[permalink] [raw]
Subject: Re: VFs go missing with latest kernel

On 02/01/2012 10:47 AM, Rose, Gregory V wrote:
> I found this in the log file you sent me. I had missed it yesterday.
>
> [ 15.835223] igb 0000:07:00.0: 7 pre-allocated VFs found - override max_vfs setting of 7
> [ 15.835393] igb 0000:07:00.0: 7 VFs allocated
>
> I think that must be a bug in the code that searches for VFs already allocated and is the source of your problem. I'll keep you updated on what I find but it has to be a bug in the VF device lookups.

Confused. What is pre-allocating the VFs during boot? Looking at my
rc-scripts I am only set MAC addresses at boot. No VMs have been started
yet. This setup has worked fine with Fedora 14 and 3.0 kernels; it's
really the move to 3.3-rc that I hit the problem.

David

2012-02-01 18:09:36

by Rose, Gregory V

[permalink] [raw]
Subject: RE: VFs go missing with latest kernel

> -----Original Message-----
> From: David Ahern [mailto:[email protected]]
> Sent: Wednesday, February 01, 2012 9:58 AM
> To: Rose, Gregory V
> Cc: 'LKML'
> Subject: Re: VFs go missing with latest kernel
>
> On 02/01/2012 10:47 AM, Rose, Gregory V wrote:
> > I found this in the log file you sent me. I had missed it yesterday.
> >
> > [ 15.835223] igb 0000:07:00.0: 7 pre-allocated VFs found - override
> max_vfs setting of 7
> > [ 15.835393] igb 0000:07:00.0: 7 VFs allocated
> >
> > I think that must be a bug in the code that searches for VFs already
> allocated and is the source of your problem. I'll keep you updated on
> what I find but it has to be a bug in the VF device lookups.
>
> Confused. What is pre-allocating the VFs during boot? Looking at my
> rc-scripts I am only set MAC addresses at boot. No VMs have been started
> yet. This setup has worked fine with Fedora 14 and 3.0 kernels; it's
> really the move to 3.3-rc that I hit the problem.

Nothing is pre-allocating the VFs. That is the bug, the function I wrote to find pre-allocated VFs is messed up somehow. Since it (mistakenly) thinks it's already found pre-allocated VFs, it doesn't call pci_enable_sriov() to allocate them.

- Greg