From: Dexuan Cui <decui@microsoft.com>
To: KY Srinivasan <kys@microsoft.com>,
        "davem@davemloft.net" <davem@davemloft.net>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "devel@linuxdriverproject.org" <devel@linuxdriverproject.org>,
        "olaf@aepfle.de" <olaf@aepfle.de>,
        "apw@canonical.com" <apw@canonical.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "vkuznets@redhat.com" <vkuznets@redhat.com>
Subject: RE: [PATCH net-next 1/1] hv_netvsc: Wait for sub-channels to be
 processed during probe
Thread-Topic: [PATCH net-next 1/1] hv_netvsc: Wait for sub-channels to be
 processed during probe
Thread-Index: AQHQv/BG8tKnHymnuUOSalik/ODCIp3faxCwgABgzQCABE4EYA==
Date: Mon, 20 Jul 2015 09:39:03 +0000
Message-ID: <da594b2ca2ed48f8b5119b7fb0269f2d@SIXPR30MB031.064d.mgd.msft.net>
References: <1437074222-13020-1-git-send-email-kys@microsoft.com>
 <4fd650437f864cf2a58e8334253f5dcd@SIXPR30MB031.064d.mgd.msft.net>
 <BY2PR0301MB1654D90488532EDD2318B7B3A0980@BY2PR0301MB1654.namprd03.prod.outlook.com>
In-Reply-To: <BY2PR0301MB1654D90488532EDD2318B7B3A0980@BY2PR0301MB1654.namprd03.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 8BIT
MIME-Version: 1.0
BN1PR0301MB0659: X-MS-Exchange-Organization-RulesExecuted
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jul 2015 09:39:16.5960
 (UTC)
X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=72f988bf-86f1-41af-91ab-2d7cd011db47;Ip=[206.191.229.116];Helo=[064-smtp-out.microsoft.com]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN1PR0301MB0659
X-Microsoft-Exchange-Diagnostics: 1;BN1PR0301MB0737;2:FgtBrL7nwN0mnfeemxPvDh+WX19j2eyicgLsiVsRQhCR0hXBzOtYSAkJXApc+pOO;3:QRGGZvunzVjsjQRssT+r7N2x9hW33wNlygfA51Bti+jGhxBBOGOUbTGjw2U4BK0BAnv3whl6WiFzIDHvTQsJzaUTRV0CP/GxrdPBuLB0kFKdzBlsxjz/6irP/M8lMasvvwUUk2Qa+b+1nPf+6xy469mhSdTNp+1HDOs0ee5abtb2i4Ty8+AN0a0aDTB1TusOdBc5UE51+oIfVLQHzx/W/Nsvn33Dg6NfIGp6Clf+QB4UuBiX9MYl9dkl0zXoTyiD;25:GIap55rXSCCDPqMswp/fSJ7ECZj8VX5kepHrFz/pCDWFt6TmAfuE5eUUUS5mIlJkwhjx6fNZuREQUlA2zdZUdqlrcSoLafNHy7iKs9lrA+RuV2GcVAR5IkP7vOPqkEEsYMijus7CK21zpEVGXUxTZiy8RJgl6b87edS21DPEVKinonYFEw9TegUAU34wm46VRxjSBpDXKdhHPlABOkk1A3WPANTtahfD0V2G7PagTINAMntEAKILHuszl1gmz4Tck7/zaeOxYyD47P5s+n/a5g==;20:tBFykZ8DS6K6w1gyOkUpmyYEAANuF4XZtSqnxZ/f+c+LEKRe29crJ24d2ZHUj8ZA8wsc3M4p15iaDriOJnKbGw==;23:vXl+f8eNk1n9Prbe1f0FJP7LxhuUmo7eV1DFB1nKguAwS6+lJAS/R4VREoOSvMFmDOX7ZiI/Ac3DEqhVmzwdmuSHtVHe3vQZVlj1McS1E4yAPXFZNKoyOwGKSbVTk9KmJ0bXB6kAqoTv0zlR/up612Y4EyzDzP52RudbgaqA8ETuNC9g5vFjhf5ruQoF41UvNoXNq9kc7bv1J7eskzLkaSJ2KyA77MtXEo04PrhlkB8HFzBjAyzM3PtF5hkphjfL
BN1PR0301MB0737: X-MS-Exchange-Organization-RulesExecuted
X-OriginatorOrg: microsoft.com
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 3960
Lines: 108

> From: KY Srinivasan
> Sent: Friday, July 17, 2015 23:33
> > From: Dexuan Cui
> > Sent: Friday, July 17, 2015 3:01 AM
> > > From: K. Y. Srinivasan
> > > Sent: Friday, July 17, 2015 3:17
> > > Subject: [PATCH net-next 1/1] hv_netvsc: Wait for sub-channels to be
> > processed
> > > during probe
> > > diff --git a/drivers/net/hyperv/hyperv_net.h
> > b/drivers/net/hyperv/hyperv_net.h
> > > ...
> > > @@ -1116,6 +1127,9 @@ int rndis_filter_device_add(struct hv_device
> > *dev,
> > >  num_possible_rss_qs = cpumask_weight(node_cpu_mask);
> > >  net_device->num_chn = min(num_possible_rss_qs, num_rss_qs);
> > >
> > > +num_rss_qs = net_device->num_chn - 1;
> > > +net_device->num_sc_offered = num_rss_qs;
> > > +
> > >  if (net_device->num_chn == 1)
> > >  goto out;
> > >
> > > @@ -1157,11 +1171,22 @@ int rndis_filter_device_add(struct hv_device
> > *dev,
> > >
> > >  ret = rndis_filter_set_rss_param(rndis_device, net_device-
> > >num_chn);
> > >
> > > +/*
> > > + * Wait for the host to send us the sub-channel offers.
> > > + */
> > > +spin_lock_irqsave(&net_device->sc_lock, flags);
> > > +sc_delta = net_device->num_chn - 1 - num_rss_qs;
> > > +net_device->num_sc_offered -= sc_delta;
> >
> > Hi KY,
> > IMO here the "-= " should be "+="?
> >
> > I think sc_delta is usually <= 0, meaning the host may allocate less
> > subchannels than
> > we expect.
> > With "-=", net_device->num_sc_offered can become bigger -- this doesn't
> > seem correct.
> We control how many sub-channels we want the host to offer (say
> sc_requested). Based on this
> number we begin to track how many have actually been processed - we
> decrement sc_requested
> each time a sub-channel offer is processed. If the host were to actually offer all
> that we have requested,
> then checking for sc_requested to be zero is sufficient to ensure that we have
> processed all the
> potentially in-flight sub-channels. However, the host  may choose to offer less
> than what we had asked for
> and the variable "delta" is tracking this difference. Since we are counting down
> from what we had asked for
> we have to subtract "delta" for proper accounting.

Yes, I understand the rationale.
Let me show the issue by example:

Let's assume sc_requested is 7 and the host actually only offers 3 sub-channels:
1. Just before sending the NVSP_MSG5_TYPE_SUBCHANNEL message, we have
net_device->num_chn == 8,
num_rss_qs ==  7
net_device->num_sc_offered == 7

2. Just after we get the reply of the message,
net_device->num_chn == 4
sc_delta = net_device->num_chn - 1 - num_rss_qs, so sc_delta == 4 - 1 - 7 = -4
net_device->num_sc_offered -= sc_delta, so
net_device->num_sc_offered == 7 - (-4) = 11. It's not zero, so we sleep on the
wait_for_completion().

3. Now we process the 3 sub-channel and net_device->num_sc_offered will
become 11 -1 -1 -1 == 8 and no complete() will be invoked!

That's why I think the "-=" in the line
net_device->num_sc_offered -= sc_delta
should be "+=".

> > Why not use
> > "net_device->num_sc_offered = net_device->num_chn - 1;" directly?
> > At this point, net_device->num_chn has been the number of the actual
> > channels.
>
> I am not sure what the question here is. num_sc_offered is initialized to the
> number we
> are going to ask and this is the number that will be decremented each time a
> sub-channel
> is processed. Since the host may decide to offer us less than what we had asked
> and some
> sub-channels may have already been processed (num_sc_offerred decremented
> accordingly)
> by the time we discover that the host has offered us less than what we asked for,
> we adjust
> num_sc_offered accordingly.

Sorry, I had a misunderstanding here.
Please just ignore this question.

Thanks,
-- Dexuan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/