Since commit 6624e780a577fc ("ice: split ice_vsi_setup into smaller
functions") ice_vsi_release does things twice. There is unregister
netdev which is unregistered in ice_deinit_eth also.
It also unregisters the devlink_port twice which is also unregistered
in ice_deinit_eth(). This double deregistration is hidden because
devl_port_unregister ignores the return value of xa_erase.
[ 68.642167] Call Trace:
[ 68.650385] ice_devlink_destroy_pf_port+0xe/0x20 [ice]
[ 68.655656] ice_vsi_release+0x445/0x690 [ice]
[ 68.660147] ice_deinit+0x99/0x280 [ice]
[ 68.664117] ice_remove+0x1b6/0x5c0 [ice]
[ 171.103841] Call Trace:
[ 171.109607] ice_devlink_destroy_pf_port+0xf/0x20 [ice]
[ 171.114841] ice_remove+0x158/0x270 [ice]
[ 171.118854] pci_device_remove+0x3b/0xc0
[ 171.122779] device_release_driver_internal+0xc7/0x170
[ 171.127912] driver_detach+0x54/0x8c
[ 171.131491] bus_remove_driver+0x77/0xd1
[ 171.135406] pci_unregister_driver+0x2d/0xb0
[ 171.139670] ice_module_exit+0xc/0x55f [ice]
Fixes: 6624e780a577 ("ice: split ice_vsi_setup into smaller functions")
Signed-off-by: Petr Oros <[email protected]>
---
v2: reword subject
v1: https://lore.kernel.org/netdev/[email protected]/
---
drivers/net/ethernet/intel/ice/ice_lib.c | 27 ------------------------
1 file changed, 27 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 11ae0e41f518a1..284a1f0bfdb545 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -3272,39 +3272,12 @@ int ice_vsi_release(struct ice_vsi *vsi)
return -ENODEV;
pf = vsi->back;
- /* do not unregister while driver is in the reset recovery pending
- * state. Since reset/rebuild happens through PF service task workqueue,
- * it's not a good idea to unregister netdev that is associated to the
- * PF that is running the work queue items currently. This is done to
- * avoid check_flush_dependency() warning on this wq
- */
- if (vsi->netdev && !ice_is_reset_in_progress(pf->state) &&
- (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state))) {
- unregister_netdev(vsi->netdev);
- clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
- }
-
- if (vsi->type == ICE_VSI_PF)
- ice_devlink_destroy_pf_port(pf);
-
if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
ice_rss_clean(vsi);
ice_vsi_close(vsi);
ice_vsi_decfg(vsi);
- if (vsi->netdev) {
- if (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state)) {
- unregister_netdev(vsi->netdev);
- clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
- }
- if (test_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state)) {
- free_netdev(vsi->netdev);
- vsi->netdev = NULL;
- clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state);
- }
- }
-
/* retain SW VSI data structure since it is needed to unregister and
* free VSI netdev when PF is not in reset recovery pending state,\
* for ex: during rmmod.
--
2.41.0
Mon, Jun 19, 2023 at 12:58:13PM CEST, [email protected] wrote:
>Since commit 6624e780a577fc ("ice: split ice_vsi_setup into smaller
>functions") ice_vsi_release does things twice. There is unregister
>netdev which is unregistered in ice_deinit_eth also.
You need to describe more relationship between ice_vsi_release() and
ice_deinit_eth(). From a quick look, I don't see that ice_deinit_eth()
is always called before/afeter ice_vsi_release().
Ice init/release flows are very hard to follow :/
>
>It also unregisters the devlink_port twice which is also unregistered
>in ice_deinit_eth(). This double deregistration is hidden because
>devl_port_unregister ignores the return value of xa_erase.
This call for another patch, doesn't it? :)
>
>[ 68.642167] Call Trace:
>[ 68.650385] ice_devlink_destroy_pf_port+0xe/0x20 [ice]
>[ 68.655656] ice_vsi_release+0x445/0x690 [ice]
>[ 68.660147] ice_deinit+0x99/0x280 [ice]
>[ 68.664117] ice_remove+0x1b6/0x5c0 [ice]
>
>[ 171.103841] Call Trace:
>[ 171.109607] ice_devlink_destroy_pf_port+0xf/0x20 [ice]
>[ 171.114841] ice_remove+0x158/0x270 [ice]
>[ 171.118854] pci_device_remove+0x3b/0xc0
>[ 171.122779] device_release_driver_internal+0xc7/0x170
>[ 171.127912] driver_detach+0x54/0x8c
>[ 171.131491] bus_remove_driver+0x77/0xd1
>[ 171.135406] pci_unregister_driver+0x2d/0xb0
>[ 171.139670] ice_module_exit+0xc/0x55f [ice]
>
>Fixes: 6624e780a577 ("ice: split ice_vsi_setup into smaller functions")
>Signed-off-by: Petr Oros <[email protected]>
>---
>v2: reword subject
>
>v1: https://lore.kernel.org/netdev/[email protected]/
>---
> drivers/net/ethernet/intel/ice/ice_lib.c | 27 ------------------------
> 1 file changed, 27 deletions(-)
>
>diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
>index 11ae0e41f518a1..284a1f0bfdb545 100644
>--- a/drivers/net/ethernet/intel/ice/ice_lib.c
>+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
>@@ -3272,39 +3272,12 @@ int ice_vsi_release(struct ice_vsi *vsi)
> return -ENODEV;
> pf = vsi->back;
>
>- /* do not unregister while driver is in the reset recovery pending
>- * state. Since reset/rebuild happens through PF service task workqueue,
>- * it's not a good idea to unregister netdev that is associated to the
>- * PF that is running the work queue items currently. This is done to
>- * avoid check_flush_dependency() warning on this wq
>- */
>- if (vsi->netdev && !ice_is_reset_in_progress(pf->state) &&
>- (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state))) {
>- unregister_netdev(vsi->netdev);
>- clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
>- }
>-
>- if (vsi->type == ICE_VSI_PF)
>- ice_devlink_destroy_pf_port(pf);
>-
> if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
> ice_rss_clean(vsi);
>
> ice_vsi_close(vsi);
> ice_vsi_decfg(vsi);
>
>- if (vsi->netdev) {
>- if (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state)) {
>- unregister_netdev(vsi->netdev);
>- clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
>- }
>- if (test_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state)) {
>- free_netdev(vsi->netdev);
>- vsi->netdev = NULL;
>- clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state);
>- }
>- }
>-
> /* retain SW VSI data structure since it is needed to unregister and
> * free VSI netdev when PF is not in reset recovery pending state,\
> * for ex: during rmmod.
>--
>2.41.0
>
>
On Mon, Jun 19, 2023 at 12:58:13PM +0200, Petr Oros wrote:
> Since commit 6624e780a577fc ("ice: split ice_vsi_setup into smaller
> functions") ice_vsi_release does things twice. There is unregister
> netdev which is unregistered in ice_deinit_eth also.
>
> It also unregisters the devlink_port twice which is also unregistered
> in ice_deinit_eth(). This double deregistration is hidden because
> devl_port_unregister ignores the return value of xa_erase.
>
> [ 68.642167] Call Trace:
> [ 68.650385] ice_devlink_destroy_pf_port+0xe/0x20 [ice]
> [ 68.655656] ice_vsi_release+0x445/0x690 [ice]
> [ 68.660147] ice_deinit+0x99/0x280 [ice]
> [ 68.664117] ice_remove+0x1b6/0x5c0 [ice]
>
> [ 171.103841] Call Trace:
> [ 171.109607] ice_devlink_destroy_pf_port+0xf/0x20 [ice]
> [ 171.114841] ice_remove+0x158/0x270 [ice]
> [ 171.118854] pci_device_remove+0x3b/0xc0
> [ 171.122779] device_release_driver_internal+0xc7/0x170
> [ 171.127912] driver_detach+0x54/0x8c
> [ 171.131491] bus_remove_driver+0x77/0xd1
> [ 171.135406] pci_unregister_driver+0x2d/0xb0
> [ 171.139670] ice_module_exit+0xc/0x55f [ice]
>
> Fixes: 6624e780a577 ("ice: split ice_vsi_setup into smaller functions")
> Signed-off-by: Petr Oros <[email protected]>
> ---
> v2: reword subject
>
> v1: https://lore.kernel.org/netdev/[email protected]/
> ---
> drivers/net/ethernet/intel/ice/ice_lib.c | 27 ------------------------
> 1 file changed, 27 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
> index 11ae0e41f518a1..284a1f0bfdb545 100644
> --- a/drivers/net/ethernet/intel/ice/ice_lib.c
> +++ b/drivers/net/ethernet/intel/ice/ice_lib.c
> @@ -3272,39 +3272,12 @@ int ice_vsi_release(struct ice_vsi *vsi)
> return -ENODEV;
> pf = vsi->back;
>
> - /* do not unregister while driver is in the reset recovery pending
> - * state. Since reset/rebuild happens through PF service task workqueue,
> - * it's not a good idea to unregister netdev that is associated to the
> - * PF that is running the work queue items currently. This is done to
> - * avoid check_flush_dependency() warning on this wq
> - */
> - if (vsi->netdev && !ice_is_reset_in_progress(pf->state) &&
> - (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state))) {
> - unregister_netdev(vsi->netdev);
> - clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
> - }
> -
> - if (vsi->type == ICE_VSI_PF)
> - ice_devlink_destroy_pf_port(pf);
> -
> if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
> ice_rss_clean(vsi);
>
> ice_vsi_close(vsi);
> ice_vsi_decfg(vsi);
>
> - if (vsi->netdev) {
> - if (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state)) {
> - unregister_netdev(vsi->netdev);
> - clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
> - }
> - if (test_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state)) {
> - free_netdev(vsi->netdev);
> - vsi->netdev = NULL;
> - clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state);
> - }
> - }
Hmm it's a bit messy:D
I agree that ice_deinit_eth() should be the one that takes care of netdev
clean up as ice_init_eth() was the one to alloc and register it. I believe
that part of the split up work was to come up with entities that are
scoped to alloc/dealloc related resources...as this was even more messed
up before.
It would be worth to exercise other code paths where ice_vsi_release() is
used - like do a loopback test (ethtool -t $IFACE) or go through a reset.
Reviewed-by: Maciej Fijalkowski <[email protected]>
> -
> /* retain SW VSI data structure since it is needed to unregister and
> * free VSI netdev when PF is not in reset recovery pending state,\
> * for ex: during rmmod.
> --
> 2.41.0
>
> _______________________________________________
> Intel-wired-lan mailing list
> [email protected]
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
On Mon, Jun 19, 2023 at 05:07:21PM +0200, Jiri Pirko wrote:
> Mon, Jun 19, 2023 at 12:58:13PM CEST, [email protected] wrote:
> >Since commit 6624e780a577fc ("ice: split ice_vsi_setup into smaller
> >functions") ice_vsi_release does things twice. There is unregister
> >netdev which is unregistered in ice_deinit_eth also.
>
> You need to describe more relationship between ice_vsi_release() and
> ice_deinit_eth(). From a quick look, I don't see that ice_deinit_eth()
> is always called before/afeter ice_vsi_release().
>
Only relation between them is that they both are called on .remove() pci
callback. Clearly ice_vsi_release() should not be cleaning up stuff that
ice_init_eth() has brought up. I was staring at the code and jumping
around these flows and fix seems reasonable.
> Ice init/release flows are very hard to follow :/
>
>
> >
> >It also unregisters the devlink_port twice which is also unregistered
> >in ice_deinit_eth(). This double deregistration is hidden because
> >devl_port_unregister ignores the return value of xa_erase.
>
> This call for another patch, doesn't it? :)
+1 :)
>
>
> >
> >[ 68.642167] Call Trace:
> >[ 68.650385] ice_devlink_destroy_pf_port+0xe/0x20 [ice]
> >[ 68.655656] ice_vsi_release+0x445/0x690 [ice]
> >[ 68.660147] ice_deinit+0x99/0x280 [ice]
> >[ 68.664117] ice_remove+0x1b6/0x5c0 [ice]
> >
> >[ 171.103841] Call Trace:
> >[ 171.109607] ice_devlink_destroy_pf_port+0xf/0x20 [ice]
> >[ 171.114841] ice_remove+0x158/0x270 [ice]
> >[ 171.118854] pci_device_remove+0x3b/0xc0
> >[ 171.122779] device_release_driver_internal+0xc7/0x170
> >[ 171.127912] driver_detach+0x54/0x8c
> >[ 171.131491] bus_remove_driver+0x77/0xd1
> >[ 171.135406] pci_unregister_driver+0x2d/0xb0
> >[ 171.139670] ice_module_exit+0xc/0x55f [ice]
> >
> >Fixes: 6624e780a577 ("ice: split ice_vsi_setup into smaller functions")
> >Signed-off-by: Petr Oros <[email protected]>
> >---
> >v2: reword subject
> >
> >v1: https://lore.kernel.org/netdev/[email protected]/
> >---
> > drivers/net/ethernet/intel/ice/ice_lib.c | 27 ------------------------
> > 1 file changed, 27 deletions(-)
> >
> >diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
> >index 11ae0e41f518a1..284a1f0bfdb545 100644
> >--- a/drivers/net/ethernet/intel/ice/ice_lib.c
> >+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
> >@@ -3272,39 +3272,12 @@ int ice_vsi_release(struct ice_vsi *vsi)
> > return -ENODEV;
> > pf = vsi->back;
> >
> >- /* do not unregister while driver is in the reset recovery pending
> >- * state. Since reset/rebuild happens through PF service task workqueue,
> >- * it's not a good idea to unregister netdev that is associated to the
> >- * PF that is running the work queue items currently. This is done to
> >- * avoid check_flush_dependency() warning on this wq
> >- */
> >- if (vsi->netdev && !ice_is_reset_in_progress(pf->state) &&
> >- (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state))) {
> >- unregister_netdev(vsi->netdev);
> >- clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
> >- }
> >-
> >- if (vsi->type == ICE_VSI_PF)
> >- ice_devlink_destroy_pf_port(pf);
> >-
> > if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
> > ice_rss_clean(vsi);
> >
> > ice_vsi_close(vsi);
> > ice_vsi_decfg(vsi);
> >
> >- if (vsi->netdev) {
> >- if (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state)) {
> >- unregister_netdev(vsi->netdev);
> >- clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state);
> >- }
> >- if (test_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state)) {
> >- free_netdev(vsi->netdev);
> >- vsi->netdev = NULL;
> >- clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state);
> >- }
> >- }
> >-
> > /* retain SW VSI data structure since it is needed to unregister and
> > * free VSI netdev when PF is not in reset recovery pending state,\
> > * for ex: during rmmod.
> >--
> >2.41.0
> >
> >
>
Petr Oros píše v Po 19. 06. 2023 v 12:58 +0200:
> Since commit 6624e780a577fc ("ice: split ice_vsi_setup into smaller
> functions") ice_vsi_release does things twice. There is unregister
> netdev which is unregistered in ice_deinit_eth also.
>
> It also unregisters the devlink_port twice which is also unregistered
> in ice_deinit_eth(). This double deregistration is hidden because
> devl_port_unregister ignores the return value of xa_erase.
>
Hi,
Is it possible to push this patch forward? I think the questions have
been answered. Or is something still unclear?
Regards,
Petr
On 7/6/2023 2:30 PM, Petr Oros wrote:
> Petr Oros píše v Po 19. 06. 2023 v 12:58 +0200:
>> Since commit 6624e780a577fc ("ice: split ice_vsi_setup into smaller
>> functions") ice_vsi_release does things twice. There is unregister
>> netdev which is unregistered in ice_deinit_eth also.
>>
>> It also unregisters the devlink_port twice which is also unregistered
>> in ice_deinit_eth(). This double deregistration is hidden because
>> devl_port_unregister ignores the return value of xa_erase.
>>
> Hi,
>
> Is it possible to push this patch forward? I think the questions have
> been answered. Or is something still unclear?
I was expecting a new version based on the question/request [1] (so set
to changes requested) before the response. Will apply and have it move
forward.
Thanks,
Tony
[1]
https://lore.kernel.org/intel-wired-lan/[email protected]/T/#mbef0a70424d01bb8fcc1ae1a4865c617fbb4befc
> -----Original Message-----
> From: Intel-wired-lan <[email protected]> On Behalf Of Petr Oros
> Sent: Monday, June 19, 2023 4:28 PM
> To: [email protected]
> Cc: [email protected]; [email protected]; Brandeburg, Jesse <[email protected]>; [email protected]; [email protected]; Nguyen, Anthony L <[email protected]>; [email protected]; [email protected]; [email protected]
> Subject: [Intel-wired-lan] [PATCH net v2] ice: Unregister netdev and devlink_port only once
>
> Since commit 6624e780a577fc ("ice: split ice_vsi_setup into smaller
> functions") ice_vsi_release does things twice. There is unregister netdev which is unregistered in ice_deinit_eth also.
>
> It also unregisters the devlink_port twice which is also unregistered in ice_deinit_eth(). This double deregistration is hidden because devl_port_unregister ignores the return value of xa_erase.
>
> [ 68.642167] Call Trace:
> [ 68.650385] ice_devlink_destroy_pf_port+0xe/0x20 [ice]
> [ 68.655656] ice_vsi_release+0x445/0x690 [ice]
> [ 68.660147] ice_deinit+0x99/0x280 [ice]
> [ 68.664117] ice_remove+0x1b6/0x5c0 [ice]
>
> [ 171.103841] Call Trace:
> [ 171.109607] ice_devlink_destroy_pf_port+0xf/0x20 [ice] [ 171.114841] ice_remove+0x158/0x270 [ice] [ 171.118854] pci_device_remove+0x3b/0xc0 [ 171.122779] device_release_driver_internal+0xc7/0x170
> [ 171.127912] driver_detach+0x54/0x8c
> [ 171.131491] bus_remove_driver+0x77/0xd1 [ 171.135406] pci_unregister_driver+0x2d/0xb0 [ 171.139670] ice_module_exit+0xc/0x55f [ice]
>
> Fixes: 6624e780a577 ("ice: split ice_vsi_setup into smaller functions")
> Signed-off-by: Petr Oros <[email protected]>
> ---
> v2: reword subject
>
> v1: https://lore.kernel.org/netdev/[email protected]/
> ---
> drivers/net/ethernet/intel/ice/ice_lib.c | 27 ------------------------
> 1 file changed, 27 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <[email protected]> (A Contingent worker at Intel)