Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 13 Apr 2021 12:03:24 -0700
From:   Jakub Kicinski <kuba@kernel.org>
To:     Dexuan Cui <decui@microsoft.com>
Cc:     davem@davemloft.net, kys@microsoft.com, haiyangz@microsoft.com,
        sthemmin@microsoft.com, wei.liu@kernel.org, liuwe@microsoft.com,
        netdev@vger.kernel.org, leon@kernel.org, andrew@lunn.ch,
        bernd@petrovitsch.priv.at, rdunlap@infradead.org,
        shacharr@microsoft.com, linux-kernel@vger.kernel.org,
        linux-hyperv@vger.kernel.org
Subject: Re: [PATCH v5 net-next] net: mana: Add a driver for Microsoft Azure
 Network Adapter (MANA)
Message-ID: <20210413120324.18983187@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
In-Reply-To: <20210413023509.51952-1-decui@microsoft.com>
References: <20210413023509.51952-1-decui@microsoft.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Mon, 12 Apr 2021 19:35:09 -0700 Dexuan Cui wrote:
> +	apc->port_st_save = apc->port_is_up;
> +	apc->port_is_up = false;
> +	apc->start_remove = true;
> +
> +	/* Ensure port state updated before txq state */
> +	smp_wmb();
> +
> +	netif_tx_disable(ndev);

In your napi poll method there is no barrier between port_is_up check
and netif_tx_queue_stopped().

> +	netif_carrier_off(ndev);
> +
> +	/* No packet can be transmitted now since apc->port_is_up is false.
> +	 * There is still a tiny chance that mana_poll_tx_cq() can re-enable
> +	 * a txq because it may not timely see apc->port_is_up being cleared
> +	 * to false, but it doesn't matter since mana_start_xmit() drops any
> +	 * new packets due to apc->port_is_up being false.
> +	 *
> +	 * Drain all the in-flight TX packets
> +	 */
> +	for (i = 0; i < apc->num_queues; i++) {
> +		txq = &apc->tx_qp[i].txq;
> +
> +		while (atomic_read(&txq->pending_sends) > 0)
> +			usleep_range(1000, 2000);
> +	}

> +		/* All cleanup actions should stay after rtnl_lock(), otherwise
> +		 * other functions may access partially cleaned up data.
> +		 */
> +		rtnl_lock();
> +
> +		mana_detach(ndev);
> +
> +		unregister_netdevice(ndev);
> +
> +		rtnl_unlock();

I find the resource management somewhat strange. Why is mana_attach()
and mana_detach() called at probe/remove time, and not when the
interface is brought up? Presumably when the user ifdowns the interface
there is no point holding the resources? Your open/close methods are
rather empty.

> +	if ((eq_addr & PAGE_MASK) != eq_addr)
> +		return -EINVAL;
> +
> +	if ((cq_addr & PAGE_MASK) != cq_addr)
> +		return -EINVAL;
> +
> +	if ((rq_addr & PAGE_MASK) != rq_addr)
> +		return -EINVAL;
> +
> +	if ((sq_addr & PAGE_MASK) != sq_addr)
> +		return -EINVAL;

PAGE_ALIGNED()