MIME-Version: 1.0
In-Reply-To: <ea15b7f9-4c02-3d4e-1c01-4e2a90d2a23c@oracle.com>
References: <9f95c2a0-e4fe-270d-790a-beeb6b3e7690@oracle.com> <ea15b7f9-4c02-3d4e-1c01-4e2a90d2a23c@oracle.com>
From: Or Gerlitz <gerlitz.or@gmail.com>
Date: Wed, 13 Dec 2017 16:28:15 +0200
Message-ID: <CAJ3xEMgqmeT7tQvJZ+5Daaz2a7wsC9rUyNstP0ZoTZkr5_p1gA@mail.gmail.com>
Subject: Re: Setting large MTU size on slave interfaces may stall the whole system
To: Qing Huang <qing.huang@oracle.com>
Cc: Linux Netdev List <netdev@vger.kernel.org>,
        Linux Kernel <linux-kernel@vger.kernel.org>,
        Jay Vosburgh <j.vosburgh@gmail.com>,
        Veaceslav Falico <vfalico@gmail.com>,
        Andy Gospodarek <andy@greyhouse.net>, Aviv Heller <avivh@mellanox.com>,
        Moni Shoua <monis@mellanox.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1285
Lines: 35

On Tue, Dec 12, 2017 at 5:21 AM, Qing Huang <qing.huang@oracle.com> wrote:
> Hi,
>
> We found an issue with the bonding driver when testing Mellanox devices.
> The following test commands will stall the whole system sometimes, with
> serial console
> flooded with log messages from the bond_miimon_inspect() function. Setting
> mtu size
> to be 1500 seems okay but very rarely it may hit the same problem too.
>
> ip address flush dev ens3f0
> ip link set dev ens3f0 down
> ip address flush dev ens3f1
> ip link set dev ens3f1 down
> [root@ca-hcl629 etc]# modprobe bonding mode=0 miimon=250 use_carrier=1
> updelay=500 downdelay=500
> [root@ca-hcl629 etc]# ifconfig bond0 up
> [root@ca-hcl629 etc]# ifenslave bond0 ens3f0 ens3f1
> [root@ca-hcl629 etc]# ip link set bond0 mtu 4500 up

> Seiral console output:
>
> ** 4 printk messages dropped ** [ 3717.743761] bond0: link status down for
> interface ens3f0, disabling it in 500 ms
[..]


> It seems that when setting a large mtu size on an RoCE interface, the RTNL
> mutex may be held too long by the slave
> interface, causing bond_mii_monitor() to be called repeatedly at an interval
> of 1 tick (1K HZ kernel configuration) and kernel to become unresponsive.

Did you try/managed to reproduce that also with other NIC drivers?

Or.