Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754172AbdFNCtw (ORCPT ); Tue, 13 Jun 2017 22:49:52 -0400 Received: from mail-io0-f181.google.com ([209.85.223.181]:35106 "EHLO mail-io0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753448AbdFNCtu (ORCPT ); Tue, 13 Jun 2017 22:49:50 -0400 MIME-Version: 1.0 From: Don Bowman Date: Tue, 13 Jun 2017 22:49:49 -0400 Message-ID: Subject: netns performance To: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1019 Lines: 35 This was talked about before (e.g. https://lkml.org/lkml/2014/6/11/42) In 3.10, this 'script' takes ~1s to run, regardless of # CPU: ip netns add foo ip netns add bar for i in `seq 0 1000` do echo -e 'netns exec foo echo\nnetns exec bar echo' >> ipnetns.batch done time ip -b ipnetns.batch > /dev/null On 3.16... 4.12rc5, it gets very slow as you add CPU. 1cpu: 0m1.3 12cpu: 1m36 32cpu: 2m36 72cpu: 3m50 if you have a 2nd copy of it running (e.g .contention) it gets much much worse. this is in turn causing me an issue in OpenStack Neutron l3-agent, which creates/destroys/enters namespaces for DHCP, Router. I had a few shots in the dark of tuning e.g. rcu_nocbs= [tried 0, 1-31, etc]. The commit that was talked about as fixing, 728dba3a39c66b3d8ac889ddbe38b5b1c264aec3, doesn't seem to have helped me. I have tried this on Centos7 and Ubuntu 16.10+ kernel configs. Some work was done characterising this linked https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1328088 Does anyone have a suggestion?