Received: by 2002:ab2:7855:0:b0:1f9:5764:f03e with SMTP id m21csp179758lqp; Wed, 22 May 2024 00:44:52 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCVo708NGaZSY3p+dG/IsJpIdYalfCuupr2OHmtAx3K3irz8IqhbFD3/LBm5zbiKriPfIBnvXB1OEtSo0MFd6WCm9q1Vppp3T2yjAZJwew== X-Google-Smtp-Source: AGHT+IEwLOLb7osdQhP+7cZfQzzE4e1ilUnRRdYomxWC+NdlM+50ncaf7q6oiZ3cnOUjrWFqEO6t X-Received: by 2002:a17:906:ae51:b0:a59:c9f3:837d with SMTP id a640c23a62f3a-a5d5ecdc534mr989877966b.30.1716363892768; Wed, 22 May 2024 00:44:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1716363892; cv=pass; d=google.com; s=arc-20160816; b=zq2Kf9Z2qlFXyi8mNNUorp9/omUdfBSLdeKI/59qSLYG3KmSqBZeXZVUbUaEuwd2AK fJSlnvIXqgvTzd8PsTKhx8XvgxiRwsuPi1A8ovWBdWOKnu1YUSzA+wTlWd42lPF8+73S GTPWV4+fTNh6QW7R1L1Q/uWBTLcuAk/ouEYnoGe5dJUEJ/9PzxNAc5958rWhG1/J0AA4 tee1NCz1YZOCYOZ78X6OrWpA3QoLsZGv9zLVtKhtsBIBZdZT0SwgtJBl+A7QdepKF99L WBK+yIcmHx7PrzscZfNPvua1Ncxm0gKETewP1B4DY9wK8laQKkzmxG8OJ9iqWZ/ujbVp xjYw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:list-unsubscribe:list-subscribe:list-id:precedence :content-transfer-encoding:content-language:accept-language :message-id:date:thread-index:thread-topic:subject:cc:to:from; bh=OyAYU9wdToTlNpCqVBj184wBvTfjYqMromOTV3WB6PM=; fh=4lp+yXpdtVeY7HI1S6QNc4xtpWXnMYzjY3UXyN1j5Wk=; b=YV1Vc2hPT2nHVfzLPiyGCjq5P8k97GcJ+QFZEPz7I67t2O9Ba+tZJRJrnt/QshJ3Qk i1zBVrYhdDNUOIZuK4KLqh9SksZtoPfX5+NlzyEuwfweVKdCm4g1a8Fpuqn1O6X6TGLv hMDH4wnLiVgEU5SpiLUo80PMqF2CqU6JQRJty2H4Te8aHdKIYFH+RgaLxy7RuK1KWpBD 4h/k0eaJ63Y9ELTLFeID+YZTkai63zEAY7D9CYlkRKeZnizQn8j1Vm7cBHqAXbZbRsSe Rlth+Y/q5oqH0ndOaZod9fSD0+Dn4Hk+9x7SR42eHbi2K/KPs/L47X0ITr0RKlEGy7si 73oQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huawei.com dmarc=pass fromdomain=huawei.com); spf=pass (google.com: domain of linux-kernel+bounces-185873-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-185873-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id a640c23a62f3a-a5cdf194dbfsi773770866b.130.2024.05.22.00.44.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 May 2024 00:44:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-185873-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huawei.com dmarc=pass fromdomain=huawei.com); spf=pass (google.com: domain of linux-kernel+bounces-185873-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-185873-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 785DC1F216D8 for ; Wed, 22 May 2024 07:44:52 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B41AC7E56B; Wed, 22 May 2024 07:44:41 +0000 (UTC) Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 942DD7D40D; Wed, 22 May 2024 07:44:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.189 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716363881; cv=none; b=UXNmhjYgEGeiU8Fw51PKRWhtEAsSiOz1CSioz+mtFjFme0AT/UvJ6MsxVt0jNMwBgpT7ki2rIrUTua0jRqy4tr8owV6Pfxq3mwIEDdYzMS5R1ZMXwn6Pc+sAC8LltX+ItgSlrJmywa2phOXWoqR6hyv3RAC5iqfzo/WUyYQQmGo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716363881; c=relaxed/simple; bh=OyAYU9wdToTlNpCqVBj184wBvTfjYqMromOTV3WB6PM=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=ppHyE2VUkMzZRUwb5bg4al3xFuwj5khgJmjlSmpTbuJWg1r/1GqWcnMIYb5sGzbuUb3oGup3BoStpjkT5tTfIxW9nXSR6GSKCNE1723TSj/EEZv/UdNO3vSpy8BLRbXkj6yw0eNxiphA2zfC/WChLul53wtHvpvKuEwUIivdHKI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=45.249.212.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.19.162.254]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4Vkjt558xVzPmKT; Wed, 22 May 2024 15:41:33 +0800 (CST) Received: from kwepemm600019.china.huawei.com (unknown [7.193.23.64]) by mail.maildlp.com (Postfix) with ESMTPS id 9B3A11800C9; Wed, 22 May 2024 15:44:35 +0800 (CST) Received: from dggpeml500007.china.huawei.com (7.185.36.75) by kwepemm600019.china.huawei.com (7.193.23.64) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Wed, 22 May 2024 15:44:35 +0800 Received: from dggpeml500007.china.huawei.com ([7.185.36.75]) by dggpeml500007.china.huawei.com ([7.185.36.75]) with mapi id 15.01.2507.035; Wed, 22 May 2024 15:44:34 +0800 From: mengkanglai To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jiri Pirko , Simon Horman , Daniel Borkmann , "Sebastian Andrzej Siewior" , Lorenzo Bianconi , "netdev@vger.kernel.org" , open list CC: "Fengtao (fengtao, Euler)" , "Yanan (Euler)" Subject: cpu performance drop between 4.18 and 5.10 kernel? Thread-Topic: cpu performance drop between 4.18 and 5.10 kernel? Thread-Index: Adqr+is3Qylu3zxvSvOZrHpOxyfQ/w== Date: Wed, 22 May 2024 07:44:34 +0000 Message-ID: <9fd382fb581e47a291ed31bfe091112c@huawei.com> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Dear maintainers: I updated my VM kernel from 4.18 to 5.10, and found that the CPU SI usage w= as higher under the 5.10 kernel for the same udp service. I captured the flame graph and compared the two versions of kernels.=20 Kernel 5.10 compared to 4.18 napi_complete_done function added gro_normal_l= ist call (ommit 323ebb61e32b4 ("net: use listified RX for handling GRO_NORM= AL skbs") Introduced), I removed gro_normal_list from napi_complete_done in 5.= 10 kernel, CPU SI usages was same as 4.18. I don't know much about GRO, so I'm not sure if it can be modified in this = way, and the consequences of such a modification?