Received: by 2002:a89:48b:0:b0:1f5:f2ab:c469 with SMTP id a11csp912711lqd; Wed, 24 Apr 2024 23:44:42 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCWH1N7qkPFRm7LBjvXWjr6kb/A81C+cnOnaWVIC2ltRbo4Um9rjCXw7JlRZ2dDeZCNdaYqG79/AxnpVIiiyFniaFBzKnZrUWcR30Z8QqQ== X-Google-Smtp-Source: AGHT+IF5dRJU0/kCyU6FSf1nah117IltEf6sd+q2z1nIBqnpMwk21ktT6NinwtvIk/OVGhaXKeUv X-Received: by 2002:a2e:8610:0:b0:2de:7cc5:7a35 with SMTP id a16-20020a2e8610000000b002de7cc57a35mr2928245lji.48.1714027481820; Wed, 24 Apr 2024 23:44:41 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1714027481; cv=pass; d=google.com; s=arc-20160816; b=BfwSmSqyPv9aNnLvd0Z8bNWniC+ueljzpHvb7gslnjN0Gor42G8XoIlDE0EBzrAhCw pEGXQV48SiXryxsNUNgO442yoWHsQFHJuR9c8TLWrVFS56Tt+DVKum3g4a+aIhB+w3Pz S2S5Cs+QWrPMLh7U6ueno6DEyc5+F3DFKNSNGdguJcLZoFZCEaZxamsmIm8g9gmKNcTQ yinzzAazErHwhVz0DRAuVIBE6ooPAZNfjKkkiqD386LOF+vEnTObRbPj+Xz+6a7Sj7ZP iaXWuBOoaUkS4i7PFdEhL03DDT7Ka42OL5Z4T6oPNDEg8Z1MlTygSXSyo9H0CTzXiAS0 seUQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:date:message-id:dkim-signature; bh=/Ulz98hKzYu1co/cS/dOgIOLEVqfXbA1NDOWLo9MEUI=; fh=PrIEA24EGWDjht/xuLxe9RY74vGG29dgBTfn193yTHc=; b=heFML78rfp2lpcE8IipIN3TgY0XbpA68l8BhH+7qGFzGBnQJvNBY93Ppzh+FBeqQ7I vd9E0qhHN6FEQH4Ii+C+8AD1Tzq2NgA9xaQlC1MK5tWHwam2/akDZ1CXA4qmQ8rUdgxj afnewBD9OiSwfXctwJ/DEpMo4xiMpm1j/NzZsPoNtNmSI+QrSUbnO39skGFK7UX+FeSN ZibhXw7QROLLydPrW/wcTpr1GCi4jMZ+e/IHCT/rvsIWmoI+KDNy+8GCtK+zA6eXS49j c0XQpFk0HOntOdY0o8FHN6hKWeYZuJF05gPk6teEv1RJx6g9Gu4KydGcMOm0h4p7ye9e gudQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@ti.com header.s=ti-com-17Q1 header.b=WtlrpY2w; arc=pass (i=1 spf=pass spfdomain=ti.com dkim=pass dkdomain=ti.com dmarc=pass fromdomain=ti.com); spf=pass (google.com: domain of linux-kernel+bounces-158081-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-158081-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=ti.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id y16-20020a056402359000b005723bc3e28asi993632edc.467.2024.04.24.23.44.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Apr 2024 23:44:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-158081-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@ti.com header.s=ti-com-17Q1 header.b=WtlrpY2w; arc=pass (i=1 spf=pass spfdomain=ti.com dkim=pass dkdomain=ti.com dmarc=pass fromdomain=ti.com); spf=pass (google.com: domain of linux-kernel+bounces-158081-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-158081-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=ti.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 3F1D31F234AE for ; Thu, 25 Apr 2024 06:44:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9998E5A4CF; Thu, 25 Apr 2024 06:44:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="WtlrpY2w" Received: from fllv0016.ext.ti.com (fllv0016.ext.ti.com [198.47.19.142]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C47CC3BBE8; Thu, 25 Apr 2024 06:44:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.47.19.142 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714027466; cv=none; b=hTjBMYSOz9LPtWGIqeg1+oUviWhJWDOch2ico8swdmq5g5CLu+pxFNTD0HTKF2RI0ceWBi78qrQOTjq+7hhCeUFBvOMHAd1+zeoP13LKILvHsJ2DqkLoSUXQxTZqLIK3AQbzqUP5LPxFhi2aNYJR9zwKJouFcB1Eo1w6TEG6rNo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714027466; c=relaxed/simple; bh=WQSMwd+tNMCVO6kH1o+7mkTtu6LUon02eBDzLfrMOo8=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=tVlKUs2reqQCZYJv2uTxqITTNy2LCSoMH0dhyOLviQpSLzyMW6gVz4NEuoGJq93zu4ZQtatHxFwgKifYFQ6Epbk/APqM7yl0D5gk2wfGwXAvLa0fdgFZNUgViPzDIPFam1MCwsOLQoJHICsxWklHN2zbaxPTOYub00FD02PA4Gg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=WtlrpY2w; arc=none smtp.client-ip=198.47.19.142 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Received: from fllv0035.itg.ti.com ([10.64.41.0]) by fllv0016.ext.ti.com (8.15.2/8.15.2) with ESMTP id 43P6i47Z085553; Thu, 25 Apr 2024 01:44:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1714027444; bh=/Ulz98hKzYu1co/cS/dOgIOLEVqfXbA1NDOWLo9MEUI=; h=Date:Subject:To:CC:References:From:In-Reply-To; b=WtlrpY2wyJsxReczN5lBNBshDzijgx/oB1qSXDL2d4EzpXmf4Rsk2Z23+BJftc7cU Kp9kOEfEcZnWJ2ddjTrDbGWYr+Amlt0dI8Vk7feoEOSPJpugq2v7MO/o2ow/wDgOla wg+YaWkVUIA80cl21P/wr1cUxsEpKkkf/CHT5wMg= Received: from DLEE100.ent.ti.com (dlee100.ent.ti.com [157.170.170.30]) by fllv0035.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 43P6i41E041580 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 25 Apr 2024 01:44:04 -0500 Received: from DLEE110.ent.ti.com (157.170.170.21) by DLEE100.ent.ti.com (157.170.170.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23; Thu, 25 Apr 2024 01:44:04 -0500 Received: from lelvsmtp6.itg.ti.com (10.180.75.249) by DLEE110.ent.ti.com (157.170.170.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.23 via Frontend Transport; Thu, 25 Apr 2024 01:44:04 -0500 Received: from [10.24.69.25] (danish-tpc.dhcp.ti.com [10.24.69.25]) by lelvsmtp6.itg.ti.com (8.15.2/8.15.2) with ESMTP id 43P6hwhf010667; Thu, 25 Apr 2024 01:43:59 -0500 Message-ID: <1c5809f2-b69d-48d1-8c27-285f164ebeb8@ti.com> Date: Thu, 25 Apr 2024 12:13:58 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH net-next] net: ti: icssg_prueth: Add SW TX / RX Coalescing based on hrtimers Content-Language: en-US To: Andrew Lunn CC: Dan Carpenter , Heiner Kallweit , Jan Kiszka , Diogo Ivo , Paolo Abeni , Jakub Kicinski , Eric Dumazet , "David S. Miller" , , , , , Vignesh Raghavendra , , Roger Quadros References: <20240424091823.1814136-1-danishanwar@ti.com> <98588a89-4970-4d75-be8a-ac410d77789f@lunn.ch> From: MD Danish Anwar In-Reply-To: <98588a89-4970-4d75-be8a-ac410d77789f@lunn.ch> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Hi Andrew, On 24/04/24 6:01 pm, Andrew Lunn wrote: > On Wed, Apr 24, 2024 at 02:48:23PM +0530, MD Danish Anwar wrote: >> Add SW IRQ coalescing based on hrtimers for RX and TX data path for ICSSG >> driver, which can be enabled by ethtool commands: >> >> - RX coalescing >> ethtool -C eth1 rx-usecs 50 >> >> - TX coalescing can be enabled per TX queue >> >> - by default enables coalesing for TX0 >> ethtool -C eth1 tx-usecs 50 >> - configure TX0 >> ethtool -Q eth0 queue_mask 1 --coalesce tx-usecs 100 >> - configure TX1 >> ethtool -Q eth0 queue_mask 2 --coalesce tx-usecs 100 >> - configure TX0 and TX1 >> ethtool -Q eth0 queue_mask 3 --coalesce tx-usecs 100 --coalesce >> tx-usecs 100 >> >> Minimum value for both rx-usecs and tx-usecs is 20us. > > Do you have some benchmark numbers? > > Did you see this patch on the mailing list: > > https://lore.kernel.org/all/20240415094804.8016-1-paul.barker.ct@bp.renesas.com/T/#md50cb07bbdd6daf985f3796508cf4b246b085268 > > This is basically a one line change, which brings big performance > gains. Did you try something as simple as that, rather than all your > hrtimer code? > I did some benchmarking today with, 1. Default driver (without any IRQ coalescing enabled) 2. IRQ Coalescing (With this patch) 3. Default IRQ Coalescing (Suggested by you in the above patch) I have pasted the full logs at [1]. Below are the final numbers, ============================================================== Method | Tput_TX | CPU_TX | Tput_RX | CPU_RX | ============================================================== Default Driver 943 Mbps 31% 517 Mbps 38% | IRQ Coalescing (Patch) 943 Mbps 28% 518 Mbps 25% | Default IRQ Coalescing 942 Mbps 32% 521 Mbps 25% | ============================================================== I see that the performance number is more or less same for all three methods only the CPU load seems to be varying. The IRQ coalescing patch (using hrtimer) seems to improve the cpu load by 3-4% in TX and 13% in RX. Whereas the default method that you have suggested doesn't give any improvemnet in tx however cpu load improves in RX with the same amount as method 2. Please let me know if this patch is OK to you based on the benchmarking? [1] https://gist.githubusercontent.com/danish-ti/47855631be9f3635cee994693662a988/raw/94b4eb86b42fe243ab03186a88a314e0cb272fd0/gistfile1.txt > Andrew -- Thanks and Regards, Danish