Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp1174368rwd; Sun, 14 May 2023 14:29:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7YQbcjTA0yvJb0SGFtuvoGqThrdbKx0xzf2DfMQDrQBZli8VtA2i0x+6QLlazBUjKkVVRS X-Received: by 2002:a05:6a21:6d84:b0:103:fcd7:544d with SMTP id wl4-20020a056a216d8400b00103fcd7544dmr16055276pzb.47.1684099742877; Sun, 14 May 2023 14:29:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684099742; cv=none; d=google.com; s=arc-20160816; b=mHIzugHYiyo42Up3FygpyGrBdPwAryTQOPbh8vYqlRe4oiLEajfE2WOISiEktPlO45 MnxUZdhExXYFGnz/ZODh/rUKT5yskNZryPfPlnpX/07iqqm2p3u5RNF6AmcKc7byPP2u Z33ygpXHAxvrz9yvBvMaNt/21t5TUINV7c32K/MRa1PydReI+aaYIpwnzbIHo80vQEJP k4lZPgaqhN4HueAgeSY8kjUMIkdUeC6FTOjS2SEdHdeo/xK1nwW4QU53jPBLW1FKaWKR png0B9Nl9AD8phKlqM9/sL6EIwRTrRKgPS09KFaZkKmiKO5ZPXaX1cOFnBk0ZcfHmNC6 KD2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:message-id:content-transfer-encoding :mime-version:subject:date:from:dkim-signature; bh=zwPLcczfHaotQQ5Bh17U80i7Ky8OX0CQkStxEqL6Fgc=; b=VAIywrf1SkglrRDEo4JAsWunbxtmF+0k8k+293G0WQXtSuoEWmNdDwky24lYfEuqCm lJZjNYNL3koDqLFEsvNPltgLkffh5UDUG6yR58cbfkojDHyhLsk7Apq8/QDX0HyM/ZNz ekT+2w8p0IU6XuR9muKIKfAPb2J3bpVjZ7lDygWgjJnhOocsm0s7q7Zbrq1FKzvrlLlU dQ5yP1MMVKpsZuAzHodJGM0e0piJVALMqS5cjPaIn/eVY+Q8NKV0r8rwWE4OOPXCr2Dz wHlUy2mEbdj0CuXQ54huRTRM8wVxjRvDv49IWw8qEJVzXnf5mqWGZsfJ0tlhYsGdxgbp Y4Ng== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=arhp3JVN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t185-20020a6381c2000000b00524eef20da6si14198220pgd.642.2023.05.14.14.28.48; Sun, 14 May 2023 14:29:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=arhp3JVN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237001AbjENUkj (ORCPT + 99 others); Sun, 14 May 2023 16:40:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229635AbjENUkf (ORCPT ); Sun, 14 May 2023 16:40:35 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 080EEE48; Sun, 14 May 2023 13:40:34 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1aad5245632so84927565ad.3; Sun, 14 May 2023 13:40:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684096833; x=1686688833; h=cc:to:message-id:content-transfer-encoding:mime-version:subject :date:from:from:to:cc:subject:date:message-id:reply-to; bh=zwPLcczfHaotQQ5Bh17U80i7Ky8OX0CQkStxEqL6Fgc=; b=arhp3JVNanZ3mDqxFjZiB8gxrZ2yc95p3ehPSMQFIs0pBDN4GbhW15bAYkDArYXWxe PNRazoUGwkZcqLpQTzdbfWNgij0PHRAnzd64cRaFqsHE+iwK1VV3UMB33oKkJFL3vI7V Nj/9qGw2lSCmCIxYcoayNVOK4aQFrp47AfwYIWqXxUNZ0m4Xj5yI1KpQ/5d9Dxf9T6zV hWYgREcS4L5Z9Pkapd8Zr0mdgoz7WOJC6wikeSf4VIRhKz3L+K9QfzxzYQig4MXwTpb1 jGf2NXr7bbPLPgYzbHpv2YYvrgWkUCNGbz+QNuu0nutgKO/mDlJHxiY+I1Hl0SoNIGlI wQIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684096833; x=1686688833; h=cc:to:message-id:content-transfer-encoding:mime-version:subject :date:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zwPLcczfHaotQQ5Bh17U80i7Ky8OX0CQkStxEqL6Fgc=; b=U331c464BuQJiID2OOyHKLU+FkmSu68zEx+m2byUnGVbsnZ9dXQpHA1qCFhFS6b15R dzN1OoQIz1MJ/9JaM0xszdiwsnvcp7ko60s2AJyLmAbGvKOuMbP+3jcVRSJbSe69gUNB xFQkvVgDX/L42LbKcB2RyW/E5vL2Rqd9t9hgHjcOFxyIW99gMJ9Rqgr1IeN3eDI5gZON lYliVK15gBeFSalaFGanZYWwsY2FSbHf+mxMzEePETTuuf6j4+pj0QA4dRrb5XXRQoqU AWBxr+dt570RvclbtMddpz/zdUfFWX7pvjs9RfHls0OGOuL24dRVbbYQJ1dVtmqVvu8m 47Ug== X-Gm-Message-State: AC+VfDzwBrv907dlOQALp4vZU5Et/oWUUwZppbpy5L99haAVacDGrbey zgKi5Ai+a+QZ0Ry6D4B8pLPpH3I+Kn/J509E X-Received: by 2002:a17:902:cece:b0:1a6:413c:4a3e with SMTP id d14-20020a170902cece00b001a6413c4a3emr41204234plg.5.1684096832828; Sun, 14 May 2023 13:40:32 -0700 (PDT) Received: from [127.0.1.1] ([2601:644:8f00:4f:728d:4faf:7256:5a34]) by smtp.gmail.com with ESMTPSA id q6-20020a170902b10600b001ab06958770sm11851906plr.161.2023.05.14.13.40.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 May 2023 13:40:32 -0700 (PDT) From: Abhijeet Rastogi Date: Sun, 14 May 2023 13:40:24 -0700 Subject: [PATCH v2] ipvs: increase ip_vs_conn_tab_bits range for 64BIT MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20230412-increase_ipvs_conn_tab_bits-v2-1-994c0df018e6@gmail.com> X-B4-Tracking: v=1; b=H4sIADdHYWQC/42OQQrDIBREr1Jc16JGQtNV71GCfI0mHxoT/CItI XevyQm6fDPDzGyMfEJP7HHZWPIFCZdYQV0vzE0QR89xqMyUUI3QUnGMLnkgb3AtZNwSo8lgjcV MXDdKuU4MoGTLaoOtOW4TRDcdHTNQ9ukw1uQDfs7ZV195QspL+p4vijzU/waL5JK3AnTognb3o J7jDPi+uWVm/b7vPyYGOiXfAAAA To: Simon Horman , Julian Anastasov , Pablo Neira Ayuso , Jozsef Kadlecsik , Florian Westphal , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, lvs-devel@vger.kernel.org, netfilter-devel@vger.kernel.org, coreteam@netfilter.org, linux-kernel@vger.kernel.org, Abhijeet Rastogi X-Mailer: b4 0.12.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1684096831; l=4400; i=abhijeet.1989@gmail.com; s=20230412; h=from:subject:message-id; bh=QKVsMjlQX/gTjNeKg6CZJYA0FFD+ARtEhFp33qX1UfA=; b=sDfvG6taPYG+iOeJr8xKMiapaAP0Fg4ONAtQiELClNWJMnRmkP3Q2skQOHBpGSL7wsrR+M1YZ WxQ2My9Kk2YArg0ZTXCHQZxUctAsIDaldhamEzKBnYNpDguqE6Era3r X-Developer-Key: i=abhijeet.1989@gmail.com; a=ed25519; pk=VinODWUuJys1VAWZP2Uv9slcHekoZvxAp4RY1p5+OfU= X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Current range [8, 20] is set purely due to historical reasons because at the time, ~1M (2^20) was considered sufficient. With this change, 27 is the upper limit for 64-bit, 20 otherwise. Previous change regarding this limit is here. Link: https://lore.kernel.org/all/86eabeb9dd62aebf1e2533926fdd13fed48bab1f.1631289960.git.aclaudi@redhat.com/T/#u Signed-off-by: Abhijeet Rastogi --- The conversation for this started at: https://www.spinics.net/lists/netfilter/msg60995.html The upper limit for algo is any bit size less than 32, so this change will allow us to set bit size > 20. Today, it is common to have RAM available to handle greater than 2^20 connections per-host. Distros like RHEL already allow setting limits higher than 20. --- Changes in v2: - Lower the ranges, 27 for 64bit, 20 otherwise - Link to v1: https://lore.kernel.org/r/20230412-increase_ipvs_conn_tab_bits-v1-1-60a4f9f4c8f2@gmail.com --- net/netfilter/ipvs/Kconfig | 26 +++++++++++++------------- net/netfilter/ipvs/ip_vs_conn.c | 4 ++-- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/net/netfilter/ipvs/Kconfig b/net/netfilter/ipvs/Kconfig index 271da8447b29..aac5d6bd82e6 100644 --- a/net/netfilter/ipvs/Kconfig +++ b/net/netfilter/ipvs/Kconfig @@ -44,7 +44,8 @@ config IP_VS_DEBUG config IP_VS_TAB_BITS int "IPVS connection table size (the Nth power of 2)" - range 8 20 + range 8 20 if !64BIT + range 8 27 if 64BIT default 12 help The IPVS connection hash table uses the chaining scheme to handle @@ -52,18 +53,17 @@ config IP_VS_TAB_BITS reduce conflicts when there are hundreds of thousands of connections in the hash table. - Note the table size must be power of 2. The table size will be the - value of 2 to the your input number power. The number to choose is - from 8 to 20, the default number is 12, which means the table size - is 4096. Don't input the number too small, otherwise you will lose - performance on it. You can adapt the table size yourself, according - to your virtual server application. It is good to set the table size - not far less than the number of connections per second multiplying - average lasting time of connection in the table. For example, your - virtual server gets 200 connections per second, the connection lasts - for 200 seconds in average in the connection table, the table size - should be not far less than 200x200, it is good to set the table - size 32768 (2**15). + Note the table size must be power of 2. The table size will be the value + of 2 to the your input number power. The number to choose is from 8 to 27 + for 64BIT(20 otherwise), the default number is 12, which means the table + size is 4096. Don't input the number too small, otherwise you will lose + performance on it. You can adapt the table size yourself, according to + your virtual server application. It is good to set the table size not far + less than the number of connections per second multiplying average lasting + time of connection in the table. For example, your virtual server gets + 200 connections per second, the connection lasts for 200 seconds in + average in the connection table, the table size should be not far less + than 200x200, it is good to set the table size 32768 (2**15). Another note that each connection occupies 128 bytes effectively and each hash entry uses 8 bytes, so you can estimate how much memory is diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c index 13534e02346c..e1b9b52909a5 100644 --- a/net/netfilter/ipvs/ip_vs_conn.c +++ b/net/netfilter/ipvs/ip_vs_conn.c @@ -1484,8 +1484,8 @@ int __init ip_vs_conn_init(void) int idx; /* Compute size and mask */ - if (ip_vs_conn_tab_bits < 8 || ip_vs_conn_tab_bits > 20) { - pr_info("conn_tab_bits not in [8, 20]. Using default value\n"); + if (ip_vs_conn_tab_bits < 8 || ip_vs_conn_tab_bits > 27) { + pr_info("conn_tab_bits not in [8, 27]. Using default value\n"); ip_vs_conn_tab_bits = CONFIG_IP_VS_TAB_BITS; } ip_vs_conn_tab_size = 1 << ip_vs_conn_tab_bits; --- base-commit: 09a9639e56c01c7a00d6c0ca63f4c7c41abe075d change-id: 20230412-increase_ipvs_conn_tab_bits-4322c90da216 Best regards, -- Abhijeet Rastogi