Received: by 2002:a05:6358:53a8:b0:117:f937:c515 with SMTP id z40csp3642183rwe; Mon, 17 Apr 2023 00:57:24 -0700 (PDT) X-Google-Smtp-Source: AKy350ZC3M4DRQWoONChsbkHT7V+dCiOgMp2U6CVK7+rKcRivT3WYJ6/yQi8bu17H2R4npiCQ7RT X-Received: by 2002:a05:6a00:2292:b0:637:f1ae:d3e with SMTP id f18-20020a056a00229200b00637f1ae0d3emr21768510pfe.25.1681718244110; Mon, 17 Apr 2023 00:57:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681718244; cv=none; d=google.com; s=arc-20160816; b=LZMwVe9WbRjfQimJMD4TqTPDmLAfe/2VZBaBzh86M87p6qAQI5VatfFO6MUfunERzw kfkYUXwmf5cIl3MiewFd3dL6lQ5lyniRvTewZYJr3IR2dW2tkFrqVidg2lSp91TOGC/V 9pcHX+3zIHO0hA5IBMqc+F9/OkC8SnlQ5e135u45G79x84M0emjBDrQs1YFQBef5hhcw ye3q0uEQrT2dhUALXNh920skoYs3OO3lbYNswrxIFfud8KuSZwLxMlWNQU5A0aQAkj/w 5x8uRDeRXHlX+oIG+m5OHvw9T4PaAWmMMAjduOytbEPDjtyQNMSTgTqzyFFdnE20+nOM sx3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=QRdnQCKny8giW3OyExxPhuJqL89+ZaGGJFW/ztJq/GQ=; b=DA/GnScvnO++0Le9JkyQnjOIVXNneEBjdemjtLvgNGNL3bZSW/B7fZWprvqgmVWsDb E89NX0fGWR0Wr+vuQ94FSXf6rvVSRKh0YI6l69BP4qM1PKO0zE4/eu7AFNmUSUPhhYgZ cwmf6M0T0IU6MMYLPK3LPMr6hdgswK0oiGx6oj3qE2u9wZR2aIzQzzjqLvbfJ9IRwDqD pAyuoAZlwpWh49uoHiaSNVrCr7fua4vEtkIdTrsPfNgdCbKTBBsK1U+v002kBSXed+Q2 ZwPJviqb++bRmtdO5Z+bJ9tlbuI+q+Qno92t5ifno5grwFZS5u+GOTbTLQsXn2QJScfB NVKw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Hchb+Dzd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a21-20020a63e855000000b004fb33a76e2csi11575332pgk.834.2023.04.17.00.57.11; Mon, 17 Apr 2023 00:57:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Hchb+Dzd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229458AbjDQHkM (ORCPT + 99 others); Mon, 17 Apr 2023 03:40:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55958 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231280AbjDQHjz (ORCPT ); Mon, 17 Apr 2023 03:39:55 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CEA54236 for ; Mon, 17 Apr 2023 00:38:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681717087; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QRdnQCKny8giW3OyExxPhuJqL89+ZaGGJFW/ztJq/GQ=; b=Hchb+DzdLmaEMJW/XuahMEGmJ/qRwq8KffahtKBOejxVNe6LYzbW1H1+u2/+IJ68l0R8n5 dTpt60TGimXkJC0R46WZbnYjq6s2D2xckS66GYQF24erkfOq19v6tTHu+XPzJJPmI95o+T opcXVCebkUS+VBzuGJZ+AlWw7nV9hEY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-549--G3PADyWMEmGXGJVJmaYgA-1; Mon, 17 Apr 2023 03:38:03 -0400 X-MC-Unique: -G3PADyWMEmGXGJVJmaYgA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 980DE85C1A7; Mon, 17 Apr 2023 07:38:02 +0000 (UTC) Received: from ovpn-8-16.pek2.redhat.com (ovpn-8-16.pek2.redhat.com [10.72.8.16]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9AEFB40C6E6E; Mon, 17 Apr 2023 07:37:55 +0000 (UTC) Date: Mon, 17 Apr 2023 15:37:50 +0800 From: Ming Lei To: Li Feng Cc: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , "open list:NVM EXPRESS DRIVER" , open list , lifeng1519@gmail.com, ming.lei@redhat.com Subject: Re: [PATCH v2] nvme/tcp: Add support to set the tcp worker cpu affinity Message-ID: References: <20230413062339.2454616-1-fengli@smartx.com> <20230413132941.2489795-1-fengli@smartx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230413132941.2489795-1-fengli@smartx.com> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Apr 13, 2023 at 09:29:41PM +0800, Li Feng wrote: > The default worker affinity policy is using all online cpus, e.g. from 0 > to N-1. However, some cpus are busy for other jobs, then the nvme-tcp will > have a bad performance. Can you explain in detail how nvme-tcp performs worse in this situation? If some of CPUs are knows as busy, you can submit the nvme-tcp io jobs on other non-busy CPUs via taskset, or scheduler is supposed to choose proper CPUs for you. And usually nvme-tcp device should be saturated with limited io depth or jobs/cpus. Thanks, Ming