Received: by 2002:a05:7412:b995:b0:f9:9502:5bb8 with SMTP id it21csp5826478rdb; Sun, 31 Dec 2023 20:09:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IFuH2e15noDS3dWVwi30+0pMdRc0rPFslQNZABaoKM3hoQbGa766cfjNq9KOgTCxqmeiRa/ X-Received: by 2002:a05:620a:b85:b0:781:d4d:c072 with SMTP id k5-20020a05620a0b8500b007810d4dc072mr18335302qkh.94.1704082174426; Sun, 31 Dec 2023 20:09:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704082174; cv=none; d=google.com; s=arc-20160816; b=HClNK4Sb2Ty7PZi6JWdxN+hd3KztPXP8jDurgTG9NTZyaRrwUjsvanAgckC2FfUceC tRZ9wUjZ0MLlS08viIjjTBzPBU1kLME7ygtHDLNuYjsGuJR+bU6FdEkx7KpYTRQx+7Jq dhhieso4VBOYNca68qpas14rMmHYLsUuTxrJZkUa5NmFzlReVu69cNFM1PckXgkYPbF6 jODQJAX0+BryycdnZj8KTFkSp3GV5Of1hdVv3oalsecPBaYI+W4h2TXbGpt3YOBUCQFA UNJuFndV481TB5zpl2n7nFkBFy//OMo6Ait4BLwIiZu5pgATyxBCBb3qWCc8VnC6oHpl KO7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:date:message-id:dkim-signature; bh=ChtrIcmjzv1mH8hapA1WonjnaM6IRYgbnIKPoH/S1kA=; fh=/8TN5tpEqOkkoN73Yz9fFu8j5QUKph09dO732LvPqE4=; b=uDe4Bz/Db+7ecaKI/rdt5quNtwEF8XsxFK6FYv0ZaQq3NXzpN6ejWKM/3UREE0c49X lo2SeDcCiCZ1m0tiA5rkfeYuu6Z3+McUeE7BsAnlOohTbJekMY5bJKgmBqNCXxjATEGl BaivzhNO6TKc1iwfbQoirc8CYTpfHBoFQii1Mk1crYjRk8w16E6E8mPnSWBgi3ksf0gB DiYJC0R2DvWyDJ75AXzUK5DPqR7Q4c6WfQiqR2HH+HRYrlScxxyA7oXHAH1PejaMjCUs bKKoP6T2QaWKxoChPuFsvFxHuTdbJgdCcSVnfLxz3muTNLXpnIyfBCl/Rdl0pEQesZEw gOYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=fJ+N7ne4; spf=pass (google.com: domain of linux-kernel+bounces-13823-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13823-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id w6-20020a05620a424600b00781c4a1a97asi1518488qko.582.2023.12.31.20.09.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 31 Dec 2023 20:09:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-13823-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=fJ+N7ne4; spf=pass (google.com: domain of linux-kernel+bounces-13823-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-13823-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 325B01C21C0A for ; Mon, 1 Jan 2024 04:09:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 41D28ECF; Mon, 1 Jan 2024 04:09:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fJ+N7ne4" X-Original-To: linux-kernel@vger.kernel.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0707BA5A for ; Mon, 1 Jan 2024 04:09:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1704082164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ChtrIcmjzv1mH8hapA1WonjnaM6IRYgbnIKPoH/S1kA=; b=fJ+N7ne4ryD443qB6cs4pP5AsXxMmZk38y7wGRlD9e16eh33lYZedE0+7KmQ8wfAUECB1i 658aPKfq14tfZ2Jp15JmugCkHeVIWU9d/oib7xXwAaxCTC4FoIBnu8hSeaaCpePWT4za3B yRy3cBjBzDnff+C+C2r+RavxMyxjhjE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-463-rawozoFjP_6dTpZyS08TyQ-1; Sun, 31 Dec 2023 23:09:20 -0500 X-MC-Unique: rawozoFjP_6dTpZyS08TyQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3039884AC60; Mon, 1 Jan 2024 04:09:20 +0000 (UTC) Received: from [10.22.16.25] (unknown [10.22.16.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id B858251D5; Mon, 1 Jan 2024 04:09:19 +0000 (UTC) Message-ID: <3a66efbd-fce0-49c3-bd71-63d52078d243@redhat.com> Date: Sun, 31 Dec 2023 23:09:19 -0500 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH next v2 2/5] locking/osq_lock: Optimise the vcpu_is_preempted() check. Content-Language: en-US To: David Laight , "'linux-kernel@vger.kernel.org'" , "'peterz@infradead.org'" Cc: "'mingo@redhat.com'" , "'will@kernel.org'" , "'boqun.feng@gmail.com'" , 'Linus Torvalds' , "'virtualization@lists.linux-foundation.org'" , 'Zeng Heng' References: <2b4e8a5816a742d2bd23fdbaa8498e80@AcuMS.aculab.com> <3a9d1782cd50436c99ced8c10175bae6@AcuMS.aculab.com> From: Waiman Long In-Reply-To: <3a9d1782cd50436c99ced8c10175bae6@AcuMS.aculab.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 On 12/31/23 16:52, David Laight wrote: > The vcpu_is_preempted() test stops osq_lock() spinning if a virtual > cpu is no longer running. > > Although patched out for bare-metal the code still needs the cpu number. > Reading this from 'prev->cpu' is a pretty much guaranteed have a cache miss > when osq_unlock() is waking up the next cpu. > > Instead save 'prev->cpu' in 'node->prev_cpu' and use that value instead. > Update in the osq_lock() 'unqueue' path when 'node->prev' is changed. > > This is simpler than checking for 'node->prev' changing and caching > 'prev->cpu'. > > Signed-off-by: David Laight > --- > kernel/locking/osq_lock.c | 16 +++++++--------- > 1 file changed, 7 insertions(+), 9 deletions(-) > > diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c > index e0bc74d85a76..eb8a6dfdb79d 100644 > --- a/kernel/locking/osq_lock.c > +++ b/kernel/locking/osq_lock.c > @@ -14,8 +14,9 @@ > > struct optimistic_spin_node { > struct optimistic_spin_node *next, *prev; > - int locked; /* 1 if lock acquired */ > - int cpu; /* encoded CPU # + 1 value */ > + int locked; /* 1 if lock acquired */ > + int cpu; /* encoded CPU # + 1 value */ > + int prev_cpu; /* encoded CPU # + 1 value */ > }; > > static DEFINE_PER_CPU_SHARED_ALIGNED(struct optimistic_spin_node, osq_node); > @@ -29,11 +30,6 @@ static inline int encode_cpu(int cpu_nr) > return cpu_nr + 1; > } > > -static inline int node_cpu(struct optimistic_spin_node *node) > -{ > - return node->cpu - 1; > -} > - > static inline struct optimistic_spin_node *decode_cpu(int encoded_cpu_val) > { > int cpu_nr = encoded_cpu_val - 1; > @@ -110,9 +106,10 @@ bool osq_lock(struct optimistic_spin_queue *lock) > if (old == OSQ_UNLOCKED_VAL) > return true; > > - node->locked = 0; > + node->prev_cpu = old; > prev = decode_cpu(old); > node->prev = prev; > + node->locked = 0; > > /* > * osq_lock() unqueue > @@ -144,7 +141,7 @@ bool osq_lock(struct optimistic_spin_queue *lock) > * polling, be careful. > */ > if (smp_cond_load_relaxed(&node->locked, VAL || need_resched() || > - vcpu_is_preempted(node_cpu(node->prev)))) > + vcpu_is_preempted(READ_ONCE(node->prev_cpu) - 1))) > return true; > > /* unqueue */ > @@ -201,6 +198,7 @@ bool osq_lock(struct optimistic_spin_queue *lock) > * it will wait in Step-A. > */ > > + WRITE_ONCE(next->prev_cpu, prev->cpu); > WRITE_ONCE(next->prev, prev); > WRITE_ONCE(prev->next, next); > Reviewed-by: Waiman Long Reviewed-by: Waiman Long