Received: by 2002:a05:6a10:9848:0:0:0:0 with SMTP id x8csp4034706pxf; Mon, 29 Mar 2021 19:30:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxvoN5f6U7qAvRXzvxdfuVKeMcGq/m80TaOfOAnqDQW8zy4ZW6D2jTymhcnObj9hfppbIW4 X-Received: by 2002:a05:6402:646:: with SMTP id u6mr31866542edx.250.1617071419920; Mon, 29 Mar 2021 19:30:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1617071419; cv=none; d=google.com; s=arc-20160816; b=v8k4iiYbzYCrPVIl9e5R9Fp4MKvD3Jkmx25CnAMXjBakfbNtLn8x3Ahj6195I/mL7+ ART590ZSVrz2Ngs7qbk5T4K8wvGvwafN36aEtDD+UBkRl4AdlD7iPhkSoi/lQuonB8bs qdB1Dt9P/V7Ysx+vHM9dCTkld6v1i3c/yhWMaUdcqgeXU2ONGI5rRBHmormLr2xOjZEQ /Cn+Ju+RyTWeD+iyAqyE97C3JGPcWJ7oPfWf47w6e4gWWErD/UNu6HQDKsG7Xp5saiE1 R977B6slL40ALb0DmM7dEHIvHBok90iv4l71k6Jiz44WQrjKQ5gPSeuGMbRYGMtkWyBu 2mvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:subject :message-id:date:from:in-reply-to:references:mime-version :dkim-signature; bh=tsS68RM7i/ke2TNbHbnYAUGYCHtNZfeGH47TPOmu90Y=; b=cw84xkXItZxKnN8sgQ0sYRAhIBa//55amnOj+GXddVl4P2vcHfUL2hGxBcrxnvlmfg GlkVLHD8gkc8PiSuQMr43Hyt6gcSbydk4oshRDba+JVgSqEZ0Vf1Jqud5ENDOAs4LG2S /SdqhnucAKvm/gcXGzMDpl0k5vmD98UOZIjcy7PRgFiLZ6HAylAxkrfzV+Fsc7WopOvA AesClTpdpUkerG39uy7pmZYx2I6msUAEG4FhpdMtxJrbidHguZOO7yNQ7x5qBRi3Z5vz TU9lvzl6M6oLrPWLWEKh6nMKSpZsfkHRUkVcSjFt9PKGKmD+AI3jjU7Jd4GZuIyNGQLV /QcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=d4cb+mFm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id gy13si14044843ejb.397.2021.03.29.19.29.57; Mon, 29 Mar 2021 19:30:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=d4cb+mFm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230100AbhC3C1F (ORCPT + 99 others); Mon, 29 Mar 2021 22:27:05 -0400 Received: from mail.kernel.org ([198.145.29.99]:53894 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230089AbhC3C0e (ORCPT ); Mon, 29 Mar 2021 22:26:34 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id E3804619A7; Tue, 30 Mar 2021 02:26:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1617071194; bh=DsNUDKNJ0Zn7ha019DqF6JpRF6ePLDXBpY2fY4Rt5zI=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=d4cb+mFmuvef/cwfP2zNQ3obGFGjzuzoUwV/X2Zigv4nsZNUWKslRrqdL34xGoLxn 96WVK3ko/y5YsTXEzBvfqnWDjP0ijpDer3lLY8MPYAeN9i8JVTGiO6vGNYDOowoWHZ KWY83N80nfMGMh9gQ6f3pRmEufiZZ3fe3+W58B6HXt6U/owpCK++KKZV9lfwu7cmZz lTuJ2/F7BpyWD8Hb7PUf5J1BMo6rogzywecLGkdu3g9KM5rt+XesuBAdPnMbzL2Pt2 XB8jdv3pb5/7T1ey8dbagvED+OrQQDtvAHHbre4u+uDUBm39lzH+oKQlmdvbJDGg+r t8BC3M/LaSv4g== Received: by mail-lf1-f43.google.com with SMTP id g8so21404523lfv.12; Mon, 29 Mar 2021 19:26:33 -0700 (PDT) X-Gm-Message-State: AOAM532xQGkROV/T4XoBJzMcIh/pLwC3669/assFnQO22haLivakrhSm AcOEQaqwBWVxJflh+IZuY3vtGDVdPuQ/n5hcWjw= X-Received: by 2002:ac2:5e26:: with SMTP id o6mr18679032lfg.355.1617071192050; Mon, 29 Mar 2021 19:26:32 -0700 (PDT) MIME-Version: 1.0 References: <1616868399-82848-1-git-send-email-guoren@kernel.org> <1616868399-82848-4-git-send-email-guoren@kernel.org> In-Reply-To: From: Guo Ren Date: Tue, 30 Mar 2021 10:26:19 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v4 3/4] locking/qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 To: Arnd Bergmann Cc: Peter Zijlstra , linux-riscv , Linux Kernel Mailing List , linux-csky@vger.kernel.org, linux-arch , Guo Ren , Will Deacon , Ingo Molnar , Waiman Long , Anup Patel , Sebastian Andrzej Siewior Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 29, 2021 at 9:56 PM Arnd Bergmann wrote: > > On Mon, Mar 29, 2021 at 2:52 PM Guo Ren wrote: > > > > On Mon, Mar 29, 2021 at 7:31 PM Peter Zijlstra w= rote: > > > > > > On Mon, Mar 29, 2021 at 01:16:53PM +0200, Peter Zijlstra wrote: > > > > Anyway, an additional 'funny' is that I suspect you cannot prove fw= d > > > > progress of the entire primitive with any of this on. But who cares > > > > about details anyway.. :/ > > > > > > What's the architectural guarantee on LL/SC progress for RISC-V ? > > > > funct5 | aq | rl | rs2 | rs1 | funct3 | rd | opcode > > 5 1 1 5 5 3 5 7 > > LR.W/D ordering 0 addr width dest AMO > > SC.W/D ordering src addr width dest AMO > > > > LR.W loads a word from the address in rs1, places the sign-extended > > value in rd, and registers a reservation set=E2=80=94a set of bytes tha= t > > subsumes the bytes in the addressed word. SC.W conditionally writes a > > word in rs2 to the address in rs1: the SC.W succeeds only if the > > reservation is still valid and the reservation set contains the bytes > > being written. If the SC.W succeeds, the instruction writes the word > > in rs2 to memory, and it writes zero to rd. If the SC.W fails, the > > instruction does not write to memory, and it writes a nonzero value to > > rd. Regardless of success or failure, executing an SC.W instruction > > *invalidates any reservation held by this hart*. > > > > More details, ref: > > https://github.com/riscv/riscv-isa-manual > > I think section "3.5.3.2 Reservability PMA" [1] would be a more relevant > link, as this defines memory areas that either do or do not have > forward progress guarantees, including this part: > > "When LR/SC is used for memory locations marked RsrvNonEventual, > software should provide alternative fall-back mechanisms used when > lack of progress is detected." > > My reading of this is that if the example you tried stalls, then either > the PMA is not RsrvEventual, and it is wrong to rely on ll/sc on this, > or that the PMA is marked RsrvEventual but the implementation is > buggy. Yes, PMA just defines physical memory region attributes, But in our processor, when MMU is enabled (satp's value register > 2) in s-mode, it will look at our custom PTE's attributes BIT(63) ref [1]: PTE format: | 63 | 62 | 61 | 60 | 59 | 58-8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 SO C B SH SE RSW D A G U X W R V ^ ^ ^ ^ ^ BIT(63): SO - Strong Order BIT(62): C - Cacheable BIT(61): B - Bufferable BIT(60): SH - Shareable BIT(59): SE - Security So the memory also could be RsrvNone/RsrvEventual. [1] https://github.com/c-sky/csky-linux/commit/e837aad23148542771794d8a2fcc= 52afd0fcbf88 > > It also seems that the current "amoswap" based implementation > would be reliable independent of RsrvEventual/RsrvNonEventual. Yes, the hardware implementation of AMO could be different from LR/SC. AMO could use ACE snoop holding to lock the bus in hw coherency design, but LR/SC uses an exclusive monitor without locking the bus. > arm64 is already in the situation of having to choose between > two cmpxchg() implementation at runtime to allow falling back to > a slower but more general version, but it's best to avoid that if you > can. Current RISC-V needn't multiple versions to select, and all AMO & LR/SC has been defined in the spec. RISC-V hasn't CAS instructions, and it uses LR/SC for cmpxchg. I don't think LR/SC would be slower than CAS, and CAS is just good for code size. > > Arnd > > [1] http://www.five-embeddev.com/riscv-isa-manual/latest/machine.html#ato= micity-pmas -- Best Regards Guo Ren ML: https://lore.kernel.org/linux-csky/