Received: by 2002:a25:f815:0:0:0:0:0 with SMTP id u21csp1527628ybd; Sun, 23 Jun 2019 09:37:22 -0700 (PDT) X-Google-Smtp-Source: APXvYqy19rq7Qo3yx2y4uulsTURBs989o8YBSxkvm4YBYB96m7TQonjDAo39gMeZLrTpv/gD9Izd X-Received: by 2002:a17:902:830c:: with SMTP id bd12mr66275896plb.237.1561307842691; Sun, 23 Jun 2019 09:37:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561307842; cv=none; d=google.com; s=arc-20160816; b=0yzz+EatJI6/yZxySanF+3/IiYRI3J2u1mg0A5Diz4JoNv07fQSgNBfYzHvk5MG7sZ yaZp+JtSsfSPrXvrm36pJVZxXFlBdUhVzfIkJvEFCjs27PlT9Wla5Pk0KG7S2KdndUFu gF9KhPcwn/rZPxZ+7EsTtH+uPvBTSR16cdEdyHFabIxGD2rJ0nebGe0oT2uZ5GM8+4Dy NvBoqKGzngjzrySCc/GUGiSLg00UWZ9LA08hs00HOkAGEee+dAKZOP8nE7cG94bEG8kW 2cx3sqrznOB5nXQt5S3z/QJmkI1paW+eztFDno5Hm9EwN9T7y0UMrDBLoRyAePe2nNxj K9Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=LjKpAzXUJiisUIZvu+C9T81E/PwzuFnRtpEl/CRPLkI=; b=wTbDVT9khy32mfqUKMCHgzlJJBXADxKi+P9MGjWPrJMoa5Nv0+ND95d+DzRUMJNrUA btewD2Rrt68w8rSEj7H9/IGeXOmJ96gx2sWn+pOGC+8IHAAcronEZyzEJhfDI/Hw2sxE paYy/jbR+h07RT+O5pF3TVDeaW7w38R8b2k633pcAIbzGDBmzNXW5h62JKXehjJoYHF0 4Hl8yQJQ6NN8k088L/z8isCBQNljdRxdXHA2Oic+RPM7mc4r24Rs8HXIgJLub8+bwSMG eHYqIoubURcYj479aGFnd5rS7vssfPGihy0FsXMrRwF6o0CX08Os2RR6jZBwVyOAzPmK 7xJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=WoTik3i7; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w6si7471728pgs.103.2019.06.23.09.36.55; Sun, 23 Jun 2019 09:37:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=WoTik3i7; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726688AbfFWQfu (ORCPT + 99 others); Sun, 23 Jun 2019 12:35:50 -0400 Received: from mail.kernel.org ([198.145.29.99]:49546 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726399AbfFWQfu (ORCPT ); Sun, 23 Jun 2019 12:35:50 -0400 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B807421530 for ; Sun, 23 Jun 2019 16:35:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1561307749; bh=f4wzwRqVtnbeGMbGLTUq9DbopFJOpD4awTGdr8HMu0c=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=WoTik3i7uVOtG9x3r9i96Ag9GXjA5+USAyJht8X/qBPra0PBApIG8JI9f6E8NDiSi CHO/c4Hsn/R99/LCKEtYzlnwWDJLEtQdF65WJ9f4axQZIsz8xhsXyuPzsiaPPUciq0 ifLAnqxrMbQFdaTEKtPF9Oo8vcgcRee+CxrRPVIA= Received: by mail-wm1-f43.google.com with SMTP id s15so10958620wmj.3 for ; Sun, 23 Jun 2019 09:35:48 -0700 (PDT) X-Gm-Message-State: APjAAAXfKT9yECyBAsCdGRywQ5rLAGir3d0d7RiOUX0V6gX4VscGcorV EeWkUg9Syd/z7XOfn63Z9dfhx3HYkFLeEn9AMUM= X-Received: by 2002:a7b:cd84:: with SMTP id y4mr12259887wmj.79.1561307747231; Sun, 23 Jun 2019 09:35:47 -0700 (PDT) MIME-Version: 1.0 References: <20190321163623.20219-1-julien.grall@arm.com> <20190321163623.20219-12-julien.grall@arm.com> <0dfe120b-066a-2ac8-13bc-3f5a29e2caa3@arm.com> <20190621141606.GF18954@arrakis.emea.arm.com> In-Reply-To: <20190621141606.GF18954@arrakis.emea.arm.com> From: Guo Ren Date: Mon, 24 Jun 2019 00:35:35 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file To: Catalin Marinas Cc: Julien Grall , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, aou@eecs.berkeley.edu, gary@garyguo.net, Atish Patra , hch@infradead.org, paul.walmsley@sifive.com, rppt@linux.ibm.com, linux-riscv@lists.infradead.org, Anup Patel , Palmer Dabbelt , suzuki.poulose@arm.com, Marc Zyngier , julien.thierry@arm.com, Will Deacon , christoffer.dall@arm.com, james.morse@arm.com Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Thx Catalin, On Fri, Jun 21, 2019 at 10:16 PM Catalin Marinas wrote: > > On Wed, Jun 19, 2019 at 07:51:03PM +0800, Guo Ren wrote: > > On Wed, Jun 19, 2019 at 4:54 PM Julien Grall wrote: > > > On 6/19/19 9:07 AM, Guo Ren wrote: > > > > Move arm asid allocator code in a generic one is a agood idea, I've > > > > made a patchset for C-SKY and test is on processing, See: > > > > https://lore.kernel.org/linux-csky/1560930553-26502-1-git-send-email-guoren@kernel.org/ > > > > > > > > If you plan to seperate it into generic one, I could co-work with you. > > > > > > Was the ASID allocator work out of box on C-Sky? > > > > Almost done, but one question: > > arm64 remove the code in switch_mm: > > cpumask_clear_cpu(cpu, mm_cpumask(prev)); > > cpumask_set_cpu(cpu, mm_cpumask(next)); > > > > Why? Although arm64 cache operations could affect all harts with CTC > > method of interconnect, I think we should keep these code for > > primitive integrity in linux. Because cpu_bitmap is in mm_struct > > instead of mm->context. > > We didn't have a use for this in the arm64 code, so no point in > maintaining the mm_cpumask. On some arm32 systems (ARMv6) with no > hardware broadcast of some TLB/cache operations, we use it to track > where the task has run to issue IPI for TLB invalidation or some > deferred I-cache invalidation. The operation of set/clear mm_cpumask was removed in arm64 compared to arm32. It seems no side effect on current arm64 system, but from software meaning it's wrong. I think we should keep mm_cpumask just like arm32. > > (there was also a potential optimisation on arm64 to avoid broadcast > TLBI if the task only ran on a single CPU but Will found that was rarely > the case on an SMP system because of rebalancing happening during > execve(), ending up with two bits set in the mm_cpumask) > > The way you use it on csky is different from how it is done on arm. It > seems to clear the mask for the scheduled out (prev) task but this > wouldn't work on arm(64) since the TLB still contains prev entries > tagged with the scheduled out ASID. Whether it matters, I guess it > depends on the specifics of your hardware. Sorry for the mistake quote, what I mean is what is done in arm32: clear all bits of mm->cpu_mask in new_context(), and set back one by one. Here is my patch: https://lore.kernel.org/linux-csky/CAJF2gTQ0xQtQY1t-g9FgWaxfDXppMkFooCQzTFy7+ouwUfyA6w@mail.gmail.com/T/#m2ed464d2dfb45ac6f5547fb3936adf2da456cb65 > > While the algorithm may seem fairly generic, the semantics have a few > corner cases specific to each architecture. See [1] for a description of > the semantics we need on arm64 (CnP is a feature where the hardware > threads of the same core can share the TLB; the original algorithm > violated the requirements when this feature was enabled). C-SKY SMP is only one hart per core, but here is a patch [1] with my thought on SMT duplicate tlb flush: [1] https://lore.kernel.org/linux-csky/1561305869-18872-1-git-send-email-guoren@kernel.org/T/#u For TLA+ model, I still need some learning before I could talk with you. > > BTW, if you find the algorithm fairly straightforward ;), see this > bug-fix which took a formal model to identify: a8ffaaa060b8 ("arm64: > asid: Do not replace active_asids if already 0"). I think it's one fo the cases that other archs also could get benefit from arm's asid allocator code. Btw, Is this detected by arm's aisd allocator TLA+ model ? Or a real bug report ? -- Best Regards Guo Ren ML: https://lore.kernel.org/linux-csky/