Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp4565753imu; Tue, 29 Jan 2019 03:55:43 -0800 (PST) X-Google-Smtp-Source: ALg8bN4XTZBFGAH3cob8iSPjYDXKQ/ZfrnSlcaxUk1+EMXGOWRFZWLe5WKhciVw72XD39TU0hd3Q X-Received: by 2002:a17:902:8a91:: with SMTP id p17mr25747086plo.316.1548762942978; Tue, 29 Jan 2019 03:55:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548762942; cv=none; d=google.com; s=arc-20160816; b=feEDhq5Tvg2cuJdt4r4TDSqQqms7cCsQB10mv3Q/eDP9BMJstk2X0fwTnxDPAuE8T2 6bUSJwNXEMw6kl+MouR3pF2WroDoJi7DyEUmSTM7O72jScZyrCMTW48KnmqHL9ypwzm4 ZrnwEDShebvKPGkPcQQ5ZBjTMOnpSSMYnZxtCQ2/7fqcLxtI3y/RskJSJRkKBHslIKJS q3bF6u/tc5UaKh2u9VoZwqPd7B86dwVd477FdSyF5E/46RYoN7Tl9zuXoAhXW2ydDbHu 1HzNVU/5h+hhKFCmsbcbYC/ypvgTeyaXF43Z2/yob7p17ELkIEIiIJ4dOdJHgEqiN0f7 94vQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=+9pmu4yIWmHA5vtzGFO1SUqPorErZxr8s9x7CgrdhXs=; b=yfToFEFU0VhX3CJrnkbF38G+ayBVjgskuGjZ9UZ1cuE8yKFI9E1xgzwCVip/uTD/Sv POOkRDBjePfDMFOsQCoVwJlYbCdpGMe52a6Wcc4aX3ojL7ToEXKKQeyf1/W1qvP2vkVz roF5qrlzzLR08m/3vV8t7esk4LcqPqS/qH0pmzkmoCC8IZWGiibh8giqDTiNO+zXx4wc RHKbncxRt9Z7nBcPVuhFxyOGP18Pth6dXz0G7YNQVsw1ERdru8gMYldmBZ3LOtrrhC29 jJ/w1MTSa39CMtl/gmL4GUhbTAAISTAUWgBKGVXphSuHGh6RcMTQ87Yy0tR9Yr/8k4Ko p9Cg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=I1RlaNE1; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a6si8073902pgc.137.2019.01.29.03.55.27; Tue, 29 Jan 2019 03:55:42 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=I1RlaNE1; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731954AbfA2LzO (ORCPT + 99 others); Tue, 29 Jan 2019 06:55:14 -0500 Received: from mail.kernel.org ([198.145.29.99]:42344 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730104AbfA2Lu4 (ORCPT ); Tue, 29 Jan 2019 06:50:56 -0500 Received: from localhost (5356596B.cm-6-7b.dynamic.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E3CD12086C; Tue, 29 Jan 2019 11:50:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1548762655; bh=9Mr8g9QAXKM2/pETEkwkeBwX0AsZUimbII2dKhnSitI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=I1RlaNE1QmJdEZS4s0TbPf8A1FXA5OB2PX0qxwjRjryNk7uUj2tIP1/gRtqOn0Kjo I3+uh5jV7ib/iVRnolyT4mDSvXVWPdkfTQvXFHuQxOAbv8q7TCE7EhT86bnXAI13+c ++bhZzCvTYAUpDyEYbve984mwq44r8t2NZOB6xIU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Eugeniy Paltsev , Vineet Gupta Subject: [PATCH 4.9 13/44] ARCv2: lib: memeset: fix doing prefetchw outside of buffer Date: Tue, 29 Jan 2019 12:36:08 +0100 Message-Id: <20190129113141.128924278@linuxfoundation.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190129113139.826927690@linuxfoundation.org> References: <20190129113139.826927690@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review X-Patchwork-Hint: ignore MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 4.9-stable review patch. If anyone has any objections, please let me know. ------------------ From: Eugeniy Paltsev commit e6a72b7daeeb521753803550f0ed711152bb2555 upstream. ARCv2 optimized memset uses PREFETCHW instruction for prefetching the next cache line but doesn't ensure that the line is not past the end of the buffer. PRETECHW changes the line ownership and marks it dirty, which can cause issues in SMP config when next line was already owned by other core. Fix the issue by avoiding the PREFETCHW Some more details: The current code has 3 logical loops (ignroing the unaligned part) (a) Big loop for doing aligned 64 bytes per iteration with PREALLOC (b) Loop for 32 x 2 bytes with PREFETCHW (c) any left over bytes loop (a) was already eliding the last 64 bytes, so PREALLOC was safe. The fix was removing PREFETCW from (b). Another potential issue (applicable to configs with 32 or 128 byte L1 cache line) is that PREALLOC assumes 64 byte cache line and may not do the right thing specially for 32b. While it would be easy to adapt, there are no known configs with those lie sizes, so for now, just compile out PREALLOC in such cases. Signed-off-by: Eugeniy Paltsev Cc: stable@vger.kernel.org #4.4+ Signed-off-by: Vineet Gupta [vgupta: rewrote changelog, used asm .macro vs. "C" macro] Signed-off-by: Greg Kroah-Hartman --- arch/arc/lib/memset-archs.S | 40 ++++++++++++++++++++++++++++++++-------- 1 file changed, 32 insertions(+), 8 deletions(-) --- a/arch/arc/lib/memset-archs.S +++ b/arch/arc/lib/memset-archs.S @@ -7,11 +7,39 @@ */ #include +#include -#undef PREALLOC_NOT_AVAIL +/* + * The memset implementation below is optimized to use prefetchw and prealloc + * instruction in case of CPU with 64B L1 data cache line (L1_CACHE_SHIFT == 6) + * If you want to implement optimized memset for other possible L1 data cache + * line lengths (32B and 128B) you should rewrite code carefully checking + * we don't call any prefetchw/prealloc instruction for L1 cache lines which + * don't belongs to memset area. + */ + +#if L1_CACHE_SHIFT == 6 + +.macro PREALLOC_INSTR reg, off + prealloc [\reg, \off] +.endm + +.macro PREFETCHW_INSTR reg, off + prefetchw [\reg, \off] +.endm + +#else + +.macro PREALLOC_INSTR +.endm + +.macro PREFETCHW_INSTR +.endm + +#endif ENTRY_CFI(memset) - prefetchw [r0] ; Prefetch the write location + PREFETCHW_INSTR r0, 0 ; Prefetch the first write location mov.f 0, r2 ;;; if size is zero jz.d [blink] @@ -48,11 +76,8 @@ ENTRY_CFI(memset) lpnz @.Lset64bytes ;; LOOP START -#ifdef PREALLOC_NOT_AVAIL - prefetchw [r3, 64] ;Prefetch the next write location -#else - prealloc [r3, 64] -#endif + PREALLOC_INSTR r3, 64 ; alloc next line w/o fetching + #ifdef CONFIG_ARC_HAS_LL64 std.ab r4, [r3, 8] std.ab r4, [r3, 8] @@ -85,7 +110,6 @@ ENTRY_CFI(memset) lsr.f lp_count, r2, 5 ;Last remaining max 124 bytes lpnz .Lset32bytes ;; LOOP START - prefetchw [r3, 32] ;Prefetch the next write location #ifdef CONFIG_ARC_HAS_LL64 std.ab r4, [r3, 8] std.ab r4, [r3, 8]