Received: by 10.192.165.148 with SMTP id m20csp1426904imm; Wed, 2 May 2018 22:09:26 -0700 (PDT) X-Google-Smtp-Source: AB8JxZroK/TXLe/XhqGyHKK/+DbLSlQdU+Y/oUmY9ujrtkEUSZw90ulbInTEMwxOYCIZ8JDN2V/F X-Received: by 2002:a63:3286:: with SMTP id y128-v6mr18603940pgy.419.1525324166238; Wed, 02 May 2018 22:09:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525324166; cv=none; d=google.com; s=arc-20160816; b=faIeGTPpje2HIJSIdn/gFr5I+m/UsCq2yzxupBPN/r1n1R2kzGNKc+cmRnHOEc7uWo BcTNJjo+ylFAJuubcBSVBFK5oyZCxR30LzpCAwBKEWjJrF1kAiAdVjjxje+P7Y1fGlIF ERWrALXAClkiDqtQg/2fpzj4l5YTILhZZVoJVdHMf7ikQni36CrPGGZ9F8CmZhCV3nzx t7QWLrMktdGEGXVkk30YSSK5CQlqNLwZ40QWLWpiUrg+5nNY8SYyUT2HrjEM0ZGUYw9s pbuPjMMwZc/7BW9wvy2Ai52OvHlFnCOCAgbmoxvEiVuD4caRVs8CWHth6imv8hHou7Yx yZCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:cc:to:from :subject:arc-authentication-results; bh=oYXRIv9brTUAGK3dKfjAkRKJMovgYwFuweprsYHJ7zE=; b=ng8yBko7CVxIj6G7HpbGMcQYbtDIXU1KxcJEgMFCubTpvT2bNhifUfnDT85AcFMyja MAEpWvavMLVROriMAH0wez1c18RRuFIdIDMmjTaYxzKkwh5ZDfX85qAi7fvEltlB9QL3 RfKOgFWmL8RV8ZGiykMi6Meqx6IEYrVZ8SHx/QGmSp9opPHWiiKLz3NlSc+ZAfatA8bh KxrPYPiPVsWks+Ntip+LsgDSmQA7j4nueph5MVtK8Mzrg+Rp+bknUEWnoOCd+0OUeWif yS1xpTfTupUX6nCrUJwvjNeQt2W/CyfQjS4s2qN8mc1VtNubwBB7lbcvTtYZYFNOmmGf MNrw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 92-v6si12712054pli.455.2018.05.02.22.09.12; Wed, 02 May 2018 22:09:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752211AbeECFJC (ORCPT + 99 others); Thu, 3 May 2018 01:09:02 -0400 Received: from mga03.intel.com ([134.134.136.65]:32039 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751765AbeECFI7 (ORCPT ); Thu, 3 May 2018 01:08:59 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 May 2018 22:08:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,356,1520924400"; d="scan'208";a="221240661" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga005.jf.intel.com with ESMTP; 02 May 2018 22:08:58 -0700 Subject: [PATCH v2 4/9] x86, memcpy_mcsafe: add write-protection-fault handling From: Dan Williams To: linux-nvdimm@lists.01.org Cc: x86@kernel.org, Ingo Molnar , Borislav Petkov , Tony Luck , Al Viro , Thomas Gleixner , Andy Lutomirski , Peter Zijlstra , Andrew Morton , Linus Torvalds , hch@lst.de, linux-kernel@vger.kernel.org, tony.luck@intel.com, linux-fsdevel@vger.kernel.org Date: Wed, 02 May 2018 21:59:02 -0700 Message-ID: <152532354268.17218.11923624861859552990.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <152532351517.17218.3583455156840230837.stgit@dwillia2-desk3.amr.corp.intel.com> References: <152532351517.17218.3583455156840230837.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for using memcpy_mcsafe() to handle user copies it needs to be to handle write-protection faults while writing user pages. Add MMU-fault handlers alongside the machine-check exception handlers. Note that the machine check fault exception handling makes assumptions about source buffer alignment and poison alignment. In the write fault case, given the destination buffer is arbitrarily aligned, it needs a separate / additional fault handling approach. The mcsafe_handle_tail() helper is reused. The @limit argument is set to @len since there is no safety concern about retriggering an MMU fault, and this simplifies the assembly. Cc: Cc: Ingo Molnar Cc: Borislav Petkov Cc: Tony Luck Cc: Al Viro Cc: Thomas Gleixner Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Andrew Morton Cc: Linus Torvalds Co-developed-by: Tony Luck Signed-off-by: Dan Williams --- arch/x86/include/asm/uaccess_64.h | 3 +++ arch/x86/lib/memcpy_64.S | 14 ++++++++++++++ arch/x86/lib/usercopy_64.c | 17 +++++++++++++++++ 3 files changed, 34 insertions(+) diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index 62546b3a398e..c63efc07891f 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -194,4 +194,7 @@ __copy_from_user_flushcache(void *dst, const void __user *src, unsigned size) unsigned long copy_user_handle_tail(char *to, char *from, unsigned len); +unsigned long +mcsafe_handle_tail(char *to, char *from, unsigned len); + #endif /* _ASM_X86_UACCESS_64_H */ diff --git a/arch/x86/lib/memcpy_64.S b/arch/x86/lib/memcpy_64.S index f01a88391c98..c3b527a9f95d 100644 --- a/arch/x86/lib/memcpy_64.S +++ b/arch/x86/lib/memcpy_64.S @@ -265,9 +265,23 @@ EXPORT_SYMBOL_GPL(__memcpy_mcsafe) mov %ecx, %eax ret + /* + * For write fault handling, given the destination is unaligned, + * we handle faults on multi-byte writes with a byte-by-byte + * copy up to the write-protected page. + */ +.E_write_words: + shll $3, %ecx + addl %edx, %ecx + movl %ecx, %edx + jmp mcsafe_handle_tail + .previous _ASM_EXTABLE_FAULT(.L_read_leading_bytes, .E_leading_bytes) _ASM_EXTABLE_FAULT(.L_read_words, .E_read_words) _ASM_EXTABLE_FAULT(.L_read_trailing_bytes, .E_trailing_bytes) + _ASM_EXTABLE(.L_write_leading_bytes, .E_leading_bytes) + _ASM_EXTABLE(.L_write_words, .E_write_words) + _ASM_EXTABLE(.L_write_trailing_bytes, .E_trailing_bytes) #endif diff --git a/arch/x86/lib/usercopy_64.c b/arch/x86/lib/usercopy_64.c index 75d3776123cc..9787f5ee0cf9 100644 --- a/arch/x86/lib/usercopy_64.c +++ b/arch/x86/lib/usercopy_64.c @@ -75,6 +75,23 @@ copy_user_handle_tail(char *to, char *from, unsigned len) return len; } +/* + * Similar to copy_user_handle_tail, probe for the write fault point, + * but reuse __memcpy_mcsafe in case a new read error is encountered. + * clac() is handled in _copy_to_iter_mcsafe(). + */ +__visible unsigned long +mcsafe_handle_tail(char *to, char *from, unsigned len) +{ + for (; len; --len, to++) { + unsigned long rem = memcpy_mcsafe(to, from, 1); + + if (rem) + break; + } + return len; +} + #ifdef CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE /** * clean_cache_range - write back a cache range with CLWB