Received: by 2002:ac0:a594:0:0:0:0:0 with SMTP id m20-v6csp4658554imm; Mon, 14 May 2018 10:41:55 -0700 (PDT) X-Google-Smtp-Source: AB8JxZr0RCQnaEvnd6VQRSfUx43OiVAt7XkhZvvX+HQ11CfjOZj70Vgo6TgcN80LOB7YfuZu2zBI X-Received: by 2002:a17:902:bc4a:: with SMTP id t10-v6mr10844494plz.343.1526319715284; Mon, 14 May 2018 10:41:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526319715; cv=none; d=google.com; s=arc-20160816; b=v5MSQbYEXCpgO+UNwq8ZQCbOkmtcPq1+6IWAhVFGl6NaUMUxZ9JPhT7yqyS0y9K035 /qltE6BfgoVr8T4h1+foLgJ4QnJWkx2eesoCuXaPQJb8FySn3kOB5dZnURP3DaaABlYj t4twxLahpJSKsylEA4SLEONM7yA+o8cD8VqHXA1MXeAXNTICJ+qRR5MC3CUC2p9WJdKH InL7tlDKR2/gGIeMJyo4veIXIycFotcbg7iLmJbveRY7yEMZ47CirfnIaC8s+4ZUgIzW vog8R9FNLHbOrMpVTU46Vn7HGjhIG62KNVLgXhBGD5WFpqYN9Ehwtp4IHfe8qgPXSRNI Mwvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature:arc-authentication-results; bh=MMrK4fGIj0q5WTv4AzAHJ94DDvw6hxE9MbUA/rxcKLo=; b=rmEBTe2cahN6VTJKj3VaW6KxGKJt2qYOcFHqfbdnYC5VtRPKqkPCe5ITzyXxGJO9Ir SnOqn7ZWOVvXwUDBdFKLyc29tFG9TytJP6eS3ptaBofwc6jqRLw/6uOlQKembgXW1yR0 p+7hZf3HjUmCwZ++JH16VE3jF+2SbWFsPe7nc4hYQgbIq4/h0m4JN7VwXSLrk9MIrgBQ Fmu7Ig5Eta3DmdGfk2rbJDt5pmzo/ElbUYUOi/Vt5/pQLDgP3nECdhf7N1ZH6+oYzEYh AaBiuyLgykd8ab340L2MjvVEOZe14gTAdshEj0uaA7KJ22AqKGYmMFlPNcgEqiR6U+AX kaxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=EByZvsUu; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i65-v6si10553397pfb.343.2018.05.14.10.41.40; Mon, 14 May 2018 10:41:55 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=EByZvsUu; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752866AbeENRZS (ORCPT + 99 others); Mon, 14 May 2018 13:25:18 -0400 Received: from mail-pl0-f66.google.com ([209.85.160.66]:41134 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752828AbeENRZM (ORCPT ); Mon, 14 May 2018 13:25:12 -0400 Received: by mail-pl0-f66.google.com with SMTP id az12-v6so7751288plb.8; Mon, 14 May 2018 10:25:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=MMrK4fGIj0q5WTv4AzAHJ94DDvw6hxE9MbUA/rxcKLo=; b=EByZvsUu/a91BMrpUGeK4miapMXV/iz0Qy7PBWgIpO7/K0uMUupe3AQ58gRmbx9GF4 19Jwayimxr45jwaeKQY2jr7S3yKj+DoLiTqkY6iggLmWiBYt57DCkOHJjb6ckykM92qu cE1+YazNkbYvTtEVF2OxRYOUiIRdc6rWHQkDJJW9f08MDoWQkHPKU9EttCFLajZLmIHs llb6/I/ARyb2Idd0OhFO8tf50hGVMapUB5f9+tLohzsma2LWYm4m245i7yRtoOYwJdNh BTR+cceAzq4zCqMOkhaLiB78khSvKkbpgqGBB2tI5kw5tIr7ydfC6LxblZRf2OvWyENX 5hPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=MMrK4fGIj0q5WTv4AzAHJ94DDvw6hxE9MbUA/rxcKLo=; b=MYWpgAoCXCxlLwkzevT/GyJSYd20FRUFXJHM6jQSviVdJK93hn6gBtcHVmwUeCg9UO +aBvgXS5K/QfC6RpHqAAEwmiFTZG5GZpgIrK22VUUe6W1x989w/CMWmfNsrK7X2QdADJ 2M9OpaMEEWuHHSzXsK1gkkS1ozczz+RFGdiRrfz3xr7s/sAxb5HUMf3boEZ23Vu7t0TG s6LdeC7oxCXFpyrhrIi6oSjfUAE42p5gCDLAV4fZmxGbWQyjysUFgxDAzL6AWrZ64gel 9xYzpkEmQUtndo1IwIpgx+F1P4wDnqOZWfAxEKDk4AIp1rDhNV9ESi0b1gWCNNiTiRfZ YpKQ== X-Gm-Message-State: ALKqPwcADHQRwoRjV2oUt+AxioGgUtZr9iypsOHqPXk7X0onCPdGFgGR 0B0PnkHnIgrrmkM/4w9J++w= X-Received: by 2002:a17:902:548f:: with SMTP id e15-v6mr11125256pli.314.1526318710867; Mon, 14 May 2018 10:25:10 -0700 (PDT) Received: from gmail.com ([2620:15c:17:3:dc28:5c82:b905:e8a8]) by smtp.gmail.com with ESMTPSA id r76-v6sm19299789pfl.1.2018.05.14.10.25.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 14 May 2018 10:25:10 -0700 (PDT) Date: Mon, 14 May 2018 10:25:08 -0700 From: Eric Biggers To: Dmitry Vyukov Cc: KVM list , karahmed@amazon.de, the arch/x86 maintainers , LKML Subject: Re: CONFIG_KCOV causing crash in svm_vcpu_run() Message-ID: <20180514172508.GC252575@gmail.com> References: <20180514030007.GH677@sol.localdomain> <20180514030205.GI677@sol.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.2 (2017-12-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, May 14, 2018 at 07:14:41AM +0200, Dmitry Vyukov wrote: > On Mon, May 14, 2018 at 5:02 AM, Eric Biggers wrote: > > Sorry, messed up address for KVM mailing list. See message below. > > > > On Sun, May 13, 2018 at 08:00:07PM -0700, Eric Biggers wrote: > >> With CONFIG_KCOV=y and an AMD processor, running the following program crashes > >> the kernel with no output (I'm testing in a VM, so it's using nested > >> virtualization): > >> > >> #include > >> #include > >> #include > >> > >> int main() > >> { > >> int dev, vm, cpu; > >> char page[4096] __attribute__((aligned(4096))) = { 0 }; > >> struct kvm_userspace_memory_region memreg = { > >> .memory_size = 4096, > >> .userspace_addr = (unsigned long)page, > >> }; > >> dev = open("/dev/kvm", O_RDONLY); > >> vm = ioctl(dev, KVM_CREATE_VM, 0); > >> cpu = ioctl(vm, KVM_CREATE_VCPU, 0); > >> ioctl(vm, KVM_SET_USER_MEMORY_REGION, &memreg); > >> ioctl(cpu, KVM_RUN, 0); > >> } > >> > >> It bisects down to commit b2ac58f90540e39 ("KVM/SVM: Allow direct access to > >> MSR_IA32_SPEC_CTRL"). The bug is apparently that due to the new code for > >> managing the SPEC_CTRL MSR, __sanitizer_cov_trace_pc() is being called from > >> svm_vcpu_run() before the host's MSR_GS_BASE has been restored, which causes a > >> crash somehow. The following patch fixes it, though I don't know that it's the > >> right solution; maybe KCOV should be disabled in the function instead, or maybe > >> there's a more fundamental problem. What do people think? > > > If __sanitizer_cov_trace_pc() crashes, I would expect there must be > few more of them here: > > if (unlikely(!msr_write_intercepted(vcpu, MSR_IA32_SPEC_CTRL))) > svm->spec_ctrl = native_read_msr(MSR_IA32_SPEC_CTRL); > > if (svm->spec_ctrl) > native_wrmsrl(MSR_IA32_SPEC_CTRL, 0); > > Compiler inserts these callbacks into every basic block/edge.. Aren't there? > > Unfortunately we don't have an attribute that disables instrumentation > of a single function. This is currently possible only on file level. > Yes, due to the code dealing with MSR_IA32_SPEC_CTRL, there were several calls to __sanitizer_cov_trace_pc() before the write to MSR_GS_BASE. The patch I tested moves the write to MSR_GS_BASE to before all of them, so it's once again the first thing after the asm block. Again I'm not sure it's the proper solution, but it did make it stop crashing. Also I'm guessing this isn't specific to nested virtualization; I just didn't have KCOV enabled on the host, thus the host didn't crash. - Eric