Received: by 2002:a05:6a10:8c0a:0:0:0:0 with SMTP id go10csp2756729pxb; Tue, 19 Jan 2021 05:32:02 -0800 (PST) X-Google-Smtp-Source: ABdhPJzknwBJZEpXhyoxReaqzE3hhFCXM4Q8F7363HX8Q3Dx5u2DW8Gbr8XlSY3bYcOoncasSL6v X-Received: by 2002:aa7:cf90:: with SMTP id z16mr3412685edx.218.1611063122530; Tue, 19 Jan 2021 05:32:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611063122; cv=none; d=google.com; s=arc-20160816; b=rqbKHUlIWs2w+9bK35lxzeZgu1XTFzOeE4PgxbRCLXRpS+AwLaSY+13LtUs+GTXZql 6wk8q4p6dxkkDjaEKEgIOjEBLfhon3EUBm011jz39OW/MFxZDZUtTL4mVI4aemIDfYoq nSZBgae1qvRuK7CMYRffTlebFkGklwTnftlor4cZ2HHqEO0d1PTKLPeEUw6g9j1UmlXd ZF/WXRnH4j8f+7BlIdzsP/ZPiW9iCX1e8QigKBKLZFYCHXh+1+CouLrwuD1yOMc79pdY K4fnlBqP4ogUEm/tDGlyv50ZBSi+3/g6LxpLVAZk6zb52KdIpQo6D++Y54Gk450PQYIW kSUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=V0WyZ0QjeeZxPEOPKkoltDAJRX7zc5vT1YGzpNcNLdg=; b=kRvGHf/cH4JLe2/Vlef+wlrMEQx8opOffc3mk1+xezWRCL8Xm+xZOdNDMvFZXc/jzI G6Q3OXZ735eUnVzw1sDm+sDzhvnYZ6SaJyoui6yCh7p9rYgWAfWBTBv9pBHjR+untJxu m4de0XawYVl/yqeNOqaxclwv2a+7Wx0uJJtOh05x4ICVLalDLMM7xQS+FCllz4wxWuHW 46OGuCuAETv9Ji4YuwlLiPm8wDv7Y0D0ZWe50X048Q039bgY1nrA9cG7ioD2hI1vNq/s ecwzIwvkSDprTjXvwH5ndH+gDagZcNzKfPUzZWbf8I97//atzfXgxW+u7V1mdpP5G6J/ gKGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=LOKDOAdb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h10si2024096ejb.554.2021.01.19.05.31.39; Tue, 19 Jan 2021 05:32:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=LOKDOAdb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392704AbhASN2L (ORCPT + 99 others); Tue, 19 Jan 2021 08:28:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2392732AbhASNSf (ORCPT ); Tue, 19 Jan 2021 08:18:35 -0500 Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com [IPv6:2a00:1450:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76B72C0613D3 for ; Tue, 19 Jan 2021 05:17:42 -0800 (PST) Received: by mail-lj1-x234.google.com with SMTP id e7so21862906ljg.10 for ; Tue, 19 Jan 2021 05:17:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=V0WyZ0QjeeZxPEOPKkoltDAJRX7zc5vT1YGzpNcNLdg=; b=LOKDOAdbletZJxnT+nSaBkOgX3NZbBgWpZRAbqYlUul6ZCkyKN8zA/ia6Ds6vE5yoj rjts812XBS1aFJraX8QRugBXIJEBGr4LGwdRH+jybZVowZvtW78kbI7rfTy96mMLSrjK W3Y8Ev5DiFpdt/ZkLfMvVp5uAUaXzntEN1Q1i/pJCmp9HjUntST5omCO4tP+jC4hqdG/ xCHgoSSBEZu8vp+rszoxN06cH5TsSSI3sTfNeWTZpZcFVz83GQi02J6w6uamqUKyFu+U XlMupOFR8JDQmhHkZxcEo2OyGgaqizv6soDyTUexM9eL7n9wxRl9WWti86pE3KwXvcde 8R1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=V0WyZ0QjeeZxPEOPKkoltDAJRX7zc5vT1YGzpNcNLdg=; b=UY6xmGwoV+R6VoS1D8AocVSJ47Gf1lgX8uF16mE6Jqotsj5udgengkCgqJKhBjD4Cf s3qNSyjo8va/61Cd2iXH1Qdozyn1dlfxYLzQVji6EaCsvkIfL1XmtPF+wP/LW2A7OzmP ODr+zkDmHRlzL7D2ttD28IpYs/KhjBU9v1BgC+hXuzUFV0E4+ZE/JuVWH1SyVe2V4CSY nreOxTdWwsRKq6504Z3X2RT0xjWEu8FFbGvt5mF91Hky+4CpNSfrLE4QzmGPWWRcqkaF Q96hwz58haO8xTn2aCnAIQg8p+2szIxDvDsDPbM2EOSrdAuUxqffQBYgKOPX4OTHD18S 5H4Q== X-Gm-Message-State: AOAM533MQiIlO+0Cr7INU22oMPd1B90s5PJM33T1OPv0mB2fmd4ZuFF5 Yd/0yhDUpWyyNczFz4psOjtrEQqAUsWWFxLcLhg= X-Received: by 2002:a2e:90c8:: with SMTP id o8mr484635ljg.43.1611062260956; Tue, 19 Jan 2021 05:17:40 -0800 (PST) MIME-Version: 1.0 References: <20201109113233.9012-1-dbrazdil@google.com> <20201110101542.GA17572@infradead.org> In-Reply-To: From: Janne Karhunen Date: Tue, 19 Jan 2021 15:17:29 +0200 Message-ID: Subject: Re: [PATCH v1 00/24] Opt-in always-on nVHE hypervisor To: Marc Zyngier Cc: Christoph Hellwig , Lorenzo Pieralisi , Andrew Walbran , Catalin Marinas , Linux Kernel Mailing List , Will Deacon , Tejun Heo , Dennis Zhou , Christoph Lameter , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Nov 10, 2020 at 1:19 PM Marc Zyngier wrote: > > Why? I thought we were trying to kill nVHE off now that newer CPUs > > provide the saner virtualization extensions? > > We can't kill nVHE at all, because that is the only game in town. > You can't even buy a decent machine with VHE, no matter how much money > you put on the table. As I mentioned it earlier, we did this type of nVHE hypervisor and the proof of concept is here: https://github.com/jkrh/kvms See the README. It runs successfully on multiple pieces of arm64 hardware and provides a tiny QEMU based development environment via the makefiles for the QEMU 'max' CPU. The code is rough, the amount of man hours put to it is not sky high, but it does run. I'll update a new kernel patch to patches/ dir for one of the later kernels hopefully next week, up to now we have only supported kernels between 4.9 .... 5.6 as this is what our development hardware(s) run with. It requires a handful of hooks in the kvm code, but the actual kvm calls are just rerouted back to the kernel symbols. This way the hypervisor itself can be kept very tiny. The s2 page tables are fully owned by the hyp and the guests are unmapped from the host memory when configured with the option (we call it host blinding). Multiple VMs can be run without pinning them into the memory. It also provides a tiny out of tree driver prototype stub to protect the critical sections of the kernel memory beyond the kernel's own reach. There are still holes in the implementation such as the virtio-mapback handling via whitelisting and paging integrity checks, and many things are not quite all the way there yet. One step at a time. -- Janne