Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   "Michael Kelley (EOSG)" <Michael.H.Kelley@microsoft.com>
To:     KY Srinivasan <kys@microsoft.com>,
        "x86@kernel.org" <x86@kernel.org>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "devel@linuxdriverproject.org" <devel@linuxdriverproject.org>,
        "olaf@aepfle.de" <olaf@aepfle.de>,
        "apw@canonical.com" <apw@canonical.com>,
        "jasowang@redhat.com" <jasowang@redhat.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "hpa@zytor.com" <hpa@zytor.com>,
        Stephen Hemminger <sthemmin@microsoft.com>,
        "vkuznets@redhat.com" <vkuznets@redhat.com>
Subject: RE: [PATCH 1/5] X86: Hyper-V: Enlighten APIC access
Thread-Topic: [PATCH 1/5] X86: Hyper-V: Enlighten APIC access
Thread-Index: AQHT3MEtyTg14b6f+k6LXXaM8ySPxaQTm/sQ
Date:   Thu, 26 Apr 2018 22:15:37 +0000
Message-ID: <DM5PR2101MB1030CCA0F5568CDED2695C58DC8E0@DM5PR2101MB1030.namprd21.prod.outlook.com>
References: <20180425181110.8683-1-kys@linuxonhyperv.com>
 <20180425181250.8740-1-kys@linuxonhyperv.com>
In-Reply-To: <20180425181250.8740-1-kys@linuxonhyperv.com>
Accept-Language: en-US
Content-Language: en-US
msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True;
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Owner=mikelley@ntdev.microsoft.com;
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2018-04-26T22:15:35.8029946Z;
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=General;
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Application=Microsoft Azure
 Information Protection;
 MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Extended_MSFT_Method=Automatic;
 Sensitivity=General
x-originating-ip: [24.22.167.197]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1;DM5PR2101MB1030;7:Q6ZClaCTVOI/Y3uUQDg7HbfbpzuwzsMYw2TwnK08eFQpv1ireLFKwCwhJeyOLHCiW1o76npguHijuWGvs9Ca/46a3HcPXMjo55M8bPFc8ehdwOJ8lGQubQzhsN/eF3CgqA9hLeBIRI1ulyIoPvj9a+IQFtw6P7AR4rHenlRmcvlWNccnBK4l6M9EmtRSQurJPPNFmm9Mpd0rMU/gvsIAx4hXHI01tZlkefHcdTXV8N+UObBSQJAtJDutMJ0djNsF;20:aYlNcfAfgAmQGrJRbAefm9aPPgUlkigmGZzesxVkzeeL5zxW516uUZ4ORcr2okLyhohHGjn2mESFdWQXqtclMPRog6yMosrY+f9Qvy5Sc7CVsf0xM70WUoL7isvlo4IMOW4p01jYCGiEgI4fT09Ia9an2jB7qZmPmhIT+oVUGQE=
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(48565401081)(5600026)(2017052603328)(7193020);SRVR:DM5PR2101MB1030;
x-ms-traffictypediagnostic: DM5PR2101MB1030:
authentication-results: spf=none (sender IP is )
 smtp.mailfrom=Michael.H.Kelley@microsoft.com; 
x-microsoft-antispam-prvs: <DM5PR2101MB1030F708F363A3BD583D3E6BDC8E0@DM5PR2101MB1030.namprd21.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(28532068793085)(89211679590171)(9452136761055)(198206253151910);
x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3231232)(944501410)(52105095)(3002001)(6055026)(6041310)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011);SRVR:DM5PR2101MB1030;BCL:0;PCL:0;RULEID:;SRVR:DM5PR2101MB1030;
x-forefront-prvs: 0654257CF5
x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(396003)(39380400002)(346002)(39860400002)(376002)(199004)(189003)(13464003)(97736004)(3280700002)(476003)(575784001)(86362001)(486006)(11346002)(446003)(3660700001)(2501003)(5250100002)(81166006)(8676002)(2906002)(110136005)(8990500004)(316002)(22452003)(106356001)(305945005)(478600001)(8936002)(14454004)(229853002)(72206003)(6436002)(81156014)(26005)(102836004)(105586002)(74316002)(2201001)(25786009)(7736002)(66066001)(9686003)(5660300001)(33656002)(59450400001)(99286004)(7416002)(1511001)(53546011)(6506007)(55016002)(53936002)(2900100001)(10290500003)(6246003)(6116002)(3846002)(10090500001)(68736007)(86612001)(76176011)(7696005)(921003)(2004002)(1121003);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR2101MB1030;H:DM5PR2101MB1030.namprd21.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1;
received-spf: None (protection.outlook.com: microsoft.com does not designate
 permitted sender hosts)
x-microsoft-antispam-message-info: OyTJKEE0oP+VGW9QLw1mY1Wp+OVQ6kj6JnQbXdvZDyCDZ9ZtkXlOFw1GMgmaKijuxJNlaayhat9o3ctCa9FMilb0FLdqMbynINGJr/sV4oitAFiwohBBaY5ZAs2FfRneGBjfxiYuIPQ7y9WVhH5rGTNxoSwHG7j4dlVSuorT3mLaJZwZhhjzUh+U8N1XKi3X
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Office365-Filtering-Correlation-Id: 28d9cd14-c590-4955-103c-08d5abc33cfd
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 28d9cd14-c590-4955-103c-08d5abc33cfd
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Apr 2018 22:15:37.1108
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR2101MB1030
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> -----Original Message-----
> From: kys@linuxonhyperv.com <kys@linuxonhyperv.com>
> Sent: Wednesday, April 25, 2018 11:13 AM
> To: x86@kernel.org; gregkh@linuxfoundation.org; linux-kernel@vger.kernel.=
org;
> devel@linuxdriverproject.org; olaf@aepfle.de; apw@canonical.com; jasowang=
@redhat.com;
> tglx@linutronix.de; hpa@zytor.com; Stephen Hemminger <sthemmin@microsoft.=
com>;
> Michael Kelley (EOSG) <Michael.H.Kelley@microsoft.com>; vkuznets@redhat.c=
om
> Cc: KY Srinivasan <kys@microsoft.com>
> Subject: [PATCH 1/5] X86: Hyper-V: Enlighten APIC access
>=20
> From: "K. Y. Srinivasan" <kys@microsoft.com>
>=20
> Hyper-V supports MSR based APIC access; implement
> the enlightenment.
>=20
> Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
> ---
>  arch/x86/hyperv/Makefile        |  2 +-
>  arch/x86/hyperv/hv_apic.c       | 98 +++++++++++++++++++++++++++++++++++=
++++++
>  arch/x86/hyperv/hv_init.c       |  5 ++-
>  arch/x86/include/asm/mshyperv.h |  6 ++-
>  4 files changed, 107 insertions(+), 4 deletions(-)
>  create mode 100644 arch/x86/hyperv/hv_apic.c
>=20
> diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
> index 367a8203cfcf..00ce4df01a09 100644
> --- a/arch/x86/hyperv/Makefile
> +++ b/arch/x86/hyperv/Makefile
> @@ -1 +1 @@
> -obj-y		:=3D hv_init.o mmu.o
> +obj-y		:=3D hv_init.o mmu.o hv_apic.o
> diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
> new file mode 100644
> index 000000000000..e0a5b36208fc
> --- /dev/null
> +++ b/arch/x86/hyperv/hv_apic.c
> @@ -0,0 +1,98 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +/*
> + * Hyper-V specific APIC code.
> + *
> + * Copyright (C) 2018, Microsoft, Inc.
> + *
> + * Author : K. Y. Srinivasan <kys@microsoft.com>
> + *
> + * This program is free software; you can redistribute it and/or modify =
it
> + * under the terms of the GNU General Public License version 2 as publis=
hed
> + * by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
> + * NON INFRINGEMENT.  See the GNU General Public License for more
> + * details.
> + *
> + */
> +
> +#include <linux/types.h>
> +#include <asm/hypervisor.h>
> +#include <asm/mshyperv.h>
> +#include <linux/version.h>
> +#include <linux/vmalloc.h>
> +#include <linux/mm.h>
> +#include <linux/clockchips.h>
> +#include <linux/hyperv.h>
> +#include <linux/slab.h>
> +#include <linux/cpuhotplug.h>
> +
> +static u64 hv_apic_icr_read(void)
> +{
> +	u64 reg_val;
> +
> +	rdmsrl(HV_X64_MSR_ICR, reg_val);
> +	return reg_val;
> +}
> +
> +static void hv_apic_icr_write(u32 low, u32 id)
> +{
> +	u64 reg_val;
> +
> +	reg_val =3D SET_APIC_DEST_FIELD(id);
> +	reg_val =3D reg_val << 32;
> +	reg_val |=3D low;
> +
> +	wrmsrl(HV_X64_MSR_ICR, reg_val);
> +}
> +
> +static u32 hv_apic_read(u32 reg)
> +{
> +	u32 reg_val, hi;
> +
> +	switch (reg) {
> +	case APIC_EOI:
> +		rdmsr(HV_X64_MSR_EOI, reg_val, hi);
> +		return reg_val;
> +	case APIC_TASKPRI:
> +		rdmsr(HV_X64_MSR_TPR, reg_val, hi);
> +		return reg_val;
> +
> +	default:
> +		return native_apic_mem_read(reg);
> +	}
> +}
> +
> +static void hv_apic_write(u32 reg, u32 val)
> +{
> +	switch (reg) {
> +	case APIC_EOI:
> +		wrmsr(HV_X64_MSR_EOI, val, 0);
> +		break;
> +	case APIC_TASKPRI:
> +		wrmsr(HV_X64_MSR_TPR, val, 0);
> +		break;
> +	default:
> +		native_apic_mem_write(reg, val);
> +	}
> +}
> +
> +static void hv_apic_eoi_write(u32 reg, u32 val)
> +{
> +	wrmsr(HV_X64_MSR_EOI, val, 0);
> +}
> +
> +void __init hv_apic_init(void)
> +{
> +	if (ms_hyperv.hints & HV_X64_APIC_ACCESS_RECOMMENDED) {
> +		pr_info("Hyper-V: Using MSR ased APIC access\n");

Typo here.  "ased" should be "based".

> +		apic_set_eoi_write(hv_apic_eoi_write);
> +		apic->read      =3D hv_apic_read;
> +		apic->write     =3D hv_apic_write;
> +		apic->icr_write =3D hv_apic_icr_write;
> +		apic->icr_read  =3D hv_apic_icr_read;
> +	}
> +}
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index cfecc2272f2d..71e50fc2b7ef 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -242,8 +242,9 @@ static int hv_cpu_die(unsigned int cpu)
>   *
>   * 1. Setup the hypercall page.
>   * 2. Register Hyper-V specific clocksource.
> + * 3. Setup Hyper-V specific APIC entry points.
>   */
> -void hyperv_init(void)
> +void __init hyperv_init(void)
>  {
>  	u64 guest_id, required_msrs;
>  	union hv_x64_msr_hypercall_contents hypercall_msr;
> @@ -298,6 +299,8 @@ void hyperv_init(void)
>=20
>  	hyper_alloc_mmu();
>=20
> +	hv_apic_init();
> +
>  	/*
>  	 * Register Hyper-V specific clocksource.
>  	 */
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyp=
erv.h
> index b90e79610cf7..bcced50037c1 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -258,7 +258,7 @@ static inline int hv_cpu_number_to_vp_number(int cpu_=
number)
>  	return hv_vp_index[cpu_number];
>  }
>=20
> -void hyperv_init(void);
> +void __init hyperv_init(void);
>  void hyperv_setup_mmu_ops(void);
>  void hyper_alloc_mmu(void);
>  void hyperv_report_panic(struct pt_regs *regs, long err);
> @@ -269,14 +269,16 @@ void hyperv_reenlightenment_intr(struct pt_regs *re=
gs);
>  void set_hv_tscchange_cb(void (*cb)(void));
>  void clear_hv_tscchange_cb(void);
>  void hyperv_stop_tsc_emulation(void);
> +void hv_apic_init(void);
>  #else /* CONFIG_HYPERV */
> -static inline void hyperv_init(void) {}
> +static __init  inline void hyperv_init(void) {}
>  static inline bool hv_is_hyperv_initialized(void) { return false; }
>  static inline void hyperv_cleanup(void) {}
>  static inline void hyperv_setup_mmu_ops(void) {}
>  static inline void set_hv_tscchange_cb(void (*cb)(void)) {}
>  static inline void clear_hv_tscchange_cb(void) {}
>  static inline void hyperv_stop_tsc_emulation(void) {};
> +static inline void hv_apic_init(void) {}
>  static inline struct hv_vp_assist_page *hv_get_vp_assist_page(unsigned i=
nt cpu)
>  {
>  	return NULL;
> --
> 2.15.1