Received: by 2002:a05:7412:37c9:b0:e2:908c:2ebd with SMTP id jz9csp383813rdb; Mon, 18 Sep 2023 19:59:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEuQ7r1epMHkDJxI3k2+tdM7kw0SzBYnTMTBXGQj+WFblgQttRGqrvKWp439XgHYgyzg5lc X-Received: by 2002:a05:622a:4a16:b0:412:2284:cb33 with SMTP id fv22-20020a05622a4a1600b004122284cb33mr12930251qtb.48.1695092350566; Mon, 18 Sep 2023 19:59:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695092350; cv=none; d=google.com; s=arc-20160816; b=jrhvDyCirXy+41jlcnzQXh7r8Ty7pesPfX3y8h5av+zNRekNms9KH5NamcHK8AAbW9 fzF8y+iN8kP7RMFQhTFRzq2KTxiQTLXwWCx2/HFxjTPEM8seGcrNhy5TwoIU4XEzock4 qjYedVdql97d88LR2qxyHzCyC0svx/WWwycVHQWhALPcjLQuGomH81Dqbq+H4MbpQnzX RwKCkRMIhcJw6b57OoPerXrnZlS4pj/4NlH9UP310G8X+34TDv64Bricgn1W4QvFRyfR hw71sXRCnhV+zfQgTfdRM2DS8j3InxSXYIwC/T34T89nppcEP9dd3P+UnZhvJP7Mo3/K zw9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:subject :message-id:date:from:in-reply-to:references:mime-version :dkim-signature; bh=lEG5BTe+fXQvb6D5LCwshGlsyCj6ZJq4Umsw46bKCG8=; fh=/g+lancHT+G4TLTOu+Uwd6L5LOHgV6oj02pY1UgLVMs=; b=ECknl2x3JJuXjq4ZlqCRdU0dijCevqELX1gEVl6ZyTLByNZ0Tcd9W0kBJSpGYh9EXK VvDz8s3Xz/qu4hGFsTEwqzoDkhMfYCdXpDXXhhpdxZ4sEgjpmsWkLaK2f4PIj8IbqPYU mwRhRpM32KAYifi0Mtm6ShQ7+XzG/IEWNG5bdH3wafGjf2uRYCqA1fav0D1m6y8AaKuk cg5Yj3q+ik41EJKicAK0ubukpuR2NYEjX/1P8SoUlK3lQzY0sxze1oTwfQWqIWjuen8P u3P4djDr4i6ZMKqugqE+W+N0Yn9Znxs9gPIh/hM9byGL3yFgiOnJxabkk/Pvoa+5KJVs 9LeA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=pv7BZGMr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id n3-20020a6546c3000000b00563a0bacbb4si801867pgr.694.2023.09.18.19.59.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Sep 2023 19:59:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=pv7BZGMr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id EE6EF8142D81; Mon, 18 Sep 2023 19:57:53 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231214AbjISC5d (ORCPT + 99 others); Mon, 18 Sep 2023 22:57:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231190AbjISC5b (ORCPT ); Mon, 18 Sep 2023 22:57:31 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D889610E; Mon, 18 Sep 2023 19:57:24 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 60900C433CB; Tue, 19 Sep 2023 02:57:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695092244; bh=qo6n8cA/KiwvnrKZ5aIeajMWB4+n4tqFxJU2jjU0b88=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=pv7BZGMruPJmiQvV2CBTlf9GJou5vMnzOQZb+abgj3k/d3UzjoXBG/c4nOsEc1RJb kVvIM+8vNz7u7+YmPPThL3KWoUS3KlHRloWFQw1r/3SuLRgSW5t6zqH9wUDx1DfAOz HTrDErCVk3LvAtaSGAO3xxh/kMRuVsyIbMZkoMoy9RrxZZCmMjrHVPBckVAXFZFFNa I2xaSXmxt5V4/mJd2zLyLbHm8ZRs7u4X+i8g7fJrq3oxhu8FO+OUCIYqo6AN3Dvasy FQux1dMIhy+cyWGiqDyNuJ7DzgZQjedTcCY+RaX/VUa7eeFqR8Uha8dVdQGlTGA3Bf djE1Eybg5JvTQ== Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-532aaba8747so759540a12.3; Mon, 18 Sep 2023 19:57:24 -0700 (PDT) X-Gm-Message-State: AOJu0YxZnkJeGtbs7V0r6L4GUCypptsysIdR8qvM3bQDluvLq++HLC7m 28hvSM0wTyONCN5CD4Otah77g5f+Vz4KEzFAC2k= X-Received: by 2002:aa7:cf94:0:b0:52c:8a13:2126 with SMTP id z20-20020aa7cf94000000b0052c8a132126mr10055343edx.37.1695092242673; Mon, 18 Sep 2023 19:57:22 -0700 (PDT) MIME-Version: 1.0 References: <20230915014949.1222777-1-zhaotianrui@loongson.cn> <20230915014949.1222777-3-zhaotianrui@loongson.cn> <525aa9e2-1ebc-6c49-55ef-9f3c7d18d3e0@loongson.cn> <8e23325a-b4d3-db80-7d65-ebadcd3ef5a9@loongson.cn> <33da46a6-45ef-ffbe-f2b3-45a1b29eaa3d@loongson.cn> <35658917-c7d6-19ad-ea23-848a4ef00a97@loongson.cn> In-Reply-To: <35658917-c7d6-19ad-ea23-848a4ef00a97@loongson.cn> From: Huacai Chen Date: Tue, 19 Sep 2023 10:57:10 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v21 02/29] LoongArch: KVM: Implement kvm module related interface To: zhaotianrui Cc: bibo mao , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Paolo Bonzini , WANG Xuerui , Greg Kroah-Hartman , loongarch@lists.linux.dev, Jens Axboe , Mark Brown , Alex Deucher , Oliver Upton , Xi Ruoyao Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 18 Sep 2023 19:57:54 -0700 (PDT) On Tue, Sep 19, 2023 at 10:53=E2=80=AFAM zhaotianrui wrote: > > > =E5=9C=A8 2023/9/19 =E4=B8=8A=E5=8D=8810:14, Huacai Chen =E5=86=99=E9=81= =93: > > On Mon, Sep 18, 2023 at 4:39=E2=80=AFPM bibo mao = wrote: > >> > >> > >> =E5=9C=A8 2023/9/18 14:25, zhaotianrui =E5=86=99=E9=81=93: > >>> =E5=9C=A8 2023/9/18 =E4=B8=8B=E5=8D=8812:03, Huacai Chen =E5=86=99=E9= =81=93: > >>>> On Mon, Sep 18, 2023 at 11:20=E2=80=AFAM zhaotianrui wrote: > >>>>> =E5=9C=A8 2023/9/18 =E4=B8=8A=E5=8D=8811:12, Huacai Chen =E5=86=99= =E9=81=93: > >>>>>> On Mon, Sep 18, 2023 at 11:08=E2=80=AFAM zhaotianrui wrote: > >>>>>>> =E5=9C=A8 2023/9/18 =E4=B8=8A=E5=8D=889:45, Huacai Chen =E5=86=99= =E9=81=93: > >>>>>>>> On Mon, Sep 18, 2023 at 9:21=E2=80=AFAM zhaotianrui wrote: > >>>>>>>>> =E5=9C=A8 2023/9/16 =E4=B8=8B=E5=8D=884:51, Huacai Chen =E5=86= =99=E9=81=93: > >>>>>>>>>> Hi, Tianrui, > >>>>>>>>>> > >>>>>>>>>> On Fri, Sep 15, 2023 at 9:50=E2=80=AFAM Tianrui Zhao wrote: > >>>>>>>>>>> Implement LoongArch kvm module init, module exit interface, > >>>>>>>>>>> using kvm context to save the vpid info and vcpu world switch > >>>>>>>>>>> interface pointer. > >>>>>>>>>>> > >>>>>>>>>>> Reviewed-by: Bibo Mao > >>>>>>>>>>> Signed-off-by: Tianrui Zhao > >>>>>>>>>>> --- > >>>>>>>>>>> arch/loongarch/kvm/main.c | 367 +++++++++++++++++++++++= +++++++++++++++ > >>>>>>>>>>> 1 file changed, 367 insertions(+) > >>>>>>>>>>> create mode 100644 arch/loongarch/kvm/main.c > >>>>>>>>>>> > >>>>>>>>>>> diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/m= ain.c > >>>>>>>>>>> new file mode 100644 > >>>>>>>>>>> index 0000000000..0deb9273d8 > >>>>>>>>>>> --- /dev/null > >>>>>>>>>>> +++ b/arch/loongarch/kvm/main.c > >>>>>>>>>>> @@ -0,0 +1,367 @@ > >>>>>>>>>>> +// SPDX-License-Identifier: GPL-2.0 > >>>>>>>>>>> +/* > >>>>>>>>>>> + * Copyright (C) 2020-2023 Loongson Technology Corporation L= imited > >>>>>>>>>>> + */ > >>>>>>>>>>> + > >>>>>>>>>>> +#include > >>>>>>>>>>> +#include > >>>>>>>>>>> +#include > >>>>>>>>>>> +#include > >>>>>>>>>>> +#include > >>>>>>>>>>> +#include > >>>>>>>>>>> +#include "trace.h" > >>>>>>>>>>> + > >>>>>>>>>>> +static struct kvm_context __percpu *vmcs; > >>>>>>>>>>> +struct kvm_world_switch *kvm_loongarch_ops; > >>>>>>>>>>> +unsigned long vpid_mask; > >>>>>>>>>>> +static int gcsr_flag[CSR_MAX_NUMS]; > >>>>>>>>>>> + > >>>>>>>>>>> +int get_gcsr_flag(int csr) > >>>>>>>>>>> +{ > >>>>>>>>>>> + if (csr < CSR_MAX_NUMS) > >>>>>>>>>>> + return gcsr_flag[csr]; > >>>>>>>>>>> + > >>>>>>>>>>> + return INVALID_GCSR; > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +static inline void set_gcsr_sw_flag(int csr) > >>>>>>>>>>> +{ > >>>>>>>>>>> + if (csr < CSR_MAX_NUMS) > >>>>>>>>>>> + gcsr_flag[csr] |=3D SW_GCSR; > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +static inline void set_gcsr_hw_flag(int csr) > >>>>>>>>>>> +{ > >>>>>>>>>>> + if (csr < CSR_MAX_NUMS) > >>>>>>>>>>> + gcsr_flag[csr] |=3D HW_GCSR; > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +/* > >>>>>>>>>>> + * The default value of gcsr_flag[CSR] is 0, and we use this > >>>>>>>>>>> + * function to set the flag to 1(SW_GCSR) or 2(HW_GCSR) if t= he > >>>>>>>>>>> + * gcsr is software or hardware. It will be used by get/set_= gcsr, > >>>>>>>>>>> + * if gcsr_flag is HW we should use gcsrrd/gcsrwr to access = it, > >>>>>>>>>>> + * else use sw csr to emulate it. > >>>>>>>>>>> + */ > >>>>>>>>>>> +static void kvm_init_gcsr_flag(void) > >>>>>>>>>>> +{ > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_CRMD); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PRMD); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_EUEN); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_MISC); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_ECFG); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_ESTAT); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_ERA); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_BADV); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_BADI); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_EENTRY); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBIDX); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBEHI); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBELO0); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBELO1); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_ASID); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PGDL); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PGDH); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PWCTL0); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PWCTL1); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_STLBPGSIZE); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_RVACFG); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_CPUID); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PRCFG1); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PRCFG2); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_PRCFG3); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS0); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS1); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS2); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS3); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS4); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS5); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS6); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_KS7); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TMID); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TCFG); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TVAL); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_CNTC); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_LLBCTL); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBRENTRY); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBRBADV); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBRERA); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBRSAVE); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBRELO0); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBRELO1); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBREHI); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_TLBRPRMD); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN0); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN1); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN2); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_DMWIN3); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_MWPS); > >>>>>>>>>>> + set_gcsr_hw_flag(LOONGARCH_CSR_FWPS); > >>>>>>>>>>> + > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IMPCTL1); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IMPCTL2); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MERRCTL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MERRINFO1); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MERRINFO2); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MERRENTRY); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MERRERA); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MERRSAVE); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_CTAG); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DEBUG); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DERA); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DESAVE); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PRCFG1); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PRCFG2); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PRCFG3); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PGD); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_TINTCLR); > >>>>>>>>>>> + > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_FWPS); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_FWPC); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MWPS); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_MWPC); > >>>>>>>>>> FWPS and MWPS are both HW CSR and SW CSR? > >>>>>>>>>> > >>>>>>>>>> Huacai > >>>>>>>>> The FWPC and MWPC should be SW GCSR, FWPS and MWPS should be HW= GCSR, it > >>>>>>>>> is my mistake. > >>>>>>>> But in user manual vol 3, section 1.5, FWPC/FWPS/MWPC/MWPS are a= ll HW > >>>>>>>> GCSR, while DBxxxx and IBxxxx are SW GCSR. > >>>>>>> Ok, It is my misunderstanding, as the FWPC and MWPC can control g= uest > >>>>>>> debug register numbers, but I know they are all HW GCSR when I lo= ok up > >>>>>>> the manual again. > >>>>>> So these lines can be removed? > >>>>>> > >>>>>> set_gcsr_sw_flag(LOONGARCH_CSR_FWPS); > >>>>>> set_gcsr_sw_flag(LOONGARCH_CSR_FWPC); > >>>>>> set_gcsr_sw_flag(LOONGARCH_CSR_MWPS); > >>>>>> set_gcsr_sw_flag(LOONGARCH_CSR_MWPC); > >>>>>> > >>>>>> And add FWPC/MWPC to hw list? > >>>> Can you discuss more with our hw engineers? I found in section 1.8: > >>>> PGD, TINTCLR, PRCFG1~3 are all HW GCSR. > >>> If guest can access hw CSR directly and do not cause a exception, it = means hw GCSR. on the other hand, if cause a exception to return to KVM to = emulate it by software, it means SW GCSR. And I re-check the PGD, TINTCLR, = PRCFG1~3 CSR, they should be hw GCSR. > >> CSR_PGD is logical read only register instead, its content is the same > >> CSR_PGDL if highest bit of BADV is 0, else its content is the same wit= h > >> CSR_PGDH. > >> > >> TINTCLR is write only register, write 1 to clear timer interrupt and r= ead > >> value is 0. > >> > >> So CSR_PGD and TINTCLR need not save and restore during vcpu switching= or > >> vm migration, it is not necessary to set CSR_PGD/TINTCLR from qemu use= r space. > >> > >> PRCFG1~3 is read only registers, it is decided by hardware and can not= be > >> set by kvm, it is not necessary to save or retore. However there maybe= brings > >> problems for vm migration on diferent hardwares if PRCFG1~3 is differe= nt. > > During my tests, reading FWPC/FWPS/MWPC/MWPS causes guest exit, but > > user manual vol 3 says they are HW GCSR, who can tell me why? > It will cause guest exiting when guest visit these registers above for > the first time, and KVM could config FWPC, MWPC to enable the debug > registers for guest, so that it will not cause exiting at next time. And > we will add this handling later when we want to use this function in gues= t. Then what type should we mark them? HW GCSR or SW GCSR? Huacai > > Thanks > Tianrui Zhao > > > > Huacai > > > >> Regards > >> Bibo Mao > >>> Thanks > >>> Tianrui Zhao > >>>> Huacai > >>>> > >>>>>> Huacai > >>>>> Yes, It is. > >>>>> > >>>>> Thanks > >>>>> Tianrui Zhao > >>>>>>> Thanks > >>>>>>> Tianrui Zhao > >>>>>>>> Huacai > >>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> Tianrui Zhao > >>>>>>>>>>> + > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB0ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB0MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB0CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB0ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB1ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB1MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB1CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB1ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB2ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB2MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB2CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB2ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB3ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB3MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB3CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB3ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB4ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB4MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB4CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB4ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB5ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB5MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB5CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB5ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB6ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB6MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB6CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB6ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB7ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB7MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB7CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_DB7ASID); > >>>>>>>>>>> + > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB0ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB0MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB0CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB0ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB1ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB1MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB1CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB1ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB2ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB2MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB2CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB2ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB3ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB3MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB3CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB3ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB4ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB4MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB4CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB4ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB5ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB5MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB5CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB5ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB6ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB6MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB6CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB6ASID); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB7ADDR); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB7MASK); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB7CTRL); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_IB7ASID); > >>>>>>>>>>> + > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL0); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR0); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL1); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR1); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL2); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR2); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCTRL3); > >>>>>>>>>>> + set_gcsr_sw_flag(LOONGARCH_CSR_PERFCNTR3); > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu) > >>>>>>>>>>> +{ > >>>>>>>>>>> + struct kvm_context *context; > >>>>>>>>>>> + unsigned long vpid; > >>>>>>>>>>> + > >>>>>>>>>>> + context =3D per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu); > >>>>>>>>>>> + vpid =3D context->vpid_cache + 1; > >>>>>>>>>>> + if (!(vpid & vpid_mask)) { > >>>>>>>>>>> + /* finish round of 64 bit loop */ > >>>>>>>>>>> + if (unlikely(!vpid)) > >>>>>>>>>>> + vpid =3D vpid_mask + 1; > >>>>>>>>>>> + > >>>>>>>>>>> + /* vpid 0 reserved for root */ > >>>>>>>>>>> + ++vpid; > >>>>>>>>>>> + > >>>>>>>>>>> + /* start new vpid cycle */ > >>>>>>>>>>> + kvm_flush_tlb_all(); > >>>>>>>>>>> + } > >>>>>>>>>>> + > >>>>>>>>>>> + context->vpid_cache =3D vpid; > >>>>>>>>>>> + vcpu->arch.vpid =3D vpid; > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +void kvm_check_vpid(struct kvm_vcpu *vcpu) > >>>>>>>>>>> +{ > >>>>>>>>>>> + struct kvm_context *context; > >>>>>>>>>>> + bool migrated; > >>>>>>>>>>> + unsigned long ver, old, vpid; > >>>>>>>>>>> + int cpu; > >>>>>>>>>>> + > >>>>>>>>>>> + cpu =3D smp_processor_id(); > >>>>>>>>>>> + /* > >>>>>>>>>>> + * Are we entering guest context on a different CPU t= o last time? > >>>>>>>>>>> + * If so, the vCPU's guest TLB state on this CPU may = be stale. > >>>>>>>>>>> + */ > >>>>>>>>>>> + context =3D per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu); > >>>>>>>>>>> + migrated =3D (vcpu->cpu !=3D cpu); > >>>>>>>>>>> + > >>>>>>>>>>> + /* > >>>>>>>>>>> + * Check if our vpid is of an older version > >>>>>>>>>>> + * > >>>>>>>>>>> + * We also discard the stored vpid if we've executed = on > >>>>>>>>>>> + * another CPU, as the guest mappings may have change= d without > >>>>>>>>>>> + * hypervisor knowledge. > >>>>>>>>>>> + */ > >>>>>>>>>>> + ver =3D vcpu->arch.vpid & ~vpid_mask; > >>>>>>>>>>> + old =3D context->vpid_cache & ~vpid_mask; > >>>>>>>>>>> + if (migrated || (ver !=3D old)) { > >>>>>>>>>>> + kvm_update_vpid(vcpu, cpu); > >>>>>>>>>>> + trace_kvm_vpid_change(vcpu, vcpu->arch.vpid); > >>>>>>>>>>> + vcpu->cpu =3D cpu; > >>>>>>>>>>> + } > >>>>>>>>>>> + > >>>>>>>>>>> + /* Restore GSTAT(0x50).vpid */ > >>>>>>>>>>> + vpid =3D (vcpu->arch.vpid & vpid_mask) << CSR_GSTAT_G= ID_SHIFT; > >>>>>>>>>>> + change_csr_gstat(vpid_mask << CSR_GSTAT_GID_SHIFT, vp= id); > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +static int kvm_loongarch_env_init(void) > >>>>>>>>>>> +{ > >>>>>>>>>>> + struct kvm_context *context; > >>>>>>>>>>> + int cpu, order; > >>>>>>>>>>> + void *addr; > >>>>>>>>>>> + > >>>>>>>>>>> + vmcs =3D alloc_percpu(struct kvm_context); > >>>>>>>>>>> + if (!vmcs) { > >>>>>>>>>>> + pr_err("kvm: failed to allocate percpu kvm_co= ntext\n"); > >>>>>>>>>>> + return -ENOMEM; > >>>>>>>>>>> + } > >>>>>>>>>>> + > >>>>>>>>>>> + kvm_loongarch_ops =3D kzalloc(sizeof(*kvm_loongarch_o= ps), GFP_KERNEL); > >>>>>>>>>>> + if (!kvm_loongarch_ops) { > >>>>>>>>>>> + free_percpu(vmcs); > >>>>>>>>>>> + vmcs =3D NULL; > >>>>>>>>>>> + return -ENOMEM; > >>>>>>>>>>> + } > >>>>>>>>>>> + /* > >>>>>>>>>>> + * There will be problem in world switch code if ther= e > >>>>>>>>>>> + * is page fault reenter, since pgd register is share= d > >>>>>>>>>>> + * between root kernel and kvm hypervisor. World swit= ch > >>>>>>>>>>> + * entry need be unmapped area, cannot be tlb mapped = area. > >>>>>>>>>>> + * In future if hw pagetable walking is supported, or= there > >>>>>>>>>>> + * is separate pgd registers between root kernel and = kvm > >>>>>>>>>>> + * hypervisor, copying about world switch code will n= ot be used. > >>>>>>>>>>> + */ > >>>>>>>>>>> + > >>>>>>>>>>> + order =3D get_order(kvm_vector_size + kvm_enter_guest= _size); > >>>>>>>>>>> + addr =3D (void *)__get_free_pages(GFP_KERNEL, order); > >>>>>>>>>>> + if (!addr) { > >>>>>>>>>>> + free_percpu(vmcs); > >>>>>>>>>>> + vmcs =3D NULL; > >>>>>>>>>>> + kfree(kvm_loongarch_ops); > >>>>>>>>>>> + kvm_loongarch_ops =3D NULL; > >>>>>>>>>>> + return -ENOMEM; > >>>>>>>>>>> + } > >>>>>>>>>>> + > >>>>>>>>>>> + memcpy(addr, kvm_vector_entry, kvm_vector_size); > >>>>>>>>>>> + memcpy(addr + kvm_vector_size, kvm_enter_guest, kvm_e= nter_guest_size); > >>>>>>>>>>> + flush_icache_range((unsigned long)addr, (unsigned lon= g)addr + > >>>>>>>>>>> + kvm_vector_size + kvm_enter_g= uest_size); > >>>>>>>>>>> + kvm_loongarch_ops->guest_eentry =3D addr; > >>>>>>>>>>> + kvm_loongarch_ops->enter_guest =3D addr + kvm_vector_= size; > >>>>>>>>>>> + kvm_loongarch_ops->page_order =3D order; > >>>>>>>>>>> + > >>>>>>>>>>> + vpid_mask =3D read_csr_gstat(); > >>>>>>>>>>> + vpid_mask =3D (vpid_mask & CSR_GSTAT_GIDBIT) >> CSR_G= STAT_GIDBIT_SHIFT; > >>>>>>>>>>> + if (vpid_mask) > >>>>>>>>>>> + vpid_mask =3D GENMASK(vpid_mask - 1, 0); > >>>>>>>>>>> + > >>>>>>>>>>> + for_each_possible_cpu(cpu) { > >>>>>>>>>>> + context =3D per_cpu_ptr(vmcs, cpu); > >>>>>>>>>>> + context->vpid_cache =3D vpid_mask + 1; > >>>>>>>>>>> + context->last_vcpu =3D NULL; > >>>>>>>>>>> + } > >>>>>>>>>>> + > >>>>>>>>>>> + kvm_init_fault(); > >>>>>>>>>>> + kvm_init_gcsr_flag(); > >>>>>>>>>>> + > >>>>>>>>>>> + return 0; > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +static void kvm_loongarch_env_exit(void) > >>>>>>>>>>> +{ > >>>>>>>>>>> + unsigned long addr; > >>>>>>>>>>> + > >>>>>>>>>>> + if (vmcs) > >>>>>>>>>>> + free_percpu(vmcs); > >>>>>>>>>>> + > >>>>>>>>>>> + if (kvm_loongarch_ops) { > >>>>>>>>>>> + if (kvm_loongarch_ops->guest_eentry) { > >>>>>>>>>>> + addr =3D (unsigned long)kvm_loongarch= _ops->guest_eentry; > >>>>>>>>>>> + free_pages(addr, kvm_loongarch_ops->p= age_order); > >>>>>>>>>>> + } > >>>>>>>>>>> + kfree(kvm_loongarch_ops); > >>>>>>>>>>> + } > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +static int kvm_loongarch_init(void) > >>>>>>>>>>> +{ > >>>>>>>>>>> + int r; > >>>>>>>>>>> + > >>>>>>>>>>> + if (!cpu_has_lvz) { > >>>>>>>>>>> + kvm_info("hardware virtualization not availab= le\n"); > >>>>>>>>>>> + return -ENODEV; > >>>>>>>>>>> + } > >>>>>>>>>>> + r =3D kvm_loongarch_env_init(); > >>>>>>>>>>> + if (r) > >>>>>>>>>>> + return r; > >>>>>>>>>>> + > >>>>>>>>>>> + return kvm_init(sizeof(struct kvm_vcpu), 0, THIS_MODU= LE); > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +static void kvm_loongarch_exit(void) > >>>>>>>>>>> +{ > >>>>>>>>>>> + kvm_exit(); > >>>>>>>>>>> + kvm_loongarch_env_exit(); > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +module_init(kvm_loongarch_init); > >>>>>>>>>>> +module_exit(kvm_loongarch_exit); > >>>>>>>>>>> + > >>>>>>>>>>> +#ifdef MODULE > >>>>>>>>>>> +static const struct cpu_feature loongarch_kvm_feature[] =3D = { > >>>>>>>>>>> + { .feature =3D cpu_feature(LOONGARCH_LVZ) }, > >>>>>>>>>>> + {}, > >>>>>>>>>>> +}; > >>>>>>>>>>> +MODULE_DEVICE_TABLE(cpu, loongarch_kvm_feature); > >>>>>>>>>>> +#endif > >>>>>>>>>>> -- > >>>>>>>>>>> 2.39.1 > >>>>>>>>>>> > >> > >