Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
MIME-Version: 1.0
References: <20231115120044.8034-1-kirill.shutemov@linux.intel.com>
 <ZVYGx1EwzjXbTEyX@MiWiFi-R3L-srv> <20231116125612.daxettqcapled7ac@box.shutemov.name>
 <ZVYkdI74X8acDtTq@MiWiFi-R3L-srv> <ZVYrA+Ks0DGFo/0p@MiWiFi-R3L-srv>
 <20231117124748.umfuc3no2qvh4shj@box.shutemov.name> <ZVeApN4lDqTLu7ma@MiWiFi-R3L-srv>
 <20231117154632.zvi6g6lblmtvikzt@box> <ZVxRBLsQcv8KRasA@MiWiFi-R3L-srv> <20231121084321.t44ntae5ohslgrme@box.shutemov.name>
In-Reply-To: <20231121084321.t44ntae5ohslgrme@box.shutemov.name>
From:   Tao Liu <ltao@redhat.com>
Date:   Tue, 21 Nov 2023 17:10:01 +0800
Message-ID: <CAO7dBbWb2UWJRm1LLXOq7MM1K6BxDetL8tTmW93H589FGz8n9g@mail.gmail.com>
Subject: Re: [PATCHv3 00/14] x86/tdx: Add kexec support
To:     "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc:     Baoquan He <bhe@redhat.com>, Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Adrian Hunter <adrian.hunter@intel.com>,
        Kuppuswamy Sathyanarayanan 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        Elena Reshetova <elena.reshetova@intel.com>,
        Jun Nakajima <jun.nakajima@intel.com>,
        Rick Edgecombe <rick.p.edgecombe@intel.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        "Kalra, Ashish" <ashish.kalra@amd.com>,
        Sean Christopherson <seanjc@google.com>,
        "Huang, Kai" <kai.huang@intel.com>, kexec@lists.infradead.org,
        linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Hi Kirill,

On Tue, Nov 21, 2023 at 4:43=E2=80=AFPM Kirill A. Shutemov
<kirill.shutemov@linux.intel.com> wrote:
>
> On Tue, Nov 21, 2023 at 02:41:08PM +0800, Baoquan He wrote:
> > > >
> > > > Still failed. And I found the normal reboot does't work either. I w=
ill
> > > > do more testing tomorrow, e.g use the tdx-tools's own rhel9 kernel
> > > > config and rebuild, and update host kernel too.
> >
> > I did more tests, resuls are summarized as below:
> >
> > 1) kexec reboot works, but always fallback to 1 cpu even though multipl=
e
> > cpus are specified;
>
> That's expected. Until you have new BIOS. See below.
>
> > 2) kdump kernel need more crashkernel memory to boot up,
> >    crashkernel=3D512M works well in our case.
>
> I guess it is due to SWIOTLB memory which requres at least 64M.
>
> >  But it failed in vmcore
> >    saving process, either makedumpfile or cp can't access the 1st
> >    kernel's old memory;
>
> Will look into it.
>
> > 3) Normal reboot always failed;
>
> It is expected. TD vCPUs are not resettable. So we need to destroy TD and
> construct a new one to emulate "reboot".
>
> I guess we can try to add some glue on QEMU side to make reboot more
> seamless.
>
> > My colleague Tao helped to double check this, he got the same testing r=
esult.
> > Plesae
> > 1) what can we do to enable the multiple cpu support for kexec reboot?
>
> You would need a patched BIOS image. I've hacked one[1] for my testing.
> But it only works if kernel runs in 4-level paging mode (specify no5lvl i=
n
> kernel command line).
>
> BIOS folks work on proper patch, but it is not ready yet.
>
> [1] https://gist.github.com/kiryl/e1dc1719e0c990b3ceee5d8de8dbf332
>
Thanks a lot for providing the BIOS patch. One more question, is the
patched BIOS image for the host physical machine or for the tdx guest
vm?

Thanks,
Tao Liu

> > 2) anything missing to allow makedumpfile/cp access 1st kernel's memory=
?
>
> It worked before for us, but I have not checked for a while.
>
> I expected you've dropped my "if (crash) return;" debug patch I asked you
> to tested before, right? If not, failure is expected.
>
> > 3) not sure if this is particular case on the system we tested on.
>
> --
>   Kiryl Shutsemau / Kirill A. Shutemov
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec
>