Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <c96a78c6a8caf25b01e450f139c934688d1735b0.camel@intel.com>
Subject: Re: [PATCH v5 07/22] x86/virt/tdx: Implement SEAMCALL function
From:   Kai Huang <kai.huang@intel.com>
To:     Dave Hansen <dave.hansen@intel.com>, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org
Cc:     seanjc@google.com, pbonzini@redhat.com, len.brown@intel.com,
        tony.luck@intel.com, rafael.j.wysocki@intel.com,
        reinette.chatre@intel.com, dan.j.williams@intel.com,
        peterz@infradead.org, ak@linux.intel.com,
        kirill.shutemov@linux.intel.com,
        sathyanarayanan.kuppuswamy@linux.intel.com,
        isaku.yamahata@intel.com
Date:   Wed, 03 Aug 2022 14:37:18 +1200
In-Reply-To: <0b20f1878d31658a9e3cd3edaf3826fe8731346e.camel@intel.com>
References: <cover.1655894131.git.kai.huang@intel.com>
         <095e6bbc57b4470e1e9a9104059a5238c9775f00.1655894131.git.kai.huang@intel.com>
         <069a062e-a4a6-09af-7b74-7f4929f2ec0b@intel.com>
         <5ce7ebfe54160ea35e432bf50207ebed32db31fc.camel@intel.com>
         <84e93539-a2f9-f68e-416a-ea3d8fc725af@intel.com>
         <6bef368ccc68676e4acaecc4b6dc52f598ea7f2f.camel@intel.com>
         <ea03e55499f556388c0a5f9ed565e72e213c276f.camel@intel.com>
         <978c3d37-97c9-79b9-426a-2c27db34c38a@intel.com>
         <0b20f1878d31658a9e3cd3edaf3826fe8731346e.camel@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.3 (3.44.3-1.fc36) 
MIME-Version: 1.0
Precedence: bulk

On Thu, 2022-07-21 at 13:52 +1200, Kai Huang wrote:
> Also, if I understand correctly above, your suggestion is we want to prev=
ent any
> CMR memory going offline so it won't be hot-removed (assuming we can get =
CMRs
> during boot).=C2=A0 This looks contradicts to the requirement of being ab=
le to allow
> moving memory from core-mm to driver.=C2=A0 When we offline the memory, w=
e cannot
> know whether the memory will be used by driver, or later hot-removed.

Hi Dave,

The high level flow of device hot-removal is:

acpi_scan_hot_remove()
	-> acpi_scan_try_to_offline()
		-> acpi_bus_offline()
			-> device_offline()
				-> memory_subsys_offline()
	-> acpi_bus_trim()
		-> acpi_memory_device_remove()


And memory_subsys_offline() can also be triggered via /sysfs:

	echo 0 > /sys/devices/system/memory/memory30/online

After the memory block is offline, my understanding is kernel can theoretic=
ally
move it to, i.e. ZONE_DEVICE via memremap_pages().

As you can see memory_subsys_offline() is the entry point of memory device
offline (before it the code is generic for all ACPI device), and it cannot
distinguish whether the removal is from ACPI event, or from /sysfs, so it s=
eems
we are unable to refuse to offline memory in  memory_subsys_offline() when =
it is
called from ACPI event.

Any comments?

--=20
Thanks,
-Kai