Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 3 Jul 2023 20:55:56 +0300
From:   "kirill.shutemov@linux.intel.com" <kirill.shutemov@linux.intel.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Dave Hansen <dave.hansen@intel.com>,
        Sean Christopherson <seanjc@google.com>,
        Isaku Yamahata <isaku.yamahata@gmail.com>,
        Kai Huang <kai.huang@intel.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        Ashok Raj <ashok.raj@intel.com>,
        Tony Luck <tony.luck@intel.com>,
        "david@redhat.com" <david@redhat.com>,
        "bagasdotme@gmail.com" <bagasdotme@gmail.com>,
        "ak@linux.intel.com" <ak@linux.intel.com>,
        Rafael J Wysocki <rafael.j.wysocki@intel.com>,
        Reinette Chatre <reinette.chatre@intel.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        Isaku Yamahata <isaku.yamahata@intel.com>,
        "nik.borisov@suse.com" <nik.borisov@suse.com>,
        "hpa@zytor.com" <hpa@zytor.com>, Sagi Shahar <sagis@google.com>,
        "imammedo@redhat.com" <imammedo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>, Chao Gao <chao.gao@intel.com>,
        Len Brown <len.brown@intel.com>,
        "sathyanarayanan.kuppuswamy@linux.intel.com" 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        Ying Huang <ying.huang@intel.com>,
        Dan J Williams <dan.j.williams@intel.com>,
        "x86@kernel.org" <x86@kernel.org>
Subject: Re: [PATCH v12 07/22] x86/virt/tdx: Add skeleton to enable TDX on
 demand
Message-ID: <20230703175556.nn5xozz7dzxjocqm@box.shutemov.name>
References: <20230628131717.GE2438817@hirez.programming.kicks-ass.net>
 <0c9639db604a0670eeae5343d456e43d06b35d39.camel@intel.com>
 <20230630092615.GD2533791@hirez.programming.kicks-ass.net>
 <2659d6eef84f008635ba300f4712501ac88cef2c.camel@intel.com>
 <20230630183020.GA4253@hirez.programming.kicks-ass.net>
 <20230630190514.GH3436214@ls.amr.corp.intel.com>
 <ZJ9IKALhz1Q6ogu1@google.com>
 <20230703104942.GG4253@hirez.programming.kicks-ass.net>
 <eb83e722-0379-1451-9c9c-9b9de33cb4cb@intel.com>
 <20230703150330.GA83892@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230703150330.GA83892@hirez.programming.kicks-ass.net>
Precedence: bulk

On Mon, Jul 03, 2023 at 05:03:30PM +0200, Peter Zijlstra wrote:
> On Mon, Jul 03, 2023 at 07:40:55AM -0700, Dave Hansen wrote:
> > On 7/3/23 03:49, Peter Zijlstra wrote:
> > >> There are also latency and noisy neighbor concerns, e.g. we *really* don't want
> > >> to end up in a situation where creating a TDX guest for a customer can observe
> > >> arbitrary latency *and* potentially be disruptive to VMs already running on the
> > >> host.
> > > Well, that's a quality of implementation issue with the whole TDX
> > > crapola. Sounds like we want to impose latency constraints on the
> > > various TDX calls. Allowing it to consume arbitrary amounts of CPU time
> > > is unacceptable in any case.
> > 
> > For what it's worth, everybody knew that calling into the TDX module was
> > going to be a black hole and that consuming large amounts of CPU at
> > random times would drive people bat guano crazy.
> > 
> > The TDX Module ABI spec does have "Leaf Function Latency" warnings for
> > some of the module calls.  But, it's basically a binary thing.  A call
> > is either normal or "longer than most".
> > 
> > The majority of the "longer than most" cases are for initialization.
> > The _most_ obscene runtime ones are chunked up and can return partial
> > progress to limit latency spikes.  But I don't think folks tried as hard
> > on the initialization calls since they're only called once which
> > actually seems pretty reasonable to me.
> > 
> > Maybe we need three classes of "Leaf Function Latency":
> > 1. Sane
> > 2. "Longer than most"
> > 3. Better turn the NMI watchdog off before calling this. :)
> > 
> > Would that help?
> 
> I'm thikning we want something along the lines of the Xen preemptible
> hypercalls, except less crazy. Where the caller does:
> 
> 	for (;;) {
> 		ret = tdcall(fn, args);
> 		if (ret == -EAGAIN) {
> 			cond_resched();
> 			continue;
> 		}
> 		break;
> 	}
> 
> And then the TDX black box provides a guarantee that any one tdcall (or
> seamcall or whatever) never takes more than X ns (possibly even
> configurable) and we get to raise a bug report if we can prove it
> actually takes longer.

TDG.VP.VMCALL TDCALL can take arbitrary amount of time as it handles over
control to the host/VMM.

But I'm not quite follow how it is different from the host stopping
scheduling vCPU on a random instruction. It can happen at any point and
TDCALL is not special from this PoV.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov