Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: wZzgOCCSU9HONFG6Ay41YdLmcva6n7OOXmYZ/PvUgcQrBBT78/9SeexGrcjN2HyYvN0D39kOGg
 YDkp/6QsoW6w==
IronPort-SDR: ZYE4sOE06IOvTvItpR8GtoR8eyOVVzhAYghfwYSFdFMybDF5U6/sjPX2HGzPeVw6MVrlkPSe1y
 kUwKb40kmF+w==
Date:   Fri, 26 Jun 2020 00:37:05 +0300
From:   Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
To:     Don Porter <porter@cs.unc.edu>
Cc:     Andy Lutomirski <luto@amacapital.net>,
        Thomas Gleixner <tglx@linutronix.de>,
        Andi Kleen <ak@linux.intel.com>,
        Sasha Levin <sashal@kernel.org>, linux-kernel@vger.kernel.org,
        bp@alien8.de, luto@kernel.org, hpa@zytor.com,
        dave.hansen@intel.com, tony.luck@intel.com,
        ravi.v.shankar@intel.com, chang.seok.bae@intel.com
Subject: Re: [PATCH v12 00/18] Enable FSGSBASE instructions
Message-ID: <20200625213705.GF20341@linux.intel.com>
References: <e9a0a521-104b-5c3a-a689-78f878e73d31@cs.unc.edu>
 <7A3EBAB0-B3B3-4CB7-AA6A-FDF29D03E30D@amacapital.net>
 <20200529152756.GA7452@invisiblethingslab.com>
 <ef8bbdff-e891-bee3-677d-3606474ecc10@cs.unc.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ef8bbdff-e891-bee3-677d-3606474ecc10@cs.unc.edu>
Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, Jun 25, 2020 at 11:27:28AM -0400, Don Porter wrote:
> On 5/29/20 11:27 AM, Wojtek Porczyk wrote:
> > On Thu, May 28, 2020 at 11:38:01AM -0700, Andy Lutomirski wrote:
> > > One useful test for the actual kernel patches would be to run your SGX
> > > workload on a loaded core.  That is, do something like taskset -c
> > > 0 graphene_thing and, simultaneously, write a trivial infinite loop program
> > > and run that under taskset -c 0 as well. For good measure, you could have
> > > perf top or perf record running at the same time.  Look for kernel errors,
> > > but also look for any evidence of your workload malfunctioning.
> > 
> > We currently run as part of CI several workloads[1], among them LTP tests[2],
> > and sometimes it's not pretty, because we encounter stability problems in
> > Graphene+SGX even without the patchset. We'll pick some stable subset and
> > will let know. Right now we'll have to retool CI for custom kernels, which
> > will take some back and forth with uni's admins.
> > 
> > [1] https://github.com/oscarlab/graphene/tree/master/Examples
> > [2] https://github.com/oscarlab/graphene/tree/master/LibOS/shim/test/ltp
> > 
> 
> Following up: we have been running a patched 5.7 kernel with v12 of this
> series on one of our CI workers.  As Wojtek mentions, infrastructure and
> other orthogonal issues took some time.
> 
> We have run our complete SGX testing pipelines successfully several times
> with no issues: no errors in Graphene or suspicious kernel messages.
> 
> I also did Andy's suggested test:
> * Graphene running nginx pinned to core 0
> * infinite loop on core 0
> * perf top running
> * Exercised with non-SGX apache bench several times (~10 minutes of testing
> time) also from core 0
> 
> Again, no apparent issues, nothing in dmesg.  I ran a similar setup with our
> SGX-specific Graphene (PAL) unit tests.  Same story: everything looks good.
> 
> Let us know if we can be of any more help here.
> 
> Thanks,
> Don

Can unmodified Graphene-SGX used with these changes?

/Jarkko