Received: by 2002:ac0:a679:0:0:0:0:0 with SMTP id p54csp47979imp; Wed, 20 Feb 2019 18:56:20 -0800 (PST) X-Google-Smtp-Source: AHgI3IZAzl4LQYyGuinihn7e8xw6zEDvaML8VdwcY1tRF8usgmyQ1AfN53o+kgo4T5JUL+jTKcqs X-Received: by 2002:a17:902:8e8b:: with SMTP id bg11mr40677748plb.332.1550717780428; Wed, 20 Feb 2019 18:56:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550717780; cv=none; d=google.com; s=arc-20160816; b=QX9wPOHF2u+aSkqpc7NY+N603ryZdNlrzVxGk7CUllkeFtH2r39B/XZoIqGie/MJih IJ4DNmlLVqHYRCyxEd6nAAam0DFycNJYQgmZ6zBv6rmPShSHmmQzW2oJ5BJFQE6IxNTH GrhEQhUsyGgu43+Czsb9QA4ApKyHgAQDn+Uu7nMHYb74ZbWfSSkXmU4Zd70W9zylzV4j WvmMt6nkxkAxlh1Pru76ttcMqf3PjljOVYqIOj/4GvX5EoHm1zZ00YWU2aOX1y8HOOSf qnUVvEa7+fnJd95gqb/j9lA8w0aw74jx0iKstrTJ2BmSRRK9V+M1PgIocjVtAg7X6vBM DUcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-language :content-transfer-encoding:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject:dkim-signature; bh=uiDp4t+EQswvCO5nmYj4R3W/t+QWzPqm8VF7HmrzOO0=; b=nyuFWQYLUKoXFd5p7YXl0WBh5x7lPdooo1ivvYQYUPEN9UwTnbf+ihrKZqjlhv5NW2 TzVAG0QCQBqUsJ+1xVMYfyQrL/5+VHsw91j21q2qnIdCXC5uFZBquU6fs9Fj6CnjC1dc bVzGcj9YyBgS0wvItxxn2F6Qa4pH40oRQsp/asbz9P9Q1JpyKeuh+xxze3KMTzJXfl71 ligmCr9oVFdBntvLh5khbOOcMvxwaTjZs6X6/682pgsGCwLU5WROafh78VVtBUhqgSJ6 +Aw0Nq5//gRsqRfOume2GtrueOeoqRbTAvfQIXtk+2Ohl7EtIXQDcKW6sBgV1jJVsPwp MlEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=3hS5xDlL; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h17si20103904pgh.495.2019.02.20.18.56.04; Wed, 20 Feb 2019 18:56:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=3hS5xDlL; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726192AbfBUCzm (ORCPT + 99 others); Wed, 20 Feb 2019 21:55:42 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:39186 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725989AbfBUCzm (ORCPT ); Wed, 20 Feb 2019 21:55:42 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x1L2nDGd002699; Thu, 21 Feb 2019 02:55:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=uiDp4t+EQswvCO5nmYj4R3W/t+QWzPqm8VF7HmrzOO0=; b=3hS5xDlLotOHvyCqqNPG6Dl95ggabCezCKU/uhDIjRdiDZ28gsmNTexyAc0eCjr/ABtG DHejBZdSmaOzQxisg0kfLQaW2yaDv6uUY2skU/mkNNEKsHhjTXfH1rlfuXi8vm5VbxxJ XTUbZEmyPSvKQ/rgNJIY65wjQJR8MPe3OUkXHVzNwHkROc4H6MSzTqN6wd32xYd6wu9g IRTykZusO75HV82yAguetNXHy31qjXjSJu0kBqZ61SZtXPuAC2KX4atKC8U0I71r97B0 w8qp7L5hTxJ7RiMC5jfPaJSh7mfDmHgJQspe4Dtzi24m0J0prtWNZwGO8VhuA8mpbgGr eg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2qpb5rn9bs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Feb 2019 02:55:20 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x1L2tIZQ009546 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Feb 2019 02:55:19 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x1L2tHuQ000668; Thu, 21 Feb 2019 02:55:17 GMT Received: from [10.132.91.175] (/10.132.91.175) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 20 Feb 2019 18:55:17 -0800 Subject: Re: [RFC][PATCH 00/16] sched: Core scheduling To: Linus Torvalds , Peter Zijlstra Cc: Ingo Molnar , Thomas Gleixner , Paul Turner , Tim Chen , Linux List Kernel Mailing , =?UTF-8?B?RnLDqWTDqXJpYyBXZWlzYmVja2Vy?= , Kees Cook , kerrnel@google.com References: <20190218165620.383905466@infradead.org> From: Subhra Mazumdar Message-ID: Date: Wed, 20 Feb 2019 18:53:08 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9173 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902210020 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2/18/19 9:49 AM, Linus Torvalds wrote: > On Mon, Feb 18, 2019 at 9:40 AM Peter Zijlstra wrote: >> However; whichever way around you turn this cookie; it is expensive and nasty. > Do you (or anybody else) have numbers for real loads? > > Because performance is all that matters. If performance is bad, then > it's pointless, since just turning off SMT is the answer. > > Linus I tested 2 Oracle DB instances running OLTP on a 2 socket 44 cores system. This is on baremetal, no virtualization.  In all cases I put each DB instance in separate cpu cgroup. Following are the avg throughput numbers of the 2 instances. %stdev is the standard deviation between the 2 instances. Baseline = build w/o CONFIG_SCHED_CORE core_sched = build w/ CONFIG_SCHED_CORE HT_disable = offlined sibling HT with baseline Users  Baseline  %stdev  core_sched     %stdev HT_disable       %stdev 16     997768    3.28    808193(-19%)   34 1053888(+5.6%)   2.9 24     1157314   9.4     974555(-15.8%) 40.5 1197904(+3.5%)   4.6 32     1693644   6.4     1237195(-27%)  42.8 1308180(-22.8%)  5.3 The regressions are substantial. Also noticed one of the DB instances was having much less throughput than the other with core scheduling which brought down the avg and also reflected in the very high %stdev. Disabling HT has effect at 32 users but still better than core scheduling both in terms of avg and %stdev. There are some issue with the DB setup for which I couldn't go beyond 32 users.