Received: by 2002:ac0:aed5:0:0:0:0:0 with SMTP id t21csp665123imb; Fri, 1 Mar 2019 10:34:44 -0800 (PST) X-Google-Smtp-Source: AHgI3Ia/Xj7gPWI759ML606DPOY5qyRggPXK52uHgH+cGtUEwS7QqhtoFKssVWYKD/qrr64nL55x X-Received: by 2002:a62:4254:: with SMTP id p81mr6928408pfa.185.1551465284152; Fri, 01 Mar 2019 10:34:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1551465284; cv=none; d=google.com; s=arc-20160816; b=leJb6SQmy3cH74rHhjapjzs9wVE2xYh/i0bW2SbL3NjbVGk0HGy/Y+A1nEjQjvbs+S hrMr0Qciee18mNu2+XzK/v035xWov5g69y1eFGmPAmmlRgmy3P+aDjPdZD2rsJYh8evR isW7aefVF4Q/0yOa/CW3Q4uw9lMKIxKmWDhBl8Wb1u38zC97pNl5zWIzZuXeRCVOkC6R TlGnsoUFizNMXzkqoEnfNT8O+xFk6ghFbFxcJ0TZaM8pxe3so5kJW1B/93sfCMjgHQG1 aMgeQPo9MDkXiStcQQJfHUz/LMufwGtUlOGugOkbhipfcf6F19IY0mkAsCcc8FqVralV EZZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject:dkim-signature; bh=9uyvylYFSkA0QW0XxHbiwXU0SqdDY56aMSbjTkF8uKk=; b=JdTkF4i/h6QIFwQ+Z/uCJL3Nik0Rtquq6k1TYkkWmtGyNho9URoFcDjGYkQ5QXfFsJ GWBBbjjPSDxMI3gOlJ+k+Q/ZwLVXc/Sm0+3tgXdRn65E1v74EDnsiqlZ0VmWKkb4ihSA le8ckPqIQZ5gaRcn8VjFMGZJO2cSYhNa8VewVSdKvcBr5O8HF3MTBq783QFNf9vowq06 AvF18bwFkzkYMrfkEMuRJRFLMRasOFFI3+5p56sRJ3rYLO81VnRQR/Bq4VcXOeilspqp HJksVEAkFbjx+tvAzVCD9OyBM0Dd7V3M/PX/vJk/lPL87HaN5SX7gos+dkL18JdEItsY fXRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=rUF1eq+v; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a25si20134424pgw.62.2019.03.01.10.34.29; Fri, 01 Mar 2019 10:34:44 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=rUF1eq+v; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389373AbfCASaU (ORCPT + 99 others); Fri, 1 Mar 2019 13:30:20 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:37979 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388934AbfCASaT (ORCPT ); Fri, 1 Mar 2019 13:30:19 -0500 Received: by mail-pl1-f194.google.com with SMTP id g37so9060889plb.5 for ; Fri, 01 Mar 2019 10:30:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=9uyvylYFSkA0QW0XxHbiwXU0SqdDY56aMSbjTkF8uKk=; b=rUF1eq+vf6Go/I4HNZxkae1KrzgzcmMoU6TWLeVKgh71HGKHJmgZr8WXTJ2+J+mnXX S4Xn6L9KRrZwyFnoWcCTZ4AGNe/eX6RN9w2Na/O9YX2SMrAm5G2DZJN0/+Cw+pJZbhEC CYExFa0RNJtIxp1mOJMWLu4JHwOutmIXchpoHa/HrWdeyV6o7JFnDmk2g3MWZeNHy8NI JbM8SoeSVzcqrn3ipKt47Qqvwv7ilui/TI3MBMT4cP7l/KyCqNhLcDEYfNCqpQvcucoF UXWecM1On6CBBQwxpkZUWPzNmz7CaUWqmZLRVGXuPcRYeD4YCDdi73gpH5251Zjpt2ox 2YlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=9uyvylYFSkA0QW0XxHbiwXU0SqdDY56aMSbjTkF8uKk=; b=fPx4NAdh59mcB0FrtsfdYJxPKS2yEtlK85kcRxjMz9UuYB+iD10KqmNZB9oVhK3vLv d53aZf0gWJbjJZ1QG1K/pTy2WO0lYbXoJQ7racgspu6FTp5lju7O40SYSMlI6aibZ3G3 olX7MspUO1gNYF9KddFwved+PNYEE+ogoOj54NvD8+xjrfFQEUbpThJx1FiClYVsQre/ VK5O7VygPsrLOsOiWkNkhv1JrFUSBMvIzzwLcAspBYACDUFgBlhlB+CScyzOCUxZHjXI LT/gmfh01dGO25FECAXOvamGCBCOL9woS5yDDOvHEEsFZmkugW9nOt7hwkGgJVxUR6aT mpSA== X-Gm-Message-State: APjAAAV/rMd7xzLbYTHBGNtYVODkIFOFHDlLbGSUJfMnzyzA83oU+Wwr D7is0qUtJTvoAJgovrlJWBSD2Q== X-Received: by 2002:a17:902:9a95:: with SMTP id w21mr6812521plp.118.1551465018990; Fri, 01 Mar 2019 10:30:18 -0800 (PST) Received: from gnomeregan.cam.corp.google.com ([2620:15c:6:14:ad22:1cbb:d8fa:7d55]) by smtp.googlemail.com with ESMTPSA id s6sm28455672pfe.37.2019.03.01.10.30.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Mar 2019 10:30:17 -0800 (PST) Subject: Re: [PATCH] percpu/module resevation: change resevation size iff X86_VSMP is set To: Eial Czerwacki , dennis@kernel.org, tj@kernel.org, cl@linux.com Cc: linux-kernel@vger.kernel.org, Shai Fultheim , Oren Twaig References: <1548071251-1849-1-git-send-email-eial@scalemp.com> From: Barret Rhoden Message-ID: Date: Fri, 1 Mar 2019 13:30:15 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <1548071251-1849-1-git-send-email-eial@scalemp.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi - On 01/21/2019 06:47 AM, Eial Czerwacki wrote: > Your main issue was that you only sent this patch to LKML, but not the maintainers of the file. If you don't, your patch might get lost. To get the appropriate people and lists, run: scripts/get_maintainer.pl YOUR_PATCH.patch. For this patch, you'll get this: Dennis Zhou (maintainer:PER-CPU MEMORY ALLOCATOR) Tejun Heo (maintainer:PER-CPU MEMORY ALLOCATOR) Christoph Lameter (maintainer:PER-CPU MEMORY ALLOCATOR) linux-kernel@vger.kernel.org (open list) I added the three maintainers to this email. I have a few minor comments below. > [PATCH] percpu/module resevation: change resevation size iff X86_VSMP is set You misspelled 'reservation'. Also, I'd just say: "percpu: increase module reservation size if X86_VSMP is set". ('change' -> 'increase'), only says 'reservation' once.) > as reported in bug #201339 (https://bugzilla.kernel.org/show_bug.cgi?id=201339) I think you can add a tag for this right above your Signed-off-by tags. e.g.: Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=201339 > by enabling X86_VSMP, INTERNODE_CACHE_BYTES's definition differs from the default one > causing the struct size to exceed the size ok 8KB. ^of Which struct are you talking about? I have one in mind, but others might not know from reading the commit message. I ran into this on https://bugzilla.kernel.org/show_bug.cgi?id=202511. In that case, it was because modules (drm and amdkfd) were using DEFINE_SRCU, which does a DEFINE_PER_CPU on struct srcu_data, and that used ____cacheline_internodealigned_in_smp. > > in order to avoid such issue, increse PERCPU_MODULE_RESERVE to 64KB if CONFIG_X86_VSMP is set. ^increase > > the value was caculated on linux 4.20.3, make allmodconfig all and the following oneliner: ^calculated > for f in `find -name *.ko`; do echo $f; readelf -S $f |grep perc; done |grep data..percpu -B 1 |grep ko |while read r; do echo -n "$r: "; objdump --syms --section=.data..percpu $r|grep data |sort -n |awk '{c++; d=strtonum("0x" $1) + strtonum("0x" $5); if (m < d) m = d;} END {printf("%d vars-> last addr 0x%x ( %d )\n", c, m, m)}' ; done |column -t |sort -k 8 -n | awk '{print $8}'| paste -sd+ | bc Not sure how useful the one-liner is, versus a description of what you're doing. i.e. "the size of all module percpu data sections, or something." Also, how close was that calculated value to 64K? If more modules start using DEFINE_SRCU, each of which uses 8K, then that 64K might run out. Thanks, Barret > Signed-off-by: Eial Czerwacki > Signed-off-by: Shai Fultheim > Signed-off-by: Oren Twaig > --- > include/linux/percpu.h | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/include/linux/percpu.h b/include/linux/percpu.h > index 70b7123..6b79693 100644 > --- a/include/linux/percpu.h > +++ b/include/linux/percpu.h > @@ -14,7 +14,11 @@ > > /* enough to cover all DEFINE_PER_CPUs in modules */ > #ifdef CONFIG_MODULES > +#ifdef X86_VSMP > +#define PERCPU_MODULE_RESERVE (1 << 16) > +#else > #define PERCPU_MODULE_RESERVE (8 << 10) > +#endif > #else > #define PERCPU_MODULE_RESERVE 0 > #endif >