Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Date:   Wed, 27 Sep 2023 10:46:17 +0530
From:   Wyes Karny <wyes.karny@amd.com>
To:     Doug Smythies <dsmythies@telus.net>
Cc:     lenb@kernel.org, linux-pm@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH] tools/power turbostat: Increase the limit for fd opened
Message-ID: <ZRO6ofa9sHbKJBCv@BLR-5CG13462PL.amd.com>
References: <20230922092823.478042-1-wyes.karny@amd.com>
 <004501d9ed96$16af12f0$440d38d0$@telus.net>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <004501d9ed96$16af12f0$440d38d0$@telus.net>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sSHpXgqSjgZnXZbXh2MgIZpqFgbqPAR/rM82LUXg7VHl/Creyk/AK1lfP6DS?=
 =?us-ascii?Q?//RwOJGP4oKWWVSCZpHREvL8pZYb59bmWqVJVPkARhMCoo08ha/GIAVCpl6C?=
 =?us-ascii?Q?PVNzJcdsNic6/ApEo5i6P7oQrVkjXBrSIbTj4El8FMxTou+Wtt1vAcHb7Vh6?=
 =?us-ascii?Q?fA4B/jX2jECOqO6nA6S9QuB87nH9CrFdjdxdlEQ6NCCWZ3pfp1VLJSj4Xn85?=
 =?us-ascii?Q?xjhd9aRmFZpGfIrQj+6hU+7eZle3jNz0BcDAKTRH3kn8CNw65hN6yodmOrXj?=
 =?us-ascii?Q?MgpmdzzZ/q2myxKN2EEs16Fj5TZiKpe3Yb43OZBfD/j23npHRrZCo1UAM9m0?=
 =?us-ascii?Q?y96Q48z8QInvpIVgPAyv2oySOZwfZCrFIk1pLd+7I8Ckr99wJ28UEm2sQkYl?=
 =?us-ascii?Q?favTIJWKmL1TxHeKi2KFHSOSnhY3h9hDGR9HhbE9EfEkOW2rYQcAyrarDy4d?=
 =?us-ascii?Q?sts6lpuRsoG2FlAGWfWBzEjhpIsecQ8fZpgtcPHipk/BTF9e2sYc7jSdYaUG?=
 =?us-ascii?Q?ZHwXqDbkazFlFTRjV26dJTUP7331RfgRSQ04jYWaVZfQg8kU++JfBsSdFSN9?=
 =?us-ascii?Q?uOr+uf+tRGN9BjjfAExIng54OMFHUWhlhmSwHJpbnjPfdPKe7BsjBw9q9jyo?=
 =?us-ascii?Q?ZxFIzoC4IzCyWgR2R7Ihode15/QbIlZlfhvwoyIXxWz6ca5T/I7l8HJx0NLM?=
 =?us-ascii?Q?S2ULzJVhNoTjcwxDJ3uXCLQRTzpJ79JGldATcf58f2R+n4IrLT6sVKKbLio7?=
 =?us-ascii?Q?oiNWAjp3dl7n6m/dX5oFiIusTLU4e5yXng9OsVyuH7+gxl/mEjBKicWYFs0M?=
 =?us-ascii?Q?II8nlTa3uVqm11/1JIg/O5D++Jvu3ijjOkxikk69ZIq7yfsAMzjOp8dUlzHe?=
 =?us-ascii?Q?7hDmM4HofXqILbCHVOgFHYU9tsqzLLfOu+Wq3VVsb8Az7/RtZqAIICXgGipK?=
 =?us-ascii?Q?rxYG8s24xuA0xtQqi8m3nKv6q1PgYFLecAioGhiogsi+lNwaeodhcv//rQL2?=
 =?us-ascii?Q?FRMz31VWDpM2PKaAm86SHDqfExZnXTzZHi9nizRcKM0sWT1dD9IkpQebeECM?=
 =?us-ascii?Q?lTDyopInsCFhza66M14kMTIruQt521j993Bf2eCQ+6iPq7L9udWLktel137e?=
 =?us-ascii?Q?0q1ApnnYwtTa47X/VrHkDofQATXrMIQMM2524DYH808JQpNAx1uD5uobwk1b?=
 =?us-ascii?Q?68GnG/dTAXM5KBA1E2Khi18eV9QNmoSGegojMMthrTPS9xXFAjDXYxAT96qV?=
 =?us-ascii?Q?M8Og99QTCEil5HeNPAiFJasK2IiDImxAQ0YvndxPkmsGXALgm0KsHnBSl9IH?=
 =?us-ascii?Q?q52WukVVbD0AWmGPe0puis6IxjtotBXT6XyGUsQAVTrIhwxqo2Xu1e18YE1Y?=
 =?us-ascii?Q?YTKjhtBD9kXPQ5RNKvOqrzg9uUwb1l8FWlnhPCcmEoKRS9SpB8O9YkLr0hUR?=
 =?us-ascii?Q?beYbryN3UkKHsQBl6Xv+dp+lbYNKneCXdzmeO6tTsL9AO10ESgBtIIOWVD5C?=
 =?us-ascii?Q?6NczDrJhjg4V6t0eeV/ZhFFOwtyXZzIIXccGxLRMhaAsHRh4YJTstQ1Eb85C?=
 =?us-ascii?Q?ddBDxPnOIhoEUAvAwkCW8U777GLrnU2ReEnhI2Qr?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3526c2e3-0fcf-49c4-16fe-08dbbf18e963
X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB3876.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2023 05:16:32.2299
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DWPDRcocp34XKSX/Kr+NW0hj9z0ljT92tpepuWd97khb+Hmps1O2DYaAvPNwOqVaBQMZWMb+i+TMpR0nq34y9w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6784
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 26 Sep 2023 22:21:31 -0700 (PDT)

Hi Doug,

Thanks for taking a look at the patch.

On 22 Sep 13:48, Doug Smythies wrote:
> On 2023.09.22 02:28 Wyes Karny wrote:
> 
> > When running turbostat, a system with 512 cpus reaches the limit for
> 
> Suggest" ... reaches the default limit for..."
> 
> > maximum number of file descriptors that can be opened. To solve this
> > problem, the limit is raised to 2^15, which is a large enough number.
> >
> > Below data is collected from AMD server systems while running turbostat:
> >
> > |-----------+-------------------------------|
> > | # of cpus | # of opened fds for turbostat |
> > |-----------+-------------------------------|
> > | 128       | 260                           |
> > |-----------+-------------------------------|
> > | 192       | 388                           |
> > |-----------+-------------------------------|
> > | 512       | 1028                          |
> > |-----------+-------------------------------|
> 
> The number of open files is a function of what is being "show"ed or "hide"en.
> They can also increase beyond the above 2 X (# of CPUs) + 4 number
> via the --add directive. 
> >
> > So, the new max limit would be sufficient up to 2^14 cpus.
> 
> Well, not quiet, but the point is valid.
> 
> Normally, I would assume that a server with a large number of
> CPUs would have set a much higher limit of the number of open
> files than the default. I use 131,072 and so this patch reduces the
> maximum.

I think below will fix the problem.

+#define MAX_NOFILE 0x8000
+
+void set_rlimit(void)
+{
+       struct rlimit limit;
+
+       if(getrlimit(RLIMIT_NOFILE, &limit) < 0) {
+               err(1, "Failed to get rlimit");
+       }
+
+       if (limit.rlim_max < MAX_NOFILE)
+               limit.rlim_max = MAX_NOFILE;
+       if (limit.rlim_cur < MAX_NOFILE)
+               limit.rlim_cur = MAX_NOFILE;
+
+       if (setrlimit(RLIMIT_NOFILE, &limit) < 0) {
+               err(1, "Failed to set rlimit");
+       }
+       return;
+}


Is this looks okay to you?

Thanks,
Wyes
> 
> Unpatched:
> root@s19:~# cat /proc/47043/limits | grep "Max open files"
> Max open files            131072               131072               files
> 
> Patched:
> root@s19:~# cat /proc/47032/limits | grep "Max open files"
> Max open files            32768                32768                files
> 
> Anyway:
> 
> Reviewed-by: Doug Smythies <dsmythies@telus.net>
> Tested-by: Doug Smythies <dsmythies@telus.net>
> 
> >
> > Signed-off-by: Wyes Karny <wyes.karny@amd.com>
> > ---
> >  tools/power/x86/turbostat/turbostat.c | 15 +++++++++++++++
> >  1 file changed, 15 insertions(+)
> >
> > diff --git a/tools/power/x86/turbostat/turbostat.c b/tools/power/x86/turbostat/turbostat.c
> > index 9a10512e3407..23f1fe58289a 100644
> > --- a/tools/power/x86/turbostat/turbostat.c
> > +++ b/tools/power/x86/turbostat/turbostat.c
> > @@ -6717,6 +6717,18 @@ void cmdline(int argc, char **argv)
> >       }
> >  }
> >
> > +void set_rlimit(void)
> > +{
> > +     struct rlimit limit;
> > +
> > +     limit.rlim_cur = 0x8000;
> > +     limit.rlim_max = 0x8000;
> > +
> > +     if (setrlimit(RLIMIT_NOFILE, &limit) < 0) {
> > +             err(1, "Failed to set rlimit");
> > +     }
> > +}
> > +
> >  int main(int argc, char **argv)
> >  {
> >       outf = stderr;
> > @@ -6729,6 +6741,9 @@ int main(int argc, char **argv)
> >
> >       probe_sysfs();
> >
> > +     if (!getuid())
> > +             set_rlimit();
> > +
> >       turbostat_init();
> >
> >       msr_sum_record();
> > --
> > 2.34.1
> >
>