Received: by 2002:ab2:1149:0:b0:1f3:1f8c:d0c6 with SMTP id z9csp2827677lqz; Wed, 3 Apr 2024 09:37:30 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCUEq6gtZUCuzFp9G3UwuIy7ypxaxjvmWArckoyB9sBKMgwQTQoAEeNT/gvAyh8YQr4UAckQCEDXfui3Qb54jCFcF3w7hliEpfHw5DoTJQ== X-Google-Smtp-Source: AGHT+IHTFtz8zRcD2tnU6zsGXwzMWuoeIm9B6e841Wn8bD5/BmXKIDXtRcS/f9P3e1/iKYof+l93 X-Received: by 2002:a17:906:248d:b0:a4d:ffcb:1f4e with SMTP id e13-20020a170906248d00b00a4dffcb1f4emr2165426ejb.75.1712162250598; Wed, 03 Apr 2024 09:37:30 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1712162250; cv=pass; d=google.com; s=arc-20160816; b=YDnu01RCGnsHx6SYwWS+DfKdXxNvSe52LwiGoxZ/Nc61QRgTl/vDn4sROEH96yJ28g QDCHNkeDfJv/SjSBRf+5wymQvtWUGijBhkLlX9vTK5avzZ0io1pYQBAZvsbh3B69OyUW mPNLrJZfYsiWK/z5BXjv/n9I77+g665QnzoVVEFwjRrJswzWogFq6T9r/7XjhGFr7Lc+ M6pommYs+117ZwHK3z6HptH5uLMd7y+pUCFsC4CBQEE2E6M9AIvGcXnckH6PT/cEXiGd 22wLbXRbDFay820GNTiGBRGExX2Ka5TOu5f54tnTuWRLub19hh01j2yBOYcWfBy0w7Qe AfHA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature:dkim-signature; bh=XdI1yWISzkbEZXlSrjJuMRoad1P879v+C1CTXZuc5rg=; fh=9DBPHsFuCJxu8EkzZTMVl4PnbPNr8Fc3iszEELIgF1I=; b=oP5N8m0t9oEt6G/IdqXIOE8G/+mBVp/Io1+9N61NvAv7YOZJkJrOnTZBYyaks2gqQj RfC5jtp5TIsj0uGuKNNbYPwcIyNU4e+9p+oyAhk01gUD+0Dz526jV9qKBio84+4B5q8Q 6ZrSsU/djEbQ7EOzaF3ppcaXahDR8Lt8hr+3Ityj01MPQg/s3lK2GgaIDjT9V/nsy3ik KE0YuQxmpVswI8HbTV9njzQoA22DXC2C4ionWw2RzONDaEpaXSI0SHQMmt47W2GNRaAC q+SSS5TfHGnJ8vcnC8FotJF5A60hmwWw/tVRlymo8ZHXroS2xtFuY11eKZZEeAZq4mK4 zVgA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@suse.cz header.s=susede2_rsa header.b=pp8YHJR4; dkim=neutral (no key) header.i=@suse.cz header.s=susede2_ed25519; arc=pass (i=1 spf=pass spfdomain=suse.cz dkim=pass dkdomain=suse.cz); spf=pass (google.com: domain of linux-kernel+bounces-130193-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-130193-linux.lists.archive=gmail.com@vger.kernel.org" Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id he36-20020a1709073da400b00a4e41d4080dsi5200414ejc.1021.2024.04.03.09.37.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Apr 2024 09:37:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-130193-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.cz header.s=susede2_rsa header.b=pp8YHJR4; dkim=neutral (no key) header.i=@suse.cz header.s=susede2_ed25519; arc=pass (i=1 spf=pass spfdomain=suse.cz dkim=pass dkdomain=suse.cz); spf=pass (google.com: domain of linux-kernel+bounces-130193-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-130193-linux.lists.archive=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EBD811F2B060 for ; Wed, 3 Apr 2024 16:27:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 228FA1509B2; Wed, 3 Apr 2024 16:27:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="pp8YHJR4"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="K5rgSLPa" Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D9D9150990; Wed, 3 Apr 2024 16:27:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712161640; cv=none; b=PwsXb0jvGqHBgNrnwRHoEqUO2I4JJbVrSdzKk3syQTyLHtub0ZXYL82evQhLFwvg9ZKbMnOq1CdD3GC8vewBcaHScGHa9PYq6RMm1qIq6XEhW++E80UTmnAoNg2fMy0UF1bqZ2MLwWti8MwRNkBsV6e9d2rY7QQg5EeR8+SmNDE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712161640; c=relaxed/simple; bh=N283gQOcSt2JQpsVAyXiyv+tS3Qj6fbPQwINwOd8nM4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=PlmnO42q3m2Rhi7ZIHGFhFsBL7rDth9UnvTmGt4Hn6XKsAjsTkEEeaVi47vQ9nX1Jdp/1fw9aN8pHiXJjpMAPqe0xzJBb/qnhDxn+vr1H6tTQNj5yTabAe9woXHEUlkfS+TNM0UDd7Le2487VISZ6rCFvR1CStJTHsPXc/kUcV4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=pp8YHJR4; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=K5rgSLPa; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Received: from imap2.dmz-prg2.suse.org (imap2.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:98]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7EF8120BFC; Wed, 3 Apr 2024 16:27:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1712161636; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=XdI1yWISzkbEZXlSrjJuMRoad1P879v+C1CTXZuc5rg=; b=pp8YHJR4fTJm2T1lBCUoAkPPEIPZ8lz/knJwXua2Qa1l+9jaqb1hZq0vFzoChcozfd4JeV EOfwxvbuFehmSHhk+SNvvUxrdLOtcR5at9GkZllfdvyE7DKta4fAyzHamFa1LtT3aRvYPY qYnFJyWxGYi10DJtb0n6k+jEDxFQqRQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1712161636; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=XdI1yWISzkbEZXlSrjJuMRoad1P879v+C1CTXZuc5rg=; b=K5rgSLPaMctwGDfFzMRnnrpUnjA0W/u0tyoQeJo5xCuhtTEOcD1Q5Y2BDc6jXid4hSMkHY D04X3mM2Ls3mT5AA== Authentication-Results: smtp-out2.suse.de; dkim=none Received: from imap2.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap2.dmz-prg2.suse.org (Postfix) with ESMTPS id 706421331E; Wed, 3 Apr 2024 16:27:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap2.dmz-prg2.suse.org with ESMTPSA id CzkpG2SDDWaVCAAAn2gu4w (envelope-from ); Wed, 03 Apr 2024 16:27:16 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 125EFA0816; Wed, 3 Apr 2024 18:27:16 +0200 (CEST) Date: Wed, 3 Apr 2024 18:27:16 +0200 From: Jan Kara To: Tejun Heo Cc: Kent Overstreet , Kemeng Shi , akpm@linux-foundation.org, willy@infradead.org, jack@suse.cz, bfoster@redhat.com, dsterba@suse.com, mjguzik@gmail.com, dhowells@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCH v2 0/6] Improve visibility of writeback Message-ID: <20240403162716.icjbicvtbleiymjy@quack3> References: <20240327155751.3536-1-shikemeng@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Spam-Flag: NO X-Rspamd-Action: no action X-Spamd-Result: default: False [-0.36 / 50.00]; BAYES_HAM(-0.55)[80.84%]; MID_RHS_NOT_FQDN(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.996]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_COUNT_THREE(0.00)[3]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:98:from]; FREEMAIL_ENVRCPT(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; FREEMAIL_CC(0.00)[linux.dev,huaweicloud.com,linux-foundation.org,infradead.org,suse.cz,redhat.com,suse.com,gmail.com,vger.kernel.org,kvack.org]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DNSWL_BLOCKED(0.00)[2a07:de40:b281:106:10:150:64:167:received,2a07:de40:b281:104:10:150:64:98:from]; RCVD_TLS_LAST(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.com:email,imap2.dmz-prg2.suse.org:rdns,imap2.dmz-prg2.suse.org:helo] X-Spam-Level: X-Spam-Score: -0.36 X-Spamd-Bar: / X-Rspamd-Queue-Id: 7EF8120BFC On Thu 28-03-24 10:46:33, Tejun Heo wrote: > Hello, Kent. > > On Thu, Mar 28, 2024 at 04:22:13PM -0400, Kent Overstreet wrote: > > Most users are never going to touch tracing, let alone BPF; that's too > > much setup. But I can and do regularly tell users "check this, this and > > this" and debug things on that basis without ever touching their > > machine. > > I think this is where the disconnect is. It's not difficult to set up at > all. Nowadays, in most distros, it comes down to something like run "pacman > -S bcc" and then run "/usr/share/bcc/tools/biolatpcts" with these params or > run this script I'm attaching. It is a signficant boost when debugging many > different kernel issues. I strongly suggest giving it a try and getting used > to it rather than resisting it. Yeah, BPF is great and I use it but to fill in some cases from practice, there are sysadmins refusing to install bcc or run your BPF scripts on their systems due to company regulations, their personal fear, or whatever. So debugging with what you can achieve from a shell is still the thing quite often. Honza -- Jan Kara SUSE Labs, CR