Date:   Thu, 16 Dec 2021 10:14:49 -0800
From:   Martin KaFai Lau <kafai@fb.com>
To:     Stanislav Fomichev <sdf@google.com>,
        Pavel Begunkov <asml.silence@gmail.com>
CC:     <netdev@vger.kernel.org>, <bpf@vger.kernel.org>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Song Liu <songliubraving@fb.com>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3] cgroup/bpf: fast path skb BPF filtering
Message-ID: <20211216181449.p2izqxgzmfpknbsw@kafai-mbp.dhcp.thefacebook.com>
References: <Yboc/G18R1Vi1eQV@google.com>
 <b2af633d-aaae-d0c5-72f9-0688b76b4505@gmail.com>
 <Ybom69OyOjsR7kmZ@google.com>
 <634c2c87-84c9-0254-3f12-7d993037495c@gmail.com>
 <Yboy2WwaREgo95dy@google.com>
 <e729a63a-cded-da9c-3860-a90013b87e2d@gmail.com>
 <CAKH8qBv+GsPz3JTTmLZ+Q2iMSC3PS+bE1xOLbxZyjfno7hqpSA@mail.gmail.com>
 <92f69969-42dc-204a-4138-16fdaaebb78d@gmail.com>
 <CAKH8qBuZxBen871AWDK1eDcxJenK7UkSQCZQsHCPhk6nk9e=Ng@mail.gmail.com>
 <7ca623df-73ed-9191-bec7-a4728f2f95e6@gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <7ca623df-73ed-9191-bec7-a4728f2f95e6@gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?li57LIqPN+XDB7e3z8CEHboLPVpRz7edGpXoffnoO9L+r8ILkicLDPQKv1tA?=
 =?us-ascii?Q?vNDf299x1L3pY8pOySHXMEC5hQFoizkWzgTByGNVFHt6iuB2kTsZKGFkFRhR?=
 =?us-ascii?Q?PT33KQaLvglowpxeUwi1MvUecciGlvh8S8m6trkk5I8zkVWmQIhXpJXuzEy6?=
 =?us-ascii?Q?WmIv2K96EKXS6DzEQAdD4E8zfc3OIwBNibRMbbjws7n/ytr4ypn6DPgVABxS?=
 =?us-ascii?Q?xSCne6s1L/EV7O9umuzZWGRGPDFTdXDbNQc58bXYzeplbJy7YqwIxO3Bfgml?=
 =?us-ascii?Q?An4vZpXF0Qrj4FUyAai8BJEVlJI36SMFVF4QNy+vUKr7qrqwmD3B/KLERokx?=
 =?us-ascii?Q?uIbiOUlasbhW+abD9kiSYoywab8cdFEA1rJSdrChc1Xuu+4i2DgJR0sQShts?=
 =?us-ascii?Q?JMFuVJ+2+MRuZcJ9LT6qLpjAtIMGbCZX2buNq6Zgwzbky1PtY+xHf0928w4Y?=
 =?us-ascii?Q?mNeViKDInURDFNHgnVLaKA1OePeSxlNYzqZvLKHLY+Pir7zAPw9Junq6hai0?=
 =?us-ascii?Q?Qk+/7jzZ4RhOIzuKbc6QhAV64IxvGf2A5Jep4oci0oO0kroIHGsoxKDJEzwr?=
 =?us-ascii?Q?JMDx81F275Z/jJDC40ZxIVTTUpnYuDjJmpKBjU4ndyiz4j6jCmBHQn46anRV?=
 =?us-ascii?Q?5CIL07CeGRv+V5iHS04+DjnLqqAhtr7QAU8daq8e4lsv/e5FH7Fv3jXyWRNm?=
 =?us-ascii?Q?KZ2cNvLSVcS47QstmGjEiWmOUe81Gzbf/+alMTmc+pklO6IACeRjSEEZ41gv?=
 =?us-ascii?Q?IK5jB/6I8aXR24nM9nyPtmltoKdYox4jA300ollqMLygAllYRxYC1nm55tis?=
 =?us-ascii?Q?Vmh9OyC5HvLppVHjj3zhMnK9IC+BAoowoPX+CV4NvndrwpNL70nzqpswOeY2?=
 =?us-ascii?Q?uLHzbKTvXXwDOxNsrllnjry7TS3+fAt5cWQeh4486tE88d8dUo3Ffv2f6sFw?=
 =?us-ascii?Q?5phnXSU09NJ1UMju5O3NfQl3o5vbsAgf32DdITVBemgVm94qN3UFl/lE+/Av?=
 =?us-ascii?Q?vvOCStVtM3lzHCgxmalDRgY9xKXCRdt1YlwYVEUWyYGsQBcYZwe+fM751KEF?=
 =?us-ascii?Q?2seqknJ4kNcwl+I9aAYOQKywOShM0NTdWxffTkkAiSjSBp4G4BKzzYMf/rxN?=
 =?us-ascii?Q?bj4zE2YFUWnZMEDl4pk3gu0IwXEWc5LpZdmcAw+i69xTc/bLboev3jNOsnen?=
 =?us-ascii?Q?Pgsl6f4koWIEHMHFXcSK320/aMHtWSV7hzs56xB9GpBmA2uzJsgYIAHdJ+xH?=
 =?us-ascii?Q?dUFsTZai0u+JaQgTKMS1ZoiqqQx1HHsJpvWWM5sRxwjCZLPZHNQ+F7D2UQQe?=
 =?us-ascii?Q?dUt71Od98XJoccTEwmTEvVegmH7TODeu6uopkkj2x19rl2yjOXCfrrhHbN9Z?=
 =?us-ascii?Q?P1NoqyH1wGKMjSrzdJ5K2gTTF/ZcQrvyD7X/AAjfNUPitnRJWhSSw5mIOec9?=
 =?us-ascii?Q?rXQxcAGbvtQduLosX1jcvn3pdHYMJ/X6+SeUJk3bYL6J43H5S6JExUmXhXmn?=
 =?us-ascii?Q?Us/qz1sCaGUAAd+EwyZzt0Pax0IC8qo3yu2zFH44emNxZ9rZi95Vwbz+PwbT?=
 =?us-ascii?Q?JPENVFwqO+y0Q2L9HKy/TW4X3T0ti9IVzYmI0Z/8?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cfff7632-91d9-434f-9d5a-08d9c0bff4bd
X-MS-Exchange-CrossTenant-AuthSource: SA1PR15MB5016.namprd15.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Dec 2021 18:14:52.8512
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8DZjnMFQ70HER0BKXK9wS/EMppw8iUtVnVwckYL08omj05H2GjT0x5NErx4CgRvx
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR15MB4323
X-OriginatorOrg: fb.com
X-Proofpoint-ORIG-GUID: LjsPYhw3c-OmxjAKEfK_on3fCw_nmKOL
X-Proofpoint-GUID: LjsPYhw3c-OmxjAKEfK_on3fCw_nmKOL
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.11.62.513
 definitions=2021-12-16_06,2021-12-16_01,2021-12-02_01
X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 suspectscore=0
 mlxscore=0 priorityscore=1501 adultscore=0 clxscore=1011
 lowpriorityscore=0 phishscore=0 mlxlogscore=999 spamscore=0 bulkscore=0
 impostorscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2110150000 definitions=main-2112160103
X-FB-Internal: deliver
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Dec 16, 2021 at 01:21:26PM +0000, Pavel Begunkov wrote:
> On 12/15/21 22:07, Stanislav Fomichev wrote:
> > On Wed, Dec 15, 2021 at 11:55 AM Pavel Begunkov <asml.silence@gmail.com> wrote:
> > > 
> > > On 12/15/21 19:15, Stanislav Fomichev wrote:
> > > > On Wed, Dec 15, 2021 at 10:54 AM Pavel Begunkov <asml.silence@gmail.com> wrote:
> > > > > 
> > > > > On 12/15/21 18:24, sdf@google.com wrote:
> [...]
> > > > > > I can probably do more experiments on my side once your patch is
> > > > > > accepted. I'm mostly concerned with getsockopt(TCP_ZEROCOPY_RECEIVE).
> > > > > > If you claim there is visible overhead for a direct call then there
> > > > > > should be visible benefit to using CGROUP_BPF_TYPE_ENABLED there as
> > > > > > well.
> > > > > 
> > > > > Interesting, sounds getsockopt might be performance sensitive to
> > > > > someone.
> > > > > 
> > > > > FWIW, I forgot to mention that for testing tx I'm using io_uring
> > > > > (for both zc and not) with good submission batching.
> > > > 
> > > > Yeah, last time I saw 2-3% as well, but it was due to kmalloc, see
> > > > more details in 9cacf81f8161, it was pretty visible under perf.
> > > > That's why I'm a bit skeptical of your claims of direct calls being
> > > > somehow visible in these 2-3% (even skb pulls/pushes are not 2-3%?).
> > > 
> > > migrate_disable/enable together were taking somewhat in-between
> > > 1% and 1.5% in profiling, don't remember the exact number. The rest
> > > should be from rcu_read_lock/unlock() in BPF_PROG_RUN_ARRAY_CG_FLAGS()
> > > and other extra bits on the way.
> > 
> > You probably have a preemptiple kernel and preemptible rcu which most
> > likely explains why you see the overhead and I won't (non-preemptible
> > kernel in our env, rcu_read_lock is essentially a nop, just a compiler
> > barrier).
> 
> Right. For reference tried out non-preemptible, perf shows the function
> taking 0.8% with a NIC and 1.2% with a dummy netdev.
> 
> 
> > > I'm skeptical I'll be able to measure inlining one function,
> > > variability between boots/runs is usually greater and would hide it.
> > 
> > Right, that's why I suggested to mirror what we do in set/getsockopt
> > instead of the new extra CGROUP_BPF_TYPE_ENABLED. But I'll leave it up
> > to you, Martin and the rest.
I also suggested to try to stay with one way for fullsock context in v2
but it is for code readability reason.

How about calling CGROUP_BPF_TYPE_ENABLED() just next to cgroup_bpf_enabled()
in BPF_CGROUP_RUN_PROG_*SOCKOPT_*() instead ?
It is because both cgroup_bpf_enabled() and CGROUP_BPF_TYPE_ENABLED()
want to check if there is bpf to run before proceeding everything else
and then I don't need to jump to the non-inline function itself to see
if there is other prog array empty check.

Stan, do you have concern on an extra inlined sock_cgroup_ptr()
when there is bpf prog to run for set/getsockopt()?  I think
it should be mostly noise from looking at
__cgroup_bpf_run_filter_*sockopt()?