Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <55c95c15-ccad-bb31-be87-ad17db7cb02a@fb.com>
Date:   Wed, 3 Nov 2021 21:23:29 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.2.1
Subject: Re: [RFC PATCH v3 0/3] Introduce BPF map tracing capability
Content-Language: en-US
To:     Alexei Starovoitov <alexei.starovoitov@gmail.com>,
        Joe Burton <jevburton.kernel@gmail.com>
CC:     Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Network Development <netdev@vger.kernel.org>,
        bpf <bpf@vger.kernel.org>, Petar Penkov <ppenkov@google.com>,
        Stanislav Fomichev <sdf@google.com>,
        Joe Burton <jevburton@google.com>
References: <20211102021432.2807760-1-jevburton.kernel@gmail.com>
 <20211103001245.muyte7exph23tmco@ast-mbp.dhcp.thefacebook.com>
 <fcec81dd-3bb9-7dcf-139d-847538b6ad20@fb.com>
 <CAN22DihwJ7YDFSPk+8CCs0RcSWvZOpNV=D1u+42XabztS6hcKQ@mail.gmail.com>
 <CAADnVQJ_ger=Tjn=9SuUTES6Tt5k_G0M+6T_ELzFtw_cSVs83A@mail.gmail.com>
From:   Yonghong Song <yhs@fb.com>
In-Reply-To: <CAADnVQJ_ger=Tjn=9SuUTES6Tt5k_G0M+6T_ELzFtw_cSVs83A@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NUJrQ0VTa2ZIY0ljQ2RWeWFza1VkaXltS3RNaVRhWmpUOTF0Q0hDbGF2L1VD?=
 =?utf-8?B?aFRSNE9jU3VOTHlpUnlNdU9nNEJidlRnWUFlVkovOWJXS0ZHZ0ttUEpLb0I1?=
 =?utf-8?B?YU9HS2d1dkZYYUNqTXAwS3JLcFFQNDJkaHA5My9nVFFmOE9OK0pSWUpRZ3pv?=
 =?utf-8?B?WDlaMnBOQ21sRUFyVWVxS1B4U1loMCtTdUNSaDg4Sld1aDUvT2ZzdmRmWDFY?=
 =?utf-8?B?bzdBVkJRNS9XYzViL0ZPU1BSeWlVcFV5QjFLLzA1MVgvN2ZVb1lBN3lLMEdO?=
 =?utf-8?B?M1pTeVQrVGhIVEozMk5obDRTTmpyZUlPZUVDamZ4S01KcVdQOW5ReUo4L0Ey?=
 =?utf-8?B?WWZRMFl1OFE2ZTBaUU9mQVN1cWg0dEFxbHNrYkh5cERVYnZSSlE3NHVBQVNR?=
 =?utf-8?B?aENqVFQ3WDFaVzRHUEs2dkJxNk1RbUY1TllXM2VuQkpGTW1CTHZ4WEk2UTRV?=
 =?utf-8?B?aDk1WU1EQlo0NVY2Yk5aNXJvd2NnUTlZb0MzOWxLSStBV1dzK0RCaU4vS0JH?=
 =?utf-8?B?Ni9aMGdIQ3RYYzB0Z3RoTi9xbU1vVmoxd3YzK1JUNzhSNjhjSC82VXZJSUVN?=
 =?utf-8?B?eXBmMWpxZUR3WjI1U3dldHBWZ1o2RFc3V3lXTzl4ZExudnp2Vkhqc1BiaENM?=
 =?utf-8?B?dEVqOUlMRUVjaGthNnVBVXhyankzYXplVzlxSVBBL1NpeHB5Q1VOQVhFekpM?=
 =?utf-8?B?MjRtYXFUcENqK2dQSTBVaVA2NnNFR3BTdUhpd2h2ZmNOa3BxYnY4b0FGY2dw?=
 =?utf-8?B?blk2NVUvYklZUVJxUytDb3prTTJlQVM4c1JQMWJCR0s5K1ovc1VMM2FnNHF2?=
 =?utf-8?B?VWJNd1N5Tm9lZnp5bTl3ZFp6amwxcHRwZG1SSE95M3p0M0VlY3djemRpV3hD?=
 =?utf-8?B?VExFUEZjeTNtRlRYblhGemRhZWhvRDlXWVZqbFM4QVQ1eDRsOHNMWFhqS0dJ?=
 =?utf-8?B?L25ueVZlejZ5ZTVKOGlrQTRzQlJtOStzaUpSQloxNlowNFYrNHJWeWlMMEYr?=
 =?utf-8?B?Q3ovRDUxMGlTR1QxNzBDN2NKZGdLM2QzbTB5TS9YZEZ1Nk1IVFZ4Y3M2UlND?=
 =?utf-8?B?TDdCR0Q0cXQ1OFR0M1lNdkRvYm5NcVhadG5jdW03RXVvaUpCSVBlancyTmcy?=
 =?utf-8?B?SkNTM0p0cXppWGFkeTRRaUtTVGVCV1BSb1NyWGhpMUtEenlxMHJKeFV5Qml2?=
 =?utf-8?B?dVBBOFByZ0Z1OVNlaFFtOEZQcUtvK3NFZmJaUlB2Y3ZaWWRQSlFXZ3B2SklC?=
 =?utf-8?B?b0NIT0ZhUGc2enRWUjRFckdybm81cWRBcndUMTNERXZOdDBGQmdwWGVBZmI5?=
 =?utf-8?B?RFdYbXRVa1ZYbmxCVTJQeHNWWXduTmtuNU42SWxsZkNqcXpZR3hIaXJZaU1o?=
 =?utf-8?B?Vm5BWXloRWlEZkthTWZrK0F4WFZwcEJvS0tpU2NNaEVicmQ4K01pOXk5T0Fo?=
 =?utf-8?B?c0JvMUZRd1dGUXh6QzdnR3ZXbHY5L3hkUk5CL0FXU3lWK254K05kN1Z1K2dt?=
 =?utf-8?B?czZrOVBHZ3BzdXdsWnQwd0NZQUlMZzVjOWhsdGhyN1p3cXlzWGJsWnN1Q0Rm?=
 =?utf-8?B?RTNtRXJSUFZNQkpmTDg5TTV4VmtRbVhXOEx2TG9RbHFiR3ZiOFQ0TzdjVTVn?=
 =?utf-8?B?aEo5alVUMDJnaFBSaGpoOWRtSktwN2FyZ2ZIVjU3bnN4eGJYVEMrcmZRVGFS?=
 =?utf-8?B?ZUhvUWd6ZjRmemlTWEs0SlprcnluYkNkdzhFQWpJZllTMXJOV0pHWDEyM1pJ?=
 =?utf-8?B?c2FLcXhsMGpJNlNGNHNqWS9ZVndSNnEvZHFyMVRNZEQ0SDFUYzB2V1pFZ1Ey?=
 =?utf-8?B?Nnl1M3dBK2dJa1dTUmVGSU1LUGp1Wk9FQ2tHUzZ5WUNsOC9DdElxdTE2NzFO?=
 =?utf-8?B?aTNYbVkvSHFCeHpGajlpc2lyYXVsMU90eVdMK01OUGV0OFZWNmRmZmVUWTVo?=
 =?utf-8?B?ODdrL25WNjM0ZE1obW4yYkJnOWhnUmd3Zml4cEhhaFlvQjBLNWNGejJLZmhq?=
 =?utf-8?B?SDlRaXQ1UDBPNDlHcDl5NFh1alcwVWFFR0ZLYVpleVo2R3Z3aFR0VkQyUEJk?=
 =?utf-8?B?aXdrSVF5cFlSQkNpN1hkTytmcFRlOXFWY21xRlZlalJPZDhQNHR5VWxjNk04?=
 =?utf-8?Q?y1AwVKKLG3q2TIBUFzCehEYBL?=
X-MS-Exchange-CrossTenant-Network-Message-Id: c157623b-94ea-4445-5a41-08d99f4adc5f
X-MS-Exchange-CrossTenant-AuthSource: SN6PR1501MB2064.namprd15.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Nov 2021 04:23:32.4817
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kJbycH5s4fn6TDHxAIIkyn4Wqkz1ujx4xwr0+AHWYbi7+zUdezmo4js2glbsV2n9
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR15MB2335
X-OriginatorOrg: fb.com
X-Proofpoint-ORIG-GUID: 30u3QrGvVm6NTE2wXVBe_QtqZzPx8VHY
X-Proofpoint-GUID: 30u3QrGvVm6NTE2wXVBe_QtqZzPx8VHY
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475
 definitions=2021-11-04_01,2021-11-03_01,2020-04-07_01
X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 bulkscore=0 spamscore=0
 clxscore=1015 malwarescore=0 phishscore=0 mlxscore=0 impostorscore=0
 lowpriorityscore=0 mlxlogscore=963 adultscore=0 suspectscore=0
 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2110150000 definitions=main-2111040024
X-FB-Internal: deliver
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 11/3/21 10:49 AM, Alexei Starovoitov wrote:
> On Wed, Nov 3, 2021 at 10:45 AM Joe Burton <jevburton.kernel@gmail.com> wrote:
>>
>> Sort of - I hit issues when defining the function in the same
>> compilation unit as the call site. For example:
>>
>>    static noinline int bpf_array_map_trace_update(struct bpf_map *map,
>>                  void *key, void *value, u64 map_flags)
> 
> Not quite :)
> You've had this issue because of 'static noinline'.
> Just 'noinline' would not have such issues even in the same file.

This seems not true. With latest trunk clang,

[$ ~/tmp2] cat t.c
int __attribute__((noinline)) foo() { return 1; }
int bar() { return foo() + foo(); }
[$ ~/tmp2] clang -O2 -c t.c
[$ ~/tmp2] llvm-objdump -d t.o

t.o:    file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <foo>:
        0: b8 01 00 00 00                movl    $1, %eax
        5: c3                            retq
        6: 66 2e 0f 1f 84 00 00 00 00 00 nopw    %cs:(%rax,%rax)

0000000000000010 <bar>:
       10: b8 02 00 00 00                movl    $2, %eax
       15: c3                            retq
[$ ~/tmp2]

The compiler did the optimization and the original noinline function 
still in the binary.

With a single foo() in bar() has the same effect.

asm("") indeed helped preserve the call.

[$ ~/tmp2] cat t.c
int __attribute__((noinline)) foo() { asm(""); return 1; }
int bar() { return foo() + foo(); }
[$ ~/tmp2] clang -O2 -c t.c
[$ ~/tmp2] llvm-objdump -d t.o

t.o:    file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <foo>:
        0: b8 01 00 00 00                movl    $1, %eax
        5: c3                            retq
        6: 66 2e 0f 1f 84 00 00 00 00 00 nopw    %cs:(%rax,%rax)

0000000000000010 <bar>:
       10: 50                            pushq   %rax
       11: e8 00 00 00 00                callq   0x16 <bar+0x6>
       16: e8 00 00 00 00                callq   0x1b <bar+0xb>
       1b: b8 02 00 00 00                movl    $2, %eax
       20: 59                            popq    %rcx
       21: c3                            retq
[$ ~/tmp2]

Note with asm(""), foo() is called twice, but the compiler optimization
knows foo()'s return value is 1 so it did calculation at compiler time,
assign the 2 to %eax and returns.

Having a single foo() in bar() has the same effect.

[$ ~/tmp2] cat t.c
int __attribute__((noinline)) foo() { return 1; }
int bar() { return foo(); }
[$ ~/tmp2] clang -O2 -c t.c
[$ ~/tmp2] llvm-objdump -d t.o

t.o:    file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <foo>:
        0: b8 01 00 00 00                movl    $1, %eax
        5: c3                            retq
        6: 66 2e 0f 1f 84 00 00 00 00 00 nopw    %cs:(%rax,%rax)

0000000000000010 <bar>:
       10: b8 01 00 00 00                movl    $1, %eax
       15: c3                            retq
[$ ~/tmp2]

I checked with a few llvm compiler engineers in Facebook.
They mentioned there is nothing preventing compiler from doing
optimization like poking inside the noinline function and doing
some optimization based on that knowledge.

> 
> Reminder: please don't top post and trim your replies.
>