Currently stackmaps can't be iterated over. The keys are obtained
through other maps and look ups have to be performed. In new usecases,
its useful to be able to iterate over the stackmap independently.
Implement bpf_get_next_key to make this possible.
More details of use case:
Currently iterating over stack maps is done like so, for example
in BCC tools offcputime script:
1. Obtain keys from the 'counts' hash map.
2. Look up stackmap with each of the keys obtained from step 1.
This makes the iteration of stackmap dependent on the counts map.
In a new tool I'm working on called BPFd [1], it gives a huge speed
up when I dump entire maps before transmitting them to BCC tools
on a different machine [2]. This patch makes it possible to dump
stackmaps independent of other maps.
Tested on x86 and arm64 machines.
[1] https://lwn.net/Articles/744522/
[2] https://github.com/joelagnel/bpfd/issues/8
Cc: Alexei Starovoitov <[email protected]>
Cc: Daniel Borkmann <[email protected]>
Cc: Brendan Gregg <[email protected]>
Cc: Brenden Blanco <[email protected]>
Signed-off-by: Joel Fernandes <[email protected]>
---
kernel/bpf/stackmap.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
index a15bc636cc98..b0bf7d009f76 100644
--- a/kernel/bpf/stackmap.c
+++ b/kernel/bpf/stackmap.c
@@ -228,7 +228,23 @@ int bpf_stackmap_copy(struct bpf_map *map, void *key, void *value)
static int stack_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
{
- return -EINVAL;
+ struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map);
+ u32 id = 0;
+
+ if (key) {
+ id = *(u32 *)key;
+ if (unlikely(id >= smap->n_buckets))
+ return -ENOENT;
+ id++;
+ }
+
+ for ( ; id < smap->n_buckets; id++) {
+ if (READ_ONCE(smap->buckets[id])) {
+ *(u32 *)next_key = id;
+ return 0;
+ }
+ }
+ return -ENOENT;
}
static int stack_map_update_elem(struct bpf_map *map, void *key, void *value,
--
2.16.0.rc1.238.g530d649a79-goog
On Wed, Jan 24, 2018 at 08:37:52PM -0800, Joel Fernandes wrote:
> Currently stackmaps can't be iterated over. The keys are obtained
> through other maps and look ups have to be performed. In new usecases,
> its useful to be able to iterate over the stackmap independently.
> Implement bpf_get_next_key to make this possible.
>
> More details of use case:
> Currently iterating over stack maps is done like so, for example
> in BCC tools offcputime script:
> 1. Obtain keys from the 'counts' hash map.
> 2. Look up stackmap with each of the keys obtained from step 1.
>
> This makes the iteration of stackmap dependent on the counts map.
> In a new tool I'm working on called BPFd [1], it gives a huge speed
> up when I dump entire maps before transmitting them to BCC tools
> on a different machine [2]. This patch makes it possible to dump
> stackmaps independent of other maps.
>
> Tested on x86 and arm64 machines.
>
> [1] https://lwn.net/Articles/744522/
> [2] https://github.com/joelagnel/bpfd/issues/8
>
> Cc: Alexei Starovoitov <[email protected]>
> Cc: Daniel Borkmann <[email protected]>
> Cc: Brendan Gregg <[email protected]>
> Cc: Brenden Blanco <[email protected]>
> Signed-off-by: Joel Fernandes <[email protected]>
> ---
> kernel/bpf/stackmap.c | 18 +++++++++++++++++-
> 1 file changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
> index a15bc636cc98..b0bf7d009f76 100644
> --- a/kernel/bpf/stackmap.c
> +++ b/kernel/bpf/stackmap.c
> @@ -228,7 +228,23 @@ int bpf_stackmap_copy(struct bpf_map *map, void *key, void *value)
>
> static int stack_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
> {
> - return -EINVAL;
> + struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map);
> + u32 id = 0;
did you check net-next tree before sending the patch?
Also please see Documentation/bpf/bpf_devel_QA.txt
On Wed, Jan 24, 2018 at 8:42 PM, Alexei Starovoitov
<[email protected]> wrote:
> On Wed, Jan 24, 2018 at 08:37:52PM -0800, Joel Fernandes wrote:
>> Currently stackmaps can't be iterated over. The keys are obtained
>> through other maps and look ups have to be performed. In new usecases,
>> its useful to be able to iterate over the stackmap independently.
>> Implement bpf_get_next_key to make this possible.
>>
>> More details of use case:
>> Currently iterating over stack maps is done like so, for example
>> in BCC tools offcputime script:
>> 1. Obtain keys from the 'counts' hash map.
>> 2. Look up stackmap with each of the keys obtained from step 1.
>>
>> This makes the iteration of stackmap dependent on the counts map.
>> In a new tool I'm working on called BPFd [1], it gives a huge speed
>> up when I dump entire maps before transmitting them to BCC tools
>> on a different machine [2]. This patch makes it possible to dump
>> stackmaps independent of other maps.
>>
>> Tested on x86 and arm64 machines.
>>
>> [1] https://lwn.net/Articles/744522/
>> [2] https://github.com/joelagnel/bpfd/issues/8
>>
>> Cc: Alexei Starovoitov <[email protected]>
>> Cc: Daniel Borkmann <[email protected]>
>> Cc: Brendan Gregg <[email protected]>
>> Cc: Brenden Blanco <[email protected]>
>> Signed-off-by: Joel Fernandes <[email protected]>
>> ---
>> kernel/bpf/stackmap.c | 18 +++++++++++++++++-
>> 1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
>> index a15bc636cc98..b0bf7d009f76 100644
>> --- a/kernel/bpf/stackmap.c
>> +++ b/kernel/bpf/stackmap.c
>> @@ -228,7 +228,23 @@ int bpf_stackmap_copy(struct bpf_map *map, void *key, void *value)
>>
>> static int stack_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
>> {
>> - return -EINVAL;
>> + struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map);
>> + u32 id = 0;
>
> did you check net-next tree before sending the patch?
Oh. I was a bit late to send it! Glad its done though.
> Also please see Documentation/bpf/bpf_devel_QA.txt
Sure, if you don't mind could you elaborate what I screwed up from the
QA with regard to this patch? Sorry if I missed something.
thanks,
- Joel