Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: tc_classid access in skb bpf context
To:     Matthew Cover <matthew.cover@stackpath.com>,
        Alexei Starovoitov <ast@kernel.org>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
        "David S. Miller" <davem@davemloft.net>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "bpf@vger.kernel.org" <bpf@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Cc:     Matthew Cover <werekraken@gmail.com>
References: <BYAPR10MB2680B63C684345098E6E7669E3070@BYAPR10MB2680.namprd10.prod.outlook.com>
From:   Daniel Borkmann <daniel@iogearbox.net>
Message-ID: <73d5b951-2598-0d7f-5b6e-8925cc61989a@iogearbox.net>
Date:   Wed, 22 May 2019 10:37:18 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.3.0
MIME-Version: 1.0
In-Reply-To: <BYAPR10MB2680B63C684345098E6E7669E3070@BYAPR10MB2680.namprd10.prod.outlook.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 05/22/2019 01:52 AM, Matthew Cover wrote:
> __sk_buff has a member tc_classid which I'm interested in accessing from the skb bpf context.
> 
> A bpf program which accesses skb->tc_classid compiles, but fails verification; the specific failure is "invalid bpf_context access".
> 
> if (skb->tc_classid != 0)
>  return 1;
> return 0;
> 
> Some of the tests in tools/testing/selftests/bpf/verifier/ (those on tc_classid) further confirm that this is, in all likelihood, intentional behavior.
> 
> The very similar bpf program which instead accesses skb->mark works as desired.
> 
> if (skb->mark != 0)
>  return 1;
> return 0;

You should be able to access skb->tc_classid, perhaps you're using the wrong program
type? BPF_PROG_TYPE_SCHED_CLS is supposed to work (if not we'd have a regression).

> I built a kernel (v5.1) with 4 instances of the following line removed from net/core/filter.c to test the behavior when the instructions pass verification.
> 
>     switch (off) {
> -    case bpf_ctx_range(struct __sk_buff, tc_classid):
> ...
>         return false;
> 
> It appears skb->tc_classid is always zero within my bpf program, even when I verify by other means (e.g. netfilter) that the value is set non-zero.
> 
> I gather that sk_buff proper sometimes (i.e. at some layers) has qdisc_skb_cb stored in skb->cb, but not always.
> 
> I suspect that the tc_classid is available at l3 (and therefore to utils like netfilter, ip route, tc), but not at l2 (and not to AF_PACKET).

From tc/BPF context you can use it; it's been long time, but I think back then
we mapped it into cb[] so it can be used within the BPF context to pass skb data
around e.g. between tail calls, and cls_bpf_classify() when in direct-action mode
which likely everyone is/should-be using then maps that skb->tc_classid u16 cb[]
value to res->classid on program return which then in either sch_handle_ingress()
or sch_handle_egress() is transferred into the skb->tc_index.

> Is it impractical to make skb->tc_classid available in this bpf context or is there just some plumbing which hasn't been connected yet?
> 
> Is my suspicion that skb->cb no longer contains qdisc_skb_cb due to crossing a layer boundary well founded?
> 
> I'm willing to look into hooking things together as time permits if it's a feasible task.
> 
> It's trivial to have iptables match on tc_classid and set a mark which is available to bpf at l2, but I'd like to better understand this.
> 
> Thanks,
> Matt C.
>