Changes from v1:
- Use unsigned access for SKF_AD_HATYPE
- Added three more patches for other problems found.
Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
identified some failures and unimplemented features.
With this patch set we get:
test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
Both big and little endian tested.
We still lack eBPF support, but this is better than nothing.
David Daney (5):
MIPS: uasm: Add support for LHU.
MIPS: BPF: Add JIT support for SKF_AD_HATYPE.
MIPS: BPF: Use unsigned access for unsigned SKB fields.
MIPS: BPF: Quit clobbering callee saved registers in JIT code.
MIPS: BPF: Fix multiple problems in JIT skb access helpers.
arch/mips/include/asm/uasm.h | 1 +
arch/mips/mm/uasm-mips.c | 1 +
arch/mips/mm/uasm.c | 3 ++-
arch/mips/net/bpf_jit.c | 41 +++++++++++++++++++++++++++++++----------
arch/mips/net/bpf_jit_asm.S | 23 ++++++++++++-----------
5 files changed, 47 insertions(+), 22 deletions(-)
--
2.9.3
If bpf_needs_clear_a() returns true, only actually clear it if it is
ever used. If it is not used, we don't save and restore it, so the
clearing has the nasty side effect of clobbering caller state.
Also, don't emit stack pointer adjustment instructions if the
adjustment amount is zero.
Signed-off-by: David Daney <[email protected]>
---
arch/mips/net/bpf_jit.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c
index a68cd36..44b9250 100644
--- a/arch/mips/net/bpf_jit.c
+++ b/arch/mips/net/bpf_jit.c
@@ -532,7 +532,8 @@ static void save_bpf_jit_regs(struct jit_ctx *ctx, unsigned offset)
u32 sflags, tmp_flags;
/* Adjust the stack pointer */
- emit_stack_offset(-align_sp(offset), ctx);
+ if (offset)
+ emit_stack_offset(-align_sp(offset), ctx);
tmp_flags = sflags = ctx->flags >> SEEN_SREG_SFT;
/* sflags is essentially a bitmap */
@@ -584,7 +585,8 @@ static void restore_bpf_jit_regs(struct jit_ctx *ctx,
emit_load_stack_reg(r_ra, r_sp, real_off, ctx);
/* Restore the sp and discard the scrach memory */
- emit_stack_offset(align_sp(offset), ctx);
+ if (offset)
+ emit_stack_offset(align_sp(offset), ctx);
}
static unsigned int get_stack_depth(struct jit_ctx *ctx)
@@ -631,8 +633,14 @@ static void build_prologue(struct jit_ctx *ctx)
if (ctx->flags & SEEN_X)
emit_jit_reg_move(r_X, r_zero, ctx);
- /* Do not leak kernel data to userspace */
- if (bpf_needs_clear_a(&ctx->skf->insns[0]))
+ /*
+ * Do not leak kernel data to userspace, we only need to clear
+ * r_A if it is ever used. In fact if it is never used, we
+ * will not save/restore it, so clearing it in this case would
+ * corrupt the state of the caller.
+ */
+ if (bpf_needs_clear_a(&ctx->skf->insns[0]) &&
+ (ctx->flags & SEEN_A))
emit_jit_reg_move(r_A, r_zero, ctx);
}
--
2.9.3
The SKB vlan_tci and queue_mapping fields are unsigned, don't sign
extend these in the BPF JIT. In the vlan_tci case, the value gets
masked so the change is not needed for correctness, but do it anyway
for agreement with the types defined in struct sk_buff.
Signed-off-by: David Daney <[email protected]>
---
arch/mips/net/bpf_jit.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c
index 880e329..a68cd36 100644
--- a/arch/mips/net/bpf_jit.c
+++ b/arch/mips/net/bpf_jit.c
@@ -1156,7 +1156,7 @@ static int build_body(struct jit_ctx *ctx)
BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff,
vlan_tci) != 2);
off = offsetof(struct sk_buff, vlan_tci);
- emit_half_load(r_s0, r_skb, off, ctx);
+ emit_half_load_unsigned(r_s0, r_skb, off, ctx);
if (code == (BPF_ANC | SKF_AD_VLAN_TAG)) {
emit_andi(r_A, r_s0, (u16)~VLAN_TAG_PRESENT, ctx);
} else {
@@ -1183,7 +1183,7 @@ static int build_body(struct jit_ctx *ctx)
BUILD_BUG_ON(offsetof(struct sk_buff,
queue_mapping) > 0xff);
off = offsetof(struct sk_buff, queue_mapping);
- emit_half_load(r_A, r_skb, off, ctx);
+ emit_half_load_unsigned(r_A, r_skb, off, ctx);
break;
default:
pr_debug("%s: Unhandled opcode: 0x%02x\n", __FILE__,
--
2.9.3
o Socket data is unsigned, so use unsigned accessors instructions.
o Fix path result pointer generation arithmetic.
o Fix half-word byte swapping code for unsigned semantics.
Signed-off-by: David Daney <[email protected]>
---
arch/mips/net/bpf_jit_asm.S | 23 ++++++++++++-----------
1 file changed, 12 insertions(+), 11 deletions(-)
diff --git a/arch/mips/net/bpf_jit_asm.S b/arch/mips/net/bpf_jit_asm.S
index 5d2e0c8..88a2075 100644
--- a/arch/mips/net/bpf_jit_asm.S
+++ b/arch/mips/net/bpf_jit_asm.S
@@ -90,18 +90,14 @@ FEXPORT(sk_load_half_positive)
is_offset_in_header(2, half)
/* Offset within header boundaries */
PTR_ADDU t1, $r_skb_data, offset
- .set reorder
- lh $r_A, 0(t1)
- .set noreorder
+ lhu $r_A, 0(t1)
#ifdef CONFIG_CPU_LITTLE_ENDIAN
# if defined(__mips_isa_rev) && (__mips_isa_rev >= 2)
- wsbh t0, $r_A
- seh $r_A, t0
+ wsbh $r_A, $r_A
# else
- sll t0, $r_A, 24
- andi t1, $r_A, 0xff00
- sra t0, t0, 16
- srl t1, t1, 8
+ sll t0, $r_A, 8
+ srl t1, $r_A, 8
+ andi t0, t0, 0xff00
or $r_A, t0, t1
# endif
#endif
@@ -115,7 +111,7 @@ FEXPORT(sk_load_byte_positive)
is_offset_in_header(1, byte)
/* Offset within header boundaries */
PTR_ADDU t1, $r_skb_data, offset
- lb $r_A, 0(t1)
+ lbu $r_A, 0(t1)
jr $r_ra
move $r_ret, zero
END(sk_load_byte)
@@ -139,6 +135,11 @@ FEXPORT(sk_load_byte_positive)
* (void *to) is returned in r_s0
*
*/
+#ifdef CONFIG_CPU_LITTLE_ENDIAN
+#define DS_OFFSET(SIZE) (4 * SZREG)
+#else
+#define DS_OFFSET(SIZE) ((4 * SZREG) + (4 - SIZE))
+#endif
#define bpf_slow_path_common(SIZE) \
/* Quick check. Are we within reasonable boundaries? */ \
LONG_ADDIU $r_s1, $r_skb_len, -SIZE; \
@@ -150,7 +151,7 @@ FEXPORT(sk_load_byte_positive)
PTR_LA t0, skb_copy_bits; \
PTR_S $r_ra, (5 * SZREG)($r_sp); \
/* Assign low slot to a2 */ \
- move a2, $r_sp; \
+ PTR_ADDIU a2, $r_sp, DS_OFFSET(SIZE); \
jalr t0; \
/* Reset our destination slot (DS but it's ok) */ \
INT_S zero, (4 * SZREG)($r_sp); \
--
2.9.3
The follow-on BPF JIT patches use the LHU instruction, so add it.
Signed-off-by: David Daney <[email protected]>
---
arch/mips/include/asm/uasm.h | 1 +
arch/mips/mm/uasm-mips.c | 1 +
arch/mips/mm/uasm.c | 3 ++-
3 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/mips/include/asm/uasm.h b/arch/mips/include/asm/uasm.h
index f7929f6..d7e84f1 100644
--- a/arch/mips/include/asm/uasm.h
+++ b/arch/mips/include/asm/uasm.h
@@ -135,6 +135,7 @@ Ip_u2s3u1(_lb);
Ip_u2s3u1(_ld);
Ip_u3u1u2(_ldx);
Ip_u2s3u1(_lh);
+Ip_u2s3u1(_lhu);
Ip_u2s3u1(_ll);
Ip_u2s3u1(_lld);
Ip_u1s2(_lui);
diff --git a/arch/mips/mm/uasm-mips.c b/arch/mips/mm/uasm-mips.c
index 763d3f1..2277499 100644
--- a/arch/mips/mm/uasm-mips.c
+++ b/arch/mips/mm/uasm-mips.c
@@ -103,6 +103,7 @@ static struct insn insn_table[] = {
{ insn_ld, M(ld_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_ldx, M(spec3_op, 0, 0, 0, ldx_op, lx_op), RS | RT | RD },
{ insn_lh, M(lh_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
+ { insn_lhu, M(lhu_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
#ifndef CONFIG_CPU_MIPSR6
{ insn_lld, M(lld_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
{ insn_ll, M(ll_op, 0, 0, 0, 0, 0), RS | RT | SIMM },
diff --git a/arch/mips/mm/uasm.c b/arch/mips/mm/uasm.c
index a829704..7f400c8 100644
--- a/arch/mips/mm/uasm.c
+++ b/arch/mips/mm/uasm.c
@@ -61,7 +61,7 @@ enum opcode {
insn_sllv, insn_slt, insn_sltiu, insn_sltu, insn_sra, insn_srl,
insn_srlv, insn_subu, insn_sw, insn_sync, insn_syscall, insn_tlbp,
insn_tlbr, insn_tlbwi, insn_tlbwr, insn_wait, insn_wsbh, insn_xor,
- insn_xori, insn_yield, insn_lddir, insn_ldpte,
+ insn_xori, insn_yield, insn_lddir, insn_ldpte, insn_lhu,
};
struct insn {
@@ -297,6 +297,7 @@ I_u1(_jr)
I_u2s3u1(_lb)
I_u2s3u1(_ld)
I_u2s3u1(_lh)
+I_u2s3u1(_lhu)
I_u2s3u1(_ll)
I_u2s3u1(_lld)
I_u1s2(_lui)
--
2.9.3
This let's us pass some additional "modprobe test-bpf" tests with JIT
enabled.
Reuse the code for SKF_AD_IFINDEX, but substitute the offset and size
of the "type" field.
Signed-off-by: David Daney <[email protected]>
---
arch/mips/net/bpf_jit.c | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)
diff --git a/arch/mips/net/bpf_jit.c b/arch/mips/net/bpf_jit.c
index 49a2e22..880e329 100644
--- a/arch/mips/net/bpf_jit.c
+++ b/arch/mips/net/bpf_jit.c
@@ -365,6 +365,12 @@ static inline void emit_half_load(unsigned int reg, unsigned int base,
emit_instr(ctx, lh, reg, offset, base);
}
+static inline void emit_half_load_unsigned(unsigned int reg, unsigned int base,
+ unsigned int offset, struct jit_ctx *ctx)
+{
+ emit_instr(ctx, lhu, reg, offset, base);
+}
+
static inline void emit_mul(unsigned int dst, unsigned int src1,
unsigned int src2, struct jit_ctx *ctx)
{
@@ -1112,6 +1118,8 @@ static int build_body(struct jit_ctx *ctx)
break;
case BPF_ANC | SKF_AD_IFINDEX:
/* A = skb->dev->ifindex */
+ case BPF_ANC | SKF_AD_HATYPE:
+ /* A = skb->dev->type */
ctx->flags |= SEEN_SKB | SEEN_A;
off = offsetof(struct sk_buff, dev);
/* Load *dev pointer */
@@ -1120,10 +1128,15 @@ static int build_body(struct jit_ctx *ctx)
emit_bcond(MIPS_COND_EQ, r_s0, r_zero,
b_imm(prog->len, ctx), ctx);
emit_reg_move(r_ret, r_zero, ctx);
- BUILD_BUG_ON(FIELD_SIZEOF(struct net_device,
- ifindex) != 4);
- off = offsetof(struct net_device, ifindex);
- emit_load(r_A, r_s0, off, ctx);
+ if (code == (BPF_ANC | SKF_AD_IFINDEX)) {
+ BUILD_BUG_ON(FIELD_SIZEOF(struct net_device, ifindex) != 4);
+ off = offsetof(struct net_device, ifindex);
+ emit_load(r_A, r_s0, off, ctx);
+ } else { /* (code == (BPF_ANC | SKF_AD_HATYPE) */
+ BUILD_BUG_ON(FIELD_SIZEOF(struct net_device, type) != 2);
+ off = offsetof(struct net_device, type);
+ emit_half_load_unsigned(r_A, r_s0, off, ctx);
+ }
break;
case BPF_ANC | SKF_AD_MARK:
ctx->flags |= SEEN_SKB | SEEN_A;
--
2.9.3
On 03/14/2017 10:21 PM, David Daney wrote:
> Changes from v1:
>
> - Use unsigned access for SKF_AD_HATYPE
>
> - Added three more patches for other problems found.
>
>
> Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
> identified some failures and unimplemented features.
Nice, thanks for working on this! If you see specific test
cases for the JIT missing, please also feel free to extend
the test_bpf suite, so this gets exposed further to other
JITs, too.
> With this patch set we get:
>
> test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
>
> Both big and little endian tested.
>
> We still lack eBPF support, but this is better than nothing.
Any future plans on this one?
Thanks,
Daniel
On 03/14/2017 03:29 PM, Daniel Borkmann wrote:
> On 03/14/2017 10:21 PM, David Daney wrote:
>> Changes from v1:
>>
>> - Use unsigned access for SKF_AD_HATYPE
>>
>> - Added three more patches for other problems found.
>>
>>
>> Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
>> identified some failures and unimplemented features.
>
> Nice, thanks for working on this! If you see specific test
> cases for the JIT missing, please also feel free to extend
> the test_bpf suite, so this gets exposed further to other
> JITs, too.
>
>> With this patch set we get:
>>
>> test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
>>
>> Both big and little endian tested.
>>
>> We still lack eBPF support, but this is better than nothing.
>
> Any future plans on this one?
Yes, my plan is to fully implement eBPF support for 64-bit MIPS, let's
see if I get enough time to do it.
David.
On Tue, Mar 14, 2017 at 02:21:39PM -0700, David Daney wrote:
> Changes from v1:
>
> - Use unsigned access for SKF_AD_HATYPE
>
> - Added three more patches for other problems found.
>
>
> Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
> identified some failures and unimplemented features.
>
> With this patch set we get:
>
> test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
>
> Both big and little endian tested.
>
> We still lack eBPF support, but this is better than nothing.
>
> David Daney (5):
> MIPS: uasm: Add support for LHU.
> MIPS: BPF: Add JIT support for SKF_AD_HATYPE.
> MIPS: BPF: Use unsigned access for unsigned SKB fields.
> MIPS: BPF: Quit clobbering callee saved registers in JIT code.
> MIPS: BPF: Fix multiple problems in JIT skb access helpers.
Thanks. Nice set of fixes. Especially patch 4.
Did you see crashes because of it?
Acked-by: Alexei Starovoitov <[email protected]>
On 03/14/2017 03:49 PM, Alexei Starovoitov wrote:
> On Tue, Mar 14, 2017 at 02:21:39PM -0700, David Daney wrote:
>> Changes from v1:
>>
>> - Use unsigned access for SKF_AD_HATYPE
>>
>> - Added three more patches for other problems found.
>>
>>
>> Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
>> identified some failures and unimplemented features.
>>
>> With this patch set we get:
>>
>> test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
>>
>> Both big and little endian tested.
>>
>> We still lack eBPF support, but this is better than nothing.
>>
>> David Daney (5):
>> MIPS: uasm: Add support for LHU.
>> MIPS: BPF: Add JIT support for SKF_AD_HATYPE.
>> MIPS: BPF: Use unsigned access for unsigned SKB fields.
>> MIPS: BPF: Quit clobbering callee saved registers in JIT code.
>> MIPS: BPF: Fix multiple problems in JIT skb access helpers.
>
> Thanks. Nice set of fixes. Especially patch 4.
> Did you see crashes because of it?
Only when running the test-bpf module.
The "JMP_JA: Jump, gap, jump, ..." test doesn't actually use any
registers, which I think is somewhat uncommon in BPF code. The system
would either crash or have weird behavior after running this test.
> Acked-by: Alexei Starovoitov <[email protected]>
>
From: David Daney <[email protected]>
Date: Tue, 14 Mar 2017 14:21:39 -0700
> Changes from v1:
>
> - Use unsigned access for SKF_AD_HATYPE
>
> - Added three more patches for other problems found.
>
>
> Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
> identified some failures and unimplemented features.
>
> With this patch set we get:
>
> test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
>
> Both big and little endian tested.
>
> We still lack eBPF support, but this is better than nothing.
What tree are you targetting with these changes? Do you expect
them to go via the MIPS or the net-next tree?
Please be explicit about this in the future.
Thank you.
On 03/14/2017 05:29 PM, David Miller wrote:
> From: David Daney <[email protected]>
> Date: Tue, 14 Mar 2017 14:21:39 -0700
>
>> Changes from v1:
>>
>> - Use unsigned access for SKF_AD_HATYPE
>>
>> - Added three more patches for other problems found.
>>
>>
>> Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
>> identified some failures and unimplemented features.
>>
>> With this patch set we get:
>>
>> test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
>>
>> Both big and little endian tested.
>>
>> We still lack eBPF support, but this is better than nothing.
>
> What tree are you targetting with these changes? Do you expect
> them to go via the MIPS or the net-next tree?
>
> Please be explicit about this in the future.
>
Sorry I didn't mention it.
My expectation is that Ralf would merge it via the MIPS tree, as it is
fully contained within arch/mips/*
David Daney
> Thank you.
>
From: David Daney <[email protected]>
Date: Tue, 14 Mar 2017 17:34:02 -0700
> On 03/14/2017 05:29 PM, David Miller wrote:
>> From: David Daney <[email protected]>
>> Date: Tue, 14 Mar 2017 14:21:39 -0700
>>
>>> Changes from v1:
>>>
>>> - Use unsigned access for SKF_AD_HATYPE
>>>
>>> - Added three more patches for other problems found.
>>>
>>>
>>> Testing the BPF JIT on Cavium OCTEON (mips64) with the test-bpf module
>>> identified some failures and unimplemented features.
>>>
>>> With this patch set we get:
>>>
>>> test_bpf: Summary: 305 PASSED, 0 FAILED, [85/297 JIT'ed]
>>>
>>> Both big and little endian tested.
>>>
>>> We still lack eBPF support, but this is better than nothing.
>>
>> What tree are you targetting with these changes? Do you expect
>> them to go via the MIPS or the net-next tree?
>>
>> Please be explicit about this in the future.
>>
>
> Sorry I didn't mention it.
>
> My expectation is that Ralf would merge it via the MIPS tree, as it is
> fully contained within arch/mips/*
Great, thanks for letting me know.
On Tue, Mar 14, 2017 at 05:34:02PM -0700, David Daney wrote:
> > What tree are you targetting with these changes? Do you expect
> > them to go via the MIPS or the net-next tree?
> >
> > Please be explicit about this in the future.
> >
>
> Sorry I didn't mention it.
>
> My expectation is that Ralf would merge it via the MIPS tree, as it is fully
> contained within arch/mips/*
Thanks, applied.
Ralf