igc_configure_rx_ring() function will be called as part of XDP program
setup. If Rx hardware timestamp is enabled prio to XDP program setup,
this timestamp enablement will be overwritten when buffer size is
written into SRRCTL register.
Thus, this commit read the register value before write to SRRCTL
register. This commit is tested by using xdp_hw_metadata bpf selftest
tool. The tool enables Rx hardware timestamp and then attach XDP program
to igc driver. It will display hardware timestamp of UDP packet with
port number 9092. Below are detail of test steps and results.
Command on DUT:
sudo ./xdp_hw_metadata <interface name>
Command on Link Partner:
echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
Result before this patch:
skb hwtstamp is not found!
Result after this patch:
found skb hwtstamp = 1677800973.642836757
Optionally, read PHC to confirm the values obtained are almost the same:
Command:
sudo ./testptp -d /dev/ptp0 -g
Result:
clock time: 1677800973.913598978 or Fri Mar 3 07:49:33 2023
Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
Cc: <[email protected]> # 5.14+
Signed-off-by: Song Yoong Siang <[email protected]>
Reviewed-by: Jacob Keller <[email protected]>
---
v2 changelog:
- Fix indention
---
drivers/net/ethernet/intel/igc/igc_base.h | 7 +++++--
drivers/net/ethernet/intel/igc/igc_main.c | 5 ++++-
2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h
index 7a992befca24..b95007d51d13 100644
--- a/drivers/net/ethernet/intel/igc/igc_base.h
+++ b/drivers/net/ethernet/intel/igc/igc_base.h
@@ -87,8 +87,11 @@ union igc_adv_rx_desc {
#define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */
/* SRRCTL bit definitions */
-#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
-#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
+#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
+#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
+#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8)
+#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
+#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25)
#define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
#endif /* _IGC_BASE_H */
diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
index 25fc6c65209b..88fac08d8a14 100644
--- a/drivers/net/ethernet/intel/igc/igc_main.c
+++ b/drivers/net/ethernet/intel/igc/igc_main.c
@@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter,
else
buf_size = IGC_RXBUFFER_2048;
- srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
+ srrctl = rd32(IGC_SRRCTL(reg_idx));
+ srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDRSIZE_MASK |
+ IGC_SRRCTL_DESCTYPE_MASK);
+ srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT;
srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
--
2.34.1
On 14/04/2023 04.09, Song Yoong Siang wrote:
> igc_configure_rx_ring() function will be called as part of XDP program
> setup. If Rx hardware timestamp is enabled prio to XDP program setup,
> this timestamp enablement will be overwritten when buffer size is
> written into SRRCTL register.
>
> Thus, this commit read the register value before write to SRRCTL
> register. This commit is tested by using xdp_hw_metadata bpf selftest
> tool. The tool enables Rx hardware timestamp and then attach XDP program
> to igc driver. It will display hardware timestamp of UDP packet with
> port number 9092. Below are detail of test steps and results.
>
> Command on DUT:
> sudo ./xdp_hw_metadata <interface name>
>
> Command on Link Partner:
> echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
>
> Result before this patch:
> skb hwtstamp is not found!
>
> Result after this patch:
> found skb hwtstamp = 1677800973.642836757
>
> Optionally, read PHC to confirm the values obtained are almost the same:
> Command:
> sudo ./testptp -d /dev/ptp0 -g
> Result:
> clock time: 1677800973.913598978 or Fri Mar 3 07:49:33 2023
>
> Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
> Cc: <[email protected]> # 5.14+
> Signed-off-by: Song Yoong Siang <[email protected]>
> Reviewed-by: Jacob Keller <[email protected]>
> ---
Reviewed-by: Jesper Dangaard Brouer <[email protected]>
> v2 changelog:
> - Fix indention
> ---
> drivers/net/ethernet/intel/igc/igc_base.h | 7 +++++--
> drivers/net/ethernet/intel/igc/igc_main.c | 5 ++++-
> 2 files changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h
> index 7a992befca24..b95007d51d13 100644
> --- a/drivers/net/ethernet/intel/igc/igc_base.h
> +++ b/drivers/net/ethernet/intel/igc/igc_base.h
> @@ -87,8 +87,11 @@ union igc_adv_rx_desc {
> #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */
>
> /* SRRCTL bit definitions */
I have checked Foxville manual for SRRCTL (Split and Replication Receive
Control) register and below GENMASKs looks correct.
> -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
> -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
> +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
> +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
Shift due to 1 KB resolution of BSIZEPKT (manual field BSIZEPACKET)
> +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8)
> +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
This shift is suspicious, but as you inherited it I guess it works.
I did the math, and it happens to work, knowing (from manual) value is
in 64 bytes resolution.
> +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25)
> #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
Given you have started using GENMASK(), then I would have updated
IGC_SRRCTL_DESCTYPE_ADV_ONEBUF to be expressed like:
#define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF
FIELD_PREP(IGC_SRRCTL_DESCTYPE_MASK, 0x1)
Making it easier to see code is selecting:
001b = Advanced descriptor one buffer.
And not (as I first though):
010b = Advanced descriptor header splitting.
> #endif /* _IGC_BASE_H */
> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
> index 25fc6c65209b..88fac08d8a14 100644
> --- a/drivers/net/ethernet/intel/igc/igc_main.c
> +++ b/drivers/net/ethernet/intel/igc/igc_main.c
> @@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter,
> else
> buf_size = IGC_RXBUFFER_2048;
>
> - srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
> + srrctl = rd32(IGC_SRRCTL(reg_idx));
> + srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDRSIZE_MASK |
> + IGC_SRRCTL_DESCTYPE_MASK);
> + srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
> srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT;
> srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
>
On Friday, April 14, 2023 5:50 PM, Jesper Dangaard Brouer <[email protected]> wrote:
>On 14/04/2023 04.09, Song Yoong Siang wrote:
>> igc_configure_rx_ring() function will be called as part of XDP program
>> setup. If Rx hardware timestamp is enabled prio to XDP program setup,
>> this timestamp enablement will be overwritten when buffer size is
>> written into SRRCTL register.
>>
>> Thus, this commit read the register value before write to SRRCTL
>> register. This commit is tested by using xdp_hw_metadata bpf selftest
>> tool. The tool enables Rx hardware timestamp and then attach XDP
>> program to igc driver. It will display hardware timestamp of UDP
>> packet with port number 9092. Below are detail of test steps and results.
>>
>> Command on DUT:
>> sudo ./xdp_hw_metadata <interface name>
>>
>> Command on Link Partner:
>> echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
>>
>> Result before this patch:
>> skb hwtstamp is not found!
>>
>> Result after this patch:
>> found skb hwtstamp = 1677800973.642836757
>>
>> Optionally, read PHC to confirm the values obtained are almost the same:
>> Command:
>> sudo ./testptp -d /dev/ptp0 -g
>> Result:
>> clock time: 1677800973.913598978 or Fri Mar 3 07:49:33 2023
>>
>> Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
>> Cc: <[email protected]> # 5.14+
>> Signed-off-by: Song Yoong Siang <[email protected]>
>> Reviewed-by: Jacob Keller <[email protected]>
>> ---
>
>Reviewed-by: Jesper Dangaard Brouer <[email protected]>
>
>> v2 changelog:
>> - Fix indention
>> ---
>> drivers/net/ethernet/intel/igc/igc_base.h | 7 +++++--
>> drivers/net/ethernet/intel/igc/igc_main.c | 5 ++++-
>> 2 files changed, 9 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/intel/igc/igc_base.h
>> b/drivers/net/ethernet/intel/igc/igc_base.h
>> index 7a992befca24..b95007d51d13 100644
>> --- a/drivers/net/ethernet/intel/igc/igc_base.h
>> +++ b/drivers/net/ethernet/intel/igc/igc_base.h
>> @@ -87,8 +87,11 @@ union igc_adv_rx_desc {
>> #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive
>Software Flush */
>>
>> /* SRRCTL bit definitions */
>
>I have checked Foxville manual for SRRCTL (Split and Replication Receive
>Control) register and below GENMASKs looks correct.
>
>> -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
>> -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
>> +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
>> +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
>
>Shift due to 1 KB resolution of BSIZEPKT (manual field BSIZEPACKET)
Ya, 1K = BIT(10), so need to shift right 10 bits.
>
>> +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8)
>> +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
>
>This shift is suspicious, but as you inherited it I guess it works.
>I did the math, and it happens to work, knowing (from manual) value is in 64 bytes
>resolution.
It is in 64 = BIT(6) resolution, so need to shift right 6 bits.
But it start on 8th bit, so need to shift left 8 bits.
Thus, total = shift left 2 bits.
I dint put the explanation into the header file because it is too lengthy
and user can know from databook.
How do you feel on the necessary of explaining the shifting logic?
>
>> +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25)
>> #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
>
>Given you have started using GENMASK(), then I would have updated
>IGC_SRRCTL_DESCTYPE_ADV_ONEBUF to be expressed like:
>
> #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF FIELD_PREP(IGC_SRRCTL_DESCTYPE_MASK, 0x1)
>
>Making it easier to see code is selecting:
> 001b = Advanced descriptor one buffer.
>
>And not (as I first though):
> 010b = Advanced descriptor header splitting.
>
You are right. Using FIELD_PREP() make the code clearer.
Thanks for your suggestion. I will submit v3 for it.
>
>> #endif /* _IGC_BASE_H */
>> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c
>> b/drivers/net/ethernet/intel/igc/igc_main.c
>> index 25fc6c65209b..88fac08d8a14 100644
>> --- a/drivers/net/ethernet/intel/igc/igc_main.c
>> +++ b/drivers/net/ethernet/intel/igc/igc_main.c
>> @@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter
>*adapter,
>> else
>> buf_size = IGC_RXBUFFER_2048;
>>
>> - srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
>> + srrctl = rd32(IGC_SRRCTL(reg_idx));
>> + srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK |
>IGC_SRRCTL_BSIZEHDRSIZE_MASK |
>> + IGC_SRRCTL_DESCTYPE_MASK);
>> + srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
>> srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT;
>> srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
>>
From: Song, Yoong Siang
> Sent: 14 April 2023 12:16
...
> >I have checked Foxville manual for SRRCTL (Split and Replication Receive
> >Control) register and below GENMASKs looks correct.
> >
> >> -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
> >> -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
> >> +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
> >> +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
> >
> >Shift due to 1 KB resolution of BSIZEPKT (manual field BSIZEPACKET)
>
> Ya, 1K = BIT(10), so need to shift right 10 bits.
I bet the code would be easier to read if it did 'value / 1024u'.
The object code will be (much) the same.
> >> +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8)
> >> +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
> >
> >This shift is suspicious, but as you inherited it I guess it works.
> >I did the math, and it happens to work, knowing (from manual) value is in 64 bytes
> >resolution.
>
> It is in 64 = BIT(6) resolution, so need to shift right 6 bits.
> But it start on 8th bit, so need to shift left 8 bits.
> Thus, total = shift left 2 bits.
>
> I dint put the explanation into the header file because it is too lengthy
> and user can know from databook.
>
> How do you feel on the necessary of explaining the shifting logic?
Not everyone trying to grok the code will have the manual.
Even writing (8 - 6) will help.
Or (I think) if the value is in bits 13-8 in units of 64 then just:
((value >> 8) & 0x1f) * 64
gcc will do a single shift right and a mask 9at some point).
You might want some defines, but if they aren't used much
just comments that refer to the names in the manual/datasheet
can be enough.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
On 14/04/2023 14.32, David Laight wrote:
> From: Song, Yoong Siang
>> Sent: 14 April 2023 12:16
> ...
>>> I have checked Foxville manual for SRRCTL (Split and Replication Receive
>>> Control) register and below GENMASKs looks correct.
>>>
>>>> -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
>>>> -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
>>>> +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
>>>> +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
>>>
>>> Shift due to 1 KB resolution of BSIZEPKT (manual field BSIZEPACKET)
>>
>> Ya, 1K = BIT(10), so need to shift right 10 bits.
>
> I bet the code would be easier to read if it did 'value / 1024u'.
> The object code will be (much) the same.
I agree. Code becomes more readable for humans and machine code will be
the same.
>>>> +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8)
>>>> +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
>>>
>>> This shift is suspicious, but as you inherited it I guess it works.
>>> I did the math, and it happens to work, knowing (from manual) value is in 64 bytes
>>> resolution.
>>
>> It is in 64 = BIT(6) resolution, so need to shift right 6 bits.
>> But it start on 8th bit, so need to shift left 8 bits.
>> Thus, total = shift left 2 bits.
>>
>> I didnt put the explanation into the header file because it is too lengthy
>> and user can know from databook.
Well, users usually don't have access to the databook (Programming
Interface) PDF. Personally I have it, but I had to go though a lot of
red-tape to get it (under Red Hat NDA).
>>
>> How do you feel on the necessary of explaining the shifting logic?
>
> Not everyone trying to grok the code will have the manual.
> Even writing (8 - 6) will help.
> Or (I think) if the value is in bits 13-8 in units of 64 then just:
> ((value >> 8) & 0x1f) * 64
> gcc will do a single shift right and a mask 9at some point).
> You might want some defines, but if they aren't used much
> just comments that refer to the names in the manual/datasheet
> can be enough.
>
After Alexander Lobakin opened my eyes for GENMASK, FIELD_PREP and
FIELD_GET, I find that easier to read and work-with these kind of
register value manipulations, see[1] include/linux/bitfield.h. It will
also detect if the assigned value exceeds the mask (like David code
handled via mask). (thx Alex)
[1]
https://elixir.bootlin.com/linux/v6.3-rc6/source/include/linux/bitfield.h#L14
So, instead of:
srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
I would write
/* BSIZEHDR value in 64 bytes resolution */
srrctl |= FIELD_PREP(IGC_SRRCTL_BSIZEHDRSIZE_MASK, (IGC_RX_HDR_LEN /
64));
--Jesper
On Friday, April 14, 2023 10:19 PM, Jesper Dangaard Brouer <[email protected]> wrote:
>On 14/04/2023 14.32, David Laight wrote:
>> From: Song, Yoong Siang
>>> Sent: 14 April 2023 12:16
>> ...
>>>> I have checked Foxville manual for SRRCTL (Split and Replication
>>>> Receive
>>>> Control) register and below GENMASKs looks correct.
>>>>
>>>>> -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
>>>>> -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
>>>>> +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
>>>>> +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
>>>>
>>>> Shift due to 1 KB resolution of BSIZEPKT (manual field BSIZEPACKET)
>>>
>>> Ya, 1K = BIT(10), so need to shift right 10 bits.
>>
>> I bet the code would be easier to read if it did 'value / 1024u'.
>> The object code will be (much) the same.
>
>I agree. Code becomes more readable for humans and machine code will be the
>same.
>
>>>>> +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8)
>>>>> +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
>>>>
>>>> This shift is suspicious, but as you inherited it I guess it works.
>>>> I did the math, and it happens to work, knowing (from manual) value
>>>> is in 64 bytes resolution.
>>>
>>> It is in 64 = BIT(6) resolution, so need to shift right 6 bits.
>>> But it start on 8th bit, so need to shift left 8 bits.
>>> Thus, total = shift left 2 bits.
>>>
>>> I didnt put the explanation into the header file because it is too
>>> lengthy and user can know from databook.
>
>Well, users usually don't have access to the databook (Programming
>Interface) PDF. Personally I have it, but I had to go though a lot of red-tape to
>get it (under Red Hat NDA).
>
>
>>>
>>> How do you feel on the necessary of explaining the shifting logic?
>>
>> Not everyone trying to grok the code will have the manual.
>> Even writing (8 - 6) will help.
>> Or (I think) if the value is in bits 13-8 in units of 64 then just:
>> ((value >> 8) & 0x1f) * 64
>> gcc will do a single shift right and a mask 9at some point).
>> You might want some defines, but if they aren't used much just
>> comments that refer to the names in the manual/datasheet can be
>> enough.
>>
>
>After Alexander Lobakin opened my eyes for GENMASK, FIELD_PREP and
>FIELD_GET, I find that easier to read and work-with these kind of register value
>manipulations, see[1] include/linux/bitfield.h. It will also detect if the assigned
>value exceeds the mask (like David code handled via mask). (thx Alex)
>
> [1]
>https://elixir.bootlin.com/linux/v6.3-rc6/source/include/linux/bitfield.h#L14
>
>So, instead of:
> srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
>
>I would write
>
> /* BSIZEHDR value in 64 bytes resolution */
> srrctl |= FIELD_PREP(IGC_SRRCTL_BSIZEHDRSIZE_MASK, (IGC_RX_HDR_LEN / 64));
>
>--Jesper
Thanks David and Jesper for the comments.
I agree to make the code more human readable.
Will refactor the code and send out v3 for review.
Thanks & Regards
Siang