The wave5 codec is a stateful encoder/decoder.
It is found on the JH7100 SoC.
The driver currently supports V4L2_PIX_FMT_HEVC, V4L2_PIX_FMT_H264.
This driver has so far been tested on pre-silicon FPGA and on the beta BeagleV
board which uses the StarFive JH7100 beta SoC.
Testing on FPGA shows it working fine, though the FPGA uses polled interrupts
and copied buffers between the host and it's on board RAM.
Testing on BeagleV shows buffer corruption that is currently attributed to a
known silicon issue in the SoC that makes the cache coherent interconnect not
so coherent.
This can likely be solved when the riscv non-coherent dma support lands and
provide optional v4l2 non-contiguous allocator, though it remains to be seen
whether support non-coherent use cases will be useful in real world hw.
Until we can test and resolve any issues on final silicon (due 2H 2021)
this driver should remain in staging.
We currently tested only the decoder.
Testing the encoder is currently impossible because of the BeagleV buffer problems,
That will need to be tested once we have fixed beaglev buffer problems.
v4l2-compliance all pass for v2 as well.
changes since v1:
Main fixes includes:
* change the yaml and dirver code to support up to 4 clks (instead of one)
* fix Kconfig format
* remove unneeded cast,
* change var types
* change var names, func names
* checkpatch fixes
Dafna Hirschfeld (4):
staging: media: wave5: Add vpuapi layer
staging: media: wave5: Add the vdi layer
staging: media: wave5: Add the v4l2 layer
staging: media: wave5: Add TODO file
Robert Beckett (2):
dt-bindings: media: staging: wave5: add yaml devicetree bindings
media: wave5: Add wave5 driver to maintainers file
.../bindings/staging/media/cnm,wave.yaml | 73 +
MAINTAINERS | 9 +
drivers/staging/media/Kconfig | 2 +
drivers/staging/media/Makefile | 1 +
drivers/staging/media/wave5/Kconfig | 12 +
drivers/staging/media/wave5/Makefile | 10 +
drivers/staging/media/wave5/TODO | 59 +
drivers/staging/media/wave5/vdi.c | 260 ++
drivers/staging/media/wave5/vdi.h | 76 +
drivers/staging/media/wave5/vpu.c | 367 ++
drivers/staging/media/wave5/vpu.h | 64 +
drivers/staging/media/wave5/vpu_dec.c | 1397 +++++++
drivers/staging/media/wave5/vpu_enc.c | 1585 ++++++++
drivers/staging/media/wave5/vpuapi.c | 1066 +++++
drivers/staging/media/wave5/vpuapi.h | 1140 ++++++
drivers/staging/media/wave5/vpuconfig.h | 85 +
drivers/staging/media/wave5/vpuerror.h | 455 +++
drivers/staging/media/wave5/wave5-hw.c | 3476 +++++++++++++++++
drivers/staging/media/wave5/wave5.h | 81 +
drivers/staging/media/wave5/wave5_regdefine.h | 638 +++
20 files changed, 10856 insertions(+)
create mode 100644 Documentation/devicetree/bindings/staging/media/cnm,wave.yaml
create mode 100644 drivers/staging/media/wave5/Kconfig
create mode 100644 drivers/staging/media/wave5/Makefile
create mode 100644 drivers/staging/media/wave5/TODO
create mode 100644 drivers/staging/media/wave5/vdi.c
create mode 100644 drivers/staging/media/wave5/vdi.h
create mode 100644 drivers/staging/media/wave5/vpu.c
create mode 100644 drivers/staging/media/wave5/vpu.h
create mode 100644 drivers/staging/media/wave5/vpu_dec.c
create mode 100644 drivers/staging/media/wave5/vpu_enc.c
create mode 100644 drivers/staging/media/wave5/vpuapi.c
create mode 100644 drivers/staging/media/wave5/vpuapi.h
create mode 100644 drivers/staging/media/wave5/vpuconfig.h
create mode 100644 drivers/staging/media/wave5/vpuerror.h
create mode 100644 drivers/staging/media/wave5/wave5-hw.c
create mode 100644 drivers/staging/media/wave5/wave5.h
create mode 100644 drivers/staging/media/wave5/wave5_regdefine.h
--
2.17.1
From: Robert Beckett <[email protected]>
Add bindings for the wave5 chips&media codec driver
Signed-off-by: Robert Beckett <[email protected]>
Signed-off-by: Dafna Hirschfeld <[email protected]>
---
.../bindings/staging/media/cnm,wave.yaml | 73 +++++++++++++++++++
1 file changed, 73 insertions(+)
create mode 100644 Documentation/devicetree/bindings/staging/media/cnm,wave.yaml
diff --git a/Documentation/devicetree/bindings/staging/media/cnm,wave.yaml b/Documentation/devicetree/bindings/staging/media/cnm,wave.yaml
new file mode 100644
index 000000000000..e18978cbba16
--- /dev/null
+++ b/Documentation/devicetree/bindings/staging/media/cnm,wave.yaml
@@ -0,0 +1,73 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/staging/media/cnm,wave.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Chips&Media Wave 5 Series multi-standard codec IP
+
+maintainers:
+ - Nas Chung <[email protected]>
+ - Robert Beckett <[email protected]>
+ - Dafna Hirschfeld <[email protected]>
+
+description: |-
+ The Chips&Media WAVE codec IP is a multi format video encoder/decoder
+
+properties:
+ compatible:
+ anyOf:
+ - items:
+ - enum:
+ - cnm,cm511-vpu
+ - cnm,cm517-vpu
+ - cnm,cm521-vpu
+ - cnm,cm521c-vpu
+ - cnm,cm521c-dual-vpu
+ - cnm,cm521e1-vpu
+ - cnm,cm537-vpu
+ reg:
+ maxItems: 1
+
+ clocks:
+ minItems: 1
+ maxItems: 4
+
+ clock-names:
+ minItems: 1
+ maxItems: 4
+
+ interrupts:
+ maxItems: 1
+
+ power-domains:
+ maxItems: 1
+
+ resets:
+ maxItems: 1
+
+ sram:
+ $ref: /schemas/types.yaml#/definitions/phandle
+ description: phandle pointing to the SRAM device node
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+
+additionalProperties: false
+
+examples:
+ - |
+ vpu: video-codec@12345678 {
+ compatible = "cnm,cm521-vpu";
+ reg = <0x12345678 0x1000>;
+ interrupts = <42>;
+ clocks = <&clks 42>;
+ clock-names = "vcodec";
+ sram = <&sram>;
+ };
+
--
2.17.1
From: Robert Beckett <[email protected]>
Add the Chips&Media wave5 encoder/decoder driver
to the maintainers file
Signed-off-by: Robert Beckett <[email protected]>
Signed-off-by: Dafna Hirschfeld <[email protected]>
---
MAINTAINERS | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 0e6594a4ad8f..5704b8c17f72 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -19962,6 +19962,15 @@ F: drivers/watchdog/
F: include/linux/watchdog.h
F: include/uapi/linux/watchdog.h
+WAVE5 VPU CODEC DRIVER
+M: Nas Chung <[email protected]>
+M: Robert Beckett <[email protected]>
+M: Dafna Hirschfeld <[email protected]>
+L: [email protected]
+S: Maintained
+F: Documentation/devicetree/bindings/staging/media/cnm,wave.yaml
+F: drivers/staging/media/wave5/
+
WHISKEYCOVE PMIC GPIO DRIVER
M: Kuppuswamy Sathyanarayanan <[email protected]>
L: [email protected]
--
2.17.1
Add a TODO file listing all that is need for destaging.
Signed-off-by: Robert Beckett <[email protected]>
Signed-off-by: Dafna Hirschfeld <[email protected]>
---
drivers/staging/media/wave5/TODO | 59 ++++++++++++++++++++++++++++++++
1 file changed, 59 insertions(+)
create mode 100644 drivers/staging/media/wave5/TODO
diff --git a/drivers/staging/media/wave5/TODO b/drivers/staging/media/wave5/TODO
new file mode 100644
index 000000000000..98dc3b923ad6
--- /dev/null
+++ b/drivers/staging/media/wave5/TODO
@@ -0,0 +1,59 @@
+* Test on real silicon once available
+
+ This driver has so far been tested on pre-silicon FPGA and on the beta BeagleV
+ board which uses the StarFive JH7100 beta SoC.
+
+ Testing on FPGA shows it working fine, though the FPGA uses polled interrupts
+ and copied buffers between the host and it's on board RAM.
+
+ Testing on BeagleV shows buffer corruption that is currently attributed to a
+ known silicon issue in the SoC that makes the cache coherent interconnect not
+ so coherent.
+ This can likely be solved when the riscv non-coherent dma support lands and
+ provide optional v4l2 non-contiguous allocator, though it remains to be seen
+ whether support non-coherent use cases will be useful in real world hw.
+
+ Until we can test and resolve any issues on final silicon (due 2H 2021)
+ this driver should remain in staging.
+
+* Handle interrupts better
+
+ Currently the interrupt handling uses an unusual design employing a kfifo to
+ transfer irq status to irq thread. This was done as a work around for dropped
+ interrupts seen with IRQF_ONESHOT based handling.
+
+ This needs further investigation and fixing properly, with the aid of
+ C&M and StarFive engineers.
+
+* Fix early end of stream handling
+
+ When using a gstreamer pipeline like this:
+ gst-launch-1.0 filesrc location=<file> ! parsebin ! h264parse ! v4l2h264dec ! jpegenc ! identity eos-after=3 ! multifilesink location=%05d.jpeg
+ i.e. forced eos after 3 buffers
+ the pipeline stalls on exit waiting for end of job.
+ We need to add some form of early exit detection, instead of the current
+ behaviour of only considering a job finish on PIC_END.
+
+* Support more formats
+
+ The current frontend v4l2 layer only supports V4L2_PIX_FMT_HEVC and
+ V4L2_PIX_FMT_H264.
+ The backend logic supports other formats, so we should wire up the support
+ for further bitsream formats.
+
+* appropreate error handling:
+allocation failure, mutex acquire failure etc.
+
+* remove all unused struct fields
+
+* change struct fields to 'bool' type or bitfield when appropreate
+Likely good candidates are fields named '*_enable' , '*_on'
+
+* handle vdi_allocate_dma_memory failure , each function has to clean after itself
+
+* make sure that 'u32', 's32' etc. are only used when reading/writing hw
+and change s32 to u32 when s32 is not required
+
+* power management handling - add (runtime_)suspen/resume cb where the clock is enabled
+
+* fix checkpatch issues (mostly fixes , only left to fix MACRO_ARG_REUSE, LONG_LINE)
--
2.17.1
Hi Dafna,
Just a high-level comment below:
On 10/11/2021 13:09, Dafna Hirschfeld wrote:
> The wave5 codec is a stateful encoder/decoder.
> It is found on the JH7100 SoC.
>
> The driver currently supports V4L2_PIX_FMT_HEVC, V4L2_PIX_FMT_H264.
>
> This driver has so far been tested on pre-silicon FPGA and on the beta BeagleV
> board which uses the StarFive JH7100 beta SoC.
>
> Testing on FPGA shows it working fine, though the FPGA uses polled interrupts
> and copied buffers between the host and it's on board RAM.
>
> Testing on BeagleV shows buffer corruption that is currently attributed to a
> known silicon issue in the SoC that makes the cache coherent interconnect not
> so coherent.
> This can likely be solved when the riscv non-coherent dma support lands and
> provide optional v4l2 non-contiguous allocator, though it remains to be seen
> whether support non-coherent use cases will be useful in real world hw.
>
> Until we can test and resolve any issues on final silicon (due 2H 2021)
> this driver should remain in staging.
>
> We currently tested only the decoder.
> Testing the encoder is currently impossible because of the BeagleV buffer problems,
> That will need to be tested once we have fixed beaglev buffer problems.
>
> v4l2-compliance all pass for v2 as well.
>
> changes since v1:
>
> Main fixes includes:
> * change the yaml and dirver code to support up to 4 clks (instead of one)
> * fix Kconfig format
> * remove unneeded cast,
> * change var types
> * change var names, func names
> * checkpatch fixes
>
>
> Dafna Hirschfeld (4):
> staging: media: wave5: Add vpuapi layer
> staging: media: wave5: Add the vdi layer
> staging: media: wave5: Add the v4l2 layer
> staging: media: wave5: Add TODO file
>
> Robert Beckett (2):
> dt-bindings: media: staging: wave5: add yaml devicetree bindings
> media: wave5: Add wave5 driver to maintainers file
>
> .../bindings/staging/media/cnm,wave.yaml | 73 +
> MAINTAINERS | 9 +
> drivers/staging/media/Kconfig | 2 +
> drivers/staging/media/Makefile | 1 +
> drivers/staging/media/wave5/Kconfig | 12 +
> drivers/staging/media/wave5/Makefile | 10 +
> drivers/staging/media/wave5/TODO | 59 +
> drivers/staging/media/wave5/vdi.c | 260 ++
> drivers/staging/media/wave5/vdi.h | 76 +
> drivers/staging/media/wave5/vpu.c | 367 ++
> drivers/staging/media/wave5/vpu.h | 64 +
> drivers/staging/media/wave5/vpu_dec.c | 1397 +++++++
> drivers/staging/media/wave5/vpu_enc.c | 1585 ++++++++
> drivers/staging/media/wave5/vpuapi.c | 1066 +++++
> drivers/staging/media/wave5/vpuapi.h | 1140 ++++++
> drivers/staging/media/wave5/vpuconfig.h | 85 +
> drivers/staging/media/wave5/vpuerror.h | 455 +++
> drivers/staging/media/wave5/wave5-hw.c | 3476 +++++++++++++++++
> drivers/staging/media/wave5/wave5.h | 81 +
> drivers/staging/media/wave5/wave5_regdefine.h | 638 +++
Please prefix the source/header filenames with 'wave5_' (or 'wave5-',
as long as it is consistent).
See e.g. drivers/staging/media/hantro/ or really any media driver.
If you just see the filename then you can at least tell for which driver
it is. Just 'vpu.c' is not descriptive enough.
For the next version I would also like to see the v4l2-compliance output.
Regards,
Hans
> 20 files changed, 10856 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/staging/media/cnm,wave.yaml
> create mode 100644 drivers/staging/media/wave5/Kconfig
> create mode 100644 drivers/staging/media/wave5/Makefile
> create mode 100644 drivers/staging/media/wave5/TODO
> create mode 100644 drivers/staging/media/wave5/vdi.c
> create mode 100644 drivers/staging/media/wave5/vdi.h
> create mode 100644 drivers/staging/media/wave5/vpu.c
> create mode 100644 drivers/staging/media/wave5/vpu.h
> create mode 100644 drivers/staging/media/wave5/vpu_dec.c
> create mode 100644 drivers/staging/media/wave5/vpu_enc.c
> create mode 100644 drivers/staging/media/wave5/vpuapi.c
> create mode 100644 drivers/staging/media/wave5/vpuapi.h
> create mode 100644 drivers/staging/media/wave5/vpuconfig.h
> create mode 100644 drivers/staging/media/wave5/vpuerror.h
> create mode 100644 drivers/staging/media/wave5/wave5-hw.c
> create mode 100644 drivers/staging/media/wave5/wave5.h
> create mode 100644 drivers/staging/media/wave5/wave5_regdefine.h
>