Audio signal processing also has the requirement for memory to
memory similar as Video.
This asrc memory to memory (memory ->asrc->memory) case is a non
real time use case.
User fills the input buffer to the asrc module, after conversion, then asrc
sends back the output buffer to user. So it is not a traditional ALSA playback
and capture case.
It is a specific use case, there is no reference in current kernel.
v4l2 memory to memory is the closed implementation, v4l2 current
support video, image, radio, tuner, touch devices, so it is not
complicated to add support for this specific audio case.
Because we had implemented the "memory -> asrc ->i2s device-> codec"
use case in ALSA. Now the "memory->asrc->memory" needs
to reuse the code in asrc driver, so the first 3 patches is for refining
the code to make it can be shared by the "memory->asrc->memory"
driver.
The main change is in the v4l2 side, A /dev/vl4-audioX will be created,
user applications only use the ioctl of v4l2 framework.
Other change is to add memory to memory support for two kinds of i.MX ASRC
module.
changes in v4:
- update document style
- separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit
changes in v3:
- Modify documents for adding audio m2m support
- Add audio virtual m2m driver
- Defined V4L2_AUDIO_FMT_LPCM format type for audio.
- Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case.
- with modification in v4l-utils, pass v4l2-compliance test.
changes in v2:
- decouple the implementation in v4l2 and ALSA
- implement the memory to memory driver as a platfrom driver
and move it to driver/media
- move fsl_asrc_common.h to include/sound folder
Shengjiu Wang (11):
ASoC: fsl_asrc: define functions for memory to memory usage
ASoC: fsl_easrc: define functions for memory to memory usage
ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
ASoC: fsl_asrc: register m2m platform device
ASoC: fsl_easrc: register m2m platform device
media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
media: uapi: Add V4L2_AUDIO_FMT_LPCM fourcc format
media: v4l2: Add audio capture and output support
media: uapi: Add V4L2_CID_USER_IMX_ASRC_RATIO_MOD control
media: audm2m: add virtual driver for audio memory to memory
media: imx-asrc: Add memory to memory driver
.../userspace-api/media/v4l/audio-formats.rst | 15 +
.../userspace-api/media/v4l/buffer.rst | 6 +
.../userspace-api/media/v4l/control.rst | 5 +
.../userspace-api/media/v4l/dev-audio.rst | 63 +
.../userspace-api/media/v4l/devices.rst | 1 +
.../media/v4l/pixfmt-aud-lpcm.rst | 61 +
.../userspace-api/media/v4l/pixfmt.rst | 1 +
.../media/v4l/vidioc-enum-fmt.rst | 2 +
.../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 +
.../media/v4l/vidioc-querycap.rst | 3 +
.../media/videodev2.h.rst.exceptions | 3 +
.../media/common/videobuf2/videobuf2-v4l2.c | 4 +
drivers/media/platform/nxp/Kconfig | 12 +
drivers/media/platform/nxp/Makefile | 1 +
drivers/media/platform/nxp/imx-asrc.c | 1058 +++++++++++++++++
drivers/media/test-drivers/Kconfig | 9 +
drivers/media/test-drivers/Makefile | 1 +
drivers/media/test-drivers/audm2m.c | 767 ++++++++++++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 1 +
drivers/media/v4l2-core/v4l2-dev.c | 17 +
drivers/media/v4l2-core/v4l2-ioctl.c | 53 +
include/media/v4l2-dev.h | 2 +
include/media/v4l2-ioctl.h | 34 +
.../fsl => include/sound}/fsl_asrc_common.h | 54 +
include/uapi/linux/v4l2-controls.h | 1 +
include/uapi/linux/videodev2.h | 25 +
sound/soc/fsl/fsl_asrc.c | 162 +++
sound/soc/fsl/fsl_asrc.h | 4 +-
sound/soc/fsl/fsl_asrc_dma.c | 2 +-
sound/soc/fsl/fsl_easrc.c | 239 ++++
sound/soc/fsl/fsl_easrc.h | 8 +-
31 files changed, 2615 insertions(+), 3 deletions(-)
create mode 100644 Documentation/userspace-api/media/v4l/audio-formats.rst
create mode 100644 Documentation/userspace-api/media/v4l/dev-audio.rst
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst
create mode 100644 drivers/media/platform/nxp/imx-asrc.c
create mode 100644 drivers/media/test-drivers/audm2m.c
rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (60%)
--
2.34.1
Linear Pulse-Code Modulation is used to represent
audio samples in buffer, the samples for each channel
are interleaved.
Signed-off-by: Shengjiu Wang <[email protected]>
---
.../userspace-api/media/v4l/audio-formats.rst | 15 +++++
.../media/v4l/pixfmt-aud-lpcm.rst | 61 +++++++++++++++++++
.../userspace-api/media/v4l/pixfmt.rst | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 3 +
5 files changed, 81 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/audio-formats.rst
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst
diff --git a/Documentation/userspace-api/media/v4l/audio-formats.rst b/Documentation/userspace-api/media/v4l/audio-formats.rst
new file mode 100644
index 000000000000..bc52712d20d3
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/audio-formats.rst
@@ -0,0 +1,15 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audio-formats:
+
+*************
+Audio Formats
+*************
+
+These formats are used for :ref:`audio` interface only.
+
+
+.. toctree::
+ :maxdepth: 1
+
+ pixfmt-aud-lpcm
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst b/Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst
new file mode 100644
index 000000000000..2231bd95212f
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-aud-lpcm.rst
@@ -0,0 +1,61 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _v4l2-audio-fmt-lpcm:
+
+****************************
+V4L2_AUDIO_FMT_LPCM ('LPCM')
+****************************
+
+Linear Pulse-Code Modulation (LPCM)
+
+
+Description
+===========
+
+LPCM audio is coded using a combination of values such as:
+
+Sample rate: which is the number of times per second that samples are taken,
+The typical rates are 8kHz, 11025Hz, 16kHz, 22050Hz, 32kHz, 44100Hz, 48kHz,
+88200Hz, 96kHz, 176400Hz, 192kHz...
+
+Sample format: which determines the number of possible digital values that
+can be used to represent each sample. The format can be SND_PCM_FORMAT_S8,
+SND_PCM_FORMAT_S16_LE, SND_PCM_FORMAT_S24_LE, SND_PCM_FORMAT_S32_LE...
+
+Channels: It is the "location" or "passageway" of a specific signal or
+data in a piece of audio. The channel number can be 1,2,3...
+
+Please refer to https://www.alsa-project.org/alsa-doc/alsa-lib/pcm.html
+for more detail
+
+Each sample contains several channels data, the channel data format is
+defined by sample format.
+
+.. flat-table::
+ :header-rows: 0
+ :stub-columns: 0
+
+ * - Sample 0:
+ - Channel 0
+ - Channel 1
+ - Channel 2
+ - Channel 3
+ - ...
+ * - Sample 1:
+ - Channel 0
+ - Channel 1
+ - Channel 2
+ - Channel 3
+ - ...
+ * - Sample 2:
+ - Channel 0
+ - Channel 1
+ - Channel 2
+ - Channel 3
+ - ...
+ * - Sample 3:
+ - Channel 0
+ - Channel 1
+ - Channel 2
+ - Channel 3
+ - ...
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..e205db5fa8af 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
colorspaces
colorspaces-defs
colorspaces-details
+ audio-formats
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index f4d9d6279094..55fd4da10ba6 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1452,6 +1452,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break;
+ case V4L2_AUDIO_FMT_LPCM: descr = "Audio LPCM"; break;
default:
/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3decf7d73870..ce71e9343705 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -839,6 +839,9 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
+/* Audio-data formats */
+#define V4L2_AUDIO_FMT_LPCM v4l2_fourcc('L', 'P', 'C', 'M') /* audio lpcm */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
--
2.34.1