Received-SPF: pass (google.com: domain of linux-crypto+bounces-3022-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: Andre Glover <andre.glover@linux.intel.com>
To: tom.zanussi@linux.intel.com,
	herbert@gondor.apana.org.au,
	davem@davemloft.net
Cc: dave.jiang@intel.com,
	fenghua.yu@intel.com,
	wajdi.k.feghali@intel.com,
	james.guilford@intel.com,
	vinodh.gopal@intel.com,
	tony.luck@intel.com,
	linux-crypto@vger.kernel.org,
	dmaengine@vger.kernel.org,
	andre.glover@linux.intel.com
Subject: [PATCH 0/4] crypto: Add new compression modes for zlib and IAA 
Date: Thu, 28 Mar 2024 10:44:41 -0700
Message-Id: <cover.1710969449.git.andre.glover@linux.intel.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This patch series adds 'canned' compression for Intel IAA and zlib. It
also adds 'dynamic' compression for Intel IAA which is compatible with zlib
deflate algorithms. 

The original IAA crypto submissions [1] included a 'canned' compression
mode, but support for 'canned' compression was removed during the review
because it didn't have an equivalent software implementation available [2].

Deflate compression can be done in a variety of modes. The dynamic mode
uses Huffman tables that are generated/optimized for that particular
input. This gives the best compression ratio but has the longest latency.
The fixed mode uses Huffman tables that are defined by the Deflate
standard. It generally gives the best latency but with a lower compression
ratio. The 'canned' compression mode implements a compression scheme that
uses a statically defined set of Huffman tables, but where the Deflate
Block Header is implied rather than stored with the compressed data. 

The 'canned' mode results in lower compression/decompression latencies
compared to using the dynamic mode, but it results in a better compression
ratio than using the fixed mode.

Below is a table showing the latency improvements with zlib, between
zlib dynamic and zlib canned modes, and the compression ratio for 
each mode while using a set of 4300 4KB pages sampled from SPEC 
CPU17 workloads:
_________________________________________________________
| Zlib Level |  Canned Latency Gain  |    Comp Ratio    |
|------------|-----------------------|------------------|
|            | compress | decompress | dynamic | canned |
|____________|__________|____________|_________|________|
|     1      |    49%   |    29%     |  3.16   |  2.92  |
|------------|----------|------------|---------|--------|
|     6	     |    27%   |    28%     |  3.35   |  3.09  |
|------------|----------|------------|---------|--------|
|     9      |    12%   |    29%     |  3.36   |  3.11  |
|____________|__________|____________|_________|________|

Using the same data set as for the above table, IAA fixed-mode compression
results in a compression ratio of 2.69. Using IAA canned-mode compression
results in a ratio of 2.88, which is a 7% improvement. This data shows that
the canned mode results in better latencies than the dynamic mode, but a
better ratio than the fixed mode. Thus, the canned mode would be preferred
when compression and decompression latencies are valued more than the
compression ratio.

'Dynamic' mode IAA compression is a HW accelerated dynamic Deflate that is
fully compatible with software decompress implementations, including zlib.
'Dynamic' IAA compression allows users to perform hardware-accelerated
compression that achieves a higher compression ratio than both 'canned'
and 'fixed' compression modes. Thus, Dynamic IAA compression should be
used when the goal is to produce the best compression ratio while
minimizing latencies by using IAA hardware acceleration.  IAA
decompression is fully compatible with software compress implementations,
when the uncompressed size is no more than 4KB.

Below is a table showing the compression ratio seen when compressing a
data set of 4300 4KB pages sampled from SPEC CPU17 workloads via various
IAA compression modes:

|---------------------------------------|
|		 Intel IAA              |
|---------------------------------------|
| compression mode | compression ratio  |
|---------------------------------------|
| fixed	           |	2.69            |
|---------------------------------------|
| canned           |	2.88		|
|---------------------------------------|
| dynamic          |	3.14		|
|---------------------------------------|

Patch 1/4 adds a software implementation of “canned’ mode to the
existing zlib software library and exposes it as “deflate-canned”.
This was done instead of creating a new canned-mode library to avoid a lot
of code duplication. Testing shows that this change has no performance
impact to the existing zlib algorithms.

Patch 2/4 adds IAA 'canned' support which is based on the original
implementation [1] and will be exposed as 'deflate-iaa-canned'

Patch 3/4 adds 'dynamic' mode IAA compression support and will be exposed
as 'deflate-iaa-dynamic'.

Patch 4/4 adds software compression stats to the optional debugfs
statistics support for IAA.

[1] https://lore.kernel.org/lkml/20230605201536.738396-1-tom.zanussi@linux.intel.com/
[2] https://lore.kernel.org/lkml/ZIw%2Fjtxdg6O1O0j3@gondor.apana.org.au/

Andre Glover (4):
  crypto: Add 'canned' compression mode for zlib
  crypto: iaa - Add deflate-canned compression algorithm
  crypto: iaa - Add deflate-iaa-dynamic compression algorithm
  crypto: iaa - Add Software Compression stats to IAA Compression
    Accelerator stats

 .../driver-api/crypto/iaa/iaa-crypto.rst      |  36 +-
 crypto/deflate.c                              |  72 +++-
 crypto/testmgr.c                              |  30 ++
 crypto/testmgr.h                              | 220 +++++++++++
 drivers/crypto/intel/iaa/Kconfig              |   1 +
 drivers/crypto/intel/iaa/Makefile             |   2 +-
 drivers/crypto/intel/iaa/iaa_crypto.h         |  44 ++-
 .../crypto/intel/iaa/iaa_crypto_comp_canned.c | 116 ++++++
 .../intel/iaa/iaa_crypto_comp_dynamic.c       |  22 ++
 .../crypto/intel/iaa/iaa_crypto_comp_fixed.c  |   1 +
 drivers/crypto/intel/iaa/iaa_crypto_main.c    | 361 ++++++++++++++++--
 drivers/crypto/intel/iaa/iaa_crypto_stats.c   |   8 +
 drivers/crypto/intel/iaa/iaa_crypto_stats.h   |   2 +
 include/linux/zlib.h                          |  10 +
 lib/Kconfig                                   |   9 +
 lib/zlib_deflate/defcanned.h                  | 118 ++++++
 lib/zlib_deflate/deflate.c                    |   8 +-
 lib/zlib_deflate/deftree.c                    |  15 +-
 lib/zlib_inflate/infcanned.h                  | 191 +++++++++
 lib/zlib_inflate/inflate.c                    |  15 +-
 lib/zlib_inflate/inflate.h                    |   5 +-
 lib/zlib_inflate/infutil.h                    |  16 +
 22 files changed, 1255 insertions(+), 47 deletions(-)
 create mode 100644 drivers/crypto/intel/iaa/iaa_crypto_comp_canned.c
 create mode 100644 drivers/crypto/intel/iaa/iaa_crypto_comp_dynamic.c
 create mode 100644 lib/zlib_deflate/defcanned.h
 create mode 100644 lib/zlib_inflate/infcanned.h

-- 
2.27.0