Writing `subprocess.Popen[str]` requires python 3.9+.
kunit.py has an assertion that the python version is 3.7+, so we should
try to stay backwards compatible.
This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
pre-existing `mypy --strict` errors and update run_checks.py"), since
mypy complains like so
> kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
Note: `mypy --strict --python-version 3.7` does not work.
We could annotate each file with comments like
`# mypy: disable-error-code="type-arg"
but then we might still get nudged to break back-compat in other files.
This patch adds a `mypy.ini` file since it seems like the only way to
disable specific error codes for all our files.
Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
but I think being explicit is better, particularly since most kernel
devs won't be familiar with how mypy works.
Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
Reported-by: SeongJae Park <[email protected]>
Link: https://lore.kernel.org/linux-kselftest/[email protected]
Signed-off-by: Daniel Latypov <[email protected]>
---
tools/testing/kunit/kunit_kernel.py | 6 +++---
tools/testing/kunit/mypy.ini | 6 ++++++
tools/testing/kunit/run_checks.py | 2 +-
3 files changed, 10 insertions(+), 4 deletions(-)
create mode 100644 tools/testing/kunit/mypy.ini
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index f01f94106129..7f648802caf6 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -92,7 +92,7 @@ class LinuxSourceTreeOperations:
if stderr: # likely only due to build warnings
print(stderr.decode())
- def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]:
+ def start(self, params: List[str], build_dir: str) -> subprocess.Popen:
raise RuntimeError('not implemented!')
@@ -113,7 +113,7 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
kconfig.merge_in_entries(base_kunitconfig)
return kconfig
- def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]:
+ def start(self, params: List[str], build_dir: str) -> subprocess.Popen:
kernel_path = os.path.join(build_dir, self._kernel_path)
qemu_command = ['qemu-system-' + self._qemu_arch,
'-nodefaults',
@@ -142,7 +142,7 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations):
kconfig.merge_in_entries(base_kunitconfig)
return kconfig
- def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]:
+ def start(self, params: List[str], build_dir: str) -> subprocess.Popen:
"""Runs the Linux UML binary. Must be named 'linux'."""
linux_bin = os.path.join(build_dir, 'linux')
params.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt'])
diff --git a/tools/testing/kunit/mypy.ini b/tools/testing/kunit/mypy.ini
new file mode 100644
index 000000000000..ddd288309efa
--- /dev/null
+++ b/tools/testing/kunit/mypy.ini
@@ -0,0 +1,6 @@
+[mypy]
+strict = True
+
+# E.g. we can't write subprocess.Popen[str] until Python 3.9+.
+# But kunit.py tries to support Python 3.7+, so let's disable it.
+disable_error_code = type-arg
diff --git a/tools/testing/kunit/run_checks.py b/tools/testing/kunit/run_checks.py
index 8208c3b3135e..c6d494ea3373 100755
--- a/tools/testing/kunit/run_checks.py
+++ b/tools/testing/kunit/run_checks.py
@@ -23,7 +23,7 @@ commands: Dict[str, Sequence[str]] = {
'kunit_tool_test.py': ['./kunit_tool_test.py'],
'kunit smoke test': ['./kunit.py', 'run', '--kunitconfig=lib/kunit', '--build_dir=kunit_run_checks'],
'pytype': ['/bin/sh', '-c', 'pytype *.py'],
- 'mypy': ['mypy', '--strict', '--exclude', '_test.py$', '--exclude', 'qemu_configs/', '.'],
+ 'mypy': ['mypy', '--config-file', 'mypy.ini', '--exclude', '_test.py$', '--exclude', 'qemu_configs/', '.'],
}
# The user might not have mypy or pytype installed, skip them if so.
base-commit: a42077b787680cbc365a96446b30f32399fa3f6f
--
2.40.1.495.gc816e09b53d-goog
On Mon, 1 May 2023 11:16:10 -0700 Daniel Latypov <[email protected]> wrote:
> Writing `subprocess.Popen[str]` requires python 3.9+.
> kunit.py has an assertion that the python version is 3.7+, so we should
> try to stay backwards compatible.
>
> This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
> pre-existing `mypy --strict` errors and update run_checks.py"), since
> mypy complains like so
> > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
>
> Note: `mypy --strict --python-version 3.7` does not work.
>
> We could annotate each file with comments like
> `# mypy: disable-error-code="type-arg"
> but then we might still get nudged to break back-compat in other files.
>
> This patch adds a `mypy.ini` file since it seems like the only way to
> disable specific error codes for all our files.
>
> Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
> but I think being explicit is better, particularly since most kernel
> devs won't be familiar with how mypy works.
Thank you for fast fix!
>
> Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
> Reported-by: SeongJae Park <[email protected]>
> Link: https://lore.kernel.org/linux-kselftest/[email protected]
> Signed-off-by: Daniel Latypov <[email protected]>
Tested-by: SeongJae Park <[email protected]>
Thanks,
SJ
[...]
On Tue, 2 May 2023 at 02:16, 'Daniel Latypov' via KUnit Development
<[email protected]> wrote:
>
> Writing `subprocess.Popen[str]` requires python 3.9+.
> kunit.py has an assertion that the python version is 3.7+, so we should
> try to stay backwards compatible.
>
> This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
> pre-existing `mypy --strict` errors and update run_checks.py"), since
> mypy complains like so
> > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
>
> Note: `mypy --strict --python-version 3.7` does not work.
>
> We could annotate each file with comments like
> `# mypy: disable-error-code="type-arg"
> but then we might still get nudged to break back-compat in other files.
>
> This patch adds a `mypy.ini` file since it seems like the only way to
> disable specific error codes for all our files.
>
> Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
> but I think being explicit is better, particularly since most kernel
> devs won't be familiar with how mypy works.
>
> Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
> Reported-by: SeongJae Park <[email protected]>
> Link: https://lore.kernel.org/linux-kselftest/[email protected]
> Signed-off-by: Daniel Latypov <[email protected]>
> ---
Thanks for jumping on this.
Looks good to me!
Reviewed-by: David Gow <[email protected]>
Cheers,
-- David
Hi David and Brendan,
On Tue, 2 May 2023 08:04:20 +0800 David Gow <[email protected]> wrote:
> [-- Attachment #1: Type: text/plain, Size: 1473 bytes --]
>
> On Tue, 2 May 2023 at 02:16, 'Daniel Latypov' via KUnit Development
> <[email protected]> wrote:
> >
> > Writing `subprocess.Popen[str]` requires python 3.9+.
> > kunit.py has an assertion that the python version is 3.7+, so we should
> > try to stay backwards compatible.
> >
> > This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
> > pre-existing `mypy --strict` errors and update run_checks.py"), since
> > mypy complains like so
> > > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
> >
> > Note: `mypy --strict --python-version 3.7` does not work.
> >
> > We could annotate each file with comments like
> > `# mypy: disable-error-code="type-arg"
> > but then we might still get nudged to break back-compat in other files.
> >
> > This patch adds a `mypy.ini` file since it seems like the only way to
> > disable specific error codes for all our files.
> >
> > Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
> > but I think being explicit is better, particularly since most kernel
> > devs won't be familiar with how mypy works.
> >
> > Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
> > Reported-by: SeongJae Park <[email protected]>
> > Link: https://lore.kernel.org/linux-kselftest/[email protected]
> > Signed-off-by: Daniel Latypov <[email protected]>
> > ---
>
> Thanks for jumping on this.
>
> Looks good to me!
>
> Reviewed-by: David Gow <[email protected]>
Looks like this patch is still not merged in the mainline. May I ask the ETA,
or any concern if you have?
Thanks,
SJ
>
> Cheers,
> -- David
On Sat, 10 Jun 2023 at 03:09, SeongJae Park <[email protected]> wrote:
>
> Hi David and Brendan,
>
> On Tue, 2 May 2023 08:04:20 +0800 David Gow <[email protected]> wrote:
>
> > [-- Attachment #1: Type: text/plain, Size: 1473 bytes --]
> >
> > On Tue, 2 May 2023 at 02:16, 'Daniel Latypov' via KUnit Development
> > <[email protected]> wrote:
> > >
> > > Writing `subprocess.Popen[str]` requires python 3.9+.
> > > kunit.py has an assertion that the python version is 3.7+, so we should
> > > try to stay backwards compatible.
> > >
> > > This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
> > > pre-existing `mypy --strict` errors and update run_checks.py"), since
> > > mypy complains like so
> > > > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
> > >
> > > Note: `mypy --strict --python-version 3.7` does not work.
> > >
> > > We could annotate each file with comments like
> > > `# mypy: disable-error-code="type-arg"
> > > but then we might still get nudged to break back-compat in other files.
> > >
> > > This patch adds a `mypy.ini` file since it seems like the only way to
> > > disable specific error codes for all our files.
> > >
> > > Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
> > > but I think being explicit is better, particularly since most kernel
> > > devs won't be familiar with how mypy works.
> > >
> > > Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
> > > Reported-by: SeongJae Park <[email protected]>
> > > Link: https://lore.kernel.org/linux-kselftest/[email protected]
> > > Signed-off-by: Daniel Latypov <[email protected]>
> > > ---
> >
> > Thanks for jumping on this.
> >
> > Looks good to me!
> >
> > Reviewed-by: David Gow <[email protected]>
>
> Looks like this patch is still not merged in the mainline. May I ask the ETA,
> or any concern if you have?
>
>
We've got this queued for 6.5 in the kselftest/kunit tree[1], so it
should land during the merge window. But I'll look into getting it
applied as a fix for 6.4, beforehand.
-- David
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit&id=e30f65c4b3d671115bf2a9d9ef142285387f2aff
On Sat, 10 Jun 2023 12:15:55 +0800 David Gow <[email protected]> wrote:
> [-- Attachment #1: Type: text/plain, Size: 2275 bytes --]
>
> On Sat, 10 Jun 2023 at 03:09, SeongJae Park <[email protected]> wrote:
> >
> > Hi David and Brendan,
> >
> > On Tue, 2 May 2023 08:04:20 +0800 David Gow <[email protected]> wrote:
> >
> > > [-- Attachment #1: Type: text/plain, Size: 1473 bytes --]
> > >
> > > On Tue, 2 May 2023 at 02:16, 'Daniel Latypov' via KUnit Development
> > > <[email protected]> wrote:
> > > >
> > > > Writing `subprocess.Popen[str]` requires python 3.9+.
> > > > kunit.py has an assertion that the python version is 3.7+, so we should
> > > > try to stay backwards compatible.
> > > >
> > > > This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
> > > > pre-existing `mypy --strict` errors and update run_checks.py"), since
> > > > mypy complains like so
> > > > > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
> > > >
> > > > Note: `mypy --strict --python-version 3.7` does not work.
> > > >
> > > > We could annotate each file with comments like
> > > > `# mypy: disable-error-code="type-arg"
> > > > but then we might still get nudged to break back-compat in other files.
> > > >
> > > > This patch adds a `mypy.ini` file since it seems like the only way to
> > > > disable specific error codes for all our files.
> > > >
> > > > Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
> > > > but I think being explicit is better, particularly since most kernel
> > > > devs won't be familiar with how mypy works.
> > > >
> > > > Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
> > > > Reported-by: SeongJae Park <[email protected]>
> > > > Link: https://lore.kernel.org/linux-kselftest/[email protected]
> > > > Signed-off-by: Daniel Latypov <[email protected]>
> > > > ---
> > >
> > > Thanks for jumping on this.
> > >
> > > Looks good to me!
> > >
> > > Reviewed-by: David Gow <[email protected]>
> >
> > Looks like this patch is still not merged in the mainline. May I ask the ETA,
> > or any concern if you have?
> >
> >
>
> We've got this queued for 6.5 in the kselftest/kunit tree[1], so it
> should land during the merge window. But I'll look into getting it
> applied as a fix for 6.4, beforehand.
Thank you for the kind answer, Gow! I was thinking this would be treated as a
fix, and hence merged into the mainline before next merge window. I'm actually
getting my personal test suite failures due to absence of this fix. It's not a
critical problem, but it would definitely better for me if this could be merged
into the mainline as early as possible.
Thanks,
SJ
>
> -- David
>
> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit&id=e30f65c4b3d671115bf2a9d9ef142285387f2aff
Hi Greg and Sasha,
On Sat, 10 Jun 2023 17:56:18 +0000 SeongJae Park <[email protected]> wrote:
> On Sat, 10 Jun 2023 12:15:55 +0800 David Gow <[email protected]> wrote:
>
> > [-- Attachment #1: Type: text/plain, Size: 2275 bytes --]
> >
> > On Sat, 10 Jun 2023 at 03:09, SeongJae Park <[email protected]> wrote:
> > >
> > > Hi David and Brendan,
> > >
> > > On Tue, 2 May 2023 08:04:20 +0800 David Gow <[email protected]> wrote:
> > >
> > > > [-- Attachment #1: Type: text/plain, Size: 1473 bytes --]
> > > >
> > > > On Tue, 2 May 2023 at 02:16, 'Daniel Latypov' via KUnit Development
> > > > <[email protected]> wrote:
> > > > >
> > > > > Writing `subprocess.Popen[str]` requires python 3.9+.
> > > > > kunit.py has an assertion that the python version is 3.7+, so we should
> > > > > try to stay backwards compatible.
> > > > >
> > > > > This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
> > > > > pre-existing `mypy --strict` errors and update run_checks.py"), since
> > > > > mypy complains like so
> > > > > > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
> > > > >
> > > > > Note: `mypy --strict --python-version 3.7` does not work.
> > > > >
> > > > > We could annotate each file with comments like
> > > > > `# mypy: disable-error-code="type-arg"
> > > > > but then we might still get nudged to break back-compat in other files.
> > > > >
> > > > > This patch adds a `mypy.ini` file since it seems like the only way to
> > > > > disable specific error codes for all our files.
> > > > >
> > > > > Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
> > > > > but I think being explicit is better, particularly since most kernel
> > > > > devs won't be familiar with how mypy works.
> > > > >
> > > > > Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
> > > > > Reported-by: SeongJae Park <[email protected]>
> > > > > Link: https://lore.kernel.org/linux-kselftest/[email protected]
> > > > > Signed-off-by: Daniel Latypov <[email protected]>
> > > > > ---
> > > >
> > > > Thanks for jumping on this.
> > > >
> > > > Looks good to me!
> > > >
> > > > Reviewed-by: David Gow <[email protected]>
> > >
> > > Looks like this patch is still not merged in the mainline. May I ask the ETA,
> > > or any concern if you have?
> > >
> > >
> >
> > We've got this queued for 6.5 in the kselftest/kunit tree[1], so it
> > should land during the merge window. But I'll look into getting it
> > applied as a fix for 6.4, beforehand.
>
> Thank you for the kind answer, Gow! I was thinking this would be treated as a
> fix, and hence merged into the mainline before next merge window. I'm actually
> getting my personal test suite failures due to absence of this fix. It's not a
> critical problem, but it would definitely better for me if this could be merged
> into the mainline as early as possible.
This patch is now in the mainline (e30f65c4b3d671115bf2a9d9ef142285387f2aff).
However, this fix is not in 6.4.y yet, so the original issue is reproducible on
6.4.y. Could you please add this to 6.4.y? I confirmed the mainline commit
can cleanly applied on latest 6.1.y tree, and it fixes the issue.
Thanks,
SJ
>
>
> Thanks,
> SJ
>
> >
> > -- David
> >
> > [1]: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit&id=e30f65c4b3d671115bf2a9d9ef142285387f2aff
On Mon, Jul 03, 2023 at 07:27:04PM +0000, SeongJae Park wrote:
> Hi Greg and Sasha,
>
> On Sat, 10 Jun 2023 17:56:18 +0000 SeongJae Park <[email protected]> wrote:
>
> > On Sat, 10 Jun 2023 12:15:55 +0800 David Gow <[email protected]> wrote:
> >
> > > [-- Attachment #1: Type: text/plain, Size: 2275 bytes --]
> > >
> > > On Sat, 10 Jun 2023 at 03:09, SeongJae Park <[email protected]> wrote:
> > > >
> > > > Hi David and Brendan,
> > > >
> > > > On Tue, 2 May 2023 08:04:20 +0800 David Gow <[email protected]> wrote:
> > > >
> > > > > [-- Attachment #1: Type: text/plain, Size: 1473 bytes --]
> > > > >
> > > > > On Tue, 2 May 2023 at 02:16, 'Daniel Latypov' via KUnit Development
> > > > > <[email protected]> wrote:
> > > > > >
> > > > > > Writing `subprocess.Popen[str]` requires python 3.9+.
> > > > > > kunit.py has an assertion that the python version is 3.7+, so we should
> > > > > > try to stay backwards compatible.
> > > > > >
> > > > > > This conflicts a bit with commit 1da2e6220e11 ("kunit: tool: fix
> > > > > > pre-existing `mypy --strict` errors and update run_checks.py"), since
> > > > > > mypy complains like so
> > > > > > > kunit_kernel.py:95: error: Missing type parameters for generic type "Popen" [type-arg]
> > > > > >
> > > > > > Note: `mypy --strict --python-version 3.7` does not work.
> > > > > >
> > > > > > We could annotate each file with comments like
> > > > > > `# mypy: disable-error-code="type-arg"
> > > > > > but then we might still get nudged to break back-compat in other files.
> > > > > >
> > > > > > This patch adds a `mypy.ini` file since it seems like the only way to
> > > > > > disable specific error codes for all our files.
> > > > > >
> > > > > > Note: run_checks.py doesn't need to specify `--config_file mypy.ini`,
> > > > > > but I think being explicit is better, particularly since most kernel
> > > > > > devs won't be familiar with how mypy works.
> > > > > >
> > > > > > Fixes: 695e26030858 ("kunit: tool: add subscripts for type annotations where appropriate")
> > > > > > Reported-by: SeongJae Park <[email protected]>
> > > > > > Link: https://lore.kernel.org/linux-kselftest/[email protected]
> > > > > > Signed-off-by: Daniel Latypov <[email protected]>
> > > > > > ---
> > > > >
> > > > > Thanks for jumping on this.
> > > > >
> > > > > Looks good to me!
> > > > >
> > > > > Reviewed-by: David Gow <[email protected]>
> > > >
> > > > Looks like this patch is still not merged in the mainline. May I ask the ETA,
> > > > or any concern if you have?
> > > >
> > > >
> > >
> > > We've got this queued for 6.5 in the kselftest/kunit tree[1], so it
> > > should land during the merge window. But I'll look into getting it
> > > applied as a fix for 6.4, beforehand.
> >
> > Thank you for the kind answer, Gow! I was thinking this would be treated as a
> > fix, and hence merged into the mainline before next merge window. I'm actually
> > getting my personal test suite failures due to absence of this fix. It's not a
> > critical problem, but it would definitely better for me if this could be merged
> > into the mainline as early as possible.
>
> This patch is now in the mainline (e30f65c4b3d671115bf2a9d9ef142285387f2aff).
> However, this fix is not in 6.4.y yet, so the original issue is reproducible on
> 6.4.y. Could you please add this to 6.4.y? I confirmed the mainline commit
> can cleanly applied on latest 6.1.y tree, and it fixes the issue.
As this was not specifically tagged with a "cc: stable..." marking, that
is why it was not picked up automatically. Also, we do not normally add
patches to any stable releases until it is in a released kernel from
Linus (i.e. a -rc release), unless you have a specific reason for it to
be merged earlier.
Should this be merged "now" into the stable trees and not wait for
6.5-rc1?
thanks,
greg k-h
On Mon, 3 Jul 2023 21:44:53 +0200 Greg KH <[email protected]> wrote:
> On Mon, Jul 03, 2023 at 07:27:04PM +0000, SeongJae Park wrote:
[...]
> > This patch is now in the mainline (e30f65c4b3d671115bf2a9d9ef142285387f2aff).
> > However, this fix is not in 6.4.y yet, so the original issue is reproducible on
> > 6.4.y. Could you please add this to 6.4.y? I confirmed the mainline commit
> > can cleanly applied on latest 6.1.y tree, and it fixes the issue.
>
> As this was not specifically tagged with a "cc: stable..." marking, that
> is why it was not picked up automatically.
I understand that. I thought this would be merged into the mainline before
v6.4 release, but I was wrong.
> Also, we do not normally add patches to any stable releases until it is in a
> released kernel from Linus (i.e. a -rc release), unless you have a specific
> reason for it to be merged earlier.
>
> Should this be merged "now" into the stable trees and not wait for
> 6.5-rc1?
The problem is not urgent, so I can wait until 6.5-rc1.
Thanks,
SJ
>
> thanks,
>
> greg k-h