Currently it's not possible to select which runner will handle a pipeline
without changing the codebase.
Add CI_TAGS environment variable, which can be used to select a runner
either from a commit message, or directly from Gitlab interface.
Also add Documentation for this variable.
Signed-off-by: Leonardo Bras <[email protected]>
---
This patch goes on top on a previous patchset sent by Helen:
https://lore.kernel.org/all/[email protected]/
With this patch I could run CI with gitlab.com runners, by setting
CI_TAGS=saas-linux-medium-amd64
The result of this pipeline can be seen in:
https://gitlab.com/linux-kernel/linux/-/pipelines/1228999646
Documentation/ci/gitlab-ci/gitlab-ci.rst | 5 +++++
ci/gitlab-ci/yml/gitlab-ci.yml | 2 ++
2 files changed, 7 insertions(+)
diff --git a/Documentation/ci/gitlab-ci/gitlab-ci.rst b/Documentation/ci/gitlab-ci/gitlab-ci.rst
index 4f7ef03cca95..18360da835bd 100644
--- a/Documentation/ci/gitlab-ci/gitlab-ci.rst
+++ b/Documentation/ci/gitlab-ci/gitlab-ci.rst
@@ -304,20 +304,25 @@ Description of Each Variable
**KCI_CHECKPATCH_OPTIONS**
Used in `checkpatch.pl "$KCI_CHECKPATCH_OPTIONS"` (see checkpatch
documentation). It is commonly used with the --ignore flag to suppress
specific warnings generated by checkpatch.pl. It can also be defined in the
commit message, since it is evaluated in run time.
**KCI_PATCH_SERIES_SIZE**
Used to define the size of the patch series, see `job: checkpatch` section
above. It is evaluated in run time, and can be set in the commit message.
+**CI_TAGS**
+ Used to help choose which runner will deal with the current pipeline.
+ If using Gitlab.com runners, set saas-linux-medium-amd64 or a better runner
+ so there is enough resources to build & commit the base image.
+
.. _triggering-pipelines-from-command-line:
Triggering Pipelines from Command Line
--------------------------------------
Pipelines can be triggered from the command line with custom variables using the
`GitLab CLI tool <https://docs.gitlab.com/ee/editor_extensions/gitlab_cli>`_.
Example:
diff --git a/ci/gitlab-ci/yml/gitlab-ci.yml b/ci/gitlab-ci/yml/gitlab-ci.yml
index 57b9c0290471..359b7715e3ab 100644
--- a/ci/gitlab-ci/yml/gitlab-ci.yml
+++ b/ci/gitlab-ci/yml/gitlab-ci.yml
@@ -33,20 +33,22 @@ workflow:
- if: $FORCE_CI == 'true'
variables:
FDO_UPSTREAM_REPO: helen.fornazier/linux # The repo where to look for cached images
# ccache builds in gitlab-runner to speed up builds
SMATCH_DB_DIR: /smatch/smatch_data
# exit code of bash script on `script` will be the exit code of the job
FF_USE_NEW_BASH_EVAL_STRATEGY: "true"
default:
+ tags:
+ - $CI_TAGS
artifacts:
paths:
- artifacts/
when: always
include:
- remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/ci-fairy.yml'
- remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/debian.yml'
- ci/gitlab-ci/yml/kernel-combinations.yml
--
2.44.0
Hi Leonardo,
Thanks for the patch.
On 26/03/2024 22:30, Leonardo Bras wrote:
> Currently it's not possible to select which runner will handle a pipeline
> without changing the codebase.
>
> Add CI_TAGS environment variable, which can be used to select a runner
> either from a commit message, or directly from Gitlab interface.
>
> Also add Documentation for this variable.
>
> Signed-off-by: Leonardo Bras <[email protected]>
> ---
>
> This patch goes on top on a previous patchset sent by Helen:
> https://lore.kernel.org/all/[email protected]/
>
> With this patch I could run CI with gitlab.com runners, by setting
> CI_TAGS=saas-linux-medium-amd64
>
> The result of this pipeline can be seen in:
> https://gitlab.com/linux-kernel/linux/-/pipelines/1228999646
>
> Documentation/ci/gitlab-ci/gitlab-ci.rst | 5 +++++
> ci/gitlab-ci/yml/gitlab-ci.yml | 2 ++
> 2 files changed, 7 insertions(+)
>
> diff --git a/Documentation/ci/gitlab-ci/gitlab-ci.rst b/Documentation/ci/gitlab-ci/gitlab-ci.rst
> index 4f7ef03cca95..18360da835bd 100644
> --- a/Documentation/ci/gitlab-ci/gitlab-ci.rst
> +++ b/Documentation/ci/gitlab-ci/gitlab-ci.rst
> @@ -304,20 +304,25 @@ Description of Each Variable
> **KCI_CHECKPATCH_OPTIONS**
> Used in `checkpatch.pl "$KCI_CHECKPATCH_OPTIONS"` (see checkpatch
> documentation). It is commonly used with the --ignore flag to suppress
> specific warnings generated by checkpatch.pl. It can also be defined in the
> commit message, since it is evaluated in run time.
>
> **KCI_PATCH_SERIES_SIZE**
> Used to define the size of the patch series, see `job: checkpatch` section
> above. It is evaluated in run time, and can be set in the commit message.
>
> +**CI_TAGS**
I would just s/CI/KCI , I'm using KCI_prefix for those that are defined
by us, since gitlab already uses CI_ prefix.
We will also need a way to specify tags per job at some point
KCI_TAGS_$JOB_NAME or something, but this can be added later.
Acked-by: Helen Koike <[email protected]>
I'll add on top of my tree for now, so we can prepare v2 with this change.
Thanks
Helen
> + Used to help choose which runner will deal with the current pipeline.
> + If using Gitlab.com runners, set saas-linux-medium-amd64 or a better runner
> + so there is enough resources to build & commit the base image.
> +
> .. _triggering-pipelines-from-command-line:
>
> Triggering Pipelines from Command Line
> --------------------------------------
>
> Pipelines can be triggered from the command line with custom variables using the
> `GitLab CLI tool <https://docs.gitlab.com/ee/editor_extensions/gitlab_cli>`_.
>
> Example:
>
> diff --git a/ci/gitlab-ci/yml/gitlab-ci.yml b/ci/gitlab-ci/yml/gitlab-ci.yml
> index 57b9c0290471..359b7715e3ab 100644
> --- a/ci/gitlab-ci/yml/gitlab-ci.yml
> +++ b/ci/gitlab-ci/yml/gitlab-ci.yml
> @@ -33,20 +33,22 @@ workflow:
> - if: $FORCE_CI == 'true'
>
> variables:
> FDO_UPSTREAM_REPO: helen.fornazier/linux # The repo where to look for cached images
> # ccache builds in gitlab-runner to speed up builds
> SMATCH_DB_DIR: /smatch/smatch_data
> # exit code of bash script on `script` will be the exit code of the job
> FF_USE_NEW_BASH_EVAL_STRATEGY: "true"
>
> default:
> + tags:
> + - $CI_TAGS
> artifacts:
> paths:
> - artifacts/
> when: always
>
> include:
> - remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/ci-fairy.yml'
> - remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/debian.yml'
>
> - ci/gitlab-ci/yml/kernel-combinations.yml
On Thu, Mar 28, 2024 at 10:17:00AM -0300, Helen Koike wrote:
> Hi Leonardo,
>
> Thanks for the patch.
>
> On 26/03/2024 22:30, Leonardo Bras wrote:
> > Currently it's not possible to select which runner will handle a pipeline
> > without changing the codebase.
> >
> > Add CI_TAGS environment variable, which can be used to select a runner
> > either from a commit message, or directly from Gitlab interface.
> >
> > Also add Documentation for this variable.
> >
> > Signed-off-by: Leonardo Bras <[email protected]>
> > ---
> >
> > This patch goes on top on a previous patchset sent by Helen:
> > https://lore.kernel.org/all/[email protected]/
> >
> > With this patch I could run CI with gitlab.com runners, by setting
> > CI_TAGS=saas-linux-medium-amd64
> >
> > The result of this pipeline can be seen in:
> > https://gitlab.com/linux-kernel/linux/-/pipelines/1228999646
> >
> > Documentation/ci/gitlab-ci/gitlab-ci.rst | 5 +++++
> > ci/gitlab-ci/yml/gitlab-ci.yml | 2 ++
> > 2 files changed, 7 insertions(+)
> >
> > diff --git a/Documentation/ci/gitlab-ci/gitlab-ci.rst b/Documentation/ci/gitlab-ci/gitlab-ci.rst
> > index 4f7ef03cca95..18360da835bd 100644
> > --- a/Documentation/ci/gitlab-ci/gitlab-ci.rst
> > +++ b/Documentation/ci/gitlab-ci/gitlab-ci.rst
> > @@ -304,20 +304,25 @@ Description of Each Variable
> > **KCI_CHECKPATCH_OPTIONS**
> > Used in `checkpatch.pl "$KCI_CHECKPATCH_OPTIONS"` (see checkpatch
> > documentation). It is commonly used with the --ignore flag to suppress
> > specific warnings generated by checkpatch.pl. It can also be defined in the
> > commit message, since it is evaluated in run time.
> > **KCI_PATCH_SERIES_SIZE**
> > Used to define the size of the patch series, see `job: checkpatch` section
> > above. It is evaluated in run time, and can be set in the commit message.
> > +**CI_TAGS**
>
> I would just s/CI/KCI , I'm using KCI_prefix for those that are defined by
> us, since gitlab already uses CI_ prefix.
I was aware of CI_ vs KCI_ prefix, but for some reason at the time I
decided that since this was about setting the gitlab tags, and not
something internal to Kernel CI, it would make more sense to use the CI_
prefix.
But I have no strong opinion on this, so if you think it's better to
call it KCI_TAGS, let's do that instead.
>
> We will also need a way to specify tags per job at some point
> KCI_TAGS_$JOB_NAME or something, but this can be added later.
Yeah, agree.
I wish we could specify tags with running requirements for each task in
some way, and let the gitlab instance to pick the better runner.
>
> Acked-by: Helen Koike <[email protected]>
>
> I'll add on top of my tree for now, so we can prepare v2 with this change.
>
> Thanks
> Helen
Thank you!
Leo
>
> > + Used to help choose which runner will deal with the current pipeline.
> > + If using Gitlab.com runners, set saas-linux-medium-amd64 or a better runner
> > + so there is enough resources to build & commit the base image.
> > +
> > .. _triggering-pipelines-from-command-line:
> > Triggering Pipelines from Command Line
> > --------------------------------------
> > Pipelines can be triggered from the command line with custom variables using the
> > `GitLab CLI tool <https://docs.gitlab.com/ee/editor_extensions/gitlab_cli>`_.
> > Example:
> > diff --git a/ci/gitlab-ci/yml/gitlab-ci.yml b/ci/gitlab-ci/yml/gitlab-ci.yml
> > index 57b9c0290471..359b7715e3ab 100644
> > --- a/ci/gitlab-ci/yml/gitlab-ci.yml
> > +++ b/ci/gitlab-ci/yml/gitlab-ci.yml
> > @@ -33,20 +33,22 @@ workflow:
> > - if: $FORCE_CI == 'true'
> > variables:
> > FDO_UPSTREAM_REPO: helen.fornazier/linux # The repo where to look for cached images
> > # ccache builds in gitlab-runner to speed up builds
> > SMATCH_DB_DIR: /smatch/smatch_data
> > # exit code of bash script on `script` will be the exit code of the job
> > FF_USE_NEW_BASH_EVAL_STRATEGY: "true"
> > default:
> > + tags:
> > + - $CI_TAGS
> > artifacts:
> > paths:
> > - artifacts/
> > when: always
> > include:
> > - remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/ci-fairy.yml'
> > - remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/debian.yml'
> > - ci/gitlab-ci/yml/kernel-combinations.yml
>