[Infrastructure as Code (IaC)] Triggering GitLab CI/CD pipelines through the API

Posted on 2021-10-09 Edited on 2023-09-02 In Infrastructure as Code (IaC) , GitLab , GitLab Runner Views: Word count in article: 5.5k Reading time ≈ 5 mins.

Triggering GitLab pipelines through the API

GitLab CI/CD is a tool for software development using the continuous methodologies:

GitLab Triggers can be used to force a pipeline rerun of a specific ref (branch or tag) with an API call.

Create a personal access token

Personal access tokens can be an alternative to OAuth2 and used to:

Authenticate with the GitLab API.
Authenticate with Git using HTTP Basic Authentication.

You can create as many personal access tokens as you like.

In the top-right corner, select your avatar.
Select Edit profile.
On the left sidebar, select Access Tokens.
Enter a name and optional expiry date for the token.
Select the desired scopes.
Select Create personal access token.

Adding a new trigger

Go to your Settings > CI/CD under Triggers to add a new trigger. The Add trigger button creates a new token which you can then use to trigger a rerun of this particular project’s pipeline.

Every new trigger you create, gets assigned a different token which you can then use inside your scripts or .gitlab-ci.yml. You also have a nice overview of the time the triggers were last used.

Adding a new trigger

In the following examples, you can see the exact API call you need to make in order to rebuild a specific ref (branch or tag) with a trigger token.

All you need to do is replace the TOKEN and REF_NAME with the trigger token and the branch or tag name respectively.

Use cURL

Copy one of the tokens above, set your branch or tag name, and that reference will be rebuilt.

curl -X POST \
     -F token=TOKEN \
     -F ref=REF_NAME \
     https://<Your GitLab URL>/api/v4/projects/<Your Project>/trigger/pipeline

Use .gitlab-ci.yml

In the .gitlab-ci.yml of another project, include the following snippet. The project will be rebuilt at the end of the pipeline.

trigger_build:
  stage: deploy
  script:
    - "curl -X POST -F token=TOKEN -F ref=REF_NAME https://<Your GitLab URL>/api/v4/projects/<Your Project>/trigger/pipeline"

Use webhook

Add the following webhook to another project for Push and Tag push events. The project will be rebuilt at the corresponding event.

1	https://<Your GitLab URL>/api/v4/projects/<Your Project>/ref/REF_NAME/trigger/pipeline?token=TOKEN

Pass job variables

Add variables[VARIABLE]=VALUE to an API request. Variable values can be used to distinguish between triggered pipelines and normal pipelines. With cURL:

curl -X POST \
     -F token=TOKEN \
     -F "ref=REF_NAME" \
     -F "variables[RUN_NIGHTLY_BUILD]=true" \
     https://<Your GitLab URL>/api/v4/projects/<Your Project>/trigger/pipeline

With webhook:

1	https://<Your GitLab URL>/api/v4/projects/<Your Project>/ref/REF_NAME/trigger/pipeline?token=TOKEN&variables[RUN_NIGHTLY_BUILD]=true

See Triggering pipelines through the API | GitLab - https://docs.gitlab.com/ee/ci/triggers/ to learn more.

Add a CI/CD variable to a project

You can add CI/CD variables to a project’s settings. Only project members with the Maintainer role can add or update project CI/CD variables. To keep a CI/CD variable secret, put it in the project settings, not in the .gitlab-ci.yml file.

To add or update variables in the project settings:

1. Go to your project’s Settings > CI/CD and expand the Variables section.
1. Select the Add Variable button and fill in the details:
- Key: Must be one line, with no spaces, using only letters, numbers, or _.
- Value: No limitations.
- Type: File or Variable.
- Environment scope: (Optional) All, or specific environments.
- Protect variable (Optional): If selected, the variable is only available in pipelines that run on protected branches or tags.
- Mask variable (Optional): If selected, the variable’s Value is masked in job logs. The variable fails to save if the value does not meet the masking requirements.

There are a few Variables should be defiend.

TRIGGER_TOKEN
BRANCH
GITLAB_URL
PROJECT_ID
HEADER, for example: HEADER="PRIVATE-TOKEN: <ACCESS_TOKEN>"

Usages

In the .gitlab-ci.yml, use the following to triggering a pipeline and waiting for the pipeline status from pending to success.

testing:
  stage: testing
  image: alpine
  script:
    # Install curl and jq packages.
    # Alpine
    # apk add --no-cache curl jq

    # Tigger a pipnline
    - CURL="curl -X POST -F token=${TRIGGER_TOKEN} -F ref=${BRANCH} ${GITLAB_URL}/api/v4/projects/${PROJECT_ID}/trigger/pipeline"
    - echo $CURL
    - TRIGGER_PIPELINE=`${CURL}`
    - echo ${TRIGGER_PIPELINE}

    # Get the pipnline id
    - PIPELINE_ID=`echo ${TRIGGER_PIPELINE} | jq .id`
    - echo ${PIPELINE_ID}

    # Wait for Pipeline success with while loop
    - while test `curl --header ${HEADER} ${GITLAB_URL}/api/v4/projects/${PROJECT_ID}/pipelines/${PIPELINE_ID} | echo | jq .status` != "success"; do sleep 10; echo "waiting 10s for PipeLine ${PIPELINE_ID} success..."; done

See The .gitlab-ci.yml file | GitLab- https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html to learn more.

FAQs

script config should be an array of strings or a string

PRIVATE-TOKEN: ${ACCESS_TOKEN} within .gitlab.ci.yml cause that syntax error.

1 2	script: - HEADER="PRIVATE-TOKEN: ${ACCESS_TOKEN}"

Use GitLab CI/CD environment variables | GitLab - https://docs.gitlab.com/ee/ci/variables/ to solve it.

Variable value is empty

If the variable is marked as protected, please remember to also mark the corresponding branch as protected, otherwise the obtained variable value is empty.