Category Archives: Gitlab

How to skip running Gitlab Pipeline when specific files change.

In some cases you might not want to run the pipeline when specific files change. For example there is no need to run the pipeline if you update only the README.md file in the repository. There is a built in function to do this called “except” the following example will show how to use it.

apply:
  stage: apply
  when: manual
  script:
    - terraform apply -auto-approve
  except:
    changes:
      - README.md
  dependencies:
    - validate and review

In the above exampe we skip running the apply job when the README.md changes. Please note that this has to be defined in EACH job you have in the pipeline configuration.

How to automatically register Gitlab Runners using runnerCreate GraphQL API mutation and Project Runners?

Gitlab has changed their Gitab Runner registration process from version 15.10. Previously there was a simple one step registration process where it was possible to register a runner by simply running gitlab-runner register command on the server/entity where the runner was installed. The old procedure was using a single registration token which was linked to the project and was used by all the runners registered for this one project.

The registration process now have changed from one step to two steps and each runner now will have to obtain their own unique authentication token.

Step 1: A new project runner will typically be created manually on the gitlab.com interface. ( scroll down for automation ).

Many of the options previously set from gitlab-runner register command as a parameter, now will be set here instead. Such as run-untagged, locked, tag-list, etc.

This new project runner will generate a unique authentication token.

Step 2: Run the gitlab-runner register command using this unique authentication token to link this runner with the project.

Needless to say that this breaks any kind of automation that has been created previously for runner registration so this would cause multiple issues for any customer using automatic registration.

After browsing some forums I found out that a new project runner can be created automatically using gitlab’s graphQL API. There is a mutation called RunnerCreate which does this part automatically. So running this call will return the authentication token, which then can be used to register the runner. I used the following call to get an authentication token:

mutation {
  runnerCreate(
    input: {projectId: "gid://gitlab/Project/00000000", runnerType: PROJECT_TYPE, tagList: "yourtag"}
  ) {
    errors
    runner {
	ephemeralAuthenticationToken
    }
  }
}

Replace the 00000000 with your project id and it should create a new project runner and return the authentication token.

I have also written a bash script that would automate the entire process end to end. The script should be running on the server/entity where the runner is installed.

 #!/usr/bin/bash
 export PRIVATE_TOKEN="Replace with your Personal Access Tokens" 
 # Go to Settings -> General and copy the numeric value from "Project ID".
 export PROJECT_ID="Replace with your Project ID" 
 export TAGLIST="yourtag" 
 export RUN_UNTAGGED="true"
 export LOCKED="true"
 # Change this is to your own hosted gitlab URL if you use gitlab.com  leave the value set.
 export GITLAB_URL="https://gitlab.com" 
 export TOKEN=$(curl "$GITLAB_URL/api/graphql" --header "Authorization:  Bearer $PRIVATE_TOKEN" --header "Content-Type: application/json" --request POST --data-binary '{"query": "mutation { runnerCreate( input: {projectId: \"gid://gitlab/Project/'$PROJECT_ID'\", runnerType: PROJECT_TYPE, tagList: \"'$TAGLIST'\", runUntagged: '$RUN_UNTAGGED', locked: '$LOCKED'} ) { errors runner { ephemeralAuthenticationToken } } }"}' |jq '.data.runnerCreate.runner.ephemeralAuthenticationToken' | tr -d '"')
 sudo gitlab-runner register --non-interactive --url $GITLAB_URL --token $TOKEN --executor shell

This script is also available on github.

How to install Ta-Lib and its python library on Ubuntu 22?

Installing TA-lib on an Ubuntu server has its challenges as not only the python library has to be installed but the product should be downloaded and compiled first. Use the following steps to perform the installation:

mkdir -p /app
sudo apt-get install build-essential autoconf libtool pkg-config python3-dev -y
cd /app
sudo wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
sudo tar -xzf ta-lib-0.4.0-src.tar.gz
cd ta-lib/
sudo ./configure
sudo make
sudo make install
sudo pip3 install --upgrade pip
sudo pip3 install TA-Lib

In case you would be using gitlab pipelines you can use the following job to do the same:

 stages:
   - prepare

 prepare:
   stage: prepare
   script:
     - mkdir -p /app
     - sudo apt-get install build-essential autoconf libtool pkg-config       python3-dev -y
     - cd /app
     - sudo wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
     - sudo tar -xzf ta-lib-0.4.0-src.tar.gz
     - cd ta-lib/
     - sudo ./configure
     - sudo make
     - sudo make install
     - sudo pip3 install --upgrade pip
     - sudo pip3 install TA-Lib

Happy trading!