In some cases you might not want to run the pipeline when specific files change. For example there is no need to run the pipeline if you update only the README.md file in the repository. There is a built in function to do this called “except” the following example will show how to use it.
In the above exampe we skip running the apply job when the README.md changes. Please note that this has to be defined in EACH job you have in the pipeline configuration.
Gitlab has changed their Gitab Runner registration process from version 15.10. Previously there was a simple one step registration process where it was possible to register a runner by simply running gitlab-runner register command on the server/entity where the runner was installed. The old procedure was using a single registration token which was linked to the project and was used by all the runners registered for this one project.
The registration process now have changed from one step to two steps and each runner now will have to obtain their own unique authentication token.
Step 1: A new project runner will typically be created manually on the gitlab.com interface. ( scroll down for automation ).
Many of the options previously set from gitlab-runner register command as a parameter, now will be set here instead. Such as run-untagged, locked, tag-list, etc.
This new project runner will generate a unique authentication token.
Step 2: Run the gitlab-runner register command using this unique authentication token to link this runner with the project.
Needless to say that this breaks any kind of automation that has been created previously for runner registration so this would cause multiple issues for any customer using automatic registration.
After browsing some forums I found out that a new project runner can be created automatically using gitlab’s graphQL API. There is a mutation called RunnerCreate which does this part automatically. So running this call will return the authentication token, which then can be used to register the runner. I used the following call to get an authentication token:
Replace the 00000000 with your project id and it should create a new project runner and return the authentication token.
I have also written a bash script that would automate the entire process end to end. The script should be running on the server/entity where the runner is installed.
#!/usr/bin/bash
export PRIVATE_TOKEN="Replace with your Personal Access Tokens"
# Go to Settings -> General and copy the numeric value from "Project ID".
export PROJECT_ID="Replace with your Project ID"
export TAGLIST="yourtag"
export RUN_UNTAGGED="true"
export LOCKED="true"
# Change this is to your own hosted gitlab URL if you use gitlab.com leave the value set.
export GITLAB_URL="https://gitlab.com"
export TOKEN=$(curl "$GITLAB_URL/api/graphql" --header "Authorization: Bearer $PRIVATE_TOKEN" --header "Content-Type: application/json" --request POST --data-binary '{"query": "mutation { runnerCreate( input: {projectId: \"gid://gitlab/Project/'$PROJECT_ID'\", runnerType: PROJECT_TYPE, tagList: \"'$TAGLIST'\", runUntagged: '$RUN_UNTAGGED', locked: '$LOCKED'} ) { errors runner { ephemeralAuthenticationToken } } }"}' |jq '.data.runnerCreate.runner.ephemeralAuthenticationToken' | tr -d '"')
sudo gitlab-runner register --non-interactive --url $GITLAB_URL --token $TOKEN --executor shell
Installing TA-lib on an Ubuntu server has its challenges as not only the python library has to be installed but the product should be downloaded and compiled first. Use the following steps to perform the installation:
mkdir -p /app
sudo apt-get install build-essential autoconf libtool pkg-config python3-dev -y
cd /app
sudo wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
sudo tar -xzf ta-lib-0.4.0-src.tar.gz
cd ta-lib/
sudo ./configure
sudo make
sudo make install
sudo pip3 install --upgrade pip
sudo pip3 install TA-Lib
In case you would be using gitlab pipelines you can use the following job to do the same: