A story of a Tanuki and a green Swan | Arroba IT
Skip to content
devops

A story of a Tanuki and a green Swan

Nov 23, 2018

Enable Gitlab CI for a Sylius project

My task was straight-forward: Add GitLab CI to a Sylius project.

While I have setup a lot of projects with TravisCI, I had never before with GitLab CI. All I knew was, analogous to TravisCI, that I would need some kind of configuration file which controls the automatization. GitLab says, that it is super easy – just create a .gitlab-ci.yml in your project root. But the more I read GitLabs documentation and blog posts, the more questions came to my mind.

  • Do I need to create a Docker container?

  • Where to store it?

  • Where to register it?

  • Installing a Gitlab Runner? Where? Why?

  • What are shared runners?

I went straight to the Sylius Slack Channel to ask if somebody already has a configuration file available … but the answer was negative. I thought that this is a common task and since Sylius is based on Symfony 4 aka Flex, somebody has to have written a config file for it. But … all I found was outdated or misleading information. At the end I cherry picked piece by piece from all those findings and literally pushed line by line to GitLab waiting for the outcome. It took me a little less than 100 commits to get it working. Another 10 for fine tuning.

In the following I will share and explain my config file with you. It can be used for a Sylius 1.3 project.

.gitlab-ci.yml

TLDR: Here you go. If your are in a hurry, just copy the file and commit it to your project.

 1stages:
 2  - test
 3 
 4variables:
 5  MYSQL_DATABASE: sylius_test_cached
 6  MYSQL_ROOT_PASSWORD: root
 7  MYSQL_USER: root
 8  # This is normally provided by the .env file. But since we have to use "mysql" as host name, its defined
 9  # here. We not using any .env file!
10  # The pattern is: mysql://user:password@host/database
11  DATABASE_URL: mysql://$MYSQL_USER:$MYSQL_ROOT_PASSWORD@mysql/$MYSQL_DATABASE
12 
13  # Set memory limit to infinite to avoid running out of memory issues
14  COMPOSER_MEMORY_LIMIT: -1
15 
16  # Set the environments variables manually
17  APP_ENV: test_cached
18  APP_DEBUG: 1
19  APP_SECRET: 1542752314
20  MAILER_URL: smtp://localhost
21 
22  PHP_DATE_TIMEZONE: Europe/Copenhagen
23 
24cache:
25  key: $CI_COMMIT_REF_NAME
26  paths:
27    - vendor
28    - node_modules
29    - public
30    - .yarn
31 
32test:
33  stage: test
34 
35  allow_failure: false # Its there for easy switching
36 
37  services:
38    - mysql:5.7
39    - memcached
40 
41  # This image was made for Laravel but works for Sylius as well
42  # https://github.com/edbizarro/gitlab-ci-pipeline-php
43  image: edbizarro/gitlab-ci-pipeline-php:7.2-alpine
44 
45  before_script:
46    - sudo yarn config set cache-folder .yarn
47    - mkdir -p public/media/cache && mkdir -p public/media/image # Avoiding error: public/media/image folder not found
48 
49  script:
50    # Install the dependencies. Don't execute scripts because the cache clean up take too long, sometimes times out
51    - composer install --prefer-dist --no-ansi --no-scripts --no-interaction --no-progress --no-suggest
52    - bin/console doctrine:database:create --if-not-exists --env=test_cached -vvv # Have to run with debug = true, to omit generation proxies before setting up the database
53    - yarn install
54    - bin/console sylius:theme:assets:install public
55    - bin/console sylius:install:assets
56    - yarn build
57    - bin/console cache:warmup --env=test_cached --no-debug -vvv
58    - bin/console doctrine:migrations:migrate --no-interaction --env=test_cached --no-debug -vvv
59 
60    # Running checks
61    - vendor/bin/phpcs -n --standard=PSR2 src
62    - vendor/bin/security-checker security:chec
63    - vendor/bin/phpspec run --no-interaction -f dot
64 
65    # Set up a frontend with fixture data
66    - bin/console sylius:fixtures:load --no-interaction --env=test_cached --no-debug -vvv
67    - bin/console server:run 127.0.0.1:8080 -d web --env test_cached > /dev/null 2>&1 &
68 
69    # Running tests
70    - echo "Starting BEHAT tests."
71    - cp behat.yml.dist behat.yml
72    - php bin/behat
73    - echo "BEHAT tests done."

Docker

I learned that I don't have to create a Docker container by myself. Some awesome people already did this and I (and you) can profit from it. The task was to find this container which comes with batteries included. The batteries here a the a fully installed LAMP stack. I tried alpine first only to find out it was missing a lot of stuff. I ended with the image from edbizarro/gitlab-ci-pipeline-php which addresses Laravel users but works very well for Flex/Sylius.

MySQL

It took me a while to configure the connection to the database. Used to use localhost or 127.0.0.1 for the --host I kept getting a Connection refused error. I learned to use mysql for the host option, since I import a MySQL Docker container when I specify mysql as a service in .gitlab-ci.yml.

.env Configuration

One of my sources for my .gitlab-ci.yml was the .travis.yml file at https://github.com/Sylius/Sylius-Standard/blob/master/.travis.yml. From there I took the rough structure and the environment. In case you wondered why I use --env=test_cached … now you know :-)

If you read the file you will see that they use a specific .env.test_cached file, which responds to the test_cached environment. The copy it and source it to make the environment variables available.

1- cp .env.test_cached.dist .env.test_cached
2- set -a && source .env.test_cached && set +a

You will miss this in my config because it sets the DATABASE_URL to mysql:://root@127.0.0.1/sylius_test_cached. This was too static for me, so I decided to set the environment variables manually and use those variables to create the database connection string.

Composer

All dependencies of Sylius are managed by Composer. There are two spots in the config I want to highlight:

1. Composer run out of memory: Sometimes while he was fetching the dependencies from upstream, sometimes while clearing the cache after installing the dependencies. Therefore I set the memory limit to infinite on line 14.

2. Composer hit max execution time: Again the clear cache command was the culprit. It takes ages to clear an–I guess already empty cache–that it slows down the whole process and sometimes even hits the max_execution_time limit. I decided to run composer install with the --no-scripts option (line 51).

I think the rest is pretty basic stuff and self explaining. If you have any question, please leave a comment.