Setting up Initial Continuous Integration

|

A continuous integration set up can be a bit tedious to initially set up, but can pay dividends in the long run.

Continuous Integration is great for testing, building and deploying projects automatically. Once the scripts are correctly set up, you never have to manually deploy again.

Luckily, I had previously set up a “Runner Steps” Continuous Integration script for deploying WordPress projects with GitLab CI, so it was relatively easy to set up the server configuration in a similar fashion.

Setting up local development environment

I like to do my local PHP development on VVV: a Vagrant stack that was created for WordPress.

To make VVV 2.0 work for Drupal I need to tweak a few of the configuration files with the following steps:

  1. Add an ‘arkofthegovernment.dev’ line to /vvv-custom.yml
  2. Add an arkofthegovernment.dev root web directory to /www
  3. Install Drupal 7 with Drush and rename that directory to arkofthegovernment.dev/public_html
  4. Add log and provision directories to the web root
  5. Create custom provision/vvv-init.sh file with VVV_DB_NAME
  6. Create custom provision/vvv-nginx.sh file with a new line include /etc/nginx/nginx-drupal-common.conf;
  7. Create the new nginx-drupal-common.conf file in /config/nginx-config/, file copied from the main nginx recipe
  8. Add a line under 411 of /provision/provision.sh to copy the new nginx-drupal-common.conf file to the main /etc/nginx directory

I also had to install drush locally on VVV, which was a bit tricky as the official docs don’t match the configuration I have exactly. Of important note, the global composer file on my Ubuntu 16.04 version is at ~/.config/composer/, not at ~/.composer like it says in the docs. I decided to install Drush 7 as I am running Drupal 7 for this project.

Additionally, I added arkofthegovernment.dev to my host laptop /etc/hosts file for the appropriate IP address I use for all my VVV sites.

Bare Drupal 7 install working as expected

 

Setting up staging environment

All of my projects run on a single Digital Ocean instance I lovingly call ‘Groucho.’

Groucho runs all my projects, including my experiments in Rails and Java, and 3rd party apps like Jira and GitLab. An nginx reverse proxy sits in front of all of it directing traffic.

If you look at the ‘runner-steps’ project above, for my WordPress projects I have a similar architecture set up like Capistrano, with a ‘release’ directory, a ‘current’ soft link to the most recent release, and a ‘shared’ directory of assets common of all releases. For now that shared directory just contains the ‘sites’ directory, which includes files and the settings.php file.

Here are the general steps it takes to install a Drupal 7 site as a staging instance:

  1. Make sure Arecords are created on the DNS record at Digital Ocean for www, stage and @
  2. Create a /srv/www/arkofthegovernment.com web root directory
  3. Create public_html and stage_html directories in the web root
  4. Create releases and shared directories in stage_html (public_html will come later)
  5. Create a database, add rights and flush privileges
    1. I use a special MySQL user for this that is only for staging databases. Not root or anything with access to prod DBs
  6. Create a stage_html/shared/sites/default/settings.php file with database credentials
  7. Create new release directory in releases with a timestamp of date +%Y%m%d%H%M%S created in bash
  8. Create a soft link from current to the new release timestamp directory
  9. Create an nginx record in /etc/nginx/sites-available
    1. I also create Let’s Encrypt SSL certs, but that is for another post
  10. Create a soft link from /etc/nginx/sites-enabled to the new nginx record
  11. Install the initial Drupal site with drush site-install standard
  12. Reload nginx!
Shiny!

 

Here is a view of the final server set up. tt is an alias for the tree command alias tt='tf -L 2', tf itself an alias for alias tf='tree -C'

Setting up Continuous Integration

Now that the local and the stage environment are set up I can create a GitLab CI runner to build, test and deploy the ‘stage’ branch on every push.

My server already has a shared runner set up, so this project will just use that as well.

For now as I am just getting started, it will simply execute the deploy step. Essentially this deploy script will create a new release folder in the staging web root, rsync the repo to the new release directory create new soft links where necessary, and reload nginx and PHP. There is also a script to harden the release directory by setting appropriate file permissions.

The steps for accomplishing this are pretty straight forward:

  1. Add a .gitlab-ci.yml file to the root of the repo. This file has instructions to:
    1. Download a new repo called drupal-runnerand call it steps
    2. Make stepsexecutable
    3. Export a variable called CI_RELEASE based on the bash timestamp above
    4. Also contains instructions for a build_stage and build_prod to execute the build, deploy, harden and backup scripts in steps
  2. Create a new stage branch
  3. Add a STAGE_PATH variable to the GitLab CI project setting so it isn’t included in the public repo
  4. git push origin stage
  5. Watch as it fails several times
  6. Make arbitrary syntax tweaks
  7. Watch as it succeeds!
The GitLab CI jobs fail easily but theyll be backAnd in greater numbers.

 

Here is a quick template test to prove it all works. I added ‘And Casey’ to to the page.tpl.php template and push the changes. They appear live on the stage server nearly instantly (after the job runs)!

I only have myself to thank.
It’s working. It’s working!

Resources

  1. Drupal deployment strategy and folder structure
  2. Continuous deployment, Drupal style

 

Leave a Reply

Your email address will not be published. Required fields are marked *