At Zero Config Rails, I (Prabin) am constantly working on automating configurations and boring setups like “Configure Minitest with Gitlab CI”.
gem install zcr-zen && zen add ci:gitlab_ci --app_test_framework=minitest
For the detailed list of configurations, you can visit Gitlab CI Generator.
Now without further ado, let’s jump right into setting up Gitlab CI for Minitest and run those tests in CI.
Assumptions
You have basic Gitlab CI configurations ready i.e.
.gitlab-ci.yml
exists in your project.If it doesn’t, you can refer to my other article Integrate Pronto with Gitlab for Rails App
You are using PostgreSQL in your app, though with minimal changes it should work for any other databases.
You are using import-maps, though I have added configurations for projects with esbuild as well, you can just uncomment them.
Tested and working in
Ruby 3.3.0
Rails 7.2.1
Minitest 5.25.1
selenium-webdriver 4.24.0
Configure Gitlab CI Variables
Firs of all, we need to add some configurations required by the CI to run tests. This should be done over at Gitlab.
Add variable for storing environment variables
I normally use Figjam which is a maintained version of the popular Figaro gem for storing environment variables which uses config/application.yml
but just the plain .env
file using dotenv gem is also very popular. Anyway, just copy the content from whatever you are using and paste it inside the Value for this new variable.
You can visit the official documentation to learn about setting up variables for Gitlab CI. You have to go to your project's setting in Gitlab and configure these in CI/CD variables.
Create a new variable for storing content in your config/application.yml
:
Type: File
Flags
Uncheck all checklists here i.e. Protect variable, Mask variable and Expand variable reference
Description
You can add “Environment Variables“ but it's optional and you can skip this as "Key" (just below this) is already clear enough on what this variable is storing.
Key:
env
In "Value", add the copied content from your env file.
NOTE: Make sure to only copy what is under "test" block or ".env.test", you don’t want to add production variables here!
Add variable for MASTER_KEY
Rails comes with config/credentials.yml.enc
for storing secrets, we generally also use ENV variables for this but since Rails credentials is the default, we will also look at how to configure those.
To decrypt credentials file, you need MASTER_KEY. If you have generated multiple credentials file per environment then you might have multiple keys like master.key, staging.key, production.key, etc..
Create a new variable for storing content in the “.key” file that can decrypt secrets configured for the test environment; normally this will be inside the config/master.key
:
Type: Variable (Default)
Flags
Uncheck all checklists here i.e. Protect variable, Mask variable and Expand variable reference
Description
Optional. You can leave it blank.
Key: MASTER_KEY
And in Value, add the content copied from the master.key
Add database.yml.ci
file
It's not considered a good practice to use config/database.yml
file for the CI so we will instead create a new file config/database.yml.ci
and add configurations required to run tests inside.
You can visit the official documentation to learn about setting up variables. You have to go to your project's setting in Gitlab and configure these in CI/CD variables.
After creating the file, add the following:
test:
adapter: postgresql
encoding: unicode
host: postgres
database: ci_db
username: postgres
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
For username it should be “postgres” which is the default user that gets created when postgres service/docker is created hence it doesn’t ask password or tries to authenticate user. You might get an error otherwise because no other users will have been created in postgres at this point: `Please check your database configuration to ensure the username/password are valid`.
For host make sure to use "postgres" instead of “localhost”. For MySQL, you will have to use "mysql" as said in the official documentation:
The service container for MySQL is accessible under the hostname mysql. To access your database service, connect to the host named mysql instead of a socket or localhost.
SQLite doesn’t need any host configurations but other configurations will most probably vary.
Configure Capybara with Selenium
We will configure Selenium with Chrome to be used both in CI and Local with Headless mode (by default) while also allowing to run in the browser if needed for debugging.
Create a new file "test/helpers/capybara.rb" and add the following code:
require "selenium-webdriver"
Capybara.register_driver :selenium_chrome_custom do |app|
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument("--headless=new") unless ENV["SELENIUM_HEADFUL"]
options.add_argument("--window-size=1400,1400")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
remote_url = ENV["SELENIUM_REMOTE_URL"]
if remote_url
Capybara::Selenium::Driver.new(
app,
browser: :remote,
url: remote_url,
options:
)
else
Capybara::Selenium::Driver.new(app, browser: :chrome, options:)
end
end
Explanation
Let's look at what each of the code block above is doing.
Custom Selenium Chrome driver
Capybara.register_driver :selenium_chrome_custom
Since existing Selenium Drivers don't provide the custom options we want, we are creating a new driver selenium_chrome_custom
which will handle Remote/Local connection as well as Headless/Headful mode.
Options
--window-size=1400,1400
Set the window size to 1400x1400 pixels. This is a reasonable size without being too large, but you can set it to whatever you like. This mostly impacts the size of debugging screenshots, but some tests may fail if you ask Capybara to click on an element which is not currently visible on the page.
--no-sandbox
Disables Chrome’s sandbox functionality because it has an issue with Docker version 1.10.0 and later.
--disable-dev-shm-usage
The "/dev/shm" shared memory partition is too small on many VM environments which will cause Chrome to fail or crash so we are disabling it.
--headless=new
Enable Chrome’s headless mode which will run Chrome without a UI.
SELENIUM_HEADFUL
will control this option. In development, you may want to run Chrome and see what's happening in the browser for debugging; you can do so by running tests withSELENIUM_HEADFUL=true bundle exec rspec spec/system
.We will see list of other commands to run system tests at the end of this explanation section in a bit.
Some guides may suggest using the --disable-gpu flag, but this is no longer necessary on any operating system.
This explanation was shamelessly copied from Remote Selenium WebDriver servers with Rails, Capybara, RSpec, and Chrome 🙈.
Selenium remote URL
remote_url = ENV[“SELENIUM_REMOTE_URL"]
Remote option is required mostly for CI but you can also test it out in local by running the Selenium Docker image e.g. with SELENIUM_REMOTE_URL=
http://localhost:4444/wd/hub
bundle exec rspec spec/system
Remote option is controlled by SELENIUM_REMOTE_URL
which needs to be passed when running tests as seen above.
Another configuration related to the remote is the use of browser: :remote
inside Capybara::Selenium::
Driver.new
which tells Capybara to run tests in remote Chrome browser instead of local one.
Add host configurations
Update the test/application_system_test_case.rb
file to include the following content so Gitlab CI can run tests in the remote browser.
# other require declarations ...
require "helpers/capybara"
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
# other codes ....
driven_by :selenium_chrome_custom
def setup
Capybara.server_host = "0.0.0.0" # bind to all interfaces
Capybara.server_port = 3000
if ENV["SELENIUM_REMOTE_URL"].present?
ip = Socket.ip_address_list.detect(&:ipv4_private?).ip_address
Capybara.app_host = "http://#{ip}:#{Capybara.server_port}"
end
super
end
end
Do note, you should replace the line driven_by …
with driven_by :selenium_chrome_custom
as shown above.
server_host
and app_host
are required for Capybara to know how it can call driver in the Remote Server.
This piece of code was extracted from the official Rails Documentation.
Commands to run tests
Lastly, let's see various commands we can use to run system tests.
Run in headless mode (default):
bundle exec rails test:system
Run in headful mode:
SELENIUM_HEADFUL=true bundle exec rails test:system
Run in headless mode inside external docker image in local:
SELENIUM_REMOTE_URL=http://localhost:4444/wd/hub bundle exec rails test:system
For CI, default command bundle exec rails test:system
will work. But SELENIUM_REMOTE_URL
will be http://selenium:4444/wd/hub
and it will be passed as an Environment Variable instead. We will look at how to do that next.
Update .gitlab-ci.yml
to run all tests
We will be adding code to enable all the following tests and you can choose to pickup or ignore as per your requirement:
Unit and Integration tests (Model, Requests, Authorization, Services etc.) which don't require us to start browser
System Tests where we will start the Chrome browser and run tests inside it
Update your .gitlab-ci.yml
with the configurations given below. Most of the configurations are accompanied by explanation, you can find clean configuration without comment at the end of the blog in the section "Final .gitlab-ci.yml
"
# change to the ruby version your application uses
image: ruby:3.3.0
variables:
MASTER_KEY: $MASTER_KEY
# explanation in next section
cache:
paths:
- vendor/
# uncomment till yarn.lock if you are using esbuild i.e. you have package.json in your project
# - node_modules/
# - yarn.lock # or package-lock.json
stages:
- test
# base configuration required for running tests
.base_db:
# add-on docker images required for running tests
services:
- postgres:latest
variables:
# set Rails environment so we don't have to prefix each command with RAILS_ENV=test
RAILS_ENV: test
# Postgres runs in a separate docker image and requires authentication to connect. Disabling that here by using "trust" so it doesn't ask for authentication
POSTGRES_HOST_AUTH_METHOD: trust
before_script:
# use same bundler version that was used in bundling the Gemfile
- gem install bundler -v "$(grep -A 1 "BUNDLED WITH" Gemfile.lock | tail -n 1)" --no-document
# install all gems to "vendor" folder which helps in caching of gem installation in between the execution of CI jobs
- bundle config set --local path "vendor"
# you can uncomment lines till `yarn install` if you are using esbuild
# - apt-get update -qq
# install "nodejs" required for yarn
# - apt-get install -y -qq nodejs
# - curl -o- -L https://yarnpkg.com/install.sh | bash
# Make yarn available in the current terminal
# - export PATH="$HOME/.yarn/bin:$HOME/.config/yarn/global/node_modules/.bin:$PATH"
# - yarn install --pure-lockfile
- bundle install --jobs $(nproc)
- cp config/database.yml.ci config/database.yml
# config/application.yml can be different for you. If you are using dotenv gem then this content will be `cat $env > .env`
- cat $env > config/application.yml
- bundle exec rails db:test:prepare
unit_and_integration_tests:
# reuse all configurations defined in .base_db above
extends: .base_db
stage: test
# run this job only when merge requests are created, updated or merged
only:
- merge_requests
script:
- bundle exec rails test
system_tests:
extends: .base_db
stage: test
services:
- name: selenium/standalone-chrome:latest
alias: selenium
# need to declare postgres again because "services" key will override the one defined in .base_db
- postgres:latest
variables:
RAILS_ENV: test
# Location of the selenium docker image. "selenium" is an alias, you can also use http://selenium-standalone-chrome:4444/wd/hub or selenium__standalone-chrome (commonly seen in other guides)
SELENIUM_REMOTE_URL: http://selenium:4444/wd/hub
only:
- merge_requests
script:
- bundle exec rails test:system
# store necessary files and folders in case of test failure for debugging the error
artifacts:
when: on_failure
paths:
- log/test.log
- tmp/screenshots/
expire_in: 1 week
Explanation
Let's look at some configurations where explanation was missing and would be lengthy to add there.
cache
cache:
paths:
- vendor/
- node_modules/
- yarn.lock
This tells Gitlab CI to cache vendor folder where we are storing all our gems, node_modules where all JS packages are stored and yarn.lock which stores the information about installed packages with their versions.
Storing all these folders and files speed up the CI in subsequent runs. bundle install
and yarn install
will only install new packages that are not already inside the cache.
stages
stages:
- test
Stages define when to run the jobs.
If you also have linting and continuous deployment configured then stages could look like this:
stages:
- lint
- test
- staging_deploy
- production_deploy
Jobs are run in the same order as configured here i.e. linting will run first then test and lastly deployments.
.base_db
All common configurations used by jobs that require database access are extracted here.
services
are add-on docker images and provide capabilities like database, redis, selenium drivers, etc.
variables
are environment variables used by Rails.
before_script
runs before the script
so anything that needs to be pre-configured can be added here.
unit_and_integration_tests
extends
will extend the configurations defined in the .base_db
and use those configurations for this job.
stage
tells this job at what stage to run. Depending on stages
defined just above this job configuration.
script
are the series of command to execute for running this job.
system_tests
selenium/standalone-chrome:latest
configures the docker image for Selenium with Chrome with the latest version.
artifacts
is used to store necessary files and folders in case of test failure. This helps us in debugging failing tests when needed. We are storing test log files for this purpose.
Final .gitlab-ci.yml
This is how your gitlab-ci.yml
should look like if you have followed everything in this blog:
image: ruby:3.3.0
variables:
MASTER_KEY: $MASTER_KEY
cache:
paths:
- vendor/
stages:
- test
.base_db:
services:
- postgres:latest
variables:
RAILS_ENV: test
POSTGRES_HOST_AUTH_METHOD: trust
before_script:
- gem install bundler -v "$(grep -A 1 "BUNDLED WITH" Gemfile.lock | tail -n 1)" --no-document
- bundle config set --local path 'vendor'
- bundle install --jobs $(nproc)
- cp config/database.yml.ci config/database.yml
- cat $env > config/application.yml
- bundle exec rails db:test:prepare
unit_and_integration_tests:
extends: .base_db
stage: test
only:
- merge_requests
script:
- bundle exec rails test
system_tests:
extends: .base_db
stage: test
services:
- postgres:latest
- name: selenium/standalone-chrome:latest
alias: selenium
variables:
RAILS_ENV: test
POSTGRES_HOST_AUTH_METHOD: trust
SELENIUM_REMOTE_URL: http://selenium:4444/wd/hub
only:
- merge_requests
script:
- bundle exec rails test:system
artifacts:
when: on_failure
paths:
- log/test.log
- tmp/screenshots/
expire_in: 1 week
Conclusion
Phew, that was a lot of configurations and explanation. And you can automate all of this with just a single command from Zero Config Rails in near future, stay tuned!
With this, your Rails app now has all type of tests running in the Gitlab CI so you can now merge changes without any worry for them breaking the production application.
Thank you for reading. Happy coding!