Felt a few pains using concourse to manage my teams very large testing & release structure. One major pain was the way script dependencies were being used within tasks and jobs. In order to provide consistency with the scripts and programs that all tasks had access to, we would build a docker image with these scripts and executable.

After a while the docker image bloated to a few GB’s in size which wasn’t ideal because every time a task would run for the first time, it would have to pull that large image making pipeline runs slower. Overall it’s probably an anti-pattern; concourse images should have what’s required to run its associated task.

I had a “novel” idea about a custom concourse resource that could download and use executable binaries needed within tasks instead of preloading them into docker images. Source code here

A felt a bit silly come to find out that the idea wasn’t novel at all and this resource already existed. Check that out here, it’s good

Anyway creating the resource helped me learn more about how concourse uses a docker images as a key piece to building pipelines. I also learned how to:

• Redirect STDIN within Rspec in order to test ruby scripts that use STDIN
• Test ruby scripts using the __FILE__ and __PROGRAM_NAME__. __FILE__ is a special variable that evaluates to the running ruby filename. This can be compared to another special variable __PROGRAM_NAME__ which evaluates to command that boots the program script. This is useful because a ruby script can be invoked on the command line (__FILE__ and __PROGRAM_FILE__ will be the same in this scenario allowing for main code to run) while simultaneously run within the Rspec framework ( __FILE__ and __PROGRAM_FILE__ will be different because of Rspec stopping main block from running in test)

Right now the resource is pretty limited in that it currently works with only linux alpine images. It also only pulls images from Github. If ever I have time, I may come back to adding more features. One feature is to allow the download of Github binaries from private repositories.

Shoutout to this blog for helping me better understand implementing the resource