It might not look like much, but it’s got it where it counts, kid.
‘Scrapcal‘ is a quick Rails project to generate images for my enormous IRL wall calendar.
Enough is working to tag this set of features as an alpha release. The purpose of this initial release was to get familiar with Rails development again, experiment with some ideas, get a proof of concept up and running, along with the whole continuous-integration deployment pipeline.
The build, test, stage and production environments are all on the same 4GB Digital Ocean instance running Ubuntu 16.04.
Two major issues this release created very difficult roadblocks and made development very painful. For both problems I decided it was best to bypass them than continue working through them.
GitLab CI Bundler Issues
As the build, staging and production environments are the same machine, the original intention was to use a GitLab Runner to build the Rails app and then simply move it locally to the correct deployment directory.
This intention ultimately created some considerable frustration but challenged my understanding of how the CI stack should work.
The first error was trying to attack too many new ideas at once. Through GitLab-CI I tried configuring Docker, Capistrano, rbenv and Bundler at the same time. This was especially painful as it would take several minutes for the CI to create the build; even simply syntax errors in the configuration could drag out the process. I had to wait a long time to determine my latest mistake.
Today's lesson: Docker sucks and/or I suck at Docker.
— Casey Driscoll (@caseydriscoll) May 2, 2017
Then in vain attempt to speed up the builds (to make mistakes faster), I started using GitLab-CI caching between builds. I believe this caused more problems as Bundler was not caching or building appropriately. It was painful to troubleshoot and sandbox the problems; it was simply too many ‘new’ things at once.
While working through this, I prematurely determined I didn’t want to use Capistrano as I would ‘only be SSHing into the same machine.” Additionally, I didn’t want to use Capistrano to trigger the deployment from my local environment to the staging server. Rather, I wanted the deployment to be triggered automatically after pushing the ‘stage’ branch to GitLab.
Is it a question of "Capistrano AND CI" or a question of "Capistrano OR CI"?
Do the complement or conflict with each other?
— Casey Driscoll (@caseydriscoll) May 2, 2017
Building and then moving the build directory seemed like the easier path and obvious path. It created more problems.
I configured Bundler to install locally with the –deployment flag, and every gem installed to the local vendor/bundler directory with no issue. However, when the entire build was moved to the deployment directory, rake could not find the necessary gems.
Ultimately, I think the problem was related to rbenv binstubs, but I could never confirm that was the issue.
After more research I decided to leverage Capistrano again. I set up the deploy user on the box, along with the appropriate private keys. It seemed silly to SSH directly into the same box I’m already on, but I realized it was a better process for a number of reasons.
- It will just work
- Moving directories simply was causing too many problems.
- Using Capistrano will allow me to move forward
- I get classic Capistrano functionality by deploying from local to external environments
- This will take away having to depend on GitLab CI
- Will also give me more experience with Capistrano scripts
- The project is ready for separate build/deployment servers in the future.
- They are the same box right now, but I’ll just need to change the server creds and it should work immediately.
- Become a better script writer
- For my WordPress deployment scripts I am borrowing a lot of the Capistrano ideas, like release directories and linked shared resources.
- However, I can’t recreate all of Capistrano and for Rails it would be best to leverage the de facto deployment tool.
- Long term, I’ll learn some Capistrano idiosyncrasies, and it should make me a better Bash scripter.
Using ‘Print’ as a model
The second major problem involved writing to the database with ActiveRecord. The summary of the issue is here.
A user should be able upload many ‘Photos,’ then collect those ‘Photos’ into ‘Groups,’ and then collect those ‘Groups’ into ‘Projects.’
The ‘Group’ is a 4×6 inch printed photo with two smaller square ‘Photos’ on it. At first I was calling the ‘Group’ a ‘Print’ and for some reason I couldn’t write to the database. I narrowed down the issue, but I couldn’t find exactly what the problem was. Ultimately I think the issue was using the name ‘Print’ as the class name.
Ultimately I refactored all references of ‘Print’ to ‘Proof,’ as in a Contact Print, and everything is working as expected.
Alpha Release Features
For this this initial release a user can perform the bare minimum of tasks:
- Login and Logout
- Upload Photos
- Create a ‘Project’
- Generate ‘Proofs’ on the Project
- Download an archived zip of the Project Proofs.
All of these actions are currently very tedious as you have to perform many of them, like uploading Photos, one at a time.
Click on the images below to see the full size gifs animate
Feature 00. Users can login and logout with Devise Authentication
This was my first time working with Devise so it was the first thing I installed.
I kept it simple. No user registration for now.
Feature 01. Users can upload Images to Photos
A ‘Photo’ is a model defined with a title:string and caption:text, but neither of those are necessary.
However, a ‘Photo’ must have an ‘Image’ reference to exist. When uploaded, Carrierwave creates these sizes:
- 150px Thumb
- 500px Medium
- 880px x 825px ‘Proof’ image (equivalent to 2.9375″ x 2.75″, the size of a day square on the wall calendar)
The Photo also stores the Date from when the Image was taken, based on Exif data. This is automatically set on upload, but can also be edited in the UI
Feature 02. Users can edit and destroy photos
Feature 03. Users can create Projects
Feature 04. Users can edit and destroy Projects
Feature 05. Users can generate ‘Proofs’
This feature was the most fun to develop. It also caused the most headaches with the ‘Print’ class problem written above.
I created a special PATCH/PUT route to handle the request, with an addition form and a ‘Generate all Proofs’ button.
When the button is clicked, Rails grabs all the photos and creates pairs them up, compositing them together with RMagick.
Learning RMagick compositing was a lot of fun, and I look forward to more photo manipulation in the future functionality.
A ‘Proof’ record is created for each pair and then attached to the Project.
Feature 06. Users can download archive
This was perhaps the biggest win of the development process.
Once all the ‘Proofs’ are generated, the User can download a zip of all the 4×6 images for that Project.
There is a custom route and controller to handle the zipping and send_file. I have to work out errors and trashing the zip after a certain length of time. Currently it just sits in the public/tmp folder indefinitely.
Bonus Feature 07: Calendar View
I forgot to mention that the default view is a monthly calendar, showing all the Photos as they would appear IRL on the wall calendar.
Each empty day has a link to the ‘Add Photo’ url (though it doesn’t populate the Date from the click)
Gems: Simple Calendar
For the beta, hopefully released at the end of next week, I’d like to have the set of features below. The focus will still be on initial CRUD app functionality over HTTP (no AJAX). Extra polish like CSS and JS updates will come later.
- Implement Unit Tests for Alpha Features
- Implement CI for Unit Tests
- ‘User’ Levels
- New User Signups
- Minimum Password Validation
- Forgotten Password Reminder
- ‘Admin’ Level Account
- /admin Screen
- CRUD Users
- Reset User Passwords
- Fine Tune Single CRUD Interactions
- If a user deletes a Photo, does that delete the image?
- Add new single Proofs to Projects
- Add pre-existing Proofs to Projects
- Alter the Photos used on a Proof
- Bulk CRUD Operations
- Upload Multiple Photos at Once
- Delete Multiple Photos at Once
- Add unused photos to pre-existing Projects
- Images in Public Folders have custom hashed filenames
- Image Editor
- Add Day Number to Photo
- Original should remain untouched
- Choose Black/White Color for Date Number
- Add Day Number to Photo