30 December 2012
On Malaysia Twitter Political Index project, I’m using delayed job extensively. I used it to pull tweets, query sentiments and update sentiments. These jobs created every 10 minutes by Heroku Scheduler, Heroku’s answer to traditional cron job.
These jobs took about 2 or 3 minutes to finish. While waiting for the next cron job, worker dyno are idle for 7 or 8 minutes. I don’t like it because I have to pay for it!
I’m in luck because Heroku provides a ruby client gem to interact with Heroku API. With this gem, we can easily start or stop dynos from within the app itself. There are few ways we can leverage on this, I chose to create rake task that will spin the dyno up and down. Then let Heroku Scheduler run it in specific intervals.
rake dynos:up to run immediately after all those sentiments job created, and scheduled
rake dynos:down 3 minutes after that. Dyno will only spin up if there’s job, and will spin down if there’s no more pending jobs. Of course if you really want to push it, you can schedule it to run every minute just be sure.
If you look at
heroku.post_ps_scale(ENV['APP_NAME'], 'worker', '0'), you can see that I assigned my app name to
APP_NAME config var. I did that because I can’t find a way to programmatically get my app name from Heroku. If you use New Relic addon you could get it from
NEW_RELIC_APP_NAME, but I feel more comfortable setting it myself.
Now I managed to save around 70% of my estimated monthly dyno cost. If you want to go gung ho with it, potentially you could add this to
after_delete callbacks. That really depends on the nature of your jobs though.
NOTE: I wrote about ‘How I Use Two Dynos On heroku For Free’ but I like Heroku API way better than using separate app to run worker. Less moving pieces to maintain.