--- title: Blog in Markdown, Deploy with Webhooks teaser: Markdown + GitHub + Webhooks for a smooth authoring experience. tags: web,rails author: - Joël Quenneville - Melanie Gilman published_on: 2015-03-06 --- We recently rewrote the code that powers this blog. Previously, the blog ran as a [Middleman app]. The new system is tailored to our preferred authoring workflow (Markdown + GitHub) and takes advantage of webhooks to automate tasks that are not writing or reviewing a post. [Middleman app]: https://middlemanapp.com/ ## Splitting content from engine The idea to rebuild the blog stemmed from a conversation about publishing new blog posts. We love our process of writing posts in Markdown, versioning them via Git and reviewing them via GitHub pull requests. However, in our previous setup, we needed to redeploy the blog to Heroku whenever a new post was published. This was tedious and frustrating. The ideal workflow would be to merge a pull request for a new blog post and have everything else happen automatically. A big obstacle to this goal was the coupling between the content of our blog and the application code that served it. This led to the decision to break up our blog into two independent repositories. One would contain the blog engine, written in Rails, while the other would be strictly Markdown documents. ![new blog workflow](https://images.thoughtbot.com/rebuilding-the-blog/new-blog-authoring-workflow.png) ## Setting up a GitHub webhook GitHub allows you to subscribe to events on a repository via a webhook. You provide them with a URL and they will post to it every time the designated event occurs. When a new post gets merged to the master branch of the content repository, we respond by kicking off an import. ![github's webhook event options](https://images.thoughtbot.com/rebuilding-the-blog/github-webhook-events.jpg) GitHub's [documentation for webhooks] is pretty good. Check it out. For security reasons, we want to restrict access to the webhook URL to only allow payloads from GitHub. GitHub allows you to set a secret key with which the incoming request is signed. If the request signature matches the payload hashed with the secret key, then we know the request is genuine. [documentation for webhooks]: https://developer.github.com/webhooks/ ## Caching with Fastly We host our blog on Heroku and use Fastly as our CDN. Jessie wrote a fantastic post on [how to set up Fastly with a Rails application]. We used this approach for the blog engine. When we import a new post, we purge the cache. However, this won't work for posts that don't show up immediately on the blog such as those with future dates. In addition, we run a daily task via [heroku scheduler] that purges the posts. Initially we were confused by the `Article#purge` and `Article.purge_all` methods included into our models by the `fastly-rails` gem. `Article#purge` will expire all pages that have the surrogate key for that individual article while `Article.purge_all` will expire all pages that have the general article surrogate key. Some pages have both, for example: ```ruby def index @articles = Article.recent set_surrogate_key_header Article.table_key, @articles.map(&:record_key) end ``` This index page can be expired by calling `Article.purge_all` or by calling `purge` on any of the article objects rendered on that page. So when should you use one over the other? * When creating a new object you want to use `purge_all`. This is a new object that isn't on any page yet so `purge` wouldn't do anything. * When updating an object, you can use `purge`. This will expire any pages that render that object. [heroku scheduler]: https://devcenter.heroku.com/articles/scheduler [how to set up Fastly with a Rails application]: https://thoughtbot.com/blog/a-guide-to-caching-your-rails-application-with-fastly ## Building a sitemap Search engines like Google and Bing use [XML sitemaps] to generate search results. The [Giant Robots sitemap] allows us to inform search engines about the relative importance of each URL on the site and how often they change. The most popular gem we found for generating sitemaps, [SitemapGenerator], generates static files and suggests setting up a cron job to update it periodically. We found that it wasn't difficult to serve our own dynamic sitemap using the `Builder` templates that ship with Rails. [XML sitemaps]: http://www.sitemaps.org [Giant Robots sitemap]: https://thoughtbot.com/blog/sitemap.xml [SitemapGenerator]: https://github.com/kjvarga/sitemap_generator ## Authoring posts locally While splitting the content from the engine simplified a lot of things, it did make previewing posts more difficult. Previously, an author could spin up a local Middleman server and preview their post exactly as it would show up on the blog. However, the new engine doesn't read files from the local repo but imports them from GitHub instead. This would force authors to: * Set up a local version of the engine * Connect it to the GitHub repository's webhook * Push to GitHub in order for GitHub to send the file back down to their local machine so the engine can render it. This whole workflow is tedious. We considered using a standard Markdown editor such as Marked to preview the posts but then they wouldn't be rendered using our stylesheet and layouts. We decided to implement an author mode that would read Markdown files from the local file system rather than the database + GitHub. In order to do this, we built a set of lightweight objects that mimicked our ActiveRecord models. `Local::Article`, `Local::Author`, and `Local::Tag`. These objects are backed by the file system rather than the database. To ensure the correct objects are called by the controller we added the following initializer: ```ruby if ENV.fetch("AUTHOR_MODE") == "true" && ENV["LOCAL_POSTS_PATH"].present? require "local/article" require "local/tag" require "local/author" Article = Local::Article Author = Local::Author Tag = Local::Tag end ``` You would expect this would cause some "already initialized constant" errors if these redefine constants from our models or if the models get loaded after this initializer. However, this is not the case. Rails' [autoloading system] will only load a model file if it finds an undefined constant named for that file. Since our constants are already defined, Rails will never load the models. [autoloading system]: http://urbanautomaton.com/blog/2013/08/27/rails-autoloading-hell/ ## Conclusion Since these changes went live, authoring blog posts is now much more streamlined. We get to focus on writing content using our favorite plain-text editor and getting feedback on GitHub. Once satisfied, we merge our post and it will automatically show up on the blog on the day it was dated for. Magical!