Welcome!

Eclipse Authors: Pat Romanski, Elizabeth White, Liz McMillan, David H Deans, JP Morgenthal

Blog Feed Post

How we use Middleman and Codeship to deploy our Documentation to Amazon S3

How we deploy our Documentation to Amazon S3 using Codeship

We always want to make our system easier to use, so we recently launched our new Documentation. You can read more about this in our last blogpost. In this post we will go into the details of how we implemented our docs and where we deploy them.

Problems with Documentation and Current Infrastructure

In the past our documentation was part of our main application. Every change that gets deployed to our main application needs to be thoroughly tested and goes through code review. This made updating our documentation a major pain and very time consuming. It hindered us from getting updates to our docs out quickly.

When we decided to redo our docs we wanted to have them hosted externally. We had some experience with Jekyll in the past, so static pages looked like a great option. We looked into Jekyll and Middleman, but Middleman looked like the better option to us, as it is easier to extend.

Using Static Pages and Amazon S3

Static pages are easy to build and design. We have full control over the layout and no complicated infrastructure is required to serve the pages . Delivering static pages from S3 is cheap and fast, so a good option for our docs.

Deployment to S3 is very easy and everything can be included in a build on Codeship.

Setting up Middleman App

Our documentation is all open source on Github: https://github.com/codeship/docs For our configuration take a look at our config.rb. We use the blog extension as it enables us to do proper permalinks and adds a few other nice options.

To build the middleman app we run

bundle install
bundle exec middleman build

which creates a build folder that will be later deployed to S3.

Deploying to Amazon S3

Of course here at Codeship everything is continuously deployed, including our docs. We decided early on to go with static pages hosted on S3, as it is fast, easy and cheap. We wanted to have the documentation on codeship.io/documentation, which meant we had to do proxy the pages through our application. Here’s how we did it.

Installing the Deployment Tools

As a first step we wanted to use the official awscli tool provided by Amazon. They can be installed through pip, so pip install awscli is all we needed.

Codeship – A hosted Continuous Deployment platform for web applications

AWS Access Keys

To be able to interact with the AWS API we needed to set environment variables. You can do this by simply exporting the following variables (or set them on the Environment tab on Codeship)

AWS_DEFAULT_REGION=us-east-1
AWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY
AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY

Removing Old Content from our Page

As a next step we wanted to make sure all docs were removed from the S3 bucket before we deployed the new ones, so we wouldn’t run into any left overs.

aws s3 rm s3://docs.codeship.io --recursive

Now the bucket was ready for the new docs.

Deploying to the S3 Bucket

Now we were ready for the deployment step. The following command syncs the build folder Middleman created to S3.

aws s3 sync build s3://docs.codeship.io --acl public-read --cache-control "public, max-age=86400"

As static pages typically don’t change very often we wanted to have caching in place, so the page is loaded quickly. By setting the cache-control header we make sure all parts of our documentation are cached for one day by the browser. The –acl public-read flag makes sure all files in the bucket are accessible for everyone.

Amazon S3 Static Website Hosting

Now that we deployed to our S3 bucket we needed to make sure it is treated as a static website by AWS. In our AWS Console we went to our S3 Bucket and enabled static website hosting.

Settings for our Bucket on S3

You can read more about this in the AWS documentation about static website hosting

Setting up Application forwarding

So now we set up our docuemtation to be served through the S3 bucket on http://docs.codeship.io.s3-website-us-east-1.amazonaws.com/documentation/

Now we needed to move it to codeship.io/documentation

Choosing rack-reverse-proxy

In the end we decided to go with rack-reverse-proxy by Jon Swope. In our config.ru we added the following as the first rack middleware:

require 'rack/reverse_proxy'

use Rack::ReverseProxy do
  reverse_proxy /^\/documentation\/?(.*)$/, 'http://docs.codeship.io.s3-website-us-east-1.amazonaws.com/documentation/$1'
end

This simply forwards every request to the S3 bucket and returns the result.

Set Asset Host to CDN for HTTPS

We didn’t want assets to be routed through our application, so we set up a cloudfront distribution to server files from the s3 bucket.

CDN

It has the added benefit that it works over https as well. In our config.rb we set the asset host:

activate :asset_host, host:'https://d1hmmog0ddn2l5.cloudfront.net'

Using Blitz.io Load Tests

To make sure our reverse proxy works well and doesn’t take down our application we ran a couple of tests through Blitz.io. The results were great, especially considering that those requests would be cached the second time.

How we use blitz.io

Conclusions

Using S3 for static pages made our documentation a lot better. We can change it easily any time while paying absolutely nothing for it. We will definitely use S3 in the future as well for different pages we want to deploy.

Further Information:

Read the original blog entry...

More Stories By Manuel Weiss

I am the cofounder of Codeship – a hosted Continuous Integration and Deployment platform for web applications. On the Codeship blog we love to write about Software Testing, Continuos Integration and Deployment. Also check out our weekly screencast series 'Testing Tuesday'!

IoT & Smart Cities Stories
René Bostic is the Technical VP of the IBM Cloud Unit in North America. Enjoying her career with IBM during the modern millennial technological era, she is an expert in cloud computing, DevOps and emerging cloud technologies such as Blockchain. Her strengths and core competencies include a proven record of accomplishments in consensus building at all levels to assess, plan, and implement enterprise and cloud computing solutions. René is a member of the Society of Women Engineers (SWE) and a m...
Early Bird Registration Discount Expires on August 31, 2018 Conference Registration Link ▸ HERE. Pick from all 200 sessions in all 10 tracks, plus 22 Keynotes & General Sessions! Lunch is served two days. EXPIRES AUGUST 31, 2018. Ticket prices: ($1,295-Aug 31) ($1,495-Oct 31) ($1,995-Nov 12) ($2,500-Walk-in)
According to Forrester Research, every business will become either a digital predator or digital prey by 2020. To avoid demise, organizations must rapidly create new sources of value in their end-to-end customer experiences. True digital predators also must break down information and process silos and extend digital transformation initiatives to empower employees with the digital resources needed to win, serve, and retain customers.
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Machine learning has taken residence at our cities' cores and now we can finally have "smart cities." Cities are a collection of buildings made to provide the structure and safety necessary for people to function, create and survive. Buildings are a pool of ever-changing performance data from large automated systems such as heating and cooling to the people that live and work within them. Through machine learning, buildings can optimize performance, reduce costs, and improve occupant comfort by ...