There are a few options for client-side code highlighting.  [Highlight.js](https://highlightjs.org/) is one of the most popular, and it's super easy to integrate into your site.  Let's do it!



## Compared to Server Side

I usually default to server-side code highlighting.  It puts the complexity of code highlighting in the build phase and keeps the runtime simple.  The most popular server-side option is [pygments](http://pygments.org/).  I have used this for many years.  Unfortunately, in my latest [conversion to hugo](/post/conversion-from-octopress-to-hugo/), I realized that the support for server-side highlighting was not good.  Not only was it slow, I found out over time that many simple things weren't even getting correctly interpreted as code blocks (eg, anything json or html, which should be common types, right?).  So I switched to Highlight.js on the client side.  I was pleasantly surprised at its simplicity.  Besides the couple new network requests, it's really simple for the consumer of this library.

## Include the Script

First, include the script near the bottom of the page that has code blocks on it:

```html
<script src="http://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.9.0/highlight.min.js"></script>
```

Next, call the Hightlight.js initialize function:

```html
<script>hljs.initHighlightingOnLoad();</script>
```

When this function runs, it'll look for the `pre > code` blocks, like this:

```html
<pre>
  <code class="language-js">
    const nacho = anaconda => squeeze(anaconda)
    console.log(nacho())
  </code>
</pre>
```

And turn them into:

```html
<pre>
  <code class="language-js hljs javascript">
    <span class="hljs-keyword">const</span> nacho = <span class="hljs-function"><span class="hljs-params">anaconda</span> =&gt;</span> squeeze(anaconda)
    <span class="hljs-built_in">console</span>.log(nacho())
  </code>
</pre>
```

## Include the Stylesheet

Now that the code snippet has been parsed and class names added to elements representing the different parts of the code syntax, we want to color it.  There are [several options for styles](https://highlightjs.org/) available on the hightlight.js home page (though the ui is weird -- just click the "style" link under the code sample there to cycle through options).

I like one called `atom-one-dark`, so I will next include that stylesheet in the `<head />` of my page that contains code blocks:

```html
<link rel="stylesheet" href="http://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.9.0/styles/atom-one-dark.min.css" type="text/css" media="all" />
```

That's it.  You're now set.  Just load your page in the browser, and it should apply the highlighting.  That was easy, right?  Do you prefer client-side or server-side code highlighting?  Do you use Highlight.js or something else?  Any favorite highlighting themes?

Highlight.js is a great option to add client-side code highlighting to your blog

Add Client-side Code Highlighting to Your Blog

Once you get a bunch of data that you want to put on your site, it's common to reach for a database.  Then you'll write an app that consumes that database and renders a dynamic site.  What if, instead, you could create a static site from that data.  What would that buy you?



## Dynamic Database

If you have a dynamic site backed by a database (eg, postgres), you'll have some good and some bad.  The good:

- Your data is well-defined, segregated into clean columns of data
- You can plug your data in anywhere in your dynamic site
- You can change data without having to redeploy your site

The bad:

- Your code is more complicated.  There are more moving parts to support the dynamic render of the site.
- Your site requires more computing power to run.  Simple crud is no big deal per se, but your site could be stressed under load.
- Your dynamic site is more expensive to host.  You have tons of options, but many will cost money.

## Markdown as a Database

But what if you could use markdown files as a database.  Instead of columns accessed by a database process, your data is stored on the filesystem as files.  The most database-like part of a markdown file is the front matter.  This is a meta data section at the top of the document that maintains some info about the document itself.  What if it contained all or some of the significant data of your doc?  

Front matter can be presented in a couple of formats like yaml, toml, or json.  I prefer yaml.  The front matter is demarcated with `---` on both sides, like:

```
aName: "for my front matter data"
```

Hmm, there are possibilities here.  It almost looks like a key-value store.  Once you have a key-value pair in the front matter, you can pull it out in the template and have access to it.  For instance, in [Hugo](http://gohugo.io/), a static site platform, you can access custom variables in your templates on the page's `.Params` object, like: `{{ .Params.aName}}`.

## Database for a Static Site

When you use markdown files as the database for your site, you get some great things:

- Good-enough data segregation with key-value pairs in the front matter.
- You get a simple, light-weight site that will take a load beating. 
- Likely cheaper and simpler infrastructure

And a couple downers:

- You redeploy to update data.  If your site doesn't change much, this might not matter to you.  Use automated build and deploy tools to make this less time-consuming.
- Mass updates are harder and less precise.  You are not writing sql update statements.  You're writing `sed` statements for string find and replacement.
- You cannot join over data sets in any meaningful way.
- You lose the flexibility to code everything else that you might want a dynamic site for

## Export Your Postgres Data

If you want to make the conversion, get your data out of your database.  There are tools for this, like [pg_dump](https://www.postgresql.org/docs/9.1/static/backup-dump.html).  I'll use my data in postgres on heroku as an example:

```
heroku pg:backups:capture            # export binary dump of database
heroku pg:backups:download           # get the dump onto your computer
pg_restore -f latest.sql latest.dump # convert to plaintext .sql file
```

Now cleanup your data in the .sql file.  Get rid of all that table, key, and sequence creation in the .sql file that won't matter in this new world.  The dumps use the postgres `copy` command.  Pare down to one data collection per file, just the rows of data for the `copy` command.  For me, a couple of those rows might look like:

```
25	The Innovator’s Dilemma	The simultaneous need and danger to reinvent oneself and one's business is the dilemma.  Do we disrupt ourselves or do we double-down on core competencies?  A dilemma indeed!	https://i.imgur.com/exW4j8F.jpg	2014-08-20		2014-11-28 19:05:28.153598	2014-11-28 19:05:28.153598	Craig Christensen	http://www.amazon.com/gp/product/B00E257S86/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00E257S86&linkCode=as2&tag=jaktre-20&linkId=5CDH2BHJK7TZT52R
26	Steve Jobs	Jobs is revealed as a man of many passions.  Reading the formative moments of the personal computing technology and industry is very interesting!  However, the coarse language in the book causes me to not recommend it.	https://i.imgur.com/hpwhvgx.jpg	2014-09-10		2014-11-28 19:08:29.530743	2014-11-28 19:08:29.530743	Walter Isaacson	http://www.amazon.com/gp/product/B004W2UBYW/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B004W2UBYW&linkCode=as2&tag=jaktre-20&linkId=KPZUBQVLGURNA3RW
```

## Create the Markdown Files

If you have a lot of data or just a little or just like writing code, you might want to automate this.  I chose to write my script in javascript.  Here it is (or this [gist](https://gist.github.com/jaketrent/87d0695ab865d4a739be59434261bbd0)) -- helper functions are at the top; actual steps are shown at the bottom.  If you read the steps at the bottom, you should see rather quickly what this script is meant to do:

```js
const fs = require('fs')

const separateColumns = line => line.split('\t')
const nameColumns = line => ({
  id: line[0],
  title: line[1],
  description: line[2],
  coverUrl: line[3],
  completionDate: line[4],
  reviewUrl: line[5],
  createdAt: line[6],
  updatedAt: line[7],
  author: line[8],
  affiliateUrl: line[9]
})
const nonEmptyColumns = line => line.id !== ''
const deriveColumns = line => Object.assign({}, line, {
  descriptionSummary: line.description.substr(0, 150),
  fileName: __dirname + '/books/' + line.title.split(' ').join('-').toLowerCase() + '.md',
  reviewUrl: line.reviewUrl.replace('http', 'https')
})
const deriveFrontMatter = line => Object.assign({}, line, {
    fileContents: `---
affiliateUrl: "${line.affiliateUrl}"
author: "${line.author}"
date: "${line.completionDate}"
categories:
  - "Book"
comments: true
description: "${line.descriptionSummary}"
draft: false
metaKeywords: ""
layout: "book"
image: "${line.coverUrl}"
reviewUrl: "${line.reviewUrl}"
title: "${line.title}"
---

${line.description}
`})
const writeFiles = line => fs.writeFileSync(line.fileName, line.fileContents)

// the steps!
const file = fs.readFileSync(__dirname + '/latest.sql', 'utf8')
const lines = file
  .split('\n')
  .map(separateColumns)
  .map(nameColumns)
  .filter(nonEmptyColumns)
  .map(deriveColumns)
  .map(deriveFrontMatter)
  .forEach(writeFiles)
```

What do you think about this idea?  Would it be a good idea for any of your projects to push your database into markdown files?  Would you have anything to gain by it?

You have a site with a cms and a database.  Would it work as a static site based on markdown?  Maybe.

Convert a Database to Markdown

Continuous delivery allows for you to release often what you are always developing.  This is even possible on a static site where we might not be hiding in-development features but content and visual styles.  Here is a strategy for doing that, executing in Hugo.



## Feature Toggles

Main line development is a joy.  It's often good to avoid the long-lived, meandering paths of feature branches.  In order to stay in the master branch, you need to keep it always ready to deploy.  So anything that's half-baked needs to be committable but not impact the main experience. 

Feature flags generally are able to be turned on/off for all or some users.  Since we don't distinguish between users in a static site via login and since it's static, with all users getting the same content, our feature flag will be on/off for all users at once.

We will be able to turn the flag on when in our development environment.  When we build for our prod environment, we turn it off.  To enable the static content once the feature is totally complete, either toggle on the feature or remove the toggle checks.

## Hugo Environment Variables

In general, environment variables are one of the low-cost ways of implementing feature toggles.  The environment config can be updated without a deploy and feature toggles can go on/off quickly.

In this specific example, we'll look at how we can do this in Hugo.  First, we'll name our flag `FEATURE_BOOK` since we're working on a book-related feature.  Then we'll connect to environment variables using the `getenv` template function:

```
{{ if getenv "FEATURE_BOOK" }}
  Book markup here
{{ end }}
```

And then to turn the feature on, just flip the flag in the environment when running the site:

```
FEATURE_BOOK=true hugo server
```

The book markup that you surrounded with the check will be visible.  Then when you build for prod, assuming the feature should be hidden, don't set the variable:

```
hugo
```

And the book markup will not be included.

## Turning Off Content

One of the biggest parts of a static site is usually the content.  In a site like Hugo, the content is mostly held in collections of markdown files.  There's usually a way, in these kinds of blogging platforms, to mark content as not ready for publishing.  In the case of Hugo, we put this in the markdown front matter:

```
---
draft: true
---
```

If we want to easily toggle all this content on in a dev environment, we can run the dev server with our normal environment variable and the `buildDrafts` flag:

```
FEATURE_BOOK=true hugo server --buildDrafts
```

## Separate Styles

In order to separate the css that we write, we need to be careful.  It would probably be wise to test both the feature toggled on and off states before we deploy, because CSS is notoriously global.  We can affect things we don't expect, even things not covered by our feature flags.

Since `getenv` and templating in general is not available in `.css` files, we might consider splitting out a stylesheet especially for our feature flag.  Then we can surround the import from our template:

```
{{ if getenv "FEATURE_BOOK" }}
  <link rel="stylesheet" href="/css/book-specific.css" />
{{ end }}
```

Have you felt empowered to deploy more freely with your use of feature toggles?  Are there any other continuous delivery devices that you use on your static sites?

A story of my migration from Octopress to Hugo static site generator

Continuous Delivery of a Static Site

Several weeks ago, I decided it was time for me to move on from Octopress.  It's given me a good run.  But there were a couple of causes that pushed me over the edge.  My experience has been good with a few pains in the conversion.



## Outgrowing Octopress

The most fundamental and important reason for me converting from Octopress is that it was getting slow to build my site.  I'd write an article, run `rake generate`, and it'd take (anecdotally) 45 seconds.  Wow, impatient, right?  Well it was apparently enough to make it less fun to write on as a platform.

Updating my site's layout on Octopress was also troublesome.  I had written my own theme, but I had worked my custom theme into the existing `sass` and template hierarchy that exists in Octopress' default theme.  These files are a tangle and hard to work with.

Octopress has made rumblings about being updated and improved upon.  But these efforts have been silent as of late.  The last commits on Jekyll's Ferrari (v3), as of this writing, were a full year ago.

Finally, I usually welcome an opportunity to learn something new.  I was on the hunt for a platform that would enccourage my writing.

## Tried Hexo

I looked at Hexo.  It looked rather promising.  It's based on Node.  The docs were extensive.  Initial speed tests were pretty good.  It seems like a pretty cool project.  But then I pushed close to 300 articles into it, and it slowed down significantly -- about 30 seconds to build the site.  Then I read some github issues about how people were having similar problems.  Then I tried to extend it with different types of content beyond blog posts (a core use case for the next generation of my blog), and got lost in creating new Hexo generators.  So I kept looking.

## Fun with Hugo

Hugo was another static site generator that appeared high on a number of top 10 lists.  It is written in golang.  I picked it up.  I identified with a marketing slogan high on their list: "Make the web fun again."  Yes!  This is essentially how I wanted to feel when I thought about adding another post to my blog.

### Hugo is Fast

Initial tests with speed were mind blowing -- near instantaneous.  Then I added almost 300 articles.  The live server and blog generation were still quite fast (606ms):

```
278 regular pages created
210 other pages created
2 non-page files copied
287 paginator pages created
198 tags created
8 categories created
total in 606 ms
```

This is a fresh build.  The caching, however, makes it even faster.  Two builds in a row will yield a second build that's faster (barely, 596ms). 

I realized that part of the fast speed was because it wasn't doing code highlighting, which I wanted on.  I read through the docs at how to turn those on.  They're actually run in python by pygments, and the docs say they're slow.  Yes, the build time went way up (15s). But again, caching still helps significantly here (in a tests, anywhere from a total 800ms to 3s) -- and is a lot more important.  The dev cycle response time is enjoyably fast.

### Templates are Easy & Intuitive

As I have rebuilt my blog on Hugo, I've been pleasantly surprised at how easy things have been to figure out and implement.  Templates are written in go templates, which are pretty nice.  There are good built-in parameters for the site and each page available in the templates.

Creating arbitrary types of content is super easy with its cli tool (via `hugo new`) that doesn't really assume anything and doesn't try to be too clever.  I can easily start new "sections" of content.

Most things are driven by a template.  They go in intuitive locations and are easy to override -- for different sections, the sitemap, and rss feeds.  There's good template fallbacks with the `_default` section.  I can find simple instructions for desired features in the docs.

### Things I Wished Hugo Had

The theme install instructions are odd.  I suppose if you want a plug n play theme, they're fine.  But I have just put my custom theme straight into my blog git repo, not using a git submodule as instructed.  

There are a few features that I miss from Octopress, including the ability to give code snippets file names.  That's apparently not generally-accepted markdown, but it's super nice.  I could write a shortcode, equivalent to Jekyll's plugins, but I'm trying to keep my Markdown vanilla parseable.  It also seems that Jekyll was more forgiving of my sloppy markdown formats.

## Data Conversion

In order to make the migration to Hugo from Octopress, my source content markdown files needed some changing.  Lots of this was in the front matter.  There were a few issues in the body of the content as well where I was using things that Octopress supported but that Hugo did not.

### Date Format

The publish date in the front matter for Octopress was in the format of:

```
date: 2015-02-28 16:20
```

Hugo wanted it in the format of:

```
date: "2015-02-28"
```

I changed this with a sed script:

```
find ./ -type f -exec sed -i '' -E "s/(date: )([^ ]+).*$/\1\"\2\"/" {} \;
```

### Split Categories

As far as I know, Octopress didn't support categories.  So I mixed my categories in as tags for my posts.  Hugo can treat them as separate things, so I split out what looked like this:

```
from: categories: [Code, heroku, dns]
```

Into this:

```
categories: 
  - "Code"
tags: [heroku, dns]
```

This is partial step, not the final format.  We'll finish it soon. 

To get categories onto a new line, it turned out that I needed a new version of `sed`.  Instead of the BSD version that's on the Mac by default, I had to install the GNU version:

```
brew install gnu-sed
```

This is apparently because the BSD `sed` can't read newline charaters `\n` in a regex.  It needs the literal new line (as in, press enter).  That seems like a pain for a nice, copy-pasteable regex.  So, in this command, I used `gsed`:

```
find . -iname "*.md" -exec gsed -i -E "s/categories: \[(\w+)(, ?)(.*)$/categories:\n  - \"\1\"\ntags: \[\3/" {} \;
```

### Format Tags

The remaining comma-separated array of tags that I was using in Octopress land wasn't apparently real yaml at all.  Anyway, Hugo didn't like it.  So, I was going to make a bulleted list.  To go from this:

```
categories: [heroku, dns]
```

To this:

```
tags:
- heroku
- dns
```

I needed to run a couple `sed` commands in a row (because I couldn't figure out how to do this with a single command).  The first command is for the posts that have one tag, then 2 tags, and so on.  I'll show you the first two.  You can extrapolate from there:

```
find . -iname "*.md" -exec gsed -i -E "s/tags: \[([a-z-]+)]$/tags:\n  - \"\1\"/" {} \;
find . -iname "*.md" -exec gsed -i -E "s/tags: \[([a-z-]+)(, ?)([a-z-]+)]$/tags:\n  - \"\1\"\n  - \"\3\"/" {} \;
```

### Moving Cover Image Into Front Matter

For a couple years, I've put a cover image in the body of the post text, right at the end of the summary section.  I wanted to move this into the front matter.  It would give me a little more visual control.  I tried many, many `grep`, `sed`, and even `perl` commands to try to make this happen.  In the end, I went through articles manually.  That turned out to be a blessing in disguise (at least I tried to squeeze lemonade out of it), and I could make other adjustments to data, where my style has changed in various ways over the years.

I found files with images using this `grep` command:

```
grep -o -c "^\!\[.*\]$.*$" * | awk -F: '{if ($2 > 0){print $1}}'
```

### Remove Code Snippet Filenames

One of the features I really liked from Octopress was the out-of-the-box support for showing file names on code snippets blocks (the ones that use 3 backticks surrounding the block). Apparently, however, this is not standard Markdown trickery, or at least Hugo doesn't like it and stops all formatting if it detects it.  It would be much nicer if it just ignored it.

Again, I was stifled in my automated solution to this.  I spent a bit of time on this, but since it affected a relatively small amount of files, I learned my lesson from the pain of time spent on the cover image transform and just went for the manual editing. I think the problem had to do with the backticks not being well-supported (or at least understood by me) in the `sed` replacement command.  So I just found the files like this:

```
grep "^$\`\{3\}[a-z]\+$ [a-zA-Z_\-\.\/\-]\+" *.md
```

### Strict Hash Separation

In markdown, you make header text using the leading hashes.  Apparently, I had hashes that butted right up next to the header text in some cases.  Octopress dealt with that ok, but this new parser used by Hugo didn't like it and would render hash literals.  To make the strict separation, I ran:

```
find . -iname "*.md" -exec gsed -i -E "s/^(#+)(\w+)/\1 \2/" {} \; 
```

In all, the Hugo conversion has led to a fast development cycle of a fast static site.  It has brought some of the blogging fun back.  I did a visual redesign at the same time, and that's been fun too.  If it still looks like a painterly celestial scene when you read this, let me know what you think of it.  

Are you using Hugo?  What do you think of it so far?