Gatsby and Asciidoc

For this blog I’ve been using the Hubpress.io — nice blog theme, online wysiwyg asciidoc editing with github automatic deployment. Unfortunately it has not developed for some time already and I wanted to switch to something. It was Gatsby that caught my interest. I’m not a front-end developer but I wanted to check what’s about that React and JAM stack thing.

I started with searching the Gatsby themes and looking for a blog. I wanted one that supports Asciidoc but 99.9% of themes work only with the Markdown. The rest 0.1% are rather showcases than themes. I decided to go for Gatsby Starter Lumen and make adjustments by hand.

Learning Gatsby

First I needed understand the Gatsby project a bit. After reading few getting started posts I found the best was to go through was this tutorial How to Build Your Coding Blog From Scratch Using Gatsby and MDX. It’s a little older and not all details work perfectly but it is a really great way to understand the structure. It elaborates on details of content processing, graphql queries, styling and others things.

Pretty soon I understood that Gatsby works with a set of plugins where one of them manages the md files. The tutorial uses the gatsby-plugin-mdx. While the Gatsby Starter Lumen works with with gatsby-transformer-remark. I.e., there are different Markdown plugins of the same purpose in the ecosystem.

The "good news" is that there is only one Asciidoc plugin available as gatsby-transformer-asciidoc. The fine thing is that the gatsby-transformer-asciidoc is pretty similar to gatsby-transformer-remark used by Lumen Starter project.

While working on the tutorial I aimed to get working .adoc transformations in parallel with .mdx ones.

The repository that contains my work is at https://github.com/ochaloup/gatsby-from-scratch.

Note	I haven’t finished whole tutorial as I wrapped it up in middle of the text when MDX plugin stopped working for me. Later I found it’s because of some Gatsby changes.

Gatsby structure

Let’s briefly recap about the Gatsby project structure.

A JavaScript project

As a non-javascript developer I want to start with few things around the JS projects in general.

dependencies are defined in package.json
JS world uses many package managers - default npm, better yarn, newer a and faster pnpm (I chose yarn)
yarn install downloads js dependencies and places them under $PWD/node_modules
when dependencies are installed a lock files (yarn.lock, package-lock.json) are created. They ensure the same versions of the libraries are used when building subsequently on other environments. They are recommended to be pushed to github.
When working with TypeScript the configuration file is tsconfig.json. It configures how the transpiler built the final js file.
package.json contains the section "scripts" that defines automation of different commands. Any defined script command may be executed with npm run <command> or yarn <command>.

A Gatsby project

Now a bit about the Gatsby itself.

The good way is to click through the structure of the Lumen Starter directory structure or the tutorial project. It may not contain all but it sheds light on the parts.

Illustrative image of art deco Gatsby font

The configuration file is gatsby-config.[js,ts]. It defines global properties and configures gatsby plugins.
Any plugin needs to be declared as a dependency in package.json as well.
The configuration file gatsby-node.js defines hooks that are executed on startup of nodejs. It’s a way to set up dynamic pages where we want to list a set of options, like categories or tags are.
The gatsby-browser.js sets up the behavior for browsers (e.g. SCSS files).
The content/config.js is used by Lumen theme to get some basic config of the site (e.g., title).

Besides the configuration files there is the directory structure (details at the Gatsby documentation).

The Gatsby is a React application thus the directory structure consists of Gatsby specific parts and the directories comming from React.

public/ is not to be touched. it’s automatically generated and it contains output of the build process (part of .gitignore)
static/ contains data that are copied to public/ as they are. used for placing image files etc.
content/ is not a standard Gatsby directory but it’s usually used by the plugins (md, adoc). it’s configured at gatsby-config.ts
src/ directory contains code
- src/pages/ contains pages that’s are generated directly at the path (e.g., it’s a place for index.js)
- src/templates templates used in programatically build pages
- src/components react directory, contains react components that can be used within templates
- src/hooks/ react directory, hooks that can be invoked from components retrieving data, …
- src/utils/ react directory, utilities
- src/types/ react directory, for typescript types definitions

The gatsby provides command line tool gatsby to generate the content (gatsby build) and to help with development (gatsby develop). If it’s installed with yarn we could have configured the script that works from node_modules.

On starting with Lumen Starter

# getting the code to be adjusted
git clone https://github.com/alxshelepenok/gatsby-starter-lumen
cd gatsby-starter-lumen
# installing dependencies under node_modules
yarn install
# starting developer mode
yarn start

The Gatsby Lumen Starter theme would be started at localhost in dev mode. The site can be accessed at http://localhost:8000 (graphql at http://localhost:8000/___graphql).

React GraphQL and Gatsby

All data that Gatsby works with comes through the GraphQL processing (i.e., React). The Markdown and Asciidoc plugins process the documents and the results are consumed through GraphQL queries. The components of the data are for example title, tags, generated HTML etc. The React component may then use that at particular places of the page.

The most of the work for met on moving from Markdown to Asciidoc was the remapping of the Markdown GraphQL queries for Asciidoc.

My work was to work with adoc and md data (MDX plugin is used) side by side. With that there could be seen the differences in the query structure. The gatsby-transformer-remark works with a similar structure as the mdx plugin. You can see this example here https://github.com/ochaloup/gatsby-from-scratch/blob/main/hello-world/src/pages/index.js#L40

query SITE_INDEX_QUERY {
  allMdx(
    sort: { fields: [frontmatter___date], order: DESC }
    filter: { frontmatter: { published: { eq: true } } }
  ) {
    nodes {
      id
      excerpt(pruneLength: 250)
      frontmatter {
        title
        date
      }
      fields {
        slug
      }
    }
  }
  allAsciidoc(
    sort: { fields: [revision___date], order: DESC }
    filter: { pageAttributes: { published: { eq: "true" } } }
  ) {
    nodes {
      id
      document {
        title
      }
      pageAttributes {
        synopsis
      }
      revision {
        date
      }
      fields {
        slug
      }
    }
  }
}

That query could be investigated in the GraphQL console (http://localhost:8000/___graphql) as running in dev mode. Both plugins place data into { nodes } where the node represents one md/adoc document.

This query example works with page metadata mainly. You can see the difference of how Asciidoc works with a pre-defined metadata structure while Markdown has not get any predefined structure just key-value placed under frontmatter part. On top the MDX plugin offers excerp which is a plain (not formatted) text. For Asciidoc I needed to work with special metadata placed in the document.

The example md document is below.

---
title: Hello World - from mdx!
date: 2022-10-01
published: true
category: "Markdown Pro"
tags:
  - "One"
---

# h1 Heading

My first post!!

The result data of the GraphQL query for the document returns

{
  "data": {
    "allMdx": {
      "nodes": [
        {
          "id": "e7b45c55-eed2-5ed4-a59d-68ae03e208cf",
          "excerpt": "My first post!!",
          "frontmatter": {
            "title": "Hello World - from mdx!",
            "date": "2022-10-01T00:00:00.000Z"
          },
          "fields": {
            "slug": "/2022/2022-10-01-first-post/"
          }
        }
      ]
    }
  }
}

The example adoc document is below.

= Hello from Asciidoc!!!
chalda <ondrej.chaloupka@proton.me>
1.0, 2022-10-08

:page-published: true
:page-synopsis: Something about my friends
:page-title: Article
:page-path: /2022/2022-10-08-a-test
:page-category: Asciidoc
:page-tags: One, Two, Three

How does it work? Good?

The GraphQL response for the document is

{
  "data": {
    "allAsciidoc": {
      "nodes": [
        {
          "id": "0d92c3b4-3549-5c8b-9dec-6dbbf98bec11",
          "document": {
            "title": "Hello from Asciidoc!!!"
          },
          "pageAttributes": {
            "synopsis": "Something about my friends"
          },
          "revision": {
            "date": "2022-10-08"
          },
          "fields": {
            "slug": "/2022/2022-10-08-a-test/"
          }
        }
      ]
    }
  }
}

Gatsby Lumen Starter changes for Asciidoc to work

Markdown to Asciidoc: Plugins

The plugins do the hard work of transforming the md/adoc document to HTML format.

I changed the Markdown plugin gatsby-transformer-remark for Asciidoc plugin gatsby-transformer-asciidoc.

That change requires to delete other Markdown/remark plugins from configuration as gatsby-remark-images (better image handling), gatsby-remark-responsive-iframe (iframe wrapped to responsive elastic container), gatsby-remark-autolink-headers (github style links hover effect), gatsby-remark-prismjs (code block syntax highlighting), gatsby-remark-copy-linked-files (copying local files linked in md to the public/ directory), gatsby-remark-smartypants` (Replaces "dumb" punctuation marks), gatsby-remark-external-links (adds the target and rel attributes to external links).

With using Asciidoc I needed to abandon that functionality (or add it differently when needed). That’s a pity but I just wanted the asciidoc syntax a lot!

Markdown to Asciidoc: GraphQL

The next step was to change all the GraphQL queries from Markdown structure to Asciidoc one. The GraphQL queries are quite similar while a bit different in detail.

The GraphQL queries are spread all over the code so it was quite annoying. The good thing was there are tests (yarn test) which helped me to understand what I forgot to cover. An example of such difference of the query is covered above or you can check the github links here:

md remark plugin : https://github.com/alxshelepenok/gatsby-starter-lumen/blob/fa2bac05139875408fe9f36bba59289ada3d3d6e/src/hooks/use-categories-list.ts#L12
adoc asciidoc plugin : https://github.com/ochaloup/blog.chalda.cz/blob/efdec3c4e5b84a9ed0ff35c4f6c72b9ca4e5e242/src/hooks/use-categories-list.ts#L12

Then there was still the trouble with metadata format. The Markdown may work with "a list" while the Asciidoc works only with strings. Check how tags are defined in md like

tags:
  - "Handwriting"
  - "Learning to write"

The tags in Asciidoc is like

:page-tags: Handwriting, Learning to write

While in md is fine to do just aggregate the list with the graphql query to get "the tag name" and "the number of occurrences"

...
group(field: frontmatter___tags) {
  fieldValue
  totalCount
}
...

Thus for asciidoc it’s needed to get all data, parse it and group it in typescript afterwards. See my way here:

get all data: https://github.com/ochaloup/blog.chalda.cz/blob/efdec3c4e5b84a9ed0ff35c4f6c72b9ca4e5e242/src/hooks/use-tags-list.ts#L14
parse and group to get totalCount: https://github.com/ochaloup/blog.chalda.cz/blob/efdec3c4e5b84a9ed0ff35c4f6c72b9ca4e5e242/src/utils/group-tags.ts#L11

Markdown to Asciidoc: Image paths

The remark consists of plugin gatsby-remark-copy-linked-files that checks what are files used in md file and then it copies them to the public/ directory where the static generated result page is placed. If you check the Starter Lumen it places pictures within the folder of the article like content/posts/<post-name>/media. The same directory structure is copied to public/.

That’s not the case for asciidoc as there is no such plugin. I didn’t want to create one and I just decided to place the pictures (and other) data to static/ folder that’s automatically placed to public/ by Gatsby. The gatsby-transformer-asciidoc can be configured to add a prefix to any image path with imagesdir. The images are placed under static/images/ but the asciidoc refers it only as image::articles/some-image.png when linked from the adoc file.

Markdown to Asciidoc: Syntax highlighting

One of the things I really want to get working is syntax highlighting. The plugin gatsby-remark-prismjs was removed and I needed to add that functionality somehow manually.

I did it with direct use of the highlight.js library. I needed to create utility that runs the hljs.highlightAll() that’s in highlightCode.ts. The hightlight.js was added to package.json and then the utility function is called from the template of posts.

Markdown to Asciidoc: CSS Style changes

This part is not finished and not examined well. I needed to adjust styling that the default one does not work nicely with HTML generated by Asciidoc transformer.

The base was adding some asciidoctor styles that I borrowed from github of asciidoctor-stylesheet-factory. I integrated it within the sass rules of the Lumen at src/assets/scss.

The other part was that I used CSS from the Hubpress.io Ghostium theme asciidoctor-default.css.

Combined together while commenting out some parts of css at components it gets working more or less good.

Gatsby blog deploy to Github pages

I used my prior article about the DNS setup of Github pages. Then now the deploy is done via Github actions that’s configured at Github under Settings → Pages

When setup first there is automatically provided a deploy script by Github and that could be later adjusted. When confirmed in Github the actions are saved into a yaml file at .github/workflows folder. You can check configuration, with slight change, of my tutorial project at .github/workflows/pages.yaml.

Summary

And that was all. The most time I spent about start understanding the Gatsby project structure, then changing the GraphQL and then fixing CSS issues.