Rendering code blocks with gatsby, mdx, and prismjs

Intro

This website is written using GatsbyJs. Technology behind gatsby is ReactJs. I am quite new to both, which isn’t surprising since I’m mainly a backend engineer. As such, it took me quite a while to sort things out, and I wanted to share some gotchas.

I started off with a gatsby-starter-blog preset which is using gatsby-remark-prismjs as a plugin to gatsby-transformer-remark.

This means gatsby-transformer-remark will transform blog posts written in markdown to HTML while during the transformation the gatsby-remark-prismjs plugin will take care of the code blocks.

The problem is very soon markdown feels too restricting. If you ever used something like notion or confluence, or even wordpress, you know it’s super easy to use beautiful callouts, different styles of quotations, different colors and backgrounds for text, etc. Markdown does not have special syntax for those.

After a little googling I stumbled upon abandoned gatsby-remark-custom-blocks which has a dependency on an equally as abandoned gatsby-remark-blocks and ruled it out as a possible solution.

Luckily a better solution was just around the corner. I mean, in the gatsby plugins library:

MDX = MD + JSX

MDX allows JSX embeds in Markdown files. Very elegant and powerful solution. Just an example of what it might look like:

## Using mdx

Head over to [mdxjs website](https://mdxjs.com/) for comprehensive intro.

<Callout icon={warn} bgColor="yellow">If you are using mdx v1, please make sure to upgrade to v2, it's 30% faster!</Callout>

<WrappedQuote>
    > To write and enjoy MDX, you should be familiar with both markdown and JavaScript (specifically JSX)
    <QuoteSource>[official mdx documentation](https://mdxjs.com/docs/what-is-mdx/)</QuoteSource>
<WrappedQuote>

I haven’t tried it yet, but it looks like it means I can use any antd component (or material UI, or chakra-ui, or any other react component library) inside my .md(x) files. What more to dream about? SOLD!

Caveats using with gatsby and gatsby-remark-prismjs

I’ve create a mdx branch in this website’s git repo and very quickly carved out gatsby-transformer-remark. Thankfully, gatsby has an awesome blog post on how to migrate from remark to mdx.

BUT. If there wasn’t this but…

I noticed my inline code blocks are screwed. Specifically, they are wrapped in divs, and as such they aren’t very “inline” anymore. Apparently, when used with gatsby-remark-prismjs, new version of mdx plugin (the one that is using mdx 2) will grab the results and visit all the html nodes (all the nodes that aren’t recognized as JSX components) and wrap them in a div. I will not go into whys, but if you are curious, it’s right there - in the source code of the plugin.

I didn’t see a non-hacky way to circumvent this. I could’ve gone with a hacky one, but I just don’t like hacky ways. I’d rather find a proper solution once and wouldn’t have to maintain hacks.

Proper solution definition

Alright, alright, alright. So I’m not askin for much. All I want is:

• Inline code highlighting
• Several languages support (python, php, go, js, css, xml, etc)
• Ability to highlight specific lines (using range notion like {1,2-10,33-35})

Sounds easy? Let’s dig into this.

Migrating from gatsby-transformer-remark to gatsby-plugin-mdx

This part is pretty straightforward and described quite well in the Migrating Remark to MDX post on gatsbyjs.com. One caveat which is NOT described there is a bug in gatsby that is preventing named GraphQL queries in templates. The issues is described here and at the moment of writing (22st of Sep, 2022) there is an open PR to fix this issue.

The issue, if exists on your site, will manifest itself with an error page which will say something like this:

Multiple "root" queries found: "PostById" and "PostById".
Only the first ("PostById") will be registered.

In case it happens, go to that component, and simply remove the name of the query, i.e.

export const pageQuery = graphql`
- query PostById (
+ query (
    $id: String!
    $previousPostId: String

Migrating from gatsby-remark-prismjs to prism-react-renderer

Once remark to MDX migration was done, I noticed all my inline codes are wrapped in divs. I might solve it by intervening into mdx-to-html compilation process, but then I’ve found a solution that would satisfy me more. And the solution was to use prism-react-renderer. The idea is to prodive <code> and <pre> components into an mdx document. Of course, we will be in cotrol of how those are rendered. Inline code will be compiled into html as <code>inline code</code>, whereas multiline code blocks will be more complex, and that’s where the prism-react-renderer will come into play.

How do we render multiline blocks with prism-react-renderer?

We need to be able to compile something like this

# Function for nth Fibonacci number
def fib(n):
    """ @TODO nahdle n < 0 """
    if n == 0:
        return 0
    elif n == 1 or n == 2:
        return 1
    else:
        return fib(n-1) + fib(n-2)

to something like this

# Function for nth Fibonacci number
def fib(n):
    """ @TODO nahdle n < 0 """
    if n == 0:
        return 0
    elif n == 1 or n == 2:
        return 1
    else:
        return fib(n-1) + fib(n-2)

There are many articles demonstrating how to use prism-react-renderer available online. I’ve been following this one by Paul Scanlon (if you are not using tailwind.css, completely misregard it). What are we interested in is the PrismSyntaxHighlight component, which I simply copied over. One thing that I’ve done was removing theme (I feel a little more comfortable with defining colors in my src/styles.css).

import React from 'react';
import { Highlight } from 'prism-react-renderer';

const PrismSyntaxHighlight = ({ children, className }) => {
  const language = className.replace(/language-/gm, '');

  return (
    <Highlight code={children} language={language}>
      {({ className, tokens, getLineProps, getTokenProps }) => {
        return (<code className={className}>
          {tokens.slice(0, -1).map((line, i) => {
            const lineProps = getLineProps({ line, key: i })
            delete lineProps.style
            return (
              <div key={i} {...lineProps}>
                {line.map((token, key) => {
                  const tokenProps = getTokenProps({ token, key })
                  delete tokenProps.style
                  return (
                    <span {...tokenProps} />
                  )
                })}
              </div>
            )
          })}
        </code>)
      }}
    </Highlight>
  );
};

export default PrismSyntaxHighlight;

import React from 'react';
- import Highlight, { defaultProps } from 'prism-react-renderer';
+ import { Highlight } from 'prism-react-renderer';

// the rest of the code remains the same

export default PrismSyntaxHighlight;

npm i prism-react-renderer prismjs @mdx-js/react gatsby-plugin-mdx

Now what’s been achieved is our block renders to an HTML which is looking something like this:

<code class="prism-code language-python">
    <div class="token-line">
        <span class="token comment"># Function for nth Fibonacci number</span>
        <span class="token plain"></span>
    </div>
    <div class="token-line">
        <span class="token plain"></span>
        <span class="token keyword">def</span>
        <span class="token plain"></span>
        <span class="token function">fib</span>
        <span class="token punctuation">(</span>
        <span class="token plain">n</span>
        <span class="token punctuation">)</span>
        <span class="token punctuation">:</span>
        <span class="token plain"></span>
    </div>

    <!--  some more html... -->

</code>

As I said, I’ve removed theme that come with prism-react-renderer, I’m preferring to style the code in the src/style.css:

.token.tag,
.token.attr-name,
.token.namespace,
.token.deleted {
  color: var(--code-color-fg);
}

.token.function-name {
  color: var(--code-color-fg);
}

.token.function {
  color: var(--code-color-blue);
}

This method seems more straightforward to me and gives me fine control over how exactly I want the code to appear on the website.

“Inject” PrismSyntaxHighlight into MDX

This is the easiest part. Just provide <code> and/or <pre> components into the mdx document via the MDXProvider. Here is a snippet from my src/templates/post.js

import { MDXProvider } from '@mdx-js/react';
import PrismSyntaxHighlight from "../components/highlight"

const components = {
  pre: ({children, className}) => { return (<div className="code-container"><pre>{children}</pre></div>) },
  code: ({children, className}) => {
    return className ? (
      <PrismSyntaxHighlight className={className}>{children}</PrismSyntaxHighlight>
    ) : <code className="language-text">{children}</code>
  }
}

// ... JSX:

<section itemProp="articleBody">
    <MDXProvider components={components}>{children}</MDXProvider>
</section>

This takes case of both multiline code blocks and inline codes.

Languages support

Prism supports many languages, but we need to import them explicitly. This is what I’ve for in my gatsby-browser.js:

import { Prism } from "prism-react-renderer";

(typeof global !== "undefined" ? global : window).Prism = Prism

// First import prism-markup-templating because otherwise it breaks php which depends on it.
// https://github.com/PrismJS/prism/issues/2769
await import('prismjs/components/prism-markup-templating')
await import('prismjs/components/prism-go')
await import('prismjs/components/prism-php')
await import('prismjs/components/prism-python')
// etc.

- import Prism from 'prism-react-renderer/prism'
+ import { Prism } from 'prism-react-renderer'

(typeof global !== 'undefined' ? global : window).Prism = Prism
// First import prism-markup-templating because otherwise it breaks php which depends on it.
// https://github.com/PrismJS/prism/issues/2769
+ await import('prismjs/components/prism-markup-templating')
- require('prismjs/components/prism-go')
- require('prismjs/components/prism-php')
- require('prismjs/components/prism-python')
- require('prismjs/components/prism-markdown')
- require('prismjs/components/prism-diff')
- require('prismjs/components/prism-css')
+ await import('prismjs/components/prism-go')
+ await import('prismjs/components/prism-php')
+ await import('prismjs/components/prism-python')
+ await import('prismjs/components/prism-markdown')
+ await import('prismjs/components/prism-diff')
+ await import('prismjs/components/prism-css')

Highlighting specific lines

gatsby-remark-prismjs has a support for highlighting specific lines of code with a special syntax that specifies those lines as a range which looks something like this python{1,2,20-22}. This syntax means that lines number 1, 2, 20, 21, and 22 should be highlighted.

It doesn’t work with the above componenent, what gets rendered instead is something like this:

<code class="prism-code language-python{1,2,20-22}">

So, apparently I need to take the range out of the language name, interpret it and highlight those specific rows myself. Quite easy with parse-numeric-range which is a js package that is capable of interpreting that exact range format. The plan is simple now:

• take out the range definition from the classname
• use parse-numeric-range to parse it
• add a highlight-line class to the highlighted lines of code

I’ll get straight into the soltuion and add comments to the code so that if you are interested, you can get a holistic view

import React from 'react';
import rangeParser from 'parse-numeric-range';
import Highlight, { defaultProps } from 'prism-react-renderer';

const calculateLinesToHighlight = (meta) => {
  const RE = /{([\d,-]+)}/
  if (RE.test(meta)) {
    const strlineNumbers = RE.exec(meta)[1]
    const lineNumbers = rangeParser(strlineNumbers)
    return (index) => (lineNumbers.includes(index + 1))
  } else {
    return () => false
  }
}

const PrismSyntaxHighlight = ({ children, className }) => {
  var language = className.replace(/language-/gm, '');
  const highlightedLinesMatch = language.match(/{[^}]+}/)
  let highlightLinesDefinition = ''
  if (highlightedLinesMatch !== null) {
    highlightLinesDefinition = highlightedLinesMatch[0]
    language = language.substr(0, highlightedLinesMatch.index)
  }
  const shouldHighlightLine = calculateLinesToHighlight(highlightLinesDefinition)

  return (
    <Highlight {...defaultProps} code={children} language={language}>
      {({ className, tokens, getLineProps, getTokenProps }) => {
        return (<code className={className}>
          {tokens.slice(0, -1).map((line, i) => {
            const lineProps = getLineProps({ line, key: i })
            if (shouldHighlightLine(i)) {
              lineProps.className = `${lineProps.className} highlight-line`
            }
            delete lineProps.style
            return (
              <div key={i} {...lineProps}>
                {line.map((token, key) => {
                  const tokenProps = getTokenProps({ token, key })
                  delete tokenProps.style
                  return (
                    <span {...tokenProps} />
                  )
                })}
              </div>
            )
          })}
        </code>)
      }}
    </Highlight>
  );
};

The calculateLinesToHighlight method has been taken from Prince Wilson’s Add line highlighting to prism-react-renderer blog post.

Now all we need is to add css styles for the highlighted lines. This is what I’ve got for it in src/styles.css:

code.prism-code .token-line.highlight-line {
  background-color: var(--code-color-line);
  display: block;
  margin-right: -1em;
  margin-left: -1em;
  padding-right: 1em;
  padding-left: 0.75em;
  border-left: 0.25em solid var(--code-color-bg);
  border-right: 0.25em solid var(--code-color-bg);
}

Conclusion

• MDX is great, I definitely enjoy how I can add anything into my markdown files, and I highly recommend moving away from just markdown on your gatsby sites
• prism-react-renderer gives a fine control over appearance of the code even though may look a little overwhelming at first (we all want “out-of-the-box” solutions, don’t we?)