
I spent some time this weekend updating a remark utility I wrote a while back. I wanted to add an 'overview' section to the beginning of every post -- basically a table of contents generated from post headers. There's a great little plugin that does this called [`remark-toc`](https://github.com/remarkjs/remark-toc), which uses `mdast-util-toc` to do just this. It did present a few problems for my workflow:

1. It requires you to define where the table of contents went in the markdown. For other folks, I'm sure this is a useful feature, but I didn't want to have to bother.
2. It removed all content between between the table of contents and the first header. Again, maybe not a problem for some folks, but I often have an introductory section that lacks a header.

Because of this, I ended up spinning up a custom implementation of `mdast-util-toc`.

### remark plugin

The [`unified`](https://github.com/unifiedjs/unified) ecosystem has a fairly robust language for describing itself. At it's core, '**unified** is an interface for processing text using syntax trees' (unified documentation). For our purposes, we can think of it this way:

- we have an `.mdx` or `.md` file
- it is run through a `parser` and converted into a syntax tree
- that syntax tree can be transformed by any number of plugins or `transformers`
- the new syntax tree is run through a `compiler` and output as html

We want to write a plugin that returns a `transformer` that modifies our syntax tree. A really basic version would look like this:

``` js
module.exports = parse;

function parse(options = {}) {
  return transformer;

  function transformer(tree) {
    // modify tree here
  }
}
```

We can pass this module to the `remarkPlugins` argument of `renderToString` to transform the tree.

### generating overview section

We'll update the above generic `parse` function to use `mdast-util-toc`, which does the heavy lifting of generating a list of headers with relevant links. `mdast-util-toc` returns a [node](https://github.com/syntax-tree/unist#node) with a `type` of `list` that we can pass to the `children` array of the syntax tree. If that `node` is returned, we'll update the `children` array in the syntax tree so the map comes first.

``` js
const tocUtil = require('mdast-util-toc');

module.exports = parse;

function parse(options = {}) {
  return transformer;

  function transformer(tree) {
    const { map } = tocUtil(tree);
    if (map) {
      tree.children = [].concat(map, tree.children)
    }
  }
}
```

This works, however, I want to generate a `div` with the class-name `toc` that includes both this table of contents as well as an `overview` heading. We'll need to generate this [`parent` node](https://github.com/syntax-tree/unist#parent) as well as the header node. I wrote up a really generic function for this:

``` js
const genOverview = map => {
  const header = {
    type: 'heading',
    depth: 3,
    children: [{ type: 'text', value: 'overview' }]
  };

  return {
    type: 'parent',
    children: [header, map],
    data: {
      hProperties: { className: 'toc' }
    }
  };
};
```

And update the `transformer` function:

``` js
const { map } = tocUtil(tree);
if (map) {
  const overview = genOverview(map);
  tree.children = [].concat(overview, tree.children);
}
```

While not the most exciting thing in the world, this should offer a brief overview for thinking about handling syntax tress. Looking forward to doing more in the future.

### resources

- [unified documentation](https://github.com/unifiedjs/unified)
- [remark plugin documentation](https://github.com/unifiedjs/unified#plugin)

### up next

- [ ] templating


generating a table of contents for markdown posts

generating theme tokens and falling back to defaults

generating theme tokens

remark generated table of contents


I'm working on a theme/token generating library based on [theme-ui](https://theme-ui.com/) and [emotion](https://emotion.sh/docs/introduction). I've found myself using some variation of the same theme object across projects, but as I iterate on that object either improving it or tweaking it, I find it increasingly difficult to keep different versions up to parity. An example: I update how I'm using breakpoints in one so I need to go through each project and update the breakpoints there.

One solution would be to publish the theme object as an npm package, but I wanted something a little more dynamic. [speculative fabulation (or sf)](https://github.com/inadeqtfuturs/if-sf) generates theme tokens (ex. colors, typography, buttons) as well as theme objects (ex. an object comprised of theme tokens). These can be passed into `emotion`'s or `styled-components`'s theme provider. `sf` operates on the idea that objects go in, objects come out -- you can extend, merge, or override theme defaults, allowing granular control over the theme you generate:

``` js
const defaultTheme = genTheme();
const defaultThemeWithNewColor = genTheme({ colors: { newColor: '#333' } });
```

Both functions return a full theme object, but you can drill down and update the specific tokens you need.

### token generators

`genTheme()` is broken down into a series of token generators -- functions that return a token object, like colors or styles for a button. In working on these token generators, I set up a few parameters for how they should behave:

0. generator files export a token object
1. generators return a token object
2. generators don't require arguments to return the token object
3. generators merge and extend the token object by default
4. generators can wholesale override the token object

For something like colors, this is simple enough:

``` js
import { merge } from 'theme-ui';

const defaultColors = {
  text: '#111827',
  background: '#F9FAFB',
  primary: '#2563EB',
  secondary: '#DE7283',
  accent: '#7C3AED',
  muted: '#88d1FF'
};

export function genColors({ override, ...restColors } = {}) {
  if (override) {
    return override;
  }
  if (!restColors) {
    return defaultColors;
  }
  return merge(defaultColors, restColors);
}

export default defaultColors;
```

By default, `genColors()` returns `defaultColors`. If we pass an `override` argument, it will return that object instead. Any other key/value pair we pass gets merged into default colors.

### token generators -- deep and defaults

Things became slightly more complicated as I began working with typography. In the past, I've defined a default text base that includes things like font size, line height, and font family, spreading that object into each typographical element (i.e. p, ul, blockquote, etc.) (I'm ultimately leveraging the `theme.styles` object from [theme-ui](https://theme-ui.com/styling-mdx)). This works for the first two generator requirements -- have a base set of styles; spread them into each element, return that:

``` js
export const defaultTextBase = {
  fontFamily: 'body',
  fontSize: ['sm', 'md'],
  lineHeight: 1.5,
  margin: 0,
  padding: 0,
  marginBottom: 4
};

const defaultText = {
  p: {
    ...defaultTextBase
  },
  ul: {
    ...defaultTextBase,
    paddingLeft: 8,
    ul: {
      margin: 0
    }
  },
  ...
}
```

The problem starts when I want to extend or merge styles into the base: if we're just spreading `defaultTextBase` into these elements, at what point do we actually merge or extend the base? One way to solve this would be to iterate through the keys in `defaultText`, spreading the base as we went, but this would prevent us from exporting an object from the generator file. Another way to handle this would be to get a diff between the new base and default base -- after all, we're just extending or merging into it. However, this presents an immediate problem if we're going to wholesale override the base at some point. So I ended up writing a utility that removes the default values (if present) from each element.

``` js
function removeDefaults(obj, defaultObj) {
  const defaultKeys = Object.keys(defaultObj);

  // mapObject does what it sounds like: allows us to map the key/value pairs in an object
  // mapObject = (obj, func) => Object.fromEntries(Object.entries(obj).map(func));
  return mapObject(obj, ([key, value]) => {
    const remainingValues = Object.keys(value).reduce((acc, curr) => {
      if (defaultKeys.includes(curr) && defaultObj[curr] === value[curr]) {
        return { ...acc };
      }
      return { [curr]: value[curr], ...acc };
    }, {});
    return [
      key,
      {
        ...remainingValues
      }
    ];
  });
}
```

Once the default values are removed, we can determine the new base (overriding, merging into, or extending the default base) and use the above `mapObject` function to pass that base into each key. You can see the full implementation [here](https://github.com/inadeqtfuturs/if-sf/blob/main/src/theme/tokens/text/index.js)

### spread defaults // remove defaults

Something about this pattern seems useful, though I'm not entirely sure where. It goes something like this:

1. Have a default base
2. Spread that default base into each key of a token object. Export that object.
3. If you need to extend or override that base, use the above function to remove those defaults
4. Return new object using new base

I like this pattern, though I think it fits a specific use case. I do think it could be used in a more generalized way, and I'd be interested to hear alternative approaches to the above problem.

### up next

- [ ] templating



In the next iteration of garden, I want to implement different content directories to field/contain different kinds of content. As I've written about [previously](/garden/rethinking-digital-gardening), I'm not thinking about content in a hierarchy, but marking a different kind of speed (reading and writing) when thinking. At a very high level, I want to be able to write notes (very short snippets or thoughts), posts (slightly longer thoughts, much like I'm doing now), and have projects (longer/larger and perhaps more polished pieces). Eventually, this schematization could be configurable by the user. In the meantime, I just want to be able to write a note and have it show up as something like `/note/note-title`.

This post gives an overview for how to setup dynamic routes for mdx content.

### configuration

Like everything else, I want to be able to control the content directories form `config.js`. I also want some granular control over how the arguments are handled. I came up with four different ways of defining content directory/ies, each offering slightly more control than the one prior, but all building on top of one another.

``` js
content: null // string. defaults to '/content'
----
content: '/garden' OR ['content/garden'] // array or string. defaults to `/${folderName}`
---
content: ['/garden', '/posts'] // array of strings. each directory produces slugs with `/${folderName}`
---
content: [ // multiple directories with options -- array of objects
    {
      path: '/garden',
      inSearch: false,
      baseSlug: // string or null [default to '${path}'],
      template: // string or path to template [default to Post],
      ... // room for future options and extensions
    }
  ]
```

In working this out, I'm trying to provide a good default (even if it's `null`) while offering a way to pass in options and extend the config in the future.

### rethinking structure

Generating static pages in Next.js requires two functions, `getStaticPaths` -- which returns a list of paths -- and `getStaticProps` -- which returns the props for those paths. Because I wanted to be able to define different paths for different content directories, I ended up moving some of the `/pages` folder structure around:

```js
// before -- nested. vested interest in /garden
/pages
|   _app.js
|   _document.js
|   index.js
|   about.js
└───/garden
    |   index.js
    |   [slug].js
    └───/tags
        |  [slug].js

//after -- flatter
/pages
|   _app.js
|   _document.js
|   index.js
|   about.js
|   garden.js
└───/[path]
    |   [slug].js
└───/tags
    |  [slug].js
```

The documentation for [dynamic routing](https://nextjs.org/docs/routing/dynamic-routes) is helpful here. If I have a general `/content` directory with something like `garden/post-01.mdx` and `notes/note-01.mdx`, `[path]/[slug].js` will use `getStaticPaths` to return an object with `{ path, slug }` corresponding to the directory and file name for each mdx file. This way, I don't need to (and shouldn't) define an explicit `/garden` folder. I can just catch all my posts with the more generalized `[path]/[slug].js`.

### helper functions

Two helper functions for generating routes and pages in `genGarden.js`, which is where all the mdx logic lives. The first is called `getFiles()`.

``` js
function getFiles(source) {
  if (Array.isArray(source)) {
    return source.reduce((acc, src) => {
      const usePath = src.path || src;
      const sourcePath = path.join(process.cwd(), usePath);
      const contentGlob = `${sourcePath}/**/*.mdx`;
      const files = glob.sync(contentGlob);
      return [...acc, ...files];
    }, []);
  }
  const sourcePath = path.join(process.cwd(), source);
  const contentGlob = `${sourcePath}/**/*.mdx`;
  const files = glob.sync(contentGlob);

  if (!files.length) return [];

  return files;
}
```

> => note
>
> `source` comes from `@config` -- it's where we define the source of our content. I can be a string, an array of strings, or an array of objects that correspond to where your content lives relative to the project's root.

`getFiles()` takes our `source` and returns an array of files that will be used to generate paths or mdx.

Once we have an array of files, we use the other helper function `getSlug()` to generate the path.

``` js
function getSlug(filepath, source) {
  const options = source.find(s => {
    if (!isObject(s)) return false;
    return filepath.includes(s.path);
  });
  const slug = filepath
    .replace(/^.*[\\\/]/, '')
    .replace(new RegExp(`${path.extname(filepath)}$`), '');
  const slugPath = path.dirname(filepath).replace(/^.*[\\\/]/, '');
  const useSlug = get(options, 'slug', slugPath);

  return { slug, slugPath: useSlug };
}
```

Here, filepath is just the path to the mdx file. `slug` and `slugPath` are exactly what they sound like -- the slug for the page and the path to get there (so something like `garden` or `posts`). The source argument provides extra information about the `slugPath` if we want to override it. The example here would be something like `source = [{ path: 'posts', slug: 'garden' }]` where our content lives in `./posts` but we want the pages to use the `garden` slug when they're generated.

### `getStaticPaths()` and `getStaticProps()`

Back in our `/[path]/[slug].js`, we need to get an array of paths. We call the `getAllPaths()` function, which uses the above helper functions.

``` js
export async function getAllPaths(source = 'garden') {
  const files = getFiles(source);
  const paths = await Promise.all(
    files.map(async filepath => {
      const { slug, slugPath } = getSlug(filepath, source);

      return { path: slugPath, slug };
    })
  );

  return paths;
}
```

Note here that `garden` is set as the fallback source. So, if you don't pass a source argument, it'll expect your content to live in a root folder called `garden`. We're passing in the source information in `/[path]/[slug].js` by grabbing it from `@config.js`:

``` js
export async function getStaticPaths() {
  const { content } = siteConfig;
  const routes = await getAllPaths(content);
  
  ...
}
```

Now that we have the paths, we can use `getStaticProps()` to return the necessary information for each file and generate the pages. Check out `genGarden.js` to see what that looks like.

### wrapping up

The extendability offered by this structure offers a less opinionated way of defining where your content lives and how it gets generated. Working through this also helped me begin to think through how content related options (indexable, searchable, etc.) might be handled in the future.

### up next

- [x] content directories
- [x] [fix tag routes (slugify)](/notes/fix-tags)
- [ ] templating


working with content directories and dynamic routes in next.js

posts tagged 'note'

remark generated table of contents

generating theme tokens

fix tags