
I spent some time this weekend updating a remark utility I wrote a while back. I wanted to add an 'overview' section to the beginning of every post -- basically a table of contents generated from post headers. There's a great little plugin that does this called [`remark-toc`](https://github.com/remarkjs/remark-toc), which uses `mdast-util-toc` to do just this. It did present a few problems for my workflow:

1. It requires you to define where the table of contents went in the markdown. For other folks, I'm sure this is a useful feature, but I didn't want to have to bother.
2. It removed all content between between the table of contents and the first header. Again, maybe not a problem for some folks, but I often have an introductory section that lacks a header.

Because of this, I ended up spinning up a custom implementation of `mdast-util-toc`.

### remark plugin

The [`unified`](https://github.com/unifiedjs/unified) ecosystem has a fairly robust language for describing itself. At it's core, '**unified** is an interface for processing text using syntax trees' (unified documentation). For our purposes, we can think of it this way:

- we have an `.mdx` or `.md` file
- it is run through a `parser` and converted into a syntax tree
- that syntax tree can be transformed by any number of plugins or `transformers`
- the new syntax tree is run through a `compiler` and output as html

We want to write a plugin that returns a `transformer` that modifies our syntax tree. A really basic version would look like this:

``` js
module.exports = parse;

function parse(options = {}) {
  return transformer;

  function transformer(tree) {
    // modify tree here
  }
}
```

We can pass this module to the `remarkPlugins` argument of `renderToString` to transform the tree.

### generating overview section

We'll update the above generic `parse` function to use `mdast-util-toc`, which does the heavy lifting of generating a list of headers with relevant links. `mdast-util-toc` returns a [node](https://github.com/syntax-tree/unist#node) with a `type` of `list` that we can pass to the `children` array of the syntax tree. If that `node` is returned, we'll update the `children` array in the syntax tree so the map comes first.

``` js
const tocUtil = require('mdast-util-toc');

module.exports = parse;

function parse(options = {}) {
  return transformer;

  function transformer(tree) {
    const { map } = tocUtil(tree);
    if (map) {
      tree.children = [].concat(map, tree.children)
    }
  }
}
```

This works, however, I want to generate a `div` with the class-name `toc` that includes both this table of contents as well as an `overview` heading. We'll need to generate this [`parent` node](https://github.com/syntax-tree/unist#parent) as well as the header node. I wrote up a really generic function for this:

``` js
const genOverview = map => {
  const header = {
    type: 'heading',
    depth: 3,
    children: [{ type: 'text', value: 'overview' }]
  };

  return {
    type: 'parent',
    children: [header, map],
    data: {
      hProperties: { className: 'toc' }
    }
  };
};
```

And update the `transformer` function:

``` js
const { map } = tocUtil(tree);
if (map) {
  const overview = genOverview(map);
  tree.children = [].concat(overview, tree.children);
}
```

While not the most exciting thing in the world, this should offer a brief overview for thinking about handling syntax tress. Looking forward to doing more in the future.

### resources

- [unified documentation](https://github.com/unifiedjs/unified)
- [remark plugin documentation](https://github.com/unifiedjs/unified#plugin)

### up next

- [ ] templating


generating a table of contents for markdown posts

generating theme tokens and falling back to defaults

generating theme tokens

remark generated table of contents


I'm working on a theme/token generating library based on [theme-ui](https://theme-ui.com/) and [emotion](https://emotion.sh/docs/introduction). I've found myself using some variation of the same theme object across projects, but as I iterate on that object either improving it or tweaking it, I find it increasingly difficult to keep different versions up to parity. An example: I update how I'm using breakpoints in one so I need to go through each project and update the breakpoints there.

One solution would be to publish the theme object as an npm package, but I wanted something a little more dynamic. [speculative fabulation (or sf)](https://github.com/inadeqtfuturs/if-sf) generates theme tokens (ex. colors, typography, buttons) as well as theme objects (ex. an object comprised of theme tokens). These can be passed into `emotion`'s or `styled-components`'s theme provider. `sf` operates on the idea that objects go in, objects come out -- you can extend, merge, or override theme defaults, allowing granular control over the theme you generate:

``` js
const defaultTheme = genTheme();
const defaultThemeWithNewColor = genTheme({ colors: { newColor: '#333' } });
```

Both functions return a full theme object, but you can drill down and update the specific tokens you need.

### token generators

`genTheme()` is broken down into a series of token generators -- functions that return a token object, like colors or styles for a button. In working on these token generators, I set up a few parameters for how they should behave:

0. generator files export a token object
1. generators return a token object
2. generators don't require arguments to return the token object
3. generators merge and extend the token object by default
4. generators can wholesale override the token object

For something like colors, this is simple enough:

``` js
import { merge } from 'theme-ui';

const defaultColors = {
  text: '#111827',
  background: '#F9FAFB',
  primary: '#2563EB',
  secondary: '#DE7283',
  accent: '#7C3AED',
  muted: '#88d1FF'
};

export function genColors({ override, ...restColors } = {}) {
  if (override) {
    return override;
  }
  if (!restColors) {
    return defaultColors;
  }
  return merge(defaultColors, restColors);
}

export default defaultColors;
```

By default, `genColors()` returns `defaultColors`. If we pass an `override` argument, it will return that object instead. Any other key/value pair we pass gets merged into default colors.

### token generators -- deep and defaults

Things became slightly more complicated as I began working with typography. In the past, I've defined a default text base that includes things like font size, line height, and font family, spreading that object into each typographical element (i.e. p, ul, blockquote, etc.) (I'm ultimately leveraging the `theme.styles` object from [theme-ui](https://theme-ui.com/styling-mdx)). This works for the first two generator requirements -- have a base set of styles; spread them into each element, return that:

``` js
export const defaultTextBase = {
  fontFamily: 'body',
  fontSize: ['sm', 'md'],
  lineHeight: 1.5,
  margin: 0,
  padding: 0,
  marginBottom: 4
};

const defaultText = {
  p: {
    ...defaultTextBase
  },
  ul: {
    ...defaultTextBase,
    paddingLeft: 8,
    ul: {
      margin: 0
    }
  },
  ...
}
```

The problem starts when I want to extend or merge styles into the base: if we're just spreading `defaultTextBase` into these elements, at what point do we actually merge or extend the base? One way to solve this would be to iterate through the keys in `defaultText`, spreading the base as we went, but this would prevent us from exporting an object from the generator file. Another way to handle this would be to get a diff between the new base and default base -- after all, we're just extending or merging into it. However, this presents an immediate problem if we're going to wholesale override the base at some point. So I ended up writing a utility that removes the default values (if present) from each element.

``` js
function removeDefaults(obj, defaultObj) {
  const defaultKeys = Object.keys(defaultObj);

  // mapObject does what it sounds like: allows us to map the key/value pairs in an object
  // mapObject = (obj, func) => Object.fromEntries(Object.entries(obj).map(func));
  return mapObject(obj, ([key, value]) => {
    const remainingValues = Object.keys(value).reduce((acc, curr) => {
      if (defaultKeys.includes(curr) && defaultObj[curr] === value[curr]) {
        return { ...acc };
      }
      return { [curr]: value[curr], ...acc };
    }, {});
    return [
      key,
      {
        ...remainingValues
      }
    ];
  });
}
```

Once the default values are removed, we can determine the new base (overriding, merging into, or extending the default base) and use the above `mapObject` function to pass that base into each key. You can see the full implementation [here](https://github.com/inadeqtfuturs/if-sf/blob/main/src/theme/tokens/text/index.js)

### spread defaults // remove defaults

Something about this pattern seems useful, though I'm not entirely sure where. It goes something like this:

1. Have a default base
2. Spread that default base into each key of a token object. Export that object.
3. If you need to extend or override that base, use the above function to remove those defaults
4. Return new object using new base

I like this pattern, though I think it fits a specific use case. I do think it could be used in a more generalized way, and I'd be interested to hear alternative approaches to the above problem.

### up next

- [ ] templating



In the next iteration of garden, I want to implement different content directories to field/contain different kinds of content. As I've written about [previously](/garden/rethinking-digital-gardening), I'm not thinking about content in a hierarchy, but marking a different kind of speed (reading and writing) when thinking. At a very high level, I want to be able to write notes (very short snippets or thoughts), posts (slightly longer thoughts, much like I'm doing now), and have projects (longer/larger and perhaps more polished pieces). Eventually, this schematization could be configurable by the user. In the meantime, I just want to be able to write a note and have it show up as something like `/note/note-title`.

This post gives an overview for how to setup dynamic routes for mdx content.

### configuration

Like everything else, I want to be able to control the content directories form `config.js`. I also want some granular control over how the arguments are handled. I came up with four different ways of defining content directory/ies, each offering slightly more control than the one prior, but all building on top of one another.

``` js
content: null // string. defaults to '/content'
----
content: '/garden' OR ['content/garden'] // array or string. defaults to `/${folderName}`
---
content: ['/garden', '/posts'] // array of strings. each directory produces slugs with `/${folderName}`
---
content: [ // multiple directories with options -- array of objects
    {
      path: '/garden',
      inSearch: false,
      baseSlug: // string or null [default to '${path}'],
      template: // string or path to template [default to Post],
      ... // room for future options and extensions
    }
  ]
```

In working this out, I'm trying to provide a good default (even if it's `null`) while offering a way to pass in options and extend the config in the future.

### rethinking structure

Generating static pages in Next.js requires two functions, `getStaticPaths` -- which returns a list of paths -- and `getStaticProps` -- which returns the props for those paths. Because I wanted to be able to define different paths for different content directories, I ended up moving some of the `/pages` folder structure around:

```js
// before -- nested. vested interest in /garden
/pages
|   _app.js
|   _document.js
|   index.js
|   about.js
└───/garden
    |   index.js
    |   [slug].js
    └───/tags
        |  [slug].js

//after -- flatter
/pages
|   _app.js
|   _document.js
|   index.js
|   about.js
|   garden.js
└───/[path]
    |   [slug].js
└───/tags
    |  [slug].js
```

The documentation for [dynamic routing](https://nextjs.org/docs/routing/dynamic-routes) is helpful here. If I have a general `/content` directory with something like `garden/post-01.mdx` and `notes/note-01.mdx`, `[path]/[slug].js` will use `getStaticPaths` to return an object with `{ path, slug }` corresponding to the directory and file name for each mdx file. This way, I don't need to (and shouldn't) define an explicit `/garden` folder. I can just catch all my posts with the more generalized `[path]/[slug].js`.

### helper functions

Two helper functions for generating routes and pages in `genGarden.js`, which is where all the mdx logic lives. The first is called `getFiles()`.

``` js
function getFiles(source) {
  if (Array.isArray(source)) {
    return source.reduce((acc, src) => {
      const usePath = src.path || src;
      const sourcePath = path.join(process.cwd(), usePath);
      const contentGlob = `${sourcePath}/**/*.mdx`;
      const files = glob.sync(contentGlob);
      return [...acc, ...files];
    }, []);
  }
  const sourcePath = path.join(process.cwd(), source);
  const contentGlob = `${sourcePath}/**/*.mdx`;
  const files = glob.sync(contentGlob);

  if (!files.length) return [];

  return files;
}
```

> => note
>
> `source` comes from `@config` -- it's where we define the source of our content. I can be a string, an array of strings, or an array of objects that correspond to where your content lives relative to the project's root.

`getFiles()` takes our `source` and returns an array of files that will be used to generate paths or mdx.

Once we have an array of files, we use the other helper function `getSlug()` to generate the path.

``` js
function getSlug(filepath, source) {
  const options = source.find(s => {
    if (!isObject(s)) return false;
    return filepath.includes(s.path);
  });
  const slug = filepath
    .replace(/^.*[\\\/]/, '')
    .replace(new RegExp(`${path.extname(filepath)}$`), '');
  const slugPath = path.dirname(filepath).replace(/^.*[\\\/]/, '');
  const useSlug = get(options, 'slug', slugPath);

  return { slug, slugPath: useSlug };
}
```

Here, filepath is just the path to the mdx file. `slug` and `slugPath` are exactly what they sound like -- the slug for the page and the path to get there (so something like `garden` or `posts`). The source argument provides extra information about the `slugPath` if we want to override it. The example here would be something like `source = [{ path: 'posts', slug: 'garden' }]` where our content lives in `./posts` but we want the pages to use the `garden` slug when they're generated.

### `getStaticPaths()` and `getStaticProps()`

Back in our `/[path]/[slug].js`, we need to get an array of paths. We call the `getAllPaths()` function, which uses the above helper functions.

``` js
export async function getAllPaths(source = 'garden') {
  const files = getFiles(source);
  const paths = await Promise.all(
    files.map(async filepath => {
      const { slug, slugPath } = getSlug(filepath, source);

      return { path: slugPath, slug };
    })
  );

  return paths;
}
```

Note here that `garden` is set as the fallback source. So, if you don't pass a source argument, it'll expect your content to live in a root folder called `garden`. We're passing in the source information in `/[path]/[slug].js` by grabbing it from `@config.js`:

``` js
export async function getStaticPaths() {
  const { content } = siteConfig;
  const routes = await getAllPaths(content);
  
  ...
}
```

Now that we have the paths, we can use `getStaticProps()` to return the necessary information for each file and generate the pages. Check out `genGarden.js` to see what that looks like.

### wrapping up

The extendability offered by this structure offers a less opinionated way of defining where your content lives and how it gets generated. Working through this also helped me begin to think through how content related options (indexable, searchable, etc.) might be handled in the future.

### up next

- [x] content directories
- [x] [fix tag routes (slugify)](/notes/fix-tags)
- [ ] templating


working with content directories and dynamic routes in next.js

04 - content directories in next.js


This post spins off of the [previous post's](/garden/02-syntax-highlighting) interest in extending themes. Because I ended up reworking both how garden handles themes (both their generation and passing them to different theme providers), I spun this off into its own post.

### color tokens and generating themes

To start, the minimum number of color tokens needed for garden is two: text and background color. Rather than go monochromatic, we also want a primary color. These should live in an easily configurable place and all components necessary for garden to work should only use these colors.

We'll add a `theme` object to `config.js`. It will expect at least a `default` object with keys corresponding to the theme object, so we'll give it a `color` object with the necessary colors. We define our themes this way so that, in the future, we can continue extending the theme with other design tokens besides colors. We can also provide config any number of other objects to generate themes. Here's an example with a default and a dark theme:

``` js
  ...
  theme: {
    default: {
      colors: {
        text: '#2d2a24',
        background: '#fdfdfd',
        primary: '#b7e2d8'
      }
    },
    dark: {
      colors: {
        text: '#ececec',
        background: '#2c2c2c',
        primary: '#888'
      }
    }
  }
```

> => question
>
> This would be really easy to accomplish with [color modes](https://theme-ui.com/color-modes), but we have to also pass the theme to the styled components provider. We'll handle this very similarly to how theme-ui handles it, though.

Next, we need to make sure the theme object supports color overrides. Garden does this using a function called `genTheme` that takes `colors` as an argument and returns a theme object.

``` js
  function genTheme({ colors } = { colors: defaultColors }) {
    ...theme
  };
```

Note here that I also have a set of default color values. You could run the function without any arguments and still get a perfectly good theme object.

### toggle between modes

Heavily influenced by theme-ui's color modes documentation, garden takes the themes specified in the config and cycles through them. At the `App.js` level, we generate an object that contains a list of themes defined in the config, and an array of theme objects generated by those themes. Here, I'm calling them 'modes' just to differentiate them (in name alone) from themes.

``` js
  const { themes } = siteConfig;
  const modeList = {
    modes: Object.keys(themes),
    modeThemes: Object.keys(themes).map(theme => {
      return genTheme({ ...themes[theme] });
    })
  };
```

In the functional component, we use a hook to setup the default state, and a toggle function to cycle among nodes.

``` js
  const [currentMode, setMode] = useState('default');
  const { modes, modeThemes } = modeList;
  const toggleMode = () => {
    const index = modes.indexOf(currentMode);
    const next = modes[(index + 1) % modes.length];
    setMode(next);
  };
```

Wrapping the whole of the app in a [context provider](https://reactjs.org/docs/hooks-reference.html#usecontext) allows us to pass the down the mode state and the toggle function.

``` js
  <ThemeContext.Provider
    value={{
      currentMode,
      toggleMode
    }}
  >
    {...app}
  </ThemeContext.Provider>
```

As an aside, I ended up liking this as a feature rather than the usual sun/moon icon for a few reasons:

1. more than one theme
2. clear what theme is on (rather than relying on iconography)
3. no need for svg imports

### resources

- [theme-ui color modes](https://theme-ui.com/color-modes)

### up next

- [x] syntax highlighting
- [x] [content directories](/garden/04-content-directories)
- [ ] templating

programmatically generating themes utilizing basic variables/tokens and adding state logic to cycle through themes

03 - extending themes


This post offers really brief documentation for adding syntax highlighting with [rehypePrism](https://github.com/mapbox/rehype-prism) to `next-mdx-remote`. It then outlines some considerations for handling that highlighting over and against the global theme.

### syntax highlighting

Syntax highlighting with next-mdx-remote can be done a few different ways. If you want to use [prism-react-renderer](https://github.com/FormidableLabs/prism-react-renderer), you can find great examples in the template repos for [mdnext](https://github.com/domitriusclark/mdnext/blob/master/packages/mdnext-components/src/components/Code.js). I opted to use `rehypePrism` to render code blocks on the node side of things. All I did was pass it as an argument in `renderToString`.

```js
  import rehypePrism from '@mapbox/rehype-prism';

  ...

  const mdx = await renderToString(content, {
    components: null,
    mdxOptions: {
      rehypePlugins: [rehypePrism]
    },
    scope: data
  });
```

### themeing and syntax highlighting

Initially, I just added the prism NightOwl theme and thought I'd move on, but the difference between the prism theme and the color scheme of the site had me wondering if there was a better way to handle syntax highlighting as an extension of the theme object. Some considerations:

1. Syntax highlighting should be an extension of the theme
2. This also means that it should be an extension of the dark theme
3. Because garden is meant to be extendable, the syntax highlighting should be as simple as possible so it can be easily overridden
4. Syntax highlighting and themeing more generally should utilize the minimum viable theme tokens or variables

Thinking about syntax highlighting ultimately opened some space to think a little bit more about colors, theme tokens, and their relation garden as a whole. You can find a post about that [here](/garden/03-extending-themes);

The approach I ended up taking, which is a design pattern I consistently fall back on, is defaults with fallbacks. To differentiate tokens in a block of code, I needed 6 colors. To accommodate undefined themes in a user config, I'm export a default color object from the theme and using that as a fallback. So now the syntax highlighting will be an extension of the current theme and fills in missing tokens with defaults. The one other thing I wanted to add was a way to define the highlighting theme based on some other theme in the config. An example: I want my syntax highlighting to be derived from the dark theme, no matter what theme is currently being used. This gives us three use cases/scenarios:

1. I didn't define any themes, so default.
2. I've defined themes. Just use those colors.
3. I've defined themes. I want to use a specific theme for syntax highlighting.

This ends up having a few moving parts:

``` js
  export const defaultColors = {
    text: '#2d2a24',
    background: '#fdfdfd',
    primary: '#b7e2d8',
    secondary: '#de7283',
    accent: '#bcadfb',
    muted: '#88d1ff'
  };
```

We need to export a set of default colors to generate the theme is a user doesn't define one. It gets imported into `App.js` and used during theme generation.

``` js
  const { themes = { default: { colors: defaultColors } } } = siteConfig;
```

In the theme function, we check if the user defined a specific theme to use for syntax highlighting, and fall back to the default color pallet if not.

``` js
  const prismTheme = pt ? themes[pt].colors : colors;
```

At that point, we have the variables available in the theme object and they get passed in.

### wrapping up

This ended up being a helpful exercise in moving from making something that works to thinking about how I wanted it to work and implementing from that direction. In thinking through different ways one would want to define their theme object, I think I also opened up space to think about extendability here and in future projects.

Working through this also highlighted the need for a different kind of post -- something shorter, and maybe just with code snippets. Next up is thinking about content directories and templating.

### resources

- [components.ai theme-ui](https://components.ai/theme-ui)
- [theme-ui color modes](https://theme-ui.com/color-modes)
- [prism theme generator](http://k88hudson.github.io/syntax-highlighting-theme-generator/www/)

### up next

- [x] syntax highlighting
- [x] [content directories](/garden/04-content-directories)
- [ ] templating


adding syntax highlighting to next-mdx-remote and rethinking themeing

02 - syntax highlighting and themeing


This post documents adding search to next.js powered static sites using [fuse.js](https://fusejs.io/). When deciding how to implement search functionality, I ended up choosing between a custom implementation (basically just filtering arrays) and using a dedicated library. While I tend not to reach for off the shelf solutions, I chose fuse because it has UMD and commonJS support and will likely be helpful to eventually build out the cli.

1. initialize fuse to search through the posts
2. hook up the search function to text input
3. return filtered list of posts

### initialize fuse

I'm adding search to the [garden page](/garden) because it already has access to all the post data needed to actually search. I have an array of posts with the following shape:

```js
  post = {
    content,
    frontmatter: {
      tags: [],
      title
    }
  }
```

Because it's searching a nested list, we need to tell fuse which fields to search for:

```js
  const options = {
    keys: ['content', 'frontmatter.title', 'frontmatter.tags']
  };
  const fuse = new Fuse(posts, options);
```

### hook up search input

We'll use a hook to keep track of the search value. The one extra piece here is [debouncing](https://css-tricks.com/debouncing-throttling-explained-examples/) the input. I like this a bit better than throttling since I'm guessing users will be entering single words rather than phrases.

```js
  import debounce from 'lodash.debounce';

  const [searchVal, setSearchVal] = useState('');

  const debounceQuery = debounce(val => setSearchVal(val), 333);
  const updateSearchVal = e => {
    const { value } = e.target;
    debounceQuery(value);
  };

  ...

  <input value={searchVal} onChange={updateSearchVal} />
```

### return filtered list of posts

Last thing to do is filter the posts based on the search criteria. The `else` statement resets the `postState` back to its initial state when the search field is empty.

```js
  const [postState, setPostState] = useState(posts);
  useEffect(() => {
    if (searchVal) {
      const filteredPosts = fuse.search(searchVal).map(p => p.item);
      setPostState(filteredPosts);
    } else {
      setPostState(posts);
    }
  }, [searchVal]);
```

### wrapping up

A tribute to fuse, to be sure, this was extremely easy. This logic will need to be extended at some point to do tag filtering, too, but that can wait until there's more content.

One thing that came up while working on this was the lack of syntax highlighting -- something I wholesale forgot -- so I'll be adding that next.

### up next

- [x] [syntax highlighting](/garden/02-syntax-highlighting)
- [x] [content directories](/garden/04-content-directories)
- [ ] templating


adding search to next.js powered static sites with fuse.js

garden 🌱

recent posts

remark generated table of contents

generating theme tokens

fix tags

04 - content directories in next.js