Syntax extensions should be configurable

[chrisjsewell]

As per https://myst-parser.readthedocs.io/en/latest/syntax/optional.html and https://github.com/executablebooks/myst-vs-code#preview-enhancement, all syntax extension should be configurable, to enable/disable.
I don't think is possible yet @agoose77?

[agoose77]

It's currently possible to disable jupyterlab-markup extensions, with a checkbox:

These settings can also be set via the disabled-plugins property in the jupterlab-markup settings.

[chrisjsewell]

Ah ok cool 👍

[agoose77]

@chrisjsewell can this be closed?

[chrisjsewell]

yep, although at some point we should write a brief bit of documentation about how to use/configure the extension (in the README or better a small RTD site)

[agoose77]

Sounds good, let's leave this open then.

[tavin]

I think it doesn't work this way anymore? In jupyterlab==3.6.1 and jupyterlab_myst==1.1.1? There seems to be no way to toggle the syntax extensions.

[agoose77]

@tavin good spot; we upgraded the package, and lost some "features" along the way. The extensibility of MyST in general is something that the community at-large need to figure out. Although there is prior art for what this might look like, this fits into the wider context of what version of MyST (e.g. 2023.02.03) is in use, i.e. the need for a version identifier. We also want to make it convenient to extend the syntax of myst at the data-level, so that it can easily be adapted to particular use cases.

So, for now, watch this space.

[tavin]

Good to know @agoose77. Where in the code is the list of enabled extensions, for the sake of hacking on it locally?

[agoose77]

Some of the plugin configuration is here:
https://github.com/executablebooks/jupyterlab-myst/blob/cc1ad5a61512c12084b83d61bdcc0ab57f45387b/src/myst.ts#L122-L131

[rowanc1]

And here for the roles/directives:

https://github.com/executablebooks/jupyterlab-myst/blob/main/src/myst.ts#L51-L54

[tavin]

Here's an idea about where to configure syntax extensions:

_config.yml
---
parse:
  myst_enable_extensions:

Maybe for this to work out, a user should somehow actively mark a directory as a "jupyter book project" or simply a "jupyter project" ...

[tavin]

@agoose77 @rowanc1 I have a partial solution for this in #115. Through a labextension "token" I expose a simple interface that encapsulates the options object which is passed to the myst parser.

The currently hardcoded roles, directives, syntax extensions, etc. are kept as defaults.

It's relatively easy to write a labextension to alter these. You can imagine backing the interface with entries from the UI settings editor, or from notebook metadata, or whatever.

I've used this to implement a few custom directives as well.

For context and easy reference these are the options I'm talking about, which can be passed through the interface:

myst-parser/src/types.ts

export type AllOptions = {
  vfile: VFile;
  markdownit: MarkdownIt.Options;
  extensions: {
    colonFences?: boolean;
    frontmatter?: boolean;
    math?: boolean | MathExtensionOptions;
    footnotes?: boolean;
    deflist?: boolean;
    tasklist?: boolean;
    tables?: boolean;
    blocks?: boolean;
  };
  mdast: MdastOptions;
  directives: DirectiveSpec[];
  roles: RoleSpec[];
};

[rowanc1]

This is super cool, still looking through the PR!

Do you have a demo of how to create a different lab extension to override the options?

[tavin]

@rowanc1 this is a fully working labextension which modifies the myst parser options: https://github.com/tavin/myst-options-poc/tree/main/src