An unofficial guide to markdown-ts-mode on Emacs 31

Intro So, Emacs 31 has been released, and a lot of shiny new stuff is there, ready for us to play with. You probably heard of this new markdown-ts-mode and decided to check it out. And guess what? On Emacs version 31, this is marked as an experimental mode. What does this mean? Should you use it or not? Is this ready? Is this just a sketch of a mode? Treat this post as a quick guide to getting this mode up and running and helping yourself find answers to these questions. Where is it in terms of features? This is an experimental mode, right? You need to opt in, so probably not everything will work flawlessly yet, and it needs more testing and feedback. That said, don't let this title mislead you. This does not mean the mode is premature in terms of features. As you will see, this is a very feature-rich mode. This mode already covers all of the https://commonmark.org/ spec, as well as most of https://github.github.com/gfm/, with some extras like code blocks even for non-ts-modes, like elisp, table of contents utilities, and interfaces with external converters, such as pandoc and gfm. Before deep diving into it yourself, you may need some help simply turning this mode on. Tree-sitter is tricky. It might even be your first time with tree-sitter, so a quick "install guide" is on our agenda. Where is it? Do I need to install the mode? Experimental means Emacs does not enable the mode by default so it is not there waiting for you to simply open a .md file or call it with M-x markdown-ts-mode RET. You need to load this library. As always on Emacs, there's more than one way of doing everything, I am a big fan of use-package so I tend to use it to organize my init file. Here is my suggested initial setup: Or if you keep use-package out of your tool belt: Now both the mode and the x (nice extra goodies) libraries are loaded, and you can simply visit your Markdown files using it. If you want to experiment with it without touching your own configuration, do the following: Save the above content in a file like testing.el. Call emacs with emacs -Q --load 'testing.el'. And there you have it, a bare Emacs session with your testing ground set up. This is what I will use for the rest of this guide. IMPORTANT: there's NO NEED to download or add this package to your package manager. The (now very old and archived) MELPA Repository will refuse to install on Emacs version 31 onward and is very, very poor in terms of features. If you are using this, you're not using the new built-in markdown-ts-mode. Right? Let's continue. Opening our first markdown file In order for you to "see what I see", we need some pictures. If it is the first time you're using a tree-sitter-based mode, let me warn you: although tree-sitter is wonderful, fast, and feature-rich, it comes with its own set of tasks to complete and perhaps debugging skills if it needs help. I will try to cover some here; I will forget others for sure. For this guide, I will be using this test file. The repository where it is hosted is our laboratory. No code lives there, remember, all code is in Emacs itself. Now go ahead and open the test.md file. IMPORTANT: At this point, many things can happen. If you have the grammar for markdown installed in your system, the file is already opened. You could, though, be prompted, as I am here, with this: It means Emacs hasn't found a grammar for markdown in my system, in this case in ~/.emacs.d/tree-sitter/ (which is the default when I start Emacs with emacs -Q ...). Emacs will offer to install it, which means downloading and compiling it from a repository already defined in markdown-ts-mode's source code. Let's install it with y. Emacs will clone the grammar repository, compile it, and continue to the second grammar. Yes, markdown uses two grammars: the main one and one for inline parsing. I will allow Emacs to install the second one with y. Success! What you should be seeing: If not, here is what you should check if something went wrong: Is Emacs compiled with the tree-sitter flag? Use M-: (featurep 'treesit) RET and check if it returns t. Do you have the tooling used for "compiling" grammars, like make, gcc, and others? Tree-sitter needs a package in your distro, usually named tree-sitter-cli which provides a tree-sitter binary, you can check you have it with tree-sitter --version. This is a common headache for all tree-sitter modes. Many people like NOT to compile their own grammars, but instead use some compiled file from a place they trust, like their own distro repository, or packages with hundreds of pre-compiled grammars. I will not dive into it; there are many ways of acquiring grammars, and I will stick with "build it yourself" for this guide. See, I kind of tricked you there. I told you that you should be seeing that, but actually, the "do you see what I see" should look like this: We provide the full file in here, with several default themes so you can compare whether your setup is complete. So, what happened? This is part of the reason markdown-ts-mode is very special. This mode can work not only with markdown, but with all other -ts-modes available! Keep this in mind; we will talk about code blocks in a while. For now, we need to understand a few things. In your test.md file, we have a special header. It is very common to have toml or yaml as headers of markdown files. This little guy here: Needs something else to fontify (aka be painted with colors by Emacs). Can you figure out what is missing? If your answer is "we need a grammar for YAML!", kudos! Whenever something does not fontify correctly in -ts-modes, you're probably missing a grammar. And as markdown-ts-mode is made to work with all available ts-modes, this is no exception. Let's install our yaml grammar with our trusty M-x treesit-install-language-grammar RET yaml. You might see now what I am seeing: Let's agree to it with y. Hmm, it looks like this time, something went wrong with yaml-ts-mode trying to register its preferred grammar with treesit-install, as there are no suggestions. We could provide it manually. But let's check something first. Taking a look at yaml-ts-mode.el, we can check which grammar it expects in its source code: Awesome! Let's simply evaluate that block and try to install the grammar again. Or manually provide the source https://github.com/tree-sitter-grammars/tree-sitter-yaml to our already-started interactive session, as I did this time: We then keep going with the defaults with RET RET RET... until the library is installed. After that, reload markdown-ts-mode, or use C-x x g, or re-open the file you're visiting. What we did here by visiting the source code is pretty rare, and most -ts-modes will automatically suggest the repository from which they are going to compile. It was nice that this happened, so I can show you what to do. Now what? We need to do the same M-x treesit-install-language-grammar for every block without fontification that we encounter. If you'd like, for our test file we could use C-x x f to force fontification and be prompted for every missing grammar used by this file. By now, you should see the entire document fontified as in here. Same as previous image: A note on grammars A -ts-mode is only as good as the tree-sitter grammar behind it. This means every -ts-mode needs to constantly keep up with improvements to the grammar, which is shared by any editor or program wanting to use tree-sitter to parse the language. This also means we are, at some point, dependent on the grammar for certain constraints and features. Almost all -ts-mode code in Emacs is filled with notes on limitations and the reasoning behind why and how something obscure is treated the way it is. Emacs mode authors and maintainers always try to suggest the grammar and the SHA commit the ts-mode is prepared to use, either in comments or in the code inside the mode, which is the same as you saw for the yaml suggestion. Part of maintaining ts-modes is keeping up with newer grammar version changes. We try our best to keep it updated with the latest versions, but the one we tested against and that should work as expected is the one in the source file of the mode. This is why I think compiling it yourself interactively with Emacs is the best possible way to guarantee a nice experience. Specifically for markdown-ts-mode, we're using the grammars provided by https://github.com/tree-sitter-grammars/tree-sitter-markdown, as this is the most complete, maintained, and broadly adopted one, both by code editors and programs in general. This doesn't mean it is free of bugs or limitations. Again, we do our best to work around these limitations and even contribute issues to the grammar and to the core tree-sitter library. I can finally open a markdown file! Congrats! Now what? How often do I need to do all of this? Only once, the first time you use a -ts-mode, or never if you already have grammars installed by some other method. Now let's see what markdown-ts-mode already provides. A quick look at markdown-ts-mode features We (BTW, this mode is authored by me and Stéphane Marks) provided an easy-menu feature for quick discoverability of functionalities. You can access it by clicking on Markdown in the mode-line, or, if you have menu-bar-mode enabled, on the menu bar, or even Ctrl + Right click (whatever Emacs maps your OS input to) on a buffer using markdown-ts-mode. This is actually this guide's TL;DR, if you want to stop now and explore it yourself (spoilers ahead). Editing The fastest way to learn the mode is to type a little of everything. Below is a speed run: what you write, what key does it for you. Marks (emphasis) Markdown is plain text, so you can always type the markers yourself: When you want You write bold **bold** bold, alt __bold__ italic *italic* italic, alt _italic_ bold + italic ***both*** strikethrough ~~gone~~ inline code `code` Or let the mode do it: C-c C-x C-f (markdown-ts-emphasize) then a single key: b bold, B bold with underscores