One white gear in the center with branching trees growing out on both sides, some branches red. AI-generated image
Development

One Grammar Engine Behind Our Language Plugins

By Paul Piper 2 min read

The 22nd plugin took me a weekend. That tells you almost everything about why shared infrastructure matters in plugin development.

The accident that became architecture

I did not set out to build a framework. I built a Freemarker plugin because I needed one. It had a lexer, a parser, a PSI tree, completion, navigation, and a handful of inspections. Standard plugin architecture.

When I started the Julia plugin, I noticed I was writing the same completion infrastructure again. The same settings framework. The same test harness. The same marketplace integration code. So I pulled the common parts into a shared module, and the Julia plugin became a grammar plus custom inspections on top of it.

That was the moment it became a framework, even though I did not call it one for another six plugins.

What is shared

The framework handles everything that is not language-specific. Completion provider registration, reference resolution patterns, settings persistence, the run configuration UI, color scheme integration, error recovery in the parser, and the test infrastructure.

Each new language plugin brings its own BNF grammar, its own lexer rules, and its own inspections. Everything else is inherited. The framework provides abstract base classes for the common patterns - “here is how a language plugin declares its file type, here is how it provides completion items, here is how it highlights errors” - and each language fills in the specifics.

The grammar surprises

Every language has at least one thing that makes the parser angry.

Julia has macros that transform the AST at parse time. You cannot fully parse Julia without executing macros, which means the IDE plugin has to make educated guesses.

Haskell has layout-sensitive parsing - indentation changes the meaning of the code. The lexer has to track indentation levels and emit synthetic tokens. It works, but the implementation is not pretty.

ABAP is case-insensitive with keywords that overlap with identifiers in ways that would make a PL/1 designer uncomfortable. The lexer needs disambiguation rules that feel like they belong in a natural language processor.

Solidity was actually straightforward to parse. The surprises were in the inspections - detecting reentrancy vulnerabilities requires understanding the control flow across external calls, which is more analysis than parsing.

What the framework does not solve

The framework makes new plugins fast to build. It does not make them fast to maintain. Each language evolves independently - Julia adds new syntax, Solidity adds new EVM features, ABAP gets new open SQL constructs. The grammar updates are per-plugin work, and they never stop. I wrote about what that ongoing cost looks like.

Twenty-two languages on one shared engine. I would do it again. I just would not have believed you if you had told me where it was going when I started that Freemarker lexer.