Custom Highlighters

A custom Highlighter lets Kazari work with grammars or engines beyond the built-in Nuri and Chroma adapters. Implementing the interface connects any tokenizer to Kazari's rendering pipeline. For consumers who need full control over the output HTML, Engine.Tokenize exposes raw tokens without any rendering.

The Highlighter interface

The interface has three methods:

1
type Highlighter interface {
2
    Tokenize(code, lang, theme string) ([][]Token, error)
3
    GetThemeColors(theme string) (ThemeInfo, error)
4
    GetLoadedLanguages() []string
5
}

Tokenize receives source code, a language identifier, and a theme name. It returns a two-dimensional slice of tokens: one inner slice per line, each token carrying Content, Color, BgColor, and FontStyle fields. Return an error for genuinely unsupported languages (the engine falls back to plaintext for unknown languages and logs a warning).

GetThemeColors returns a ThemeInfo struct with the theme's background, foreground, selection, line number, and fold colors. Kazari calls this once per theme at engine construction to generate CSS variables.

GetLoadedLanguages returns all language identifiers the highlighter can tokenize. Kazari uses this to distinguish "unknown language" (plaintext fallback with warning) from "known language that errored" (hard error).

Minimal implementation

A highlighter that returns unstyled tokens for any language:

1
type PlaintextHighlighter struct{}
2
3
func (p *PlaintextHighlighter) Tokenize(code, lang, theme string) ([][]kazari.Token, error) {
4
    lines := strings.Split(code, "\n")
5
    result := make([][]kazari.Token, len(lines))
6
    for i, line := range lines {
7
        result[i] = []kazari.Token{{Content: line}}
8
    }
9
    return result, nil
10
}
11
12
func (p *PlaintextHighlighter) GetThemeColors(theme string) (kazari.ThemeInfo, error) {
13
    return kazari.ThemeInfo{
14
        FG: "#24292f",
15
        BG: "#ffffff",
16
    }, nil
17
}
18
19
func (p *PlaintextHighlighter) GetLoadedLanguages() []string {
20
    return []string{"plaintext"}
21
}

1
engine := kazari.New(
2
    kazari.WithHighlighter(&PlaintextHighlighter{}),
3
)

For production implementations, see nuri/highlighter.go (full TextMate grammar support with dual-theme optimization) and chroma/highlighter.go (Chroma lexer/style adaptation).

DualThemeTokenizer

When Kazari renders with both a light and dark theme, it needs two token streams with identical boundaries (same line count, same token count per line, same Content values). By default, the engine calls Tokenize twice and merges the results.

Highlighters that can resolve both themes in a single pass implement the optional DualThemeTokenizer interface:

1
type DualThemeTokenizer interface {
2
    TokenizeDual(code, lang, lightTheme, darkTheme string) (light, dark [][]Token, err error)
3
}

The engine type-asserts this interface on its Highlighter at tokenization time. When present and a dark theme is configured, a single TokenizeDual call replaces two Tokenize calls. The Nuri adapter implements this interface; the Chroma adapter does not (Chroma's lexer API does not support multi-theme tokenization, so two passes are used).

The two returned streams must have identical token boundaries. TextMate scanning is theme-independent, so any TextMate-based highlighter can satisfy this constraint naturally.

Raw tokens with Engine.Tokenize

Engine.Tokenize exposes raw token data for consumers who build their own HTML instead of using Kazari's renderer:

1
tokens, err := engine.Tokenize(code, "go")
2
// tokens is [][]Token: one inner slice per source line

Each Token carries:

Tokenize uses the engine's light theme only. It resolves language aliases (via WithLanguageAliases) before calling the underlying highlighter. A configured highlighter is required; calling Tokenize without one returns an error.

Observing failures

When an unknown language falls back to plaintext, the engine emits a warning through WithWarningHandler. For custom highlighters, this lets the consumer log or react to languages the highlighter does not cover:

1
engine := kazari.New(
2
    kazari.WithHighlighter(customHL),
3
    kazari.WithWarningHandler(func(msg string) {
4
        log.Printf("[kazari] %s", msg)
5
    }),
6
)

Without a handler, warnings go to the standard logger via log.Print.

Related

• Types and Interfaces for full field definitions of Token, ThemeInfo, and font style constants
• Nuri Adapter and Chroma Adapter for production adapter implementations
• Post-Render Callbacks for modifying rendered HTML after tokenization