Zig Patterns: Labelled Switch Loops

I recently migrated this site from an Astro project to a custom Zig static site generator. As part of that, I built parsers both for html and markdown, which means tokenizing source files before parsing them into an AST.

One common thing that you will find when building something like a lexer/tokenizer is that you'll need to loop over a set of bytes and keep some state about what you are currently doing. One such way of doing this in modern Zig is to use a labelled switch statement.

const Lexer = struct {
    input: [:0]const u8,
    pos: usize = 0,

    const State = enum {
        start,
        word,
    };

    fn nextToken(l: *Lexer) Token {
        var token: Token = undefined;

        state: switch (State.start) {
            .start => switch (l.input[l.pos]) {
                ' ', '\t', '\r', '\n' => {
                    l.pos += 1;
                    continue :state .start;
                },
                0 => {
                    token.kind = .eof;
                    token.start = l.pos;
                },
                else => {
                    token.start = l.pos;
                    continue :state .word;
                },
            },
            .word => switch(l.input[l.pos]) {
                ' ', '\t', '\r', '\n', 0 => {
                    token.end = l.pos;
                },
                else => {
                    l.pos += 1;
                    continue :state .word;
                },
            },
        }

        token.end = l.pos;
        return token;
    }
};

This code snippet is a simple lexer that will output words in a string that are separated by whitespace, until it reaches EOF, at which point it will output an EOF token.

Notice that there is no foror whileloop in the code. All of the "looping" is done via the labelled switch; when we notice a non-whitespace character, we set the token start position, and then use continue :state .word;to tell Zig to re-run the switch with the state being .word.

This can result in more optimized instructions being output than if you were to use a standard loop, meaning that for use cases like a lexer this is often the favoured implementation.