⚙️ Compiler Pipeline
Interactive compiler pipeline simulation. Type an arithmetic expression and watch it become tokens, an abstract syntax tree, stack and three-address code, then a constant-folding optimisation pass — all with a real recursive-descent parser.
About Compiler Pipeline
A compiler is a program that translates source code written in a high-level language into a lower-level representation — typically machine code or bytecode — that a processor or virtual machine can execute. The pipeline is a sequential series of well-defined stages: the lexer (tokeniser) breaks the raw character stream into meaningful tokens; the parser builds an Abstract Syntax Tree (AST) that represents the grammatical structure; the intermediate representation (IR) stage emits a language-independent form suitable for analysis; and the code generator (codegen) produces target instructions, often with an optimisation pass such as constant folding that evaluates compile-time expressions to reduce runtime work. Understanding this pipeline is foundational to programming-language design, IDE tooling, and security analysis.
Type or paste an arithmetic expression into the editor and watch each pipeline stage update in real time. The AST panel shows the parse tree built by a recursive-descent parser; the IR panel shows three-address code; and the optimisation panel highlights constants that were folded. Experiment with nested brackets and large literal values to see how the tree depth and IR length change.
Frequently Asked Questions
What does a lexer do and what is a token?
A lexer (also called a tokeniser or scanner) reads the raw source characters one by one and groups them into tokens — the smallest meaningful units of the language. For a simple arithmetic expression like "3 + 4 * x", the tokens are NUMBER(3), PLUS, NUMBER(4), STAR, IDENT(x). Each token has a type and often a value, and whitespace is typically discarded at this stage. Lexers are usually implemented as finite automata derived from regular expressions.
What is an Abstract Syntax Tree (AST)?
An AST is a tree data structure that represents the grammatical structure of source code, with each interior node representing an operator or construct and each leaf node representing an operand or literal. Unlike a concrete parse tree, the AST omits syntactic noise such as parentheses and semicolons, retaining only the semantic structure needed for further analysis. Compilers, interpreters, linters, and code formatters all operate primarily on the AST rather than on raw source text.
What is a recursive-descent parser?
A recursive-descent parser implements the grammar as a set of mutually recursive functions, one per grammar production rule. To parse an expression, it calls the expression function, which calls the term function for multiplication, which calls the factor function for atoms — mirroring the operator precedence hierarchy directly in the call stack. Recursive-descent parsers are easy to write by hand, produce excellent error messages, and are used by production compilers including GCC (C front-end) and Clang.
What is three-address code (TAC) and why is it used as an IR?
Three-address code (TAC) is an intermediate representation where each instruction has at most one operator and three operands (two sources and one destination): e.g. t1 = a * b; t2 = t1 + c. This simple, uniform structure makes dataflow analysis easy — each temporary is defined once, facilitating static single assignment (SSA) form. Most modern compilers (GCC, LLVM) use a form of TAC internally before lowering to assembly; LLVM's IR is a well-known, human-readable example.
What is constant folding and how much does it improve performance?
Constant folding is an optimisation that evaluates constant sub-expressions at compile time rather than emitting code to compute them at runtime. For example, 2 * 3 + 1 becomes 7 with zero runtime cost. Modern compilers also propagate constants through variable assignments (constant propagation), enable further simplifications like dead-code elimination, and even evaluate pure function calls whose arguments are all constants. The combined effect can dramatically reduce instruction count in code with many literal calculations.
What is the difference between a compiler and an interpreter?
A compiler translates source code into a target representation ahead of time (AOT), producing an artefact — a binary, bytecode, or IR — that can run independently of the compiler. An interpreter reads source (or bytecode) and executes it directly, statement by statement, at runtime. Many modern systems combine both: Python compiles to bytecode (.pyc) which the CPython interpreter then executes, and JavaScript engines compile to machine code using just-in-time (JIT) compilation for frequently executed code paths.
What is a grammar and how does it define a language?
A formal grammar is a set of production rules that define which strings of tokens are valid in a language. Context-free grammars (CFGs), described in Backus-Naur Form (BNF) or Extended BNF, are used for most programming languages. A parser checks that the input conforms to the grammar and builds the AST. The famous Chomsky hierarchy ranks grammars by expressive power; context-free grammars are powerful enough to describe nested structures like balanced parentheses, which regular expressions cannot.
What happens during the semantic analysis phase?
Semantic analysis checks constraints that the grammar cannot express: type checking (is an integer being added to a string?), scope resolution (is this variable declared before use?), and use-before-initialisation detection. In statically typed languages, this phase constructs a symbol table mapping identifiers to their types and builds type annotations into the AST. Errors here produce the familiar "type mismatch" or "undeclared variable" messages that appear after successful parsing.
What are common compiler optimisations beyond constant folding?
Modern compilers apply dozens of optimisation passes: dead code elimination removes instructions whose results are never used; loop unrolling reduces branch overhead; inlining substitutes function call sites with the function body to eliminate call overhead; common subexpression elimination (CSE) avoids recomputing identical expressions; and auto-vectorisation rewrites scalar loops to use SIMD (Single Instruction, Multiple Data) CPU instructions. LLVM's optimisation pipeline applies around 60 passes at -O2, and the speedups achieved routinely range from 2x to 10x over unoptimised code.
How does a just-in-time (JIT) compiler work?
A JIT compiler compiles code at runtime, typically after it has been observed running frequently ("hot paths"). The V8 engine (used in Chrome and Node.js) starts by interpreting JavaScript bytecode, profiles which functions are called most, then compiles those to optimised native machine code. If a previously made assumption (such as a variable always holding an integer) is later violated, V8 "deoptimises" and falls back to interpreted execution. JIT-compiled JavaScript can run within 2-3x the speed of equivalent C code for many workloads.
Watch source code become machine code: lexing to tokens, recursive-descent parsing to an AST, three-address codegen and a constant-folding optimisation pass — type any arithmetic expression and step through the stages.
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install