Lexical Analysis
Computer Science: OCR A Level, topic 2.2.14
Lexical Analysis
Lexical analysis is the first stage a compiled language goes through to compile code.
White space
- Firstly, white space is removed from the code.
- This is the white space that is present to help humans read the code.
- The compiler doesn’t need the white space so it is removed.
Tokens
- The compiler will then look through the code and create tokens out of the keywords and operators it finds (lexemes).
- A token is a string that is recognised by the computer.
- For example, the number 8 might be assigned the token 'number:8'.
- Any comments are ignored by the compiler.
Symbol table
- These tokens are assembled into a symbol table ready for the next step of the compiling process.
Test yourself on Lexical Analysis
Lexical Analysis
Where are the tokens produced by lexical analysis stored?