2.2.14

Lexical Analysis

Test yourself on Lexical Analysis

Test your knowledge with free interactive questions on Seneca — used by over 10 million students.

Lexical Analysis

Lexical analysis is the first stage a compiled language goes through to compile code.

White space

White space

  • Firstly, white space is removed from the code.
  • This is the white space that is present to help humans read the code.
  • The compiler doesn’t need the white space so it is removed.
Tokens

Tokens

  • The compiler will then look through the code and create tokens out of the keywords and operators it finds (lexemes).
  • A token is a string that is recognised by the computer.
    • For example, the number 8 might be assigned the token 'number:8'.
  • Any comments are ignored by the compiler.
Symbol table

Symbol table

  • These tokens are assembled into a symbol table ready for the next step of the compiling process.
Jump to other topics
1

Components of a Computer

1.1

Structure & Function of the Processor

1.1.1Von Neumann Architecture1.1.2Registers1.1.3Buses1.1.4Fetch, Decode, Execute Cycle1.1.5Factors Affecting CPU Performance1.1.6Pipelining1.1.7CPU Architecture
1.2

Types of Processors

1.2.1CISC & RISC1.2.2GPUs & Their Uses1.2.3Multicore & Parallel Systems
1.3

Input, Output & Storage

1.3.1Elements of Computer Systems1.3.2Types of Computer Systems1.3.3How Magnetic Storage Works1.3.4Properties of Magnetic Storage1.3.5Examples of Magnetic Storage1.3.6How Optical Storage Works1.3.7Properties of Optical Storage1.3.8Examples of Optical Storage1.3.9Types of Optical Disc1.3.10Random Access Memory1.3.11Read Only Memory1.3.12Uses of Flash Memory1.3.13Properties of Flash Memory1.3.14What to do When We Run Out of Memory1.3.15How Virtual Memory Works
2

Software & Software Development

2.1

Systems Software

2.1.1Operating Systems2.1.2Managing Memory2.1.3Paging2.1.4Segmentation2.1.5Device Management & Drivers2.1.6Interrupts2.1.7Interrupt Service Routine (ISR)2.1.8Scheduling2.1.9Types of Scheduling2.1.10Types of Operating System2.1.11BIOS2.1.12Virtual Machines
2.2

Applications Generation

2.2.1Applications Software2.2.2Utilities2.2.3Encryption Software2.2.4Defragmentation Software2.2.5Data Compression Software2.2.6Backup Software2.2.7Open Source Software2.2.8Proprietary Software2.2.9Licensing Issues2.2.10Compilers2.2.11Interpreters2.2.12Assemblers2.2.13Compiling a Program2.2.14Lexical Analysis2.2.15Compilation Stages2.2.16Linkers, Loaders & Libraries
2.3

Software Development

2.3.1Algorithmic Thinking2.3.2Waterfall Lifecycle2.3.3Waterfall Lifecycle - Strengths & Weaknesses2.3.4Agile Methodology2.3.5Agile Methodology - Strengths & Weaknesses2.3.6Extreme Programming2.3.7Extreme Programming - Strengths & Weaknesses2.3.8Spiral Methodology2.3.9Spiral Methodology - Strengths & Weaknesses2.3.10Rapid Application Development2.3.11RAD - Strengths & Weaknesse
2.4

Types of Programming Language

2.4.1High Level Programming Languages2.4.2Low Level Programming Languages2.4.3Programming Paradigms2.4.4Procedural Programming Languages2.4.5Assembly2.4.6Machine Code2.4.7Modes of Addressing Memory2.4.8Object-Oriented Languages2.4.9Classes2.4.10Objects2.4.11Encapsulation2.4.12Polymorphism
3

Exchanging Data

3.1

Compression, Encryption & Hashing

3.1.1Compression3.1.2Run Length Encoding3.1.3Dictionary Coding3.1.4Encryption3.1.5Symmetric Encyrption3.1.6Hashing
3.2

Databases

3.2.1Records3.2.2Databases3.2.3Relational Databases3.2.4Entity Relationship Modelling3.2.5Managing Databases3.2.6Normalisation3.2.7SQL3.2.8Principles of Data Storage3.2.9Atomicity, Consistency, Isolation & Durability
3.3

Networks

3.3.1The Benefits of Networks3.3.2Network Performance3.3.3Types of Networks3.3.4Network Protocols3.3.5Transmission Protocols3.3.6What is the Internet?3.3.7Uniform Resource Locators3.3.8Domain Name Service3.3.9Web Hosting3.3.10Layering Concepts3.3.11TCP &. OSI Models3.3.12The Advantages of Layering3.3.13What's in a Packet?3.3.14How do Packets get Routed?3.3.15Did my Data Arrive Safely?3.3.16Network Hardware3.3.17Transmission Media3.3.18Firewalls3.3.19Proxies3.3.20Client-Server Model3.3.21Advantages of the Client Server Model3.3.22Disadvantages of the Client Server Model3.3.23Peer-to-Peer Model3.3.24Advantages of the Peer-to-Peer Model3.3.25Disadvantages of the Peer-to-Peer Model
3.4

Web Technologies

3.4.1HTML3.4.2CSS3.4.3JavaScript3.4.4Search Engine Indexing3.4.5PageRank Algorithm3.4.6Server & Client-Side Processing
4

Data Types, Data Structures & Algorithms

4.1

Data Types

4.1.1Data Types4.1.2Casting4.1.3Arrays4.1.42D Arrays4.1.5Strings4.1.6Binary4.1.7Sign & Magnitude4.1.8Binary Addition4.1.9Binary Shifts4.1.10Hexadecimal4.1.11Using Hexadecimal4.1.12Converting Binary & Hexadecimal4.1.13Converting Denary & Hexadecimal4.1.14Floating Points in Binary4.1.15Normalisation of Floating Points4.1.16Floating Point Addition4.1.17Floating Point Subtraction4.1.18Bitwise Manipulation - Shifts4.1.19Bitwise Manipulation - Masks4.1.20Character Sets4.1.21ASCII4.1.22Unicode
4.2

Data Structures

4.2.13D Arrays4.2.2Linked List4.2.3Queues4.2.4Graphs4.2.5Hash Tables4.2.6Stacks4.2.7Trees4.2.8Binary Search Trees
4.3

Boolean Algebra

4.3.1Binary Systems4.3.2Logic Circuits4.3.3Boolean Operators4.3.4Evaluating Expressions4.3.5Boolean Logic4.3.6Karnaugh Maps4.3.7Boolean Identities4.3.8Boolean Rules4.3.9Flip Flops4.3.10Adder Circuits
5

Legal, Moral, Cultural & Ethical Issues

5.1

Computing Related Legislation

5.1.1Data Protection Act (1998)5.1.2Computer Misuse Act (1990)5.1.3Copyright Designs & Patents Act (1988)5.1.4Creative Commons Licensing5.1.5Regulation of Investigatory Powers Act (2000)
5.2

Moral & Ethical Issues

5.2.1Online Activity Tracking5.2.2Censorship5.2.3Positive Cultural Impacts5.2.4Negative Cultural Impacts5.2.5E-Waste5.2.6Energy Consumption5.2.7Positive Environmental Impact5.2.8Layout, Colour Paradigms & Character Sets5.2.9Computers in the Workplace5.2.10Automated Decision-Making5.2.11Artificial Intelligence5.2.12Monitoring Behaviour5.2.13Analysing Personal Information5.2.14Piracy & Offensive Communication
6

Elements of Computational Thinking

6.1

Thinking Abstractly

6.1.1Abstraction
6.2

Thinking Ahead

6.2.1Inputs & Outputs
6.3

Thinking Procedurally

6.3.1Decomposition
6.4

Thinking Logically

6.4.1Pseudocode6.4.2Flow Diagrams
6.5

Thinking Concurrently

6.5.1Concurrent Processing
7

Problem Solving & Programming

7.1

Programming Techniques

7.1.1Variables & Constants7.1.2Comments7.1.3Inputs & Outputs7.1.4Subroutines7.1.5Scope
7.2

Programming Construction

7.2.1Integrated Development Environments7.2.2The Purpose of Testing7.2.3Types of Testing7.2.4Test Plans
8

Algorithms

8.1

Algorithms

8.1.1Interpreting Algorithms8.1.2Correcting Algorithms8.1.3Completing Algorithms8.1.4Sort Algorithms8.1.5Bubble Sort8.1.6Insertion Sort8.1.7Merge Sort8.1.8Search Algorithms8.1.9Linear Search8.1.10Binary Search8.1.11Quick Sort Algorithm8.1.12A* Algorithm

Practice questions on Lexical Analysis

Can you answer these? Test yourself with free interactive practice on Seneca — used by over 10 million students.

  1. 1
    Lexical AnalysisPut in order
  2. 2
    Where are the tokens produced by lexical analysis stored?Multiple choice
Answer all questions on Lexical Analysis

Unlock your full potential with Seneca Premium

  • Unlimited access to 10,000+ open-ended exam questions

  • Mini-mock exams based on your study history

  • Unlock 800+ premium courses & e-books

Get started with Seneca Premium

Compiling a Program

Compilation Stages