There are two ways for designing a lexical analyzer, they are
Hand coding
Lexical analyzer generator
Hand Coding: Programmer has to perform the following task
Specify the tokens by writing regular expressions
Construct Finite Automata equivalent to a regular expression
Recognize the tokens by a constructed Finite Automata
Where as Lexical analyzer generator, programmer will do step1 remaining two steps are done by lexical analyzer generator automatically.
Hand coding
Lexical analyzer generator
Hand Coding: Programmer has to perform the following task
Specify the tokens by writing regular expressions
Construct Finite Automata equivalent to a regular expression
Recognize the tokens by a constructed Finite Automata
Where as Lexical analyzer generator, programmer will do step1 remaining two steps are done by lexical analyzer generator automatically.
Lexical Analyzer Generator
Lexical Analyzer Generator introduce a tool called Lex, which allows one to specify a lexical analyzer by specifying regular expressions to describe pattern for tokens
The input for the lex tool is lex language
A program which is written in lex language will compile through lex compiler and produce a C Code called lex.yy.c always i.e.,
Now, C code is compiled by C compiler and produce a file called a.out as always i.e.,
The C compiler output is a working lexical analyzer that can take a stream of input character and produce a stream of tokens i.e.,
Structure of Lex Program
A Lex program has the following form
declarations
%%
transition rules
%%
auxiliary functions
The declaration section includes declaration of variables, manifest constants (LT, LE, GT, GE) and regular definitions
The general form of transition rule is
pattern{Action}
where pattern is a regular expression and action defines what action should take when it matches with pattern
Auxiliary function second holds whatever additional functions are used in the actions
These functions can be compiled separately and loaded with the lexical analyzer
Structure of Lex Program
A Lex program has the following form
declarations
%%
transition rules
%%
auxiliary functions
The declaration section includes declaration of variables, manifest constants (LT, LE, GT, GE) and regular definitions
The general form of transition rule is
pattern{Action}
where pattern is a regular expression and action defines what action should take when it matches with pattern
Auxiliary function second holds whatever additional functions are used in the actions
These functions can be compiled separately and loaded with the lexical analyzer
