Compiler Design Part-1

Compiler: It is a language translator which translates from one language to other

Contents:

•  Lexical analysis, parsing, syntax-directed translation
• Runtime environments
• Intermediate code generation
• Local optimization
• Data flow analysis: constant propagation, liveness analysis, common subexpression elimination.  

Language Processing System:

    

       Structure of a compiler/Phases of a compiler

    Usage of Symbol Table

Phase	Usage
Lexical Analysis -->	create new entites for new Identifiers
Syntax Analysis:-->	Adds information regarding attributes like type, scope, dimension, line of reference & line of use
Semantic Analysis:-->	Use the available information to check for semantics & is updated
Intermediate code--> generation	to add temporary variables
code optimization -->	Information in symbol table used in machine-dependent optimization by considering addresses & aliased variables information.
Target code generator->	Generates the code by using the addresses information of identifiers.

       

      Symbol table entries

Each entry in the symbol table is assciated with attributes that support the compiler in different phases

Attributes are: Name, Size, Dimension, Type, Line of declaration, line of usage, Address.

          Lexical analysis:

•      We need to define the rules to construct the expression based on source code.
•      The lexer takes the regular expression as input and then converts it into the equivalent FA
•      Every string which is scanned for source code is given as an input for Finite Automata to check the validity of the string.
•      If the string is accepted by FA, then the string becomes a token

P     Problem Example

     Difference between lexeme and token:
Lexeme is a token name, where token contains token name and attribute value.

Design of Lexical Analyzer
We can either write a manual program for lexical analyzer or use tool.(Lex tool)
Manual design: Token -> Pattern -> reg Exp -> FA -> Transistion table -> transistion function

     
Secondary Function of a Lexical analyzer

•         Elimination of white spaces.
•         Removal of comment lines
•        Correlating the error msg by tracing the line number
  
  

     LA uses DFA for Tokenization, It is the only phase that reads the program char by char

Lexical Error: will be updated
Lexical Error Recovery

    Left Factoring:

     To get rid of non-determinism and common prefixes.

     Eliminationg non-determinism doesn't effect Ambuguity.

Support Vector Machine

 Support Vector Machines (SVM) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that assigns new examples into one category or the other, making it a non-probabilistic binary linear classifier.

Input vectors that support the margin are called "support vectors"

A hyperplane is a subspace of one dimension less than its ambient space. If a space is 3-dimensional then its hyperplanes are the 2-dimensional planes, while if the space is 2-dimensional, its hyperplanes are the 1 dimensional lines.

“The goal of support vector machines (SVM) is to find an optimal hyperplane that separates the data into classes.”

In case of SVM, we have a training data with the help of which we build an SVM model which can best predict the test data. There are two approaches in SVM for fitting a model using training data – Hard Margin SVM and Soft Margin SVM.

In case of a hard margin classifier we find “w” and “b” such that
ø(w)=2/||w|| is maximized for all {(xi,yi)} where yi(wT.xi+b)>=1

Soft margin doesn’t require to classify all the training data correctly, unlike hard margin. As a result, soft margin misclassifies some of the training data. However, on an average it has comparatively higher prediction accuracy than hard margin classifier for test data.
Concept of “Slack Variable” - "εi" is introduced to allow misclassification, where εi represents the distance of that point from the boundary margin for that class.

In case of a soft  margin classifier we find “w” and “b” such that
ø(w)=(1/2)wTw+C∑εi is minimized for all {(xi,yi)} where yi(wT.xi+b) >= (1-εj) and εj >= 0 where j is the set of indices of violates the boundary hyper plane.
Parameter "C"  can be viewed as a way to control over-fitting.

For a given point,

• If 0 ≤ εi ≤ 1 then the point is classified correctly, lies in between the hyper plane and the margin on the correct side of the hyperplane. This point exhibits a margin violation.

• If εi > 1  then the point is misclassified, lies on the wrong side of the hyperplane and beyond the margin.

C is a regularization parameter that controls the margin as follows:

• A small value of C implies that the model is more tolerant and hence has a larger margin.

• A large value of C makes the constraints hard to ignore, and hence the model has a smaller margin.

• When the value of C is infinity, then all the constraints are enforced and thus the SVM model is considered a hard-margin classifier

SVM can classify non-linearly separable data as well using the kernel trick.

method of classifying the data by transforming it into a higher dimension is called "kernel trick"

How to measure the agreement between two raters? - Cohen's Kappa Coefficient

 The kappa coefficient is a statistic that measures the agreement between raters. It is commonly used in mental health and psychosocial studies. The kappa coefficient can be used for scales with more than two categories, and its range of possible values is from −1 to 1.

Cohen’s kappa is a quantitative measure of reliability for two raters that are rating the same thing, correcting for how often the raters may agree by chance.

In other words, a model will have high kappa score if there is a big differefence between the accuracy and null rate error.

kappa = (Observed agreement - Expected Agreement) / ( 1 - Expected agreement )

Logistic Regression

 Definition:

Logistic Regression is a Supervised learning algorithm that makes use of logistic functions to predict the probability of a Binary outcome.

- outcome limited to two possible outcomes: yes/no, 0/1, or true/false.

Logistic regression analyzes the relationship between one or more independent variables and classifies data into discrete classes. It is extensively used in predictive modeling, where the model estimates the mathematical probability of whether an instance belongs to a specific category or not.

Logistic regression uses a logistic function called a sigmoid function to map predictions and their probabilities. The sigmoid function refers to an S-shaped curve that converts any real value to a range between 0 and 1.

Types of Logistic Regression:

1. Binary Logistic Regression:
   The dependent variable has only two 2 possible outcomes/classes.
   Example-Male or Female.
2. Multinomial Logistic Regression:
   The dependent variable has only two 3 or more possible outcomes/classes without ordering.
   Example: Predicting food quality.(Good,Great and Bad).
3. Ordinal Logistic Regression:
   The dependent variable has only two 3 or more possible outcomes/classes with ordering. Example: Star rating from 1 to 5

Reference:

Wikipedia

Origin of LR

Linear Discriminant Analysis - LDA Explanation with Numerical Problem

 Linear Discriminant Analysis (LDA) is a dimensionality reduction technique commonly used for supervised classification problems. The goal of LDA is to project the dataset onto a lower-dimensional space while maximizing the class separability.

LDA is very similar to Principal Component Analysis (PCA).

LDA can be performed in 5 steps:

1. Compute the mean vectors for the different classes from the dataset.
2. Compute the scatter matrices (in-between-class and within-class scatter matrices).
3. Compute the eigenvectors and corresponding eigenvalues for the scatter matrices.
4. Sort the eigenvectors by decreasing eigenvalues and choose k eigenvectors with the largest eigenvalues.
5. Use this eigenvector matrix to transform the samples onto the new subspace.

 Linear Discriminant Analysis (LDA) is like PCA, but it focuses on maximizing the seperability among the known categories.

Assumptions:

LDA is parametric - assumes Normal Distribution of data.

LDA assumes that each input have same variance.

Why Use Linear Discriminant Analysis?

Dimensionality Reduction

Feature Extraction

Handling Multiclass Problems

Reducing Overfitting

Applications:
image recognition, text classification, bioinformatics, face recognition

Numerical Problem: Conversion of 2D to 1D

References:
LDA clearly Explained - StatQuest

LDA Code Implementation

LDA Paper