General Programming Information

The Front End of Compilation (Preprocessing, Lexing, Parsing)

There are typically two steps that occur during the Front End in order to change the human-written code into machine-readable bytecode. In C++, the first step is that the compiler does a starting pass over the code using what is known as a preprocessor. This pass will search for any preprocessor directives that have been defined and mutate the code in some way by computing the directive in the manner that has been defined by the language. In C++ all preprocessor directives are lines that start with the hashtag character (#) and the most clear one to illustrate how this works is the #include directive, an example of which is shown here:

// this is the very top of the file
#include <iostream>
// rest of the file goes here

In this example, the #include preprocessor directive was used in order to call for the C++ iostream library files to be copy-pasted into the file for conversion into bytecode. This inclusion allows for use of C++'s built-in Input/Output features in code without needing to redefine them.

Once the preprocessor has completed expanding all of the directives included in the file, the compiler will then run through the preprocessed code and perform Lexical Analysis, also called Lexing or Tokenization, in which each token of the code is scanned and separated. These scanned tokens may also be evaluated to determine the type of token they are such as an identifier, keyword, separator, operator, literal, comment, etc. in order to improve post-processing of the tokens in the next step. The part that performs this is known as a Lexer, and once it has scanned the tokens it passes them to a Parser to perform Syntax Analysis, also known as Parsing.

When the token sequence is parsed, the Parser will typically create a Parse Tree, which essentially breaks the tokens up into groups depending on the rules of the language. For an example of this, see this image from Wikipedia that creates a Parse Tree of the sentence "John hit the ball."

In this tree, the sentence (S) is broken into a noun (N) and verb phrase (VP). The verb phrase is then further broken into a verb (V) and a noun phrase (NP). This is then broken into a determiner (D) for particles like "the" and the noun (N) that acts as the direct object of the verb.

When a Parser produces this kind of tree it does so based on the rules of the programming language itself. As such, the C++ statement "int a = 1;" may be broken into the following series of tokens:

KEYWORD int (I call this a keyword because it makes sense to me)
IDENTIFIER a (an identifier is just a name of a variable kind of thing)
EQUALS (this is a defined operator, so it is just equals)
1 (literal value of 1)
SEMICOLON (Can be ignored since it holds no bearing on the code meaning)
          

This series of tokens may then be parsed to produce a tree that appears as such:

Keep in mind I made this myself so it is almost guaranteed not accurate, just a visualization to help understand.

Once this tree is created, the Parse Tree is then evaluated in a process known as Semantic Analysis. This is where the code is checked for Semantic Errors in order to send out errors or cancel the compilation process. More importantly, however, the Symbol Table is created, associating Identifiers, Constants, Procedures, and Functions with their defined information such as scope, type, return type, etc. For an example, take this C++ code and a Symbol Table that could possibly be created by the compiler:

int foo (double param) {
  int bar = 0;
  for (int i = 0; i < 10; i++) {
    bar += (i * (int) param);
  }
  return bar;
}

(Yes, this example was plagiarized from Wikipedia, shhhhh)

The symbol table will store much more information than this, including intermediate expression values, such as the one that typecasts param into an int.

Creation of this table marks the end of the Front End of compilation, and we can finally move on to the Middle End. Also, here is where lots of my beginning information is from.

The Middle End of Compilation (Optimization and Control Flow)

The most optional aspect of Compilation, the Middle End, is entirely about Optimizing the code that has been passed to it through simple analysis independent of machinery as well as developing graphs for potential control flow and the call structure of each function. Here, the compiler begins by analyzing many different aspects of the code in order to find potential ways to optimize the code. This is where the Control-Flow Graph and Call Graph are created by the compiler in order to further analyze the code for optimization. An example situation where these are helpful is in determining that a function is never called throughout the program and can thus be omitted from the final compiled file. Once the file is optimized based on coding patterns, the Middle End is finished and the Back End can begin. Here is where I got my information from.

The Back End of Compilation (Specific Optimization and Code Generation)

The final step of Compilation involves more specific optimization and the generation of a machine-code file known as an Object File. Similar to the Middle End, the compiler will analyze the code for certain criteria in order to determine ways to optimize; However, unlike the Middle End, the optimization done in this step is done in order to optimize the code depending on hardware specifications. An example of this is peephole optimization, wherein short sequences of instructions are rewritten for hardware efficiency. Following these optimizations, the code is finally translated into machine-readable code in a process known as Code Generation. As stated earlier, the file produced from this is known as an Object File and typically ends in .obj, .o, or something else depending on the compiler. Once again, here is where my information is from.

Why Understanding Multiple Paradigms is Important

When programming, it is most important to not learn a language per se, but rather build a toolkit of concepts that can be used later. Many of these foundational ideas will have crossover between languages regardless of paradigm. This rings true especially for general-purpose, multi-paradigm languages: For example, big ones like Python, C#, and C++ are all multi-paradigm. Even Java (after Java 8) has implemented Functional programming ideas despite being considered an annoyingly Object Oriented language. Knowing the abilities and concepts of multiple paradigms is essentially a necessary aspect of being a good programmer. Overall, learning these concepts now is important to allow for a better understanding of just how professional coding should be on different projects: a deliberate choice of paradigms depending on the specific problem at hand. As such, please take the time to learn about these concepts at least in some depth just to understand the options you have as a programmer.

Introduction

In programming, there are considered to be 4 different paradigms (a language's approach to the way code is written) that are used when coding. Before reading, remember that many languages have overlap in the way that they implement paradigms, as stated before in the precursor. Building on this, I would also like to state that some paradigms are just extensions of others and are more a superset, so those are included by default. With that said, I will begin at the very baseline with the terminology used to describe different paradigms: the concepts of imperative versus declarative code.

If you want just a quick overview of paradigm differences, click here to see a chart detailing such. I recommend looking at that first and then reading my stuff if needed.

A Precursor About Imperative vs Declarative

This should serve as a short and easy descriptor to both Imperative and Declarative Programming ideologies, but I do still recommend reading the more in-depth versions as they give information about more specific aspects on top of this general description. Ultimately, Imperative Programming is defined as telling a computer every step of the process to execute in very specific terms. On the other hand, Declarative Programming is defined as telling a computer what we want to have happen in order to make the computer do things we want without necessarily defining each step. However, there is a large caveat to Declarative Programming in that it is actually all just an abstraction of Imperative Programming. For example, looking for keys Imperatively would be written as "look under the mattress, then on the couch, then in the bed, then …" whereas looking for keys Declaratively would just be "look for keys." Either way the same steps may be done, but Declarative programming separates the defined execution from the desired logic. As such, Declarative programs are just Imperative code hidden behind a language itself, as there is no other way to do so. This video is great to watch to understand

Imperative Programming and Structured Programming

As said above, Imperative Programming is the act of defining every step for the computer to execute in a sequential order. In Imperative Programming, the order of execution matters heavily as there is a large reliance on data state and expressions, which are known as statements in Imperative Programming as they modify data's state in some way. As an example, Pure Assembly Language, which is very low-level programming that depends on the computer architecture's available instruction set, is a great example of Imperative Programming, as every single step that the computer executes is defined clearly. Essentially, code would be written in one big chunk of statements that mutate the state of the program's data in order to achieve the desired result.

A purely Imperative Programming language is extremely bare-bones and really is just simply a "do this, then this, then this, and so on" form of writing code. As such, there was a large possibility for improving the quality of life aspects of Imperative languages. Given this idea, Imperative Programming was later extended by the concepts of Structured Programming, which sought to improve the flow of purely Imperative languages. Structured Programming is essentially exactly the same as Imperative Programming, but adds various ideas such as if/then/else statements, code blocking, and subroutines (functions). I include Structured Programming here because it is essentially just pure Imperative code but with some quality of life changes. This is the essence of languages with different paradigms: all are extensions of Imperative code in some form or another.

Declarative Programming

I would like to start by building on the common definition of Declarative programming given previously: Declarative programming is telling a computer what we want to have happen rather than how to do it. I stated previously that Declarative Programming is just an abstraction of Imperative programming, and so the computer in this situation is actually the language itself which has some built-in implementation of some actions. Take, for example, the following code written in C#:

int count = 0;
int[] vals = {1, 2, 3, 4, 5};
for (int i = 0; i < 5; i++) {
  if (vals[i] <= 3) {
      count++;
  }
}
int[] underFourDoubled = new int[count];
int cursor = 0;
for (int j = 0; j < 5; j++) {
  if (vals[j] <= 3) {
    underFourDoubled[cursor] = vals[j] * 2;
    cursor++;
  }
}

int[] vals = {1, 2, 3, 4, 5};
vals = vals.Where(vals => vals <= 3).Select(vals => vals * 2).ToArray();

In this code, we begin with vals equaling {1, 2, 3, 4, 5}, but then take every value that is less than 4 and double it, only adding those values to a new array that we then set vals equal to. In the imperative method, I have to parse to find the number of values less than 4, create a new array of that size, then parse them again to add their doubled values to the new array. Contrastingly, the Declarative approach uses the data querying methods included in C#'s LINQ framework, specifically Where() and Select() .

Where() filters the passed in input stream that can be iterated through and adds the values to a new list dependent on if they get returned by the passed lambda expression. Select() takes an iterable input stream and applies the passed lambda expression to that data, returning a list of those changed values. These methods are built in to C# to make data-querying easier to write as they are often-done procedures. There isn't necessarily anything special about how these methods are done, they are all written imperatively, but coding using functions that hide the implementation is what makes it declarative code.

In terms of benefit to programming it this way: the main benefit of writing Declarative code whenever possible is that it allows for faster programming, debugging, and is more safe to program as the way the computer interacts with the queried function is consistent. There is also a potential for having some efficiency benefits in the form of parallel processing, but the majority of cases will be slightly more inefficient, which is really not a problem unless having very efficient code is super necessary, like when the code is being repeated over and over and over in a very short span of time.

You may have recognized by now that Declarative programming is pretty much only applicable when functions have already been defined and, due to this, not all code can be written Declaratively when only using libraries, frameworks, or language features written by others. However, as Declarative programming is essentially just done by using a clearly-defined function, or one that essentially self-comments, Declarative code can be done by abstracting a certain type of function. The functions that define a Declarative function differ from typical Imperative coding functions in the way that they must be Referentially Transparent.

int pure(int x, int y) {
  int val = x + y;
  val = val * 2;
  return val;
}

int globalVal = 3;
int impure(int x, int y) {
  int val = x + y;
  val = val * globalVal;
  globalVal += 1;
  return val;
}

Logical Programming

Logical programming is a subset of the Declarative Programming ideology and involves defining a set of rules and facts, allowing the computer to then decide on its own how to interpret those rules when a question, called a query, is posed to the computer. The computer does this based on the concepts of formal logic, which is a complex field in itself, so I will not explain that here. For an example of how a logical program would look, here is some abstract code written in ProLog, a logic programming language:

food(sandwich).
lunch(sandwich).

meal(X) :- food(X).
?- meal(X), lunch(X).

// sandwich is a food (Fact)
// sandwich is a lunch (Fact)

// Any food is a meal/Anything is a meal if it is food (Rule)
// What is both a meal and lunch (query)

In this instance, the computer would look at the query, determine what rules and facts it would need to look at, then return an answer. For this program, the computer would return "Answer : X = sandwich."

If you want to learn more, here is the Wikipedia article on logical programming. As said before, logic programming works based on the rules of formal logic and is defined by those ideas.

Procedural Programming

Procedural programming, similar to Structured programming, adds more ideas to the concepts of the Imperative Programming ideology by extending Structured Programming even more. As it is still Imperative Programming, Procedural Programming involves directly telling a computer step-by-step what to do, but Procedural Programming modularizes Imperative Programming through the use procedures (functions/routines/subroutines). Procedural Programming added the concepts of modularization (separating functions into different, interactable modules), local variables, iteration and other things to Structured Programming. Ultimately, there really isn't much logical difference between Procedural, Structured, and Imperative programming, but they still have different capabilities technically.

Object-Oriented Programming

Object-Oriented programming is a further extension of the Imperative Programming ideology and adds even more onto the ideas of the Procedural Programming paradigm, and is one of the easiest paradigms to understand as it is one of the most analogous to real-world concepts. In Object-Oriented programming, concepts are represented as classes (objects) that can have data, alter data, both, or neither. Object-Oriented Programming added many concepts associated with Objects, such as data abstraction, inheritance, polymorphism, and much more. This paradigm is liked due to its allowance for easy code-sharing and understandability. Polymorphism is the main defining feature of Object-Oriented Programming and can allow for Data Abstraction through interfaces and inheritance and is ultimately what makes this paradigm powerful. Encapsulation is also important, but can very easily be done wrong by creating too many unnecessary classes to just do one thing, so be careful when using it on smaller scale projects. It is the strict dedication to encapsulation that makes this paradigm hated by so many despite the incredible benefits that can be gained from smartly using polymorphism and inheritance.

Functional Programming

Functional programming, like Logical Programming, is also a subset of the Declarative Programming ideology, and is effectively just a close representation of that ideology. Ultimately, functional programming doesn't necessarily depend on the representation of objects, but rather as a representation of repeatable actions to be done given certain data. In a functional programming language, functions are considered to be first-class citizens, meaning that they are able to be used like any other data entity. A first-class citizen function can be stored in a variable, named, returned from other functions, and passed to other functions. The main descriptor of this language comes in the way that purely functional languages will often restrict the programmer from mutating the state of the program, forcing the program to be Referentially Transparent throughout. This allows for the first-class citizen status of functions, making Functional programs essentially doing actions based on, or with, actions. There are many benefits to using functional programming, if it is supported, as it can eliminate a lot of the boilerplate code (code that is repeated with little or no change) that occurs often in Object-Oriented programming. See the declarative topic for an example of functional programming since my example is pretty much Functional.