Introduction to Generative AI

From Artificial Intelligence to Foundation Models

Generative AI did not appear out of nowhere. It is the result of decades of progress in Artificial Intelligence, Machine Learning, and Deep Learning. Understanding Generative AI without understanding what came before it leads to shallow knowledge and misuse.

This article follows the natural evolution of intelligence in machines:

Artificial Intelligence (AI)
        ↓
Machine Learning (ML)
        ↓
Deep Learning (DL)
        ↓
Generative AI (GenAI)
        ↓
Foundation Models & Applications

Each layer introduces new capabilities, new abstractions, and new responsibilities for engineers and users.

What Is Artificial Intelligence?

Artificial Intelligence is best understood as a scientific and engineering discipline, not a product or a single technology.

AI is a branch of computer science focused on creating intelligent agents—systems that can:

• Reason about their environment
• Learn from experience
• Act autonomously toward goals

In this sense, AI is similar to physics: it defines theories, methods, and tools used to solve a class of problems.

AI = Theories + Algorithms + Systems
     that mimic intelligent behavior

AI aims to answer a fundamental question:

How can machines behave intelligently?

Machine Learning: Teaching Computers to Learn

Machine Learning (ML) is a subfield of AI where systems learn patterns from data instead of relying on explicitly coded rules.

Traditional Programming

Data + Rules → Output

Here, humans explicitly define the logic.

Machine Learning

Data + Output → Rules (Model)

Instead of writing rules, we train models using examples.

A machine learning system:

• Observes patterns in data
• Builds a statistical representation
• Uses that representation to make predictions on new data

This enables computers to learn without explicit programming.

Supervised vs Unsupervised Learning

Supervised Learning

Supervised learning uses labeled data, meaning each example includes the correct answer.

Input (X) ──► Model ──► Output (Y)
                     ↘ Compare with Label

Example:

A restaurant owner wants to predict tips using:

• Total bill amount
• Order type (pickup or delivery)

The model learns from past examples and minimizes error, which is the difference between predicted and actual values.

This process is a classic optimization problem.

Unsupervised Learning

Unsupervised learning uses unlabeled data.

The goal is not prediction but discovery.

Example: clustering employees based on income and tenure.

Income ↑
       |      ● ●
       |   ● ●
       |         ● ●
       |   ● ●
       +----------------→ Tenure

The model discovers patterns that humans may not explicitly define.

Deep Learning and Neural Networks

Deep Learning is a specialized form of Machine Learning that uses Artificial Neural Networks.

These networks are inspired by the human brain and consist of interconnected neurons organized into layers.

Input Layer → Hidden Layers → Output Layer
     ○             ○○○○            ○
     ○         ○○○○○○○○         ○○○

Key characteristics:

• Many layers (depth)
• Ability to learn complex, abstract representations
• High performance on unstructured data (text, images, audio)

Semi-Supervised Learning

Neural networks can combine:

• Small labeled datasets
• Large unlabeled datasets

[Labeled Data] + [Unlabeled Data]
             ↓
      Better Generalization

This allows models to scale when labeled data is scarce or expensive.

Where Generative AI Fits

Now we reach Generative AI.

AI
└── Machine Learning
    └── Deep Learning
        └── Generative AI

Generative AI:

• Uses neural networks
• Works with labeled and unlabeled data
• Can be trained using supervised, unsupervised, and semi-supervised learning

Large Language Models (LLMs) are a subset of Deep Learning and often a subset of Generative AI.

Discriminative vs Generative Models

Discriminative Models

Discriminative models answer:

“What is this?”

They learn the conditional probability:

P(y | x)

Example:

Image → Model → Dog / Cat

They are excellent for classification and prediction.

Generative Models

Generative models answer:

“What could this be?”

They learn the joint probability:

P(x, y)

This allows them to generate new data.

Learn data distribution
        ↓
Generate new samples

Example: generating a new image of a dog instead of just classifying one.

Is It Generative AI or Not?

A simple rule

If output is:
- A number
- A class
- A probability
→ NOT GenAI

If output is:
- Text
- Image
- Audio
- Video
→ GenAI

Mathematical View

All models can be expressed as:

Y = f(X)

• X = input data (text, images, audio, CSV files)
• Y = model output

If Y is creative content, the model is generative.

Traditional ML vs Generative AI

Traditional Machine Learning

Training Code + Labeled Data
        ↓
      Model
        ↓
Prediction / Classification / Clustering

Generative AI

Training Code + Labeled Data + Unlabeled Data
                ↓
          Foundation Model
                ↓
Text | Code | Images | Audio | Video

Generative AI models are broader, more flexible, and more powerful.

Evolution of Paradigms

Rule-Based Programming

IF ears=2 AND legs=4 AND fur=yes → Cat

Neural Networks

Images → Model → "Cat" / "Not Cat"

Generative AI

Prompt → Model → New Content

Formal Definition of Generative AI

Generative AI is a type of artificial intelligence that creates new content based on what it has learned from existing data.

Training produces a statistical model that predicts the most likely response to a prompt.

This is why the word _generative_ is used.

Types of Generative Models

Text-to-Text

• Translation
• Question answering
• Summarization

Text-to-Image

• Diffusion-based image generation

Text-to-Video

• Video generation from scripts or descriptions

Text-to-3D

• 3D assets for games and simulations

Text-to-Task

• Performing actions such as searching, navigating UIs, or editing documents

Pattern Learning Example

"I'm making a sandwich with peanut butter and..."
→ jelly

Language models:

• Learn statistical patterns in language
• Predict what comes next
• Generate coherent, contextual responses

Transformers (The Breakthrough)

Transformers enabled the 2018 revolution in NLP.

Input → Encoder → Representation → Decoder → Output

They allow:

• Context awareness
• Parallel processing
• Long-range dependencies

Hallucinations (A Critical Limitation)

Hallucinations are incorrect or nonsensical outputs.

They occur when:

• Training data is insufficient
• Data is noisy
• Context is missing
• Constraints are weak

Hallucinations reduce trust and must be actively mitigated.

Prompts and Prompt Design

A prompt is the control surface for Generative AI.

Better Prompt → Better Output

Prompt design shapes:

• Accuracy
• Style
• Relevance
• Safety

Foundation Models

Large pre-trained models adaptable to many tasks.

Foundation models are:

• Large
• Pre-trained
• Adaptable

They can be fine-tuned for tasks such as:

• Sentiment analysis
• Fraud detection
• Image understanding
• Customer support

Generative AI for Code

Generative models can:

• Debug code
• Explain logic line-by-line
• Translate languages
• Generate SQL
• Write documentation

This fundamentally changes software development workflows.

Platforms and Tools

• Vertex AI Studio: explore and customize models
• Vertex AI Agent Builder: low-code AI agents
• Gemini: multimodal AI (text, image, audio, code)

21. Final Takeaway

Generative AI marks a shift from rule-based programming to content generation driven by learned patterns. Understanding this foundation is the first step toward becoming an AI practitioner.