Confucius (551–479 BCE)

Confucius was an influential Chinese philosopher whose teachings focused on morality, ethics, and the cultivation of wisdom.

“By three methods we may learn wisdom:
First, by reflection, which is noblest;
Second, by imitation, which is easiest;
And third by experience, which is bitterest.”
— The Analects of Confucius

Aristotle (384–322 BCE)

While Aristotle didn’t use the modern term “intelligence,” his philosophical framework includes key concepts:

Vatsyayana (c. 200-499 CE)

Vatsyayana was an ancient Indian philosopher best known for his work on logic and epistemology.

Intelligence is the ability to discern valid knowledge from error and illusion.
— Nyaya Sutras

• Emphasized pramana (means of knowledge) as essential to understanding truth.
• Identified perception, inference, and testimony as valid sources of knowledge.
• Believed that right knowledge is that which is uncontradicted and corresponds with reality.

Al-Farabi (872–950)

Al-Farabi was a renowned Islamic philosopher and polymath, often called the “Second Teacher” after Aristotle.

“The perfected intellect is that which distinguishes true from false by means of demonstrative reasoning.”
— Book of Letters

• Emphasized the role of reason in acquiring knowledge and achieving truth.
• Argued that aql (intellect) allows humans to differentiate between true and false beliefs.

Alan Turing (1912–1954)

Alan Turing was a British mathematician and computer scientist, widely regarded as a pioneer of artificial intelligence.

“If a machine can behave indistinguishably from a human, then it is intelligent.”
— Computing Machinery and Intelligence (1950)

• Defined intelligence by behavioral indistinguishability.

John McCarthy (1927–2011)

John McCarthy was an American computer scientist who coined the term “Artificial Intelligence” and advanced the field of AI.

“Intelligence is the computational part of the ability to achieve goals in the world.”
— What is Artificial Intelligence? (2007)

• Coined the term Artificial Intelligence in 1955.

Marcus Hutter (b. 1967)

Marcus Hutter is a German computer scientist, a professor of artificial intelligence and senior researcher at DeepMind, known for his work on universal artificial intelligence.

“Intelligence measures an agent’s ability to achieve goals in a wide range of environments.”
— A Collection of Definitions of Intelligence (2007)

• Developed the AIXI model, a mathematical formalization of a maximally intelligent agent based on Solomonoff induction and sequential decision theory.
• Introduced the Universal Intelligence Measure, quantifying intelligence as expected reward averaged over all computable environments, weighted by their simplicity.
• Co-authored a survey of over 70 definitions of intelligence from psychology, philosophy, and artificial intelligence, “A Collection of Definitions of Intelligence” (2007).

Marvin Minsky (1927–2016)

Marvin Minsky was an American cognitive scientist and co-founder of the MIT AI Laboratory, known for his work on the theory of mind.

“Our minds contain processes that enable us to solve problems we consider difficult. Intelligence is our name for whichever of those processes we don’t yet understand.” — The Society of Mind (1985)

• Proposed the “society of mind” model — multiple interacting mechanisms working together.

Robert Sternberg (b. 1949)

Robert Sternberg is an American psychologist who developed influential theories on intelligence and creativity.

Intelligence is mental activity directed towards purposive adaptation to, and selection and shaping of, real-world environments relevant to one’s life. — Beyond IQ: A Triarchic Theory of Human Intelligence (1985)

• Environment selection includes selecting problems to solve or designing experiments to test hypotheses
• Proposed three domains of intelligence:
  • Analytical (logic, problem-solving)
  • Creative (innovation, adaptation)
  • Practical (street smarts, real-world skills)

Howard Gardner (b. 1943)

Howard Gardner is an American developmental psychologist best known for his theory of multiple intelligences.

“Intelligence is the capacity to solve problems or to fashion products that are valued in one or more cultural settings.”
— Multiple Intelligences Go to School (1989)

• Proposed 8+ types of intelligence:

An intelligent being

• Perceives and experiences the environment through experiments and observation
• Can imitate and analyze those experiences
• Infers new knowledge and models of the environment by reflecting and learning from experience and using logic, adapting to its environment
• Understands first principles, such as cause and effect
• Can discern truth from falsehood, e.g. by devising experiments to test hypotheses or using logic
• Can select problems to solve, solve those problems, create valued products, act wisely and achieve goals, shaping its environment
• Uses language and behaves like a human (or other intelligent biological systems)

The environment, problems solved and products created can be: linguistic, logical-mathematical, musical, bodily-kinesthetic, spatial, interpersonal, intrapersonal, or naturalistic.

Sub-fields of AI

• Perception, e.g. computer vision, speech recognition and sensing
• Knowledge representation
• Machine learning
• Formal logic and symbolic reasoning
• Natural language processing
• Robotics
• Planning, search and optimization
• Human-AI interaction and explainable AI
• Multi-agent systems

Sub-fields of AI

• Human emotion and cognition modeling
• AI safety, ethics, fairness and value alignment
• Nature-inspired algorithms, e.g. evolutionary and swarm optimization

Herbert Simon (1916–2001)

Herbert Simon was a renowned cognitive psychologist and economist, co-founder of AI.

“Learning and discovery mechanisms permit the system to adapt with gradually increasing effectiveness to the particular environment in which it finds itself.”
— The Sciences of the Artificial (1996)

• Emphasizes the “adaptiveness of the human organism and its ability to acquire new representations and strategies, becoming adept in dealing with highly specialized environments.”

Tom Mitchell (b. 1951)

Tom Mitchell is a computer scientist and a pioneer in the field of machine learning.

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.”
— Machine Learning (1997)

• General definition but intuitive and useful
• Introduces the notion of a task
• The performance measure is task-specific

What is a task?

• A task is the problem you want to solve with ML.
• It defines what the model should do, not how it learns.
• Examples: classifying emails, predicting house prices, grouping customers.

Core ML tasks

1. Classification: Assign inputs to categories (e.g., spam vs. not spam).
2. Regression: Predict continuous values (e.g., housing prices).
3. Clustering: Group similar data points without labels.
4. Dimensionality Reduction: Simplify data while preserving structure.
5. Anomaly Detection: Identify unusual or rare data points.
6. Ranking: Order items by relevance (e.g., search results).
7. Recommendation: Suggest items based on user preferences.
8. Time Series Forecasting: Predict future values based on past data.
9. Data Generation: Create new data similar to the training set.
10. Causal Inference: Discover cause-effect relationships.
11. Semantic Segmentation: Assign a label to every pixel in an image.

Core ML tasks

12. Object Detection: Identify and locate objects within an image.
13. Speech Recognition: Convert spoken language into text.
14. Machine Translation: Automatically translate text from one language to another.
15. Question Answering: Provide answers to questions posed in natural language.
16. Policy Learning: Learn a strategy for decision-making in environments.
17. Density Estimation: Estimate the probability distribution of a dataset.

Learning terminology

1. Supervised learning: Learning from labeled data, basically classification and regression.
2. Unsupervised learning: Finding patterns in unlabeled data, e.g. clustering and dimension reduction.
3. Semi-supervised learning: Combining labeled and unlabeled data.
4. Self-supervised learning: Using pretext tasks, treating part of the unlabeled data as labels.
5. Reinforcement learning: Learning via interaction and rewards.
6. Online learning: Continuously updating the model with streaming data.
7. Active learning: Identifying and requesting labels for the most informative or uncertain data points to improve learning efficiency, related to design of experiments.
8. Weakly supervised learning: Learning from noisy or incomplete labels.

Learning terminology

9. Multi-task learning: Learning several related tasks simultaneously, sharing knowledge to improve overall performance.
10. Transfer learning: Reusing knowledge learned from one task or domain to improve learning on another, usually sequentially (pretrain then fine-tune) as opposed to multi-task learning.
11. Few-shot learning: Quickly learning a new task from only a few labeled examples, often leveraging transfer learning or meta-learning techniques.
12. Meta-learning: Learning how to learn better and faster, typically by training models to quickly adapt to new tasks.
13. Federated learning: Training machine learning models across decentralized devices or servers while keeping raw data local, enhancing privacy.

Main domains in machine learning

Machine learning can be applied across many domains, each with unique challenges and requirements. Many of these domains are also broadly studied in AI, not just in ML.

1. Healthcare and Biomedical

   • Applying ML to medical data and biology.
   • Tasks: Disease diagnosis, medical imaging, drug development, patient monitoring.

2. Computer Vision

   • Processing and understanding images and videos.
   • Tasks: Image classification, object detection, segmentation, video analysis.

3. Speech and Audio Processing

   • Understanding and generating audio signals.
   • Tasks: Speech recognition, speaker identification, speech synthesis, emotion detection.

Main domains in machine learning

4. Natural Language Processing (NLP)

   • Working with text and language data.
   • Tasks: Text classification, machine translation, sentiment analysis, question answering.

5. Sensor and IoT Data

   • Using data from physical sensors like accelerometers, GPS, environmental sensors.
   • Tasks: Activity recognition, environmental monitoring, predictive maintenance.

6. Robotics and Control Systems

   • Controlling robots and autonomous systems.
   • Tasks: Path planning, manipulation, autonomous navigation, reinforcement learning.

Main domains in machine learning

7. Time Series and Sequential Data

   • Analyzing data indexed by time or sequence.
   • Tasks: Forecasting, anomaly detection, sequence modeling.

8. Finance and Economics

   • Financial market analysis and risk management.
   • Tasks: Fraud detection, algorithmic trading, credit scoring, customer segmentation.

9. Recommendation Systems

   • Personalizing content or products.
   • Tasks: Product recommendations, content filtering, ranking.

Main domains in machine learning

10. Cybersecurity

    • Detecting threats and protecting systems.
    • Tasks: Intrusion detection, malware classification, phishing detection.

11. Manufacturing and Industry 4.0

    • Optimizing industrial processes.
    • Tasks: Predictive maintenance, quality control, supply chain optimization.

12. Gaming and Simulation

    • Training agents or models for games or simulations.
    • Tasks: Game playing AI, strategy learning, environment simulation.