Skip to content

History of AI - Sebastian Regelmann

Video sources - Summary and Reflection

1 Alan Turing Broadcast

In the broadcast the main talking point is: Can computers think like brains. One argument against that is, that computers only follow instructions that where created by a human so they do not think like a brain. Turing on the other hand thinks that a computer can be programmed to behave like a brain. A computer can be programmed to perform the same calcualtion like a other device and a brain is just a big calculator so with enough storage and calculation power could a program calculate in the same way as a brain does. But for this to work a brain needs to actually perform determenistic calculations and to program a computer like a brain we need to understand the brain really good, to copy it into a program. But still with the problems that nobody knows exactly how a brain works does Alan Turing think that at the end of the 20th century will there be a machine that can answer so human like that a human can't guess if the answer was from a human or a computer.

2 Lecture Sebastian Raschka

  • Mathematical formula of a neuron to perfomr AND/OR/NOT operations
  • Algorithm to learn wheights of perceptron with the help of training data
  • 1969 Perceptorns can only learn simple classification boundaries (AI Winter)
  • Multilayer Perceptron (Neural Networs) for more complex classification with backpropagation for training
  • Diffrent learning algorithms
  • Connvolutional Neural Networs capture positional related data
  • Recurrent Neural Netoworks caputure sequenzial data
  • LSTM are RNN with memory cells
  • Deep Learning stands for a machine recognising patterns in data on their own
  • 2004 Using GPUs to speed up training
  • In the lecture the concept of transformer models with attention is missing

3 Alfredo Canziani

  • McCulloch and Pitts (MCP) provided the biological blueprint. They modeled a mathematical neuron as a binary logical device similar to a neuron cell. It could perform AND, OR and NOT operation. But the wheights and what operation to perform where static.
  • Norbert Wiener showed how a system can adjust itself to reach a target state. With this a error can be defined to meassure how good/bad the perceptron in
  • Claude Shannon’s Information Theory helps to view inputs as statistical signal. The fixed MCP inputs could be replaced by variable Weights. Each input signal could be strengthened or weakened based on its statistical importance.

4 "Perceptrons" and AI Winter

The book described how a single layer perceptron can only classify linear problems. It points out that for more complex clasification a multilayer perceptron is needed but that there are no algorithm to train a multilayer model. Becaus of that the field got no more funding and research slowed down a lot.

AI history events

McCulloch & Pitts (1943)

Mathematical formula that descibes a biological neuron

Rosenblatt's Perceptron (1958)

A physical computer that could learn to recognise simple shapes. It trained wheights from random start values untill it could detect shapes.

Minsky and Papert - Critique of Perceptrons (1969)

A book that shows that a single layer perceptron can only classify linear problems. It is missing the capabilities to classify XOR problems. So many problems could not be solved with a single layer perceptron

Backpropagation

A algorithm that allowes multilayer perceptrons to learn. It caluculates from the back to the front of the model how much influence each neuron has on the error and adjusts the weights to minimize the error

Deep Learning Revolution (2012)

A CNN Model performed really well on the ImageNet test because of the following reasosn:

  • Models had more layers
  • Models could be trained in parallel on GPUs
  • There was a lot of training data on the internet

It showed that with enough compute power and data it is possible to train a NN to perform better than a hand written programm.