Skip to content

Session 2: History and basics of AI

First we review the results of the first session and discuss what students want to learn and create in this course.

Then we give an overview of the history of AI, the basic concepts and terminology, and the main types of models used in generative AI.

Materials

Audio samples for the course

We listened to the rerecorded radio lecture of Alan Turing. This was broadcasted in 1951.

Results

AI tool usage

We discussed the mindmaps and compared the usage of AI tools like ChatGPT and Gemini. All students use AI tools, some on mobile devices less regularly, most on desktop systems on a daily basis.

Alan Turing

We listened to the lecture of Alan Turing and afterwards the student had to guess when the recording had been done. The guesses were between the 1970s and 1990s. We further discussed the lecture in order to create a common understanding. The main outcomes have been:

  • Turing refers to Ada Lovelace and Charles Babbage. Ada Lovelace stated that machines cannot create anything, they can only follow orders. But the better the plan, the better the results and there are no limits.
  • Machines do not have a free will which humans have. Machines are deterministic and it is necessary to create programs that generate randomness to get some kind of creative results.
  • Turing brings up a metaphor of two men writing their biography. The one man that had a life full of events can easily write a longer biography, while someone who has not done many various things in his life, will not be able to write an elaborate biography. He refers to computers that they depend on the input data. The more data is available, the more can be created.
  • Current computers (in 1951) are not capable to compute complex outputs, but Turing estimates that a computer that is 100 times faster and has 100 times the memory capacities could be able to work like the human brain. We checked the numbers and figured out that we now have computers that are about 1 million times faster and have about one billion times more capacity.
    • Note from aftermath: This is probably the machine Turing had in mind: Manchester Mark 1
  • Turing describes the potential of the computer, while at the time there was no program yet that could solve complex programs. However, Turing stated that by the end of the century, there might be a dialogue machine that is indistinguishable from a human. This is nowadays known as the Turing test.

Notebook LM

We listened to a German podcast created with Google's NotebookLM based only on the lecture slides of Sebastian Raschka.

We then discussed the connection between a biological neuron and the artificial neuron as well as the mathematical meaning of it. See https://en.wikipedia.org/wiki/Perceptron#Definition for another illustration and explanation. We further learned how the basic logical operations AND, OR and NOT can be reallized with an artificial neuron.

From the perceptron we came to the book "Perceptrons" in which it was proved that it is impossible to implement an XOR operation with a single layer of perceptrons.

After a long AI winter, not before the late 1980s and mid 1990s multi-layer neural networks came up that were able to solve more complex tasks.

AI history events

McCulloch & Pitts (1943)

  • First mathematical model of a neuron --> metaphor for biological neurons
    • Note that the human brain works differently, it is more like fish inspires building submarines or birds inspired how to build a plane.
  • Draw a neuron with inputs, weights, and activation function (threshold) and show how AND, OR, NOT can be implemented

Rosenblatt's Perceptron (1958)

  • First learning algorithm for a single-layer neural network
  • Show the basic idea of weight adjustment based on error

Widrow and Hoff - ADALINE (1960)

  • First neural network to use the mean squared error loss function
  • Introduced the concept of gradient descent for training

Minsky and Papert - Critique of Perceptrons (1969)

  • Show the limitations of single-layer perceptrons (e.g., XOR problem) --> show with a drawing of linear classifier
  • Led to a temporary decline in neural network research - the "AI winter"

Backpropagation

  • Rumelhart, Hinton, and Williams (1986) showed that it works for multi-layer networks
  • but the idea was not new as Schmidhuber points out in his blog post

Side results

  • It is recommended to take notes while listening to lectures or audio samples. The notes should be in the same language as the audio in order to reduce the mental load.

Homework

Generate a list of important events in AI history. We compare the generated lists in the next session.