Summer of Math Exposition

Presented by 3Blue1Brown 3blue1brown

How does AI learn?

Audience:

Tags: optimizationmachine-learningaioptimization-theorylearninggradient

This blog post explains how AI's learn, with a focus on gradient descent. I explain the intuition behind gradient descent using analogies, drawings and animations, structured in the form of a dialog with a (fictional) student. The entry starts out with arguing why AI is overall learning the same way humans do, by means of trial and error. Then follows an example of how one can measure the accuracy of the AI model using a cost function, followed by a description of how gradient descent can be used to efficiently find the optimal model parameters. Details such as step size, stopping criteria etc. are also briefly discussed. My goal with the entry is to make the topic of how AI's learn understandable for a target audience of people who are curious towards AI and technology, but without a requirement on them having any mathematic foundation in calculus, linear algebra or optimisation theory. My aim is to keep things very high-level, as to not scare readers away by mathematical nitpicking of details. The target audience could therefore be e.g. highschool students or students starting university - but it is intended for any curious adult.


Analytics

5.5 Overall score*
44 Rank
13 Votes
6 Comments

Comments

6.6

I like the approach where you use a fictional conversation between two persons to help develop the idea intuitively.

Overall, I think your approach does a great job motivating the concept as I go along. Each step follows naturally from the previous. And the dialog format certainly helps in bringing out this quality.

6.5

I enjoyed reading the text and appreciated all the nice animations

The last bit before Part 3 is a bit weird, because the sentence structure seems to be incorrect. The title of Part 3 also mentions gradient descent, which has not been mentioned until that point. The first animation in Part 3 also disappears to quickly, such that it would be good to have a figure with just the data and then a line drawn.

\nabla is not a Greek symbol, as stated in Part 4. Since the whole text is quite long and Part 5 is just a bonus, I would move Part 5 after the summary.

7.8

This was greatly explained! The concept of AI learning is commonly known, but this is the first time I got a look into the details through a simple prediction of salary. It really does reinforces my intuition of math and AI. Moreover, the analogies used is very effective in explaining technical math concepts such as the gradient. The GIF animation is clearly illustrated as well. The conversation did lead me through the flow of the post. However, it sometimes distracted me a little bit from the main content.

Overall, this is a really well-explained post of the mathematical details of AI learning and has made me interested!

4

The explanation is good and covers the whole idea and training mechanism of ML however the same topic is already available in tons of blogs in the same format and covering the same points. I feel that splitting it in two parts would be better if aimed at high school students and then expand each part with less conversion and more rigidity.

4.8

Good explainer, I think this is quite illustrative on how AIs learn. However, this is a competition about math, and your article was devoid of any math equations. I think that even for complicated concepts such as gradient descent, you can simplify the math to something easier: for example, use a 2D function with x=parameter weight and y=loss, and demonstrate how a gradient descent would work just with the 2D slope, then say that this is actually in a high dimensional space and loss landscape. You also need to talk more about back propagation and dropout, as those are critical to the training process as well.

8.5

Bravo! For a topic revolving around a concept from multivariable calculus, this article does an excellent job making it digestible for its target audience. I can’t give much credit for novelty given the modern abundance of resources explaining how AI works, but those commonly seen concepts are particularly well-articulated in this piece. If a newcomer trying to feel their way around AI reads this article, then I’m sure they’ll get a lot of use out of the mental image of being trapped in a snowstorm on a mountain.

P.S. Nabla is not a Greek letter; it is a symbol resembling the Greek letter delta that was first introduced in a mathematical context. I could make other nitpicks like that, but this one in particular annoyed me for some reason.