Feedback | SoME

But what does entropy mean and why does AI rely on it?

Audience:

Tags: information-theory

This video is about information theory which explains the concepts of “information” and “entropy” from first-principles. It dives into questions like “where does the equation for information come from?”, “what does entropy really mean?”. It aims to provide a strong intuition which can help you deeply understand these concepts from the world of information theory. The video also goes into conceptual details to answer questions such as why does entropy come up in the world of artificial intelligence (AI), and why do AI models such as neural networks rely on it so often? The ideal audience is undergraduate and graduates in maths, physics, computer science or a related field.

Analytics

6.52 Overall score*

52 Rank

18 Votes

9 Comments

Comments

3.8

Could use some more motivation rather than resorting to the viewer’s previous familiarity with the topic. It also seems like quite a basic topic for an undergraduate/graduate audience, but I liked the examples you used.

Amazing video! Well explained with the perfect balance between intuition and fundamental definitions. The only improvement would be to remove the tiny glitches you have in your animations, but otherwise very well written and organized video!

Clear exposition; compelling presentation; for me “high unpredictability” or “low unpredictability” don’t make intuitive sense; this part could have been explained better

6.4

there was a good connection made b/w entropy, probabilty and little bit of computer science

I liked the motivation for the logarithm in the formula, where each bit allows you to reduce the number of possible states by half.

6.6

Nice presentation although I think this topic is very overdone and there are many many videos about this already so it’s hard to add something new to the conversation. I really like the background music.

Initial Thoughts

Well done! The topic seemed a little daunting at first, but you did a good job of keeping the explanation simple. This was a new concept to me, and I came away from your video feeling like I understood the basics of it.

What Went Well

Both the audio and visual explanations were simple and easy to follow. I liked the visual for bits of information, and I thought the card example at the beginning was a really good one to start off with to show how information and probabilities are related.

Ideas for Improvement

One thing I thought could be improved is the text in the visuals. Sometimes the text would start writing overtop of previous text before the other text had faded. Other times the text would transform into almost the same text as it was before, and those changes were subtle enough that I wasn’t sure what changed, so I had to rewind slightly to make sure I wasn’t missing anything.

Clear and concise. The examples of dartboard and cards are nicely used to explain formula and theoretical concepts. I liked it.

Very clear explanations and flow of the whole video, which made it easy to understand the concept of entropy and why it is important!