Summer of Math Exposition

Presented by 3Blue1Brown 3blue1brown

The Art of T{e,a}sting Cookies

Audience:

Tags: statisticshypothesis-testing

Use statistical hypothesis testing to decide on which cookie to eat next! The aim of this video is to explain in simple terms the concept of safe anytime hypothesis testing that has been popularised in the last few years through the concept of e-values (https://en.wikipedia.org/wiki/E-values). Hypothesis tests are a way to use the data at hand to make a decision on the average behavior of the data. Here we take the example of preference data (like or dislike and decide on which we prefer on average). Hypothesis tests are a core tool for scientists as it allows one to assess if some observed phenomenon is due to chance or if there is really something to understand from the data. The main idea of anytime hypothesis tests compared to usual hypothesis tests is that we can collect data sequentially and decide to stop whenever we want and still have a controlled probability of error, whatever the reason for stopping. This contrasts with the usual tests which are in general very strict on the methodology one has to use. This video's main target are undergraduate with some basic understanding of what is a random variable, and expectation... However, we tried to make it so that the main concepts are accessible to a broader audience.


Analytics

6.5 Overall score*
54 Rank
19 Votes
16 Comments

Comments

9

I liked this video. I am biased because I know the topic but I think the authors made good use of the animation.

8.1

This is an easily digestible explainer with a toy example regarding cookies. Would have liked a real-world example as well to really motivate using this sort of math and stats. Great animations and voice-over.

7.5

Great animation and clear explanation, although the video does speed up a bit near the part where batches are created.

5

Cute and friendly animations, and a fun demo of the concept.

I would have liked to see more emphasis on the qualitative differences between alternate statistics, to see the “big picture” better.

7

It’s a little slow. Somewhat awkward pauses.

6.5

This video acts as a good introduction to statistics and probability. However, I feel it does not use many of the advantages intrinsic to web video; rather it seems like it would work much better as a lecture or a section of a textbook. That said, it indeed seems to be a good learning resource.

4

I didn’t expect martingale would appear. Very interesting.

5.2

It’s a good explanation, maybe it tries to cover too much topics in a few minutes since it’s starts from the beginning so maybe some students get lost in the end.

5

I would have liked an explanation of why the false positive rate is additive. I struggled a lot with probability in school because I find it extremely unintuitive when you are or aren’t allowed to add or multiply things together.

7.3

This was a very good video. There was a clear structure and I, having never heard of e-values, think I know for which kind of applications I might look more deeply into them. However, I did not understand all of the concepts you laid out, even after rewatching some sections. When you introduce the notion of just eating more, I don’t see how exactly the false positive ratio is 0.1. I would expect that if the two tests contradict each other, I would’t yet draw a conclusion or give prevalence to the second or some other, more complex strategy. When updating evidence later, what does it mean for my test if M_n crosses the line? Do I reject H_0? I don’t recall you ever stating how I evaluate the new test. How does the fact that my test gets more reliable with more samples translate into this framework? It would also have been interesting to mention a scenario where the “default” test is preferable to the one you described, though that’s an add-on, not a critique. Nevertheless, the video was instructive and illustrated that there are several ways to do a hypothesis test.

7.3

This video is very clear when defining terms to make sure everyone is on the same page from the start. Since I am not a statistics major, I found it very helpful to go through all the vocabulary. I thought that the jump to threshold testing was a little fast, but I could see this being useful for a statistics student. Great video (also raise the speaker volume a bit because the speaker was hard to hear, especially with the music)

7.9

Omg I need to watch this bit about martingale again

3

Very incomplite explantion. some unessary details, likw the alis and bob framing. the ruls are not cleare, and chaionging (What are H0 and H1)…

7.5

The author used a very good “hook” to get the viewer’s interest: cookies. I wish a little more explanation were given: Why was the scale in multiples of 10? What was happening when the bell curve was shown on its side? The graphics were clear.

5.2

200 cookies make cookie monster sick? :) Clever cartoon, but I think this needs some practical uses of how this is used.

6

I very much liked the video’s way of discussing an abstract concept by easing into it, starting with a relatable example involving cookie preferences. Some of the explanations are pretty unclear, however. I still don’t know why that one horizontal chart with the histogram is shown as logarithmic