The Geometry of AI Minds in Superposition (By Anthropic): Visually Explained
What truly goes on in the inner thoughts of AI language models like ChatGPT? We will explore the strange geometric structures that concepts in their mind are arranged in. This video is based on this research paper: https://transformer-circuits.pub/2022/toy_model/index.html It is also related to my other video on AI mind reading based on recent OpenAI and Anthropic work: https://youtu.be/krINuMZhJmU
Analytics
Comments
I really enjoyed the format of an old fashion psa. I found I enjoyed the more when I slightly sped up the video speed to 1.25. Speeding up the video, at least for me, did not seem to overly hurt the video format, while also helping smooth over the videos flow. Overall, this video was beautifully polished.
7.5Little bit scary but great
5.6Expositions of neural networks are common right now. Yet the author presented the content in quite a novel way. The video is simple and elegant and could be shared with people with just a small background in Math. Concepts such as super position, interference and privileged basis are quite new to me. I thank the author for a well made video.
7.2This video is interesting and well-made. The discussion is quite well motivated. The mathematical content is described well. The video follows the source paper by Elhage et al. quite closely, so the video itself is totally novel (it's not clear if the video creator is one of the authors). Educationally, the "balancing" of the information is slightly off, so some simple concepts (the dot product, for instance) are explained in depth, but more complicated concepts are rushed through more. The final video is for some reason quite grainy (beyond the deliberate 1950s aesthetic) and JPEG compression artefacts are very obvious when watching it in full screen mode. The presentation was quite good otherwise.
7Speaking as a sentient Perl script I was very impressed with how you AI guys are getting on. Very good marks for novelty, and no significant criticisms on any other areas. Keep it up!
7Goal Orientation: 5/10 Novelty: 6/10 Thought-Provoking: 2/10 Comprehensibility: 7/10 Technological: 6/10 Overall Average: 5.2/10
4.8This isn't a real voice, isn't it? Creepy.
5.4I like the emphasis on the issue of interference vs. true similarity. I hadn't appreciated before now how the choice of where to squeeze in unrelated features may be about minimizing interference from what was actually observed in the training data. Hopefully I got the right idea there :)
5.9I love this video, super creative refreshing format and presentation style, and explained very well. I usually hate ai generated or professional sounding voices in YouTube but this one was done well. Topic was very interesting and insightful as well. Some small details maybe could be explained a little more, and more motivation or examples and analogies of why we should care or applications might be good
7.9