Summer of Math Exposition

very well

Bit long? But overall very nice!

Very readable, Well-sequenced, Motivates before it defines, Concrete before abstract, Visual scaffolding. Exact solution/PINN prediction/side-by-side comparison, Honest about limitations. I’ve learned about PINN without knowing anything about it earlier.

As I understand it the core idea is that normal neural networks learn purely from data. But for many scientific problems, data is scarce or expensive to collect, while we often already know the governing physical law (a differential equation). PINNs exploit that: instead of relying only on labeled data, you also penalize the network whenever its predictions violate a known physics equation. PINNs shine when you need to combine physics + sparse data.

Very complete walkthrough, with good motivation and explanation.

This was a very informative, and a pleasant read.

The article doesn’t hold my hand with the math, and the math is relatively simple to follow anyway. At first I thought this would be an intimidating article about Neural Networks, but since this was physics-based, like the article mentions, we learn HOW this works as well as WHAT falls short.

This is a valuable resource introducing PINNS to people that are interested in solutions to problems without needing to know a ton of mathematics or physics to understand it. The sources/links provided are handy when we want to learn more about an aspect of this article/model.

I can also appreciate that this PINN model is still only a tool next to lots of other neural networks and other numerical methods, and should be taught its perks and shortcomings.

I found the article incredibly interesting and gives plenty of additional sources to follow up on your own research. And that’s the point/purpose of some articles I feel.

Interesting variation on Neural nets. I wonder what best practical example problems you could show as a follow up? Maybe give some examples of where this is used now in software?

You seem inconsistent in your article’s prerequisites. “Not an intro to neural networks or solving PDEs”, you write, then introduce partial derivatives as if new. I expect the audience that already knows, or could quickly learn, neural networks would also know well how derivatives and partial derivatives work, even if they can’t solve PDEs. You clearly tried to write a hook at the start, but I don’t find it compelling. Here’s my attempt at a better one: “Who doesn’t love a good neural network? They’ve gotten remarkably good at learning patterns from data. With an expressive architecture, enough compute, and a big pile of examples, they can learn functions that would be hard to intuit, let alone write explicitly. But is a pile of examples the best we can give it? What if we already know general facts about the function we’re trying to learn?” It might be instructive to compare how a PINN converges against how a conventional ANN (with training temperatures sampled from the analytic solution) converges. Show the respective shapes/plots at a couple stages early in training. Maybe they’ll look about the same. Maybe the PINN’ll be smoother or faster somehow. Good and distinctive explanation overall. I finally know what a PINN really is!

Where you “define a physics loss,” what is the N? I think there’s something interesting happening over here (in fact, would it be fair to say that a major contribution of the technique is in figuring out how to translate PDE into a loss on a real network?) This seems like something worth emphasizing (or clarifying, if I’m misunderstanding).

The topic as a brief introduction to PINNs made sense but the mathematical specifics could use work. For example, in $\hat y = f_\theta (x)$ it is never explained at a high level what $f$ is with respect to the sets it maps between as well as its goals. Just defining a loss function as well “training” as some process is not specific enough for this level of explanation. Additionally, the discussion about the derivatives and motivating the heat equation seemed backwards and unclear what the added benefit was. Finally, more discussion of error must be given if this is being approached from a scientific computing lens. Just the absolute and training errors are not sufficient especially for the heat equation. Relative error, error order in space and time, as well as error in PINN implementation and FDM or FEM. It would have been nice to investigate the computational cost too.

Nice article! It really helps with understanding how PINNs work.