Explainability in Neural Networks
Explainability in neural networks for the benefit of physics simulations.
Unfortunately, many of us have many of us have found ourselves in projects where we had to decide between selecting a more accurate model or a model that was physically interpretable to the physics we were trying to simulate. For example, this could be random forests verses neural networks.
Back then, I looked at a ‘poor man approach’ to solving this problem:
- Complete training for the NN
- Hold all inputs at their mean values
- Vary each input, one at a time, through entire range (store the predictions)
- Take one standard deviation from the mean in the predicted results for each input
The result was a crude way to gather insight on the sensitivity of each input on the resulting predictions.
Now, I found this! Enjoy.