Interpreting Neural Networks through the Polytope Lens

The paper presents a new approach called the polytope lens to interpret neural networks' behavior and understand what they have learned. The polytope lens uses the way that piecewise linear activation functions partition the activation space into discrete polytopes to identify monosemantic regions of activation space and semantic boundaries.

The polytope lens can help businesses understand why their neural networks make certain decisions and improve their interpretability. It can also guide the design of better neural networks by enabling researchers to identify and correct problems with their models.