Comparing Apples and Oranges: is Stitching Similarity a Load of Spheres?

Model stitching is used in the literature to assess the extent to which models capture similar information.The intuition is that if two models classify samples in the same way, they must be capturing the same information.We construct a series of experiments to show that two models can reach the same conclusion but represent very different information. We therefore argue that unlike previously claimed, stitching cannot reflect the extent to which models represent or capture similar information. This paper draws the community's attention to the need to correctly interpret the results of such functional similarity measures and highlights the need for similarity measures that capture informational similarity.