Towards Interpretation of Node Embeddings

Abstract

Recently there have been a large number of studies on embedding large-scale information networks using low-dimensional, neighborhood and community aware node representations. Though the performance of these embedding models have been better than traditional methods for graph mining applications, little is known about what these representations encode, or why a particular node representation works better for certain tasks. Our work presented here constitutes the first step in decoding the black-box of vector embeddings of nodes by evaluating their effectiveness in encoding elementary properties of a node such as page rank, degree, closeness centrality, clustering coefficient, etc. We believe that a node representation is effective for an application only if it encodes the application-specific elementary properties of nodes. To unpack the elementary properties encoded in a node representation, we evaluate the representations on the accuracy with which they can model each of these properties. Our extensive study of three state-of-the-art node representation models (DeepWalk, node2vec and LINE) on four different tasks and six diverse graphs reveal that node2vec and LINE best encode the network properties of sparse and dense graphs respectively. We correlate the model performance obtained for elementary property prediction tasks with the high-level downstream applications such as link prediction and node classification, and visualize the task performance vector of each model to understand the semantic similarity between the embeddings learned by various models. Our first study of the node embedding models for outlier detection reveals that node2vec and DeepWalk identify outliers well for sparse and dense graphs respectively. Our analysis highlights that the proposed elementary property prediction tasks help in unearthing the important features responsible for the given node embedding model to perform well for a given downstream task. This understanding would facilitate in picking the right model for a given downstream task.