Contextual intelligence for unified data governance

Abstract

Current data governance techniques are very labor-intensive, as teams of data stewards typically rely on best practices to transform business policies into governance rules. As data plays an increasingly key role in today's data-driven enterprises, current approaches do not scale to the complexity and variety present in the data ecosystem of an enterprise as an increasing number of data requirements, use cases, applications, tools and systems come into play. We believe techniques from artificial intelligence and machine learning have potential to improve discoverability, quality and compliance in data governance. In this paper, we propose a framework for 'contextual intelligence', where we argue for (1) collecting and integrating contextual metadata from variety of sources to establish a trusted unified repository of contextual data use across users and applications, and (2) applying machine learning and artificial intelligence techniques over this rich contextual metadata to improve discoverability, quality and compliance in governance practices. We propose an architecture that unifies governance across several systems, with a graph serving as a core repository of contextual metadata, accurately representing data usage across the enterprise and facilitating machine learning, We demonstrate how our approach can enable ML-based recommendations in support of governance best practices.