Cognitive Advisory Agent

Abstract

Machine Learning workflow involves extracting data from a data lake, training the model, predicting future behavior, and finally saving the results. At times, the initial dataset given to train the model may require more datapoints in order to perform better. Merging features from similar dataset or appending additional features to existing dataset may help improve trained model performance. However, merging data is a complex operation. While the state-of-the-art has existing works in resolving the merging of datasets, there is a need for a cognitive system to performs this task efficiently. In this work-in-progress paper, we propose a novel cognitive advisory agent, that dynamically resolves the above challenges to enhance model performance. It provides actionable items to improve the performance of the model. The agent performs reinforcement learning in order to assess the merged feature dataset. We demonstrate the efficacy of our method on a real-world use case.