Towards Feature Engineering with Human and AI’s Knowledge: Understanding Data Science Practitioners’ Perceptions in Human&AI-Assisted Feature Engineering Design

Abstract

As AI technology continues to advance, the importance of human-AI collaboration becomes increasingly evident, with numerous studies exploring its potential in various felds. One vital feld is data science, including feature engineering (FE), where both human ingenuity and AI capabilities play pivotal roles. Despite the existence of AI-generated recommendations for FE, there remains a limited understanding of how to efectively integrate and utilize humans’ and AI’s knowledge. To address this gap, we design a readily-usable prototype, human&AI-assisted FE in Jupyter notebooks. It harnesses the strengths of humans and AI to provide feature suggestions to users, seamlessly integrating these recommendations into practical workfows. Using the prototype as a research probe, we conducted an exploratory study to gain valuable insights into data science practitioners’ perceptions, usage patterns, and their potential needs when presented with feature suggestions from both humans and AI. Through qualitative analysis, we discovered that the “Creator” of the feature (i.e., AI or human) signifcantly infuences users’ feature selection, and the semantic clarity of the suggested feature greatly impacts its adoption rate. Furthermore, our fndings indicate that users perceive both diferences and complementarity between features generated by humans and those generated by AI. Lastly, based on our study results, we derived a set of design recommendations for future human&AI FE design. Our fndings show the collaborative potential between humans and AI in the feld of FE.