Usama Fayyad,
Data Mining Grand Challenges,
In Proceedings of the 8th Pacific-Asia Conference on Advances
in Knowledge Discovery and Data Mining (PAKDD 2004),
Sydney, Australia, No. 3056 in Lecture Notes in Computer Science, p. 2, Springer-Verlag, May 26-28 2004. (keynote speech)The past two decades has seen a huge wave of computational systems for the ``digitization'' of business operations from ERP, to manufacturing, to systems for customer interactions. These systems increased the throughput and efficiency of conducting ldquotransactionsrdquo and resulted in an unprecedented build-up of data captured from these systems. The paradoxical reality that most organizations face today is that they have more data about every aspect of their operations and customers, yet they find themselves with an ever diminishing understanding of either. Data Mining has received much attention as a technology that can possibly bridge the gap between data and knowledge. While some interesting progress has been achieved over the past few years, especially when it comes to techniques and scalable algorithms, very few organizations have managed to bene t from the technology. Despite the recent advances, some major hurdles exist on the road to the needed evolution. Furthermore, most technical research work does not appear to be directed at these challenges, nor does it appear to be aware of their nature. This talk will cover these challenges and present them in both the technical and the business context. The exposition will cover deep technical research questions, practical application considerations, and social/economic considerations. The talk will draw on illustrative examples from scienti c data analysis, commercial applications of data mining in understanding customer interaction data, and considerations of coupling data mining technology within database management of systems. Of particular interest is the business challenge of how to make the technology really work in practice. There are many unsolved deep technical research problems in this eld and we conclude by covering a sampling of these.