Electronic commerce is emerging as the killer domain for data mining technology. Is there support for such a bold statement? Data mining technologies have been around for decades, without moving significantly beyond the domain of computer scientists, statisticians, and hard-core business analysts. Why are electronic commerce systems any different from other data mining applications?
In his book Crossing the Chasm (Moore & McKenna 1995), Moore writes, “There were too many obstacles to its adoption . . . inability to integrate it easily into existing systems, no established design methodologies, and lack of people trained in how to implement it . . . ” (p. 23). What was “it”? Artificial intelligence technology, as a product. Data mining shares many traits with AI technologies in general, so we should be concerned that they do not share the same business fate.1
Notwithstanding several notable successes, data mining projects remain in the realm of research: high potential reward, accompanied by high risk. The risk stems from several sources. It has been reported by many (Langley & Simon 1995, Piatetsky-Shapiro, Brachman, Khabaza, Kloesgen & Simoudis 1996), and has been our experience, that the “data mining,” or algorithmic modeling phase of the knowledge discovery process occupies at most 20% of the effort in a data mining project. Unfortunately, the other 80% contains several substantial hurdles that without heroic effort may block the successful completion of the project.