Data Miners Come Clean

The biggest challenges faced by data miners are dirty data, data access, and explaining data mining to others, according to a survey of more than two hundred professionals worldwide, by Rexer Analytics.

Predictive modeling and segmentation / clustering are the most common types of analyses conducted (by 89% and 77% of respondents, respectively); and the most commonly used algorithms are regression (79%), decision trees (77%) and cluster analysis (72%).

Rexer's first annual survey of the data mining community was conducted in the spring and looked at the experiences, priorities, and challenges of the sector via an online survey. Fully 100 of the 314 responses were removed from the key analysis as they came from employees of software firms that produce data mining tools, and were considered a possible source of bias. 35 different countries were represented.

SPSS, SPSS Clementine, and SAS were the three most frequently utilized analytic tools in 2006, each by more than 40% of data miners. 45% also used their own code. Comments on planned use this year show increasing interest in the Oracle Data Mining tool, and decreasing interest in C4.5/C5.0/See5.

The most important factors in selection of tools are the dependability and stability of software, followed by the ability to handle large data sets, and then data manipulation capabilities.

A complete analysis of the survey will be presented at the fall SPSS Directions Conference and the Oracle BIWA Summit. Analytics and CRM firm Rexer is online at www.RexerAnalytics.com.

All articles 2006-23 written and edited by Mel Crowther and/or Nick Thomas, 2024- by Nick Thomas, unless otherwise stated.