Analytical thinking

(These specs may be changed as required by the instructor)

For the project, you will work on a problem of your choosing that is interesting, significant, and relevant to apply the data mining algorithms and techniques you learned in the class to some real-world problems. You will apply the elements and standards of analytical thinking approaches to solve your problem. You will have great latitude in what you choose to work on, so take advantage of this opportunity to make a big impact!

The primary requirements of the project are:

· Your project must use some non-trivial data that you collects. Some interesting web sites are listed below including data sources.

· Your project must implement more than one algorithm that is presented in class or is closely related to the course topic (e.g., classification, clustering, association rule, anomaly detection and simulation). Different models may either discover different patterns or confirm the same pattern.

The interesting links are

Each student should post a two-page project proposal (in double spaced 12-font size of Times New Roman) and Logic of Problem (a PDF file) prepared from the Critical Thinking Community website to Proposal dropbox . Provide a brief, descriptive name of your project. Your name should be something memorable!

In the proposal, you should address the following issues by applying the elements and standards of analytical thinking approaches:

· What are the problems to be addressed? If selected from the Kaggle, it should be an Active contest.

· What data do you plan to use?

· Are any pre-processing steps necessary?

· What data mining (machine learning) approach (Supervised or Unsupervised) do you plan to use? Justify why Supervised or Unsupervised.

1) Data Exploration

For the second project checkpoint, you must have collected a significant portion of the data that your project will ultimately use. You should describe the data that you will be using for your project in 2-3 pages. You should show figures or tables highlighting the key characteristics of the data. For example, you might show the following aspects for each file:

· The source of your data

· Number of records

· Number of attributes and a brief descriptions of the file

· Data type and description for each attribute

· Missing values

· Outliers

2) Logic of Problem

Update your Logic of the Problem, and summarize your update including the following:

· What/why you have updated during this data exploration phase

· Do you have any suggestions to the Logic of Problem as an Analytical Thinking tool?

( 1 )

Post your 2-3-page (in double spaced 12-font size of Times New Roman) report and updated Logic of Problem to the checkpoint 2 dropbox.

1) Data Preprocessing and Preliminary Learning Algorithms

For the third project checkpoint, you will describe how you have preprocessed the data using such techniques as aggregation, feature selection, feature/attribute selection and discretization. And by this checkpoint, you must have tried building at least your core- learning algorithm. Describe what algorithms you have tried and discuss any findings and/or problems.

2 Logic of Problem

Update your Logic of the Problem, and summarize your update including the following:

· What/why you have updated during this data preprocessing & preliminary learning algorithm phase

· Do you have any suggestions to the Logic of Problem as an Analytical Thinking tool?

Post your 2-3-page (in double spaced 12-font size of Times New Roman) report and updated Logic of Problem to the checkpoint 3 dropbox.

· Data Exploration

· Methodology

· Should have chosen at least two data mining techniques to build models

· What performance measures used

· Conclusions

· Any patterns confirmed by different models?

· Summarize your problem and discuss what conclusions you have drawn from the models you have built

· References

· All data files used in the project