Project Technical Report
Project Description
CIS 4321 Spring 2020
Dr. Batarseh
In this project, you experience the full cycle of the data mining process. Below, I explain the different stages of the project.
Project Objectives
At the conclusion of this project assignment, participants should be able to:
· Write a project proposal
· Identify a dataset to mine
· Mine a dataset and write-up the insights gathered from the results
Requirements
For the final project in CIS4321 , you are going to mine a dataset and define a project scope, implementation and analysis. The dataset should be interesting, non-trivial and should have at least 6 attributes and on the order of 1000s (or more) instances. Some examples include data related to business, consumer behaviors, social-network information, etc. You could select a business problem that can be addressed through data mining. The following links are some sites to public datasets.
· www.archive.ics.uci.edu/ml/datasets.html
· www.pewinternet.org/datasets
· www.labrosa.ee.columbia.edu/millionsong
· www.wunderground.com/history
· www.yelp.com/academic_dataset
· www.developer.bestbuy.com/apis
Project Proposal (Due April 20th)
1. Project name (descriptive and concise).
2. Significance of the project
3. Dataset description
a. Describe the contents of the dataset.
b. Link to where it can be located
c. Dataset format
d. Provide a description of the attributes and target variable.
4. Implementation
a. What type of pre-processing, EDA and modeling you anticipate using?
5. Results
a. Why are the results useful?
b. Who would be interested in the results?
Dataset Mining
Your project should deliver on the functionality described in your project proposal. As part of this, you will need to perform data preprocessing (as needed), exploratory analysis of the dataset (including visualizations), modeling and testing and evaluation. You should also consider feature selection to help improve the predictive power (accuracy) of you approach.
Technical Report (Integrated in Jupyter Notebook).
You need to write a technical report describing your approach and findings. Your report must be written in Jupyter Notebook and interleaved with your python code. The report should be organized, clear, concise and easy to understand and follow. Your notebook should have the following sections at a minimum (in the order given below):
1. Introduction: This section must briefly describe the dataset you used and the data mining task you implemented. Briefly describe your findings.
2. Data Analysis: This section must provide details about the dataset. You must include:
a. Information about the dataset itself, e.g., the attributes and attribute types, the number of instances, and the attribute being used as the label.
b. Relevant summary statistics about the dataset.
c. Data visualizations highlighting important/interesting aspects of your dataset. Visualizations may include frequency distributions, comparisons of attributes (scatterplot, multiple frequency diagrams), box and whisker plots, etc. The goal is not to include all possible diagrams, but instead to select and highlight diagrams that provide insight about the dataset itself.
d. Note that this section must describe the above (in paragraph form) and not just provide diagrams and statistics. Also, each figure included must have a figure caption (Figure number and textual description) that is referenced from the text (e.g., “Figure 2 shows a frequency diagram for ...”). You should provide you source code using Jupyter Notebook and files.
3. Modeling Results: This section should describe the modeling approach you developed and its performance. Explain what techniques you used, briefly how you designed and implemented model, how you tested the predictive ability, and how well it performs.
4. Conclusion: Provide a conclusion of your project, including a short summary of the dataset you used and any of its inherent challenges, the modeling approach you developed and any ideas you have on ways to improve its performance
Project Submission
Submit your project to blackboard by the due date, no late submissions will be accepted.
You should submit a well-documented Jupyter Notebook and dataset files. Submit both .ipynb and .pdf files, name your files First_Lastname_FinalProject.ipynb.
Grading Guidelines
This assignment is worth 100 points + 10 points bonus. Your assignment will be evaluated based on a successful compilation and adherence to the program requirements. We will grade according to the following criteria:
· 15 pts for project proposal
· 50 pts for implementation
· 25 pts for relevance/originality of project
· 25 pts for technical rigor and complexity
· 35 pts for technical reporting in a Jupyter Notebook