PYTHON

CSCI 333.01W

Final Project (10% of Final Grade)

Part I

50 points + 10 bonus points

Due date: 4/13/2020 Tue. by 11:59pm

Overview

The Final project consists of two Parts. This is Part 1.

Part 2 (50 points) will be available next week.

The goal of project Part 1 is

1) Get familiar with Jupyter Notebook with Anaconda

2) SciPy.stats and Linear Regression

1. Jupyter Notebook

When we installed anaconda, Jupyter Notebook is one of the apps that’s got installed. It has direct access to all the packages/libraries installed with anaconda.

Jupyter Notebook is “an open-source web-based application which allows you to create and share documents containing live code, equations, visualizations, and narrative text”. This notebook not only supports Python but also has support for over 40 programming languages.

It’s the top popular Python Open Source IDE for Data Scientists.

How to open Jupyter Notebook on your computer:

· Click Start on the bottom left corner, browse to Anaconda3 -> click Jupyter Notebook (Anaconda3), to open it.

· There are other ways to open Jupyter Notebook …

How to create a new notebook:

Watch the following video for a quick review on how to use Jupyter Notebook: https://www.youtube.com/watch?v=3C9E2yPBw7s

For more information, check these two links, or search topics you’re interested online.

a) https://realpython.com/jupyter-notebook-introduction/

b) https://jupyter.org/

Use Jupyter Notebook for the rest of this project.

2. SciPy.stats and Linear Regression

Up to now, we learned and practiced with some Python Libraries for Data Science, such as NumPy, Pandas, Matplotlib. There are many, many more libraries that are not possible for us to go over in this class. I encourage you to investigate, learn, and practice more whenever you can, by studying books, or internet resources, most importantly, practice, practice, practice!

In this project Part 1, you are required to get your foot wet for scipy.stats module:

https://docs.scipy.org/doc/scipy/reference/stats.html

SciPy (Scientific Python) Library is widely used for engineering, science and math in Python. This library’s linregress function (from the scipy.stats module) performs simple linear regression:

https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.linregress.html

Read the following topics:

The first three topics covers how to create lots of data (numpy), how to plot data to visualize them (matplotlib). The 4th topic uses scipy.stats module linregress() function to fit the data with a line of Linear Regression. After calling linregress(), plug the resulting slope and intercept into the y=mx+b equation to make predictions, and see how good the prediction is. (m is the line’s slope, b is the line’s intercept with the y-axis at x=0, x is the independent variable, y is the dependent variable). In simple linear regression, y is the predicted value for a given x.

1. https://www.w3schools.com/python/python_ml_data_distribution.asp

2. https://www.w3schools.com/python/python_ml_normal_data_distribution.asp

3. https://www.w3schools.com/python/python_ml_scatterplot.asp

4. https://www.w3schools.com/python/python_ml_linear_regression.asp

Run the sample codes from these topics in Jupyter Notepad to get familiar with the IDE.

After reading and practicing these topics, solve the following Task 1 problem. Save your program as FinalProjectPart1_YourFirstLastName.ipynb. (Submission of your program in .ipynb format will get extra 5 bonus points. This is to encourage your effort to use Jupyter Notebook).

Glance at “What to Submit" when you start working on the task so that you know what information to provide for the tasks.

What to Submit:

1. One doc file “csci333-FinalProject-Part1-YourFirstLastName.docx", including the screenshot of the source code, and the outputs of your program(s).

2. Your python file (*.ipynb format) (.ipynb format will get 5 bonus points).

3. In well-defined programs, proper comments are required. For programs without comments, some points will be deducted.

4. Note: if some part of your program or code does not work, you can explain the status of the program or code and then attach your explanation and description in your word document. Partial credit will be given so do what you can and make sure you show your work.

Submission Example:

· csci333-FinalProject-Part1-YanLi.docx

· FinalProjectPart1Code-YanLi.ipynb

Part 1 Tasks (50 points):

1) (3 points each step)

a) Create 100,000 values of normal distribution with mean value at 10.0, standard deviation at 1.0

b) Draw a histogram with 100 bars

c) Explain your data

2) (3 points each step)

Follow the following code to initialize two arrays, x and y

import numpy

import matplotlib.pyplot as plt

from scipy import stats

# Creat two arrays: "x" and "y":

# 1) create a "temp" array of 50 evenly spaced points between -5 to 5 [-5,..,5]

temp = numpy.linspace(-5,5,50)

print(temp)

# 2) create a "x" array which is polynomial values (-1*temp+4) for each temp element

x = numpy.polyval([-1, 4], temp)

print(x)

# 3) add some noise to x to create a "y" array

y = x + numpy.random.randn(n)

print(y)

Continue with your code:

a) Draw a scatter plot to view your x, y data, does it look like you can use linear regression to fit the data and predict future values?

b) Use scipy function to derive the parameters (slope, intercept) for the linear regression model.

c) Use the derived parameters to create a function for the model for prediction.

d) Then use this function to derive new y values for each x element.

e) On a plot, draw the original scatter plot (x, y), and draw the line of linear regression (x, and new y).

f) What’s the r-squared value? Explain how well your data fits in the linear regression, according to the r-squared value.

g) Predict future value for x = 4, or a number you’d like to pick.

h) Visually examine the predicted y result on the plot, does it seem like a good fit? Is your model reasonably good?

Grading Rubric (total 60 points):

· 33 points for finishing part1 question 1) and 2) (a)-(h).

· 3 points for defining function(s) in your code

· 4 points for appropriate comments / Markdown

· 5 points for a runnable python program with correct data visualization

· 5 points for screenshots of the program(s) and output results

· 5 bonus points for source code submitted in .ipynb format

· 5 bonus points for on-time submission

2