Foundations Of Geographic Information Systems Assignment Package

--- title: "Untitled" author: "Michael Silano" date: "8/19/2019" output: html_document --- ## Objectives The objective of today's hands-on activity is to provide a basic overview of R. For each topic covered the first part will be directed – you will follow the prescribed sequence of R commands in order to familiarize yourself with what they do. The second part of each topic will ask you to apply these R commands. ## Assignment For this activity you will turn in a Rmd file (prefered) or R Script file that shows your work. If you turn in a Rmd file, you should check that the code "knits" before submitting! ``` 3+12 ## addition 5 - pi ## subtraction 2*8 ## multiplication 14/5 ## division 14 %/% 5 ## integer division 14 %% 5 ## modulus (a.k.a. remainder) sqrt(25) ## square root z = 5 ## assignment to a variable z ## value of a variable y = sqrt(36) ## square root y ``` The window should show ```{r, echo=TRUE} 3+12 ## addition 5 - pi ## subtraction 2*8 ## multiplication 14/5 ## division 14 %/% 5 ## integer division 14 %% 5 ## modulus (a.k.a. remainder) sqrt(25) ## square root z = 5 ## assignment to a variable z ## value of a variable y = sqrt(36) ## square root y ``` The number [1] before the answers just means that this item is the first element of a vector (vectors can be thought of as a collection of related values, such as a column in a data table). From these examples we can see a few things about R. First is that it knows the value of common constants (e.g. π) and can use them like numbers. Second, that we can assign both constants and the results of functions to variables and we can see the value of a variable by entering it at the command prompt. This also demonstrates “sqrt”, the square root function, and that values are passed to functions within parenthesis. Typically, values passed to function are referred to as the arguments of the functions. Make sure you understand what each operator (+, -, *, /, %%, %/%) does before you proceed. Some of the other common mathematical operators in R that you should try running are: ``` 10^2 ## power exp(1) ## exponential log(10) ## natural logarithm (i.e. ln) log10(10) ## log base 10 log2(10) ## log base 2 sin(pi/2) ## sine cos(pi) ## cosine tan(pi/4) ## tangent asin(0.5) ## arc-sine acos(0.5) ## arc-cosine atan(1)*180/pi ## arc-tangent atan2(-1,-1)*180/pi ## arc-tangent (alternate version) factorial(10) ## factorial ``` Note that the atan2 function takes TWO arguments. In R there are many functions that can take more than one argument, and these arguments are always separated by commas. To get information on a function in R you can precede the function name with a ?. So if we wanted to look up what the two arguments of the atan2 function are, we could type ``` ?atan2 ``` The rest of this activity will often list the names of functions you might find useful, and ? can be used to find out about the syntax of these functions. If on the other hand you are looking for a command (e.g. because you've forgotten its name) you can use help.search. For example: ``` help.search("mantel") ``` This will return a list of relevant functions with the parenthesis after the name indicating what package it is in. help.search will only search packages you have already installed, not all the ones that exist -- you'll want to use CRAN to find new packages. Within RStudio you can also search for help with commands in the **Help** tab in the bottom-right window. This search window combines the functionality of both ? and help.search. Packages are installed using the ‘Install Packages’ button under the “Packages” tab or by using 'install.packages' function on the command line. Packages are loaded by clicking the check box next to a packages name or by using the 'library' function at the command line. Packages only need to be installed once but need to be loaded every time you start R (hint: you'll probably want to list library commands near the top of your script files [described below]) ### Questions: ``` 1. Evaluate the following: a. ln(1) b. ln(0) c. ln(e) d. ln(-5) e. -ln(5) f. ln(1/5) g. How does R represent when the output of a function is not a number? 2. Why are we multiplying the results of the atan and atan2 functions by 180/pi? 3. How is the atan2 function different from the atan function? 4. What is the difference between log and log10? 5. Given a right triangle with sides x = 5 and y = 13, calculate the length of the hypotenuse (show code) 6. If a population starts with a density of 5 individuals/hectare and grows exponentially at a growth rate r=0.04/year, what is the size of the population in π years? (show code) 7. Subtract the month you were born from the remainder left after you divide the year you were born by the day you were born (show code) ``` # R Scripts Now click on File>New>R Script to open up a script window. It is often useful to work on R from a script window because it provides a record of what you did in your analysis and can be reused for similar analyses. It is particularly essential for more complicated analyses. In the script window type ``` x = 1:10 x #y ``` Unlike at the command prompt nothing happens when you hit return at the end of a line. Highlight the code and hit the Run button. At the command line you should see ```{r} x = 1:10 x #y ``` Here the result, a vector of ten values from 1 to 10, demonstrates the basic R syntax for creating a sequence of numbers. This example also demonstrates that the comment character '#' also works in scripts. Putting comments in scripts is very useful for remembering what you did when you come back to a file later. For the rest of this activity I'll use boxes to indicate text to be typed in and run, and will use > to indicate that it should be typed on the command prompt and no prompt to indicate that you probably want to type it in a script instead. There are a number of other ways of generating useful sequences besides the ':' syntax ```{r} seq(1,10,by=0.5) seq(1,by=0.5,length=10) rep(1,10) x=seq(0,3,by=0.01) ``` In all cases you need to provide the first value in a sequence (the first argument to seq and rep), and after that you need to provide some combination of step size (by), length, and finishing value. Most functions in R can be applied to vectors of data, not just individual data points. Indeed, many only make sense when applied to vectors, such as the following that calculate sums, first differences, and cumulative sums. ```{r} sum(1:10) ## sum up all values in a vector diff(1:10) ## calculate the differences between adjacent values in a vector cumsum(1:10) ## cumulative sum of values in a vector prod(1:10) ## product of values in a vector ``` ``` Questions: 8. Describe the difference in output between sum and cumsum. 9. Generate a sequence of even numbers from -6 to 6 10. Generate a sequence of values from -4.8 to -3.43 that is length 8 (show code) a. What is the difference between values in this sequence? b. What is the sum of the exponential of this sequence? 11. Calculate a second difference [a difference of differences] for the sequence 1:10 (show code) ``` ## Loading and Saving Data There are a number of ways to get information into and out of R, but the most simple is in ASCII text formats, such as tab-delimited (.txt) or comma-separated (.csv). It’s usually straightforward to export data in one of these formats from most any program (e.g. Excel). Lets begin by opening the “Lab1_frogs.txt” file in the "data" folder. This data and some of the examples below come from Ben Bolker's handy book "Ecological Models and Data in R". Note that if you just click on the file from within the **Files** tab, or try to open the file from File > "Open File..."" that this opens the file as a text file, but it it doesn't load the data into R in any way we can use it. Instead, we'll want to run the following command ```{r} dat = read.table("data/Lab1_frogs.txt",header=TRUE) ``` The second variable “header=TRUE” informs R that your data file has column headers that should be read rather than treated as just another line of data. For the above command to work **R has to be looking at the correct folder**. You can find out what folder R is currently looking at (its “working directory”) using “getwd” and you can change that directory using “setwd” or within the “Files” window tab under **More > Set As Working Directory**. <table border="1" bgcolor="yellow"><tr><td> <h3>AUTO-COMPLETE</h3> <p>Be aware that RStudio has the capacity to auto-complete function names, function arguments, and file names. So, for example, if you type ‘read.t’ and then hit TAB, RStudio will finish typing read.table and it would also show what information you can specify for the read.table function. If you type read.table( and then hit TAB, RStudio will allow you to select the function argument that you want to fill in. If you type read.table(“ and then hit TAB, RStudio will show you the files in your current working directory and allow you to select one. If there are a lot of files in the directory, you can start typing the file name you want and then hit TAB again and RStudio will limit what it shows to just those files that match what you’ve typed so far. The same ‘search’ functionality applies to file names as well. So, if you type just ‘re’ and then hit TAB, then RStudio will show you all function that begin with re. </p> </td></tr></table> For saving ASCII data there is an equivalent command “write.table”. Type ?read.table and ?write.table to learn more about these functions. While read.table and write.table can read and write CSV (comma separated value) files by just specifying the ‘sep’ argument as sep = “,” (i.e. a comma in quotes), there are also predefined functions read.csv and write.csv. ```{r} write.table(dat,"my_frogs.csv",row.names=FALSE,sep=",") ## save as CSV ``` R also has a built-in binary data format that is good for saving results for later use in R and can store any number of data types of different shapes and sizes (note just single tables). The function “save” is used to save data ```{r} save(dat,x,y,z,file="Lab1.RData") ``` This command saves any number of data objects, separated by commas, that come before the ‘file=’ argument, which tells the function the name of the file you want to write to. This data can then be reloaded at a later time, or on a different computer, using “load” ```{r} load("Lab1.RData") ``` There is also a “save.image” function that saves every variable you have defined so far ```{r} save.image("Lab1_all.RData") ``` ## Data types and dimensions One of the first things you’ll do with any data set when you first load it up is some basic checks to see what you are dealing with. Typing the variable name will show you its contents, but if you just loaded up something with a million entries then you’ll sit for a long time as R lists every number on the screen. The function class will tell you the type of data you’ve just loaded. ```{r} class(dat) ``` In this case the data is in a “data.frame”, which is like a matrix but can also contain non-numeric data. The basic (or atomic) data types in R are integer, numeric (decimal), logical (TRUE/FALSE), factors, and character. Character data in R is usually displayed in double quotes to indicate that it is character data (e.g. the character “1” rather than the number 1). When writing character data in R (e.g. file names in read.data) it is necessary to use double quotes as well so that R can tell the difference between character data and the names of variables and functions. By contrast, R usually reads character data from files correctly even if the data isn’t in quotes. In addition to the basic data types in R, there are a wide variety of derived data types built up from these basic types that are used for a wide range of purposes. A common example is the Date type, which can be useful for analyzing and plotting data through time, and which you can learn more about by looking at the help for **as.Date** and **strptime**. At the most basic level R organizes data into vectors of data of a given data type and each column of a data.frame consists of a vector. R also has a matrix data type, which must contain data of a single basic data type (usually numeric). It is important to be aware of data types because certain operations can only be applied to certain data types (e.g. you can multiply two matrices but not two data frames). ```{r} str(dat) ``` will tell you the basic structure of the data. From these we learn that there are four columns of data named “frogs”, “tadpoles”, “color”, “spots” and that there are 20 rows of data, and that the data is numeric for the first two, a factor for the third, and logical for the fourth. If we didn’t need all this information ```{r} names(dat) ``` will tell you the names of the columns in you data frame. ```{r} dim(dat) ``` **dim** will tell you the dimensions of the data, in this case [1] 20 4 which means we 20 rows and 4 columns. Each of these pieces of information is accessible individually using **nrow** and **ncol**. ```{r} nrow(dat) ncol(dat) ``` Note that dim will not work on a single vector (e.g. dim(x)) but that **length(x)** will tell you the length of a vector. ```{r} head(dat) tail(dat) ``` The functions **head** and **tail** show just the first and last few lines of a data set, respectively ```{r} summary(dat) ``` will give you basic summary statistics on a data set Data within vectors, matrices, and data frames can be accessed using [ ] notation. Subsets of data can also be accessed by specifying just rows, just columns, or ranges within either. These are often referred to as subscripts or indices and the first is the row number while the second is the column. ```{r} x[5] # select the 5th element only dat # select the entire data frame dat[5,1] # select the entry in the 5th row, 1st column dat[,2] # select all rows of the second column dat[1:5,] # select rows 1 through 5, all columns dat[6:10,2:3] # select rows 6 through 10, columns 2 and 3 ``` We can also refer to specific columns of data by name using the $ syntax ```{r} dat$frogs # show just the ‘frogs’ column dat$color[6:10] # show the 6th though 10th elements of the color column ``` In general, it is better to **access data by name**, rather than using the row and column numbers, because this makes your code easier to understand and debug, making the process of coding less error prone. It also makes it much easier to adapt your code to new situations or data sets, where the columns might not come in the same order, or the data might not have the same number of rows or columns. This highlights a more general point, that you should use variables to represent names and numbers, especially if those names and numbers are reused, rather than ‘hard coding’ numeric values into the code. Finally, there are functions for converting data from one data type to another ```{r} as.character(dat$color) as.numeric(dat$spots) as.logical(0:1) as.matrix(dat) ``` ``` 12. Show just the spots data as characters 13. Show the 3rd through 8th rows of the 1st though 3rd columns 14. Show the first 3 rows ``` ## Combining vectors There is a simple function **c( )** in R that “combines” vectors or numbers into a single vector. You use it like this: ```{r} x=c(1,7) x y=c(10:15,3,9) y c(x,y) ``` Vectors can also be used for indexing other vectors. For example: ```{r} y[x] ## return the 1st and 7th element of y ``` We can also combine vectors to build up data frames by “binding” them together either are rows or as columns ```{r} p = 1:10 q = 10:1 cbind(p,q) # bind as columns rbind(q,p) # bind as rows ``` **cbind** and **rbind** can also be applied to existing data frames, for example to add another column to an existing data frame or to take two data sets with the same columns and bind them together by row to make a larger data set. ``` 15. Create a character vector that contains the names of 4 super heros. 16. Show just the odd numbered rows in the frog data. Write this code for the GENERAL CASE (i.e. don’t just type c(1,3,5,…) but use functions that you learned in previous sections to set up the sequence. ``` ## Logical operators and indexing R can perform standard logical comparisons, which can be very useful for comparing and selecting data. It's important to know the syntax for the different logical operators, some of which are odd: > greater than < less than >= greater than or equal to <= less than or equal to == equal to (TWO equals signs...you were very close!) != not equal As a simple example you could compare individual numbers: ```{r} 1 > 3 5 < 7 4 >= 4 -11 <= pi log(1) == 0 exp(0) != 1 ``` You can also combine multiple logical operators using the symbols for ‘and’ (&) and ‘or’ ( | ) ```{r} w = 4 w > 0 & w < 10 w < 0 | w > 10 ``` You can also apply logical operators to vectors and matrices. When you type a "logical" expression like "y > x" in R you get a TRUE/FALSE answer of the same shape as the inputs. e.g.: ```{r} z = y>13 z ``` You will notice that by default logical operations are performed element-by-element. If you want to apply a logical test to a whole vector at a time you can use the function **any** to test if any of the values are true and **all** to test if all values are true ```{r} any(y>13) all(y>13) ``` You also need to know that logical vectors like "z" above can be used as indices for other vectors of the same length. Commonly, you'll use them as indices to one of the vectors that produced them. e.g.: ```{r} y[z] ``` Or, skipping the middleman "z": ```{r} y[y>13] ``` These simple comparisons can provide a powerful means for subsetting data. These comparisons can also be used in matrices and data frames the same way we were using sequences of row or column numbers above. For example, if you just wanted the rows where there were 3 or more frogs, you could type ```{r} dat[dat$frogs >= 3,] ``` R also has a built in function **subset** for doing this sort of subsetting that takes the data set as the first argument and the condition used for subsetting as the second argument. So the above could also be rewritten as ```{r} subset(dat,frogs >= 3) ``` **subset** also has an optional 3rd argument for just returning specific columns. So if you wanted to run the previous subset but only needed the columns tadpoles and spots you could run ```{r} subset(dat,frogs >= 3,c("tadpoles","spots")) ``` Note that when your data is characters you'll need double-quotes in your comparison. e.g. ```{r} a=c("north","south","east","west") a == "east" ``` Two more things about logical vectors. First, sometimes it's easier or necessary to have a list of the indices to the TRUE values only (e.g. when the list includes NA values). The R function "which" is just for this purpose: ```{r} which(y==3) # The seventh element of y equals 3 ``` And finally, sometimes TRUE/FALSE behave just like 0/1, which can be very useful. For example, this handy syntax: ```{r} sum(y>13) # two elements of y are >13 ``` Logicals don't work exactly like 0/1 in some situations, so be careful. You can always convert them explicitly with as.numeric( ) too if need be: ```{r} as.numeric(y>13) ``` ``` 17. For the frog data set: a. display just the rows where frogs have spots b. display just the rows where frogs are blue c. how many blue tadpoles are there? d. create a new object containing just the rows where there are between 3 and 5 tadpoles e. display just the rows where there are less than 2.5 red frogs f. display where either frogs do not have spots or there are more than 5 frogs ``` ## Plots, tables, and exploratory analysis Often understanding our data requires more than just being able to subset the raw data, but also the ability to summarize and visualize data. The **table** command can do basic tabulation and cross tabulation of data ```{r} table(dat$color) table(dat$color,dat$spots) ``` There are also a number of commands for calculating basic statistical measures ```{r} mean(dat$frogs) median(dat$tadpoles) var(dat$frogs) ## variance sd(dat$frogs) ## standard deviation cov(dat$frogs,dat$tadpoles) ## covariance cor(dat$frogs,dat$tadpoles) ## correllation quantile(dat$tadpoles,c(0.05,0.90)) ## 5% and 95% quantiles min(dat$frogs) ## smallest value max(dat$frogs) ## largest value ``` R also has a set of apply functions for applying any function to sets of values within a data structure. ```{r} apply(dat[,1:2],1,sum) # calculate sum of frogs & tadpoles by row (1st dimension) apply(dat[,1:2],2,sum) # calculate sum of frogs & tadpoles by column (2nd dimension) ``` The function *apply* will apply a function to either every row (dimension 1) or every column (dimension 2) of a matrix or data.frame. In this example the commands apply the “sum” function to the first two columns of the data (frogs & tadpoles) first calculated by row (the total number of individuals in each population) and second by column (the total number of frogs and tadpoles) ```{r} tapply(dat$frogs,dat$color,mean) # calculate mean of frogs by color tapply(dat$frogs,dat[,c("color","spots")],mean) # calculate mean of frogs by color & spots ``` The function **tapply** will apply a function to an R data object, grouping data according to a second variable or set of variables. The first example applies the “mean” function to frogs grouping them by color. The second shows that tapply can be used to apply a function over multiple groups, in this case color X spots. There are a lot of options for plotting data in R. The simplest of these include ```{r} plot(dat$frogs,dat$tadpoles) ## x-y scatter plot abline(a=0,b=1) ## add a 1:1 line (intercept=0, slope=1) hist(dat$tadpoles) ## histogram abline(v=mean(dat$tadpoles)) ## add a vertical line at the mean pairs(dat) ## all pairwise scatter plots barplot(tapply(dat$frogs,dat$color,mean)) ## barplot of frogs by color abline(h=3) ## add a horizontal line at 3 ``` The functions **lines** and **points** are also frequently used to add additional lines and points (respectively) to an existing plot. Within the Plot window, graphs can be cut-and-pasted into other documents or saved to file fairly simply by using Export. If you want to automate the process of exporting graphics, for example when you generate a whole bunch of figures at once and don’t want to Export each one by hand, you'll want to use the graphical functions such as 'postscript', 'pdf', or 'tiff'. For all of these plot functions there are numerous additional (optional) arguments that control the formatting of the plots. The help for par (i.e. ?par) gives a fairly detailed list of these options, some of which you will see in further examples below. 18. Plot the following lines from 0 to 3 (hint: define x as a sequence with a small step size). Make sure to make the resolution of x sufficiently small to see the curves a. ln(x) b. $e^{-x}$ 19. Make a barplot of the median number of frogs grouped by whether they have spots or not. 20. Plot a histogram of blue frogs 21. Use apply to calculate the across-population standard deviations in the numbers of frogs and tadpoles ## Classical tests Since R evolved out of the statistical programming language S, it can easily perform a wide variety of statistical tests and analyses. In R linear regression is done with the function “lm” (linear models) ```{r} reg1 = lm(tadpoles ~ frogs,data=dat) #model syntax: y ~ x reg1 # default return from lm summary(reg1) # detailed summary of results anova(reg1) # ANOVA table of results plot(reg1) # diagnostic plots (4 panels) plot(residuals(reg1)) # residuals by row coef(reg1) # parameter coefficients vcov(reg1) # parameter covariance matrix plot(dat$frogs,dat$tadpoles) abline(reg1) # adding regression line to the scatter plot ``` The “equation” syntax for models in R often confuses people because while the order of the data is that you would use for writing down the equation [y = f(x)], it is the opposite order from the scatterplot (x,y). The equation syntax allows one to add additional variables to the regression model (e.g. y ~ x1 + x2). This syntax also makes it easy to specify interaction terms (y ~ x1 + x2 + x1*x2). Note that the linear model is returning an object and that all the other functions are acting on this object. You can use all the functions you used to explore data objects (e.g. class, names, str, summary) to explore the objects returned by functions. Similar to 'lm', ANOVA models are done with 'aov' ```{r} anov1 = aov(tadpoles ~ color + spots + color*spots,data=dat) summary(anov1) ``` Finally, we can get a bit more sophisticated in our graphs to display these results. Note that R doesn’t care about white space—you can add spaces, tabs, and/or carriage returns wherever you want to make your code more readable. ```{r} plot(dat$frogs,dat$tadpoles, cex=1.5, # increase the symbol size col=as.character(dat$color), # change the symbol color by name pch=dat$spots+1, # change the symbol (by number) cex.axis=1.3, # increase the font size on the axis xlab="Frog Density", # label the x axis ylab="Tadpole Density", # label the y axis cex.lab=1.3, # increase the axis label font size main="Frog Reproductive Effort", # title cex.main=2 # increase title font size ) abline(reg1,col="green", # add the regression line ,lwd=3) # increase the line width legend("topleft", c("Red no spot","Blue no spot","Red spots","Blue Spots"), pch=c(1,1,2,2), col=c("red","blue","red","blue"),cex=1.3 ) ``` ## Vector and Matrix Math While loops are a powerful tool, there are many places where a calculation we want to perform can be done so directly on a vector or a matrix as a whole rather than having to loop through each value. For example, in question 26 we had a vector x = 1:10 and we wanted to add 5 to every value. We could have done this more simply as ```{r} x + 5 ``` which adds 5 to each element. We can also multiply, divide, and subtract with vectors, and the operations are applied to each element ```{r} 5*x x/5 x-5 ``` Furthermore, if we have two or more vectors, and we perform mathematical operations on them, the operations are performed element by element ```{r} y = 10:1 x*y x-y x/y dat$frogs + dat$tadpoles ``` R also allows us to define matrices and perform element-wise math on them as well ```{r} z = matrix(1:25,5,5) z z + 5 z*5 z - 1:5 ``` The last example shows that we can also perform element-wise math between a matrix and a vector. Similarly, matrix-to-matrix math is also allowed, though for these cases you need to pay a lot of attention to the dimensions of the matrices and the order that vectors are being applied. In addition to element-wise math, R can also perform standard matrix math. If you have not seen this math before you don’t need to worry about what it is doing at this point, any matrix operations will be explained on a case-by-case later in the semester. ```{r} t(z) ## transpose diag(z) ## diagonal of an existing matrix diag(1,5) ## create a diagonal matrix w = z + diag(1,5) ## matrix addition w solve(w) ## inversion crossprod(z,z) ## cross product z %*% z ## matrix multiplication ``` ## Synopsis Most of this document is meant to familiarize you with R, so that when you encounter a problem you need to solve you will have a vague memory of something that might work and can use this as a reference. Over time you will come to memorize a larger fraction of this and through exploration you will find many additional functions and coding techniques that are useful. However, here are some parting thoughts on what you should take home at the end of the day. If you have never programmed before: * Focus first on the core concepts that define almost all programming languages + Mathematical operators + Logical operators + Indexing of vectors, matrices, and data frames + If statements + For loops + Functions * Programming, at its essence, is a process of breaking down a complicated problem into a series of very simple steps, and then translating those steps into a symbolic language (code). This document is mostly about the vocabulary and syntax of one such language (R), the fun and creative process of using this language for problem solving will come through examples and experience. If you know another language and are picking up R: * The hardest transition will be SYNTAX Everyone: * Save code early and often + Even better is to commit your code in a version control system like git + Commit code frequently, typically one new feature per commit * Keep code well documented * Use meaningful variable names * Develop a habit of actively searching and exploring R. + Read the help documents for a function + Search for new functions and techniques + Scour the web when debugging. * The key to good programming is being able to teach yourself