Psychology Week four assignment

all right so here we are you all in week four we are at the halfway point now you're doing great i'm almost finished with grading last week's assignments really proud overall i do want to take a minute at the beginning to say participation participation questions are built into this class but i often get through lecture and get distracted and forgot forget to ask a discussion question. So when I do that, you all, I'm not trying to trick you. There's not like a hidden discussion or a participation question or anything. If I forget to ask it, which I most likely will, then I'm just going to give everyone full points who completes their discussion on time. The participation points aren't weighted as heavily. And so if I forget to ask, that's my bad. I get caught up in lecture. I'm just going to give you all full points for it, as long as you complete your initial post on time, okay? All right, so let's jump in this week with a little bit of content review. Of course, you all should be doing all the readings that are in the course materials. That will help you, but I'll give you a little review here so in research methods in statistics we are dealing with inferential statistics so this type of work involves making inferences about population so like everyone that we're interested in we could never reach the full population we only aren't able to collect a sample. So, inferential statistics involve making inferences about populations based on the information we gather from our samples. So, if you collect data, if you do a study, you are taking a sample from the larger population that you're interested in, okay? Okay. So inferential statistics allow us to actually make inferences about that larger population based on the small sample that we're actually dealing with. So we contrast this with descriptive statistics like means and standard deviations and those types of things, which merely summarize known information about your sample. Inferential statistics are especially important because they give us the basis for being able to draw conclusions about the larger world, about the populations that we can never measure as a whole. And we do this based on the results from particular groups of people that we actually study, so our actual sample. So, the core logic of hypothesis testing, this was one of those aha moments for me as a student researcher. So, hypothesis testing considers the probability, the likelihood, that the result of a study, so specifically your test statistic, could have occurred if the experimental procedure actually had no effect. So, if there was actually no effect at all, what is the probability that you would get your test statistic? That is what hypothesis testing is doing. So, if that probability is low, like the standard in our field of less than 0.05, then the probability of no effect is rejected. So, if we have less than a 5 % chance of getting a test statistic as large as we did for the null hypothesis to be true, then we're going to reject that null hypothesis and say the odds of that being true are so low, given the size of this test statistic, that we are going to reject the null hypothesis. So then the theory behind the procedure is considered to be supported or your alternative hypothesis is supported, not proven, if you reject the null hypothesis. So the steps of null hypothesis testing are always the same, no matter what statistical test you're using, you all. So step one is always formulate a research question and operationalize your variables, your independent and dependent variables. make them measurable, make them quantifiable. Step two, you state your null and alternative hypotheses. The null hypothesis always says there's no effect. That's what we're testing statistically. But the alternative hypothesis is what you actually think is happening. And that's what you hope to find support for by rejecting the null hypothesis. Step three, this is the part so many students freak out about, but it's just one of the steps in the process of hypothesis testing. You choose the appropriate statistical test. So the nice thing about this is it actually depends on your research question and the types of variables that you outlined in step one. Those will flow naturally. The appropriate statistical test will flow naturally from your research question you set your significance level your alpha level so the conventional level in behavioral science is 0.05 meaning a five percent chance of getting a test statistic that large if the null hypothesis is really true and then the fun part you go out and you collect your data and input the data into your statistical software of choice. So, it doesn't matter which one you're using. SPSS, R, Jamovi, Excel, so many other different statistical analysis programs should give you all the same result. So, you'll, whatever your program of choice, you're going to run the analysis and interpret the output, no matter what type of statistical test you're using. Is it a correlation or a regression or a t-test or an ANOVA, no matter the type of analysis, what you're going to do immediately is look at the output, look at the p-value and determine if that probability is less than 0.05. You're going to look at the test statistic. And so the p-value, the probability tells us the probability of getting that test statistic, given that the null hypothesis is true, given our assumption that the null hypothesis is true so that p-value is the first thing we look at we're always going to look at the test statistic itself and then most of the time we are also going to look at confidence intervals and effect size so the p-value simply tells you if there is statistical significance or not if you can reject the null hypothesis but looking at the confidence intervals, and effect sizes gives you more information about the meaningfulness of your findings because we can have statistically significant results that are actually meaningless, like a low effect, a small effect size in the long run. So, we have to look at all that information together. One single piece of information is not enough. And then after looking at all that information, then you make your decision. You either reject the null hypothesis or fail to reject the null hypothesis, and then draw conclusions in the context of your original research question. So, you always take it back to the beginning. Ultimately, you're going to report your results and the limitations of your study, and there will always be limitations. All right, so we go through those steps, right? let me just back up here a second we go through these steps say we get a p value less than 0.05 the test statistic exceeds the the critical cutoff value and it's not that big of an effect um but the confidence interval doesn't contain zero so it's technically significant and we we feel pretty confident in saying we're going to reject the null and say this gives support for the alternative hypothesis. But remember, we are always working with that margin of error. So this alpha level that we set back here at step four, that conventional 0.05 level, it's somewhat arbitrary because it leaves that 5 % chance of error. So what if we're wrong? What if we you know either reject the null hypothesis and say yeah there was something happening or we fail to reject the null hypothesis and say yeah our our intervention didn't work so we we do that but what if we're wrong so this drink brings us to the discussion of type 1 and type 2 errors These are the most common types of errors in null hypothesis testing. So there's always a chance that the researcher will make an error in their conclusions. We're never claiming 100% certainty. We are always willing to accept that chance of error. So a type 1 error is a false positive. So when you reject a true null hypothesis. this. So I always got tripped up on type one and type two errors and trying to remember which one was which. And this silly little meme here that I have, hopefully will help you remember. So a type one error, a really silly example of that would be this doctor looking at this elderly gentleman and saying, congratulations, sir, you're pregnant. So something was wrong with that test, right? You all, this elderly gentleman is not pregnant. And so the test was saying there's something happening here, but in reality, that's not the case, right? So a type one error is a false positive. We reject the null and we say there is an effect, but we were wrong. So that alpha level of 0.05 means there's a 5 % chance that the researcher will conclude there is an effect when really there is not. So, in those cases, when we see that happening, the difference between the means wasn't due to the independent variable, but was actually due to sampling error or due to chance. Now, a type 2 error is failing to reject a null hypothesis. So, that would be the example of this doctor looking at this clearly pregnant woman and saying, no, you're not pregnant. So that's saying nothing is happening here when actually there is something happening, but the test didn't pick it up. So interestingly, there's this converse trade-off with type one and type two errors. So your chance of a type two error is essentially the same as your statistical power. So your power to reject a null hypothesis, but there's this kind of trade-off. As you decrease your chances of making a type two, you increase your chances of making a type one and vice versa, which is how behavioral science has settled on that 5% chance of error, okay? But regardless of how well we're trying to guard against these things, they absolutely still can happen. So, try to give you a little visual explanation of this, of how it can happen. So, looking at A here, these are two normal distributions that happen to overlap a lot, right, in the top. B shows you, demonstrates that the increase of average distance, so the farther apart the means get, this reduces the probability of one of those classification errors, a type 1 or type 2 error. and then conversely, the probability of a classification error increases when we have more dispersion. So the more overlap and the more spread out distributions are, this increases our chances of making an error. All right, so let me give you some good illustrations here. So, let's consider T-tests, where we're just comparing two means, okay? So, if we were trying to look for differences between, so X1 and X2, or sample mean one and sample mean two, if we got these results, we would conclude that there is no difference, that they all belong to part of the same population, and we would be correct, right? easy peasy. Case B or case 2, here the sample mean of group 1 is so different from the sample mean of group 2 that they've created their own separate distributions. They're so spread apart that we can easily identify the difference and we would be correct. But it's not always as simple as case A1 or B2 here. So looking here at case three or C, now you know where the cutoffs are for the 5% probability. They're down there in the tails of that distribution, exactly where you see the sample means of group one and group two. So if we actually got these results, we would likely reject the null hypothesis because the difference in these means would fall within that 5% range. And so, we would reject the null hypothesis, but we would be incorrect. Those two sample means did actually belong to the same population, to the same distribution, just at very opposite ends. Okay. So, that would be a type one error. You would reject the null hypothesis, but you were actually wrong. Okay. Now, a type two error is when we get that overlap. So, two completely separate distributions who happen to share a lot of common ground. So, you see the difference in these, the blue and red lines here between sample mean one and two. So we would conclude that there was no effect, but we would be wrong here. So we would fail to reject the null, but we would be wrong here because of all of that overlap that we're missing. All right. So what are the different factors that can influence a hypothesis test? First is the variability of scores. So, if scores are all over the place, there's a lot of variability, it's more difficult to see the patterns and the differences between groups when there's so much variability. So, that's a big issue. And then the second is the number of scores in the sample. So, you need a large enough sample size to have sufficient statistical power to test the questions that you're trying to test. All right. So what is statistical significance for those who are still confused? We'll just kind of run back through it. So remember that we are dealing with samples. We never have access to the full population. If you're interested in people with PTSD or people with depression, you are never going to get access to everyone in the world who has depression or PTSD. So you take a sample. And so the larger your sample, the more representative it is of the population that you're interested in studying. So as I mentioned earlier, we always test the null hypothesis. Statistically speaking, we are testing the hypothesis of no effect. We are trying to determine the probability of getting that test statistic if the null hypothesis is really true. So if the probability is so low, less than 0.05, that we can reject the null, then that's what gives us support for the alternative hypothesis. So inferential statistics like we're using make inferences about the population at large, but how do we know that we're not making a mistake? Well, all we can do is set that significance level and know that we can never be 100% positive that what occurred is actually the result of what we tested. So we never claim 100% confidence in science, but our goal through good experimental design and methodology is to eliminate as many other possible competing reasons for the differences between the groups. So essentially statistical significance is the degree of risk that you're willing to take that you would reject a null hypothesis when it's actually true. So what is the, how much risk are you willing to take that you would reject the the whole hypothesis and say there is an effect when actually there might not be. And the answer to that is 5% typically in behavioral science. Now, in addition to these considerations, we have to think about confounding variables. So those other variables out there that could be contributing to our findings that we aren't directly measuring. So we need to think about and try to measure and control for as many of those things as we can. So, please beware of spurious and confounding variables. Coming back to that bit about the difference between significance and actually meaningful results. So, oftentimes we can get statistically significant results, often with correlational research that's not always meaningful. So it's important that you only interpret your findings for the context in which they occurred. You can't make grand causal claims about the whole population of interest. You need to be reasonable about what your actual sample and data show. So we look at not only the p-value, but we look at the size of the effect and we consider our statistical power. So this little illustration here is a very simple thing to demonstrate the impact of sample size on power. So with an N of one, with a sample size of one, trying to find differences between baseline and post-treatment, you would get a power level of say 0.26. Now tripling that sample size to three gives you more than double the power. So you get power of 0.53. And then upping that sample size to seven, and you see that distribution spread apart, right? As our sample size increases, we're more easily able to see differences. And so the larger your sample, the more power you have. all right so there's the quick and dirty material review for you and now let's dig into your tasks for the week so with your knowledge of significance testing read over the cohen and schmidt papers that are in the course materials and i've given you all the great things about null hypothesis testing, but I want you all to describe the significant flaws of null hypothesis testing. We touched on some of them and what it means to our field. So as a note, over the coming weeks, we are going to be discussing alternative approaches to null hypothesis significance testing, but let's dig in this week. So those are the two articles, Cohen, 1990, things I have learned so far. And then Schmidt 1996, statistical significance testing and cumulative knowledge and psychology implication for training. Okay. And so now for your week four assignment, the data files are there. You'll use this same data this week and next week, But don't get ahead of yourself. So this week, you're going to be, the file contains all these variables that you see here. So subject number, time DRS is number of visits to health professionals. That's your outcome. PHY HEAL is the number of physical health symptoms. MEN HEAL is the number of mental health symptoms. Those are both independent variables. And the third independent variable is stress. So that's a stressful life event score. Now, for this week, you're only going to focus on the variables time DRS, so number of visits to health professionals, and stress, okay? Okay. So, those variables are there for you in the data set. And so, the first step anytime you start, anytime you get your hands on data, it's important to examine potential univariate and bivariate outliers. So, use the procedures you've done in your earlier assignments to examine outliers and get to know your data. For this assignment, be sure to include your Z-score analysis, your box plot visualization, and your tests of normality. Now, if you see potential outliers in the data, I want you to describe the outliers and how you would handle them. So, would you keep them, delete, or transform them and why? But do not actually delete, transform them because I want everyone working with the same data set. So, before you conduct any of the actual analyses, you need to examine the data for potential outliers and ensure that it meets the key assumptions for the regression analysis. So, I've given you very detailed instructions. So, use Z-scores and box plots to identify those outliers. Let's stick with a plus minus 3.3. Tell me whether the distributions are approximately normal, and discuss whether those outliers appear to be true outliers and how you would handle them. So would you retain them, transform them, or exclude them? But again, do not actually do that. For part two, you're going to create a scatterplot. So in Jamovi, you will click on exploration. And then you will move stress over to the IV and time to the doctors on the DV, so the X and Y axis. Make sure you label the plot properly. So figure one, scatterplot of stress and visits to the doctor. Then look at the scatterplot itself and describe what you see. Describe the relationship between the independent and dependent variable, whether it appears to be positive or negative, or if there's no clear relationship at all. Comment on the strength of the relationship. So how closely do those data points create a linear line? How strong is that linear line? Weak, moderate, or strong? And then note any potential outliers or unusual patterns in the data. So clusters of scores, gaps in the data, those kinds of things. And then make sure you let me know if the data points follow a linear or straight line pattern. For part three, you will conduct your correlation analysis. So you will go to analyses and regression, and then you will move over the two variables of interest. You will ask Jamovi or whatever software you're using to give you a Pearson's correlation to report significance. And your hypothesis is that they are correlated. it. So you'll do that and then report your results. I've given you a bit of a template here. A Pearson correlation analysis was conducted to determine the relationship between, and I know in correlations, independent variables aren't, and dependent variables aren't true, but we'll be taking it a step further. So report your variables and the results. So you will report R, degrees of freedom equals the correlation coefficient, comma, and the p-value. To interpret this, you're going to say, is this correlation positive or negative? For the strength, you can use standard benchmarks to describe the strength. So anything less than 0.3 would be considered weak. Anything between 0.3 and 0.5 would be considered moderate. and anything greater than 0.5 would be considered strong. Now, keep in mind that is whether those are positive or negative. A correlation of negative 0.5 or higher is just as strong as a correlation of positive 0.5 or higher. The sign of the correlation only tells you about the direction. it doesn't tell you anything about the strength, the absolute number value, how close it is, the higher that value, the stronger the correlation. So you're going to explain what the relationship means in context. Something like this, the correlation is positive or negative, indicating that as this variable increases or decreases, this other variable tends to increase or decrease as well. Now for part four, we move on to regression, which is a step up from correlation where we actually are making predictions. So for part four, you're going to conduct a simple linear regression with stress as the independent variable and visits to the doctor, time DRS as the dependent variable. So what you're trying to do here is you're trying to predict the number of visits a person makes to the doctor based on how much stress they have in their life. Okay. So you will choose regression in Jamovi, and then you will move your variables over appropriately. So this would consider your independent variable to be a covariant. So you You will move stress into that box and the time DRS variable into the DV box. Also, you will click these additional options in Jamovi and ask for it to give you the R value, R squared, which gives you the effect size, tells you how much variance is explained, and the overall omnibus F test. So make sure you click those things. And then I've deleted the actual results here. I've hidden them, but you will get output that looks like this with actual values. And you will report it in this manner. So you will say a simple linear regression was conducted with IV as the independent variable and DV as the dependent variable. You will report the overall model significance first using the F-test results. Then we move on to the model fit. So you report R-squared and interpret the percentage of variance explained. You give the model fit. So the R-squared value indicates that whatever percentage of the variance in dependent variable is explained by independent variable. So that's how you do that. And then you also need to give the regression equation. So you will need this information from the output. you're going to need the intercept and the slope. So you will, you'll give the, the formula will go dependent variable equals the intercept plus the slope times the independent variable. So the intercept value represents the predicted dependent variable when, excuse me, when the IV is zero, so when that's held constant, and the slope indicates how much each one unit increase in the IV, the dependent variable is then predicted to increase in those value units. Okay. So, you're also going to need to report your assumption checks. In order to do these analyses, you have to make sure that the assumptions have been met. So, the first is linearity. So, confirm linearity was assessed earlier with that scatter plot, and then summarize the findings. You need to create a histogram or QQ plot of the residuals, not the raw data, but the residuals, and then discuss whether those are approximately normally distributed. it, then I want you to dig in and compare what you were able to learn from the correlation versus the regression analysis. So, the correlation coefficient, that R value, that measures the strength and direction of the relationship between variables independent of units. So, the correlation coefficient is a standardized measure ranging from negative one to positive one. Negative one and positive one mean they would be perfectly correlated. So the further, the closer you get to the whole one value, either negative or positive, the stronger your correlation. The closer you are to zero, the weaker your correlation. Okay. So correlation is dimensionless and independent of the scales of the variable. So it's limited in what we can do with it. It's just simply going to tell us, are these variables associated or not? Now, the regression slope, the beta value, actually represents the rate of change in the dependent variable for each one unit increase in the independent variable. So it's much more precise and actually maintains the units of the original variables, which correlation analysis does not do. So regression absolutely depends on the scale of the variables. So dig into those differences, what you learned from each analysis, what one could tell you that the other couldn't, and give me a good explanation of the difference between correlation coefficient and the regression slope in the actual context of this study. So, talk about it in terms of the variable names, not just broadly in a textbook fashion. All right, then I want you to get into some limitations in your conclusion. So, discuss any limitations of the study. For example, small sample size, if there were outliers in the data, if the assumptions were not fully met, these are limitations that you must address. And then write a conclusion paragraph that brings together all the findings and write up the results in APA format. So make sure that you include all of the key findings. So the regression equation, the significance of the model and its R-squared value, key results from both the correlation and regression analyses, and the implication of the relationship between stress and doctor visits, which are the variables that we're actually interested in. All right. Stop sharing. Come back out here. Everyone's always like off camera when I come off my lecture, but I feel like if I could see your your faces, there would be like smoke coming from your ears. Like I just fried your brains a little. agreed yeah it's a lot i know we cover we cover a lot in a little bit do you all have questions um as of now i don't but i might email you if i do please email me yes so i think with all of the data work it always feels overwhelming just being taught about it until you actually get in there and do it yourself. So I recommend pulling up the assignment and pulling the lecture back up and looking through the examples that I've given you throughout lecture, because I pretty much gave you the template language that you need, and all you have to do is fill in the values. yes that actually does help a lot so i'm definitely going to use that and follow you know those steps so thank you so much good good you're so very welcome all right all my quiet others out there are we doing okay are we just out there traumatized or right if you do not have any more questions comments concerns you are so free to just go ahead and log on out. I am available, as always, for questions, comments, concerns. Reach out sooner rather than later, but I'm really proud of the work everyone is doing so far. This was a really tough class, and I feel like you're all doing great. So, I'm... We appreciate you, Dr. Rettliff. Yes, thank you. Thank you. All right. Whoever wants to go, go ahead and get on out of here. And if you want to hang around for a minute, hang around and ask questions or chat. Okay. Dr. Ratliff. I'll probably send you an email. Okay. I'll watch out for it, Asia. Have a good night. You too. Thank you. All right. Bye-bye. Did you say something, Kevin, or are you saying bye? Bye -bye. Good night. Okay. Bye. Thank you. Have a good one. Hi, Dr. Ratliff. Hi. Are we recording? Yes. Do you want me to stop? I can stop. Okay. Thank you.