Psychology week two assignment

Interesting. You learn something new every day. All right. Welcome to class. This is a week to a program evaluation. Hi, Alyssa. I was just chatting with Cameron getting caught up. How are you doing tonight? I'm doing well. How are you? Pretty good. Good. Good, good. Talking about Chicagoland. It's in the news a lot lately. don't miss it i used to live in uh gary so i'm right across the water from chicago so very familiar with the the cold they're starting you're starting to get don't miss it well it's i think it's a great city but we have we've been had a most beautiful um fall it's a really nice day today as well i didn't really get outside um but But it's actually really, really, we've been very lucky. But, yeah, I just am not, like, thrilled with what's going on in the news that they've kind of, or I don't know about the news, but certain groups, they have portrayed Chicago as being very dangerous, and it's not even in the top ten most dangerous cities. Mayor Biff, I think his name is. he's been he was on Rachel the other night and I don't know we've been seeing a lot of the mayor Biff I think his name is yeah I don't know anyway well let's get started on talking about program evaluation because it's fun to talk about that but I know we all have things to do um hi Lou thank you for coming I know you told me she's going to be just listening in but just wanted to say hello. Welcome. So first off, I just want to say that I was very impressed. I think the class did a great job overall on the week one assignment with the logic models. You know, it is something new and different, but overall, I thought it was, you know, people did a good job. Um, there were a few people who, um, didn't turn in the logic model or turn it in in the wrong format. So I've reached out to those people, um, individually, but overall, you know, people who did turn it in, in the correct format seem to get it and, um, do a good job overall. How was your experience with that? Did you guys feel like that was a useful experience? Do you have any feedback on that assignment? I always, you know, I'm open to feedback. So I do have a little survey form going, which I will, you know, send out a new one for each week. So feel free to use that even if you attend, if there's something confusing to you or something that's unclear or something that you particularly like, that's helpful, you know, as, you know, we take that feedback and and try to make it like a clear, you know, clear, clear format so that it's something very actionable that you can use. So if you can tell from my little memes and stuff, obviously logic models are very common. You know, that's something that's a typical thing that's used in program evaluation. So you can find a lot of things about it online. So regardless, you guys did a nice job with the first week. All right. I'm going to take that as everybody's good. If you don't have any comments. Okay. So let's just jump in and talk about the week two discussion and assignment. So what we are doing in this class, as I've mentioned before, is you're going to be writing a program evaluation report. So you're kind of, what I'm trying to do is give you the opportunity to think like a program evaluator and what that means is that if you're a program evaluator you would be designing the research and then you would also be analyzing the research but you would not be the one who would be collecting the data you would be telling an administrator within that program what data they need to collect and give them the tools you know, to collect that data. But as part of your role as the program evaluator, you would need to define your evaluation goals and questions. So there's two parts this week. We're going to do the discussion, which is going to focus on defining the evaluation goals and questions, and then the assignment, which is going to be the first part of your evaluation report. So your evaluation report, the full report will be due in week seven, but each week that you have to turn in an assignment, you're going to turn in a piece of that evaluation report. And that's why that logic model is so important. So, you know, for people who didn't have the completed logic model or it done correctly, that's why it's important to still do that this week, because, um you know we're going to just keep building off of this you're going to use that logic model then you're going to develop your evaluation goals and questions and then you're going to figure out the methodology and then we're going to you know kind of collect data but we're you're just going to be creating a data file and then analyzing data in week six and writing an evaluation you know an results section, and then week seven, pulling it all together. So save your feedback each week and try to incorporate your feedback each week. You may even want to, you know, do the incorporations of feedback each week as you go along, because that's going to just make it easier when you get to week seven, if you just incorporate that feedback as we go along. Okay, so getting it into the discussion and the nitty gritty here. So for the discussion this week, as I mentioned earlier, we are going to, you know, we're, what we're trying to do is figure out whether this program is effective, right? So whether what they're doing with the program is really working. So we're the program evaluators. We're hired to come in to be the program evaluators, and we're going to figure out what assessment we want to do as if this were a real project and we were really collecting data. Okay, so again, for your discussion this week, there's several different questions. and what we're going to do is the discussion is going to help you with your week one assignment. So with your discussion, you're going to think about first the program's impact vision. So that should be about one paragraph. So just imagine your agency's program is highly successful. what would success look like for clients or participants? Okay, so what is it that you would want to see as an outcome? Like, what do you think success would look like in that program that you identified. And so I gave a little template example here. And it says, like, this is something that you could use as a template, something like this, like if this program, obviously, you would put in the program name, if this program were highly successful, and then whoever it's for, right, youth, adolescents, clients, moms, dads, whatever the participants are, would, you know, what would be the ideal change? So let's say it was a a program where we wanted to reduce stress for single parents. So like if this program were successful, single parents would have reduced stress. So in other words, just identify what success would look like in terms of what kind of outcome would you want to be seeing whether it's an increase in something or a reduction in something we would want to identify what what we would want to be seeing with that program and the impact in terms of the impact or success Okay, next, I'd like you to identify, specifically identify some outcomes. What are one or two ideal outcomes that you would want to measure if resources and time were unlimited? And why are these outcomes important indicators of the program's effectiveness? So again, only choose one or at most two outcomes. So, you know, I would want to see improvements in social support and or perceived stress for single parents. These outcomes are important because they relate to parents' well -being and ability to support their children or whatever it is, right? So, you know, you would just be writing a paragraph about, you know, what you're looking for in terms of outcomes and why they matter. So, you know, is the service trying to reduce mental health symptoms? Is the program trying to reduce substance use? Is it trying to increase educational attainment? What are the outcomes that are important and what is the program goal for the agency? So again, you should have one overall outcome that you're trying to measure for this agency and program, and you might pick a secondary goal as well. Um, so maybe if you're looking for a reduction in mental health symptoms or a reduction in substance use, you'd secondarily look at a goal like family engagement or family involvement or employment or something like that. But primarily you're interested in the reduction of mental health symptoms or substance use, or maybe we're looking for increases in education or pursuing higher ed, something like that. So the primary goal is like either the reduction of something or an increase in something. And the secondary goal might be something like the level of functioning, like functioning well within a family, functioning well within the employment system. So something like that. Okay. So here in part, the second part, you're stating the particular goal in terms of what you want to see improvements in and why they're important. okay so next um in part three we're going to get into what measurements you might use um so for example if you're interested in substance use um you know you might do something with respect to um measuring you know how much they're they're using particular substances Um, if you're interested in mental health reduction, symptom reduction, you'll look at either something like mood disorders or anxiety or whatever the agency is mostly treating. Um, but the key is you want to see, is there effectiveness in the program? And is there a change between when they come in, when the measures are, you know, when it's measured at the beginning of the program, before the program starts, and from when they leave or when they're discharged from the program. So are the symptoms higher, significantly higher when they come in than when they leave? So that would be a reduction. Or, you know, or is the symptoms lower when they start than then when they leave, then that would be an increase or an improvement. So like if we're looking at something in the education realm, are they doing better off at discharge in terms of educational attainment than they were at intake? So you're going to want to look at, you know, Typically, we would want to look at something like measurement at intake and at discharge, but you also remember that you have long-term goals, and so you'd probably look at something like intake, discharge, and six months after they leave, like six months after discharge. So we'll talk about that more when we get to measurement, but right now we're not talking about specific measures, but we're talking more generally about, you know, what we'd want to be looking at, like what kinds of things we'd want to be examining or evaluating. okay so then next um we would be getting into um ethical and practical considerations so what challenges might come up in trying to collect this kind of data for example privacy concerns um you know time concerns like if we're dealing with something like single parents and we want them to attend a program are we offering child care so that they can attend the program um you know uh missing data what about you know people who miss a certain number of sessions how much is too much um you know how does how much does staff need to be trained, so on and so forth. So you could, you should at least talk about at least one challenge that it would be very important to address to ensure that the evaluation is ethical and practical. Okay, so here we're just kind of looking at an overall, what do we want to accomplish with this evaluation. So you should support your discussion with at least two scholarly sources that focus on evidence-based practices in social service agencies. So again, you can use your textbook but it's important to also bring in some um outside resources as well any questions about the discussion and again i know this probably seems like a lot but it's actually going to lead in part to your assignment so hopefully it will be helpful for you okay so if there are no questions about the discussion that I'm going to move on to the assignment and how this leads to the assignment. So it's a little hard to see. Let me see if I can increase the size here. Okay, so this week you're going to write the first part of your program evaluation report. Remember that you're going to be writing the program evaluation report over the next five or six weeks, right? Because you're going to write a part, each assignment is part of the program evaluation report, and then you're going to revise it, those different parts based on feedback, and pull it all together for your week seven assignment. So again, make sure you just incorporate feedback each week, because then you will be pulling it all together for your week of an assignment. So this week, we're going to start off with the introduction. So one thing I will say is that you're, you know, make sure you have a title page. That's a title page like you always do in your classes. And then on page two, centered at the top of the page, you write the word introduction in bold. Now that's a little different than what we do usually in APA style. I feel like when I read APA papers, I'm always like saying, don't write the word introduction. You just have the title at the top of the page. So this is different. We're going to actually have the word introduction. So have the word introduction centered at the top of your second page. And then your introduction is going to have three parts, and those could be left justified subheadings, the problem, the program, and the evaluation. So the problem is going to be the issue or need the program's designed to address. The program is an overview of the agency and program you're evaluating, and the evaluation explains what the evaluation will examine. that's really, you know, directly from your discussion this week. Okay, so let's start off with part one. Part one is the problem, and this should be about one to two paragraphs. So the problem, this is really, think about if you were trying to sell this to a grant, right? You're trying to get money for doing this project or showing that this program is effective. that it's really working. So one thing you're going to want to do is really identify what is the problem that this agency's program is seeking to address, and why does it matter? Like, who's affected and why does it matter? So you should include support for support with relevant statistics or research on the issue's prevalence, and you can use credible sources like the CDC, National Institute of Health, National Institute of Mental Health, and so on, any of those types of sources, like what's the prevalence of, you know, whatever it is you're trying to solve, alcohol addiction, or, you know, and what is it and what does it matter? So who is affected? And then what happens if this issue is not addressed? What are the risk, costs, community or community impact? You know, something like that. So be sure to talk about, you know, what it is and why it's important. Next, in one to two paragraphs, I'd like you to describe the program. So describe the one specific program that you chose. You can also discuss the agency as part of that, which would be a good thing to do. But I want you to also talk specifically about that program, because as you know, there are a lot of different programs within that agency. And so to the best of your ability, based on the information that's available, talk about what that program does in terms of services, structure, where it is, who they're serving, and if it's available, like what the treatment or intervention approach is and why that's appropriate for the population. Now, I know not all this information is available on the website. So if you need to make inferences, you can also, you know, say that like, you know, you know, based on what I've read of other types of similar programs, my assumption is that this is being done, you know, as XYZ. Okay. So again, you know, we don't necessarily have all the information. In an ideal world, the way this, I guess, I don't know if I say in an ideal world, but the way this class used to be is that when it was very small and it was an elective, it was people had to find an agency and interview people at the agency and find out specific information. But there's just not really enough time in eight weeks um and so it seemed like people often had to do what we're doing now anyway they didn't really weren't able to you know get in with agencies um in enough time to really find out information so um so you know just do the best you can with that aspect of it okay um Next, in section three, I'd like you to state what the evaluation will assess. So what are you, you know, what are you trying to assess? This is really just one sentence long. That's all part three needs to be. So in section three here, you're going to incorporate the goals and objectives that you discussed in the discussion this week. So there's a reason that we do the discussion first. It's a clarifying assignment, and then you'll go into the report production. So, you know, what I'd like you to do for this part is, you know, incorporate the goals and objectives from your discussion this week. and then you could use this template you don't have to use this template but it may help you so this program evaluation will assess the effectiveness of and insert your program name by determining whether there's a significant increase or decrease in your target outcome between time points like intake and outtake or over the over the 10 -week course or whatever your timeframe is that you're looking at. Okay. So here are some examples like this program evaluation will assess the effectiveness of the youth resilience group by determining whether there's a significant reduction in anxiety symptoms between intake and discharge. Or this program evaluation will evaluate the parenting support program by determining whether positive parent child interaction scores increase over the 10-week course. Okay, so again, don't name or describe the measurement tools yet. We're just talking about measurement more generally here in terms of what the outcomes are we want to be looking at. And because you're writing a report rather than a manuscript, your formatting will not follow the same sections or use the same headings as an APA style manuscript. However, you should have APA formatted citations. So, and make sure that your references is an APA formatted list as well. Um, any questions about, um, the assignment? Okay. Um, all right. So I'm going to give you an example of it. It's in the classroom, but I'm going to share with you an example of, um, um how you might talk about some of these things in your report um because we do have um like i said an example in the classroom that um looks at um you know that has an evaluation and so that can be helpful just to kind of see how some other people talk about evaluations when they're dealing with funding agencies and things like that. Okay, so I'm going to just share my desktop and kind of briefly go over a few of the slides. okay so um i'm just gonna try to show this so this is the broward um homeless program this was a single site pro well okay so you can actually watch this so why do they need to have a program evaluation this is back in 2001 to 2003 and that was about the time that funding agencies began to require that the social service agencies now provide evidence that the services that they provide actually work prior to that they could say we provide you know X number of group therapy sessions a week or X number of individual sessions per month and it was just a number of services that were reported but now they wanted to know do they actually work and donors also require evidence that they're donating money to nonprofit agencies that that really provide a service that's effective okay so I would go, you know, into there and actually watch it because it is useful. Dr. Hansen recorded this overview, but you can see, like, for example, here on the goals page, there are some goals for this program, such as to increase behavioral health services, engagement in center residents, behavioral stability, reduce symptoms severity, increase functionality, um so those are examples of some things that might be considered um you know outcomes that they wanted to focus on um here is an example of how things might be measured like they used a pre-post design um and um you know some things that you need to think about when you're doing assessment like is what is the main purpose who will do it what type of assessment is it We're not getting into measures yet, but here are some examples of measures, and it outlines some of the specific measures. And then here are some of the issues or challenges that may come up. Like, for example, you know, there's an aid for training, oversight, supervision, proper data collection and data entry. And then, you know, this goes, there is just one slide here that looks at the results, where compared to those treated to those not treated, there was a reduction in substance ***** symptoms, psychiatric symptoms, and so on. Okay, so I just wanted to provide that as a little overview of a write-up. It is in the week two area. um your assignment for this week should be one and a half to two pages and what's the reason for that well the reason is that we don't want you know to end up with a huge report when you're dealing like with granting agencies or funding sources um you know they don't want to see a you know 40 page report so your intro this week should be about a page and a half maybe at most two pages um so you know if you have any questions as you're going through the week feel free to reach out and ask um questions i'm here to help um i think i have just one more slide to share with you just to kind of emphasize some of the points from this week um Um, let's see if I can reduce this cause now it's looking, showing up very big. Um, okay. Oh, here it is. So I had a little, um, graphic here where it says, um, so, so we're talking about outcomes, right? One of the things we're talking about is outcomes. So this, um, is a little program evaluation here evaluation here is like outcomes normally we just measure the height of the files um but we're going to look at outcome as change for clients not um paperwork volume because you know we could of course um just look at something like how many sessions people attend or how many forms there are, but that's really just what the program does. Outcomes are about what changes for participants. So this week, I want you to just choose one thing that you think that program truly hopes to improve and one simple way to measure that change. So don't worry, it doesn't need to be fancy. We just want to look at what are we really hoping to change here. And, you know, focusing on like a short term or intermediate goal, obviously, the long term goals, we would need to be looking further out at data collection, like six months after the program. And so what do we want to look at in terms of what's a hopeful goal in terms of the outcome for the clients or participants? All right. Well, thank you guys for coming. um does i guess my last question would just be i know there's just a few of you here but i'm wondering if this time works for you um if it doesn't work great feel free to let me know that as well um but i do appreciate that you came even though the time was was changed um so thank you again. All right. Have a good, have a good week. Um, all right. Thank you, Lou. Thank you, Alyssa. Um, thanks and have a, have a great week and feel free to reach out if you have any questions, um, or concerns. Hi, Watchner. Okay, great. Um, okay. So that's three for three of the people who responded. Um, Um, I don't know. Um, but yeah, obviously everybody else feel free to reach out and let me know, um, your thoughts about the timing. All right. Thank you guys and have a great week.