Program evaluation

Evaluation Design 

In terms of the type of evaluation that this program will follow is the formative evaluation. For the time before the beginning of the program it will aim to provide the project’s design and to use the information based on the surveys before the start of the program to guide us on what our population has questions about and need help with in regards to contraceptive use and other concerns about their reproductive health. Whilst, using the formative evaluation approach our team will work to monitor the effectiveness of the program and alter the program if needed. For instance, our approach to using the pretest-posttest control group evaluation design will help us determine if the program is effective. Before the seminars, we provide a survey (pre-test) to a specific group of individuals, in our case young women between the ages of 18-24 who are sexually active and gather baseline data. Furthermore, throughout the program we evaluate these individuals, through surveys and at the end of the program we provide a survey (post-test) to these participants to determine what behavior changes occur and if the information and resources provided about safe sex and other reproductive concerns help achieve the outcome of our program and the quality of life of these participants. 

Sampling Design and Description

Additionally, the best method to use for our program is probability sampling in terms that we are selecting based on the criteria of young sexually active women and choosing our participants at random but women within the target population have an equal probability of being selected to participate in the program. For this reason, our program follows the simple random sampling approach because it saves time, and also our budget and resources may be limited. This method of sampling also helps us narrow down the 100 participants we would like to work with instead of all the young women between the ages of 18 and 24 located in all of Collin County. This method allows our group to gather substantial data from the sample and therefore more appropriate and constructive.

Instrumentation

The main instrument the program is going to use are surveys. A survey allows participants to answer anonymously. By allowing participants to answer anonymously and not in open discussions they are more open to giving truthful answers (Wilson, 2018). In New York, there is a program called Comprehensive Adolescent Pregnancy Prevention (CAPP) and they do pre and post surveys. CAPP does this to get information about the participant's past and newly acquired behaviors. For this program, there is going to be a pre and post survey throughout the program. 

The process objective of the program is to learn how much or how little the participants know about contraceptives and its benefits. The instrument that will be used is a pre-survey and it will be given to 100 participants 3 months before the first conference. The survey will present different types of contraceptives of which are the ones most commonly used to the least knowledgeable among the population. The contraceptives that will be presented are Birth Control Implant, IUD, Birth Control Shot, Birth Control Patch, Birth Control Pill, Condom, Internal Condom, Fertility Awareness, and Withdrawal (pull out method). The program coordinators obtained these contraceptive methods from Planned Parenthood. The participants will respond using 1 being knowledgeable about the contraceptive and the benefits of it and how it is used, 2 knowing about the product but does not know about the benefits nor how it's used, and 3 not having any knowledge about the contraceptive. The survey is going to be evaluated by totaling how many 1’s, 2’s, and 3’s there were. This survey will allow program coordinators to get an insight on which contraceptives are most commonly known and how many different contraceptives methods are young women informed about. 

The impact objective of the program is after the first conference 40% of the participants will be fully knowledgeable about three contraceptive methods. To get this data the same instrument that was used for the process objective will be used. The survey will be almost the same as the pre-survey as it will have all the contraceptives that were presented to the participants before the program began. However, now the program coordinators expect participants to be informed about more contraceptive methods after the first conference. The participants will respond to the survey with 1 as being fully informed of its use and benefits, 2 just knowing the product, and 3 not knowing anything about that product. The program coordinators will total the final responses and they will see that 40% of the participants were educated about three contraceptive methods. 

For the outcome objective, the instrument the program is going to use is a questionnaire. The outcome objective is after 6 months, 75% of the participants will be knowledgeable about different contraceptive methods and have the knowledge and resources on how to use it and where to purchase it. The questionnaire is located in the appendix. This questionnaire will allow program coordinators to learn if the program made an impact on participants. The program taught participants about different kinds of contraceptive methods and it also informed them about resources available to them that can help with their sexual health and how to obtain the contraceptives. The questionnaire will show if participants have used contraceptives and if they have used the resources that they were informed about. 

Reliability and Validity

In the contemporary epoch, data collection has been eased significantly. In essence, this has been achieved sufficiently through the aid of the copious knowledge instruments at the disposal. Apparently, one of those particular knowledge instruments happens to be that of surveys. Survey instruments have and are still playing a pivotal role in acquiring imperative information (Wanzer et al., 2019). However, there are two critical facets to consider whenever formulating and testing any survey instrument to obtain essential data. It is the adherence to these two factors that aid in ensuring the survey instrument's eminence and the information gathered for evaluation and utilization. 

Conferring to the two aspects, the survey instrument is only valid if it adequately measures what ought to be measured (Vaske, 2019). Besides, it is deemed valid if it can measure the initial and ultimate indulgence regarding any concern that a specific individual may envision collecting data. On the other hand, the survey instrument is taken to be reliable if it yields imperative and consistent retorts from the selected audience in the event presented to them. In essence, this means that the survey instrument is only considered reliable based on the responses that an individual acquires. Therefore, it is significant for an individual to formulate surveys appropriately so that they can be deemed reliable based on the vital responses obtained from them once they are administered to the targeted audience. 

Ultimately, even though the survey reliability and validity happen to be conspicuously dissimilar, they are inextricably affiliated. The factor of reliability does not entail validity;although it prompts a limit on general validity. Significantly, the proof of validity and reliability is a requirement to guarantee a survey instrument's integrity and eminence.

Data Collection Methods 

The program will use instruments such as a survey to collect quantitative data. Due to COVID-19 and the safety protocol, we will minimize face to face contact as much as possible for the data collection methods. For quantitative data, pre and post-survey will test participants' knowledge on contraceptives benefits and risks and the survey must be completed within the listed dates. Both pre and post-surveys, as stated before, will be done anonymously and submitted through the organization's online portal. The participants have an option to decline the survey under any circumstances. Before the survey is sent out, the organization will post ads requesting participants and there will be the organization contact information attached to the ad. Potential participants can send an email to the provided contact information from the organization requesting a survey. After getting participant contact information through an email, the pre-test survey will be sent to at least 100 random participants 3 weeks before the first seminar. Along with the link to the survey, the email will also contain directions on how to upload the survey once completed. In the survey, the participant will not be asked to list their name on the survey to keep it anonymous but they will be asked to provide their age. Participants will complete the survey within two weeks after receiving it. We will analyze the survey by categorizing them into different age groups and noted on contraceptives that are least known to the audience. Our conference coordinators and data management team will be responsible for the reports of the results. 

After the first survey, the participant will be asked to send in a list of 3 contraceptives that they would like to try the next time they have sex. They can send an attachment through an email or upload the document through a portal. Then 5 months after the program, all participants are asked to take a post-survey containing the same materials as a pre-survey handed out before the program. The only difference is that the new survey will ask what one form of contraceptives the participant will pledge to use after the program. We will also use secondary data to support our need assessment and let others see why the lack of contraceptive use is an issue among young women. We also use pre-existing research data to compare our new data to see if there is a correlation between the two. 

Data Analysis Technique

All of the estimates and percentages in this report are based on sampling weights methods that were designed to produce unbiased national estimates representing women between the ages of 18-24 at the time of each respective survey. Statistics for this report, including sampling errors, were produced using the survey analysis procedure. Overall associations between independent variables and variables on condom use were evaluated as well. Women who used a highly reliable method like IUD, Norplant, patch, Depo Provera, or birth control pills, all of the time indicated that they had a tubal ligation were classified as high in terms of the effectiveness of their contraceptive use whereas women who used other reliable methods (female or male condoms ) all of the time classified as moderate in terms of the effectiveness of contraceptive use (Bloch et al., 2012). Pills use decreased with increasing age. There were 16.6% of women aged 15–19 currently using the pill as compared with 20-29 aged women was 19.5% (FastStats - Contraceptive Use 2019).

References

Act for Youth (n.d.). CAPP Pre-Post Surveys. Retrieved from http://actforyouth.net/sexual_health/community/capp/ebp/pre-post.cfm

FastStats. Cdc.gov. https://www.cdc.gov/nchs/fastats/contraceptive.htm. Published 2020. Accessed November 1, 2020.

Bloch JR, Webb DA, Mathew L, Culhane JF. Pregnancy intention and contraceptive use at six months postpartum among women with recent preterm delivery. Journal of obstetric, gynecologic, and neonatal nursing: JOGNN. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3409429/ . Published 2012. Accessed November 1, 2020. 

Planned Parenthood (2020). Birth Control. Retrieved from https://www.plannedparenthood.org/learn/birth-control

Wilson, K. (2018). How (And Why!) To Write A Pre-Course Survey Or Questionnaire. Northwestern School of Professional Studies. Retrieved from https://dl.sps.northwestern.edu/blog/2018/04/write-pre-course-survey-questionnaire/

Vaske, J. J. (2019). Survey research and analysis. Sagamore-Venture. 1807 North Federal Drive, Urbana, IL 61801. https://eric.ed.gov/?id=ED605453

Wanzer, D., McKlin, T., Edwards, D., Freeman, J., & Magerko, B. (2019, February). Assessing the Attitudes Towards Computing Scale: A Survey Validation Study. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (pp. 859-865). https://dl.acm.org/doi/abs/10.1145/3287324.3287369

Appendix 

1. Are you more informed about different contraceptive methods after going through the program?

a. Yes

a. No

1. Do you know at least 3 different kinds of contraceptive methods?

b. Yes

b. No

1. Of those 3 contraceptive methods do you know how they work/used?

c. Yes

c. No

1. Do you know the benefits of those contraceptive methods?

d. Yes

d. No

1. Do you know where to obtain and where to go to get these contraceptives?

e. Yes

e. No

1. Have you used contraceptive methods in the last 6 months?

f. Yes

f. No

1. Do you know of resources available to you that can help with your sexual health?

g. Yes

g. NO

1. Have you used any of those resources in the last 6 months?

h. Yes

h. No