615 forum

Week

Lesson - Week One

Boulmitus gives you two definitions of evaluation. One that suggests collecting and analyzing data to determine if and to what degree objectives are being achieved, and the other used to enhance the decision-making process. At different times and from different perspectives, the evaluation may be looking at effectiveness, efficiency or impact of a program. It may just give a comparison of activities. Or it may identify areas in need of improvement. Many people and agencies that actively engage in the evaluation process do so in order to secure or keep funding for their programs. The evaluations are used to show benefits and opportunities which they would pursue with funding. In order to "talk the talk" though, you need to be aware of the vocabulary. You will see terms such as program objectives, resources, stakeholders, standards, data source, population sample, design, data analysis and audience. Pay attention to these terms and please try to use them in your forum discussions and assignments. As in law, these are "technical" terms and have a specific meaning in this community. 

Evaluations can be done at all stages of a program or project cycle. The typical program cycle is: Philosophy and Goals, Needs Assessment, Program Planning, Implementation and Formative Evaluation, and Summative Evaluation. Within those parameters, there are many data sources and many stakeholders that should be involved in the process. 

So, when assessing a program, one must thoroughly understand the social problem the program is intended to address; assess the program design in relationship to the needs of the target population; and, involve the relevant experts and stakeholders in the process.

I would like to introduce the program we will use throughout this class to add a practical application for some of the concepts you have started to learn. Everyone is familiar with the  National Park Service (NPS) . We have all visited a number of the sites they own and maintain. The Lincoln Memorial, the Grand Canyon and Yellowstone come to mind immediately but there are many smaller entities within your state that are also owned or managed by the NPS. But did you know that the NPS is mandated by Congress to evaluate their services and report these findings?

The  Federal Lands Recreation Act  (REA) authorizes/requires five agencies to collect and spend recreation fees on lands they manage: the Department of the Interior's (DOIs) Bureau of Land Management (BLM), Bureau of Reclamation, National Park Service (NPS), US Fish and Wildlife (FWS) and the US Department of Agriculture's Forest Service (USFS).  Overall the Act requires visitor fees to be plowed back into the parks to enhance the visitor experience, enhance programs, provide educational experiences and maintain the recreation sites. 

As a consequence of the  Government Performance and Results Act of 1993 , NPS is required to perform visitor surveys. As part of our studies we are going to look at the NPS and how they collect the data as required by law for this report. The data is tracked by many stakeholders:

   –NPS leadership, planners, project managers

  –Congress

–Academic researchers

–The media

–Gateway communities

–Members of the public

 

The data is used for:

•Allocating resources within NPS

–Budget

•Projects within NPS

–Designing / sizing visitor facilities at a park

•Economic Impact Assessments for NPS units

•Planning in Gateway Communities

•Business Development

          “How many visitors” tells businesses about local market for goods and services related to parks

The following lesson tabs will go into more detail on the definitions we will need for this study.

Boulmetis, John & Boulmetis, Phyllis Dutwin (2011). The ABCs of Evaluation: timeless techniques for program and project managers. San Francisco, CA: Jossey-Bass.

· Week One Part Two Week One

· Part of the requirements for the NPS is to track stats of visitors and overnight visitors to their parks. As you evaluate these stats, it is important to establish definitions for what they measure (this will be important for any evaluation you do as the final decision maker may not be familiar with the details of the program). For their purposes, the definitions of recreational visit and overnight stay are important. They follow:

NPS Public Use Statistics Definitions

BACKCOUNTRY – One or more primitive or wilderness areas which are reached primarily by hiking, boating, or horseback.

CAMPGROUND – An area of land designated and developed for use as a camp.

CONCESSIONER – A private company or an individual granted the privilege of providing facilities and services considered necessary by the NPS for accommodating visitors.

CONCESSIONER CAMPGROUND – An element of a concessioner operation involving services for overnight camping.

CONCESSIONER LODGING – An element of a concessioner operation involving facilities for overnight lodging such as hotels, motels, cabins, cottages, trailer villages, and trailer rental.

MISCELLANEOUS OVERNIGHT STAY – Any overnight stay not otherwise defined (sleeping aboard boats, camping in organized groups, and/or any other overnight stays not included in other categories of overnight stays, except inholders, which are non-reportable).

NATIONAL PARK SERVICE (NPS) – A Department of the Interior (DOI) bureau which administers approximately two dozen types of federal land, nationally significant for their scenic, natural, scientific, historical, or archeological interest (see CLASSIFICATION). The agency was established as a Bureau of the DOI by an Act of Congress on August 25, 1916. The NPS does not administer National Forests (United States Department of Agriculture/Forest Service), Wildlife Refuges (United States Department of Interior/Fish and Wildlife Service), or a variety of other lands available for public use.

NON-RECREATION OVERNIGHT STAY – A reportable non-recreation overnight stay includes leaseholders, line shacks for ranchers, and government personnel (other than NPS employees).

NON-RECREATION VISIT – A reportable non-recreation visit includes:

·       Persons going to and from inholdings across significant parts of park land;

·       Commuter and other through traffic;

·       Tradespeople with business in the park;

·       Any civilian activity a part of or incidental to the pursuit of a gainful occupation (e.g., guides);

·       Government personnel (other than NPS employees) with business in the park;

·       Citizens using NPS buildings for civic or local government business, or attending public hearings;

·       Outside research activities (visits and overnights) if independent of NPS legislated interests (e.g. meteorological research).

NON-REPORTABLE VISIT – The entry into a park by NPS employees, their families, concessionaire employees, members of cooperating associations, NPS contractors, and service personnel are not reportable for public use.  Specific situations include:

·       Employees of the NPS who are assigned to the park or are visiting the park in connection with their duty assignment;

·       NPS contractors, concessionaires, cooperating associations and their employees;

·       Temporary or permanent members in household of personnel otherwise included in this definition whose residence is in the park;

·       Private tenants within the NPS boundaries (inholders) if not crossing significant NPS territory for access;

·       Persons engaged in pursuit of specific legal rights of use (e.g., subsistence hunting and fishing);

·       Any other persons whose presence in the park is to help the Service fulfill its mission (e.g., volunteers in the park, research activities associated with the NPS mission).

OVERNIGHT STAY – One night within a park by a visitor.  For example, three visitors for two nights yields 6 overnight stays.

RECREATION VEHICLE – Any enclosed vehicle used for camping which is more elaborate than a simple truck or car, such as pickup truck with camper body, popup tent trailer, travel trailer, bus, motor coach, mobile home, etc.

RECREATION VISIT – The entry of a person onto lands or waters administered by the NPS for recreational purposes, excluding government personnel, through traffic (commuters), tradespeople, and people residing within park boundaries.  Visits originating on surface vehicles (trains, boats, other) and aircraft may be counted if they stop and disembark passengers on NPS administrated territory.  The applicable rule is that one entrance per individual per day is countable.

RV OVERNIGHT STAY – One night within a park by a visitor in an RV in an NPS-owned campground.

TANDEM VISITATION – A visitor going from park to park in one day.  Visits to different parks are counted separately as long as the areas visited are independently authorized units of the National Park System, not separate portions of the same unit.

TENT OVERNIGHT STAY – One night within a park by a visitor in a tent in an NPS-owned campground.

VISIT - The entry of any person, except NPS and service personnel, onto lands or waters administered by the NPS. A visit may occur as a recreation visit or a non-recreation visit. A same-day reentry, negligible transit, and an entry to a detached portion of the same park on the same day are considered to be a single visit. Such adjustments are made insofar as practicable for noncontiguous parts of the same park. However, visits are reported separately for two contiguous parks.

VISITOR DAY - Twelve visitor hours in a park.

VISITOR HOUR - The presence of one or more persons, excluding NPS personnel, in a park for continuous, intermittent, or simultaneous periods of time aggregating one hour (e.g., one person for one hour, two persons for one-half hour).

(Definitions provided for this class specifically aligned with the visitor use survey by the Social Science Program of NPS but a complete list can be found at Street, B. 2014. Statistical abstract: 2013. Natural Resource Data Series NPS/NRSS/EQD/NRDS—2014/635. National Park Service, Fort Collins, Colorado. )Part Three Lesson Week One

There are many evaluation models but four that are highlighted in your text. The Discrepancy Model is used within a complex structure to give assurances on evidence to allow for cause and effect inferences. The Goal-Free Model looks at the actual effect on needs. It is not the program goal but the needs of the audience that are important. The Transaction Model allows for constant feedback between the evaluator, the participants and the project staff. Finally, the Decision-Making Model, as its name implies, looks to make decisions on the future of a program. Table 5.1 will help you determine which models might be most appropriate for your study. Please see the Center for Disease Control's Program Evaluation website. It offers a wealth of resources on evaluation in general and logic models specifically. This site offers many government agency evaluation links as well. </</</