JOURNAL ARTICLE CRITIQUES

TOOLS OF EVALUATION

SPECIAL ISSUE

2015

VOCATIONAL EVALUATION

AND CAREER ASSESSMENT PROFESSIONALS JOURNAL

AND

VOCATIONAL EVALUATON AND WORK ADJUSTMENT ASSOCIATION JOURNAL

TOOLS OF EVALUATION SPECIAL ISSUE

2015

TOOLS OF EVALUATION

SPECIAL ISSUE

2015

Vocational Evaluation and Career Assessment Journal

Steven R. Sligar, Co-Editor Nancy Simonds, Co-Editor

Vanessa Perry, Managing Editor

Vocational Evaluation and Work Adjustment Association Journal

Andrea Nerlich, Co-Editor

Randall S. McDaniel, Co-Editor

Special Issue Reviewers

Samuel Castiglione, Workforce & Technology Center

Debra Homa, University of Wisconsin–Stout Pamela Leconte, The George Washington University

Amanda McCarthy, Northern Illinois University Pat McCarthy, Virginia Department of Aging and Rehabilitative Services

Randall S. McDaniel, Auburn University Andrea Nerlich, Hofstra University

Michael  O’Brien,  New  Mexico  Highlands  University Steven R. Sligar, East Carolina University

Fran Smith, The George Washington University, Recognizing Differences

The Tools of Vocational Evaluation

The story of the development of this joint issue is reflective of our past, present, and hopefully of our future. Pruitt (1986) noted that VEWAA became a professional division of the National Rehabilitation Association (NRA) in 1967, which is a good milestone date. VECAP was established in 2003 (vecap.org). Both organizations have vocational evaluation as a foundational underpinning and the members use tools in the practice of their profession. Fast forward to the 2011 NRA Conference in Chicago when a chance conversation occurred between Randall McDaniel, incoming Co-Editor of the VEWAA Journal and Steve Sligar, Co- Editor of the VECAP Journal about the possibility of a joint effort between the two organizations. Later, Andrea Nerlich, Co-Editor of the VEWAA Journal, joined the conversation. After more discussion between the editors and their respective boards of directors, a collective decision was made to go forward with a joint journal and select a theme.

A survey of the VECAP members (Ahlers-Schmidt, 2010) indicated a strong interest in information on the tools used by practicing evaluators. In 1972, a national task force had been created  to  examine  the  field  of  vocational  evaluation  that  subsequently  published  “The   Vocational  Evaluation  Project  Final  Report”  with  seven  different  sections (Crow, 1975). One section, The Tools of the Vocational Evaluator, lists three types of tools: situations (i.e., on- the-job evaluation, work samples, psychometrics), resources (e.g., occupational and client information, job analysis) and applied tools (i.e., interviewing, observing, and reporting). Forty years  following  that  seminal  publication,  the  timing  seemed  right  to  select  “Tools  of   Evaluation”  as  the  overall  theme  of  this  joint  journal.  With  respective  board  approvals  for  this   collaboration, the editors of the VEWAA and VECAP journals were off to a good start. As part of examining the Tools of Evaluation, the editors decided this special edition should focus on three areas, i.e. assessment tools, methods, and technology. Around these three focus areas, the editors solicited co-authors to produce two manuscripts for each area.

In the spring of 2014, a joint call for contributors on these topics was issued with an overwhelming response from rehabilitation professionals who wanted to participate in these writing teams. The editors produced a rough outline of each manuscript that was shared with the writing teams and a writing team leader was selected for each manuscript. The writing volunteers were eventually organized into five teams and the writing began. Andrea Nerlich took the lead with Randall McDaniel and Steve Sligar providing assistance during the writing, peer review, and editing processes. Further support was provided by Vanessa Perry, VECAP Journal Managing Editor, who designed the cover and layout, and Nancy Simonds, VECAP Journal Co-Editor, who proofread the special issue. The key to the success of this effort lay with the teams listed below.

 Chad Betters and Steve Sligar contribute the results from their research on tools used by vocational evaluators in state vocational rehabilitation programs.

 Lee Ann Rawlins-Alderman, Robin E. Dock, Megon Steele, and Leslie Wofford

EDITORIAL

provide a thorough discussion of current vocational assessment methods.  Bryan S. Austin, Carl W. Sabo, Amanda K. McCarthy, Matthew E. Sprong, and

Lauren N. Noble describe interviewing as an assessment tool.  Frances Smith, Pamela Leconte, William E. Garner, and Veronica I. Umeasiegbu

provide an overview of current technology used in vocational evaluation.  Randall S. McDaniel, Scott Beveridge, Christian Chan, and Jeremy Cushen review

potential future technology and its probable impact on vocational evaluation. The editors of this first combined VEWAA and VECAP journal are proud of this collaborative effort and similar to the 1972 task force—a diverse group of vocational evaluators from around the country pulled together to discuss tools for evaluation.

Andrea Nerlich Randall S. McDaniel Steven R. Sligar Co-Editor, VEWAA Co-Editor, VEWAA Co-Editor, VECAP

References Ahlers-Schmidt, C. R. (2010). Results of the VECAP Journal readership survey. Vocational

Evaluation and Career Assessment Journal, 6(2), 45–47. Crow, S. H. (July 1975, Ed.). Vocational evaluation project final report. Vocational Evaluation

and Work Adjustment Bulletin, 8, 1–165. Pruitt, W. A. (1986). Vocational evaluation (2nd ed.). Menomonie, WI: Walt Pruitt

Associates.

Editors’  Note:

Reference to this special issue should be cited as Nerlich, A. P., McDaniel, R. S., & Sligar, S. R. (Eds.). (2015). Tools of Evaluation [Special

issue]. Vocational Evaluation and Work Adjustment Association Journal and Vocational Evaluation and Career Assessment Journal.

Reference to an article should be cited as Betters, C. J. & Sligar, S.R. (2015). The tools of the trade: A national study on tool utilization

in vocational evaluation. [Special issue]. Vocational Evaluation and Work Adjustment Association Journal and Vocational Evaluation and Career Assessment Journal, 8–16.

VOCATIONAL EVALUATION AND WORK ADJUSTMENT

ASSOCIATION JOURNAL AND

VOCATIONAL EVALUATION AND CAREER ASSESSMENT JOURNAL

TOOLS OF EVALUATION, SPECIAL ISSUE

CONTENTS Page

ISSUE The Tools of the Trade: A National Study on Tool Utilization in Vocational Evaluation Chad J. Betters and Steven R. Sligar Current Vocational Assessment Methods Lee Ann Rawlins-Alderman, Robin E. Dock, Megon Steele, and Leslie Wofford Utilization of Interviewing as an Assessment Tool to Enhance Vocational Rehabilitation Service Delivery: Fostering the Therapeutic Alliance  and  Professionals’  Judgment  Accuracy   Bryan S. Austin, Carl W. Sabo, Amanda K. McCarthy, Matthew E. Sprong, and Lauren N. Noble Current Technology in Vocational Evaluation: Trends and Opportunities Frances Smith, Pamela Leconte, William E. Garner, and Veronica I. Umeasiegbu The Impact of Future Technology on Vocational Evaluation Randall S. McDaniel, Scott Beveridge, Christian Chan, and Jeremy Cushen

8

18

37

53

72

Tools of Evaluation, Special Issue 2015 8

The Tools of the Trade: A National Study on

Tool Utilization in Vocational Evaluation

Chad J. Betters

Winston-Salem State University

Steven R. Sligar East Carolina University

Abstract The results from a national study are presented. Key employees from 14 state vocational rehabilitation programs identified tools used by vocational evaluators in their day-to-day practice. There were 433 individual tools listed with 197 (45.5%) psychometric tests representing eight constructs: achievement, aptitude, career planning/development, intelligence, personality, vocational  interest,  work  values,  and  “other.”  There  were  236  (54.5%)  work  samples  listed  (14   complete commercial work sampling systems, 59 individual samples from commercial systems, and 153 evaluator-created work samples). Listings of the most frequently reported tools are provided. Results indicate that tools are similar to those described in three seminal works, warn of the danger of defining the field by its tools, and identify potential ethical violations. Keywords: Vocational Evaluator, Evaluation Tools, Work Sampling, Psychometric Testing

The Tools of the Trade: A National Study on Vocational Evaluation Tool Utilization

Benjamin Franklin stated: The best

investment is in the tools of one's own trade. Vocational evaluators use a variety of tools to accomplish their job, which is to empower clients to choose a career. Three seminal works on vocational evaluation contain descriptions of tools. The Vocational Evaluation and Work Adjustment (VEWAA) Project (Crow, 1975) participants stated there are three types of tools: situations (i.e., on-the-job evaluations,

work samples, and psychometrics); resource (i.e., occupational and client information, job analysis, and audio-visual); and applied (i.e., interviewing, observing, and reporting). Pruitt (1986) listed ten components of a vocational  evaluation:  “occupational   information, work samples, situational assessment, community based assessment, psychological testing, special projects, observation, A-V material, client information, feedback sessions, and interviewing”  (p.  22).  Pruitt  also  emphasized   the importance of work samples and using actual tools from the job. Thomas (1999)

Tools of Evaluation, Special Issue 2015 9

described three categories of tools: instruments (i.e., tests and work samples); techniques (e.g., situational or community- based assessments); and strategies (i.e., accommodations, modifications, and learning style assessment to identify supports). This study sought to identify tools used by vocational evaluators in their day- to-day practice.

Methods

This analysis is a component of a

larger benchmarking study focusing on employment conditions for vocational evaluators working within the state vocational rehabilitation (VR) system (Sligar & Betters, 2012). A 32-item survey was created and data collected from June 2010 to April 2011. The specific questions related  to  this  analysis  were  “Is  there  a  list   of tools (e.g., tests, work samples, specific techniques) used by vocational evaluators? If  yes,  will  you  share  it  with  us?”  The  survey   was administered by research assistants (RAs) who were trained to follow a protocol of how to identify participants, record responses, and follow-up to obtain lists of tools and other information.

The target sample was employees of the 64 general, blind, or combined VR programs in the United States. In order to collect statewide information, purposeful sampling was used. An incumbent with statewide responsibilities for the VE program  was  sought  in  each  state’s  central   office. This position was typically a program specialist. The RA recorded the VR contact person’s  information  and  responses  in   Survey Monkey, an online survey tool. Descriptive statistics were used in the data analysis. The study has Institutional Review Board approval through East Carolina University.

Results

As indicated in the original benchmarking study (Sligar & Betters, 2012), 63 of the 64 state vocational rehabilitation programs provided data. Of the 63 reporting programs, 26 programs employ vocational evaluators. When specifically looking at tools used by vocational evaluators working within the state vocational rehabilitation system, 24 programs maintain a listing of tools that their evaluators utilize when working with clients. When asked, 14 (12 general/combined and 2 blind services) of the 24 programs shared their list.

Upon combining the lists for all 14 programs, there were 433 individual tools identified. The 433 tools consisted of 197 (45.5%) psychometric tests that included 171 from various constructs: achievement, aptitude, career planning/development, intelligence, personality, vocational interest, and work values. Another 26 psychometric tests were also identified, which were grouped  as  “other”  given  they  did  not  fall   into the previously recognized constructs. There were also 236 (54.5%) work samples provided, including 14 complete commercial work sampling systems, 59 individual samples from commercial systems, and 153 evaluator-created work samples, which can also be considered specific task-related samples. The following data represents, by construct, the most frequently used tools and the number of the 14 reporting state vocational rehabilitation programs utilizing the tools. Achievement

Achievement testing measures

clients’  knowledge  from  formal  learning  and   life experiences (Power, 2013, p. 269). A total of 32 achievement instruments were

Tools of Evaluation, Special Issue 2015 10

identified, with top five by frequency indicated in Table 1. Additional achievement tools that were also mentioned include the Woodcock-Johnson (three programs), the Nelson Denny Reading Test (two programs), and the Gates-MacGinitie Reading Test (two programs). Aptitude

Aptitude testing assesses “individuals’  skills  and  abilities”  (Parker,   2008, p. 123). The 14 programs listed 41 measures of aptitude, with the top five by frequency provided in Table 1. Notable aptitude instruments also included the Minnesota Ability Test Battery (four programs), the Purdue Pegboard (four programs), the Crawford Small Parts Dexterity Test (three programs), and the SRA Test of Mechanical Concepts (two programs). Although the Purdue Pegboard is also considered by some vocational evaluators as a work sample, it was reported as an aptitude tool by the respondents, and therefore was included in this section of the analysis.

Career Planning/Development

Formal career assessment uses psychometric instruments to help a client develop  an  inventory  of  “personal  and   environmental  characteristics”  that  facilitate   career choice (Power, 2011, pp. 207–208). A total of 17 tools were mentioned, with the top four by frequency provided in Table 1. Additional career planning/development tools that were included were the Career Maturity Inventory, the Student Styles Questionnaire, the College Survival and Success Scales, and the Career Development Inventory, each utilized by one program.

Intelligence

Intelligence is the capacity to utilize mental abilities, such as memory, abstract reasoning, and analogic reasoning, in order to solve novel problems (Power, 2013, p. 193). There were 14 instruments used by the programs, including the top four by frequency provided in Table 1. Other intelligence tools included the Kaufman Brief Intelligence Test, the Slosson Intelligence Test, and The Test of Nonverbal Intelligence, each utilized by one program. Personality

Personality assessment is used to quantify  “influences  that  explain  a  person’s   behavior  in  a  specific  situation”  (Krug,   2008, p. 153). Nine personality measures were reported with the top four by frequency provided in Table 1. The Taylor-Johnson Temperament Analysis and the Tennessee Self-Concept Scale were both used by one program.

Vocational Interest

Vocational interest indicates a “preference  for  work  environments  and   outcomes”  (O*Net,  n.d.a) and the use of inventories empowers clients to make informed career choices (Fouad, Smothers, Kantamneni, & Guillen, 2008, p. 216). The 14 programs listed 43 vocational interest tools, with the top five by frequency provided in Table 1. Other notable measures included the Picture Interest Survey (four programs), the Geist Picture Interest Inventory (two programs), the Strong Interest Inventory (one program), and the Campbell Interest and Skill Survey (one program).

Tools of Evaluation, Special Issue 2015 11

Work Values

Work  values  are  “global  aspects  of   work  that  are  important  to  a  person’s   satisfaction”  (O*Net,  n.d.b) and these instruments assess the work values construct. There were 15 instruments shared,

with the top five by frequency provided in Table 1. Additional tools used to measure work values included the World of Work Inventory, Work Values, Personal Audit, and the Entrepreneurial Readiness Inventory, each used by one program.

Table 1 Top Five Psychometric Tools by Frequency for Vocational Evaluation Constructs

CONSTRUCTS TOOLS # OF PROGRAMS

Achievement Wide Range Achievement Test (WRAT 3/WRAT 4) 3/7 SRA Arithmetic/Reading/Verbal Forms 5/5/2 Test of Adult Basic Education 7 Adult Basic Learning Examination 5 Wonderlic Basic Skills Test 4

Aptitude Career Ability Placement Survey 7 Career Scope 6 Computer Operator/Programmer Aptitude Battery 5 Differential Aptitude Test 5 Revised Minnesota Paper Form Board Test Series 5

Career Planning* Barriers to Employment Success Inventory 6 Asset Career Skills 3 Career Exploration Inventory 3 Career Thoughts Inventory 2

Intelligence* Beta III 6 Raven’s  Progressive  Matrices 3 MECA Emotional Intelligence Profile 2 Shipley Institute of Living Scale 2

Personality* Myers Briggs Type Indicator 5 Hogan Personality Inventory 2 Holland Code Exercise 2 16 Personality Factors 2

Tools of Evaluation, Special Issue 2015 12

Table 1 (continued) Top Five Psychometric Tools by Frequency for Vocational Evaluation Constructs

CONSTRUCTS TOOLS # OF PROGRAMS

Vocational Interest

Self-Directed Search 12 Career Assessment Inventory 9 Becker Reading-Free Interest Inventory 8 Career Occupational Preference System 8 Wide Range of Interest Opinion Test 2 6

Work Values Career Orientation Preference Evaluation Survey 6 Work Motivation Scale 3 Becker Work Adjustment 2 Work Orientation and Values Survey 2 Work Preference Match 2

*These constructs had fewer than five tools with a greater than one program frequency. “Other” Instruments categorized into “other”  include   psychometric instruments that did not fit with the previously mentioned constructs. Overall, the instruments within this group served  as  various  measures  of  a  client’s   functionality. Although there were 26 instruments mentioned, the majority of the tools were used by only one program each. Instruments that were used by more than one program are included in Table 2.

Work Samples

Work  samples  are  “generalized  work  like   tasks that are administered under specific instructions”  (Power,  2011,  p.  246) that measure  a  person’s  work  capabilities   (Power, 2013, p. 275). In the collected tool

data, work sampling clearly separated into two different categories: commercial work sample systems and specific work samples from these systems, and evaluator-created work samples. Among the 14 state vocational rehabilitation programs, 14 commercial systems were reported, with the top eight systems (those used by more than one program) provided in Table 3. Commercial work samples were also reported as individual measures as well, since a single component (e.g., Valpar Component Work Sample [VCWS] #9) may be used by a program, but not the entire VCWS system. When looking at the data, 59 individual samples from various commercial work sampling systems were reported by the 14 programs.

Tools of Evaluation, Special Issue 2015 13

Table 2 “Other”  Psychometric Tools by Frequency

TOOLS # OF PROGRAMS Transition to Work Inventory 3 Daily Living Skills Inventory 2 Emotional Behavior Checklist 2

Kenexa Prove It! 2 OASYS* 2

Offender Reintegration Scale 2 *Although not a psychometric test, it was reported as such by two programs. Table 3 Commercial Work Sampling Systems by

Frequency

SYSTEMS # OF PROGRAMS

Valpar CWS 7 JEVS 3

Minnesota Clerical Test

3

Aviator 3000 2 Gilbertson Basic

Accounting Work Sample

2

McCarron-Dial 2 Tower Clerical Series 2

VITAS 2 Evaluator-Created Work Samples

The 14 programs also reported 153

evaluator-created work samples, which appeared to serve as specific task-related measures. The tasks varied from basic skills (measuring) to simple tasks (e.g., dusting, mopping, Bissell [vacuuming], silverware wrapping) to semi-skilled jobs (e.g., bike repair, oil/lubrication, small engine assembly).

Discussion The discussion focuses on three

areas. First is a comparison of tools used with the seminal works. All three authors (Crow, 1975; Pruitt, 1986; Thomas, 1999) included psychometric tests and work samples as tools used in vocational evaluation. The results indicate this is still the case today with a close split between work samples (54.5%) and psychometric tests (45.5%). Second is the continuing need to be aware of the profession being identified by its tools (Thirtieth Institute on Rehabilitation Issues [30th IRI], 2003). The IRI scholars cautioned that using one approach  or  method  “…can  be  quite   restrictive, especially for those who are severely disabled or represent an ethnic minority”  (p.  233).  Despite  the  variety  of   tests and work samples listed in the survey, the respondents did not report other tools, such as measures of physical capacity or situational assessment.

The third area has two parts that fall under the Guidelines for Professional Conduct for the Professional Vocational Evaluator (RPVE; Registry of Professional Vocational Evaluators, 2011). First is that the tests and work samples used are within the limits of competence (p. 4), which means the vocational evaluators must have the educational background to use tests and work samples. Evaluators meet this

Tools of Evaluation, Special Issue 2015 14

requirement though additional training may be  required  to  enhance  the  evaluator’s  skills.   Additionally,  evaluators  meet  the  vendor’s   minimal requirements to purchase various tests. The second part, use and selection of the instruments (p. 4), is of greater concern especially in regards to norms and standardization. The evaluator is in a caveat emptor situation with many of the tests. Not only must the evaluator determine if the test measures the construct needed but must factor such characteristics as disability and ethnicity into the decision to select. Some tests have different editions, such as the Wide Range Achievement Test 3 and 4 (Wilkinson & Robertson, 2006) and both the newer and older editions are apparently in use. Other instruments, such as the Crawford Small Parts Dexterity Test that was discontinued in April 2011 (Pearson Canada, 2014), are reported as still in use. The RPVE Guidelines (2011) indicate that using outdated tests is unethical.

Even more disconcerting are the commercial work sample systems. The respondents listed the Jewish Employment and Vocational Services Work Samples or JEVS, which is no longer available. The JEVS became the Vocational Interest Temperament, and Aptitude Scale (VITAS) that is also no longer supported by the vendor (Vocational Research Institute, 2011). Valpar International only supports VCS #9 Whole Body Range of Motion (Valpar International, 2014) and the other VCWS are still available from BASES of Virginia (http://www.basesofva.com/about/). Valpar has developed new products such as the Joule, a functional capacity evaluation, and Pro3000, a modular assessment, database, and reporting system (http://www.valparint.com/). Simwork Systems has Sim Work Samples and the ERGOS II Work Simulator System (http://www.simwork.com/Home.aspx). None of the new products were mentioned.

The continued use of older systems when newer technology is available impedes delivery of services that are state of the art.

Limitations

This study has several limitations. First,  this  study’s  goal  was  to  benchmark   utilized tools within state vocational rehabilitation programs, and therefore can only depict the currently used tools at the time of data collection. Second, data was collected via a telephonic survey, which inherently has potential concerns regarding internal validity. Third, there is an assumption that the individual from each state vocational rehabilitation program who provided the data did indeed provide correct information. Although highly unlikely, there is a possibility that a listing of tools that are currently not utilized was accidentally sent rather than a current listing.

Finally, delimitations in the study must be noted. The data was delimited to state vocational rehabilitation programs that were willing to participate. One state chose to not participate. No data from community- based or private sector rehabilitation programs was collected, and therefore tool utilization within these two arenas is not addressed.

Future Research

Additional future research on vocational evaluation tool utilization is warranted. While a listing of tools was generated,  the  frequency  of  each  tool’s  use   would reveal more of the actual day-to-day practices of the vocational evaluator. Tool availability also needs to be studied to determine if a common protocol can be developed. Community-based and private sector vocational evaluation needs to be examined in order to see what differences, if

Tools of Evaluation, Special Issue 2015 15

any, exist compared to state vocational rehabilitation programs. As indicated in the article for the larger study conducted by Sligar & Betters (2012), an exact replication of this study would be advantageous, as changes in tool utilization could be noted given the benchmark established with this data. Future research should also examine how variance in tool utilization among state vocational rehabilitation programs may or may not influence vocational rehabilitation services, including any impact on successful closures. What role does psychometric testing vs. work sampling play in promoting successful vocational outcomes? How do evaluator-created work samples assist in the process?

Conclusions

There are two conclusions from this

study: one is troubling and the other is positive. The identification and implied use of older editions or even discontinued tests and work samples is troubling. This practice may be more reflective of current decreased funding levels that leads vocational evaluators to use older, available tools, rather than a conscious decision not to add new items to the inventory. Regardless, the practice of using potentially out-of-date tools is not only unethical but also limiting to the collection of accurate data about clients’  interests,  aptitudes,  and  the  other   constructs described in this paper. Vocational evaluators must continue in their role as educators (Thirtieth Institute on Rehabilitation Issues, 2003) in order to advocate for the necessary tools to continue to provide excellent services.

The second conclusion is that vocational evaluators have a variety of tools. There were 433 identified by 14 programs. If the other programs had shared their lists, then more tools would probably have been

identified. Considering the 433 tools, vocational evaluators are able to use a variety of different tools to collect data and this variance adds to the usability and accuracy of the results (Thirtieth Institute on Rehabilitation Issues, 2003).

References

Crow, S. (1975). Vocational evaluation

project final report. Vocational Evaluation and Work Adjustment Association Bulletin (reprint). Menomonie, WI: University of WI– Stout Materials Development Center.

Fouad, N. A., Smothers, M. K., Kantamneni, N., & Guillen, A. (2008). Vocational inventories. In B. Bolton & R. M. Parker (Eds.), Handbook of measurement and evaluation in rehabilitation (4th ed.) (pp. 209– 240). Austin, TX: PRO-ED.

Krug, S. E. (2008). Assessment of personality. In B. Bolton & R. M. Parker (Eds.), Handbook of measurement and evaluation in rehabilitation (4th ed.) (pp. 151– 173). Austin, TX: PRO-ED.

O*Net OnLine (n.d.a). Browse by O*Net Data Interests. Retrieved from http://www.onetonline.org/find/descr iptor/browse/Interests/

O*Net OnLine (n.d.b). Browse by O*Net Data Work Values. Retrieved from http://www.onetonline.org/find/descr iptor/browse/Work_Values/

Parker, R. M. (2008). Aptitude testing. In B. Bolton & R. M. Parker (Eds.), Handbook of measurement and evaluation in rehabilitation (4th ed.) (pp. 121–150). Austin, TX: PRO- ED.

Pearson Canada. (2014). Crawford Small Parts Dexterity Test discontinued as of April 2011. Retrieved from

Tools of Evaluation, Special Issue 2015 16

http://www.creativeorgdesign.com/te sts_page.htm?id=72

Power, P. W. (2011). A guide to career management and programming for adults with disabilities: A 21st century perspective. Austin, TX: PRO-ED.

Power, P. W. (2013). A guide to vocational assessment (5th ed.). Austin, TX: PRO-ED.

Pruitt, W. A. (1986). Vocational evaluation. Menomonie, WI: Walt Pruitt and Associates.

Registry of Professional Vocational Evaluators. (2011). Guidelines for professional conduct for the Professional Vocational Evaluator. Retrieved from http://pveregistry.org/wp- content/uploads/2011/03/PVEGuidel inesofProfessionalConduct.pdf

Sligar, S. R., & Betters, C. J. (2012). The state of the state vocational evaluator: A national study. Journal of Rehabilitation, 78(4). 21–30.

Thirtieth Institute on Rehabilitation Issues. (2003). A new paradigm for vocational evaluations: Empowering

the VR consumer through vocational information (30th Institute on Rehabilitation Issues). Washington, DC: Rehabilitation Services Administration, U.S. Department of Education. Retrieved from http://www.iriforum.org/download/I RI30.pdf

Thomas, S. W. (1999). Vocational evaluation in the 21st century: Diversification and independence. Journal of Rehabilitation, 65(1), 10– 15.

Valpar International (2014). Valpar replacement parts and supplies catalog. Retrieved from http://www.valparint.com/supplies.p df

Vocational Research Institute. (2011). Products: Legacy products. Retrieved from http://www.vri.org/products

Wilkinson G. S., & Robertson, G. J. (2006). Wide Range Achievement Test 4 professional manual. Lutz, FL: Psychological Assessment Resources.

Author Note Chad J. Betters, PhD, CRC, CDMS, CVE, PVE, is an Assistant Professor in the Rehabilitation Studies program at Winston-Salem State University. He completed his MHS degree in Rehabilitation Counseling and his PhD in Rehabilitation Science at the University of Florida. Betters  has  more  than  ten  years’  experience  as  a  rehabilitation  educator.  His  research  and  clinical   interests include spirituality and vocational rehabilitation, as well as vocational evaluation methods and practices. With more than a dozen publications and more than forty presentations, Betters is a nationally recognized expert in these areas. Prior to becoming an educator, he operated a private rehabilitation consulting and evaluation practice. Betters is past president of NCVEWAA, past NRA board representative for VEWAA and past president of RPVE.

Tools of Evaluation, Special Issue 2015 17

Steven R. Sligar, EdD, CVE, PVE, serves as an Associate Professor and Coordinator of the Graduate Certificate in Vocational Evaluation in the Department of Addictions and Rehabilitation Studies at East Carolina University. He has a master’s in Rehabilitation and Special Education with a specialization in vocational evaluation from Auburn University and doctorate in Adult Education and Human Resource Development from Northern Illinois University. Presently, he serves as a subject matter expert for a national grant with Ohio State University to develop career tests for persons who are deaf and use American Sign Language. He is also the co-editor of the Vocational Evaluation and Career Assessment Professionals Journal and President of the NC chapter of the Vocational Evaluation and Work Adjustment Association.

Return to Table of Contents

Tools of Evaluation, Special Issue 2015 18

Current Vocational Assessment Methods

Lee Ann Rawlins-Alderman Auburn University

Robin E. Dock Winston Salem State University

Megon Steele Roosevelt Warm Springs Institute for Rehabilitation

Leslie Wofford Auburn University

Abstract The vocational evaluation process is an organized and individualized approach for accurately predicting the consumers’  vocational  functioning  potential,  developing  meaningful  vocational   objectives, and ultimately, finding successful employment. This process is a multifactorial, individualized assessment with an ultimate goal of meeting the specific needs of the unique consumer. While a variety of vocational evaluation methods exists, each method and its related assessment tools is selected based on the specific needs, and tailored to answer the referral questions for the identified consumer. The appropriateness of the methods used in each unique evaluation will depend on the functional strengths and limitations, past experiences, educational background, and specific needs of the consumer. Psychometric, work sampling, informal, and observational assessment methods are frequently used within the vocational evaluation process for different purposes and in a variety of situations. The authors discuss the strengths and limitations of current vocational assessment methods, and provide examples of application to the field.

Keywords: Vocational Evaluation, Testing, Work Samples

Current Vocational Assessment

Methods The vocational evaluation process

involves the use of multiple methods of gathering information that will assist the consumer in efficient vocational decision- making (Frey, 1984; Havranek, Field, & Grimes, 2005). This process is an ongoing activity that leads to a successful plan for employment. The vocational evaluation process is an organized and individualized approach for accurately predicting the

consumers’  vocational  functioning  potential,   developing meaningful vocational objectives, and ultimately, successfully finding employment. Referrals for vocational evaluations can span from asking about the skills and aptitudes of a consumer for a particular job to identifying specific interests within the wide world of work.

The vocational evaluation process is a multifactorial, individualized assessment to meet the specific needs of the unique consumer. While a variety of vocational evaluation methods exists, each method and

Tools of Evaluation, Special Issue 2015 19

its related assessment tools is selected based on the specific needs and tailored to answer the referral questions for the identified consumer. The appropriateness of the methods used in each unique evaluation will depend on the functional strengths and limitations, past experiences, educational background, and specific needs of the consumer. Psychometric instruments, work sampling, informal, and observational assessment methods are frequently used within the vocational evaluation process for different purposes and in a variety of situations. Within this article the authors discuss the strengths and limitations of vocational evaluation methods, including examples of application in the field.

Psychometric Assessment in the Vocational Evaluation Process

Psychometric assessment is one

method within the multidisciplinary, as well as interdisciplinary, vocational evaluation process (Flansburg, 2011; Havranek et al., 2005; Leconte, 1994). Psychometric assessment includes a set of formal, structured measures that have been scientifically researched and tested to ensure that they are fair, reliable, and valid. They are administered and scored in a standardized manner, allowing for comparison to a norm group of similar populations who have completed the measures in the past. Since objectivity is key to using these assessments, a good psychometric measure provides fair and accurate results each time it is given. Psychometric assessment commonly includes personality profiles, reasoning tests, motivation questionnaires, and ability assessments to determine disability status, vocational rehabilitation services eligibility, and vocational needs (Condon, Gandolfo, Brugnaro, Thomas, & Donnelly, 2004; Flansburg, 2011; Havranek et al., 2005).

Fundamentals of Psychometric Assessments

Within the vocational evaluation process, the most common psychometric assessments cover the categories of intelligence, aptitude, educational achievement, personality, and temperament. Psychometric assessment typically precedes other types of assessment methods to satisfy the need for collection of information relevant to eligibility for vocational rehabilitation (VR) services, disability status, basic academic achievement levels, and functional capacity (Havranek et al., 2005). Due to the education, training, and supervised experience required for most psychometric assessment measures, a master’s  prepared  rehabilitation  counselor  or   vocational evaluator may not have the credentials to administer, score, and interpret these measures (Power, 2013; Robinson & Drew, 2014). However, the rehabilitation counselor and vocational evaluator must be competent in understanding this data to be able to integrate it with the data obtained by other assessment methods in the vocational evaluation process. If a vocational evaluator is unsure whether his or her credentials meet the national requirements for administering psychometric assessments, he or she can review the requirements available at the American  Psychological  Association’s  web   page (www.apa.org) or contact the test publisher.

Frequently, psychometric assessment measures are the only way of obtaining certain kinds of information. Basic reading, spelling, writing, and math skills are commonly assessed through psychometric assessment measures. The legal definition of some disabilities varies by state and is based on IQ scores from specific psychometric measures that are accepted as incrementally

Tools of Evaluation, Special Issue 2015 20

valid data. Eligibility determination for some state VR agencies requires that information be obtained through tests that are considered psychometric measures (Campbell & Fox, 2002). Although many traits measured by a psychometric instrument are abstract in nature (e.g., reasoning, temperament, emotional functioning), these instruments often identify vocational assets and limitations that are not determined by other assessment methods. Psychometric assessment measures are  commonly  used  to  clarify  a  consumer’s   functional capacity for worker traits, such as general education achievement (e.g., verbal reasoning, math computation, reading, numerical reasoning, and written language), aptitude for general learning ability, verbal traits, and numerical processing (Havranek et al., 2005; Power, 2013).

Psychometric Assessment Limitations

One criticism of psychometric assessment use is that it is removed from assessing ability for jobs and a poor predictor of employment success for persons with certain disabilities (Anthony & Jansen, 1984; Brown, McDaniel, & King, 1995). These tests often measure cognitive abilities rather than the psychomotor abilities and work behaviors that are more closely related to skills used on the job (Geisinger, Boodoo, & Noble, 2002). Frequently, psychometric measures have not included individuals with disabilities in the normative sample population, and individuals with disabilities are being compared to peers without disabilities. Psychometric assessment measures are also considered ineffective in evaluating people with disabilities who have low literacy levels (Power, 2013).

Jobs are often ruled out for individuals with disabilities within the psychometric assessment process due to comparison with established or normed

performance levels. This use of cut-off scores can unfairly restrict vocational options for many individuals with disabilities (Choubon, Stewart, & McGrew, 1991; Lightner, 1994). Thomas (1994) noted that the use of standardized tests may offer questionable validity for many individuals with disabilities due to low reading levels, test anxiety, accommodation needs, and lack of concreteness, which was not considered with the normative sample of comparison. Psychometric assessment measures may be discriminatory and may actually screen out individuals with disabilities (Lightner, 1994).

Although several psychometric measures offer alternative administration guidelines for individuals with certain disabilities, the premise behind the psychometric assessment process (standardization, reliability, validity) often presents a significant disadvantage to the vocational evaluator (Geisinger et al., 2002). The standardized administration of psychometric assessment measures is frequently violated for individuals with disabilities when modifications and/or accommodations are applied during the assessment process (Campbell & Fox, 2002). Modifications or accommodations influence the normed comparison and interpretation of many psychometric assessment measures, yet valuable data may be obtained about the consumer (Geisinger et al., 2002; Robinson & Drew, 2014; Warschausky, et al., 2011). However, Warschausky and colleagues (2011) identified measurement stability between select standard and modified versions of psychometric measures when the source of the modification and/or accommodation involved the use of assistive technology with children ages 6 to 12 who had cerebral palsy.

Tools of Evaluation, Special Issue 2015 21

Section Conclusions

Psychometric assessment, within the vocational evaluation process, is commonly used to gather specific information about the consumer as the results relate to his or her employment potential. For example, if the consumer’s  cognitive  abilities  and  academic   achievement are below average based on psychometric assessment measures, the rehabilitation counselor may not want to consider employment that requires a 4-year college degree for this consumer. Although psychometric measures are frequently used to determine eligibility for VR services (e.g., psychological evaluations), these assessment measures may be modified or adapted for the purpose of gaining useful information, rather than mere comparison with a normative sample . The use of psychometric assessments provides the consumer and the rehabilitation professional with vocationally relevant information to facilitate more accurate vocational planning, decision- making, and career development.

Work Sampling in the Vocational

Evaluation Process

In addition to more traditional testing methods, many professionals include work samples in their overall rehabilitation evaluation and decision-making processes. One of the best ways to assess an individual’s  job  potential  is  to  have  the   person complete tasks he or she will be asked to perform in a potential work environment. Work samples are typically used to assess current skills and abilities. Based on the premise of behavioral consistency—where the way a person acts in a simulated situation is assumed to be the same as he or she might perform on the job—work samples can also be used to analyze the ability to learn new skills. Work sampling is seen as a fact finding

mechanism, utilizing instantaneous and unbiased observations at random times. These pieces of work are performed to make inferences surrounding the functional capacity of the individual in the areas of vocational aptitude, worker characteristics, and vocational interests (Corthell & Griswold, 1987; Nadolsky, 1981). The main objective of work sampling is to determine productivity by using the proportion of activities observed to estimate the actual activities performed during working hours, while assessing job skills potential and work related behaviors. Further, work samples include essential functions of the jobs that they represent, which assists the evaluator in identifying  a  consumer’s  potential  for  a   specific type of job or area of work.

Historically, work sampling—also referred to as activity sampling or ratio delay study—reflects a work measurement technique, with early proponents such as L. H. C. Tippett and R. L. Morrow (Pruitt, 1986). Snap reading method was the first term used, referring to its most important feature of taking activity snapshots. In 1952, the work sampling term was first used by C. L. Brisley, where it originated within the industrial engineering and management fields. Later, the method was extended to the health care industry and other areas. Today, work sampling is one of the most appreciated work measurement methods because of the speed, low cost, limited training requirement, and accuracy it provides in the vocational evaluation process.

To determine strategically the suitability of developing work samples for a given job area, the vocational evaluator must consider the labor market the sample will assess, as well as whether or not the skills required for the job are possessed by the consumer. Each work sample should focus on an array of activities and components of the actual job setting. Work activities,

Tools of Evaluation, Special Issue 2015 22

materials, tools, layout, and physical conditions of the work sample must resemble the actual jobs as closely as possible. Further, the work sample should be developed for maximum applicability and be strong in content validity. The single most important criterion for a good work sample is content validity, and this valid approach incorporates job analysis, task analysis, standardization of instructions, layout, and scoring (Power, 2013). Types of Work Samples

There are several types of work samples. Some applied work samples are samples of work that have been taken in entirety from an industry or business and brought into the evaluation unit for the purpose  of  determining  an  individual’s   interests, skills, and abilities to perform a particular job. These jobs are noted as replicating a segment of essential work factors emerging from the community (Power, 2013; Pruitt, 1986). Other work samples include both single and cluster traits. A single trait sample is seen as assessing a single work trait or characteristic that may have relevance to a specific job or many jobs, but is intended to assess a single trait in isolation. In contrast, cluster traits contain a number of traits inherent to a job and/or a variety of jobs. These types of traits, based on an analysis of occupational grouping and traits necessary for successful performance, are intended to assess the consumer’s  potential  to  perform  various  jobs   and are assessed inclusively. As noted by Power (2013), the differences among these types are a matter of emphasis with all disclosing specific abilities, with results emerging from a series of work samples combined to inform both the evaluator and the consumer of the overall profile of performance.

Work Sampling Advantages

There are several major advantages of the work sample method. This method is the closest approximation of the reality of work that can be achieved in the evaluation setting. It can provide exposure to and experiences in a wide variety of jobs while not only assessing skills, but also revealing aspects  of  the  consumer’s  personality,   interests, and attitudes toward the world of work. Further, performance of work-related activities and concrete tasks can provide direct feedback to both the evaluator and the consumer regarding issues of worker performance, as performance identical to work is required in a work sample system. Additionally, it can be advantageous when an  evaluee’s  motivation  is  considered  in   relation to performance standards of the actual job, with the individual responding more naturally to work- related than abstract- related tasks. Finally, as noted by Power (2013), the use of work sampling can eliminate cultural, educational, and language barriers in the assessment of overall vocational potential, as individuals realize that they are working on practical tasks that are directly related to the vocational planning process. Work Sampling Limitations

A criticism of work samples is that they lack a theoretical basis and are related to an empiricist and Western view of the individual and work (Searle, 2003). Work samples must be carefully designed to test specific items, as the lack of standardization may result in dissimilar results in diverse environments. Further, problems may emerge when more attention is paid to face validity (i.e., how relevant a test appears) than content validity (i.e., how accurately a test measures the skill it is designed to measure) in the evaluation process. In any

Tools of Evaluation, Special Issue 2015 23

concern for fairness, work samples are of particular value as they have both higher face validity and greater fairness for all participants (Lievens & Klimosky, 2001). Further, there exists a limited comparison between the environment in industry and the work sample setting. This comparison could be due to rapid changes in technology that may make work samples inapplicable or irrelevant based on the current workforce (Enders, 2002). Finally, work sampling is based on a quantitative measurement. It can offer qualitative insights, like the quality of work,  individual’s  strengths  and   weaknesses, or distinguishing between aptitude and achievement, therefore making inferences possible concerning the overall total experience of the individual throughout the evaluation process. Section Conclusions

Traditionally, work samples have been used when traditional paper-and-pencil tests or psychometric assessments were not viable options in the evaluation process. Because of the cost and time attributes, work sampling and other related methods have been used mainly when evaluating physical tasks. More recent research (e.g., Hagner, 2010; Peer & Tenhula, 2010; Thirtieth Institute on Rehabilitation Issues [30th IRI], 2003) looks at the possibility to encompass, also, the application of these methods to measure behavioral, cognitive, and affective aspects of work for individuals with disabilities. Assessing the qualitative aspects of processes, such as communication skills, problem solving, professional behavior, or ethics, often requires the examination of behavior from multiple cognitive and affective levels. This application of work sampling in assessing these aspects allows evaluators the ability to explore more broadly an individual's contribution to the evaluation outcome, while fully taking into

account the complexities of the multiple layers of disability.

Informal Assessments in the Vocational

Evaluation Process

Informal assessments are those which are less dependent on standardized development and structure (Power, 2011). Informal assessments include, but are not limited to, observations, interviewing sessions, self-report inventories, and even preference questionnaires. Establishing rapport between an evaluator and a consumer is a significant part of the overall evaluation process. Depending on how the evaluee feels about standardized testing, the evaluator may be perceived as intimidating, resulting in a rocky foundation for the entire process. Informal assessments are often used as techniques to establish rapport and create a relaxed, but effective environment.

An experience most people share is test anxiety, and this issue is prominent in the vocational evaluation process, as the individual may look at the process as another area in which to test, perform poorly, and be labeled (30th IRI, 2003). In some instances the vocational evaluation process has been viewed as lengthy. This process, for some, may seem time- consuming due to differing situations. These situations may include long waiting lists for evaluations, lack of availability of a qualified evaluator, lack of the presence of an evaluator in the area, or any number of other situations that cause this process to be time-consuming. No matter the elapsed timeframe, an effective and individualistic assessment approach can save the evaluator time,  as  well  as  change  the  person’s   perception of the entire process, therefore making the overall process well worth the wait.

Tools of Evaluation, Special Issue 2015 24

Fundamentals of Informal Vocational Assessments

Building rapport in the evaluation process is critical in order to understand fully the impact of the overall evaluation on the individual with disabilities. Though the evaluee may appear to be comfortable and have a well-rounded understanding of the services she or he is receiving (e.g., needs and preferences), the evaluee may often share in the context of the evaluation feelings and/or concerns that may not have been voiced during the initial intake interview. Often, evaluees are reluctant to voice concerns due to their personal bias regarding the process, or a concern over losing this service in the future. When considering these issues related to consumer hesitancy in the assessment process, utilizing an informal assessment approach may be one of the ways the professional can bring underlying issues to the surface in order for them to be addressed fully in the overall assessment process. Keeping in mind that there are multiple ways to assess a consumer using informal techniques, the use of multiple informal methods may provide the evaluator qualitative information regarding consumer preferences in order to understand more fully transferable skills and abilities, and to assist the individual in the selection of his or her overall vocational goals. Once these underlying issues or experiences emerge, the evaluator and the evaluee can address and use these together as they move forward in the overall evaluation process. Examples of Informal Assessments

There are numerous examples of informal assessments. These range from improvised tasks to alternative ways of demonstrating competency. The latter is often referenced as authentic assessment and

includes techniques such as graphic organizers, performance products, and live performances. (For more information and discussion of these tools see Scott, 2000.) Following are two tools used in evaluation: one is interviewing (see article by Austin et al., in this Special Issue) and the other is an example of an emerging technique.

Interviews. The initial interview is a foundational  area  of  an  evaluator’s  training,   and becomes the cornerstone of the assessment process (Power, 2013). Intake interviews are the first step in determining a starting point from which to clarify known information, discover new information, and begin the overall process of acknowledging individual preference. By presenting the individual with the opportunity to voice his or her concerns and preferences, the evaluator demonstrates to the individual that the process facilitates empowerment to make informed decisions regarding future vocational goals. Further, this type of assessment allows the evaluator to obtain descriptive  data  on  the  individual’s   emotional, behavioral, and social issues, thus providing further insight into the individual’s  motivations  regarding   rehabilitation planning and ultimate path in life.

SWOT analysis. Another area that could be used to support individuals to resolve their concerns are work trials, also referred to as the Strengths, Weaknesses, Opportunities, and Threats,  or  “SWOT”   analysis (see http://www.mindtools.com/pages/article/ne wTMC_05_1.htm for an example). Strengths (S) and Weaknesses (W) are considered to be internal factors over which individuals have some measure of control. Further, Opportunities (O) and Threats (T) are considered to be external factors over which an individual may have little or no control. Using this method could build on the consumer-evaluator/counselor

Tools of Evaluation, Special Issue 2015 25

relationship by providing one-on-one attention to the individual, which actively demonstrates evaluator involvement in working towards a successful vocational outcome. A SWOT analysis can be an important informal tool for understanding and analyzing the thoughts and environment of the individual. Its key purpose is to identify the strategies that will create an understanding of the resources, and capabilities,  in  the  consumer’s  personal   environment. Therefore, SWOT serves as a foundation for evaluating the internal potential and limitations, and the probable/likely opportunities and threats from the external environment. It provides a context and view of positive and negative factors inside and outside of the individual that could affect successful vocation outcomes. An understanding of the individual’s  environment  assists  the   evaluator in forecasting and predicting trends, and demonstrates the overarching need to include the evaluee at every level in the vocational evaluation and decision- making process. Informal Vocational Assessment Advantages

When informal assessments are tailored  to  a  consumer’s  functional  needs, strengths, and preferences, the ultimate goals of vocational evaluation may be enhanced. Vocational assessment has often depended principally on aptitude testing, job analysis, work sample techniques, and systematic use of behavioral observations (30th IRI, 2003).

Informal assessments can assist in establishing evaluator-consumer rapport, but what makes this possible are the true advantages of informal assessments (e.g., lower consumer test anxiety, more relaxed administration). An evaluator must continue to seek and find informal methods of

assessment that will be effective in evaluating all types of individuals with disabilities. With the use of informal methods, the evaluator will be able to obtain needed information, and can demonstrate to the consumer that this evaluation process can be flexible but structured at the same time (30th IRI, 2003). Using this structure a consumer has the opportunity to disclose information and promote openness in the overall evaluation process.

Informal Vocational Assessment Limitations

Disadvantages of informal assessments center on two concerns. The first concern emerges in the form of non- standardized assessment forms and techniques, which in turn allow information to be subjectively interpreted. Standards are put in place to ensure structured rules and boundaries for administering, scoring, and interpreting of assessments. Assessments deemed informal are not viewed as having consistent measures that can be validated. Therefore, concerns emerge surrounding the utility of the results. The second area surrounds issues of the interpretation of the assessment. Informal assessments are seen as being subjective, and are often criticized because results may be erroneously identified  as  “factual”  data.    Subjective information or writing is based on personal opinions, interpretations, points of view, emotions, and judgment. As noted by Power (2013), the interpretation of the assessment information  as  the  evaluator’s  personal   reaction  to  the  consumer’s  evaluation   experience opens the evaluation to criticism, and can make this type of assessment ill- suited for the overall evaluation.

Section Conclusions

Tools of Evaluation, Special Issue 2015 26

By breaking from the norm of standardized assessments, evaluators have the opportunity to explore skills and abilities in a different way with evaluees who are trying to discover their transferable skills, especially if they are trying to return to the world of work. The evaluator must support the  consumer’s  informal  self-expression and encourage understanding that evaluation is not only about tests, but also personal experiences. By exploring informal assessments the richness of the individuals experience begins to emerge, and true understanding of skills and abilities from the consumer’s  perspective  may  emerge  in  the   process. By having a holistic view of the individual through the utilization of the informal process, the evaluator will have the capability to match skills and abilities more fully, thereby encouraging consumers to maximize their rehabilitation potential and attain their highest level of functioning.

Community-Based Vocational Assessment Methods

Vocational evaluation is

distinguished from other forms of evaluation by the presence of real or simulated work activities (Dowd, 1993). As support for inclusion of all groups, an emphasis on offering these activities within the community, rather than behind the walls of vocational rehabilitation facilities, is increasing. Community-based vocational assessment methods afford evaluators a view of consumers in real-world situations and rely on keen observations made during those situations. The quality of the documentation of these observations is critical, as it must later be interpreted and used for vocational planning. Community- based methods may include the observation and recording of actual or simulated work activities, as with situational assessments, on-the-job evaluations, or job try-outs. Also,

they may include activities of daily living that can be related to work, as is found with the process of Discovery (Callahan, n.d.a), discussed later in this article. Common across all community-based methods is the element of individualization. The key in this process of individualization is considering the consumer's unique needs, as well as the systematic approach to gathering and documenting the objective observations, and interpreting their relevance to future work activities.

Basics of Observation and Recording

Community-based activities during vocational evaluation allow the consumer to be assessed authentically during tasks relevant to work in settings that most closely resemble real life (Nerlich, 2012). Observation and documentation of a consumer’s  vocational  behaviors  and   performance should be systematic and deliberate. The process should start with careful consideration of the vocational evaluation referral questions and information already gathered regarding the consumer (Pruitt, 1986). This consideration will affect the type of environments and tasks that are chosen for assessment. For example, a referral inquiring about the appropriateness of a skilled occupation for an individual with demonstrated average range academic skills would require a different approach than one seeking recommendations regarding supported employment options for an individual with complex layers of disability. Thorough understanding of the individual consumer, familiarity with potential community-based assessment sites, knowledge of the tasks available, and awareness of critical vocational factors for a wide variety of occupations lays the groundwork for planning meaningful, work-based observations in the community (Power,

Tools of Evaluation, Special Issue 2015 27

2013). For those options that may not be overtly vocational, activities should be carefully chosen so that they are relevant to the goals of the overall assessment process and the individual.

Recording Tool

Once planned and executed, the use of a consistent recording tool is key to the success of community-based vocational assessment methods. An effective evaluator will keep referral questions in mind while observing work behaviors and performances that are exceptional, in or out of keeping with expectations, or significant to providing answers in any way (Pruitt, 1986). Task observation forms for each occurrence of observed activity should be completed using descriptive language, while avoiding opinionated and judgmental adjectives, or accounts of what the consumer did not do (Corthell & Lesnick, 1974). These forms should include information about the time and date, environmental factors, accommodations or modifications provided, and any relevant behaviors observed. Keeping recordings objective and factual allows the reader to synthesize accurately what occurred as a part of the vocational evaluation in its entirety (Pruitt, 1986). Today’s  evaluators  have  an  advantage  in  the   documentation process over their predecessors. The use of tablets and portable technology makes it possible to record observations on-the-go in an electronic format that can easily be added to reports when the evaluator returns to the office. The use of multimedia devices also allows vocational evaluators to capture footage of the tasks performed by consumers for a closer review at a later time, rather than having to rely solely on notes and memory.

Community-Based Assessment Options

Most often, community-based assessment options include activities that are the same or similar to an actual employment situation. Examples include situational assessments, on-the-job evaluations, and job try-outs. These methods often include the identification of comparative standards of performance  by  which  a  consumer’s   performance and behaviors are evaluated. Tasks typical to the vocational area being explored are presented along with instructions, expectations, and feedback. Other methods of community-based assessment practices include activities that are not overtly vocational, but are intended to lead to a more intimate view of what a person can do based on his or her ability to perform tasks in everyday situations.

Situational assessments. Situational assessment is a method that is useful for exposing inexperienced consumers to aspects of the world of work. Prior to job development, situational assessment can provide  information  about  the  consumer’s   preferences for work and interpersonal skills for community employment (Condon, Enein-Donovan, Gilmore, & Jordan, 2004). These short-term opportunities provide a naturalistic environment in which consumers can perform real work around actual coworkers, with feedback and supports. A variety of supports and accommodations, in a range of intensities and durations, can be tried, adjusted, and recorded according to the  consumer’s  response  to  them.  This   method of assessment might involve a sampling of tasks across multiple community-based settings within a short span of time to give consumers reference points for more than one field or vocation. It can help consumers solidify employment preferences and interests, build confidence in abilities, as well as expose them to potential employment options (Power, 2013).

Tools of Evaluation, Special Issue 2015 28

On-the-job evaluation. As an adaptation of situational assessment, on-the- job evaluation (OJE) begins with a particular job in mind and gives the consumer the opportunity to experience the full demands of the job to determine whether or not the job is a fit. Concentrated on a specific job, rather than a general occupation, the OJE provides feedback from an actual on-site supervisor (Hagner, 2010). While an evaluator’s  knowledge  of  a  specific  position   usually comes from research and second- hand information, on-the-job evaluation offers the chance to receive feedback from a person who is a subject matter expert and regularly assesses performance of employees in the same position. On-the-job evaluations are comparative in nature, weighing  the  person’s  aptitudes,  abilities,   skills, and physical condition against the requirements of the job in question. At the end of this experience, predictions and recommendations are expected to be made about  a  person’s  potential  for  success  in  the   given position. This method carries the concern that often consumer evaluations are done by job incumbents with little knowledge of evaluation standards who can insert their own bias in their assessment of the evaluee.

Job try-outs. As another related method, job try-outs are usually longer evaluations that may extend from a period of weeks to months. When vocational evaluation is seen not only as a stand-alone service, but is utilized as part of the continuum of the vocational rehabilitation process, a job try-out can be arranged. The major difference between a job try-out and other forms of work-based assessment is the possibility of an actual offer of employment being made if the experience goes well. For a job try-out, there are likely to be more check-ins to gather information about a consumer’s  performance  from  a  supervisor,   and less one-on-one interaction between the

evaluator and consumer onsite. There should be a defined start and end date, complete with a deadline for when a hiring decision can be expected (Jordan, 2008). The emphasis is placed on giving the consumer and the site supervisor as accurate a picture as possible of what it would be like if the consumer were actually employed at the site.

Discovery. While work-based assessments within a community setting are widely accepted as an important tool for understanding work preferences, behaviors, and performance, other methods that are gaining popularity and acceptance stretch the use of observation to encompass life activities that may later be translated into work activities. One such method is Discovery, which has roots in the Employment for All movement. In his white paper Discovery  Is…, Callahan (n.d.a) defines  it  as  “a  process  that  involves  getting   to  know  people”  and  “spending  time  with   people, instead of testing or evaluating them, as  a  means  of  finding  out  who  they  are.”   That process excludes comparison as a rule, relying instead on the planning of and participation in various community and life activities with which the consumer is both familiar and unfamiliar to provide information for carefully documented behavioral observations. As with any observational method, the recording of accurate Discovery-based observations is paramount to its usefulness. Descriptive and objective language, limited use of adjectives, and the systematic recording of short bursts of significant behaviors should be recorded on activity logs. Callahan (n.d.b) even encourages the use of photography to capture imagery of the consumer during observations to enhance documentation; these observations are then translated into the contributions, conditions, and interests that a person demonstrates toward employment.

Tools of Evaluation, Special Issue 2015 29

Rather than prescribing Discovery for a specific disability type, Callahan (2007) champions it for those with the most significant impact of disability as a means toward employment for all. The method has been used for those consumers who have not traditionally been successful in standardized testing, who have attempted employment several times and failed, or who need significant assistance to secure employment, including representation during job seeking and development. The information gathered can be used to answer referral questions about the appropriate type of employment for consumers, supports that must be in place for success, and consumer strengths. This observational lens has also been used to gain functional answers to questions of appropriateness for admission to residential training  given  a  person’s  present  strengths   and needs, and to help vocational rehabilitation counselors better understand their consumers before developing residential service plans. As community inclusion continues to gain popularity and the terms job readiness and employability are slowly replaced with an assumption of a fully integrated work world, Discovery becomes more relevant to the field of vocational assessment.

Community-Based Assessment Advantages

Community-based assessment methods, while varying in specifics of technique, duration, and use, have advantages and disadvantages. These methods are based on observing the actual function of consumers in naturalistic settings, adding to the face validity of the evaluation process. In other words, the information gathered would appear to be directly applicable to determining how a person functions in work and life activities in the community. Because they can

generally be utilized by professionals with a good understanding of objective observation and  writing  techniques,  a  master’s  prepared   evaluator has the skills to perform these observations with required corresponding documentation. The cost savings, per se, is that expensive tests are not required. Anecdotally, professionals have noted better rapport with consumers who receive more observation-based assessment, as it is conducive to taking more time with the consumer without the traditional stressors of test taking. It allows a show-me approach to assessment that consumers and families tend to appreciate and consequently are more motivated to participate in (Wiggins, 1993). Rather  than  relying  on  a  single  day’s  testing   data, observational techniques can last days, or longer. This gives the evaluator information over a prolonged period of time, allowing the consumer to be seen during a wider variety of circumstances and experiences.

There are particular advantages for specific populations. Community-based assessment has been shown to be an important part of vocational evaluation for consumers with traumatic brain injury who may continue to perform well on standardized testing after an injury, but struggle functionally to put all the pieces together during real-life activities. Research has shown that these individuals feel the opportunity to try out work is paramount to the return-to-work process (Stergiou-Kita, Rappolt, & Dawson, 2012). Transition-aged high school students with disabilities benefit from situational assessments to try out different jobs before making decisions about their future and to identify strengths and areas of performance for improvement before entering the workforce (Trainor, Smith, & Kim, 2012). People seeking integrated, supported employment are well- served by work evaluations that focus on consumer work behaviors and adaptation to

Tools of Evaluation, Special Issue 2015 30

feedback, the social aspects of the job that will need to be considered in employment, and development of natural supports in actual work situations (Power, 2013). Individuals who experience communication and interpersonal needs associated with autism spectrum disorders find it beneficial to participate in job try-outs that give them a chance to show what they can do in jobs where the traditional hiring process may have excluded them (Jordan, 2008). Moreover, with the addition of customized employment as a defined best practice in the Workforce Innovation and Opportunities Act (WIOA), and an increased emphasis on services for high school transition age students, Discovery as a pathway to customized employment and situational assessments during high school becomes even more relevant to meeting the needs of vocational rehabilitation today (U.S. Department of Education, 2014).

Community-Based Assessment Limitations

Objectivity in this method is essential. Purposefully looking for relationships among apparently isolated behavioral events, considering the impact of the  environment  on  the  consumer’s   behaviors, and being able to recognize and compensate for influences of bias and personal values are all integral to being a successful observer (Pruitt, 1986). Observational evaluation is often a qualitative technique that can be easily affected by service provider opinion, bias, and misunderstanding about the purpose of the activities. So, while face validity may seem high, the usability of the documentation can easily be compromised by a poor observer. And for job try-outs or on-the-job evaluations where site supervisors make most of the judgment calls, subjectivity or rater bias can be

problematic (Hagner, 2010). Information can be interpreted incorrectly, especially when the interpretation fails to take into account that behaviors are the product of the person within a specific environment (Gilbert & Malone, 1995). As Pruitt (1986) noted,  “observational  sensitivity, as any other skill, is developed and improved through  practice  (p.  31).”  

Also, the evaluator must understand that there are limits on the kinds of sites that can be arranged for community-based assessments. In particular, simulating jobs with higher levels of academic, technical skill, and cognitive requirements for situational evaluations are difficult. Provision of this method of assessment is also highly dependent on the availability of staff to perform observations, as it requires significant one-on-one attention (Hall, 2009). While at first glance, community- based methods may seem to reduce costs associated with expensive psychometric testing supplies, the hourly rates of evaluators can add up with prolonged observation. In general, time-consuming community-based assessment sites are more difficult to set up (Hagner, 2010; Power, 2013). Nerlich (2012) offered some guidance for best practices in securing community-based assessment sites. The time investment is substantial, ranging from several hours to several weeks, and the method lacks the relative comfort and convenience of in-office tests and measurements. Evaluators must also be sure the placement is in compliance with prevailing wage and hour regulations. However, in the end, making observations and allowing consumers to actually experience work and life activities may prove to be the most effective and useful. Section Conclusions

Tools of Evaluation, Special Issue 2015 31

Where does community-based assessment fit within the framework of individualized vocational evaluation? Observation of actual tasks related to employment confirms or changes the hypotheses formed during individualized vocational evaluation planning and testing. Using descriptive documentation, it puts a face to the consumer and breathes life into the process with action rather than scores. Observation can confirm consumer reports of work skills, preferences, and familiarity in specific work environments mentioned during an interview. Observational assessments can be used as a complement to psychometrics and work sampling or as a stand-alone service to answer questions of function related to specific employment interests, independent living, and transferrable skills. Paper-and-pencil measurements can only be enhanced by the addition of observational assessment. However, in individualized vocational evaluation there exists a paradigm where the parts are often best utilized in harmony, so preliminary testing can be helpful in determining the appropriateness of observational opportunities before their deployment (Hagner, 2010). Individualization of the vocational evaluation plan means that specific and appropriate observational assessments should be readily introduced to complement other methods of assessment. Certainly professionals who do not observe consumers in life or work activities risk missing the opportunity to experience the essence of who they really are outside the quiet and control of a testing room.

Final Thoughts

Work is one of the most central components  in  an  individual’s  life.  It  has   been confirmed by a number of authors that work transcends culture, socio-economics,

ethnicity, and gender (Szymanski & Parker, 2010). Work not only has the ability to provide safety and security, but it enhances intrinsic qualities, such as self-efficiency and self-worth. Given these factors, when an individual is unable to work, the impact can have a negative and often devastating effect on  a  person’s  physical  health,  mental  health,   and overall well-being. Vocational assessment and evaluation are often necessary in the vocational rehabilitation process to assist consumers in understanding their valued skills, aptitudes, and abilities when considering vocational outcomes. The vocational evaluation process is a multifactorial, individualized assessment to meet the specific needs of the unique consumer. While a variety of vocational evaluation methods exists, each method and its related assessment tools is selected based on the specific needs and tailored to answer the referral questions for the identified consumer. The appropriateness of the methods used in each unique evaluation will depend on the functional strengths and limitations, past experiences, educational background, and specific needs of the consumer.

Vocational evaluation has continued to evolve from its beginning in the vocational rehabilitation movement of the 1950s and 1960s (Pruitt, 1986). From the first graduating class of Vocational Evaluation graduate students at UW–Stout in 1968 and Auburn University in 1970 to the recent creation of the Registry of Professional Vocational Evaluators (pveregistry.org), and more currently the inclusion of vocational evaluation as a major standard for rehabilitation counselor education (Council on Rehabilitation Education [CORE], 2012), rehabilitation professionals have tried to answer consumer and  referring  counselors’  questions  in  the   most comprehensive and accurate ways possible. Methods have taken many forms

Tools of Evaluation, Special Issue 2015 32

and have changed over time as best practices have emerged, and new methods continue to be developed in order to meet better the needs of consumers today. An overall shift toward real-world methods and a focus on function rather than diagnosis has driven a trend in consumer-centered observational techniques and the use of situational elements in community-based settings (30th IRI, 2003). Further, focusing on issues related to significant disabilities has provided a change in thinking from job- readiness to employment for all, and have necessitated that evaluators become more adept at identifying transferable skills and abilities. Technology has made it possible to evaluate these aspects more efficiently, but increased understanding of the impact of the methods used when providing vocational evaluation and assessment is key to ensuring the most comprehensive assessment outcomes.

While it is easy to become attached to a method of assessment and comfortable in its administration, an evaluator must hold to the value of individualization in evaluation (30th IRI, 2003). Each consumer is different and presents with individualized nuances, background, and specifics of functioning. Where one consumer may be served well with a work evaluation or work sampling only, another may require psychometric testing to discern an academic functioning problem and to predict success in post-secondary education. The key is to understand how informal and formal assessments work in tandem, not in competition with one another, to create a holistic picture of the individual. The assessment process should be unique to the consumer and pull from sound practices that will make it effective, efficient, and accurate.  Like  vocational  rehabilitation’s   goal of employment for all, vocational evaluation should be an inclusive process.

References Anthony W., & Jansen, M. (1984).

Predicting the vocational capacity of the chronically mentally ill. American Psychologist, 39(5), 537– 544.

Brown, C., McDaniel, R., & King, W. (1995). The role of vocational evaluation in the rehabilitation process. Vocational Evaluation and Work Adjustment Bulletin, 28(4), 91– 98.

Callahan,  M.  (n.d.a).  Discovery  Is….   Retrieved from http://marcgold.com/Publications/W hite%20Papers/Discovery%20is.pdf

Callahan, M. (n.d.b). Documenting discovery: Developing a descriptive profile to guide employment plans. Retrieved from http://marcgold.com/Publications/ White%20Papers/Profiles% 20Documenting%20Discovery%20A rticle.pdf

Callahan, M. (2007, January/February). Embracing a working life for all. TASH Connections, 16–18.

Campbell, W. J., & Fox, H. R. (2002). Testing individuals with disabilities in the employment context: An overview of issues and practices. In R. B. Ekstrom, & D. K. Smith (Eds.), Assessing individuals with disabilities in educational, employment, and counseling settings (pp. 193–204). Washington, DC: American Psychological Association.

Choubon, R. A., Stewart, W. W., & McGrew, G. W. (1991). The implications of technology for vocational assessment. Rehabilitation Education, 5, 253– 259.

Tools of Evaluation, Special Issue 2015 33

Condon, C., Gandolfo, C., Brugnaro, L., Thomas, C., & Donnelly, P. (2004). Getting the most from the public vocational rehabilitation system. Institute for Community Inclusion, Tools for Inclusion19. Retrieved from https://www.communityinclusion.org /article.php?article_id=129

Condon, C., Enein-Donovan, L., Gilmore, M., & Jordan, M. (2004). When existing  jobs  don’t  fit:  A  guide  to  job   creation. The Institute Brief, Issue 17. Retrieved from http://www.communityinclusion.org/ article.php?article_id=126

Corthell, D. W., & Griswold, P. P. (1987). The use of vocational evaluation in VR. Menomonie, WI: Stout Vocational Rehabilitation Institute.

Corthell, D. W., & Lesnick, M. J. (1974, September). Observation and recording behavior. Vocational Evaluation and Work Adjustment Bulletin. 7(3), 21–27. Retrieved from http://library.ncrtm.org/pdf/J050.007 1.01C.pdf

Council on Rehabilitation Education. (2012). Standards. Retrieved on September 29, 2014 from http://www.core- rehab.org/COREStandards

Dowd, L. (1993). VEWAA Glossary of terminology for vocational assessment, evaluation, and work adjustment. Menomonie, WI: University of Wisconsin–Stout, Materials Development Center.

Enders, A. (2002). Technology and the employment of people with disabilities. Unpublished manuscript.

Flansburg, J. D. (2011). Vocational evaluation: A primer. Journal of Employment Counseling, 48, 114– 120.

Frey, N. D. (1984). Functional assessment in the  80’s:  A  conceptual  enigma,  a   technical challenge. In A. Halpern, & M. Fuhrer (Eds.), Functional assessment in rehabilitation (pp. 11– 44). Baltimore, MD: Brookes.

Geisinger, K. F., Boodoo, G., & Noble, J. P. (2002). The psychometrics of testing individuals with disabilities. In R. B. Ekstrom, & D. K. Smith (Eds.), Assessing individuals with disabilities in educational, employment, and counseling settings (pp. 33–42). Washington, DC: American Psychological Association.

Gilbert, D. T., & Malone, P. S. (1995). The correspondence bias. Psychological Bulletin, 117, 21–38.

Hagner, D. (2010). The role of naturalistic assessment in vocational rehabilitation. Journal of Rehabilitation, 76(1), 28–34.

Hall, A. C. (2009). Mandatory situational assessments in Tennessee. Retrieved on December 5, 2014 from https://www.communityinclusion.org /article.php?article_id=289

Havranek, J., Field, T., & Grimes, J. E. (2005). Vocational assessment: Evaluating employment potential (4th ed.). Athens, GA: Elliott & Fitzpatrick, Inc.

Jordan, M. (2008). Supporting individuals with autism spectrum disorders: Quality employment practices. The Institute Brief, Issue 25. Retrieved from https://www.communityinclusion.org /article.php?article_id=266

Leconte. P. J. (1994). Vocational appraisal services: Evolution from multidisciplinary origins and applications to interdisciplinary practices. Vocational Evaluation and

Tools of Evaluation, Special Issue 2015 34

Work Adjustment Association Bulletin, 27(4), 119–126.

Lievens, F., & Klimoski, R. J. (2001). Understanding the assessment center process: Where are we now? In C. L. Cooper, & I. T. Robertson (Eds.), International review of industrial and organizational psychology (vol. 16) (pp. 245–286). Chicester: John Wiley & Sons, Ltd.

Lightner J., Jr. (1994).Vocational evaluation and assessment in public sector rehabilitation: Trends, changes, and an interdisciplinary approach. Vocational Evaluation and Work Adjustment Association Bulletin, 27(4), 133–135.

Nadolsky, J. M. (1981). The use of work samples in the vocational evaluation process. Diagnostique, 6(2), 28–39.

Nerlich, A. P. (2012). Securing the site: Locating and negotiating community-based assessments. Vocational Evaluation and Work Adjustment Association Journal, 39, 23–30.

Peer, J., & Tenhula, W. (2010). Assessment of vocational functioning in serious mental illness: A review of situational assessment and performance based measures. Journal of Vocational Rehabilitation, 32, 175–189.

Power, P. W. (2011). A guide to career management and programming for adults with disabilities: A 21st Century perspective. Austin, TX: PRO-Ed.

Power, P. W. (2013). A guide to vocational assessment (5th ed.). Austin, TX: PRO-ED.

Pruitt, W. (1986). Vocational evaluation. Menomonie, WI: Walt Pruitt Associates.

Robinson, R. H., & Drew, J. L. (2014). Psychometric assessment in forensic

vocational rehabilitation. In R. Robinson (Ed.), Foundations of forensic vocational rehabilitation (pp.87–131). New York, NY: Springer Publishing Co.

Scott, J. (2000). Authentic assessment tools. In R. L. Custer Ed.) Using authentic assessment in vocational education (Information Series 138; pp. 33–48). Columbus, OH: ERIC Clearinghouse on Adult, Career, and Vocational Education, The Ohio State University.

Searle, R. (2003). Selection and recruitment: A critical text. London: Palgrove Macmillan and Milton Keynes: The Open University.

Stergiou-Kita, M., Rappolt, S., & Dawson, D. (2012). Towards developing a guideline for vocational evaluation following traumatic brain injury: The qualitative  analysis  of  clients’   perspectives. Disability & Rehabilitation, 34(3), 179–188.

Szymanski, E. M., & Parker, R. M. (2010). Work and disability: Issues and strategies in career development and job placement. (3rd ed.). Austin, TX: PRO-ED

Thirtieth Institute on Rehabilitation Issues. (2003). A new paradigm for vocational evaluations: Empowering the VR consumer through vocational information (30th Institute on Rehabilitation Issues). Washington, DC: Rehabilitation Services Administration, US Department of Education. Retrieved from http://www.iriforum.org/download/I RI30.pdf

Thomas, S. W. (1994). Vocational evaluation: Dynamics, politics, and creative application. Vocational Evaluation and Work Adjustment Association Bulletin, 27(4), 155–119.

Tools of Evaluation, Special Issue 2015 35

Trainor, A. A., Smith, S. A., & Kim, S. (2012). Four supportive pillars in career exploration and development for adolescents with LD and EBD. Intervention in School and Clinic, 48(1), 15–21. doi:10.1177/1053451212443129

U.S. Department of Education. (2014, July 22). The Workforce Innovation and Opportunity Act overview of Title IV: Amendments to the Rehabilitation Act of 1973. Retrieved from http://www2.ed.gov/about/offices/list

/osers/rsa/publications/wioa- changes-to-rehab-act.pdf

Warschausky, S., Van Tubbergen, M., Asbell, S., Kaufman, J., Ayyangar, R., & Donders, J. (2011). Modified test administration using assistive technology: Preliminary psychometric findings. Assessment, 19(4), 472–479.

Wiggins, G. P. (1993). Assessing student performance. San Francisco: Jossey- Bass Publishers.

Author Note Lee Ann R. Rawlins-Alderman, Ph.D., CRC, CFLE, is an Assistant Professor and Program Coordinator in the Department of Special Education, Rehabilitation and Counseling at Auburn University. Rawlins-Alderman has professional experiences in the areas of adult mental health, family support, geriatrics, rehabilitation, and disability. She has conducted national and international research in the areas of special education, transition, disability, and rehabilitation. She serves in leadership roles within many organizations at the state, regional, and national levels. Robin E. Dock, Ph.D., CRC, LPC, is an Associate Professor in the Department of Rehabilitation Counseling of the School of Health Sciences at Winston-Salem State University. Dock publishes in peer-reviewed journals and book chapters and presents at local, regional, national, and international conferences on topics of deafness, accommodations for adult-workers with disabilities, college students with disabilities, and vocational evaluation with a transition population. Megon Steele, M.S., C.R.C., P.V.E., is the Manager of Vocational Assessment Services at Roosevelt Warm Springs Institute for Rehabilitation (RWS). Steele has worked to develop a unique hybrid approach to vocational assessment at RWS that combines traditional evaluation techniques with the tenets of Discovery to serve the entire Georgia Vocational Rehabilitation Agency. Leslie Wofford is a master’s  degree  student  in  the  area  of  Rehabilitation  Counselor  Education at Auburn University. Wofford is furthering her career in order to broaden her role in the field of rehabilitation. She has succeeded academically, which is reflected with her membership in Delta Epsilon Iota Academic Honor Society.

Return to Table of Contents

Tools of Evaluation, Special Issue 2015 36

Tools of Evaluation, Special Issue 2015 37

Utilization of Interviewing as an Assessment Tool to

Enhance Vocational Rehabilitation Service Delivery:

Fostering  the  Therapeutic  Alliance  and  Professionals’  Judgment  Accuracy  

Bryan S. Austin University of Idaho–Boise

Carl W. Sabo

Wright State University

Amanda K. McCarthy Northern Illinois University

Matthew E. Sprong

Montana State University–Billings

Lauren N. Noble University of Idaho–Boise

Abstract An interview is an essential component of the vocational rehabilitation (VR) process (intake, eligibility, assessment, plan development, employment and follow-up, and case closure). Interviewing is primary to establishing professional-consumer rapport where professional judgments are needed to elicit vital data from consumers to develop an accurate consumer profile, determine appropriate rehabilitation goals, and ensure consumers are given valid information to inform their decision-making. Therefore, the purpose of this article was to describe how interviewing can be utilized by VR professionals as an assessment tool throughout the VR process. Recommendations for incorporating interviewing techniques as a means to build strong professional-consumer therapeutic alliances and more accurate professional judgments are provided. Specifically, Motivational Interviewing (MI) strategies of rapid engagement, debiasing techniques, and clinical supervision are proposed. Keywords: Interview Assessment, Working Alliance, Motivational Interviewing, Counselor Bias, Clinical Supervision

Utilization of Interviewing as an

Assessment Tool to Enhance Vocational Rehabilitation Service Delivery:

Fostering the Therapeutic Alliance and Professionals’  Judgment  Accuracy

Tools of Evaluation, Special Issue 2015 38

Interviewing is most often utilized in vocational rehabilitation (VR) as a method for collecting relevant information from a consumer for VR service provision (Power, 2013). Vocational rehabilitation is a program that provides supportive and individualized services to assist consumers with disabilities in obtaining and maintaining employment (Parker & Patterson, 2012). These services are provided to identify occupations that are compatible  with  a  consumer’s  skills,   abilities, and interests. Among state-federal VR programs and related providers (e.g., community rehabilitation programs, One- Stop Centers, mental health agencies) that provide direct services to consumers they serve, the need for effective interviewing is well documented (Berven, 2004; Drummond & Jones, 2010; Power, 2006, 2013; Shahnasarian, 2004). Therefore, interviewing is an essential function performed by VR professionals (e.g., rehabilitation counselors, employment specialists, vocational evaluators, career assessment specialists) and is of particular importance to career assessment and vocational evaluation professionals.

The interviewing process is beneficial because it provides an avenue for establishing VR professional-consumer rapport and for collecting data to build an accurate  “working  picture”  or  vocational   profile of a consumer. For instance, the interviewing process can help VR professionals understand consumer concerns, goals, strengths, potential barriers, desired services, or even appropriateness for VR services, and be used to develop a realistic rehabilitation goal with a consumer. The VR research literature shows that strong VR professional-consumer therapeutic alliances (also referred to as working relationship, rapport, working alliance), often originally formed during the initial interview, are linked to positive

rehabilitation outcomes for consumers (Donnell, Lustig, & Strauser, 2004; Lustig, Strauser, Rice, & Rucker, 2002; Lustig, Strauser, & Weems, 2004; Strauser, Lustig, & Donnell, 2004). The therapeutic alliance can be defined as the agreement on goals, tasks and emotional bonds between the VR professional and consumer (Bordin, 1979). Goals of VR are established and agreed upon to address the purpose of why a consumer is seeking rehabilitation services. Tasks of therapeutic activities and/or responsibilities by both the VR professional and consumer are developed in a collaborative manner that is acted upon to achieve specified employment goals. Bonds describe the nature of the relationship to include differing levels of trust and attachment between the VR professional and the consumer. Consequently, the ability of VR professionals to establish such therapeutic alliances with their consumers from the onset of a VR professional- consumer initial interview is essential.

Additionally, VR professionals must work toward making accurate professional judgments during the interview to ensure they collect the essential consumer data necessary to form a valid consumer profile (Berven, 2008). Errors or bias in professional judgments can be detrimental for consumers and limit their opportunities, rehabilitative services provided, and outcomes (Rosenthal, 2004; Rosenthal & Berven, 1999). To aid in developing an accurate vocational profile, information obtained during the interview is often corroborated with information received from other sources (e.g., medical/psychological reports, standardized assessments, person- center planning; Hagner, 2010; Meyer et al., 2001).

Besides being a critical source of data to VR professionals, the interview also provides a method for consumers to learn about the VR process, such as how services

Tools of Evaluation, Special Issue 2015 39

will be provided, possible outcomes of services, consumer rights, and consumer and VR professional expectations (Carlisle & Neulicht, 2010). In short, an effective interview involves an exchange of information that helps the VR professional and consumer prepare for and make informed decisions over the course of the VR process. Utilizing specific techniques to form a trusting professional-consumer relationship (i.e., therapeutic alliance) and valid vocational profile during the interview process will not only enhance service delivery, but will also promote appropriate rehabilitation goals and positive outcomes for consumers with disabilities (Austin & Leahy, 2015; Lustig et al., 2002). Therefore, the purpose of this article is to provide VR professionals with a discussion on how interviewing can be applied as an assessment tool throughout the VR process (intake, eligibility, assessment, plan development, employment and follow-up, and case closure) with a focus on establishing strong therapeutic alliances and making accurate professional judgments while working with consumers. Specifically, the use of Motivational Interviewing (MI) strategies of rapid engagement, debiasing techniques, and clinical supervision will be presented.

Interviewing throughout VR Process

Intake

The primary focus of the intake

interview is for VR professionals to:  develop a relationship with a

consumer;  share relevant agency information

(e.g., informed consent process); and  gather information to develop a

working picture of a consumer and his or her needs to develop a

comprehensive and valid consumer vocational profile (Power, 2013).

Although interviewing skills (e.g., multicultural understanding, active listening, reflecting feeling, influencing) are essential during all phases of the VR process (Ivey, Ivey, Zalaquett, & Quirk, 2012), the intake interview provides a unique opportunity to utilize these skills to build a therapeutic alliance with a consumer and to assess a consumer’s readiness for change early in the process (Wagner & McMahon, 2004). Thus, a reasonable expectation is that the intake interview and resulting strength of the therapeutic alliance may affect all subsequent VR professional-consumer interactions. For example, the professional- consumer therapeutic alliance has been found to be a factor in the achievement of positive consumer outcomes across helping professions (Horvath & Symonds, 1991; Martin, Garske, & Davis, 2000). Assuming that a VR professional-consumer therapeutic alliance causes these beneficial outcomes, VR professionals should try to maximize the therapeutic alliance with consumers. This may be achieved by:

 exhibiting behaviors such as empathy, warmth, respect, genuineness, and acceptance during the intake interview (Lustig et al., 2002); and

 balancing reflective statements (e.g., “It  appears  that  this  has  been  difficult   for  you”)  with  instructional   statements  (e.g.,  “Let’s  review  job   interviewing  skills”).  

Authentic reflective statements must be utilized to help consumers perceive that the VR professional is genuinely interested in helping and respects the consumer as an individual (Safran & Muran, 1998). Notably, many state VR professionals work from a solution-focused counseling orientation where the emphasis is on addressing a consumer’s  current  needs  with  active  

Tools of Evaluation, Special Issue 2015 40

involvement from the VR professional (Trepper et al., 2010). Some may view a solution-focused orientation as discrepant with building the therapeutic alliance. However, despite a solution-focused framework, attempts at building a therapeutic alliance with consumers is achievable and of primary importance during the intake process.

Besides building rapport, interviewing skills can also be used to assess a  consumer’s  readiness  for  change  during the intake process. Formally or informally assessing  a  consumer’s  readiness  for  change   early in the VR process will allow for actions to be taken in order to retain or refer the consumer (Wagner & McMahon, 2004). For example, during the intake process a consumer who indicates she/he has taken action to identify employment prior to the first appointment (i.e., internal locus of control) with the VR professional will likely have a different level of readiness than a consumer who believes that the VR professional’s  job  is  to  identify  jobs  with   little assistance from the consumer (i.e., external locus of control). A consumer who has taken prior action may need job-seeking skills training (e.g., interviewing skills, résumé development). A consumer with the perception  that  it  is  the  VR  professional’s   duty to identify jobs may need information about his or her roles and responsibilities.

Careful attention and follow-up on consumers’  statements  can  help  VR   professionals design an individualized picture of a consumer’s  readiness  to  begin   the VR process along with initial interventions that can help consumers progress towards their rehabilitation goals. Finally, a skilled VR professional must attend to not only what consumers vocalize, but also to the nonverbal indicators (e.g., eye contact, vocal tone, facial expressions) during all in-person interactions. A nonverbal communication assessment

administered by a VR professional can provide various types of information regarding  consumers’  feelings  about  the  VR   process and/or questions/information being sought.  A  consumer’s  nonverbal   communication may confirm or contradict the thoughts/feelings being conveyed verbally by the consumer. Moreover, VR professionals must also be cognizant of and make accurate judgments about their own nonverbal behaviors and their effect on consumer responses—important to cultivating a supportive climate to encourage consumers to speak freely about their rehabilitation needs and goals (Ivey et al., 2012).

Eligibility

The eligibility requirements for VR services indicate that someone has a physical or mental impairment (i.e., functional  limitations)  that  limit  the  person’s   ability to find, secure, and maintain employment (Parker & Patterson, 2012). Functional limitations that a consumer experiences may include limitations in mobility, communication, interpersonal skills, self-care, self-direction, work performance, or work skills. Interviewing skills can be applied during the eligibility process as a way to obtain valid, reliable, and comprehensive information about the consumer’s  functional  limitations.   Professional judgment errors and oversights made by VR professionals during this phase of the process could result in a consumer not receiving services (LeBlanc & Smart, 2007) or, when order of selection is in place, being forced to wait for services.

To improve accuracy of VR professional judgments during the consumer eligibility determination process, the interviewing skills of questioning (i.e., open and closed questions) is essential. More specifically, VR professionals should ask

Tools of Evaluation, Special Issue 2015 41

consumers explicit questions that relate to their reported disabilities; and explore consumers’  perceived  disability-related barriers to employment for every eligibility criterion category (e.g., interpersonal skills, self-care, self-direction). For example, a VR professional may address the category of interpersonal skills by asking a consumer, “Can  you  tell  me  about  more  about  your   anxiety  as  it  relates  to  work?”  This  type  of   related questioning for each criterion category will likely develop a more comprehensive and valid consumer profile and better ensure appropriate diagnostic reports (i.e., medical/psychological) and referrals to specialists (e.g., physiatrist, audiologist) are sought and used for eligibility determination purposes.

Assessment

Once a consumer has been determined eligible for services, VR professionals work collaboratively with consumers and begin the vocational assessment process. Vocational assessment is important in assisting consumers to achieve employment and can improve the efficient use of financial resources that is oftentimes limited (Thirtieth Institute on Rehabilitation Issues [30th IRI], 2003). This phase requires interactive dialogue between the VR professional and consumer to mutually agree upon realistic and attainable rehabilitation goals. The VR professional- consumer therapeutic alliance sets the foundation of this assessment process and should be used to empower consumers and support them in realizing their vocational potential (i.e., strengths, interests, and aptitudes; Kosciulek, 2004; Power, 2006). This interview encounter further facilitates consumers’  self-awareness; knowledge about work, job opportunities and resources; and ultimately, career goals in line with their indicated interests, abilities, and specified

rehabilitative and accommodation needs (e.g., on-the-job supports, rehabilitation technology; Power, 2006).

Vocational rehabilitation professionals also need to be accurate and unbiased in their assessment appraisals, which is critical to predicting consumer potential and identifying suitable rehabilitation interventions, services, and goals (Strohmer & Leierer, 2000). Vocational rehabilitation professionals must accurately evaluate the consumer-related information obtained to develop a valid vocational profile of a consumer. This involves  evaluating  a  consumer’s   rehabilitation goal, job seeking skills, the likelihood of maintaining employment, and any environmental barriers (e.g., employer attitudes, job accommodation needs) that may exist. With regards to the rehabilitation goal, the VR professional (in collaboration with the consumer) must determine if there is enough information to decide if the consumer’s  employment  goal  is  suitable.  For   example, if a consumer has identified a specific job that he or she would like to obtain, the VR professional needs to determine if the consumer understands the necessary prerequisites (e.g., abilities and aptitudes) and physical and learning capacities needed to perform the job, and identify whether this job exists within the local economy. Likewise, does the consumer understand the demands of the job and what to expect, and is any training needed to help the consumer acquire the skills necessary to be effective in the job?

When creating a vocational profile, VR professionals need to evaluate the job seeking skills of their consumers. As mentioned previously, observing the physical appearance of the consumer during the interview will allow a VR professional to evaluate whether job-seeking skills training is needed. For instance, a VR professional may have concluded that a

Tools of Evaluation, Special Issue 2015 42

consumer was not enthusiastic, well- groomed, and not interpersonally responsive during the interview. When interviewing for employment, these factors are vital and may jeopardize  a  consumer’s  employability. Consequently, a VR professional needs to make an accurate judgment as to whether or not  a  consumer’s  job  seeking  skills  are   satisfactory or determine if more information is needed to make that decision.

Once a determination has been made that a consumer is qualified for work and has adequate job-seeking skills to obtain a specified job position, a VR professional must determine if enough information is available to assess whether a consumer will be able to maintain employment. Finally, the VR professional must evaluate if any environmental barriers exist that may impact the rehabilitation goal of a consumer. For example, a VR professional may internally ask  the  question,  “Are  there  any   health/physical restrictions that would preclude certain types of employment for this consumer (e.g., unable to work in environments with frequent exposure to pulmonary irritants, transportation barriers that  may  lead  to  tardiness  or  absenteeism)?”   After VR professionals evaluate all relevant barriers that may affect  a  consumer’s   rehabilitation goal, rehabilitation services or interventions are targeted to eliminate or reduce such barriers.

Plan Development

Following completion of the vocational assessment process, the VR professional and consumer work together to develop the Individualized Plan for Employment (IPE). It is important for the IPE to be simple (understandable) and realistic (attainable) for consumers. Oftentimes the VR administrative requirements in completing an IPE make it confusing and hard for consumers to follow.

Hence, the written IPE document should be developed collaboratively with consumers and  include  consumers’  language  to  increase   their ownership of the IPE (having created it), clarity (about what to do), and therefore, follow through (actually doing what is specified in the plan). Consequently, during the interview process, the VR professional can gain benefit by:

 structuring time to meet and develop the IPE with a consumer during a face-to-face interview meeting;

 serving as facilitators or guides and asking questions that are centrally connected to agreed-upon employment goals and the rehabilitation services required to overcome disability and other related barriers to achieving such goals; and

 using  consumers’  language  that  can   be typed directly into the IPE module of the computer-based case management system when listing the objectives and rehabilitative services that address the disability- related barriers (e.g., supported employment, rehabilitation technology) and other related barriers (e.g., benefits planning, college or university training) considered necessary for a consumer to successfully achieve his or her rehabilitation goal.

Employment and Follow-up

When a consumer is ready for employment, VR professionals should continue to utilize the interview as an assessment tool. A face-to-face interview at this phase of the VR process can better inform  VR  professionals’  judgments  related   to  a  consumer’s  state  of mind and readiness to work. Accordingly, VR professionals may best accomplish this task during the VR

Tools of Evaluation, Special Issue 2015 43

professional-consumer interaction (interview) by:

 gauging consumer emotional readiness for employment;

 evaluating and supporting consumer readiness prior to employment, at the time it is obtained, and once a consumer starts employment; and

 focusing  on  a  consumer’s  thoughts,   fears, and expectations as he or she enters into employment.

To illustrate, the VR professional may ascertain how well any purchased adaptive equipment/software is working for the consumer and employer. Centered on the nature of answers provided by a consumer and information discovered from the interview, both VR professional and consumer can determine if any additional rehabilitation services or interventions are necessary. Case Closure

At the time of VR case closure, VR professionals should have what could be described as an exit interview with their consumers. More precisely, VR professionals should draw out consumers’ information regarding their:

 overall job satisfaction;  comfort level of performing

essential functions of the job;  interactions with

supervisors/coworkers/the public;  integration into the workplace

culture;  tardiness or absenteeism issues;  child care concerns (if applicable);

and  related disability management

problems. Vocational rehabilitation professionals can integrate and synthesize this information to determine the appropriateness of closing a consumer’s  case.  If  case  closure  is  not  

deemed appropriate, then a VR professional, in collaboration with the consumer, can determine what additional rehabilitation services are necessary to achieve a level of job stability that is agreeable to both the professional and consumer. Once a consumer is at a place in his or her employment that is deemed appropriate for closure, then the VR professional can outline case closure processes (e.g., post- employment services) and answer any remaining questions from the consumer prior to closure. For example, a VR professional may outline steps a consumer may need to follow to re-engage with the state VR agency or to request additional rehabilitation services if employment is terminated and/or disability-related issues arise. Again, it is of utmost importance, whenever possible, that the closure/exit interview between VR professionals and consumers take place face-to-face. This allows for full engagement between VR professionals and consumers at a potentially stressful time for consumers, thus enhancing the  VR  professional’s  ability  to  evaluate  the   consumer’s  state  of  mind  and  readiness  for   case closure.

Recommendations for Interviewing Techniques

Given that most encounters with

consumers are limited in both time and frequency, an extraordinary amount of interpersonal skillfulness by VR professionals is necessary. Large caseload sizes and comprehensive rehabilitative service needs for consumers with significant disabilities are at odds with the administrative pressures VR professionals face to move consumers through the VR process (Kierpiec, Phillips, & Koscuik, 2010). By establishing a VR professional- consumer therapeutic alliance quickly, consumers may be more likely to self-

Tools of Evaluation, Special Issue 2015 44

disclose pertinent information important to developing effective IPEs that can, in turn, lead to more accurate vocational profiles, greater satisfaction, and enhanced outcomes for consumers (e.g., employment; Lustig et al., 2002; Timmons, Schuster, Hamner, & Bose, 2002). Without a promptly established trusting relationship, critical consumer information may be overlooked (Berven, 2004). Missing important consumer information  may  then  derail  consumers’  and   VR  professionals’  rehabilitative  efforts  and   lead to ineffective IPEs. This may, therefore, result in a greater likelihood of negative consumer outcomes that fall short of expectations, and quite possibly, substantial inefficiencies that can cost the VR agency time and money (Chan, Shaw, McMahon, Koch, & Strauser, 1997; Kierpiec et al., 2010). Hence, within each professional- consumer interview encounter, accurate VR professional judgments are vitally important (Berven, 2008). Vocational rehabilitation professionals must recognize when and how to  adapt  to  meet  each  consumer’s   individualized needs in a culturally sensitive manner (Ivey et al., 2012). Such professional judgments are needed not only to formulate complete and valid vocational profiles, but to further establish strong therapeutic alliances that, ultimately, can enhance the quality of consumer outcomes (Austin & Leahy, 2015; Ivey et al., 2012; Lustig, 1996; Lustig et al., 2002; Strohmer & Leierer, 2000). Consequently, this prior discussion that has substantiated the importance of using interviewing as an effective assessment tool throughout the VR process, establishing strong therapeutic alliances, and making accurate VR professional judgments has led to the recommendations for interviewing techniques that follow below.

Rapid Engagement

Vocational rehabilitation professionals may want to incorporate rapid engagement (Miller & Rollnick, 2002) into their interviewing practice. This term is drawn from the counseling theory of Motivational Interviewing (MI) and maintains a philosophy congruent with the spirit of VR policy—taking a person- centered stance with consumers (Wagner & McMahon, 2004), where consumers are empowered to make informed choices that ultimately affect the direction of their rehabilitative planning and services (Kosciulek, 1999). Consumer motivation plays a vital role in this approach and can increase consumer involvement, self- determination, and development of goals. Motivational Interviewing seeks to obtain the maximum amount of vocationally applicable information in a realistic and reasonable period of time (Wagner & McMahon, 2004).

Rapid engagement can be defined as a focus on productive VR professional- consumer dialogue and factors (i.e., consumer issues) that make a difference in relation to employment (Rollnick, Butler, Kinnersley, Gregory, & Mash, 2010). MI has substantial empirical evidence to support its effectiveness and is emerging as a best practice in state VR agencies (Chan et al., 2012; Jackson & Franklin, 2014; Wagner & McMahon, 2004). However, the explicit focus  and  use  of  MI’s  rapid  engagement   framework and, thereby, the need for VR professionals to hone in on and adapt their interpersonal style and skills to establish a therapeutic alliance quickly within short time intervals (e.g., one hour, 30 minutes), may be particularly useful and most efficient. Likewise, this approach can be used to focus on employment from the first interface between the State VR agency and consumer, where a standard can be set about the structure of the relationship that revolves centrally around employment. Rollnick et al.

Tools of Evaluation, Special Issue 2015 45

(2010) proposed a three-pronged approach to working effectively and efficiently with consumers including (a) using a guiding style, (b) employing MI techniques, and (c) encouraging change talk.

A guiding style integrates three core interviewing skills of asking, listening, and informing. VR professionals ask consumers open-ended questions that encourage consumers to think about how and why they may want to make career or employment- related changes in their lives. This helps orient consumers to focus on employment and encourages them to think critically about their commitments to make necessary changes to pursue their employment goals. Vocational rehabilitation professionals listen in a manner that expresses empathy toward their consumers. This is accomplished by using reflective listening statements and paraphrasing techniques that encourage consumers to proceed with sharing their stories from their lens or worldview. With permission from the consumer, VR professionals also provide information about possible rehabilitative services and employment-related opportunities. This behavior demonstrates respect for and belief in a consumer and facilitates hope for a better future.

The use of a guiding style may be additionally enhanced and thereby, more effective, if counselors utilize additional MI tools that can encourage consumers to further engage and make a commitment to the VR process, including:

 Help the consumer identify what he or she wants to change immediately, and evaluate the strengths and weaknesses of these decisions.

 Guide the consumer to prioritize what is most important. This will lead to appropriate resources being allocated and rendered.

 Ask the consumer if he or she understands the information

presented, or if clarification is needed. A consumer needs to completely understand the information, or an informed decision will not be made when he or she is involved in rehabilitation planning and service choice.

MI’s  central  intention  is  to  increase   consumer motivation for change. Incorporating self-motivational statements (i.e., I want..., I can...,  I  will…)  or   promoting such change talk are behavioral examples  of  a  consumer’s  motivation  or   commitment to behavioral change. Therefore,  focusing  on  a  consumer’s   language with clinical intention will increase consumer dialogue, particularly when discussing why or how a consumer wants to change.

Applying these interviewing practices within a short timeframe may appear simplistic; however, the effective application of these strategies may be easier said than actually accomplished. VR professional self-awareness and the ability to  identify  the  need  to  adapt  one’s   interpersonal style and skill to meet the unique needs of consumers can be challenging, especially when feedback on clinical performance may not occur often enough. The use of efficient questions is an art form that must be purposefully orchestrated in the moment with consumers and it takes explicit practice to master this rapid engagement approach. Vocational rehabilitation professionals must believe that a consumer has the ability to change and that an employment outcome is possible, allowing consumers to lead in the process of change and when making final decisions about their rehabilitative planning and services.

Such an approach may not be adopted or come as easy for some VR professionals. Accordingly, obtaining feedback from respected colleagues or

Tools of Evaluation, Special Issue 2015 46

supervisors  on  one’s  clinical  performance   may be instrumental in learning and being effective in applying this approach. Vocational rehabilitation professionals interested in enhancing their interviewing skills in applying MI techniques should seek additional information and training resources, such as the Motivational Interviewing Online Training Resource (http://motivationalinterview.org) or the Technical and Continuing Education Center (TACE: http://www.tace5.siu.edu/programs.aspx). Such resources may enhance VR professionals’  interviewing  skills  of   reflective listening, questioning, and affirming  consumer’s  strengths  and  intrinsic   motivation for change.

Debiasing Techniques

One approach to enhance VR professionals’  judgment  accuracy  is  to   integrate the use of debiasing techniques into their interviewing skill repertoire (Austin & Leahy, 2015). Debiasing techniques can  be  defined  as  “…the  use  of   internal dialogues that serve as reminders to counselors to question their biases and use of other strategies or tools including scientific-based methods to improve clinical judgment  accuracy”  (Austin  &  Leahy,  2015,   p. 32). Underlying this definition are techniques that can be used during the VR professional-consumer interview to improve the  accuracy  of  VR  professionals’   judgments about the consumers they serve. Several of these techniques are described in detail below.

Use systematic and comprehensive interviews. To ensure important consumer information necessary for effective rehabilitation planning is not ignored and is gathered consistently across consumers, VR professionals are recommended to use a structured interview guide that intends to

capture essential consumer details (Garb, 1998). One example is the Integrated Structured Interview (ISI; McMahon & Watson, 2012). The ISI is meant to be applied during the interview to intermix effectively standardized interest tests (e.g., Self-Directed Search, Career Assessment Inventory) with narrative, meaning-making questions. For instance, the ISI can be used to  interpret  better  John  Holland’s  codes  (i.e.,   personality-environment compatibility) to maximize  consumers’  interpretation  of   results and create new, positive stories about their future careers. (See McMahon and Watson [2012] for example of structured interview guide).

Attend to quality of professional- consumer relationship. Strong VR professional-consumer therapeutic alliances in VR have been associated with positive rehabilitation outcomes for consumers (Donnell et al., 2004; Lustig et al., 2002; Lustig et al., 2004; Strauser et al., 2004). Consequently, the therapeutic alliance should be an essential part of the VR process, which may also instill hope, active participation, and lead to increased consumer job satisfaction (Lustig et al., 2002). Most importantly, high quality therapeutic alliances are primarily established during VR professional- consumer interviewing encounters. Bordin (1979) believed that consumer readiness to accept particular goals, professional theoretical orientation, and professional or consumer personality characteristics must all be considered as interacting variables that contribute to the strength of the therapeutic alliance. Vocational rehabilitation professionals should, therefore, check in often with themselves and their consumers throughout the VR process to ensure rapport is maintained. The Working Alliance Inventory (WAI) can be a useful clinical tool as one way to monitor the helping relationship (Horvath & Greenburg, 1989).

Tools of Evaluation, Special Issue 2015 47

Information to obtain the WAI can be found online at http://wai.profhorvath.com/.

Continuously evaluate and address own biases. Vocational rehabilitation professionals should consciously ask themselves  this  question,  “How  do  I  feel   about this consumer based on his/her culture (e.g., race/ethnicity, disability, age, social- economic  status,  gender)?” (Garb, 1998). This type of internal dialogue is needed to prompt VR professionals to take explicit action to address actively their own personal biases to maintain their effectiveness and expand their competencies in working with culturally diverse consumers. Vocational rehabilitation professionals are referred to Middleton et al. (2000) for ideas on how to address and overcome their own cultural biases.

Adapt use of counseling style. Vocational rehabilitation professionals must have the ability to identify accurately and meet the individualized needs of consumers based on their unique situations and cultures (Ivey et al., 2012). Use of Friedlander and Ward’s    (1984) relational styles (i.e., attractiveness, interpersonally sensitive, and task oriented) is one model that can be conceptualized for application to match moment-to-moment  changes  of  consumer’s   needs (See Appendix located in Bernard & Goodyear [2009] for copy of Supervisory Styles Inventory that can be used as a guide for shifting professional styles).

Use multiple sources of assessment. An interview integrated with multiple modes of assessments/tests (e.g., community-based work assessment, career interest inventories, report from family/significant others) is best practice and most valid (Berven, 2008; Meyer et al., 2001). Vocational rehabilitation professionals need to synthesize all relevant pieces of assessment data and communicate this information to consumers in an accurate and sensitive manner (Whiston, 2009).

Vocational rehabilitation professional assessment appraisals of their consumers using multiple methods will likely lead to more effective IPEs and positive rehabilitation outcomes (Hagner, 2010).

Clinical Supervision

Due to the complex nature of interviewing within the VR process, the potential impact interviewing has on service delivery, and the skills needed to apply techniques presented (e.g., rapid engagement, debiasing techniques), VR professionals are encouraged continually to develop and refine their interviewing skills. Clinical supervision is a process that can be used as a method for working practitioners to enhance their skills as an interviewer (Aasheim, 2012). Clinical supervision is a process consisting of regular, ongoing, and supportive interactions between a junior member of a profession and a more senior member of a profession (e.g., field supervisor, senior rehabilitation counselor) where the interactions focus on enhancing the clinical skill set of the junior member, as well as protecting consumer welfare (Bernard & Goodyear, 2009; Herbert, 2004). Examples of activities completed during clinical supervision include (a) a VR professional and supervisor addressing the VR professional’s  biases  when  working  with   a consumer who has a criminal history, (b) a supervisor modeling use of reflective statements for the VR professional, and (c) a VR professional working with a senior rehabilitation counselor to review an eligibility decision based on information collected during an initial interview. Clinical supervision includes many different types of interactions with a focus on enhancing the clinical skill set of the junior member while protecting consumer welfare; and interviewing skills can be enhanced through the use of clinical supervision.

Tools of Evaluation, Special Issue 2015 48

Regardless of interview structure type (e.g., unstructured, structured, semi- structured) a VR professional uses, or phase of the VR process (e.g., intake, eligibility, assessment), VR professionals can utilize clinical supervision. Within the helping professions, clinical supervision has been linked to increases in counselor awareness, wellness, multicultural competency, and case conceptualization skills (Bernard & Goodyear, 2009). Increases in these areas are typically associated with decreases in professional bias (or professional judgment error), which contributes to more effective VR professional clinical decision-making and consumer assessment. Despite the potential benefits of clinical supervision, the advantage of incorporating clinical supervision within state VR agencies tends to be poorly understood and irregular (Herbert, 2004; Herbert & Trusty, 2006). Lack of time, lack of training, and limited contact between supervisor and VR professional have been cited by rehabilitation counselors as barriers to clinical supervision (Austin, 2012; Herbert, 2004; McCarthy, Michiels, Blissett, & Shemshedini, 2013).

To address some of the barriers to receiving clinical supervision, peer supervision has been found to be helpful and to supplement clinical supervision received from a formal VR supervisor (Herbert, 2012). In general, when arrangements are made for peers to work together for mutual benefit, they are considered to be engaging in peer supervision and this can also be referred to as peer consultation (Bernard & Goodyear, 2009). A peer supervision or consultation model encourages self- evaluation by decreasing reliance on the expert supervisor. Peer support, encouragement, and enhancement of self- confidence are noted benefits of peer supervision (Benshoff, 1992). Reviewing individual cases during peer group

supervision provides opportunities for VR professionals to increase their awareness of potential biases they may hold, as well as receive feedback for future action to enhance  professionals’  judgment  accuracy.  

Conclusion

The work of VR professionals is

complex as many variables are involved in assisting individuals with disabilities to meet their vocational, educational, and independent living goals. Interviewing is one of many skills VR professionals must possess in order to accomplish the goal of assisting VR consumers with disabilities through the VR process to obtain and maintain employment. Interviewing skills are  considered  “foundational  skills”  that   each VR professional should continue to develop and refine. With the significance of using interviewing as a tool of assessment throughout the VR process, it is important for VR professionals to be thoughtful and focused on all of their consumer contacts that involve any element of interviewing and information gathering. This diligence will serve the rehabilitation field and VR consumers well.

Furthermore, the argument has been made in our discussion of the importance of cultivating a strong VR professional- consumer therapeutic alliance, as well as the critical need to make accurate professional judgments during the interview process. Interviewing techniques such as rapid engagement and debiasing techniques presented provide VR professionals with practical suggestions. Such techniques may not only improve the quality of VR professional-consumer rapport, but the validity  of  consumer’s  vocational  profiles;;   and thereby, decision-making that can lead to better rehabilitation outcomes for consumers. Given the potential benefits of clinical  supervision  for  consumer’s  and  VR  

Tools of Evaluation, Special Issue 2015 49

professionals alike (Austin, 2012; McCarthy, 2013), the use of clinical supervision  to  enhance  further  professionals’   interviewing skills and techniques proposed in this article is also recommended, which we believe is essential to enhancing VR professionals’  interviewing  skill   development. Finally, the importance of interviewing and its use as an effective assessment tool for VR professionals cannot be stated enough. Consequently, VR professionals (and consumers) will be well served if they recognize the strengths associated with the full utilization of interviewing and the positive implications it may have on the rehabilitation outcomes of consumers.

References

Aasheim, L. L. (2012). Practical clinical

supervision for counselors: An experimental guide. New York, NY: Springer Publishing Company, LLC.

Austin, B. S. (2012). A qualitative analysis of vocational rehabilitation counselor perceptions of clinical supervision. Journal of Applied Rehabilitation Counseling, 43(3), 25–33.

Austin, B. S., & Leahy, M. J. (2015). Construction and validation of the Clinical Judgment Skill Inventory (CJSI): Clinical judgment skill competencies that measure counselor debiasing techniques. Rehabilitation Research, Policy, and Education, 29(1), 27–46.

Benshoff, J. M. (1992). Peer consultation for professional counselors. Ann Arbor, MI: ERIC/CASS.

Bernard, J. M., & Goodyear, R. K. (2009). Fundamentals of clinical supervision. Upper Saddle River, NJ: Pearson Education.

Berven, N. L. (2004). Assessment. In T. F. Riggar, & D. R. Maki (Eds.),

Handbook of rehabilitation counseling (pp. 199–217). New York, NY: Springer Publishing Company, Inc.

Berven, N. L. (2008). Assessment interviewing. In B. Bolton, & R. Parker (Eds.), Handbook of measurement and evaluation in rehabilitation (pp. 243–261). Austin, TX: Pro-Ed.

Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research and Practice, 16(3), 252–260.

Carlisle, J., & Neulicht, A. (2010). The necessity of professional disclosure and informed consent for rehabilitation counselors. Rehabilitation Counseling Bulletin, 53(4), 218–225.

Chan, F., Shaw, L. R., McMahon, B. T., Koch, L., & Strauser, D. (1997). A model for enhancing rehabilitation counselor-consumer working relationships. Rehabilitation Counseling Bulletin, 41(2), 122–137.

Chan, F., Sung, C., Muller, V., Wang, C., Fujikawa, M., & Anderson, C. A. (2012). Evidence-based practice and research utilization. In D. R. Maki & V. M. Tarvydas (Eds.), The professional practice of rehabilitation counseling (pp. 391– 412). New York, NY: Springer Publishing Company, LLC.

Donnell, C. M., Lustig, D. C., & Strauser, D. R. (2004). The working alliance: Rehabilitation outcomes for persons with severe mental illness. Journal of Rehabilitation, 70(2), 12–18.

Drummond, R. J., & Jones, K. D. (2010). Assessment procedures for counseling and helping professionals. Upper Saddle River, NJ: Pearson.

Tools of Evaluation, Special Issue 2015 50

Friedlander, M. L., & Ward, L. G. (1984). Development and validation of the supervisory styles inventory. Journal of Counseling Psychology, 31(4), 541–557.

Garb, H. N. (1998). Studying the clinician: Judgment research and psychological assessment. Washington, DC: American Psychological Association.

Hagner, D. (2010). The role of naturalistic assessment in vocational rehabilitation. Journal of Rehabilitation, 76(1), 28–34.

Herbert, J. T. (2004). Qualitative analysis of clinical supervision within the public vocational rehabilitation program. Journal of Rehabilitation Administration, 28(2), 51–73.

Herbert, J. T. (2012). Clinical supervision. In D. Maki & V. Tarvydas (Eds.), The professional practice of rehabilitation counseling (pp.429– 448). New York, NY: Springer Publishing Company, LLC.

Herbert, J. T., & Trusty, J. (2006). Clinical supervision practices and satisfaction within the public vocational rehabilitation program. Rehabilitation Counseling Bulletin, 49(2), 66–80.

Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the working alliance inventory. Journal of Counseling Psychology, 36(2), 223–233.

Horvath, A. O., & Symonds, B. D. (1991). Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology, 38(2), 139–149.

Ivey, A. E., Ivey, M. B., Zalaquett, C. P., & Quirk, K. (2012). Essentials of intentional interviewing: Counseling in a multicultural world (2nd ed.). Belmont, CA: Brooks/Cole.

Jackson, C., & Franklin, K. (2014, March). Motivational interviewing affecting successful system change: Data and impact summary. Paper presentation at the National Rehabilitation Educators Conference, Manhattan Beach, CA.

Kierpiec, K. M., Phillips, B. N., & Koscuik, J. F. (2010). Vocational rehabilitation caseload size and the working alliance: Implications for rehabilitation administrators. Journal of Rehabilitation Administration, 34(1), 5–14.

Kosciulek, J. F. (1999). The consumer- directed theory of empowerment. Rehabilitation Counseling Bulletin, 42(3), 196–213.

Kosciulek, J. F. (2004). Empowering people with disabilities through vocational rehabilitation counseling. American Rehabilitation, 28(1), 40–47.

LeBlanc, S. M., & Smart, J. (2007). Outcome discrepancies in rehabilitation counseling. Journal of Applied Rehabilitation Counseling, 36, 12–19.

Lustig, D. C. (1996). Clinical judgment in vocational assessment. Vocational Evaluation and Work Adjustment Bulletin, 29(3), 59–63.

Lustig, D. C., Strauser, D. R., Rice, N. D., & Rucker, T. F. (2002). The relationship between working alliance and rehabilitation outcomes. Rehabilitation Counseling Bulletin, 46(1), 24–32.

Lustig, D. C., Strauser, D. R., & Weems, G. H. (2004). Rehabilitation service patterns: A rural/urban comparison of success factors. Journal of Rehabilitation, 70(3), 13–19.

Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review.

Tools of Evaluation, Special Issue 2015 51

Journal of Consulting and Clinical Psychology, 68(3), 438–450.

McCarthy, A. K. (2013). Relationships between supervisory working alliance and client outcomes in state vocational rehabilitation counseling. Rehabilitation Counseling Bulletin, 57(1), 23–30.

McCarthy, A. K., Michiels, P., Blissett, C., & Shemshedini, A. (2013). State vocational  rehabilitation  counselors’   perceptions towards supervision. Rehabilitation Counselors and Educators Journal, 6(2), 26–34.

McMahon, M., & Watson, M. (2012). Telling stories of career assessment. Journal of Career Assessment, 20(4), 440–451.

Meyer, G. J., Finn, S. E., Eyde, J. D., Kay, G. G., Moreland, K. L., Dies, R. R., …&  Reed,  G.  M.  (2001).   Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56(2), 128–165.

Middleton, R. A., Rollins, C. W., Sanderson, P. L., Leung, P., Harley, D. A., Ebener, D., & Leal-Idrogo, A. (2000). Endorsement of professional multicultural rehabilitation competencies and standards: A call to action. Rehabilitation Counseling Bulletin, 43(4), 219–245.

Miller, W. R., & Rollnick, S. (2002). Motivational Interviewing: Preparing people for change (2nd ed.). New York, NY: The Guilford Press.

Parker, R. M., & Patterson, J. B. (2012). Rehabilitation counseling: Basics and beyond (5th Ed.). Austin, TX: PRO-ED.

Power, P. (2006). A guide to vocational assessment (4th ed.). Austin, TX: Pro-Ed.

Power, P. (2013). A guide to vocational assessment (5th ed.). Austin, TX: Pro-Ed.

Rollnick, S., Butler, C. D., Kinnersley, P., Gregory, J., & Mash, B. (2010). Competent novice: Motivational Interviewing. British Medical Journal, 340(7758), 1242–1245.

Rosenthal, D. A. (2004). Effects of client

race on clinical judgment of practicing European American vocational rehabilitation counselors. Rehabilitation Counseling Bulletin, 47(3), 131–141.

Rosenthal, D. A., & Berven, N. L. (1999). Effects of client race on clinical judgment. Rehabilitation Counseling Bulletin, 42(3), 243–264.

Safran, J. D., & Muran, J. C. (Eds.) (1998). The therapeutic alliance in short term psychotherapy. American Psychological Association Publications.

Shahnasarian, M. (2004). The clinical interview in vocational assessment. In F. Flash, & S. Powell (Eds.), Directions in rehabilitation counseling: Volume 15 (pp. 1099– 1116). Long Island City, NY: Hatherleigh.

Strauser, D. R., Lustig, D. C., & Donnell, C. (2004). The relationship between working alliance and therapeutic outcomes for individuals with mild mental retardation. Rehabilitation Counseling Bulletin, 47(4), 215–223.

Strohmer, D. C., & Leierer, S. J. (2000). Modeling rehabilitation counselor clinical judgment. Rehabilitation Counseling Bulletin, 44(1), 3–9.

Thirtieth Institute on Rehabilitation Issues (2003). A new paradigm for vocational evaluation: Empowering the VR consumer through vocational information. (30th Institute on

Tools of Evaluation, Special Issue 2015 52

Rehabilitation Issues). Washington, DC: Rehabilitation Services Administration, U.S. Department of Education. Retrieved from http://www.iriforum.org/download/I RI30.pdf

Timmons, J. C., Schuster, J., Hamner, D. & Bose, J. (2002). Ingredients for success: Consumer perspectives on five essential elements to service delivery. Journal of Vocational Rehabilitation, 17(3), 183–194.

Trepper, T. S., McCollum, E. E., De Jong, P., Korman, H., Gingerich, W., & Franklin, C. (2010). Solution focused

therapy treatment manual for working with individuals: Research committee of the Solution Focused Brief Therapy Association, 1–16. Retrieved at http://www.sfbta.org/research.pdf

Wagner, C. C., & McMahon, B. T. (2004). Motivational interviewing and rehabilitation counseling practice. Rehabilitation Counseling Bulletin, 47(3), 152–161.

Whiston, S. C. (2009). Principles and applications of assessment in counseling (3rd ed.). Belmont, CA: Brooks/Cole Cengage Learning.

Author Note Bryan S. Austin, Ph.D., is an Assistant Professor/Program Coordinator of the Rehabilitation Counseling and Human Services Program at the University of Idaho-Boise. He is a Licensed Professional Counselor in the State of Idaho and Certified Rehabilitation Counselor. Carl W. Sabo, Ph.D., CRC, is an Assistant Professor in the Department of Human Services at Wright State University in Dayton, Ohio. Sabo is the Director of the Rehabilitation Counseling/Severe Disabilities Master's Degree Program. Amanda K. McCarthy, Ed.D., is an Assistant Professor in the School of Allied Health and Communicative Disorders at Northern Illinois University. She is a Certified Rehabilitation Counselor, Certified Vocational Evaluation Specialist, and Licensed Professional Counselor in the State of Illinois. Matthew E. Sprong, Ph.D., is an Assistant Professor in the Department of Rehabilitation and Human Services at Montana State University-Billings. He is a Certified Rehabilitation Counselor and Licensed Clinical Professional Counselor in Montana and Illinois. Lauren N. Noble is a master's student in the Rehabilitation Counseling and Human Services Program at the University of Idaho-Boise. She is interested in pursuing research regarding education of rehabilitation and human services. Noble is planning to finish her master's degree program in December 2015.

Return to Table of Contents

Tools of Evaluation, Special Issue 2015 53

Current Technology in Vocational Evaluation: Trends and Opportunities

Frances Smith Recognizing Differences.com

Pamela Leconte

George Washington University

William E. Garner University of Arkansas at Little Rock

Veronica I. Umeasiegbu

Southern University and A & M College

Abstract The role of technology in vocational evaluation has grown exponentially, particularly since the early 1980s with the introduction of desktop computers. Use of the Internet is now an essential part of vocational evaluation practice. Considerations of how evaluators have weaved computer technology into their assessment processes, based work samples and community assessments on industry technology, designed and developed reports, accessed web-based resources, and integrated assistive technology for best practices have played paramount roles in services. Fast forward to the 21st century where the digital reach and ubiquitous nature of technology have transformed  vocational  evaluators’  everyday  practices.  Today  vocational  evaluators  compose   reports through multiple computerized and web-based means, using tools such as voice activated word-processing, cloud-based collaboration and editing, and electronic-based work sampling. Assistive technologies are more present in services and inclusive frameworks such as universal design and universal design for learning shape best practices. The growing reliance on digital means to access inexpensive tools, communicate, and connect increases the likelihood that prior predictions for virtual ways to provide career assessment and vocational evaluation services are increasingly the norm of the future. The purpose of this article is to trace highlights in the history of  technology’s  role  in  VE  and  consider  potential  trends  and  opportunities. Keywords: Vocational Evaluation, Technology, Career Assessment, Virtual Vocational Evaluation, Universal Design, Universal Design For Learning, Assistive Technology, Tele- Evaluation

Current Technology in Vocational Evaluation: Trends and Opportunities

Tools of Evaluation, Special Issue 2015 54

The viability of our profession requires that vocational evaluation (VE) services be based upon contemporary and projected requirements for careers and work; therefore, professionals must integrate technology into administrative and direct service practices. For VE services to reflect accurately the current labor market, vocational evaluators must continue to incorporate real or simulated work, including current technology used in that work, into daily practices. To represent accurately  consumers’  skills  and abilities to enter or succeed in the labor market, assessment methods such as work samples and the various types of community-based vocational assessment options must mirror the jobs we recommend for evaluees. Finally, to ensure that consumers are able to demonstrate optimal performances and work behaviors, assessment integrating assistive technology (AT) should occur prior to and/or during the vocational evaluation process. All of this requires that vocational evaluators stay abreast of current and future technology developments and collaborate with business and industry to access technology that is intrinsically related to work for use in VE practices. Incorporating technology into VE services provides opportunities for vocational evaluators to acquire new skills and exercise their creativity.

In 1984, John Naisbitt released a second edition of Megatrends: Ten New Directions Transforming Our Lives, which provided a forecast for trends that would shape our future. Our comfort with an industrialized society and economy was quickly changing into one characterized by information-rich, technology-enhanced, and globally expansive approaches. In particular, the focus of technology was moving from one  characterized  as  “forced  technology”  to   one  of  “high-tech and high-touch”  (Naisbitt,   1984). This message is in sharp contrast to

the connected, ubiquitous world of technology we know today (New Media Consortium, 2014a). Technology and online learning have become common tools for practitioners to perform and share their work, communicate with one another, and stay abreast of information in the technology-rich 21st century. The advancing role of the Internet and social media as major sources for information sharing has redefined how most individuals seek answers, locate resources, and communicate with others. Online means for collaborative learning, evaluative tools such as learning analytics, and the proliferation of online growth point towards a future full of limitless possibilities (Jenkins, Clinton, Purushotma, Robinson, & Weigel, 2009; Li & Bernoff, 2008).

The purpose of this article is to trace highlights  in  the  history  of  technology’s  role   in VE and consider potential trends and opportunities. As we review the historical underpinnings of technology in the field and how policies, national positions, and innovations have had an impact on the way evaluators work, we also consider the importance of universal design and universal design for learning as frameworks to guide best practice in assessment, including the role of AT in socially just services (Leconte, Smith, & Johnson, 2007; Smith, Leconte, & Vitelli, 2012). Also, we explore how technology is and should be integrated into assessment practices across the methods and information resources that we use. Finally, we address the impact of distance learning and opportunities for virtual vocational evaluation practice and offer recommendations for future practice.

Historical  Perspectives:  Technology’s  

Role in Vocational Evaluation

Not long ago vocational evaluators used postal or inter-office/agency mail

Tools of Evaluation, Special Issue 2015 55

services to develop schedules, receive referrals, communicate with referral sources, and share reports. Most information was transmitted through paper or in-person conferences, reports were composed on legal pads or typewriters, and recommendations and career research were investigated through paper-bound sources such as the Dictionary of Occupational Titles and the Occupational Outlook Handbook. Much has changed!

Technological Innovations within Assessment for Employment Opportunities

Today, most evaluators conduct scheduling via electronic mail (e.g., email); receive referrals and send reports they develop on computers and other electronic devices via email, by using Google Docs® (http://docs.google.com), Wiggio® (http://www.wiggio.com), or other virtual communication platforms; and are in touch with referral sources almost immediately using texting or tweeting. Evaluators working with state agencies, such as vocational rehabilitation, may have access to agency Intranets, which allow communication in a safe, Health Insurance Portability and Accountability Act of 1996 (HIPAA)-compliant way. In the past, vocational evaluators dictated or wrote reports in longhand and gave these to secretaries to type. Now, most evaluators compose their reports using a variety of technologies such as laptops, tablets, or “smart”  devices  that  use  a  variety  of   software programs; some evaluators use Dragon Naturally Speaking Professional® (https://www.cdw.com) to dictate their behavioral observations and compose reports by voice. Evaluators also photograph and video record consumer work performances with cell phones and integrate these digital documents into electronic

reports or portfolios (e.g., e-portfolios). These multiple uses of technology have served to enrich the VE process for consumers and professionals. In fact, evaluators can provide VE to consumers from great distances via video conferencing platforms such as Skype® (http://www.skype.com), or Google Video Chat® (https://support.google.com), which is more secure than Skype®.

Technology’s  role  in  VE  always  has   been a critical consideration in the profession. As the desktop computer debuted in the mid-1980s, the value of computer-assisted report processing (Smith & Rothacker, 1986) and computer-aided assessment practices (Cusick, 1989) were explored and promoted. Over the years, educators and practitioners considered new innovations and proposed promising practices that opened pathways for expanding employment and educational opportunities for their consumers (Ashley & McGuire-Kuletz, 1999; Johnson, Hannon, & Leconte, 2003; McDaniel, Beadles, & McDaniel, 1997). Opportunities to capitalize upon  technology’s  innovation  were   proposed and, in some cases, adopted by the profession (MacIsaac, 2003; McDaniel et al., 2001; Smith, 1997). Virtual ways to expand VE practice have been modeled (Smith, 2010) and distance courses for graduate coursework have been explored (Tilton, McDaniel, & Lott, 2003; Smith & Leconte, 1999). Low-tech and high-tech assistive technology has been incorporated into VE services in a variety of ways (Noll & Lawler, 1998).

Professional associations, notably the Vocational Evaluation and Work Adjustment Association (VEWAA) and Vocational Evaluators and Career Assessment Professionals (VECAP) have promoted forward-thinking professional practices by inviting technology leaders to speak at the National Issues Forums in

Tools of Evaluation, Special Issue 2015 56

Vocational Assessment and Evaluation; for example, invitees included O*NET developers prior to the official roll-out of the system in 1998 (Hester & Baltrukenas, 1997), Brian Kurth of Vocation Vacations (http://www.vocationvacation.com) in 2010 demonstrating the use of the web-based career mentorship as another way to explore careers for adults seeking new career experiences, and Dr. Skip Rizzo (http://ict.usc.edu/profile/albert-skip-rizzo) who demonstrated in 2012 how virtual reality can be used as work samples and simulated work to help rehabilitate people with disabilities or others who face barriers to gaining or maintaining employment. Research conducted by Dr. Connie McReynolds at the Institute of Research, Assessment, and Professional Development regarding how neurofeedback can contribute to assessment interventions signifies an intriguing technological advancement (Gutierrez, 2013).

While electronic assessment instruments provide more options for accessing pay-as-you-go  or  free  “tools,”   they also potentially threaten the integrity of the assessment process if users fail to understand the characteristics of well- researched instruments or that the presence of a professional (preferably a vocational evaluator) is essential to guide and monitor the process. The prediction made by Thomas (1999), that the growth of VE will involve the use of technology and that “cyber- evaluation will take advantage of interactive computer technology in all aspects of assessment”  (p.  10)  has  materialized.   However, opportunities to maximize the assessment process and promote efficiency have been debated topics in the field (Thomas, 1989), partly because evaluators had and have so few resources to integrate new technology into practice.

The Necessity of Integrating Assistive Technology

In addition to reliance on electronic

communication and technology to operate our programs, vocational evaluators have integrated AT into services or have been encouraged to do so. When VE began to emerge from sheltered workshops and workforce development programs in the 1960s, and from school-based programs in the 1970s through the 1990s, AT consisted primarily of low-tech strategies and devices, such as jigs related to specific job tasks, magnifiers, and holders (Gugerty, Roshal, Tradewell, & Anthony, 1981). These devices allowed people with cognitive and physical disabilities to have access to, participate in, and perform work. For example, if someone with a severe cognitive ability can assemble some parts of a task, but not all, the task can be analyzed and divided into smaller tasks that two or more people can share to accomplish the larger task (Gold, 1989). Items, such as finger grips on pens, job-related jigs, Dycem material to hold items in place, cover guards for keyboards, and Velcro holders, allow people with physical disabilities or dexterity problems to perform required employment tasks. Though high tech devices may seem to dominate popular literature, low-tech devices can be as effective (Tibbs, 2002). However, examples of high tech devices have become more sophisticated as electronics, nanotechnology (Leconte, 2003; L. Johnson, personal communication, July 7, 2014), and scientific research have targeted specific disabilities (Gutierrez, 2013; Rizzo, 2012). For instance, robotic limbs are being refined daily to assist Veterans and others who have lost appendages—many of these rely on digital or nanotechnology systems to work. Again, vocational evaluators have long advocated for integrating AT (meaning both services and devices) into their

Tools of Evaluation, Special Issue 2015 57

programs, but unfortunately, insufficient fiscal resources remain barriers.

Federal legislation highlighting reasonable accommodations (Rehabilitation Act of 1973; Rehabilitation Act Amendments, 1978, 1986, 1998, 2014), technology assistance (Technology-Related Assistance for Individuals with Disabilities Act of 1988, 2004), and equal access and participation (Americans with Disabilities Act of 1990; Americans with Disabilities Act Amendments Act of 2008) underscored the role of AT in VE practice (Fried, 1993; Reed, 1993). The inclusion of AT was promoted by evaluators who also became experts in AT (Langton & Lown, 1995; Noll, 1993; Reed, Fried, & Grimm, 1993; Reed, Thomas, Lown, & Smith, 1995). Best practices were shared via short-term training (Noll & Lawler, 1998), published guidelines (Langton, 1993; Langton, Smith, Lown, & Chatham, 1998), and professional associations adopted national positions. In 1996, VEWAA appointed an AT taskforce and later published a position paper advocating the integration of AT in vocational evaluation, which was adopted by VECAP as well (Reed, 1996).

Impact of Universal Design on Vocational

Evaluation and Work

Also in the 1990's, the concept of universal design (UD) emerged and evolved into a field for promoting accessible product and environmental design. Coupled with the importance of assistive technology, the seven principles of UD provided guidelines for inclusive design that could be accessed by all (Story, Mueller, & Mace, 1998). The UD movement infiltrated VE; in fact, the combination of civil rights laws and UD demanded re-designs of assessment services and work environments used during VE and required that vocational evaluators recommend customized workplace

accommodations (Langton et al., 1998; Mueller, 1990, 1992).

In 2003, the Thirtieth Institute on Rehabilitation Issues (30th IRI): A New Paradigm for Vocational Evaluation highlighted the value of four key paradigms that would shape the future: empowerment, culture, universal design [emphasis added], and individualization. These key frameworks provide enhanced opportunities and expanded services and opportunities to all individuals seeking career awareness and development.  Technology’s  role  within  each   of these was featured as the use of computer technology and the Internet became more mainstream.

Universal Design for Learning (UDL): A

Framework to Guide Best Practice

The importance of UDL as an educational framework (Higher Education Opportunity Act, 2008; Rose & Gravel, 2010) continues to be recognized, as well as the value AT plays in enabling success for individuals with disabilities (Center for Applied Special Technology [CAST], 2014; Meyer, Rose, & Gordon, 2013; Rose & Meyer, 2002). The UDL framework began through early research efforts at Harvard University and CAST (Rose & Meyer, 2006). Based upon research from neuroscience on variability in learning, core principles of UDL underscore the importance of providing instruction and assessment opportunities in multiple ways to maximize learning across neural networks (Meyer & Rose, 2000). In 2008, UDL was fully defined in the Higher Education Opportunity Act of 2008.

The UDL approach works in harmony with central tenets of the VE process as it prioritizes the importance of multiple approaches to understand best the career strengths and needs of the evaluee (Smith, 2003). Since 2006, VECAP has

Tools of Evaluation, Special Issue 2015 58

developed national positions on the importance of UDL in VE practice (Leconte et al., 2007; Smith et al., 2012) and continues to contribute to the expansion of this knowledge. As a member of the national UDL taskforce, VECAP has been a voice for the profession and the importance of considering career assessment and VE through a UDL lens (National UDL Taskforce, 2014).

Using UDL, vocational evaluators can better design their planning, assessment, and reports in ways that consider the range of learning variability expected in all learners. A two-year grant funded project in Virginia, Expanding Career Options through Universal Design for Learning (ECO-UDL), demonstrated the success of melding career assessment with technology and instructional strategies through a UDL lens for students with disabilities in career transition (Harris, Dowd, & Smith, 2012).

Paradigm Shifts and Future Expectations

Since 2006, the rapid growth of Web 2.0 technologies has offered an array of freely available, web-based software tools that anyone can develop and recreate into tools for communication, collaboration, and resource development. The last five years of the New  Media  Consortium’s  (NMC)  annual   Horizon Reports have forecast the rapid adoption of social networking, grassroots video, user-generated content, collaboration webs, mobiles, cloud computing, open source, e-books and tablets (NMC, 2014b). Recent digital developments in our field have included the digitizing of all VEWAA Journals by the National Clearinghouse of Rehabilitation Training Materials. Both VECAP and VEWAA distribute information to members through a webpage and select social media channels including weblogs, social networks, Facebook, and Twitter. Also, VECAP and VEWAA provide

digitized versions of seminal texts in the field on their websites.

The use of distance or online learning has become a necessity in the field and a few universities are offering online certificates in VE. In 2013, Massively Open Online Courses (MOOCs) and tablet technologies continue to be prioritized as important to consider (NMC, 2013) in education. Open source technologies, such as wikis, Google Hangouts®, YouTube®, Wordpress® blogs, continue to grow and offer free, creative solutions for developing and sharing web content that is helpful to consumers and professionals. The importance of online learning has been heightened at a time when there are fewer financial resources for both individuals and agencies to access traditional learning venues. According to the Sloan Consortium students taking at least one online course grew to 6.7 million between 2003–2013 (Allen & Seaman, 2013). Sponsoring Third Thursday webinars, VECAP has taken advantage of the web to encourage and provide online learning (http://www.vecap.org). The potential to build and promote these types of rich learning opportunities is limitless with the tools available today and points towards new possibilities for VE practice.

Web-based Assessment Instruments and

Resources

Online assessment instruments have grown exponentially; this proliferation has become so vast that professional associations and authors have published cautions to the field (Leconte, 2000; Sampson & Lumsden, 2000), including updated Codes of Ethics and revised ethical guidelines for practice (American Counseling Association [ACA], 2014; Commission on Rehabilitation Counselor Certification [CRCC], 2010; National

Tools of Evaluation, Special Issue 2015 59

Career Development Association [NCDA], 2015). While the cost of online assessment instruments varies, there are a large number of free or low-cost assessment resources that show up in a simple search of search engines (Garner & Szirony, 2005). Identifying the reading level of online assessment instruments represents an initial caution for potential users as many providers do not take this into consideration.

Examples of online assessment instruments, both government- and private sector-based, suggest that these resources are available to assist vocational evaluators and can provide preliminary information for career and rehabilitation counselors who may not have in-house resources or geographic availability of comprehensive VE services. This is especially helpful for practitioners working in rural areas where limited resources and travel time prohibit on-site vocational screening of a new consumer’s  skills  and  abilities  to  determine   his or her needs, interests, and goals (30th IRI, 2003). Given that some professional training programs, especially those with the majority of their degree program online, are adapting their instructional strategies to utilize online assessment resources, many newly trained practitioners can develop the skills to seek out and take advantage of online assessment instruments (Garner & Dickerson, 2008).

Free or Low-Cost Online Assessment Instruments—The Private Sector

The following sections provide a few examples of free or low-cost private sector instruments available on the Internet across the categories of interest, temperament, aptitude, and achievement assessment. These are offered as examples of available instruments, with the caveat that neither VEWAA nor VECAP are endorsing these products, nor are we advocating their use.

Interest and temperament assessments. Commonly, vocational evaluators select an interest inventory as an initial assessment instrument to help plan the remainder of the assessment or evaluation process (Power, 2013). The Career Key (currently $12.95) is an online interest inventory that takes approximately 10- minutes  to  take  and  is  based  on  Holland’s  R- I-A-S-E-C Interest Codes. The Career Key provides a self-evaluation of skills, abilities, values, interests, and personality. It also identifies promising jobs and gives accurate information about them (Jones, 2014). Free temperament assessment instruments on the Internet often take the form of light-hearted, self-insight instruments of questionable validity (Garner & Szirony, 2005). The Keirsey Temperament Sorter II is an exception. Based on the well-researched Myers-Briggs Type Inventory, four character types or mega-temperaments— Rationals, Idealists, Artisans, and Guardians—are identified as well as 16 sub- types (Keirsey, 1996). Most if not all web- based assessment instruments rely on self- report, which needs to be verified and triangulated by using other methods and techniques (Smith et al., 1996).

Aptitude and achievement assessments. Aptitude and achievement instruments are also available free of charge. One example of a single aptitude assessment is the Free Practice Typing Test. It has one-, two-, and three-minute tests available with variable text choices, and provides automatic word-per-minute calculations as a quick way to assess keyboarding skills (TypingMaster Test, n.d.). Another example, the Free Reading Assessment, features two distinct sections. The phonics section pronounces the word audibly and asks the consumer to identify the appropriate word from a written word list. The Comprehension section follows the more traditional read-and-answer questions-on-

Tools of Evaluation, Special Issue 2015 60

the-content format (Sound Reading Solutions, n.d.). Again, vocational evaluators and others need to determine if these limited instruments meet the unique needs of their evaluees and can be used to direct or supplement additional assessment.

Free online assessment instruments tend to be user-friendly, self-directed, and self-reported. Often the private sector site offers a free or low-cost test hoping to attract buyers to their for-pay instruments. The downside is that free private sites may disappear (Barak, 2003; Leconte, 2000); however, sometimes a web search can yield the same instrument (or something similar) on another website (Garner & Szirony, 2005). These private vendor instruments are used to greatest advantage when trying to “locate”  one’s  interest,  temperament,   aptitude achievement or learning style preferences initially and when they are used to supplement findings from standardized, researched instruments.

Free or Low-Cost Online Assessment Instruments—Government-Supported Sites

Interest and aptitude assessments. The Occupational Network Service (O*NET) is an Internet-based database designed to replace the out-of-date Dictionary of Occupational Titles (DOT). During the development of the O*NET, vocational evaluators advocated to have the Department of Labor (DOL) include the details of specific jobs that help evaluators and consumers make employment “matches”  that  are  safe  and  doable  by  the   consumer. The DOL reported that it did not have sufficient fiscal resources to provide these levels of specificity (J. Ellis, personal communication, September 1997). The current database contains approximately 1000 occupations (Occupational Information Network [O*NET], n.d.a) that are classified

by an eight-digit code as the Standard Occupational Classification code (U. S. Department of Labor Employment and Training Administration, 2004). Vocational evaluators research the O*NET along with evaluees for career exploration and to make specific recommendations in reports. The O*NET Version 19 database (2014) has many uses, such as career counseling, job placement, and transferable skills analysis (Field & Field, 2004). The Career Exploration Tools comprise the actual vocational assessment portion of the O*NET and include three free assessment instruments; the Interest Profiler, Work Importance Locator or Profiler, and the Ability Profiler (O*NET, n.d.b).

The Interest Profiler comes in both downloadable paper and computerized formats. The software can be self- administered and self-interpreted, but most evaluees require support from the evaluator or counselor to gain the greatest benefit. It utilizes Holland's R-I-A-S-E-C interest structure (Power, 2013) and is based solely on self-report. Without the presence of a vocational evaluator who has the responsibility to monitor the administration and interpretation, results may be suspect or inaccurate and the attention to a systematic appraisal process may be lost (Smith et al., 1996).

The Work Importance Locator is in paper format and its software equivalent, the Work Importance Profiler, is free. The Work Locator and Profiler both measure six types of work values: Achievement, Independence, Recognition, Relationships, Support, and Working Conditions. The Work Locator and the Profiler match work values to the characteristics of O*NET occupations.

The Ability Profiler includes free paper assessment instruments supplemented by low-cost apparatuses for dexterity testing. The Ability Profiler measures nine job-

Tools of Evaluation, Special Issue 2015 61

relevant abilities: Verbal Ability, Arithmetic Reasoning, Computation, Spatial Ability, Form Perception, Clerical Perception, Motor Coordination, Finger Dexterity, and Manual Dexterity. It is based on many of the old General Aptitude Test Battery (GATB) concepts (Power, 2013), and is administered by practitioners in individual or group settings. Free software is available for computerized scoring. The results can be linked to occupations in the O*NET. The Ability Profiler optional apparatus sections include a Manual Dexterity Pegboard and a Finger Dexterity Board. The O*NET is dynamic and ever-changing. The potential for growth and development of the assessment components of the O*NET could have significant impact on private, for-pay assessment sites (Garner & Szirony, 2005).

Career information resources. Another resource, that also utilizes O*NET databases,  is  America’s  Career  Infonet,  part   of the CareerOneStop suite of web-based products funded and developed by the U.S. Department of Labor (CareerOneStop, n.d.). Use  of  America’s  Career  Infonet  can identify wage and employment trends, occupational requirements, state-by-state labor market conditions, nationwide employer contacts, and an extensive online career resource library (Disability.gov, n.d.).

Other government-supported resources are state electronic occupational databases, such as Texas Career Alternative Resource Evaluation System (CARES) which is O*NET and Holland-code based, but it has applications that considerably expand its utility (Garner, 2012). The online version of Texas CARES is a product of the Texas Workforce Commission/Labor Market and Career Information (TWC/LMCI) system and includes two major groupings – the World of Work and the World of Learning. Most states have similar databases and career information systems and many include assessment

instruments and activities, including the Virginia View (http://www.vaview.vt.edu) and  California’s  Eureka   (http://www.eureka.org). The Quality of Online Assessment Resources

For all testing resources, practitioners should consider the source with caution. Is the instrument valid and reliable as demonstrated by solid research studies? Are the norm groups appropriate to the individual or groups being tested? Reading levels vary so it is critical for the evaluator to make sure the instrument chosen is appropriate for the specific consumer. Careful examination of the documentation provided is critical for all professionals evaluating potential assessment instruments (Power, 2013). Vocational evaluators may use free e-assessments to determine if they meet the needs of the professionals and the consumers involved; however, it may be necessary to delve more in-depth and consider commercial assessment software to which higher costs are attached.

As previously noted, utilizing online rehabilitation resources for consumer service purposes requires a studied vigilance for ethical and practical difficulties that may present problems for practice—and the consumers. Many VE and counseling professionals and credentialing organizations have included technology- related assessment in their Codes of Ethics and are providing guidelines to help all professional minimize the negative impact of using invalid and unreliable assessment instruments (ACA, 2014; CRCC, 2008, 2010; NCDA, 2015). These guidelines apply to online assessment, as well as more traditional paper and pencil versions, and often include specific sections addressing technology-related issues. Of special concern for online assessment instruments is

Tools of Evaluation, Special Issue 2015 62

confidentiality of results and quality utilization of assessment information in vocational planning and placement. An additional caution regarding the availability of online instruments, especially free ones, is the possibility that an instrument available today may have changed significantly or may not be accessible the next time a vocational evaluators or other rehabilitation practitioner wishes to use it (Barak, 2003; Leconte, 2000). Then a new Internet search for a satisfactory tool must begin again (30th IRI, 2003).

Technology Innovation: Emerging Assessment Opportunities

The 21st Century has been

established as a unique period regarding the use of technology to provide services to individuals facing employment-related challenges. Technology is also playing a pivotal role in access to healthcare, an increase in community participation of individuals with disabilities, and new assessment practices. Two examples of innovative practice based in technology are tele-rehabilitation and using live, real-time video for conducting VE (Kirby, 2003; Tilton et al., 2003). McCue, Fairman, and Pramuka  (2010)  state  that  “tele- rehabilitation is an emerging method of delivering rehabilitation services that uses technology to serve consumers, clinicians, and systems by minimizing the barriers of distance,  time,  and  cost”  (p.  196)  and  this   growing discussion in the literature supports the use of tele-rehabilitation for remote evaluation, assessment and provision of care (Dinesen, Seeman, & Gustafsson, 2011; Piron, Tonin, Trivello, Battistin & Dam, 2004; Schmeler, Schein, McCue, & Betz, 2009). For years, vocational evaluators and other rehabilitation professionals have been conducting  “wired”  or  distance  services  to  

rural, remote areas such as Alaska (C. Veir, personal communication, 1994).

In 2002, a tele-health program using technology in conjunction with vocational evaluators in Washington, DC designed for immigrants was funded by the U.S. Departments of Commerce and Labor. The Community Preservation and Development Corporation (CPDC) worked with bio- medical engineering professionals at The Catholic University of America, Howard University’s  School  of  Social  Work,  and   interns from the Collaborative Vocational Evaluation  Master’s  degree  program  at  The   George Washington University (GWU) to develop a high tech career assessment process and training program (L. Johnson, personal communication, July 7, 2014). Nanotechnology,  “smart”  wiring  the   apartments of senior citizens to link directly to the hospital bio-medical laboratory and nursing stations, formed the basis of the services.  Some  “nano”  wearing  apparel  was   also used. High tech and entry-level skills work samples of High Tech Home Health Care Workers were created by the CPDC vocational evaluators and GWU interns.

Tele- or Distance Vocational Evaluation

Unfortunately, literature on the use tele-, cyber, and distance technology to provide VE is limited. Practice is driving the field to reconsider how services are or can be provided. With the availability of cyber and electronic assessment capabilities, VE services can be expanded to reach remote areas and to create more virtual work sampling. Having partnerships between universities and service programs has proven to help university faculty, as well as professionals, provide mutually beneficial research activities. One such collaboration example occurred between the Postsecondary Education Rehabilitation Transition (PERT) Program

Tools of Evaluation, Special Issue 2015 63

(http://www.wwwrc.net/pert), part of Woodrow  Wilson  Rehabilitation  Center’s   (WWRC) services; local high schools; state vocational rehabilitation; and a university. The project focused on the expansion of VE, occupational therapy assessment, speech assessment, and independent living assessment to transition-aged youth with disabilities to Virginia high schools. Video cameras were connected to a high school site and at WWRC via the state vocational rehabilitation connectivity system. The project was used with different high schools and was facilitated by a rehabilitation counselor who worked closely with schools. Other VE outreach from WWRC extended the use of digital technology via the Project Train IT (Kirby, 2003), using assessment and distance learning to improve employment outcomes.

As illustrated, distance VE can use image-based technology or videoconferencing for consultations, VE provision, and management of consumer services. Videoconferencing (tele-health) for remote management of prosthetic and orthotic needs of patients (Lemaire & Jeffreys, 1999; Lemaire, Boudrias, & Greene, 2001) was occurring about the same time the WWRC project was being piloted. Vocational evaluators could study the work of tele-medical researchers (Forducey, Ruwe, Dawson, Scheideman-Miller, McDonald, & Hantla, 2003; Savard, Borstad, Tkachuck, Lauderdale, & Conroy, 2003) for ideas about extending VE services and learn from the distance techniques used to assess and manage neurological impairments of medical consumers (Russell, 2007). Continued research efforts exploring applications of virtual reality therapy provide examples of the potential for vocational evaluators and other rehabilitation practitioners (Creative Technologies, 2013).

Expanded Opportunities in Rural Areas

Access to services is an important determinant of VE, and community services and access to transportation are some of the conditions that can be overcome by distance technology services. When one considers that about 20% of the U.S. population lives in rural areas (Mazurek et al., 2011), distance technology makes sense. Rural areas are known for limited services and poor rehabilitation outcomes for individuals with disabilities (Young, Strasser, & Murphy, 2004; Umeasiegbu, 2013). In addition to the examples noted earlier, more researchers and service providers are using the  “power  of  technology”  to  reach  more   consumers who live in rural locations. For example, Schopp, Johnstone, and Merveille (2000) used videoconferencing to provide cognitive assessment and psychotherapy for individuals with brain injury with positive rehabilitation outcomes. Schmeler and colleagues (2009) posited that tele- rehabilitation is a needed option to serve those who will need to travel far distance in order to receive rehabilitation services. The use of technology in the provision of services such as distance (tele-) VE can prove to be cost effective and lead to more effective caseload management (Umeasiegbu, 2013). Tindall and Huebner (2009) studied the impact of tele- rehabilitation on caregiver burden and found that time and costs were saved when professionals used such technology to provide speech therapy. As the scope of VE expands via technology, vocational evaluators can expand their expertise and competencies to reach more evaluees. The authors assume that most VE and rehabilitation practitioners are computer and technology literate, yet the lack of technology and digital connectivity by some may continue to foster a digital divide and prove to be a barrier that may hinder the

Tools of Evaluation, Special Issue 2015 64

innovative use and application of technology in practice.

Conclusion

A recent study by Sligar and Betters

(2012) suggested that the sociopolitical and scientific climate of the times will drive future VE training and service provision. We can see evidence of this—technological advancements are moving faster and are proving challenging for vocational evaluators and local resources to keep up. The key is to grasp opportunities created by science (e.g., technology) and integrate and build our services around them.

This requires that we prioritize a national advocacy and capacity-building agenda. If government entities cannot do this, we should do it on our own via VECAP, VEWAA, and partnerships with other national groups and industries, as well as local community employers. Further research is needed to explore the types of technologies necessary to provide effective and productive VE, to expand the number of consumers who can benefit from career opportunities yet to be developed, and explore innovative ways to share and deepen knowledge and understanding in the field.

References

Allen, I. E., & Seaman, J. (2013). Changing

the course: Ten years of tracking online education in the United States, Babson Survey Research Group. Retrieved from http://onlinelearningconsortium.org/s urvey_report/changing-course-ten- years-tracking-online-education- united-states/

American Counseling Association. (2014). 2014 ACA code of ethics. Retrieved from

http://www.counseling.org/resources /aca-code-of-ethics.pdf

Ashley, J. M., & McGuire-Kuletz, M. (1999). Integrating rehabilitation technology into the vocational assessment process. In R. Fry (Ed.), Ninth national forum on issues in vocational assessment: The issues papers (pp. 231–240). University of Wisconsin–Stout. Menomonie,WI: Stout Vocational Rehabilitation Center.

Barak, A. (2003). Ethical and professional issues in career assessment on the internet. Journal of Career Assessment, 11(1), 3–21.

CareerOneStop. (n.d.) America’s  career   infonet. Retrieved from http://www.careerinfonet.org/

Center for Applied Special Technology (2014). About UDL: What is universal design for learning. Retrieved from http://cast.org/udl/index.html

Commission on Rehabilitation Counselor Certification. (2010). Code of professional ethics for rehabilitation counselors. Retrieved from http://www.crccertification.com/fileb in/pdf/CRCCodeOfEthics.pdf

Commission on Rehabilitation Counselor Certification (2008). Code of professional ethics for vocational evaluation specialists, work adjustment specialists, and career assessment associates. Retrieved from http://www.crccertification.com/fileb in/pdf/VE/VECodeofEthics.pdf

Creative Technologies (2013). Marketplace features Skip Rizzo and virtual reality therapy for PTSD. Retrieved from http://ict.usc.edu/news/marketplace- features-skip-rizzo-and-virtual- reality-therapy-for-ptsd/

Tools of Evaluation, Special Issue 2015 65

Cusick, G. M. (1989). Computer-assisted vocational assessment. Vocational Evaluation and Work Adjustment Bulletin, 22(1), 19–23.

Dinesen, B., Seeman, J., & Gustafsson, J. (2011). Development of a program for tele-rehabilitation of COPD patients across sectors: Co- innovation in a network. International Journal of Integrated Care, 11, 1–12.

Disability.gov. (n.d.) America’s  career   infonet. Retrieved from https://www.disability.gov/resource/ americas-career-infonet/

Field, J. E., & Field, T. F. (2004). The transitional classification of jobs (6th ed.). Athens, GA: Elliot & Fitzpatrick.

Forducey, P. G., Ruwe, W. D., Dawson, S. J., Scheideman-Miller, C., McDonald, N. B., & Hantla, M. R. (2003). Using telerehabilitation to promote TBI recovery and transfer of knowledge. Neuro Rehabilitation, 18, 103–111.

Fried, J. H. (1993). The use of assistive technology in rehabilitation. Vocational Evaluation and Work Adjustment Bulletin, 26(1), 29–31.

Garner, W. E. (2012). Using Texas CARES web-based resources as an online classroom instructional tool. In Ed- Media 2012 World Conference on Educational Media & Technology. Norfolk, VA: Association for the Advancement of Computing in Education.

Garner, W. E., & Dickerson, L. R. (2008). Accreditation issues in online education programs: How to use electronic delivery methods to meet traditional standards. In Ed-Media 2008 World Conference on Educational Multimedia, Hypermedia & Telecommunications.

Norfolk, VA: Association for the Advancement of Computing in Education.

Garner, W. E., & Szirony, G. M. (2005). Free internet-based vocational assessment resources: Quantity and quality issues. In Ed-Media 2005 World Conference on Educational Multimedia, Hypermedia & Telecommunications. Norfolk, VA: Association for the Advancement of Computing in Education.

Gold, M. (1989). Try another way. Retrieved from http://mn.gov/mnddc/extra/marc- gold1.html

Gugerty, J., Roshal, A. F., Tradewell, M. D. J., & Anthony, L. (1981). Tools, equipment, and machines adapted for the vocational education and employment of handicapped people. Madison, WI: University of Wisconsin–Madison, Wisconsin Vocational Studies.

Gutierrez, J. (2013). Neurofeedback: Retraining the brain. Cal State University San Bernardino Magazine, 22(1), 1–8.

Harris, L. A., Dowd, L. R., & Smith, F. G. (2012, April). Expanding career options through universal design for learning (ECO-UDL). Paper session presented at the 15th VECAP National Issues Forum, San Bernardino, CA.

Hester, E. J., & Baltrukenas, V. (1997). Ability  measurement  in  DOL’s   occupational information network (O*NET): A factorial approach. In R. Fry (Ed.), Eighth national forum on issues in vocational assessment: The issues papers (pp. 22–31). University of Wisconsin–Stout. Menomonie, WI: Stout Vocational Rehabilitation Center.

Tools of Evaluation, Special Issue 2015 66

The Higher Education Opportunity Act. (2008). PL 110–315, 122 § 3078.

Jenkins, H., Clinton, K., Purushotma, R., Robinson, A. J., & Weigel, M. (2009). Confronting the challenges of participatory culture: Media education for the 21st century. Chicago, IL: The MacArthur Foundation.

Johnson, L., Hannon, C., & Leconte, P. (2003). The empowerment of vocational evaluation through electronic portfolio. In P. Leconte, & M. McGuire-Kuletz (Eds.), Eleventh national forum on issues in vocational assessment: The issues papers (pp. 29–45). Suffolk, VA: Suffolk Public Schools.

Jones, L. K. (2014). The Career Key. Retrieved from http://www.careerkey.org/

Keirsey, D. (1996). Keirsey Temperament Sorter II. Retrieved from http://keirsey.com/

Kirby, S. E. (2003). Project train IT: Linking assessment, distance learning, and employment outcomes. In P. Leconte, & M. McGuire-Kuletz (Eds.), Eleventh national forum on issues in vocational assessment: The issues papers (pp. 59–66). Suffolk, VA: Suffolk Public Schools.

Langton, A. J. (1993). Making more effective use of assistive technology in the vocational evaluation process. Vocational Evaluation and Work Adjustment Bulletin, 26(1), 13–19.

Langton, A. J., & Lown, N. (1995). Rehabilitation technology resources and services: How well are we using them in vocational evaluation? In R. Fry (Ed.), Seventh national forum on issues in vocational assessment: The issues papers (pp. 19–26). University of Wisconsin–Stout, Menomonie,

WI: Stout Vocational Rehabilitation Center.

Langton, A. J., Smith, F., Lown, N. F., & Chatham, L. J. (1998). Guide to using assistive technology in assessment and vocational evaluation. West Columbia, SC: South Carolina Vocational Rehabilitation Department.

Leconte, P. (2000, March). Potentials and perils of web-based vocational assessment instruments: Status of e- assessment. Presentation at the Virginia Transition Forum in Williamsburg, VA.

Leconte, P. (2003, March). Keeping up with cyber-humans: Vocational evaluation methods and techniques. Presentation at the Eleventh National Issues Forum in Vocational Assessment and Evaluation in Charleston, SC.

Leconte, P., Smith, F. G., & Johnson, C. (2007). The VECAP position paper on universal design for learning: Another step toward social justice. Vocational Evaluation and Career Assessment Professionals Journal, 4(1), 39–49.

Lemaire, E. D., Boudrias, Y., & Greene, G. (2001). Low-bandwidth, Internet- based videoconferencing for physical rehabilitation consultants. Journal of Telemedicine and Telecare, 7, 82– 89.

Lemaire, E. D, and Jeffreys, Y. (1999). Low-bandwidth telemedicine for remote orthotic assessment. Prosthetics and Orthotics International, 22, 155–167.

Li, C., & Bernoff, J. (2008). Groundswell: Winning in a world transformed by social technologies. Boston, MA: Harvard Business Press.

MacIsaac, K. (2003). Keeping current in the age of information: A challenge to

Tools of Evaluation, Special Issue 2015 67

vocational evaluation. In P. Leconte, & M. McGuire-Kuletz (Eds.), Eleventh national forum on issues in vocational assessment: The issues papers (pp. 156–162). Suffolk, VA: Suffolk Public Schools.

Mazurek, M., Johnstone, B., Hagglund, K., Mokelke, E., Lammy, A., & Yamato, Y. (2011). Geographic differences in traumatic brain injury and spinal cord injury rehabilitation. International Journal of Therapy and Rehabilitation, 18(10), 551–556.

McCue, M., Fairman, A., & Pramuka, M. (2010). Enhancing quality of life through telerehabilitation. Physical Medicine and Rehabilitation Clinics of North America, 21, 195–205.

McDaniel, R. S., Beadles, R. J., & McDaniel, N. B. (1997). Utilizing the World Wide Web for disability resources and vocational information for vocational evaluators. In R. Fry (Ed.), Eighth national forum on issues in vocational assessment: The issues papers (pp. 162–170). University of Wisconsin–Stout, Menomonie, WI: Stout Vocational Rehabilitation Center.

McDaniel, R. S., Brown, C. D., Browning, D., Tew-Washburn, S., Lott, A., McDaniel, N. B., & Barton, C. (2001). Get your degree on the Internet. In R. Fry & W. E. Garner (Eds.), Sixth national forum on issues in vocational assessment: The issues papers (pp. 61–75). University of Wisconsin–Stout, Menomonie, WI: Stout Vocational Rehabilitation Center.

Meyer, A., & Rose, D. H. (2000). The future is in the margins: The role of technology and disability in education reform. Retrieved from http://www.udlcenter.org/sites/udlce

nter.org/files/Meyer- Rose_FutureisintheMargins_2.pdf

Meyer, A., Rose, D. H., & Gordon, D. (2013). Universal design for learning: Theory and practice. Retrieved from http://udltheorypractice.cast.org/logi n

Mueller, J. (1990). The workplace workbook: An illustrated guide to job accommodation an assistive technology. Washington, DC: The Dole Foundation.

Mueller, J. (1992). The workplace workbook 2.0: An illustrated guide to workplace accommodation and technology. Washington, DC: The Dole Foundation.

Naisbitt, J. (1984). Megatrends: Ten new directions transforming our lives. New York, NY: Warner Books, Inc.

National Career Development Association. (2015). Two thousand and fifteen NCDA code of ethics. Retrieved from http://www.ncda.org/aws/NCDA/ass et_manager/get_file/3395

National Universal Design for Learning Taskforce. (2014). About the National UDL Taskforce. Retrieved from http://www.udlcenter.org/advocacy/t askforce

New Media Consortium (2013). NMC horizon report: 2013 higher education edition. Retrieved from http://www.nmc.org/pdf/2013- horizon-report-HE.pdf

New Media Consortium (2014a). NMC horizon report: 2014 higher education edition. Retrieved from http://www.nmc.org/pdf/2014- horizon-report-HE.pdf

New Media Consortium (2014b). NMC publications. Retrieved from http://www.nmc.org/publications

Tools of Evaluation, Special Issue 2015 68

Noll, A. J. (1993). Reasonable accommodations: Assistive technology and vocational evaluation. Vocational Evaluation and Work Adjustment Bulletin, 26(1), 25–28.

Noll, A., & Lawler, M. (1998). The integration and implementation of assistive technology into the vocational evaluation process. Menomonie, WI: Assistive Technology and Assessment Center.

Occupational Information Network. (2014). Data publication schedule. Retrieved from http://www.onetcenter.org/dataPubli cation.html

Occupational Information Network. (n.d.a). O*NET online. Retrieved from http://www.onetonline.org/

Occupational Information Network. (n.d.b). O*NET career exploration tools. Retrieved from http://www.onetcenter.org/tools.html

Piron, L., Tonin, P., Trivello, E., Battistin, L., & Dam, M. (2004). Motor tele- rehabilitation in post-stroke patients. Medical Informatics and the Internet in Medicine, 29(2), 119-125.

Power, P. W. (2013). A guide to vocational assessment (5th ed.). Austin, TX: Pro-Ed.

Reed, B. J. (1993). Historical overview of assistive technology. Vocational Evaluation and Work Adjustment Bulletin, 26(1), 7–11.

Reed, B. J. (1996). Position paper: Role of assistive technology in assessment and vocational evaluation. Vocational Evaluation and Work Adjustment Bulletin, 29(2), 34.

Reed, B. J., Fried, J. H., & Grimm, R. L. (1993). Expanding the vocational evaluation process through assistive technology. In R. Fry, & W. E. Garner (Eds.), Sixth national forum

on issues in vocational assessment: The issues papers (pp. 217–220). University of Wisconsin–Stout, Menomonie, WI: Stout Vocational Rehabilitation Center.

Reed, B., Thomas, S., Lown, N. & Smith, F. (1995). The role of the vocational evaluator in assistive technology assessments and accommodations. In R. Fry (Ed.), Seventh national forum on issues in vocational assessment: The issues papers (pp. 99–104). University of Wisconsin–Stout. Menomonie, WI: Stout Vocational Rehabilitation Center.

Rizzo, A. (2012). Clinical virtual reality: A brief review of the future. Paper session presented at the15th National Issues Forum in San Bernardino, CA.

Rose, D. H., & Gravel, J. W. (2010). Universal design for learning. In P. Peterson, E. Baker, & B. McGraw (Eds.), International encyclopedia of education (pp. 119–124). Oxford: Elsevier.

Rose, D. H., & Meyer, A. (2002). Teaching every student in the digital age: Universal design for learning. Alexandria, VA: Association for Supervision and Curriculum Development.

Rose, D. H., & Meyer, A. (2006). Preface. In D. H. Rose, & A. Meyer (Eds.), A practical reader in universal design for learning (pp. vii–xi). Cambridge, MA: Harvard Education Press.

Russell, T. G. (2007). Physical rehabilitation using telemedicine. Journal of Telemedicine and Telecare, 13, 217– 220.

Sampson, J. P., & Lumsden, J. A. (2000). Ethical issues in the design and use of Internet-based career assessment. Journal of Career Assessment, 8(1), 21–35

Tools of Evaluation, Special Issue 2015 69

Savard, L., Borstad, A., Tkachuck, K., Lauderdale, D., & Conroy, B. (2003). Tele-rehabilitation consultants for clients with neurologic diagnoses: Cases from rural Minnesota and American Samoa. Neurorehabilitation, 18, 93– 102.

Schmeler, M. R., Schein, R. M., McCue, M., & Betz, K. (2009). Tele- rehabilitation clinical and vocational applications for assistive technology: Research, opportunities, and challenges. International Journal of Telerehabilitation, 1(1), 12–24.

Schopp, L. H., Johnstone, B. R., & Merveille, O. C. (2000). Multidimensional telecare strategies for rural residents with brain injury. Journal of Telemedicine and Telecare, 6(S1), 146-149.

Sligar, S. R., & Betters, C. J. (2012). The state of state vocational evaluators: A national study. Journal of Rehabilitation, 78(4), 21–30.

Smith, F. G. (1997). VEWAA online and other World Wide Web resources for practitioners. In R. Fry (Ed.) Eighth national forum on issues in vocational assessment: The issues papers (pp. 171–176). University of Wisconsin–Stout, Menomonie, WI: Stout Vocational Rehabilitation Center.

Smith, F. G. (2003). Universal design for learning and vocational evaluation: Recognizing the parallels. In P. Leconte, & M. McGuire-Kuletz (Eds.), Eleventh national forum on issues in vocational assessment: The issues papers (pp. 67–72). Suffolk, VA: Suffolk Public Schools.

Smith, F. G. (2010). Social media: New opportunities for the profession. Feature session presented at the 11th

National Issues Forum in Oklahoma City, OK.

Smith, F. G., & Leconte, P. (1999). Designing and delivering an online graduate course in vocational evaluation. Paper session presented at the 10th International Conference of the Division on Career Development and Transition in Charleston, SC.

Smith, F. G., Leconte, P., & Vitelli, E. (2012). The VECAP position paper on universal design for learning for career assessment and vocational evaluation. Vocational Evaluation and Career Assessment Professionals Journal, 8(1), 13–26.

Smith, F. G., Lombard R., Neubert D., Leconte, P., Rothenbacher, C., & Sitlington, P. (1996). The position statement of the Interdisciplinary Council on Vocational Evaluation and Assessment, April,1996. Career Development for Exceptional Individuals, 19, 73–76.

Smith, F. G., & Rothacker, L. B. (1986). Computer assisted report processing in vocational evaluation. In R. Fry (Ed.), Second national forum on issues in vocational assessment: The issues papers (pp. 163–167). University of Wisconsin–Stout, Menomonie, WI: Stout Vocational Rehabilitation Center.

Sound Reading Solutions. (n.d.). Free reading assessment. Retrieved from http://soundreading.com/online- reading-assessment.php

Story, M. F., Mueller, J. L., & Mace, R. L. (1998). The universal design file: Designing for people of all ages and abilities. Raleigh, NC: North Carolina State University: The Center for Universal Design. Retrieved from

Tools of Evaluation, Special Issue 2015 70

http://www.ncsu.edu/ncsu/design/cu d/pubs_p/pudfiletoc.htm

Thirtieth Institute on Rehabilitation Issues. (2003). A new paradigm for vocational evaluations: Empowering the VR consumer through vocational information (30th Institute on Rehabilitation Issues). Washington, DC: Rehabilitation Services Administration, US Department of Education. Retrieved from http://www.iriforum.org/download/I RI30.pdf

Thomas, S. W. (1989). Commercial marketers: Do they help or hinder the field? In R. R. Fry (Ed.), Fourth national forum on issues in vocational assessment: The issue papers (pp. 1–4). University of Wisconsin–Stout, Menomonie, WI: Stout Vocational Rehabilitation Center.

Thomas, S. W. (1999). Vocational evaluation in the 21st century: Diversification and independence. Journal of Rehabilitation, 65(1), 10– 15.

Tibbs, S. F. (2002). Low-tech assistive technology: Changing roles and paradigms in rehabilitation. Heath Newsletter. Washington, DC: Heath Resource Center, George Washington University.

Tilton, J. S., McDaniel, R. S., & Lott, A. C. (2003). Best practices for web based distance education instruction. In P. Leconte, & M. McGuire-Kuletz (Eds.), Eleventh national forum on issues in vocational assessment: The issue papers (pp. 171–175). Suffolk, VA: Suffolk Public Schools.

Tindall, L. R., & Huebner, R. A. (2009). The impact of an application of telerehabilitation technology on caregiver burden. International Journal of Telerehabilitation, 1(1), 3–8.

TypingMaster. (n.d.). TypingMaster online test. Retrieved from http://www.typingtest.com/

Umeasiegbu, V. I. (2013). Determinants of healthcare access after spinal cord injury: A literature review. Journal of Applied Rehabilitation Counseling, 44(2), 11–17.

U. S. Department of Labor Employment and Training Administration. (2004). O*NET –beyond information - intelligence. Retrieved from http://www.doleta.gov/programs/one t/

Young, A. E., Strasser, R., & Murphy, G. C. (2004).  Agricultural  workers’  return   to work following spinal cord injury: A comparison with other industry workers. Disability and Rehabilitation, 26(17), 1013–1022.

Author Note Frances G. Smith, Ed.D.,CVE, is an Adjunct Professor at George Washington University and an Independent Consultant of http://recognizingdifferences.com. She has been affiliated with the field of vocational evaluation since 1978. She has provided direct service as a certified vocational evaluation specialist to secondary through college age students as well as adult consumers. She has an M.A from the Collaborative Vocational Evaluation Training Program at GW University and has taught as an adjunct for them since 1988. She has served in leadership roles for VEWAA, VECAP, and the former Commission on Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES). She is a Past-President of VECAP and represents their interests on the National Universal Design for Learning (UDL) Taskforce.

Tools of Evaluation, Special Issue 2015 71

Pamela J. Leconte, Ed.D., CVE, is a Professorial Lecturer/Retired Assistant Research Professor at George Washington University. She has worked in the field of vocational evaluation since 1968 and recently retired from George Washington University where she directed the Collaborative Vocational Evaluation Training graduate program. The program was funded by the U.S. Department of Education, Rehabilitation Services Administration and prepared vocational evaluators for both rehabilitation and transition services. She has served in leadership positions for VECAP, VEWAA, and the former Commission on Certification of Work Adjustment and Vocational Evaluation Specialists (CCWAVES). William E. Garner, Rh.D., CRC, LPC, is an Associate Professor at University of Arkansas at Little Rock. He has 38 years of experience in the rehabilitation counseling profession including 33 years in rehabilitation education. He has worked with distance education for the last 18 years and was instrumental in the founding the online rehabilitation counseling program at the University of Arkansas at Little Rock, which was the first completely online program, of any type, in the state of Arkansas. Garner has acted as a consultant to the Commission on Rehabilitation Counselor Certification for the last 20+ years and served as a member of the Arkansas Board of Examiners in Counseling. Veronica I. Umeasiegbu, PT, PhD, CRC, is an Assistant Professor at Southern University and A & M College. She has diverse knowledge on disability and rehabilitation. Previously, she taught physical therapy courses at the University of Nigeria. In 2008, she served as an intern at the Rehabilitation and Disability Unit of the WHO in Geneva Switzerland. She is a contributor to the CBR Guidelines published by the WHO. Umeasiegbu is a graduate of rehabilitation counseling programs at University of Pittsburgh (Masters) and University of Kentucky (Ph.D). Umeasiegbu is a Certified Rehabilitation Counselor (CRC), and she is currently an assistant professor at the Southern University and A&M College.

Return to Table of Contents

Tools of Evaluation, Special Issue 2015 72

The Impact of Future Technology on Vocational Evaluation

Randall S. McDaniel Auburn University

Scott Beveridge Christian Chan

The George Washington University

Jeremy Cushen

Abstract Vocational evaluation can benefit from future technology developments; therefore vocational evaluators must stay abreast of those changes that not only will have an impact on their service functions but support better consumer outcomes. Changes in the physical technology, the ability to offer services and receive education at a distance, and concern for data security are three areas of future technology impacting vocational evaluation. Future vocational evaluation will be more technology-based providing both evaluators and their consumers improved access to information relevant to better assessment outcomes. Improved access promises better evaluator resources to facilitate matches from evaluator recommendations to consumer outcomes, while technology development holds the promise of providing sophisticated education, training, and assessment tools for evaluators. There are possibilities to use emerging technologies more broadly in vocational evaluation, including providing evaluation services at a distance and gaining more efficient online education and knowledge sharing with fellow practitioners. A number of developing technologies hold promise in supporting the typical evaluation processes and have the potential of providing more objective methods of performing vocational evaluation functions. With the continuing usage of big data to aid in service delivery, the risks of a breach of that data has surged from Internet-based hacking. Security measures to protect personal data have to be considered in the design and provision of future technology-driven vocational evaluation services. Thus, this paper explores the potential future development of computing technologies and its use in evaluator functions and education, with suggested methods to improve data security concerns. Keywords: Technology, Security, Vocational Evaluation

The Impact of Future Technology on Vocational Evaluation

The integration of technology in

American culture has affected the work methods and procedures used in occupations from cabinet making to medicine. Over a decade ago, the authors of Thirtieth Institute on Rehabilitation Issues (30th IRI), A New

Paradigm for Vocational Evaluation, (2003) suggested that vocational evaluators need to keep pace with current tools and technology in the workforce in order to maintain competence in their role. In the assessment process, however, a number of the evaluation methods and procedures have remained relatively static (e.g., testing, interview, observation), even though

Tools of Evaluation, Special Issue 2015 73

technology has been somewhat integrated into the process. For example, vocational evaluators can perform computer-based testing that administers, scores, and summarizes results. Additionally, behavioral observations can be recorded on touch screen tablets and report writing often is done with the aid of a word processing program. The advance in hardware, software programs, and applications (apps) suggests that methodological changes are on the forefront for service provision in vocational evaluation (VE). Some of these advances and potential impact on vocational evaluation will be discussed here and it is recommended that vocational evaluators stay informed of developing technologies that can be adapted to improve evaluation outcomes.

The seemingly overnight introduction of new technologies makes it difficult to foresee technology developments a decade or more in the future. However, current technology trends suggest future possibilities. These trends affect the provision of vocational evaluation services both in-house and at a distance. For example, the delivery of vocational evaluation educational information to and between practitioners is changing with the aid of technology. Additionally, vocational evaluation can use technology to facilitate knowledge gain and sharing within the community of practice, regardless of physical location and time boundaries, through asynchronous online education. Similarly, in service provision, having the consumer and evaluator in the same space has been a requirement to provide the service since the beginning. However, online technologies expand the possibility of reaching those in need of service at alternate sites, including their home setting, facilitating the provision of services in rural areas or to locations more convenient to the evaluee. The bright side of technology in

vocational evaluation is that it can provide enormous gains in ease, efficiency, evidence-based practice, and robustness of recommendations. The dark side of technology is that any data transported on the Internet can be intercepted and exposed, requiring the use of more sophisticated data security to keep personal data confidential. The themes of developing physical technology, distance education and service, and data security, as applied to vocational evaluation, are explored in this paper.

Physical Technology

Moore’s  Law  in  computing  suggests   that computing power (i.e., the number of transistors in the central processing chip) will  double  every  two  years  (Moore’s  Law,   n.d.).  Moore’s  Law  has  held  reasonably true during the advent and development of computing power since the early days of transistors resulting in computer processors capable of executing calculations 5,000 times faster than the original processor on the first IBM personal computer (Brain, n.d.).  Combining  Moore’s  Law  with   calculation speed gains and the shrinking size of processors has resulted in a proliferation of computing devices and activities. That proliferation has affected VE and is expected to have an increasing impact on the provision of VE services. The shrinking size of computer processors has allowed them to be wedged into any electronic device we purchase. With the advent of nanotechnology, it is theoretically possible that future computers can exist at the microscopic level making them even more of an increasing part of everyday function. In addition to providing information, smaller computing devices are being used not only to monitor all things human but also to provide enhanced views of our current reality. A down side of the ubiquitous use of computers is that they can

Tools of Evaluation, Special Issue 2015 74

overload practitioners with data they may or may not know how to analyze and this overload tends to turn practitioners into computer technicians who spend increasing time in front of a computer and less time in front of people. Vocational evaluators are, however, accustomed to using computers in their testing, communication, and report writing, and therefore current developments in technology will be adapted for the evaluation process. One of the more exciting future trends in technology that may affect VE is wearable technology.

Wearable Technology

Fashion meets technology with wearable technologies such as bracelets, fitness monitors, and enhanced watches that offer body functioning monitoring, application use across a spectrum of functions, and Internet connectivity. Recently introduced from several manufacturers are glasses with head mounted displays that can both record what the wearer is seeing and provide stored or Internet-derived information to the wearer projected to the inside of the glasses. One example, the Google glass, attempted to provide full-time mobile computing power in our everyday life in wearable form (https://developers.google.com/glass/design/ ui). Early versions of this hardware contained an optical head mounted recorder and display, a touchpad, a camera that can save and/or broadcast pictures and video, and voice interaction with the Internet through an imbedded microphone and earpiece. Other companies, such as Microsoft (Luk & Ovide, 2013), have provided their own version and it is expected that the large technology players, such as Apple and Amazon, will have similar devices; current anticipation is these devices will become much more functional.

This type of wearable headset provides the functions of a smart phone or tablet computer but in a small unit attached to eyeglasses. Examples of early uses of Google glass included helping a surgeon virtually augment an operation, recording a medical  exam  on  a  patient’s  retina,   identifying shrapnel in a wound, viewing breastfeeding instructions for mothers to use while nursing, monitoring endangered species in Nepal, and broadcasting surgeries and autopsies for teaching purposes. Thus, a wearable headset offers the potential of providing information on the world seen through the glasses, serving as a video recording and Internet broadcasting device, and providing informational or behavioral prompts to the wearer.

The use of wearable technologies has potential to provide service changes in VE. The bulk of current vocational evaluation testing done is through the provision of paper and pencil and work sample testing. In these testing methods, stimuli are presented, responses recorded, and results are interpreted. Responses (quality or speed) are most often compared to a normative sample (i.e., norm referenced interpretation) or a minimum job criterion to evaluate a consumer’s  performance  in  relation  to  that   reference group or standard. However, other fields such as medicine and engineering have adopted more accurate measurements of human body responses from their patients to enable their results to be understood utilizing an individualized interpretation. In rehabilitation, this method is most often seen in physical capacity evaluations. Individualized or ipsative interpretation of test results is similar to a cyclist comparing ride time over a set distance to his or her personal best or individual average rather than his or her placement in a group. This rating method allows self-competition and opens up improved training possibilities with increased health benefits to the cyclist.

Tools of Evaluation, Special Issue 2015 75

Similarly, personal monitors that analyze physical responses such as heart rate, blood pressure, sweat intensity, brain wave changes, and blood markers could offer more objective measurements of response to vocational testing protocols, which are harder to fake or misinterpret. The use of wearable technologies, such as monitoring bracelets, placed on evaluation consumers can provide objective measures that better compare their performance with previous performance. These data can certainly be norm referenced but offer the added advantage of improved objectivity and self- analysis.

Newer technologies, such as head mounted displays, provide the vocational evaluators an alternative method of performing their work functions and learning from VE experts who are better trained and experienced in the field. Vocational evaluators typically read multiple reports from other practitioners about the consumer, provide vocationally relevant testing, make behavioral observations, and integrate the data collected. They then compare that information to a variety of factors they find predict success in training or employment. This information is compiled into a vocational evaluation report that can be shared with the consumer, the referral source, and potential employers (Thirtieth Institute on Rehabilitation Issues [30th IRI], 2003). Wearable technologies offer help with all of these functions. For example, reading of previous reports can be done using a head mounted display with voice notes or summaries made by the evaluator. Evaluators are required to record behaviors in work-like settings. Mobile recording/broadcasting devices offer a more efficient method to record snippets of typical behaviors or test results for later review and communication with the consumer, the referral source, and potential employers as

part of a narrative and video final report. An area not well utilized is the comparison of the consumer data with data from previous research that can guide the vocational evaluation recommendations within a framework of researched best-evidenced based recommendations. This technology provides a means to that end.

In addition to its usage for service provision, wearable technologies offer a training opportunity for the practicing vocational evaluator. Using the broadcasting feature of headset recording technology would easily allow live peer or expert review  of  the  evaluator’s  performance.  In   this scenario, a supervisor or master-level practitioner could make behavioral prompts or suggestions in the real time environment by watching and listening to the reality occurring in front of the evaluator. These types of emerging wearable technologies are just the beginning of a more broadly included usage of technology in the evaluation process and do not even consider the capacity of these developing technologies on ameliorating functional limitations with assistive technologies. In addition to wearable and smaller technologies used in evaluation, changes in the use of technology will alter the way services are provided and for vocational evaluator education.

Service Provision and Education

at a Distance Changes to Practice and New Challenges

The changes in technology are dynamically shifting the field of rehabilitation and VE to new opportunities. Within the scope of those opportunities, there are shifts in responsibilities, roles, and ethics, which require a dynamic body of research (Barnett, 2005), additional strategies for practice, and innovative tools

Tools of Evaluation, Special Issue 2015 76

in rehabilitation education pedagogy (Layne & Hohenshil, 2005). Considering that vocational rehabilitation was focused on building face-to-face therapeutic alliances with consumers, changes in technology have altered perspectives for how practitioners are to form appropriate and ethical therapeutic alliances. Addressing those changes has been emphasized across international boundaries, where counseling presents an entirely different cultural shift (Goss & Anthony, 2009; Kolog, Sutinen, & Vanhalakka-Ruoho, 2014).

Because these changes in technology present opportunities and challenges, a significant component of rehabilitation practice is how to adopt the commonly used face-to-face counseling and evaluation methods into an online modality, especially when the modality consists of several potential services (e.g., telephone, emails, texting, video chatting). Guidelines and codes of ethics should be addressing how to adopt a variety of interview skills (Ivey, Ivey, & Zalaquett, 2010) when those interview skills may not fully translate into the environment using online evaluation methods. For example, evaluators may not have the ability to utilize fully interview skills, such as examining nonverbal actions of consumers, which are important components of communication. Due to this particular issue, rehabilitation service providers may need to operate on different technology platforms and recognize what interventions are most effective for each individual consumer.

Multiple researchers have also noted that research regarding online counseling and evaluation is scarce, which demonstrates the gap in providing guidelines and defining effectiveness in practices. These researchers (Mallen, Vogel, & Rochlen, 2005a; Mallen, Vogel, Rochlen, & Day, 2005b) observed that investigators rarely take on the topic of service delivery in online settings as a

scholarly interest. This lack of interest reinforces the literature gap in having appropriate resources to conduct online service, despite the change of practices from exclusively face-to-face interactions to online interactions (Chang, 2005). Other researchers also support the notion that technological changes will allow practitioners to reach more consumers and assist in reducing barriers to receiving services (Mallen & Vogel, 2005; Sanchez- Page, 2005). Examples of these consumers include rural populations, members of the military serving in remote theatres of operation (e.g., Afghanistan), or persons with disabilities who may not be able to travel to a service center. While utilizing technology for consumer services can be assistive, Caspar and Berger (2005) suggested that researchers have new ethical responsibilities to develop research that addresses best practices and opportunities for online services. Similarly, Barros-Bailey and Saunders (2010) noted the gaps in the literature regarding the impact of technology in counseling, which necessitated their adaptation in the Commission on Rehabilitation Counseling Certification (CRCC) 2010 Code of Ethics to connect with the new competencies of using technology in counseling and rehabilitation services.

Impact of Technology on Education/VE Training

Increasing competencies. Altering the modality of vocational evaluation will necessitate an updated set of competencies, as well as a revision of pedagogical tools and psychometric testing instruments. Barros-Bailey and Saunders (2010) noted the significant role technology has played in changing  rehabilitation  practitioners’  code   of ethics, including shifts in the prior curriculum in graduate training programs

Tools of Evaluation, Special Issue 2015 77

and changes in the services provided. One major point they observed was the challenge of securing the administration of web-based pedagogical tools (e.g., video lectures, podcasts, iTunesU, YouTube) when those tools can easily be altered from their original format. In addition to this challenge, Barros- Bailey and Saunders (2010) explained that there is very little research defining the effectiveness of training tools for rehabilitation counseling and vocational evaluation training programs, which leaves rehabilitation educators with limited resources.

There is a paucity of peer-reviewed research on conducting vocational testing at a distance or utilizing new technologies. The comprehensive nature of a vocational evaluation presents challenges to obtain objective results when utilizing new communication technologies. For example, when completing a vocational evaluation, an evaluator uses clinical interview and psychometric testing to obtain data on a consumer’s  vocational  interests,  aptitudes,   academic achievement, personality, preferences, work history, learning style, behavioral observations, medical and/or psychological history, and work-affecting limitations. The instruments utilized to collect this information have been standardized for face-to-face assessment and relatively few of these are available online. Utilizing these instruments at a distance raises a number of questions. Can an evaluator send the instruments to the consumer via the mail? How does technology accommodate this challenge? How would the reliability and validity of the findings (e.g., testing environment, duration of the instrument, taking breaks, privacy, assistance) be affected?

Mallen et al. (2005a) provided several recommendations to address the continuing use of technology in providing career counseling. Their recommendations

specifically targeted the development of professionals in training regarding online counseling, as many service providers are not fully equipped or have even considered providing services online. Television programs (e.g., Web Therapy) have even poked fun at the counseling profession for having counselors who are ill-equipped to attempt online counseling services. Mallen et al. (2005a) offered many options that rehabilitation educators and supervisors could take when developing the competencies and transferrable skills from face-to-face interactions and the new service provision environment. They explained that service provider trainees would benefit heavily from increasing their online interactions (e.g., Skype), which would increase their familiarity and experience with online platforms. The trainees would then develop connections between their face-to-face training in helping skills to their experiences online.

Online education and supervision. The 30th IRI (2003) noted a desperate need for increased vocational evaluator training, which could be assisted through online education and supervision. There are major benefits to implementing programs and courses that are delivered in an online or technology-based format. Layne and Hohenshil (2005) offered that supervision and education can work with more flexible schedules for instructors, supervisors, and students. Cicco (2012) emphasized the importance of utilizing diverse tools to engage trainees in their coursework. Specifically, Cicco (2012) identified the combination of classroom technology, especially online assessments, surveys, and Blackboard technology, to develop creative formats in which learning can take place. McAdams and Wyatt (2010) also identified that distance education can provide access to more students working with a diversity of populations, especially rural populations and

Tools of Evaluation, Special Issue 2015 78

lower socioeconomic status (SES), who do not have the funding to transport themselves to institutions or organizations on a frequent basis. In addition, Beveridge, Karpen, Hadjiyane, Weiss, and Liu (2014) explained that graduate students who were currently practicing in their field found a specific online education program to be extremely beneficial, even when the course content was distributed exclusively online. Significant results from the Beveridge et al. (2014) study reported that students who completed the program had more satisfaction in their learning, experienced a significant increase in their earnings, and were given advanced roles in their employment as rehabilitation service providers and vocational experts.

There are also significant difficulties when examining the quality of supervision provided by technology-based formats. Mallen et al. (2005a) offered multiple perspectives on major issues that occur when providing online supervision. For example, immediate consultation or supervision would be difficult because supervisors may not be readily available due to variables such as separate time zones and physical presence. Supervisors may have more challenges in ascertaining particular feelings of their supervisees, especially if the supervisees appear anxious or happen to be in different geographic locations. The potential challenge in being unable to ascertain those feelings can become barriers to a successful supervisory relationship. Mallen et al. (2005a) did not posit that a successful supervisor relationship was not capable through online formats, but they concluded that beneficial relationships would take a longer time to develop and would have new unique challenges.

Certificates vs. Degrees (MOOCs). Development of training should address the ideas about how pedagogy and curriculum are organized when delivering education and

supervision. There has been a major debate about what would be most effective and ethical for service providers. One idea is the application of massive open online courses (MOOCs), which allow numerous individuals to partake in educational content or coursework at very low costs (Spector, 2014). This avenue could address the significant changes in technology that are affecting higher education (McAdams & Wyatt, 2010; Moon, 2013). Although this method sounds ideal, the concept can also be lacking, because training modules and resources are developed without supervision (Spector, 2014). Consequently, many individuals who participate in these MOOCs would be left without feedback from an expert and/or without ideas of how to implement these tools into practice. The idea from  Spector’s  (2014)  perspective  reinforces   the outcome of the Beveridge et al. (2014) study, where students found their interaction with the instructor to be the most meaningful experience (e.g., weekly video lectures, personal telephone calls, Skype), which allowed them to engage with the content at a higher level. Additionally, individuals taking MOOCs would also not be able to receive a degree from an institution; they would receive a certificate of completion rather than college credits. Spector (2014) noted that there is much potential to implement the concept of MOOCs into instructional learning, but the suggestion is to combine the MOOCs into “mini-MOOCs,”  which  would  target   personalized learning in addition to web- based learning with experts. Spector (2014) suggested that MOOCs are not fully ready to implement, as initial troubleshooting issues have not been resolved and research into their effectiveness is scarce; this suggestion coincides with the call of many other researchers (Barros-Bailey & Saunders, 2010; Caspar & Berger, 2005; Mallen & Vogel, 2005, Mallen et al., 2005a; Mallen et

Tools of Evaluation, Special Issue 2015 79

al., 2005b) to engage in more research regarding the effectiveness of online education and incorporation of technology into providing services.

Providing Services at a Distance

Advantages. One of the major benefits of providing services using a diversity of technology (e.g., phone, text, chat, video conference) is its accessibility to a diverse clientele (Riemer-Reiss, 2000). Utilizing several technology platforms to provide VE and services can extend the outreach beyond the areas that are local to the service provider. Vocational evaluation services at a distance can especially utilize web-based video chatting and live conferencing to provide interactions similar to face-to-face services. This access to services addresses several concerns posed by the rehabilitation profession in how to access underserved populations. Many of these underserved populations come from rural regions and regions with low socioeconomic status (SES) that do not have VE services readily available. Common occurrences for these populations are longer periods of time for an evaluation, lower frequency of rehabilitation services, and higher rates of consumer dropout.

An additional benefit of utilizing technology  to  provide  services  is  the  field’s   adaptation to societal changes. Communication in society and rehabilitation is transitioning beyond face-to-face interactions. As a result, this change requires rehabilitation providers to enhance their services by providing options and recognizing new forms of communication. Furthermore, significant advantages in the use of technology with services have highlighted the barrier to seeking services, especially when consumers come from cultures that have stigmatized receiving these services. An interesting point that

Mallen et al. (2005a) posited was that some practitioners who utilized English as a second language mitigated their own barriers through communication with consumers through the use of technology. They provided the example of a practitioner who had less anxiety in providing services because of reduced self-consciousness about his/her accent.

Disadvantages and concerns. There are multiple concerns behind the scope of providing services through diverse technology formats. One significant concern is an array of ethical dilemmas that needs to be examined before working with these technological formats. For example, ethical and legal boundaries may preclude evaluators from providing services to consumers who are in different states or countries. There are different governing bodies regarding the provision of services that are not parallel with licensure laws that vary from state to state (McAdams & Wyatt, 2010). For instance, Licensed Professional Counselors in Pennsylvania have different requirements for licensure than New York. When a counselor in Pennsylvania uses online technology to provide services to consumers in New York, there is a risky ethical or legal situation due to the different licensure requirements. Similarly, consumers who are in different countries (e.g., Afghanistan), including military service members, may also present another ethical dilemma for service providers utilizing this type of technology to provide rehabilitation services. The codes of ethics from several professional identities (e.g., American Counseling Association, Commission on Rehabilitation Counselor Certification) often warn service providers of the implications for providing services beyond the limitations of licensure. The requirements vary by state and among countries.

Tools of Evaluation, Special Issue 2015 80

Using technology in counseling and assessment, especially through the use of telephone, messaging, chat, or Skype, also remains questionable when falling under the critiques of interview skills and helping relationships (Ivey et al., 2010). Much of the non-verbal communication that is a significant component of helping relationships will be lost (Mallen et al., 2005a). Service providers may not be able to view  the  consumer’s  body  language. In addition, providers may also miss the ability to  attune  to  consumers’  facial  expressions,   vocal tone, and volume, especially in the cases of telephone or chat utilization. When these skills are challenged, establishing a working alliance and trust is more difficult. When the possibility of a working alliance is hindered, connecting with a consumer may become more challenging.

Another perspective to consider is the assumptions that are created when developing the use of technology in VE. A major challenge is that low SES individuals may not have the access or competency to utilize technology, despite the ability to mitigate barriers of transportation or geography (Sampson & Makela, 2014). Another important group to consider is the older adult population, who may not have the familiarity or comfort level in using technology (Riemer-Reiss, 2000). Additionally, services provided online may not be appropriate for consumers diagnosed with severe psychiatric disabilities, especially when the level of risk is high (e.g., suicidal or homicidal ideation). The other major assumption is that technology will be completely reliable (Layne & Hohenshil, 2005). That situation, however, may not always be the case, as wireless frequencies are interrupted and in some rural geographical areas the latest high speed technology may not be available (or affordable). Interruptions in session can detract from the service provision.

These challenges require continuous scholarship to highlight best practices about how to implement technology in assessment (Mallen & Vogel, 2005; Sampson & Makela, 2014). Counselors and evaluators will need to think comprehensively about the appropriateness of the services, the ethical risks, and the well-being of each unique consumer and what works best for that individual. With service provision and online education, personal and private consumer and student data is generated in the process that is expected to be confidential for the evaluator and student/consumer. Unfortunately, sophisticated cyber criminals have regularly hacked into that confidential data and exposed it or sold it to anyone with an Internet connection. With reports of rogue groups and nation backed hackers repeatedly breaching personal data, data security has moved to the forefront of technology.

Data Security

Computing and technology have progressed over the past 30 years, especially in the areas of computing speeds and processing power. Through these technological advances, three truths have become more apparent: (1) computers have become part of a global network of information sharing resources and tools capable of instantaneous speeds; (2) miniaturization has allowed computers to take the form of phones and mobile devices; and (3) data security is a balance between a safe environment for our business and personal information, and the ability to access it at any time or from any place.

In order to understand data security, understanding the risks these evolutions present is important. The great thing about a world of inter-connected systems is the ubiquitous availability of information. For example, a recent advance in computer

Tools of Evaluation, Special Issue 2015 81

technology is cloud computing, which provides a solution to store and retrieve files anywhere our wired or wireless systems can access: two common providers are Google Drive (http://www.google.com/drive/ ) and Dropbox (https://www.dropbox.com/). In addition to file storage, these technologies allow for sharing and collaboration of information  with  “trusted”  people  and   protection  from  “untrusted”  people.  As   access to personal or private data becomes easier, so does the opportunity for malicious actors to find and obtain this data. To ensure that information remains secure and free from theft, we have to look at two sides of the cloud-computing picture.

When using a cloud system from service providers, such as those listed above, to what means do they go to ensure that your data is safe and can they be trusted? Do they understand that the data stored is more than just pictures and spreadsheets, but data that contains personally identifiable information or personal health information? In addition to the cloud providers, the members of the VE community are trusted representative of consumer records, and must ensure the end user side of data protection. In an article released on lifehacker.com, Gordon (2014) indicated that five million Google accounts were hacked and usernames and passwords were released. A great deal of personally identifiable information or personal health information could be gained in such a theft. Companies such as Google and Dropbox would be out of business if they could not ensure protection of the data they store. This is one example of what professionals working with confidential consumer information should consider before utilizing cloud storage.

As with cloud computing and the required security of the service provider, the sometimes overlooked tool is the mobile device used to access the cloud. We utilize these systems every day to connect to the

cloud, store data, and connect to others for collaboration. As we use these devices, how do we ensure the same level of security that is placed on the cloud-computing providers is placed on the personal mobile device? There exists both a physical and logical threat to security with such devices. Due to the size of mobile devices, they present a security threat due to the ease of being lost or stolen and the type of information they possess. Most users do not have the technical knowledge to protect data stored on these devices to prevent unauthorized access in the event of a stolen or lost device. The logical threat is that of the applications that are installed, the types of connections we allow (e.g., WiFi, Bluetooth, Near Field Communications) and the way we restrict their activity. Some of the ways to ensure that systems remain safe is by using software from reputable companies, avoiding connections to unknown wireless access points, and disabling unused components, such as Bluetooth, when not in use. These examples show that even basic users must know the importance of understanding system configurations and account and password management. This basic knowledge is vital to securing information systems.

Everyday users have the ability to send real-time video across the world to facilitate both business and personal interaction. Individuals can now use this tele-presence to oversee patient care and recovery and come together as a group to solve real problems. Cisco and other video conferencing companies have created conference rooms that are simply monitors attached to devices to give the appearance of a fully functioning conference room for users around the world to participate. This technology allows for individuals to collaborate over great distances from the comfort of their homes or businesses. As this technology used in medicine allows

Tools of Evaluation, Special Issue 2015 82

patient assessments to be completed, it also allows for worldwide expertise to be shared, ensuring a patient gets the best treatment possible with the least impact to financial resources. This solution provides quality care without the cost limiting the level of treatment provided.

While this technology provides advantages, it still requires a great deal of end-user understanding of technology and data protection from service providers to ensure confidentiality. Cloud computing is set so that information users share individual resources but have logical boundaries between the data. As with the physical world, where we can build fences and walls and utilize guards, the logical world has similar protections but the most basic of users can sometimes overcome these boundaries intentionally or deliberately. Much like cloud computing, the same concerns exist with the need to protect the information that allows these collaborations to remain private. Whether it is a file or video being sent across the network, it still represents data that must be protected.

Technology provides interactions like any seen in previous generations. Today’s  security  system  and  home   automation provide instant feedback if a sensor is tripped or the system is disarmed, as well as providing the gratification of knowing  one’s  homes  and  valuables  are   secure through remote communications. With these technologies allow this type of connectivity, they also open the door to real time monitoring of individuals with conditions that require in depth tracking, though this would occur within call centers instead of individual nurses at a single home. Most people would say this is depersonalizing health care, but given the current level of medical treatment availability, we have the technology that can allow greater support with less personnel thereby resulting in cost savings. Again,

security is a prime concern that needs to be tracked in order to establish such resources for the future.

Data security typically revolves around three topics: confidentiality, integrity, and availability (Price, 2008). As vocational evaluators utilize the technologies mentioned, each new method must define what the most important component of data security is. All three techniques of data security may be required, but it is very dependent on the type of information being shared or stored. When dealing with information such as medical treatment or care history, confidentiality is pivotal in how we utilize this information. We must ensure the proper encryption of the data and the means that is transported. As we add these techniques to the data, we begin losing the speed at which the information can be shared as well as storage limitations.

Within the VE process, integrity, the next component of data security, can change an individual’s  vocational  outcome if not ensured. As the individuals are tested on various levels of vocational ability, they are scored and rated on various scales. Without data security to ensure integrity, these scores could be modified in transit or maliciously altered to provide an unfair advantage when determining individual assessments. Implementing the security features to ensure integrity may not be as costly as that of confidentiality, but still requires systems that can support the software that ensures data integrity. An example of this type of protection is through the use of hashing algorithms that allow detection of data tampering.

The least critical of the three components of data security in dealing with VE is that of data availability. During the vocational evaluation process, if a system takes several hours to download a file versus a few minutes, nothing is really lost. Where

Tools of Evaluation, Special Issue 2015 83

this becomes extremely important is in the redundancy of the data, which still falls into the availability category. As previously discussed, the use of cloud computing connects practitioners to available information; however if proper checks are not in place, the data one expects to retrieve may be lost in the cloud forever.

As described above, data security, involving the confidentiality, integrity, and availability of information, is technologically advanced and must be implemented early in the VE process. How does all of this information affect the technology and services afforded to the VE process? Technology has become increasingly more intertwined into our lives with devices such as the smart thermostat by NEST (https://store.nest.co) and other home automation systems. As these solutions provide methods to make day-to-day tasks more manageable, they also lead to the ability to provide capabilities for people with disabilities. These sophisticated systems are just sensors with a connection to the Internet. Similar devices are being used to monitor treatment or facilitate the recovery of individuals, with the same goal. As more of these systems are developed that are dependent on the Internet, extra precautions, such as restricting access and implementing protection systems, are required.

The technology to advance the vocational evaluation process is available and will only continue to benefit the community. From a data security perspective, these technologies all rely on the ability to protect the data as it rests and as it transits the Internet. In order to utilize these advances in technology, it will be the responsibility of ALL who are entrusted with the protection of this data.

Summary

Technology is producing changes in the VE process and future technologies offer the potential for radical changes in VE. For example, it will be possible for future vocational evaluators to collect more accurate and objective physical human body responses to stimuli that can improve measurement by increasing accuracy. By allowing the evaluator to tap into the wealth of knowledge from evidence-based practice, integration and interpretation of the collected evaluation data will be more scientifically based, providing better outcomes. In addition, information collected during the evaluation process can be processed efficiently using technology. With technology, VE will expand its boundaries from the centralized service center to locations convenient to the consumer. In addition to service provision improvement, future technologies are expected to provide improved educational opportunities through the provision of online education and possibly evaluator monitoring and prompting by supervisors and/or master level evaluators. In this scenario, the instructor/mentor can see and hear everything the evaluator sees and hears in real time and can provide informational prompts to aid the evaluator in his or her duties. While information flow is expected to be increasingly available over larger distances, the concern was raised about how to keep that information (e.g., personal data or consumer information) confidential and secure from the public. Data security concerns relate to confidentiality, data integrity, and availability of the data. Increasing use of cloud-based data storage presents unique data security issues but improves the availability of evaluation data to a multitude of devices. As future technologies become available and affordable, VE methods, procedures, and appearance will change drastically from today’s  service  while  offering  an  improved  

Tools of Evaluation, Special Issue 2015 84

and more efficient product with better consumer outcomes. That time cannot get here fast enough.

References

Barnett, J. E. (2005). Online counseling:

New entity, new challenges. The Counseling Psychologist, 33(6), 872–880. doi: 10.1177/0011000005279961

Barros-Bailey, M., & Saunders, J. L. (2010). Ethics and the use of technology in rehabilitation counseling. Journal of Applied Rehabilitation Counseling, 41(2), 60–64.

Beveridge, S., Karpen, S., Hadjiyane, M. C., Weiss, B. A., & Liu, X. (2014). A mixed-methods case study of an online forensic rehabilitation counseling graduate certificate program. The Rehabilitation Professional, 22(3), 149–162.

Brain, M. (n.d.). How microcomputers work. Retrieved August 22, 2014 from http://computer.howstuffworks.com/ microprocessor1.htm.

Caspar, F., & Berger, T. (2005). The future is bright: How can we optimize online counseling, and how can we know whether we have done so? The Counseling Psychologist, 33(6), 900–909. doi: 10.1177/0011000005280183

Chang, T. (2005). Online counseling: Prioritizing psychoeducation, self- help, and mutual help for counseling psychology and practice. The Counseling Psychologist, 33(6), 881–890. doi: 10.1177/0011000005279962

Cicco, G. (2012). Designing innovative counseling courses: Combining technology, theory, and practice. Journal on School Educational Technology, 7(4), 1–7.

Gordon, W. (2014, September). 5 million online passwords leaked, check yours now [Updated]. Retrieved from http://lifehacker.com/5-million- gmail-passwords-leaked-check- yours-now-1632983265

Goss, S. S., & Anthony, K. K. (2009). Developments in the use of technology in counselling and psychotherapy. British Journal of Guidance & Counselling, 37(3), 223–230. doi: 10.1080/03069880902956967

Ivey, A. E., Ivey, M. B., & Zalaquett, C. P. (2010). Intentional interviewing and counseling: Facilitating client development in a multicultural society (7th ed.). Belmont, CA: Brooks/Cole, Cengage Learning.

Kolog, E. A., Sutinen, E., & Vanhalakka- Ruoho, M. (2014). E-counselling implementation: Students' life stories and counselling technologies in perspective. International Journal of Education and Development using Information and Communication Technology, 10(3), 32–48.

Layne, M. C., & Hohenshil, T. H. (2005). High tech counseling: Revisited. Journal of Counseling and Development: JCD, 83(2), 222–226.

Luk, L., & Ovide, S. (2013, October, 22) Microsoft tests eyewear similar to Google Glass. The Wall Street Journal. Retrieved from http://www.wsj.com/news/articles/S B1000142405270230440210457915 0952302814782

Mallen, M. J., & Vogel, D. L. (2005). Online counseling: A need for discovery. The Counseling Psychologist, 33(6), 910–921. doi: 10.1177/0011000005280182

Mallen, M. J., Vogel, D. L., & Rochlen, A. B. (2005a). The practical aspects of online counseling: Ethics, training,

Tools of Evaluation, Special Issue 2015 85

technology, and competency. The Counseling Psychologist, 33(6), 776–818. doi: 10.1177/0011000005278625

Mallen, M. J., Vogel, D. L., Rochlen, A. B., & Day, S. X. (2005b). Online counseling: Reviewing the literature from a counseling psychology framework. The Counseling Psychologist, 33(6), 819–871. doi: 10.1177/0011000005278624

McAdams, C. R., & Wyatt, K. L. (2010). The regulation of technology- assisted distance counseling and supervision in the United States: An analysis of current extent, trends, and implications. Counselor Education and Supervision, 49(3), 179–192.

Moon, J. (2013). MOOCs plus. Issues in Science & Technology, 30(1), 16–19.

Moore’s  Law.  (n.d.).  Retrieved  July  10  2014   from http://www.mooreslaw.org/.

Price, S. (2008). A matter of integrity [Web log message]. Retrieved July 21, 2014 from http://blog.isc2.org/isc2_blog/2008/0 5/a-matter-of-int.html

Riemer-Reiss, M. (2000). Utilizing distance technology for mental health

counseling. Journal of Mental Health Counseling, 22(3), 189–203.

Sampson, J. P., & Makela, J. P. (2014). Ethical issues associated with information and communication technology in counseling and guidance. International Journal for Educational and Vocational Guidance, 14(1), 135–148. doi:10.1007/s10775-013-9258-7

Sanchez-Page, D. (2005). The online- counseling debate: A view toward the underserved. The Counseling Psychologist, 33(6), 891–899. doi: 10.1177/0011000005279964

Spector, J. M. (2014). Remarks on MOOCS and mini-MOOCS. Educational Technology Research and Development, 62 (3), 385–392.

Thirtieth Institute on Rehabilitation Issues. (2003). A new paradigm for vocational evaluations: Empowering the VR consumer through vocational information (30th Institute on Rehabilitation Issues). Washington, DC: Rehabilitation Services Administration, U.S. Department of Education. Retrieved from http://www.iriforum.org/download/I RI30.pdf

Author Note Randall S. McDaniel, Ed.D., CRC is the Wayne T. Smith Distinguished Professor Emeritus of Rehabilitation at Auburn University where he taught for more than 40 years in the Department of Special Education, Rehabilitation, and Counseling. His work and educational background are all in the area of rehabilitation. Some of his professional roles include: Co-Editor of the VEWAA Journal; board member for the National Council on Rehabilitation Education, National Rehabilitation Association, Vocational Evaluation and Work Adjustment Association, and Vocational Evaluation   and   Career   Assessment   Professionals’   organizations;;   and   President   of   several professional organizations including VEWAA. McDaniel has authored more than fifty journal articles and has edited or contributed to the publication of five books. He has been an invited or keynote presenter at more than 100 regional, national, and international meetings. He continues work in private practice as a vocational expert and life care planner. Scott Beveridge, Ph.D., LCPC-S, CRC, CDMS began working as a rehabilitation counselor in

Tools of Evaluation, Special Issue 2015 86

the private-for-profit sector in 1989. He began his academic career at the University of Maryland in 2004 as a Research Associate Professor and joined the George Washington University in 2007 as an Assistant Professor. His current duties include development of new academic programs, counseling courses and concentrations (TBI, Autism, and Substance Abuse), methods of instruction and administration of academic programs. He developed and implemented the Forensic Rehabilitation Counseling certificate program (second forensic rehabilitation counseling program in the United States and the only one to collaborate with law school faculty). Beveridge has authored 22 journal articles, several grants, contributed to three books and has completed 38 regional, national and international presentations. In addition to the above duties, Beveridge has a private practice providing forensic rehabilitation services in Maryland, Pennsylvania, and DC, and provides pro bono mental health counseling for veterans via Give an Hour. Christian D. Chan, MA, NCC is a third-year doctoral student in Counseling with the Graduate School of Education and Human Development at The George Washington University (GWU). He received his MA in Clinical Mental Health Counseling with GWU and currently serves as an admissions counselor with the GWU College of Professional Studies. His broad interests revolve around multicultural counseling, multicultural supervision, and social justice. He is specifically interested in intersectionality of cultural identity, acculturative stress, intergenerational conflict, and ethnic identity development and socialization. His prior professional experiences include working as a case manager with foster care adolescents and working as an outpatient counselor, providing individual, couples, parent-child, and family counseling services. Due to his passion for mentorship, collaboration, professional identity development, and advocacy, his career goals include working as a full-time counselor education faculty while working part-time to provide counseling services in the community.

Return to Table of Contents

Tools of Evaluation, Special Issue 2015 87

2015 VECAP National Executive Council President

Catherine Burzio, MA 540.247.1888

[email protected]

President-Elect Vacant

Immediate Past-President Kathleen Kaegi, MS, CRC

804.241.7936 [email protected]

Treasurer Jen Hemme, CVE

Goodwill of Southwestern Pennsylvania 412.632.1875

[email protected]

Secretary Jenny Garifalos

Working Wonders of Maryland, LLC 443.632.5448

[email protected]

Journal Co-Editor Steven R. Sligar, EdD, CVE, PVE

East Carolina University 252.744.6293

[email protected]

Journal Co-Editor Nancy Simonds, MA

Nancy Simonds Communication, LLC 860.254.5418

[email protected]

Journal Managing Editor Vanessa M. Perry, MS, CRC

East Carolina University 252.744.6300

[email protected]

Advocacy Co-Coordinator K. Ashley McFall

Pittsburgh Public Schools 412.323.4078

[email protected]

Advocacy Co-Coordinator Crystal Evans

Pittsburgh Public Schools 412.323.3970

[email protected]

Communications Co-Coordinator Jill Flansburg, PhD

Florida TRADE St. Petersburg College

727.791.2508 [email protected]

Communications Co-Coordinator Matthew Shapiro

6 Wheels Consulting, LLC 804.317.0819

[email protected]

Education Co-Coordinator Elizabeth M. Crunkilton, MS, CRC, CRP

Division of Rehabilitative Services 540.812.5078

[email protected]

Membership Co-Coordinator Sam Castiglione

410.404.3580 [email protected]

Membership Co-Coordinator Liz Jones

New Horizons Supported Services [email protected]

Standards Co-Coordinator John Thomason

Oklahoma Department of Rehabilitation Services 405.635.2765

[email protected]

Standards Co-Coordinator Pam Leconte, EdD

The George Washington University 301.587.2370

[email protected]

Board Member at Large Jennifer Harris

Arkansas PROMISE 870.541.9482

[email protected]

Board Member at Large Cathy Cook, MS, CRC, PVE

Woodrow Wilson Rehabilitation Center 540.851.2511

[email protected]

Return to Table of Contents

Tools of Evaluation, Special Issue 2015 88

The Vocational Evaluation and Career Assessment Professionals (VECAP) is a

nonprofit organization originally founded in 1967 to promote the professions and

services of vocational evaluation and work adjustment. Formerly known as the

Vocational Evaluation and Work Adjustment Association (VEWAA), the name was

changed in 2003 to better reflect the focus of the organization as well as emphasize the

independent status of the organization. This group has no affiliation with the National

Rehabilitation Association (NRA) or the NRA/VEWAA.

The VECAP organization is committed to advance and improve the fields of

vocational evaluation and career assessment and represents the needs of the

professionals who provide those services. Its scope of services encompasses

individuals who need assistance with vocational development and/or career decision-

making.

VECAP’s  membership comprises professionals who provide vocational evaluation,

assessment, and career services and others interested in these services.

VECAP members identify, guide, and support the efforts of persons served to

develop and realize training, education, and employment plans as they work to attain

their career goals.

For manuscript and membership information visit VECAP.org.

VECAP MISSION

Tools of Evaluation, Special Issue 2015 89

2015 VEWAA Board of Directors President

Andrea Perkins Nerlich, PhD, CRC, CVE Past-President

Shawn Zimmerman, MS, CVE, CRC, PVE

Secretary Debra Homa, Ph.D., CRC, CVE, PVE

Treasurer John Lui, PhD, MBA, CRC, PVE

Board Member at Large Juliet H. Fried, EdD, CRC, CVE, PVE

Board Member at Large Paige Tidwell, CRC, CVE, PVE

Board Member at Large Priscilla Matthews, MS, CRC, PVE

Board Member at Large Cindy Larson, MS, CRC, PVE, CBIS

Committee Chair: Membership Paige Tidwell, MRC, CRC, CVE

Committee Chair: Student Representative Sabrina Harris, MS

Committee Chair: Representative to the NRA Board Jean E. Johnson, EdD, CRC

Committee Chair: Publications Andrea Perkins Nerlich, PhD, CRC, CVE

Committee Chair: Publications Randall S. McDaniel, EdD, CRC

Committee Chair: Website Anthony J. Langton, MS, CVE

Return to Table of Contents

Tools of Evaluation, Special Issue 2015 90

The Vocational Evaluation and Work Adjustment Association (VEWAA)

represents the interests and needs of human service professionals who work with

people in support of their endeavors in training, employment, and education to enhance

successful attainment of career goals. VEWAA has a diverse membership that provides

services to individuals who need assistance with vocational development and or career

decision making, are involved in education and training of professionals entering the

field, and conduct research and policy development that enhance the lives of persons

with disabilities. Individual members provide services within a variety of settings such

as community rehabilitation programs, schools, state and federal agencies, universities,

the private sector and directly in business and industry.

The mission of VEWAA remains to improve and advance the fields of

vocational evaluation and work readiness and it is committed to representing the needs

of all professionals who provide related services. Through training, education, and

communications, VEWAA seeks to provide all members with access to professional

growth opportunities, to respond to current issues, and to monitor the development of

public policy and workforce trends at the national, state, and local levels.

For manuscript and membership information visit VEWAA.com.

VEWAA MISSION

Tools of Evaluation, Special Issue 2015 91