PADM530Wk2@

GettyImages

Comparative analysis is a good tool to have in one’s toolbox, but a good analyst will use multiple analytical tools to complement each other. Comparative analysis is in some ways a very basic concept – compare the results of different policies to see how they differ. However, it can be challenging to use effectively in practice. It should not be confused with a mere list of how two times or places differ (or do not differ).

Policy analysis can be used at any level of government. There are a number of ways to analyze policy. Usually, analysts use more than one method to give a better over-all idea of how well the policy serves the citizens. When doing policy analysis, the wise

Comparative Analysis--Selecting the Correct Things to Compare

Elizabeth Keavney, Ph.D. Professor at American Public University System Published Apr 21, 2020

+ Follow

analyst will work to examine all sides of the issue and carefully include long-term consequences and effects as well as short-term consequences and effects.

The greatest challenge in comparative analysis is finding jurisdictions or time periods to compare that differ much more in the policy of interest than in any other variable that might affect the outcome of interest, such as demographics, culture, or economic conditions. Most outcomes of interest can be affected by many variables, and many variables differ significantly from place to place and time to time.

Once the analyst begins, he or she might examine how effective the policy is—that is, how well it meets its objectives. The problem with this is that not all those who write policy put a clear and specific purpose in the policy. The analyst might do a cost- benefit analysis. This method of analysis is frequently used in order to ensure the policy is worth the cost and will help determine its efficiency. Another method of analysis to include would be comparative analysis. It compares two or more jurisdictions or times that are similar but differ in the policy of interest. It doesn’t just list ways in which they are alike and ways in which they are different. It focuses on policy outcomes or results.

Limits on the applicability of Comparative Analysis

 There is no way for a researcher to prove that he or she has captured all the relevant variables (this would be proving a negative – a logical impossibility), so he or she must rely on expert judgement from experience and from the literature to capture as many as practical. The better one matches the variables of greatest relevance, the stronger one’s conclusions will be.

This example will use countries to show the importance of selecting the correct things for comparison. Remember that the same concepts apply for other comparisons.

Figure 1 shows two countries dissimilar in many respects. The large population difference may be overlooked for many purposes, since many variables are measured on a per capita basis. However, as you can see, the other variables shown here also have substantial differences. Thus, India and the United States are not a good choice for most comparative analysis purposes. For example, health care outcomes may be strongly influenced by at least three of the four variables chosen to illustrate this point, and by other variables not shown in the chart. The effects of these variables are likely to be much greater than the effects of any policy differences, so separating differences

caused by the policy choices from differences caused by the other variables is essentially impossible.

Figure 1. Comparison of Dissimilar Countries

Figure 1 sources: Population data from One World – Nations Online (2020), per capita income data from U.S. Central Intelligence Agency (2020), Diversity score data from Fearon (2003), and age distribution data from United Nations (2019).

 Choosing a developed western nation for such an analysis would not be as obviously flawed an approach, but still would still require attention to the differences in relevant variables. Figure 2 shows some of the relevant variables for the United States and Australia, Canada, Norway and the United Kingdom. Note that the portions of the populations less than 40 years old, though not dramatically different, could differ enough to cause statistically significant variations in frequencies of certain diseases or recovery times from injuries. A thorough analysis would require more detailed age distribution information than the single number used to illustrate the point here. Note

also that the level of diversity is significantly less than in the United States in every country listed except Canada, and ethnic diversity could affect the occurrence of certain genetically linked illnesses. Note also that there are probably other relevant variables not included in this illustrative example.

Figure 2. Comparison of the U.S. to Selected Western Nations

Figure 2 sources: Population data from One World – Nations Online (2020), per capita income data from U.S. Central Intelligence Agency (2020), diversity score data from Fearon (2003), and age distribution data from United Nations (2019).

The real world is a messy place, and it may be difficult to find excellent cases for comparative analysis, even with an extensive search. This may mean deeper analysis (analyzing more cases), using other methods instead of or in addition to comparative analysis, or tolerating less certainty in one’s conclusions. Comparative analysis can be a very useful tool, but using it without understanding its limits is dangerous.

Although perfectly matching all relevant variables except the one under investigation is never possible, this does not mean that comparative analysis is not a useful tool. Instead, it means that it must be employed with proper attention to how closely all the variables relevant to the outcome of interest match, and that one must carefully consider any differences in those when drawing conclusions based on this type of analysis.

References

 Fearon, J.D. (2003). Ethnic and cultural diversity. Journal of Economic Growth. 8(2), 195- 222, Retrieved from http://www.jstor.org/stable/40215943 .

One World - Nations Online. (2020). Countries of the World by Population. Retrieved from https://www.nationsonline.org/oneworld/population-by-country.htm

United Nations. (2019) Population by age, sex and urban/rural residence. Retrieved from ttps://data.un.org/Data.aspx?d=POP&f=tableCode%3a22

United States Central Intelligence Agency. (2020). The World Factbook. Retrieved from https://www.cia.gov/library/publications/the-world- factbook/rankorder/2004rank.html

 

 

 

 

 

Vincente Carreon Outreach Specialist @ Federal Energy Regulatory Commission

2mo

1 CommentLike Comment Share

To view or add a comment, sign in

Insights from the community

Marketing Research

How can you design a survey for cluster analysis?

Like · Reply

Good read, Dr. K!

More articles by this author

When Policy and Basic Economics Collide Dec 17, 2022

Paying the Bill Jun 19, 2021

No Majority in Electoral College—What Happens? Nov 20, 2020

See all

Market Research

How can you tell primary and secondary market research apart?

Market Research

What are the advantages of using a Likert scale in market research?

Survey Research

How do you report and communicate your survey findings using random sampling?

Market Research

What are the most effective skip patterns to use in a market research questionnaire?

Content Creation

What are the most effective ways to analyze and present survey data?

Show more

Explore topics Sales

Marketing

Business Administration

HR Management

Content Management

Engineering

Soft Skills

See All

© 2023 About

Accessibility User Agreement

Privacy Policy Your California Privacy Choices

Cookie Policy Copyright Policy

Brand Policy Guest Controls

Community Guidelines Language