FOR BUSINESS INTELLIGENCE
ozaiqii
factor_analysis.ppt9. Factor Analysis
*
Factor Analysis
- Factor analysis is a general name denoting a class of procedures primarily used for data reduction and summarization.
- In regression, DA, and ANOVA, one variable is condisered as DV, and the other variables as IVs. However, no such distinction is made in factor analysis.
- In factor analysis (FA) an entire set of interdependent relationships is examined without making the distinction between dependent and independent variables.
- Factor analysis is used in the following circumstances:
- To identify underlying dimensions, or factors, that explain the correlations among a set of variables.
- To identify a new, smaller, set of uncorrelated variables to replace the original set of correlated variables in subsequent multivariate analysis (regression or discriminant analysis).
- To identify a smaller set of salient variables from a larger set for use in subsequent multivariate analysis.
FA application in marketing research
- Used in market segmentation for identifying the underlying variables on which to group the customers. New car buyers might be grouped based on the relative emphasis they place on economy, convenience, performance, comfort, and luxury. This might result in five segments economy seekers, convenience seeker…
- In pricing studies, FA can be used to identify the characteristics of price-sensitive consumers. For example, these consumers might be methodical, economy minded, and home centered.
Explain loadings and communalities
- Communalities are related to loadings. The sum of squares for a row in the loading matrix is equal to the communality of a variable.
- Factor loadings. Factor loadings are simple correlations between the variables and the factors.
- Factor matrix. A factor matrix contains the factor loadings of all the variables on all the factors extracted.
- Communality. The proportion of variance explained by the common factors.
Statistics Associated with Factor Analysis
*
Explain loadings and communalities
- Loadings are the correlations between a factor (columns) and a variable (rows). For the purpose of interpretation, loading matrix is the most important output of factor analysis. Analogous to Pearson’s r, the squared factor loading is the percent of variance in that variable explained by the factor. For the purpose of interpretation, loadings are most important output for factor analysis.
- Communality (h2) represents the proportion of the variance of each variable that is explained by the selected factors. In another term, it is how much an item is shared among factors.
Statistics Associated with Factor Analysis
- Bartlett's test of sphericity. Bartlett's test of sphericity is a test statistic used to examine the hypothesis that the variables are uncorrelated in the population. In other words, the population correlation matrix is an identity matrix; each variable correlates perfectly with itself (r = 1) but has no correlation with the other variables (r = 0).
- Correlation matrix. A correlation matrix is a lower triangle matrix showing the simple correlations, r, between all possible pairs of variables included in the analysis. The diagonal elements, which are all 1, are usually omitted.
- Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy is an index used to examine the appropriateness of factor analysis. High values (between 0.5 and 1.0) indicate factor analysis is appropriate. Values below 0.5 imply that factor analysis may not be appropriate.
- Factor scores. Factor scores are composite scores estimated for each respondent on the derived factors.
- Eigenvalue. Column sum of squared loadings for a factor; also referred to as the latent root. It conceptually represents that amount of variance accounted for by a factor.
- Scree plot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction.
Statistics Associated with Factor Analysis
Scree Plot
0.5
2
5
4
3
6
Component Number
0.0
2.0
3.0
Eigenvalue
1.0
1.5
2.5
1
- In principal components analysis, the total variance in the data is considered. The diagonal of the correlation matrix consists of unities, and full variance is brought into the factor matrix. Principal components analysis is recommended when the primary concern is to determine the minimum number of factors that will account for maximum variance in the data for use in subsequent multivariate analysis. The factors are called principal components.
- In common factor analysis, the factors are estimated based only on the common variance. Communalities are inserted in the diagonal of the correlation matrix. This method is appropriate when the primary concern is to identify the underlying dimensions and the common variance is of interest. This method is also known as principal axis factoring.
Conducting Factor Analysis
Determine the Method of Factor Analysis
Difference Between PAF and PCA
- Principal components factor analysis inserts 1's on the diagonal of the correlation matrix, thus considering all of the available variance.
- Most appropriate when the concern is with deriving a minimum number of factors to explain a maximum portion of variance in the original variables, and the researcher knows the specific and error variances are small.
- Common factor analysis only uses the common variance and places communality estimates on the diagonal of the correlation matrix.
- Most appropriate when there is a desire to reveal latent dimensions of the original variables and the researcher does not know about the nature of specific and error variance.
Factor Analysis example: toothpaste attribute ratings
Sheet17
| RESPONDENT NUMBER | V1 | V2 | V3 | V5 | V6 | |
| 1 | 7.00 | 3.00 | 6.00 | 4.00 | 2.00 | 4.00 |
| 2 | 1.00 | 3.00 | 2.00 | 4.00 | 5.00 | 4.00 |
| 3 | 6.00 | 2.00 | 7.00 | 4.00 | 1.00 | 3.00 |
| 4 | 4.00 | 5.00 | 4.00 | 6.00 | 2.00 | 5.00 |
| 5 | 1.00 | 2.00 | 2.00 | 3.00 | 6.00 | 2.00 |
| 6 | 6.00 | 3.00 | 6.00 | 4.00 | 2.00 | 4.00 |
| 7 | 5.00 | 3.00 | 6.00 | 3.00 | 4.00 | 3.00 |
| 8 | 6.00 | 4.00 | 7.00 | 4.00 | 1.00 | 4.00 |
| 9 | 3.00 | 4.00 | 2.00 | 3.00 | 6.00 | 3.00 |
| 10 | 2.00 | 6.00 | 2.00 | 6.00 | 7.00 | 6.00 |
| 11 | 6.00 | 4.00 | 7.00 | 3.00 | 2.00 | 3.00 |
| 12 | 2.00 | 3.00 | 1.00 | 4.00 | 5.00 | 4.00 |
| 13 | 7.00 | 2.00 | 6.00 | 4.00 | 1.00 | 3.00 |
| 14 | 4.00 | 6.00 | 4.00 | 5.00 | 3.00 | 6.00 |
| 15 | 1.00 | 3.00 | 2.00 | 2.00 | 6.00 | 4.00 |
| 16 | 6.00 | 4.00 | 6.00 | 3.00 | 3.00 | 4.00 |
| 17 | 5.00 | 3.00 | 6.00 | 3.00 | 3.00 | 4.00 |
| 18 | 7.00 | 3.00 | 7.00 | 4.00 | 1.00 | 4.00 |
| 19 | 2.00 | 4.00 | 3.00 | 3.00 | 6.00 | 3.00 |
| 20 | 3.00 | 5.00 | 3.00 | 6.00 | 4.00 | 6.00 |
| 21 | 1.00 | 3.00 | 2.00 | 3.00 | 5.00 | 3.00 |
| 22 | 5.00 | 4.00 | 5.00 | 4.00 | 2.00 | 4.00 |
| 23 | 2.00 | 2.00 | 1.00 | 5.00 | 4.00 | 4.00 |
| 24 | 4.00 | 6.00 | 4.00 | 6.00 | 4.00 | 7.00 |
| 25 | 6.00 | 5.00 | 4.00 | 2.00 | 1.00 | 4.00 |
| 26 | 3.00 | 5.00 | 4.00 | 6.00 | 4.00 | 7.00 |
| 27 | 4.00 | 4.00 | 7.00 | 2.00 | 2.00 | 5.00 |
| 28 | 3.00 | 7.00 | 2.00 | 6.00 | 4.00 | 3.00 |
| 29 | 4.00 | 6.00 | 3.00 | 7.00 | 2.00 | 7.00 |
| 30 | 2.00 | 3.00 | 2.00 | 4.00 | 7.00 | 2.00 |
Sheet1 (2)
| RESPONDENT | V1 | V2 | V3 | V5 | V6 | |
| NUMBER | ||||||
| 1 | 7.00 | 3.00 | 6.00 | 4.00 | 2.00 | 4.00 |
| 2 | 1.00 | 3.00 | 2.00 | 4.00 | 5.00 | 4.00 |
| 3 | 6.00 | 2.00 | 7.00 | 4.00 | 1.00 | 3.00 |
| 4 | 4.00 | 5.00 | 4.00 | 6.00 | 2.00 | 5.00 |
| 5 | 1.00 | 2.00 | 2.00 | 3.00 | 6.00 | 2.00 |
| 6 | 6.00 | 3.00 | 6.00 | 4.00 | 2.00 | 4.00 |
| 7 | 5.00 | 3.00 | 6.00 | 3.00 | 4.00 | 3.00 |
| 8 | 6.00 | 4.00 | 7.00 | 4.00 | 1.00 | 4.00 |
| 9 | 3.00 | 4.00 | 2.00 | 3.00 | 6.00 | 3.00 |
| 10 | 2.00 | 6.00 | 2.00 | 6.00 | 7.00 | 6.00 |
| 11 | 6.00 | 4.00 | 7.00 | 3.00 | 2.00 | 3.00 |
| 12 | 2.00 | 3.00 | 1.00 | 4.00 | 5.00 | 4.00 |
| 13 | 7.00 | 2.00 | 6.00 | 4.00 | 1.00 | 3.00 |
| 14 | 4.00 | 6.00 | 4.00 | 5.00 | 3.00 | 6.00 |
| 15 | 1.00 | 3.00 | 2.00 | 2.00 | 6.00 | 4.00 |
| 16 | 6.00 | 4.00 | 6.00 | 3.00 | 3.00 | 4.00 |
| 17 | 5.00 | 3.00 | 6.00 | 3.00 | 3.00 | 4.00 |
| 18 | 7.00 | 3.00 | 7.00 | 4.00 | 1.00 | 4.00 |
| 19 | 2.00 | 4.00 | 3.00 | 3.00 | 6.00 | 3.00 |
| 20 | 3.00 | 5.00 | 3.00 | 6.00 | 4.00 | 6.00 |
| 21 | 1.00 | 3.00 | 2.00 | 3.00 | 5.00 | 3.00 |
| 22 | 5.00 | 4.00 | 5.00 | 4.00 | 2.00 | 4.00 |
| 23 | 2.00 | 2.00 | 1.00 | 5.00 | 4.00 | 4.00 |
| 24 | 4.00 | 6.00 | 4.00 | 6.00 | 4.00 | 7.00 |
| 25 | 6.00 | 5.00 | 4.00 | 2.00 | 1.00 | 4.00 |
| 26 | 3.00 | 5.00 | 4.00 | 6.00 | 4.00 | 7.00 |
| 27 | 4.00 | 4.00 | 7.00 | 2.00 | 2.00 | 5.00 |
| 28 | 3.00 | 7.00 | 2.00 | 6.00 | 4.00 | 3.00 |
| 29 | 4.00 | 6.00 | 3.00 | 7.00 | 2.00 | 7.00 |
| 30 | 2.00 | 3.00 | 2.00 | 4.00 | 7.00 | 2.00 |
Sheet1
| TABLE 19.1 | ||||||
| TOOTHPASTE ATTRIBUTE RATINGS | ||||||
| RESPONDENT | V1 | V2 | V3 | V5 | V6 | |
| NUMBER | ||||||
| 1 | 7.00 | 3.00 | 6.00 | 4.00 | 2.00 | 4.00 |
| 2 | 1.00 | 3.00 | 2.00 | 4.00 | 5.00 | 4.00 |
| 3 | 6.00 | 2.00 | 7.00 | 4.00 | 1.00 | 3.00 |
| 4 | 4.00 | 5.00 | 4.00 | 6.00 | 2.00 | 5.00 |
| 5 | 1.00 | 2.00 | 2.00 | 3.00 | 6.00 | 2.00 |
| 6 | 6.00 | 3.00 | 6.00 | 4.00 | 2.00 | 4.00 |
| 7 | 5.00 | 3.00 | 6.00 | 3.00 | 4.00 | 3.00 |
| 8 | 6.00 | 4.00 | 7.00 | 4.00 | 1.00 | 4.00 |
| 9 | 3.00 | 4.00 | 2.00 | 3.00 | 6.00 | 3.00 |
| 10 | 2.00 | 6.00 | 2.00 | 6.00 | 7.00 | 6.00 |
| 11 | 6.00 | 4.00 | 7.00 | 3.00 | 2.00 | 3.00 |
| 12 | 2.00 | 3.00 | 1.00 | 4.00 | 5.00 | 4.00 |
| 13 | 7.00 | 2.00 | 6.00 | 4.00 | 1.00 | 3.00 |
| 14 | 4.00 | 6.00 | 4.00 | 5.00 | 3.00 | 6.00 |
| 15 | 1.00 | 3.00 | 2.00 | 2.00 | 6.00 | 4.00 |
| 16 | 6.00 | 4.00 | 6.00 | 3.00 | 3.00 | 4.00 |
| 17 | 5.00 | 3.00 | 6.00 | 3.00 | 3.00 | 4.00 |
| 18 | 7.00 | 3.00 | 7.00 | 4.00 | 1.00 | 4.00 |
| 19 | 2.00 | 4.00 | 3.00 | 3.00 | 6.00 | 3.00 |
| 20 | 3.00 | 5.00 | 3.00 | 6.00 | 4.00 | 6.00 |
| 21 | 1.00 | 3.00 | 2.00 | 3.00 | 5.00 | 3.00 |
| 22 | 5.00 | 4.00 | 5.00 | 4.00 | 2.00 | 4.00 |
| 23 | 2.00 | 2.00 | 1.00 | 5.00 | 4.00 | 4.00 |
| 24 | 4.00 | 6.00 | 4.00 | 6.00 | 4.00 | 7.00 |
| 25 | 6.00 | 5.00 | 4.00 | 2.00 | 1.00 | 4.00 |
| 26 | 3.00 | 5.00 | 4.00 | 6.00 | 4.00 | 7.00 |
| 27 | 4.00 | 4.00 | 7.00 | 2.00 | 2.00 | 5.00 |
| 28 | 3.00 | 7.00 | 2.00 | 6.00 | 4.00 | 3.00 |
| 29 | 4.00 | 6.00 | 3.00 | 7.00 | 2.00 | 7.00 |
| 30 | 2.00 | 3.00 | 2.00 | 4.00 | 7.00 | 2.00 |
Sheet2
Sheet3
Sheet4
Sheet5
Sheet6
Sheet7
Sheet8
Sheet9
Sheet10
Sheet11
Sheet12
Sheet13
Sheet14
Sheet15
Sheet16
Example: toothpaste ratings
Suppose the researcher wants to determine the underlying benefits consumers seek from the purchase of a toothpaste. A sample of 30 respondents were interviewed using mall-intercept method. The respondents were asked to indicate their degree of agreement with the following statement using a 7-point scale (1=strongly disagree, 7=strongly agree).
- V1: it is important to buy a toothpaste that prevents cavities
- V2: I like a toothpaste that gives shiny teeth
- V3: A toothpaste should strengthen your gums
- V4: I prefer a toothpaste that freshens breath
- V5: Prevention of tooth decay is not an important benefit offered by a toothpaste.
- V6: The most important consideration in buying a toothpaste is attractive teeth.
- The analytical process is based on a matrix of correlations between the variables.
- For the FA to be appropriate, the variables must be correlated.
- Bartlett's test of sphericity can be used to test the null hypothesis that the variables are uncorrelated in the population. If this hypothesis cannot be rejected, then the appropriateness of factor analysis should be questioned.
- Another useful statistic is the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. Small values of the KMO statistic indicate that the correlations between pairs of variables cannot be explained by other variables and that factor analysis may not be appropriate. Generally, a value greater than 0.5 is desirable.
Conducting Factor Analysis
Construct the Correlation Matrix
Correlation Matrix
Results of PCA
The null hypothesis, that the population correlation matrix is an identity matrix (not correlated), is rejected by the Bartlett’s test of sphericity. The chi-square statistic is 111.314 with 15df, which is significant at the 0.05 level (p<0.001).
The value of KMO statistic (0.660) is large (>0.5). Thus, factor analysis is considered an appropriate technique for this study.
The eigenvalue of factor 1 (F1) is 2.731, and it accounts for a variance of 45.52% of the total variance. The second factor (F2) has an eigenvalue of 2.218, which is 36.96% of the total variance. The first two factors combined account for 82.49% of the total variance.
The communality table gives relevant information after the desired number of factors have been extracted.
results
From the scree plot, the first two components have eigenvalues greater than 1.
- A Priori Determination. Sometimes, because of prior knowledge, the researcher knows how many factors to expect and thus can specify the number of factors to be extracted beforehand.
- Determination Based on Eigenvalues. In this approach, only factors with Eigenvalues greater than 1.0 are retained. An Eigenvalue represents the amount of variance associated with the factor. Hence, only factors with a variance greater than 1.0 are included. Factors with variance less than 1.0 are no better than a single variable, since, due to standardization, each variable has a variance of 1.0. If the number of variables is less than 20, this approach will result in a conservative number of factors.
Conducting Factor Analysis
Determine the Number of Factors
- Determination Based on Scree Plot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction. Experimental evidence indicates that the point at which the scree begins denotes the true number of factors. Generally, the number of factors determined by a scree plot will be one or a few more than that determined by the Eigenvalue criterion.
- Determination Based on Percentage of Variance. In this approach the number of factors extracted is determined so that the cumulative percentage of variance extracted by the factors reaches a satisfactory level. It is recommended that the factors extracted should account for at least 60% of the variance.
Conducting Factor Analysis
Determine the Number of Factors
Results
The component matrix shows that some variables has cross loading (absolute value of factor loading greater than 0.3) on component 1 and component 2. In such a complex matrix, it is difficult to interpret. Through rotation, the component matrix is transformed into a simpler one that is easier to interpret.
In the rotated component matrix table, F1 has high coefficients for V1 (prevention of cavities) and V3 (strong gum), and a negative coefficient for V5 (prevention of tooth decay is not important). Therefore, this factor will be labeled a health benefit factor. Note the negative coefficient for a negative variable (V5) leads to a positive interpretation that prevention of tooth decay is important.
F2 is highly related with V2 (shiny teeth), V4 (fresh breath), and V6 (attractive teeth). Thus F2 may be labaled a social benefit factor.
Factor Matrix Before and After Rotation
Factors/component
Variables
1
2
3
4
5
6
1
X
X
X
X
X
2
X
X
X
X
1
X
X
X
2
X
X
X
Variables
1
2
3
4
5
6
Factors/component
(a)
High Loadings
Before Rotation
(b)
High Loadings
After Rotation
- Although the initial or unrotated factor matrix indicates the relationship between the factors and individual variables, it seldom results in factors that can be interpreted, because the factors are correlated with many variables. Therefore, through rotation the factor matrix (or component matrix) is transformed into a simpler one that is easier to interpret.
- The rotation is called orthogonal rotation ant, if the axes are maintained at right angles. The most commonly used method for rotation is the varimax procedure. This is an orthogonal method of rotation that minimizes the number of variables with high loadings on a factor, thereby enhancing the interpretability of the factors. Orthogonal rotation results in factors that are uncorrelated.
- The rotation is called oblique rotation when the axes are not maintained at right angles, and the factors are correlated. Sometimes, allowing for correlations among factors can simplify the factor pattern matrix. Oblique rotation should be used when factors in the population are likely to be strongly correlated.
Conducting Factor Analysis
Rotate Factors
Results
In principal component analysis, component scores are uncorrelated. In common factor analysis, estimates of these scores are obtained, and there is no guarantee that the factors will be uncorrelated with each other.
Fi = Wi1X1 + Wi2X2 + Wi3X3 + . . . + WikXk
The weights, or component/factor score coefficient, are obtained from the component/factor score coefficients matrix. Using the component score coefficient matrix, one could compute two component score for each respondents.
- A factor can then be interpreted in terms of the variables that load high on it.
- Another useful aid in interpretation is to plot the variables, using the factor loadings as coordinates. Variables at the end of an axis are those that have high loadings on only that factor, and hence describe the factor.
Conducting Factor Analysis
Interpret Factors
Factor Loading Plot
- The correlations between the variables can be deduced or reproduced from the estimated correlations between the variables and the factors.
- The differences between the observed correlations (as given in the input correlation matrix) and the reproduced correlations (as estimated from the factor matrix) can be examined to determine model fit. These differences are called residuals.
Conducting Factor Analysis
Determine the Model Fit
Determine the model fit
The final step is to determine the model fit.
A basic assumption underlying FA is that the observed correlation between variables can be attributed to common factors. Hence, the correlations between the variables can be reproduced from the estimated correlations between the variables and the factors. The difference between the observed correlations (as given in the input correlation matrix) and the reproduced correlations (as estimated from the factor matrix) can be examined to determine model fit. These differences are called residuals. If there are many large residuals, the model does not provide a good it. The following table only have five residuals greater than 0.05, indicating an acceptable model fit.
Results of Common Factor Analysis
Results of Common Factor Analysis
Results of Common Factor Analysis
SPSS Windows
To select this procedure using SPSS for Windows click:
Analyze>Data Reduction>Factor …
RESPONDENT
NUMBER V1 V2 V3
V4 V5 V6
1
7.003.006.004.002.004.00
2
1.003.002.004.005.004.00
3
6.002.007.004.001.003.00
4
4.005.004.006.002.005.00
5
1.002.002.003.006.002.00
6
6.003.006.004.002.004.00
7
5.003.006.003.004.003.00
8
6.004.007.004.001.004.00
9
3.004.002.003.006.003.00
10
2.006.002.006.007.006.00
11
6.004.007.003.002.003.00
12
2.003.001.004.005.004.00
13
7.002.006.004.001.003.00
14
4.006.004.005.003.006.00
15
1.003.002.002.006.004.00
16
6.004.006.003.003.004.00
17
5.003.006.003.003.004.00
18
7.003.007.004.001.004.00
19
2.004.003.003.006.003.00
20
3.005.003.006.004.006.00
21
1.003.002.003.005.003.00
22
5.004.005.004.002.004.00
23
2.002.001.005.004.004.00
24
4.006.004.006.004.007.00
25
6.005.004.002.001.004.00
26
3.005.004.006.004.007.00
27
4.004.007.002.002.005.00
28
3.007.002.006.004.003.00
29
4.006.003.007.002.007.00
30
2.003.002.004.007.002.00
