interaction effects between endogenous variables
Source:vignettes/interaction_two_etas.Rmd
interaction_two_etas.RmdThe Problem
Interaction effects between two endogenous (i.e., dependent)
variables work as you would expect for the product indicator methods
("dblcent", "rca", "ca", "uca"). For the lms- and qml
approach however, it is not as straight forward.
The lms- and qml approach can (by default) handle interaction effects between endogenous and exogenous (i.e., independent) variables, but not interaction effects between two endogenous variables. When there is an interaction effect between two endogenous variables, the equations cannot easily be written in ‘reduced’ form – meaning that normal estimation procedures won’t work.
The Solution
This being said, there is a work-around for these limitations for both the lms- and qml-approach. In essence, the model can be split into two parts, one linear and one non-linear. Basically, you can replace the covariance matrix used in the estimation of the non-linear model, with the model-implied covariance matrix from a linear model. Thus you can treat an endogenous variable as if it were exogenous – given that it can be expressed in a linear model.
Example
Let’s consider the the theory of planned behaviour (TPB) where we wish to estimate the quadratic effect of INT on BEH (INT:INT). With the following model:
tpb <- '
# Outer Model (Based on Hagger et al., 2007)
ATT =~ att1 + att2 + att3 + att4 + att5
SN =~ sn1 + sn2
PBC =~ pbc1 + pbc2 + pbc3
INT =~ int1 + int2 + int3
BEH =~ b1 + b2
# Inner Model (Based on Steinmetz et al., 2011)
INT ~ ATT + SN + PBC
BEH ~ INT + PBC
BEH ~ INT:INT
'Since INT is an endogenous variable, its quadratic term (i.e., an interaction effect with itself) would include two endogenous variables. Thus we would ordinarily not be able to estimate this model using the lms- or qml-approach. However, we can split the model into two parts, one linear and one non-linear. While INT is an endogenous variable, it can be expressed in a linear model – since it is not affected by any interaction terms:
tpb_linear <- 'INT ~ PBC + ATT + SN'We could then remove this part from the original model, giving us:
tpb_nonlinear <- '
# Outer Model (Based on Hagger et al., 2007)
ATT =~ att1 + att2 + att3 + att4 + att5
SN =~ sn1 + sn2
PBC =~ pbc1 + pbc2 + pbc3
INT =~ int1 + int2 + int3
BEH =~ b1 + b2
# Inner Model (Based on Steinmetz et al., 2011)
BEH ~ INT + PBC
BEH ~ INT:INT
'We could now just estimate the non-linear model, since INT now is an
exogenous variable. This would however not incorporate the structural
model for INT. To address this, we can make modsem replace the
covariance matrix (phi) of (INT, PBC, ATT, SN) with the model-implied
covariance matrix from the linear model, whilst estimating both models
simultaneously. To acheive this, we can use the cov.syntax
argument in modsem:
est_lms <- modsem(tpb_nonlinear, data = TPB, cov.syntax = tpb_linear, method = "lms")
#> Warning: It is recommended that you have at least 48 nodes for interaction
#> effects between endogenous variables in the lms approach 'nodes = 24'
summary(est_lms)
#> Estimating null model
#> EM: Iteration = 1, LogLik = -28467.33, Change = -28467.332
#> EM: Iteration = 2, LogLik = -28124.48, Change = 342.852
#> EM: Iteration = 3, LogLik = -27825.10, Change = 299.377
#> EM: Iteration = 4, LogLik = -27581.12, Change = 243.980
#> EM: Iteration = 5, LogLik = -27370.69, Change = 210.431
#> EM: Iteration = 6, LogLik = -27175.43, Change = 195.264
#> EM: Iteration = 7, LogLik = -27000.48, Change = 174.946
#> EM: Iteration = 8, LogLik = -26848.56, Change = 151.919
#> EM: Iteration = 9, LogLik = -26711.51, Change = 137.051
#> EM: Iteration = 10, LogLik = -26592.54, Change = 118.968
#> EM: Iteration = 11, LogLik = -26504.04, Change = 88.504
#> EM: Iteration = 12, LogLik = -26466.85, Change = 37.190
#> EM: Iteration = 13, LogLik = -26452.38, Change = 14.465
#> EM: Iteration = 14, LogLik = -26439.05, Change = 13.331
#> EM: Iteration = 15, LogLik = -26430.51, Change = 8.541
#> EM: Iteration = 16, LogLik = -26422.81, Change = 7.698
#> EM: Iteration = 17, LogLik = -26403.89, Change = 18.924
#> EM: Iteration = 18, LogLik = -26402.25, Change = 1.642
#> EM: Iteration = 19, LogLik = -26401.21, Change = 1.037
#> EM: Iteration = 20, LogLik = -26400.26, Change = 0.955
#> EM: Iteration = 21, LogLik = -26399.30, Change = 0.960
#> EM: Iteration = 22, LogLik = -26398.64, Change = 0.658
#> EM: Iteration = 23, LogLik = -26398.02, Change = 0.615
#> EM: Iteration = 24, LogLik = -26397.74, Change = 0.278
#> EM: Iteration = 25, LogLik = -26397.33, Change = 0.420
#> EM: Iteration = 26, LogLik = -26397.20, Change = 0.128
#> EM: Iteration = 27, LogLik = -26396.77, Change = 0.425
#> EM: Iteration = 28, LogLik = -26396.48, Change = 0.292
#> EM: Iteration = 29, LogLik = -26396.34, Change = 0.145
#> EM: Iteration = 30, LogLik = -26396.31, Change = 0.022
#> EM: Iteration = 31, LogLik = -26395.95, Change = 0.365
#> EM: Iteration = 32, LogLik = -26395.73, Change = 0.215
#> EM: Iteration = 33, LogLik = -26395.68, Change = 0.055
#> EM: Iteration = 34, LogLik = -26395.37, Change = 0.304
#> EM: Iteration = 35, LogLik = -26395.30, Change = 0.079
#> EM: Iteration = 36, LogLik = -26395.10, Change = 0.198
#> EM: Iteration = 37, LogLik = -26395.04, Change = 0.057
#> EM: Iteration = 38, LogLik = -26394.99, Change = 0.054
#> EM: Iteration = 39, LogLik = -26394.96, Change = 0.028
#> EM: Iteration = 40, LogLik = -26394.93, Change = 0.031
#> EM: Iteration = 41, LogLik = -26394.91, Change = 0.019
#> EM: Iteration = 42, LogLik = -26394.89, Change = 0.023
#> EM: Iteration = 43, LogLik = -26394.87, Change = 0.015
#> EM: Iteration = 44, LogLik = -26394.85, Change = 0.019
#> EM: Iteration = 45, LogLik = -26394.84, Change = 0.013
#> EM: Iteration = 46, LogLik = -26394.82, Change = 0.018
#> EM: Iteration = 47, LogLik = -26394.81, Change = 0.012
#> EM: Iteration = 48, LogLik = -26394.79, Change = 0.018
#> EM: Iteration = 49, LogLik = -26394.78, Change = 0.013
#> EM: Iteration = 50, LogLik = -26394.76, Change = 0.020
#> EM: Iteration = 51, LogLik = -26394.74, Change = 0.015
#> EM: Iteration = 52, LogLik = -26394.72, Change = 0.028
#> EM: Iteration = 53, LogLik = -26394.69, Change = 0.022
#> EM: Iteration = 54, LogLik = -26394.63, Change = 0.062
#> EM: Iteration = 55, LogLik = -26394.58, Change = 0.057
#> EM: Iteration = 56, LogLik = -26394.29, Change = 0.284
#> EM: Iteration = 57, LogLik = -26394.04, Change = 0.248
#> EM: Iteration = 58, LogLik = -26393.97, Change = 0.075
#> EM: Iteration = 59, LogLik = -26393.73, Change = 0.240
#> EM: Iteration = 60, LogLik = -26393.72, Change = 0.011
#> EM: Iteration = 61, LogLik = -26393.71, Change = 0.013
#> EM: Iteration = 62, LogLik = -26393.70, Change = 0.005
#> EM: Iteration = 63, LogLik = -26393.69, Change = 0.008
#> EM: Iteration = 64, LogLik = -26393.69, Change = 0.003
#> EM: Iteration = 65, LogLik = -26393.68, Change = 0.007
#> EM: Iteration = 66, LogLik = -26393.68, Change = 0.003
#> EM: Iteration = 67, LogLik = -26393.67, Change = 0.007
#> EM: Iteration = 68, LogLik = -26393.67, Change = 0.003
#> EM: Iteration = 69, LogLik = -26393.66, Change = 0.006
#> EM: Iteration = 70, LogLik = -26393.66, Change = 0.002
#> EM: Iteration = 71, LogLik = -26393.66, Change = 0.006
#> EM: Iteration = 72, LogLik = -26393.65, Change = 0.002
#> EM: Iteration = 73, LogLik = -26393.65, Change = 0.005
#> EM: Iteration = 74, LogLik = -26393.65, Change = 0.002
#> EM: Iteration = 75, LogLik = -26393.64, Change = 0.005
#> EM: Iteration = 76, LogLik = -26393.64, Change = 0.002
#> EM: Iteration = 77, LogLik = -26393.63, Change = 0.004
#> EM: Iteration = 78, LogLik = -26393.63, Change = 0.002
#> EM: Iteration = 79, LogLik = -26393.63, Change = 0.004
#> EM: Iteration = 80, LogLik = -26393.63, Change = 0.002
#> EM: Iteration = 81, LogLik = -26393.62, Change = 0.004
#> EM: Iteration = 82, LogLik = -26393.62, Change = 0.002
#> EM: Iteration = 83, LogLik = -26393.62, Change = 0.004
#> EM: Iteration = 84, LogLik = -26393.61, Change = 0.002
#> EM: Iteration = 85, LogLik = -26393.61, Change = 0.003
#> EM: Iteration = 86, LogLik = -26393.61, Change = 0.002
#> EM: Iteration = 87, LogLik = -26393.60, Change = 0.003
#> EM: Iteration = 88, LogLik = -26393.60, Change = 0.003
#> EM: Iteration = 89, LogLik = -26393.60, Change = 0.003
#> EM: Iteration = 90, LogLik = -26393.60, Change = 0.003
#> EM: Iteration = 91, LogLik = -26393.59, Change = 0.003
#> EM: Iteration = 92, LogLik = -26393.59, Change = 0.003
#> EM: Iteration = 93, LogLik = -26393.59, Change = 0.003
#> EM: Iteration = 94, LogLik = -26393.59, Change = 0.003
#> EM: Iteration = 95, LogLik = -26393.58, Change = 0.003
#> EM: Iteration = 96, LogLik = -26393.58, Change = 0.003
#> EM: Iteration = 97, LogLik = -26393.58, Change = 0.002
#> EM: Iteration = 98, LogLik = -26393.58, Change = 0.003
#> EM: Iteration = 99, LogLik = -26393.57, Change = 0.002
#> EM: Iteration = 100, LogLik = -26393.57, Change = 0.003
#> EM: Iteration = 101, LogLik = -26393.57, Change = 0.002
#> EM: Iteration = 102, LogLik = -26393.57, Change = 0.003
#> EM: Iteration = 103, LogLik = -26393.56, Change = 0.002
#> EM: Iteration = 104, LogLik = -26393.56, Change = 0.003
#> EM: Iteration = 105, LogLik = -26393.56, Change = 0.002
#> EM: Iteration = 106, LogLik = -26393.56, Change = 0.003
#> EM: Iteration = 107, LogLik = -26393.55, Change = 0.002
#> EM: Iteration = 108, LogLik = -26393.55, Change = 0.003
#> EM: Iteration = 109, LogLik = -26393.55, Change = 0.002
#> EM: Iteration = 110, LogLik = -26393.55, Change = 0.003
#> EM: Iteration = 111, LogLik = -26393.54, Change = 0.002
#> EM: Iteration = 112, LogLik = -26393.54, Change = 0.003
#> EM: Iteration = 113, LogLik = -26393.54, Change = 0.002
#> EM: Iteration = 114, LogLik = -26393.54, Change = 0.003
#> EM: Iteration = 115, LogLik = -26393.53, Change = 0.002
#> EM: Iteration = 116, LogLik = -26393.53, Change = 0.003
#> EM: Iteration = 117, LogLik = -26393.53, Change = 0.002
#> EM: Iteration = 118, LogLik = -26393.53, Change = 0.003
#> EM: Iteration = 119, LogLik = -26393.53, Change = 0.001
#> EM: Iteration = 120, LogLik = -26393.52, Change = 0.003
#> EM: Iteration = 121, LogLik = -26393.52, Change = 0.001
#> EM: Iteration = 122, LogLik = -26393.52, Change = 0.003
#> EM: Iteration = 123, LogLik = -26393.52, Change = 0.001
#> EM: Iteration = 124, LogLik = -26393.51, Change = 0.003
#> EM: Iteration = 125, LogLik = -26393.51, Change = 0.001
#> EM: Iteration = 126, LogLik = -26393.51, Change = 0.003
#> EM: Iteration = 127, LogLik = -26393.51, Change = 0.001
#> EM: Iteration = 128, LogLik = -26393.51, Change = 0.003
#> EM: Iteration = 129, LogLik = -26393.50, Change = 0.001
#> EM: Iteration = 130, LogLik = -26393.50, Change = 0.003
#> EM: Iteration = 131, LogLik = -26393.50, Change = 0.001
#> EM: Iteration = 132, LogLik = -26393.50, Change = 0.003
#> EM: Iteration = 133, LogLik = -26393.50, Change = 0.001
#> EM: Iteration = 134, LogLik = -26393.49, Change = 0.003
#> EM: Iteration = 135, LogLik = -26393.49, Change = 0.001
#> EM: Iteration = 136, LogLik = -26393.49, Change = 0.003
#> EM: Iteration = 137, LogLik = -26393.49, Change = 0.001
#> EM: Iteration = 138, LogLik = -26393.48, Change = 0.003
#> EM: Iteration = 139, LogLik = -26393.48, Change = 0.001
#> EM: Iteration = 140, LogLik = -26393.48, Change = 0.003
#> EM: Iteration = 141, LogLik = -26393.48, Change = 0.001
#> EM: Iteration = 142, LogLik = -26393.48, Change = 0.003
#> EM: Iteration = 143, LogLik = -26393.48, Change = 0.001
#> EM: Iteration = 144, LogLik = -26393.47, Change = 0.003
#> EM: Iteration = 145, LogLik = -26393.47, Change = 0.001
#> EM: Iteration = 146, LogLik = -26393.47, Change = 0.003
#> EM: Iteration = 147, LogLik = -26393.47, Change = 0.000
#> EM: Iteration = 148, LogLik = -26393.46, Change = 0.003
#> EM: Iteration = 149, LogLik = -26393.46, Change = 0.000
#> EM: Iteration = 150, LogLik = -26393.46, Change = 0.003
#> EM: Iteration = 151, LogLik = -26393.46, Change = 0.000
#> EM: Iteration = 152, LogLik = -26393.46, Change = 0.004
#> EM: Iteration = 153, LogLik = -26393.46, Change = 0.000
#> EM: Iteration = 154, LogLik = -26393.45, Change = 0.004
#> EM: Iteration = 155, LogLik = -26393.45, Change = 0.000
#> EM: Iteration = 156, LogLik = -26393.45, Change = 0.004
#> EM: Iteration = 157, LogLik = -26393.45, Change = 0.000
#>
#> modsem (version 1.0.3):
#> Estimator LMS
#> Optimization method EM-NLMINB
#> Number of observations 2000
#> Number of iterations 136
#> Loglikelihood -23781.36
#> Akaike (AIC) 47670.72
#> Bayesian (BIC) 47973.16
#>
#> Numerical Integration:
#> Points of integration (per dim) 24
#> Dimensions 1
#> Total points of integration 24
#>
#> Fit Measures for H0:
#> Loglikelihood -26393
#> Akaike (AIC) 52892.89
#> Bayesian (BIC) 53189.74
#> Chi-square 66.72
#> Degrees of Freedom (Chi-square) 82
#> P-value (Chi-square) 0.889
#> RMSEA 0.000
#>
#> Comparative fit to H0 (no interaction effect)
#> Loglikelihood change 2612.09
#> Difference test (D) 5224.18
#> Degrees of freedom (D) 1
#> P-value (D) 0.000
#>
#> R-Squared:
#> BEH 0.235
#> INT 0.365
#> R-Squared Null-Model (H0):
#> BEH 0.210
#> INT 0.367
#> R-Squared Change:
#> BEH 0.025
#> INT -0.002
#>
#> Parameter Estimates:
#> Coefficients unstandardized
#> Information expected
#> Standard errors standard
#>
#> Latent Variables:
#> Estimate Std.Error z.value P(>|z|)
#> INT =~
#> int1 1.000
#> int2 0.915 0.016 58.24 0.000
#> int3 0.807 0.015 54.49 0.000
#> ATT =~
#> att1 1.000
#> att2 0.876 0.012 71.51 0.000
#> att3 0.787 0.012 66.55 0.000
#> att4 0.693 0.011 60.50 0.000
#> att5 0.885 0.012 71.68 0.000
#> SN =~
#> sn1 1.000
#> sn2 0.893 0.017 52.59 0.000
#> PBC =~
#> pbc1 1.000
#> pbc2 0.912 0.013 69.14 0.000
#> pbc3 0.801 0.012 66.52 0.000
#> BEH =~
#> b1 1.000
#> b2 0.959 0.033 29.32 0.000
#>
#> Regressions:
#> Estimate Std.Error z.value P(>|z|)
#> BEH ~
#> INT 0.196 0.026 7.60 0.000
#> PBC 0.238 0.022 10.62 0.000
#> INT:INT 0.129 0.018 7.29 0.000
#> INT ~
#> PBC 0.218 0.029 7.51 0.000
#> ATT 0.210 0.025 8.28 0.000
#> SN 0.172 0.028 6.22 0.000
#>
#> Intercepts:
#> Estimate Std.Error z.value P(>|z|)
#> int1 1.005 0.020 49.42 0.000
#> int2 1.004 0.019 53.24 0.000
#> int3 0.998 0.017 57.46 0.000
#> att1 1.007 0.024 42.32 0.000
#> att2 1.001 0.021 47.17 0.000
#> att3 1.011 0.019 51.87 0.000
#> att4 0.994 0.018 55.74 0.000
#> att5 0.986 0.021 46.02 0.000
#> sn1 1.000 0.024 42.10 0.000
#> sn2 1.005 0.021 47.14 0.000
#> pbc1 0.991 0.023 43.04 0.000
#> pbc2 0.979 0.021 45.56 0.000
#> pbc3 0.986 0.019 51.24 0.000
#> b1 0.995 0.024 42.27 0.000
#> b2 1.014 0.022 45.68 0.000
#> BEH 0.000
#> INT 0.000
#> ATT 0.000
#> SN 0.000
#> PBC 0.000
#>
#> Covariances:
#> Estimate Std.Error z.value P(>|z|)
#> PBC ~~
#> ATT 0.668 0.079 8.48 0.000
#> SN 0.675 0.054 12.48 0.000
#> ATT ~~
#> SN 0.625 0.029 21.63 0.000
#>
#> Variances:
#> Estimate Std.Error z.value P(>|z|)
#> int1 0.161 0.009 18.28 0.000
#> int2 0.161 0.008 20.89 0.000
#> int3 0.170 0.007 23.51 0.000
#> att1 0.167 0.007 23.23 0.000
#> att2 0.150 0.006 24.81 0.000
#> att3 0.160 0.006 26.51 0.000
#> att4 0.163 0.006 27.46 0.000
#> att5 0.159 0.006 24.85 0.000
#> sn1 0.181 0.015 12.48 0.000
#> sn2 0.155 0.012 13.27 0.000
#> pbc1 0.145 0.008 18.27 0.000
#> pbc2 0.160 0.007 21.74 0.000
#> pbc3 0.154 0.007 23.69 0.000
#> b1 0.185 0.020 9.23 0.000
#> b2 0.136 0.018 7.52 0.000
#> BEH 0.475 0.024 19.71 0.000
#> PBC 0.960 0.037 26.13 0.000
#> ATT 1.000 0.058 17.32 0.000
#> SN 0.968 0.086 11.29 0.000
#> INT 0.481 0.019 24.97 0.000
est_qml <- modsem(tpb_nonlinear, data = TPB, cov.syntax = tpb_linear, method = "qml")
summary(est_qml)
#> Estimating null model
#> Starting M-step
#>
#> modsem (version 1.0.3):
#> Estimator QML
#> Optimization method NLMINB
#> Number of observations 2000
#> Number of iterations 76
#> Loglikelihood -26360.52
#> Akaike (AIC) 52829.04
#> Bayesian (BIC) 53131.49
#>
#> Fit Measures for H0:
#> Loglikelihood -26393
#> Akaike (AIC) 52892.45
#> Bayesian (BIC) 53189.29
#> Chi-square 66.27
#> Degrees of Freedom (Chi-square) 82
#> P-value (Chi-square) 0.897
#> RMSEA 0.000
#>
#> Comparative fit to H0 (no interaction effect)
#> Loglikelihood change 32.70
#> Difference test (D) 65.41
#> Degrees of freedom (D) 1
#> P-value (D) 0.000
#>
#> R-Squared:
#> BEH 0.239
#> INT 0.370
#> R-Squared Null-Model (H0):
#> BEH 0.210
#> INT 0.367
#> R-Squared Change:
#> BEH 0.029
#> INT 0.003
#>
#> Parameter Estimates:
#> Coefficients unstandardized
#> Information observed
#> Standard errors standard
#>
#> Latent Variables:
#> Estimate Std.Error z.value P(>|z|)
#> INT =~
#> int1 1.000
#> int2 0.914 0.015 59.04 0.000
#> int3 0.807 0.015 55.65 0.000
#> ATT =~
#> att1 1.000
#> att2 0.878 0.012 71.56 0.000
#> att3 0.789 0.012 66.37 0.000
#> att4 0.695 0.011 61.00 0.000
#> att5 0.887 0.013 70.85 0.000
#> SN =~
#> sn1 1.000
#> sn2 0.888 0.017 52.62 0.000
#> PBC =~
#> pbc1 1.000
#> pbc2 0.913 0.013 69.38 0.000
#> pbc3 0.801 0.012 66.08 0.000
#> BEH =~
#> b1 1.000
#> b2 0.960 0.032 29.91 0.000
#>
#> Regressions:
#> Estimate Std.Error z.value P(>|z|)
#> BEH ~
#> INT 0.197 0.025 7.76 0.000
#> PBC 0.239 0.023 10.59 0.000
#> INT:INT 0.128 0.016 7.88 0.000
#> INT ~
#> PBC 0.222 0.030 7.51 0.000
#> ATT 0.213 0.026 8.17 0.000
#> SN 0.175 0.028 6.33 0.000
#>
#> Intercepts:
#> Estimate Std.Error z.value P(>|z|)
#> int1 1.014 0.022 46.96 0.000
#> int2 1.012 0.020 50.41 0.000
#> int3 1.005 0.018 54.80 0.000
#> att1 1.014 0.024 42.01 0.000
#> att2 1.007 0.021 46.97 0.000
#> att3 1.016 0.020 51.45 0.000
#> att4 0.999 0.018 55.65 0.000
#> att5 0.992 0.022 45.67 0.000
#> sn1 1.006 0.024 41.66 0.000
#> sn2 1.010 0.022 46.71 0.000
#> pbc1 0.998 0.024 42.41 0.000
#> pbc2 0.985 0.022 44.93 0.000
#> pbc3 0.991 0.020 50.45 0.000
#> b1 0.999 0.023 42.64 0.000
#> b2 1.017 0.022 46.25 0.000
#> BEH 0.000
#> INT 0.000
#> ATT 0.000
#> SN 0.000
#> PBC 0.000
#>
#> Covariances:
#> Estimate Std.Error z.value P(>|z|)
#> PBC ~~
#> ATT 0.678 0.029 23.45 0.000
#> SN 0.678 0.029 23.08 0.000
#> ATT ~~
#> SN 0.629 0.029 21.70 0.000
#>
#> Variances:
#> Estimate Std.Error z.value P(>|z|)
#> int1 0.158 0.009 18.22 0.000
#> int2 0.160 0.008 20.38 0.000
#> int3 0.168 0.007 23.63 0.000
#> att1 0.167 0.007 23.53 0.000
#> att2 0.150 0.006 24.71 0.000
#> att3 0.160 0.006 26.38 0.000
#> att4 0.162 0.006 27.64 0.000
#> att5 0.159 0.006 24.93 0.000
#> sn1 0.178 0.015 12.09 0.000
#> sn2 0.157 0.012 13.26 0.000
#> pbc1 0.145 0.008 18.44 0.000
#> pbc2 0.160 0.007 21.42 0.000
#> pbc3 0.154 0.006 23.80 0.000
#> b1 0.185 0.020 9.42 0.000
#> b2 0.135 0.018 7.60 0.000
#> BEH 0.475 0.024 19.74 0.000
#> PBC 0.962 0.036 27.04 0.000
#> ATT 0.998 0.037 26.93 0.000
#> SN 0.988 0.039 25.23 0.000
#> INT 0.488 0.020 24.59 0.000