Revista Methodology
La revista Methodology es el órgano oficial de la European Association of Methodology (EAM), una asociación de expertos en metodologÃa que desarrollan su labor en distintas áreas de las ciencias sociales y del comportamiento. La revista constituye una plataforma para el intercambio interdisciplicinar de la investigación metodológica y sus aplicaciones en distintos campos. Se abordan tres disciplinas principales: análisis de datos, investigación metodológica y psicometrÃa.
Methodology: European Journal of Research Methods for the Behavioral and Social Sciences
http://www.psycontent.com
Methodological Advances for Detecting Physiological Synchrony During Dyadic Interactions- Wed, 01 Feb 2012 14:59:53 GMTA defining feature of many physiological systems is their synchrony and reciprocal influence. An important challenge, however, is how to measure such features. This paper presents two new approaches for identifying synchrony between the physiological signals of individuals in dyads. The approaches are adaptations of two recently-developed techniques, depending on the nature of the physiological time series. For respiration and thoracic impedance, signals that are measured continuously, we use Empirical Mode Decomposition to extract the low-frequency components of a nonstationary signal, which carry the signal?s trend. We then compute the maximum cross-correlation between the trends of two signals within consecutive overlapping time windows of fixed width throughout each of a number of experimental tasks, and identify the proportion of large values of this measure occurring during each task. For heart rate, which is output discretely, we use a structural linear model that takes into account heteroscedastic measurement error on both series. The results of this study indicate that these methods are effective in detecting synchrony between physiological measures and can be used to examine emotional coherence in dyadic interactions.
|
Non-Graphical Solutions for Cattell?s Scree Test- Wed, 01 Feb 2012 14:59:52 GMTMost of the strategies that have been proposed to determine the number of components that account for the most variation in a principal components analysis of a correlation matrix rely on the analysis of the eigenvalues and on numerical solutions. The Cattell?s scree test is a graphical strategy with a nonnumerical solution to determine the number of components to retain. Like Kaiser?s rule, this test is one of the most frequently used strategies for determining the number of components to retain. However, the graphical nature of the scree test does not definitively establish the number of components to retain. To circumvent this issue, some numerical solutions are proposed, one in the spirit of Cattell?s work and dealing with the scree part of the eigenvalues plot, and one focusing on the elbow part of this plot. A simulation study compares the efficiency of these solutions to those of other previously proposed methods. Extensions to factor analysis are possible and may be particularly useful with many low-dimensional components.
|
Misestimation of Reliability Using Coefficient Alpha and Structural Equation Modeling When Assumptions of Tau-Equivalence and Uncorrelated Errors Are Violated- Wed, 01 Feb 2012 14:59:52 GMTCoefficient alpha (?) has been described as a lower
bound for test reliability. However, previous research indicates that when
certain assumptions are violated, ? can either
overestimate or underestimate reliability.
|
?Is the Hypothesis Correct? or ?Is it Not?- Fri, 28 Oct 2011 13:57:05 GMTResearchers in the behavioral and social sciences often have one informative hypothesis with respect to the state of affairs in the population from which they sampled their data. The question they would like an answer to is ?Is the Hypothesis Correct? or ?Is it Not.? Classical statistics has not yet provided an approach with which this question can be answered. In this paper it will be shown that there is a Bayesian approach that does provide an answer to this question. Using two ANOVA examples the context of this paper will be sketched. Subsequently it will be shown how the Bayes factor can be used to quantify the support in the data for an informative hypothesis (?It is?) and its complement (?It is not?). Subsequently, the performance of the method proposed will be evaluated by means of error probabilities and evaluation of the robustness with respect to violations of the assumption of homogeneous within group variances. Finally, the methodology will be elaborated and it will be illustrated how the approach proposed can be implemented using WinBUGS.
|
The Impact of Controlling for Extreme Responding on Measurement Equivalence in Cross-Cultural Research- Fri, 28 Oct 2011 13:57:04 GMTPrior research has shown that extreme response style can seriously bias responses to survey questions and that this response style may differ across culturally diverse groups. Consequently, cross-cultural differences in extreme responding may yield incomparable responses when not controlled for. To examine how extreme responding affects the cross-cultural comparability of survey responses, we propose and apply a multiple-group latent class approach where groups are compared on basis of the factor loadings, intercepts, and factor means in a Latent Class Factor Model. In this approach a latent factor measuring the response style is explicitly included as an explanation for group differences found in the data. Findings from two empirical applications that examine the cross-cultural comparability of measurements show that group differences in responding import inequivalence in measurements among groups. Controlling for the response style yields more equivalent measurements. This finding emphasizes the importance of correcting for response style in cross-cultural research.
|
Sample Size Issues for Cluster Randomized Trials With Discrete-Time Survival Endpoints- Fri, 28 Oct 2011 13:57:04 GMTWith cluster randomized trials complete groups of subjects are randomized to treatment conditions. An important question might be whether and when the subjects experience a particular event, such as smoking initiation or recovery from disease. In the social sciences the timing of such events is often measured in discrete time by using time intervals. At the planning phase of a cluster randomized trial one should decide on the number of clusters and cluster size such that parameters are estimated accurately and sufficient power on the test on treatment effect is achieved. On basis of a simulation study it is concluded that regression coefficients are estimated more accurately than the variance of the random cluster effect. In addition, it is shown that power increases with cluster size and number of clusters, and that a sufficient power cannot always be achieved by using larger cluster sizes at a fixed number of clusters.
|
The Effects of Purification and the Evaluation of Differential Item Functioning With the Likelihood Ratio Test- Fri, 28 Oct 2011 13:57:03 GMTThe current research compares the effects of several strategies to establish the anchor subtest when detecting for differential item functioning (DIF) using the IRT likelihood ratio test in one- and two-stage procedures. Two one-stage strategies were examined: (1) ?One item? and (2) ?All other items? used as anchor. Additionally, two two-stage strategies were tested: (3) ?One anchor item with posterior anchor test augmentation? and (4) ?All other items with purification.? The strategies were compared in a simulation study, where sample sizes, DIF size, type of DIF, and software implementation (MULTILOG vs. IRTLRDIF) were manipulated. Results indicated that Procedure (1) was more efficient than (2). Purification was found to improve Type I error rates substantially with the ?all other items? strategy, while ?posterior anchor test augmentation? did not yield a significant improvement. In relation to the effect of the software used, we found that MULTILOG generally offers better results than IRTLRDIF.
|
Analyzing Observed Composite Differences Across Groups- Fri, 28 Oct 2011 13:57:03 GMTAlthough the use of structural equation modeling has increased during the last decades, the typical procedure to investigate mean differences across groups is still to create an observed composite score from several indicators and to compare the composite?s mean across the groups. Whereas the structural equation modeling literature has emphasized that a comparison of latent means presupposes equal factor loadings and indicator intercepts for most of the indicators (i.e., partial invariance), it is still unknown if partial invariance is sufficient when relying on observed composites. This Monte-Carlo study investigated whether one or two unequal factor loadings and indicator intercepts in a composite can lead to wrong conclusions regarding latent mean differences. Results show that unequal indicator intercepts substantially affect the composite mean difference and the probability of a significant composite difference. In contrast, unequal factor loadings demonstrate only small effects. It is concluded that analyses of composite differences are only warranted in conditions of full measurement invariance, and the author recommends the analyses of latent mean differences with structural equation modeling instead.
|
Drawing Inferences From Multiple Intervals in the Single-Factor Design- Fri, 28 Oct 2011 13:57:03 GMTAlthough confidence intervals for means are excellent vehicles for making inferences about population values, they are not always efficient and practical for making inferences about the differences among the means. This article reviews and elaborates on methods that modify the calculations and the graphical presentations of confidence intervals in order to make them appropriate for both single value inferences and pairwise comparisons in a single-factor between-subjects design. Extensions of the procedure, as well as potential problems, are also discussed.
|
One Size Fits All?- Thu, 04 Aug 2011 22:59:43 GMTThe use of latent curve models (LCMs) has increased almost exponentially during the last decade. Oftentimes, researchers regard LCM as a ?new? method to analyze change with little attention paid to the fact that the technique was originally introduced as an ?alternative to standard repeated measures ANOVA and first-order auto-regressive methods? (Meredith & Tisak, 1990, p. 107). In the first part of the paper, this close relationship is reviewed, and it is demonstrated how ?traditional? methods, such as the repeated measures ANOVA, and MANOVA, can be formulated as LCMs. Given that latent curve modeling is essentially a large-sample technique, compared to ?traditional? finite-sample approaches, the second part of the paper addresses the question to what degree the more flexible LCMs can actually replace some of the older tests by means of a Monte-Carlo simulation. In addition, a structural equation modeling alternative to Mauchly?s (1940) test of sphericity is explored. Although ?traditional? methods may be expressed as special cases of more general LCMs, we found the equivalence holds only asymptotically. For practical purposes, however, no approach always outperformed the other alternatives in terms of power and type I error, so the best method to be used depends on the situation. We provide detailed recommendations of when to use which method.
|
Managing Heterogeneity of Variance in Studies of Reliability Generalization With Alpha Coefficients- Thu, 04 Aug 2011 22:59:41 GMTIn Reliability Generalization (RG) meta-analyses, the importance of bearing in mind the problems of range restriction or biased sampling and their influence on reliability estimation has often been highlighted. Nevertheless, the presence of heterogeneous variances in the included studies has been diagnosed in a subjective way and has not been taken into account in later analyses. Procedures to detect the presence of a variety of sampling schemes and to manage them in the analyses are proposed. The procedures are further explained with an example, by applying them to 25 estimates of Cronbach?s alpha coefficient in the Hamilton Scale for Depression.
|
Detection of Differential Item Functioning- Thu, 04 Aug 2011 22:59:40 GMTThis study analyzes Differential Item Functioning (DIF) with three combined decision rules and compares the results with the variation of the Mantel-Haenszel procedure (vaMH) proposed by Mazor, Clauser, and Hambleton (1994). One decision rule combines the Mantel-Haenszel procedure (MH) with the Breslow-Day test of trend in odds ratio heterogeneity (BDT), having performed the Bonferroni adjustment, as Randall Penfield proposed. The second uses both MH and BDT without the Bonferroni adjustment. The third combines MH with the Breslow-Day test for homogeneity of the odds ratio without the Bonferroni adjustment. The three decision rules yielded satisfactory results, showed similar power, and none of them detected DIF erroneously. The second rule proved to be the most powerful in the presence of nonuniform DIF. Only in the presence of uniform DIF with the smallest difference of difficulty parameters, was there evidence of vaMH?s superiority.
|
Change? What Change?- Thu, 04 Aug 2011 22:59:39 GMTA primary objective of panel studies is to analyze change. The same questionnaire is used to compare data recorded at various times. Panel designs assume that the meaning of the questions and the concept of interest are stable over time. Analyses of measurement invariance often show the contrary. A qualitative part supplementing a panel survey can help us understand this phenomenon. In this study, 261 first-year psychology students completed questionnaires about their study motivation on two occasions; we interviewed some students as well. The survey showed that study motivation is not invariant over time. The qualitative data converged with the quantitative outcomes and explained the lack of invariance by the students? overall transition during the first study year. We conclude that mixing quantitative and qualitative research methods for panel studies helps us understand change in constructs over time. We can study change at the macrolevel and better understand such change at the microlevel.
|
Variance Heterogeneity in Published Psychological Research- Thu, 04 Aug 2011 22:59:37 GMTParametric assumptions for statistical tests include normality and equal variances. Micceri (1989) found that data frequently violate the normality assumption; variances have received less attention. We recorded within-group variances of dependent variables for 455 studies published in leading psychology journals. Sample variances differed, often substantially, suggesting frequent violation of the assumption of equal population variances. Parallel analyses of equal-variance artificial data otherwise matched to the characteristics of the empirical data show that unequal sample variances in the empirical data exceed expectations from normal sampling error and can adversely affect Type I error rates of parametric statistical tests. Variance heterogeneity was unrelated to relative group sizes or total sample size and observed across subdisciplines of psychology in experimental and correlational research. These results underscore the value of examining variances and, when appropriate, using data-analytic methods robust to unequal variances. We provide a standardized index for examining and reporting variance heterogeneity.
|
Assessing Content Validity Through Correlation and Relevance Tools- Thu, 04 Aug 2011 22:59:29 GMTContent validity elicits expert opinion regarding items of a psychometric instrument. Expert opinion can be elicited in many forms: for example, how essential an item is or its relevancy to a domain. This study developed an alternative tool that elicits expert opinion regarding correlations between each item and its respective domain. With 109 Registered Nurse (RN) site coordinators from National Database of Nursing Quality Indicators®, we implemented a randomized Bayesian equivalence trial with coordinators completing ?relevance? or ?correlation? content tools regarding the RN Job Enjoyment Scale. We confirmed our hypothesis that the two tools would result in equivalent content information. A Bayesian ordered analysis model supported the results, suggesting that evidence for traditional content validity indices can be justified using correlation arguments.
|
Subjective p Intervals- Thu, 04 Aug 2011 22:59:27 GMTSuppose you obtain p = .02 in an experiment, then replicate the experiment with new samples. What p value might you obtain, and what interval has an 80% chance of including that replication p? Under conservative assumptions the answer is, perhaps surprisingly (.0003, .30). The authors report three email surveys that asked authors of articles published in leading journals in psychology, medicine, or statistics to estimate such intervals. Overall response rate (7%) was low, but responses from 360 researchers gave intervals with an average 40% to 50% chance of including replication p, rather than the target 80%. Results were similar for all three disciplines. Respondents generally found the task unfamiliar and difficult. There was great variability over respondents, but almost all of them gave intervals that were too short. This widespread, and often severe, underestimation of the variability of p may help to explain why researchers place too much interpretive weight on single p values.
|
When the Truth Hits You Between the Eyes- Thu, 04 Aug 2011 22:59:26 GMTVisual data analysis is an important first step when evaluating intervention effects. This also holds for analyzing data from single-case experiments. Because most software packages do not offer customized facilities for constructing single-case graphs and are not particularly suited to perform single-case visual data analyses, we created an R package to help researchers in making graphical representations of single-case data and to transform graphical displays back to raw data. In addition to a basic plotting function, we included some tools to facilitate the use of three interpretative principles for visually analyzing single-case data: plotting a measure of central location as a horizontal reference line; displaying variability with (trimmed) range bars, range lines, and trended ranges; and displaying trends with a vertical line graph, by fitting a robust linear trend, or by plotting running medians. Finally, we included a function to extract raw data values from published graphs.
|
Estimation of and Confidence Interval Formation for Reliability Coefficients of Homogeneous Measurement Instruments- Thu, 04 Aug 2011 22:59:24 GMTThe reliability of a composite score is a fundamental and important topic in the social and behavioral sciences. The most commonly used reliability estimate of a composite score is coefficient ?. However, under regularity conditions, the population value of coefficient ? is only a lower bound on the population reliability, unless the items are essentially ?-equivalent, an assumption that is likely violated in most applications. A generalization of coefficient ?, termed ?, is discussed and generally recommended. Furthermore, a point estimate itself almost certainly differs from the population value. Therefore, it is important to provide confidence interval limits so as not to overinterpret the point estimate. Analytic and bootstrap methods are described in detail for confidence interval construction for ?. We go on to recommend the bias-corrected bootstrap approach for ? and provide open source and freely available R functions via the MBESS package to implement the methods discussed.
|
Exploiting Prior Information in Stochastic Knowledge Assessment- Thu, 04 Aug 2011 22:59:24 GMTVarious adaptive procedures for efficiently assessing the knowledge state of an individual have been developed within the theory of knowledge structures. These procedures set out to draw a detailed picture of an individual?s knowledge in a certain field by posing a minimal number of questions. While research so far mostly emphasized theoretical issues, the present paper focuses on an empirical evaluation of probabilistic assessment. It reports on simulation data showing that both efficiency and accuracy of the assessment exhibit considerable sensitivity to the choice of parameters and prior information as captured by the initial likelihood of the knowledge states. In order to deal with problems that arise from incorrect prior information, an extension of the probabilistic assessment is proposed. Systematic simulations provide evidence for the efficiency and robustness of the proposed extension, as well as its feasibility in terms of computational costs.
|
An Improved Model for Evaluating Change in Randomized Pretest, Posttest, Follow-Up Designs- Thu, 04 Aug 2011 22:59:22 GMTRandomized pretest, posttest, follow-up (RPPF) designs are often used for
evaluating the effectiveness of an intervention. These designs typically address
two primary research questions: (1) Do the treatment and control groups differ
in the amount of change from pretest to posttest? and (2) Do the
treatment and control groups differ in the amount of change from posttest to
follow-up? This study presents a model for answering these questions
and compares it to recently proposed models for analyzing RPPF designs due to
|
Página anterior: Datos de contacto
Página siguiente: Congresos y Eventos
aemcco@ub.edu