Combining Emic and Etic Studies: Recent Developments

Combining Emic and Etic Studies: Recent Developments

Emic and Etic Measurement in Large-Scale Assessments: Towards an Integration Fons J. R. van de Vijver Structure Focus in large-scale assessment on asking identical questions (etic approach) Emicetic distinction Relevance for large-scale assessment From a dichotomy to complementarity Example: personality assessment Implications for large-scale assessment N of Countries in Large-Scale Assessment Programme for International Student Achievement (PISA): 78 countries (2018) Starting Strong Survey (3S): 9 countries (2018) Teaching and Learning International Survey (TALIS): 48 countries (2018) Programme for the International Assessment of Adult Competencies (PIAAC): 47 countries Current Paradoxes of Large-Scale Assessments Due to Growth General: Growth is accompanied by paradoxes/trade-offs Increase in number of countries (with increase in cultural variation) will

simultaneously make such studies more informative for all stakeholders (countries, policymakers, scientists,) studies more difficult to conduct (e.g., adaptation issues) Analysis paradox: more countries, fewer conclusions can be drawn Scalar invariance as the desired/required, yet impossible goal Test design paradox: more countries, less cultural coverage Focus on what has at least some relevance in all countries Can easily gravitate towards Western-based stimuli (common to most OECD countries); Wober (1969): How well can they do our tricks? Test length paradox: more items, more design and analysis problems Long instruments are both desirable and undesirable (paradox that long instruments are more informative, yet less likely to show high level of invariance) Ways Forward? Maintain that more countries and longer instruments are more informative, but Adjust/refine invariance procedures OECD conference later this year on this topic Relax the framework that only identical stimuli can be used and are informative as the

framework will increasingly become a straitjacket This approach will be further explored here Need to Combine Universal and Culture-Specific Elements Psychological phenomena are typically neither completely universal, nor completely culture specific More universalism at a higher level of abstraction (extroversion) than at a lower level (indicators of extroversion) As a Venn Diagram: Construct X in Three Countries Large-scale assessment now focuses on the intersection Yet, any subset can add validity to specific comparisons or enhance internal consistency Options Adoption (Close, literal translation) Advantage: maintains metric equivalence Disadvantage: adequacy (too) readily assumed, should be

demonstrated Adaptation (changing contents of one or more items so as to increase cultural appropriateness) Advantage: more flexible, more tailored to the context Disadvantage: fewer statistical techniques available to compare scores across cultures Assembly (composing a new instrument) Advantage: very flexible Disadvantage: almost no comparability maintained Taxonomy of Adaptations Adaptation as generic term; Need for adaptation can be due to: Construct equivalence Similarity of construct in source and target culture Cultural equivalence Linguistic equivalence Norms about interaction (modes of address)

Cultural fact sheet Translation accuracy: Retention of denotation and connotation Measurement equivalence Retention of psychometric features (response styles) Similarity of factors measured by a test and comparability of scores A Note on the Term Adaptation Adaptations are always meant to enhance ecological validity, but Minimum definition: In OECD studies placeholders are often used when developing instruments Involving places where a close translation would not work, but where the rest of the item remains

intact Maximum definition: Adaptation as inserting or replacing items Emic-etic in personality research Pike (1967): "Emic: an emic account comes from a person within the culture; indigenous perspective E.g., what is intelligence in rural Zambia? "Etic: account is a description of a behavior or belief by an observer, in terms that can be applied to other cultures E.g., is the structure of intelligence universal? Long considered to be conflicting Example: (In)completeness of the Five-Factor Model Financially supported by NRF and SANPAD The SAPI, an acronym for South African Personality Inventory, is a project that aims to develop an indigenous personality measure for all 11 official languages in South Africa. Participants are Byron Adams (University of Johannesburg and Tilburg University, the Netherlands), Deon de Bruin (University of Johannesburg), Karina de Bruin (University of Johannesburg), Carin Hill (University of Johannesburg), Leon Jackson (North-West University), Deon Meiring (University of Pretoria and University of Stellenbosch), Alewyn Nel (North-West University), Ian Rothmann (North-West University), Michael Temane (North-West University), Velichko

Valchev (Tilburg University, the Netherlands), and Fons van de Vijver (North-West University and Tilburg University, the Netherlands). Cross-Cultural Personality Models Universality (etic approach) Using imposed etic concepts emphasizes universals or core similarities in all human beings Are the imposed etic personality constructs universally applicable? Cultural specificity (emic approach) Utilizes a culture-specific orientation - emic concepts Indigenous and culturally relevant constructs may be missing Are there personality characteristics that are important to the understanding of personality in the local culture that have been left out in Western measures?

Personality: Evidence for Universality Evidence largely based on two lines of research 1. Five-Factor Model (Costa, McCrae) Five factors: Neuroticism, Extroversion, Openness, Agreeableness, Conscientiousness 2. Three-Factor Model (Eysenck) Three factors: Extroversion, Neuroticism, Psychoticism Evidence for Cultural Specificity Chinese concepts of interpersonal relatedness Japanese concept of amae (sweet indulgence) Korean concept of chong (affection) Indian concept of selfless self South-African concept of Ubuntu Meaning "humanity towards others"

"I am because we are" "A person 'becomes human' through other persons" "A person is a person because of other persons" Emic versus Etic or Emic and Etic? Personality is neither completely universal, nor completely culture-specific In retrospect, it is difficult to see how only one perspective could be presented as THE perspective Emic and Etic instead of Emic vs. Etic How can we combine the two? Much evidence in cross-cultural psychology that basic structure of psychological functioning is universal, but manifestations can differ across cultures Languages of South Africa Language % Zulu 23.8% Xhosa 17.6% Afrikaans 13.3%

Northern Sotho 9.4% Tswana 8.2% English 8.2% Sotho 7.9% Tsonga 4.4% Swati 2.7% Venda 2.3% Ndebele 1.6%

Other languages 0.5% Aim SAPI Project No test available with demonstrated validity in all groups Develop a personality instrument for South Africa that can accommodate its diversity Stages 1. Qualitative Stage Identifying personality structure in an indigenous, comparative study Generating and piloting items/scales 2. Quantitative Stage Method Participants Adults from all eleven language groups Sample guidelines: Socio-economic Status, Education (low, medium, High), Males/Females, Age (18-35 yrs & 35 older) Interview done in participants first language

Instrument Person descriptions of familiar persons Your own personality Describe yourself A parent . Stages in Data Analysis Phase 1 Preparing Data Translations Removing synonyms, Phase 2 Iterations of Condensing data Consulting language and cultural experts Phase 3: Conceptual Cluster Analysis 22 Condensing the Data Facets abusive (34) Categorising Cleaning Response in English

Response in Ndebele abusive abused He is abused Uhlukulumezekile epilweni abusive abused him he abused him even when abuse was not there kade ambuser lokha ingekho abusive abuses me He abuses me Uyangi hlukumeza abusive Abusive person

An abusive person Umuntu ohlukumezako SOFTHEARTEDNESS RELATIONSHIP HARMONY Conflict Seeking Meddlesomeness Hostility Amiability Interpersonal Relatedness Egoism Active support Approachability FACILITATIN G Gratefulness Guidance

Empathy EXTRAVERSI ON Dominance CONSCIENTIOUSNE SS Sociability Expressiveness Positive Emotionality Achievement Orientation Thoughtlessness EMOTIONAL STABILITY OPENNES S Materialism Emotional Control Emotional Sensitivity Neuroticism

Balance INTEGRI TY Dedication Self-discipline Ego Strength Encouraging others Epistemic Curiosity Integrity Fairness Orderliness INTELLEC T Aesthetics Reasoning Broadmindedness Openness to Experience

Skillfulness Social Intellect Cluster: Relationship Harmony Subcluster Approachability Conflict-Seeking Facet Example Response (Language) Accommodating Addressed us in English so we could understand (Xhosa) Approachable She is approachable, I could speak to her about anything (S Sotho) Arrogant He thinks he is better than all the other people (N Sotho) Flexible

Flexible to situation (Tswana) Humble She is a humble and down to earth person (Ndebele) Open for Others Accepts people for who and what they are (English) Proud Is proud and thinks of herself better than others (Swati) Stubborn Was stubborn, did not listen to anybody (Tswana) Tolerant Tolerant (Afrikaans) Welcoming Welcoming to everyone (Venda) Argumentative Likes to quarrel (Xhosa)

Provoking Provocative and calls people names (Swati) Troublesome Creates tension for nothing (Zulu) Cluster: Soft-Heartedness Active Support Community Involvement (11/143) There is one person who is always looking after the community (Zulu) Heedful She listens when you talk to her (S Sotho) Helpful Is helpful when you are in need (Swati) Protective Protective (Xhosa) Solving Problems of Others If I have a problem, she knows how to solve it (Ndebele) Amiability

Egoism Supportive I like to give people my support (Tswana) Friendly She is a friendly person (Tsonga) Irritating He is annoying and irritating (S Sotho) Kind Kind (Venda) Likeable He is loved by everyone (S Sotho) Pleasant He was a nice person to live with (Zulu) Stern Always serious, not smiling (Xhosa) Generous

One who is generous and gives food when asked (Swati) Greedy Greedy (Afrikaans) Jealous A person who is jealous of other peoples possessions (Zulu) Self-Centered All revolves around her, she thinks (English) Selfish Wants everything for himself (Xhosa) Quantitative Findings 262 items (incl. 12 SD items) administered to 1,364 students and community adults 671 Blacks, 198 Coloureds, 104 Indians, 391 Whites EFA per cluster; items with loadings < .30 or .40 were removed 158 items in latest version (incl. 12 Social Desirability items) 18 empirically derived scales EFA on scale scores 27 Cronbachs alpha

Black Coloured Indian White Facilitating (10) .84 .85 .87 .88 Integrity (12) .76 .83 .84 .79 Social Intellect (4) .71

.75 .82 .74 Interpersonal Relatedness (9) .77 .81 .83 .78 Warm-Heartedness (12) .83 .89 .83 .87 Deceitfulness (3) .58 .59

.75 .53 Conflict-Seeking (6) .67 .70 .71 .64 HostilityEgoism (13) .80 .84 .89 .83 Emotional Balance (8) .71 .77 .73

.74 Negative Emotionality (10) .75 .73 .76 .77 Playfulness (6) .71 .76 .84 .81 Sociability (7) .75 .80 .84 .86

Achievement Orientation (10) .79 .78 .81 .82 Orderliness (11) .81 .85 .88 .86 TraditionalismReligiosity (4) .57 .65 .78 .75 Intellect (10)

.74 .81 .79 .76 Broad-Mindedness (5) .60 .66 .72 .75 Epistemic Curiosity (6) .66 .77 .83 .80 Scale (# items) 28

SAPI Structure 29 RELATIONSHIP HARMONY Conflict Seeking Meddlesomeness Qualitative SOFTHEARTEDNESS Hostility Amiability Interpersonal Relatedness Egoism Active support Approachability FACILITATIN G Gratefulness

Guidance Empathy EXTRAVERSI ON Dominance CONSCIENTIOUSNE SS Sociability Expressiveness Positive Emotionality Achievement Orientation Thoughtlessness EMOTIONAL STABILITY Courage OPENNES S Materialism Emotional Control

Emotional Sensitivity Neuroticism Balance INTEGRI TY Dedication Self-discipline Ego Strength Encouraging others Epistemic Curiosity Integrity Fairness Orderliness INTELLEC T Aesthetics

Reasoning Broadmindedness Openness to Experience Skillfulness Social Intellect 30 Quantitative SOCREL POSITIVE SOCREL NEGATIVE Conflict Seeking Interpersonal Relatedness Facilitating Hostile Egoism Deceitfulness WarmHeartedness

Social Intelligence EXTRAVERSI ON CONSCIENTIOUSNE SS Orderliness Integrity Playfulness Sociability Achievement Orientation Emotional Maturity Traditionalism Religiosity NEUROTICIS M Negative Emotionality INTELLEC T/ OPENNES S

Intellect Epistemic Curiosity Broadmindedness 31 Conclusion Both emic and etic factors were found Emic notably in the social-relational domain Five Factor Model well replicated for N, E, O, C If we would have started by applying a standard Big Five/FFM instrument, the salience of social relational aspects would never have been found Emic and Etic in Large-Scale Studies Etic/one-size-fits-all approach has strengths Allows for cross-cultural comparisons Is efficient (can use short instruments) Can often rely on tried-and-tested instruments Etic/one-size-fits-all approach has limitations Focus on short instruments Construct coverage can become problematic Focus on what is shared across cultures, which will become less and less when cultural variation is

increased Combined emicetic approach would amount to adding culture-specific items to etic measures Note: different from international option that involves all items about a construct Emic approach can help to Increase ecological validity of instruments Increase construct coverage Enhance validity and internal consistency in cultures Overcome the criticism that large-scale assessments can show Western bias Note: item response theory can deal with the combination of etic and emic items (cf. matrix designs) Global Conclusion From emic versus etic to emic and etic Emphasis on identical questions in large-scale assessment (one-size-fits-all) becomes problematic with the increased cultural/linguistic diversity of large-scale assessments Culture-specific items may help policy makers

Recently Viewed Presentations

  • CARBON FOOTPRINT THE RIGHT ONES Isabel Dominguez 0175615

    CARBON FOOTPRINT THE RIGHT ONES Isabel Dominguez 0175615

    Proposal for change. In order to reduce our emissions most effectively, we need to know where they come from. Our study shows that our emissions come from five areas: transportation, heating/cooling your home, appliances and other energy users in your...
  • Phylum Mollusca - Weebly

    Phylum Mollusca - Weebly

    Phylum Mollusca. 75000 species. 2nd only to Phylum Arthropoda. Includes octopuses, oysters, snails, slugs, squids, clams… Divided into 6 classes based on shell and foot
  • Money Text Set READ 436 Dr. Sullivan May

    Money Text Set READ 436 Dr. Sullivan May

    Written By: Stuart J. Murphy. Publisher: Harper Collins Publisher (2002) Reading Level: 3.0 (Flesch-Kincaid) Summary: It is a great book for students who are learning about adding money, subtracting money, and making change. The book is about a group of...
  • Intro to Chinas Policymaking Process The creation and

    Intro to Chinas Policymaking Process The creation and

    "Opinions" are like a teaser to the National Five-Year Plan. Every fifth year, the Party Central Committee holds a plenary session to determine China's key areas of focus for the next half-decade. The result is a strategic document typically called...
  • Ablative Absolute - University of Florida

    Ablative Absolute - University of Florida

    Literally translates: With noun having been verbed (ppp) With noun verbing (pap) In smooth English: [When, since, although] noun had been verbed Passive Periphrastic The Future Passive Participle aka Gerundive + a form of sum The participle part of the...
  • The New Cosmology: I. Basic Cosmology

    The New Cosmology: I. Basic Cosmology

    Arial Wingdings 1_Default Design Big Bang Nucleosynthesis: Theory vs. Observation Slide 2 Slide 3 Slide 4 Slide 5 Slide 6 Slide 7 Slide 8 Slide 9 Slide 10 Light Element Abundances D, He-4 and Li-7 Slide 12 Detecting Deuterium Slide...
  • Faculty Proposal to Teach in the Honors College

    Faculty Proposal to Teach in the Honors College

    Honors LBST (continued) LBST 2101-Hxx: Western Civilization . All sections of this course explore a major aspect of western culture. Particular attention is given to an examination of the constructed nature of the present through a close examination of the...
  • Jesus Shall Reign - Traditional Music Library

    Jesus Shall Reign - Traditional Music Library

    Jesus shall reign where'er the sun. Doth his successive journeys run; His kingdom stretch from shore to shore, Till moons shall wax and wane no more.