Sociologický časopis / Czech Sociological Review 2021, 57(6): 683-705 | DOI: 10.13060/csr.2022.001

Improving Knowledge Production in Comparative Survey Research: Cross-Using Data from Four International Survey Programmes

Brina Malnar1, Lorna Ryan2
1 University of Ljubljana
2 City, University of London

In the context of flourishing cross-national and multi-level research, the variety of macro and micro data sources available to researchers have evolved into an interdependent ecosystem of social indicators. Focusing on four comparative social surveys, this paper examines the extent to which secondary data users take advantage of a range of complementary data sources to broaden the breadth or strengthen the robustness of their research. Using two Google Scholar-based datasets of 2789 and 796 publications, we find that, despite the complex equivalence issues in comparative survey research, users combine data to a considerable extent, aiming to increase conceptual, geographic, and temporal coverage and cross-validate findings. Selecting the example of the European Social Survey, 183 journal articles are qualitatively examined to identify specific epistemic gains attained by analysts when combining ESS survey data with data from other comparative programmes. The strategy involves risks, emanating from either analysts' own misjudgements or arising from the wider issues of comparability and transparency in cross-national survey research. However, a number of data harmonisation platforms have recently emerged that may facilitate the standardisation of measures across surveys, augmenting the possibilities for future theory development and research.

Klíčová slova: comparative surveys, social indicators, academic users, data ecosystem, multi-level analysis

Vloženo: 25. září 2020; Revidováno: 17. červenec 2021; Přijato: 6. leden 2022; Zveřejněno: 17. leden 2022  Zobrazit citaci

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Malnar, B., & Ryan, L. (2021). Improving Knowledge Production in Comparative Survey Research: Cross-Using Data from Four International Survey Programmes. Sociologický časopis / Czech Sociological Review57(6), 683-705. doi: 10.13060/csr.2022.001
Stáhnout citaci

Reference

  1. Andress, H.-J., D. Fetchenhauer and H. Meulemann. 2019. 'Cross-National Comparative Research - Analytical Strategies, Results, and Explanations.' KZfSS Kölner Zeitschrift Für Soziologie Und Sozialpsychologie 71 (S1): 1-28, https://doi.org/10.1007/s11577-019-00594-x. Přejít k původnímu zdroji...
  2. Blair, E. 2016. 'A Reflexive Exploration of Two Qualitative Data Coding Techniques.' Journal of Methods and Measurement in the Social Sciences 6 (1): 14-29, https://doi.org/10.2458/azu_jmmss.v6i1.18772. Přejít k původnímu zdroji...
  3. Boukes, M. and A. C. Morey. 2018. 'Survey Context Effects and Implications for Validity: Measuring Political Discussion Frequency in Survey Research.' Journal of Survey Statistics and Methodology 7 (2): 201-226, https://doi.org/10.1093/jssam/smy008. Přejít k původnímu zdroji...
  4. Briceno-Rosas, R., S. Butt and J. Kappelhof. 2020. 'Improving Central Monitoring of Fieldwork in Cross-National Surveys: The Case of the Fieldwork Management System in the European Social Survey.' Survey Methods: Insights from the Field. Retrieved 20 September 2021 (https://surveyinsights.org/?p=11866).
  5. Bryan, M. L. and S. P. Jenkins. 2015. 'Multilevel Modelling of Country Effects: A Cautionary Tale.' European Sociological Review 32 (1): 3-22, https://doi.org/10.1093/esr/jcv059. Přejít k původnímu zdroji...
  6. Burkhauser, R. V. and D. R. Lillard. 2005. 'The Contribution and Potential of Data Harmonization for Cross-National Comparative Research.' DIW Diskussionspapiere 486. Retrieved 20 July 2020 (https://www.econstor.eu/bitstream/10419/18337/1/dp486.pdf).
  7. Curty, R. G. 2016. 'Factors Influencing Research Data Reuse in the Social Sciences: An Exploratory Study.' International Journal of Digital Curation 11 (1): 96-117, https://doi.org/10.2218/ijdc.v11i1.401. Přejít k původnímu zdroji...
  8. Damian, E., B. Meuleman and W. van Oorschot. 2019. 'Transparency and Replication in Cross-National Survey Research: Identification of Problems and Possible Solutions.' Sociological Methods & Research. Retrieved 20 July 2020 (https://journals.sagepub.com/doi/abs/10.1177/0049124119882452), https://doi.org/10.1177/0049124119882452. Přejít k původnímu zdroji...
  9. Davies, T. 2012. 'Open Data: Infrastructures and Ecosystems.' Paper presented at the ACM Web Science Conference, Evanston, Illinois. Retrieved 20 July 2020 (https://www.emergentworks.net/sites/default/files/ikmemergent_archive/Social_Life_of_Data_-_Infrastructure_and_Ecosystem_Paper.pdf).
  10. Dong, Q., M. R. Elliott and T. E. Raghunathan. 2014. 'Combining Information from Multiple Complex Surveys.' Survey Methodology 40 (2): 347-354.
  11. Doolan, D. M. and E. S. Froelicher. 2009. 'Using an Existing Data Set to Answer New Research Questions: A Methodological Review.' Research and Theory for Nursing Practice 23 (3): 203-215, https://doi.org/10.1891/1541-6577.23.3.203. Přejít k původnímu zdroji...
  12. Drost, E. 2011. 'Validity and Reliability in Social Science Research.' Education Research and Perspectives 38 (1): 105-123.
  13. Dubrow, J. K. and I. Tomescu-Dubrow. 2015. 'The Rise of Cross-National Survey Data Harmonization in the Social Sciences: Emergence of an Interdisciplinary Methodological Field.' Quality & Quantity 50 (4): 1449-1467, https://doi.org/10.1007/s11135-015-0215-z. Přejít k původnímu zdroji...
  14. Duca, D. and K. Metzler. 2019. The Ecosystem of Technologies for Social Science Research (White Paper). London, UK: Sage, https://doi.org/10.4135/wp191101. Přejít k původnímu zdroji...
  15. European Strategy Forum on Research Infrastructures (ESFRI). 2018. ESFRI Roadmap 2018 - Strategy Report on Research Infrastructures. Retrieved 20 July 2020 (http://roadmap2018.esfri.eu/).
  16. Faniel, I. M., A. Kriesberg and E. Yakel. 2015. 'Social Scientists' Satisfaction with Data Reuse.' Journal of the Association for Information Science and Technology 67 (6): 1404-1416. Přejít k původnímu zdroji...
  17. Franco, A., N. Malhotra and G. Simonovits. 2014. 'Publication Bias in the Social Sciences: Unlocking the File Drawer.' Science 345 (6203): 1502-1505, https://doi.org/10.1126/science.1255484. Přejít k původnímu zdroji...
  18. Goerres, A., M. B. Siewert and C. Wagemann. 2019. 'Internationally Comparative Research Designs in the Social Sciences: Fundamental Issues, Case Selection Logics, and Research Limitations.' KZfSS Kölner Zeitschrift für Soziologie Und Sozialpsychologie 71: 75-97, https://doi.org/10.1007/s11577-019-00600-2. Přejít k původnímu zdroji...
  19. Groves, R. M. 2011. 'Three Eras of Survey Research.' Public Opinion Quarterly 75 (5): 861-871, https://doi.org/10.1093/poq/nfr057. Přejít k původnímu zdroji...
  20. Hadler, M., A. Eder, M. Haller and F. Höllinger. 2015. 'Methodological Problems of Quantitative Comparative Social Research.' International Journal of Sociology 45 (4): 247-255, https://doi.org/10.1080/00207659.2015.1098056. Přejít k původnímu zdroji...
  21. Haerpfer, C. and K. A. Kizilova. 2016. 'The World's Largest Social Science Infrastructure and Academic Survey Research Program: The World Values Survey in the New Independent States.' RUDN Journal of Sociology 16 (4): 719-741.
  22. Harden, A. 2010. 'Mixed Methods Systematic Reviews: Integrating. Quantitative and Qualitative Findings.' FOCUS - Technical Brief No. 25. Retrieved 8 July 2020 (https://ktdrr.org/ktlibrary/articles_pubs/ncddrwork/focus/focus25/Focus25.pdf).
  23. Harkness, J. 1999. 'In Pursuit of Quality: Issues for Cross-National Survey Research.' International Journal of Social Research Methodology 2 (2): 125-140, https://doi.org/10.1080/136455799295096. Přejít k původnímu zdroji...
  24. Harzing, A.-W. 2012. 'A Preliminary Test of Google Scholar as a Source for CitationData: A Longitudinal Study of Nobel Prize Winners.' Scientometrics 94 (3): 1057-1075, https://doi.org/10.1007/s11192-012-0777-7. Přejít k původnímu zdroji...
  25. Heath, A., S. Fisher and S. Smith. 2005. 'The Globalization of Public Opinion Research.' Annual Review of Political Science 8 (1): 297-333, https://doi.org/10.1146/annurev.polisci.8.090203.103000. Přejít k původnímu zdroji...
  26. Jacobides, M. G., C. Cennamo and A. Gawer. 2018. 'Towards a Theory of Ecosystems.' Strategic Management Journal 39 (8): 2255-2276, https://doi.org/10.1002/smj.2904. Přejít k původnímu zdroji...
  27. Jeffers, K., M. King, L. Cleveland and P. Kelly Hall. 2017. 'Data Resource Profile: IPUMS-International.' International Journal of Epidemiology 46 (2): 390-391, https://doi.org/10.1093/ije/dyw321. Přejít k původnímu zdroji...
  28. Johnston, M.P. 2017. 'Secondary Data Analysis: A Method of Which the Time Has Come.' Qualitative and Quantitative Methods in Libraries (QQML) 3: 619-626.
  29. Jowell, R. 1998. 'How Comparative Is Comparative Research?' American Behavioral Scientist 42 (2): 168-177, https://doi.org/10.1177/0002764298042002004. Přejít k původnímu zdroji...
  30. Jowell, R., C. Roberts, R. Fitzgerald and G. Eva. 2007. Measuring Attitudes Cross-Nationally. Lessons from the European Social Survey. London: Sage, https://doi.org/10.4135/9781849209458. Přejít k původnímu zdroji...
  31. Keeter, S. 2012. 'Presidential Address: Survey Research, Its New Frontiers, and Democracy.' Public Opinion Quarterly 76 (3): 600-608, https://doi.org/10.1093/poq/nfs044. Přejít k původnímu zdroji...
  32. Klingemann, H.-D. 2017. 'Prolegomena to a Comparative Second-order Analysis of the World Values Survey.' Pp. 133-144 in Societal Enlightenment in Turbulent Times, edited by B. Malnar and K. Müller. Vienna: Echoraum.
  33. Kroneberg, C. 2019. 'Theory Development in Comparative Social Research.' KZfSS Kölner Zeitschrift für Soziologie Und Sozialpsychologie 71: 29-51, https://doi.org/10.1007/s11577-019-00604-y. Přejít k původnímu zdroji...
  34. Kuechler, M. 1998. 'The Survey Method: An Indispensable Tool for Social Science Research Everywhere?' American Behavioral Scientist 42 (2): 178-200, https://doi.org/10.1177/0002764298042002005. Přejít k původnímu zdroji...
  35. Lagos, M. 2008. 'International Comparative Surveys: Their Purpose, Content and Methodological Challenges.' Pp. 580-593 in The SAGE Handbook of Public Opinion Research, edited by W. Donsbach and M. Traugott. London: SAGE, https://doi.org/10.4135/9781848607910.n55. Přejít k původnímu zdroji...
  36. Lindstrøm, M. D. and K. Kropp. 2017. 'Understanding the Infrastructure of European Research Infrastructures - The Case of the European Social Survey (ESS-ERIC).' Science and Public Policy 44 (6): 855-864, https://doi.org/10.1093/scipol/scx018. Přejít k původnímu zdroji...
  37. Lyberg, L., L. Japec and C. Tongur. 2019. 'Prevailing Issues and the Future of Comparative Surveys.' Pp. 1053-1082 in Advances in Comparative Survey Methods, edited by T. Johnson, B. Pennell, I. Stoop and B. Dorer. Hoboken: Wiley, https://doi.org/10.1002/9781118884997.ch48. Přejít k původnímu zdroji...
  38. Lynn, P. 2003. 'Developing Quality Standards for Cross-National Survey Research: Five Approaches.' International Journal of Social Research Methodology 6 (4): 323-336, https://doi.org/10.1080/13645570210132848. Přejít k původnímu zdroji...
  39. Lynn, P., L. Japec and L. Lyberg. 2006. 'What's So Special about Cross-National Surveys?' ZUMA Nachrichten Spezial 12: 7-20.
  40. Malnar, B. 2019. ESS Annual Bibliographic Report 2019 (ESS ERIC Deliverable 11.8). Retrieved 21 May 2020 (http://www.europeansocialsurvey.org/docs/findings/ESS-ERIC-Annual-Bibliographic-Report-2019.pdf).
  41. Massey, D. S. and R. Tourangeau. 2013. 'Introduction. New Challenges to Social Measurement.' The ANNALS of the American Academy of Political and Social Science 645 (1): 6-22, https://doi.org/10.1177/0002716212463314. Přejít k původnímu zdroji...
  42. Mathison, S. 1988. 'Why Triangulate?' Educational Researcher 17 (2): 13-17, https://doi.org/10.3102/0013189X017002013. Přejít k původnímu zdroji...
  43. Mayr, P. and A.-K. Walter. 2007. 'An Exploratory Study of Google Scholar.' Online Information Review 31 (6): 814-830, https://doi.org/10.1108/14684520710841784. Přejít k původnímu zdroji...
  44. Mochmann, E. 2008. 'Improving the Evidence Base for International Comparative Research.' International Social Science Journal 59 (193): 489-506, https://doi.org/10.1111/j.1468-2451.2009.01681.x. Přejít k původnímu zdroji...
  45. Müller-Bloch, C. and J. Kranz. 2015. 'A Framework for Rigorously Identifying Research Gaps in Qualitative Literature Reviews.' Proceedings of the 36th International Conference on Information Systems (ICIS), Fort Worth, USA, 13-16 December 2015.
  46. NASEM - National Academies of Sciences, Engineering, and Medicine. 2017. Federal Statistics, Multiple Data Sources, and Privacy Protection: Next Steps. Washington, DC: The National Academies Press.
  47. Norris, P. 2009. 'The Globalization of Comparative Public Opinion Research.' Pp. 520-540 in The Sage Handbook of Comparative Politics, edited by T. Landmann and N. Robinson. Thousand Oak, CA: Sage. Přejít k původnímu zdroji...
  48. Oh, D.-S., F. Phillips, S. Park and E. Lee. 2016. 'Innovation Ecosystems: A Critical Examination.' Technovation 54: 1-6, https://doi.org/10.1016/j.technovation.2016.02.004. Přejít k původnímu zdroji...
  49. Oliver, S., A. Harden, R. Rees, J. Shepherd, G. Brunton, J. Garcia and A. Oakley. 2005. 'An Emerging Framework for Including Different Types of Evidence in Systematic Reviews for Public Policy.' Evaluation 11 (4): 428-446, https://doi.org/10.1177/1356389005059383. Přejít k původnímu zdroji...
  50. Olshannikova, E., T. Olsson, J. Huhtamäki and H. Kärkkäinen. 2017. 'Conceptualizing Big Social Data.' Journal of Big Data 4 (1). Retrieved 20 July 2020 (https://journalofbigdata.springeropen.com/articles/10.1186/s40537-017-0063-x#citeas), https://doi.org/10.1186/s40537-017-0063-x. Přejít k původnímu zdroji...
  51. Pennell, B., K. C. Hibben, L. E. Lyberg, P. P. Mohler and G. Worku. 2017. 'A Total Survey Error Perspective on Surveys in Multinational, Multiregional, and Multicultural Contexts.' Pp. 179-201 in Total Survey Error in Practice, edited by P. Biemer, E. de Leeuw, S. Eckman, B. Edwards, F. Kreuter, L. Lyberg, C. Tucker and B. West. Hoboken, NJ: Wiley, https://doi.org/10.1002/9781119041702.ch9. Přejít k původnímu zdroji...
  52. Petticrew, M. and H. Roberts. 2006. Systematic Reviews in the Social Sciences. A Practical Guide. Malden, MA: Blackwell, https://doi.org/10.1002/9780470754887. Přejít k původnímu zdroji...
  53. Quandt, M. and R. Luijkx. 2015. 'Data Bases and Statistical Systems: International Comparative Research.' Pp. 789-797 in International Encyclopedia of the Social & Behavioral Sciences, Volume 5, edited by J. D. Wright. 2nd ed. Oxford: Elsevier, https://doi.org/10.1016/B978-0-08-097086-8.41017-2. Přejít k původnímu zdroji...
  54. Roberts, G. and D. Binder. 2009. 'Analyses Based on Combining Similar Information from Multiple Surveys.' Pp. 2138-2147 in Proceedings of the Joint Statistical Meetings Section on Survey Methods Research. Retrieved 10 November 2021 (https://www.semanticscholar.org/paper/Analyses-Based-on-Combining-Similar-Information-Roberts-Binder/70ef9ec615640c12f029fe0d026b0166ca88e2a7).
  55. Schenker, N., J. F Gentleman, D. Rose, E. Hing and I. M. Shimizu. 2002. 'Combining Estimates from Complementary Surveys: A Case Study Using Prevalence Estimates from National Health Surveys of Households and Nursing Homes.' Public Health Reports 117 (4): 393-407, https://doi.org/10.1016/S0033-3549(04)50177-2. Přejít k původnímu zdroji...
  56. Schenker, N. and T. E. Raghunathan. 2007. 'Combining Information from Multiple Surveys to Enhance Estimation of Measures of Health.' Statistics in Medicine 26 (8): 1802-1811, https://doi.org/10.1002/sim.2801. Přejít k původnímu zdroji...
  57. Seppanen, M., S. Hyrynsalmi, K. Manikas and A. Suominen. 2017. 'Yet Another Ecosystem Literature Review: 10+1 Research Communities.' Pp. 1-8 in 2017 IEEE Přejít k původnímu zdroji...
  58. European Technology and Engineering Management Summit (E-TEMS). Munich: IEEE Institute of Electrical and Electronic Careers, https://doi.org/10.1109/E-TEMS.2017.8244229. Přejít k původnímu zdroji...
  59. Smith, T. W. 2019. 'Improving Multinational, Multiregional, and Multicultural (3MC) Comparability Using the Total Survey Error (TSE) Paradigm.' Pp. 13-44 in Advances in Comparative Survey Methods: Multinational, Multiregional, and Multicultural Contexts, edited by T. P. Johnson, B. Pennell, I. Stoop and B. Dorer. Hoboken, NJ: Wiley, https://doi.org/10.1002/9781118884997.ch2. Přejít k původnímu zdroji...
  60. Tengö, M., E. S. Brondizio, T. Elmqvist, P. Malmer and M. Spierenburg. 2014. 'Connecting Diverse Knowledge Systems for Enhanced Ecosystem Governance: The Multiple Evidence Base Approach.' AMBIO 43 (5): 579-591, https://doi.org/10.1007/s13280-014-0501-3. Přejít k původnímu zdroji...
  61. Tomescu-Dubrow, I. and K. M. Slomczynski. 2014. 'Democratic Values and Protest Behavior: Data Harmonization, Measurement Comparability, and Multi-Level Modeling in Cross-National Perspective.' Ask: Research and Methods 23 (1): 103-114.
  62. United Nations Economic Commission for Europe (UNECE). 2017. Guide on Poverty Measurement. New York and Geneva: United Nations.
  63. Vezzoni, C. 2015. 'Secondary Analysis in the Social Sciences and Its Relation to Futures Studies.' On the Horizon 23 (2): 128-139, https://doi.org/10.1108/OTH-02-2015-0006. Přejít k původnímu zdroji...
  64. Ware, M. and M. Mabe. 2012. The STM Report. An Overview of Scientific and Scholarly Journal Publishing. The Hague: International Association of Scientific, Technical and Medical Publishers.
  65. White, P. and R. Saylor Breckenridge. 2014. 'Trade-Offs, Limitations, and Promises of Big Data in Social Science Research.' Review of Policy Research 31 (4): 331-338, https://doi.org/10.1111/ropr.12078. Přejít k původnímu zdroji...
  66. Winters, K. and S. Netscher. 2016. 'Proposed Standards for Variable Harmonization Documentation and Referencing: A Case Study Using QuickCharmStats 1.1.' PLOS ONE 11(2): e0147795, https://doi.org/10.1371/journal.pone.0147795. Přejít k původnímu zdroji...
  67. Wysmułek, I. 2018. 'Europe of Uneven Data: Country Representation in International Surveys on Corruption, 1989-2017.' Ask: Research and Methods 27 (1): 87-104, https://doi.org/10.18061/ask.v27i1.0005. Přejít k původnímu zdroji...

Tento článek je publikován v režimu tzv. otevřeného přístupu k vědeckým informacím (Open Access), který je distribuován pod licencí Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0), která umožňuje nekomerční distribuci, reprodukci a změny, pokud je původní dílo řádně ocitováno. Není povolena distribuce, reprodukce nebo změna, která není v souladu s podmínkami této licence.