Staff

Journal article

Schmidt, Felix, Frank Mangold, Sebastian Stier, and Roberto Ulloa. 2024. "Facebook as an Avenue to News: A Comparison and Validation of Approaches to Identify Facebook Referrals." Political Communication online first. doi: https://doi.org/10.1080/10584609.2024.2342983.

Kirkizh, Nora, Roberto Ulloa, Sebastian Stier, and Jürgen Pfeffer. 2024. "Predicting political attitudes from web tracking data: a machine learning approach." Journal of Information Technology & Politics online first. doi: https://doi.org/10.1080/19331681.2024.2316679.

Zucker, Eve M., Mykola Makhortykh, Roberto Ulloa, Daniel Bultmann, and David Simon. 2023. "AI and Archives: How can Technology Help Preserve Holocaust Heritage Under the Risk of Disappearance?" Eastern European Holocaust Studies 1 (2): 357-363. doi: https://doi.org/10.1515/eehs-2023-0052.

Urman, Aleksandra, Mykola Makhortykh, Roberto Ulloa, Maryna Sydorova, and Juhi Kulshrestha. 2023. "Constants and Variables: How Does the Visual Representation of the Holocaust by AI Change Over Time." Eastern European Holocaust Studies 1 (2): 365-371. doi: https://doi.org/10.1515/eehs-2023-0055.

Ulloa, Roberto, Mykola Makhortykh, Aleksandra Urman, and Juhi Kulshrestha. 2023. "Novelty in News Search: A Longitudinal Study of the 2020 US Elections." Social Science Computer Review 42 (3). doi: https://doi.org/10.1177/08944393231195471.

Ulloa, Roberto, and Celina Kacperski. 2023. "Search engine effects on news consumption: Ranking and representativeness outweigh familiarity in news selection." New Media & Society online first. doi: https://doi.org/10.1177/14614448231154926.

Makhortykh, Mykola, Eve M. Zucker, David Simon, Daniel Bultmann, and Roberto Ulloa. 2023. "Shall androids dream of genocides? How generative AI can change the future of memorialization of mass atrocities." Discover Artificial Intelligence 1 (3): 28. doi: https://doi.org/10.1007/s44163-023-00072-6.

Zagovora, Olga, Roberto Ulloa, Katrin Weller, and Fabian Flöck. 2022. ""I updated the <ref>": The evolution of references in the English wikipedia and the implications for altmetrics." Quantitative Science Studies 3 (1): 147-173. doi: https://doi.org/10.1162/qss_a_00171.

Urman, Aleksandra, Mykola Makhortykh, and Roberto Ulloa. 2022. "Auditing the representation of migrants in image web search results." Humanities and Social Sciences Communications 9 (130). doi: https://doi.org/10.1057/s41599-022-01144-1.

Kacperski, Celina, Roberto Ulloa, Sonja Klingert, Benedikt Kirpes, and Florian Kutzner. 2022. "Impact of incentives for greener battery electric vehicle charging: A field experiment." Energy Policy 161 (February): 112752. doi: https://doi.org/10.1016/j.enpol.2021.112752.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2022. "Memory, counter-memory and denialism: How search engines circulate information about the Holodomor-related memory wars." Memory Studies 15 (6): 1330–1345. doi: https://doi.org/10.1177/17506980221133732.

Ulloa, Roberto, Ana Carolina Richter, Mykola Makhortykh, Aleksandra Urman, and Celina Kacperski. 2022. "Representativeness and face-ism: Gender bias in image search." New Media & Society 26 (6). doi: https://doi.org/10.1177/14614448221100699.

Ulloa, Roberto, Mykola Makhortykh, and Aleksandra Urman. 2022. "Scaling up search engine audits: Practical insights for algorithm auditing." Journal of Information Science 50 (2). doi: https://doi.org/10.1177/01655515221093029.

Aleksandra, Urman, Mykola Makhortykh, and Roberto Ulloa. 2022. "The matter of chance: auditing web search results related to the 2020 US presidential primary elections across six search engines." Social Science Computer Review 40 (5): 1323–1339. https://doi.org/10.1177/08944393211006863.

Makhortykh, Mykola, Aleksandra Urman, Teresa Gil-Lopez, and Roberto Ulloa. 2022. "To track or not to track: Examining perceptions of online tracking for information behavior research." Internet Research 32 (7): 1066-2243. doi: https://doi.org/10.1108/INTR-01-2021-0074.

Urman, Aleksandra, Mykola Makhortykh, Roberto Ulloa, and Juhi Kulshrestha. 2022. "Where the earth is flat and 9/11 is an inside job: A comparative algorithm audit of conspiratorial information in web search results." Telematics and Informatics 72 (August): 101860. doi: https://doi.org/10.1016/j.tele.2022.101860.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2021. "Hey, Google, is this what the Holocaust looked like? Auditing algorithmic curation of visual historical content on Web search engines." First Monday 26 (10). doi: https://doi.org/10.5210/fm.v26i10.11562.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2020. "How search engines disseminate information about COVID-19 and why they should do better." The Harvard Kennedy School (HKS) Misinformation Review 1 (3): 1-12. doi: https://doi.org/10.37016/mr-2020-017.

Ulloa, Roberto. 2016. "CulSim: A simulator of emergence and resilience of cultural diversity." doi: https://doi.org/10.1016/j.softx.2016.07.005.

Kacperski, Celina, Roberto Ulloa, and Craig Hall. 2016. "Do athletes imagine being the best, or crossing the finish line first? A mixed methods analysis of construal levels in elite athletes’ spontaneous imagery." Journal of Mixed Methods Research 13 (2): 216-241. doi: https://doi.org/10.1177/1558689816674563.

Ulloa, Roberto, Celina Kacperski, and Fernando Sancho-Caparrini. 2016. "Institutions and cultural diversity: Effects of democratic and propaganda processes on local convergence and global diversity." PLoS ONE 11 (4): e0153334. doi: https://doi.org/10.1371/journal.pone.0153334.

Chapter in an edited book

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2023. "This Is What Pandemic Looks Like: Visual Framing of COVID-19 on Search Engines." 1. In COVID Communication: Exploring Pandemic Discourse, edited by Douglas A. Vakoch, John C. Pollock, and Amanda M. Caleb, 113-123. Springer, Cham. doi: https://doi.org/10.1007/978-3-031-27665-1_9.

Urman, Aleksandra, Mykola Makhortykh, and Roberto Ulloa. 2021. "Auditing source diversity bias in video search results using virtual agents ." In WWW '21: Companion Proceedings of the Web Conference 2021, edited by Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia, 232-236. New York: ACM. doi: https://doi.org/10.1145/3442442.3452306.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2021. "Detecting Race and Gender Bias in Visual Representation of AI on Web Search Engines." In Advances in Bias and Fairness in Information Retrieval : Second International Workshop on Algorithmic Bias in Search and Recommendation, BIAS 2021, Lucca, Italy, April 1, 2021, Proceedings, edited by Ludovico Boratto, Stefano Faralli, Mirko Marras, and Giovanni Stilo, Communications in Computer and Information Science 1418, 36-50. Springer, Cham. doi: https://doi.org/10.1007/978-3-030-78818-6_5.

Ulloa, Roberto, and Tom Froese. 2016. "Nobility-targeting raids among the Classic Maya: Cooperation in scale-free networks persists under tournament attack when population size fluctuates." In Proceedings of the Artificial Life Conference 2016, 472-479. MIT Press. doi: https://doi.org/10.7551/978-0-262-33936-0-ch077.

Ulloa, Roberto. 2012. "Snapshot of the position of a site in the Internet: Vector space model applied to external pages." In Proceedings of the XXXVIII Latin American Informatics Conference, 1-7. IEEE. doi: https://doi.org/10.1109/CLEI.2012.6427223.

Ulloa, Roberto, and Gabriela Barrantes. 2009. "Parallel opponent modelling with knowledge domain: Direct search of strategies in multi-agent systems." In Proceedings of the XXXV Latin American Informatics Conference, CLEI. https://clei.org/proceedings_data/CLEI2009/56887.pdf.

Ulloa, Roberto, and Gabriela Barrantes. 2008. "Building a scenario for the simulation of attacks on systems of agents." SCCC. In Jornadas Chilenas de Computación, doi: https://doi.org/10.13140/RG.2.1.4785.4324.

Monograph

Ulloa, Roberto. 2016. Cultural Diversity in Artificial Societies: Case Studies of the Maya Peoples. Electronic Thesis and Dissertation Repository. The University of Western Ontario. https://ir.lib.uwo.ca/etd/4231.

Ulloa, Roberto. 2015. Kivy: Interactive Applications and Games in Python. 2. Packt Publishing.

Ulloa, Roberto. 2013. Kivy: Interactive Applications in Python . 1. Packt Publishing.

Ulloa, Roberto. 2011. Uso de aleatorización como un mecanismo de defensa frente a la inferencia de estrategias en sistemas multiagente. MSc Dissertation. Universidad de Costa Rica. Sistema de Estudios de Posgrado. http://repositorio.sibdi.ucr.ac.cr:8080/jspui/handle/123456789/2545.

Working and discussion paper

Zucker, Eve M., Roberto Ulloa, David Simon, Mykola Makhortykh, and Daniel Bultmann. 2024. Archiving Holocaust Digital Memorialization: The Lublin District Camps. Digital Archive of Memorialization of Mass Atrocities (DAMMA) Workshop Whitepaper..

Adam, Silke, Mykola Makhortykh, Michaela Maier, Viktor Aigenseer, Aleksandra Urman, Teresa Gil-Lopez, Clara Christner, Ernesto de León, and Roberto Ulloa. 2024. Improving the quality of individual-level online information tracking: challenges of existing approaches and introduction of a new content- and long-tail sensitive academic solution. https://arxiv.org/abs/2403.02931.

Kacperski, Celina, Roberto Ulloa, Jeremy Wautelet, Tobias Vogel, and Florian Kutzner. 2023. Creating inclusive mobility systems: towards age and education sensitive interventions for enhancing autonomous vehicle acceptance. https://arxiv.org/abs/2311.16780.

Kacperski, Celina, Mona Bielig, Mykola Makhortykh, Maryna Sydorova, and Roberto Ulloa. 2023. Examining bias perpetuation in academic search engines: an algorithm audit of Google and Semantic Scholar. https://arxiv.org/abs/2311.09969.

Kacperski, Celina, Roberto Ulloa, Denis Bonnay, Juhi Kulshrestha, Peter Selb, and Andreas Spitz. 2023. Who are the users of ChatGPT? Implications for the digital divide from web tracking data. https://arxiv.org/abs/2309.02142.

Bultmann, Daniel, Mykola Makhortykh, David Simon, Roberto Ulloa, and Eve M. Zucker. 2022. Digital Archive of Memorialization of Mass Atrocities (DAMMA): Workshop Whitepaper. Yale University Genocide Studies Program. https://gsp.yale.edu/digital-archive-memorialization-mass-atrocities-damma-workshop-whitepaper.

Ulloa, Roberto, and Celina Kacperski. 2020. A simulation of the resilience of cultural diversity in the face of large-scale events. ArXiV Preprint. http://arxiv.org/abs/2003.05322.

Variable Report

2012. Recognition between Baroque and Renaissance Style of Classic Paintings. doi: https://doi.org/10.13140/RG.2.1.3154.9203.

Presentation at a conference

Ulloa, Roberto, Celina Kacperski, Juhi Kulshrestha, Andreas Spitz, Denis Bonnay, and Peter Selb. 2023. "An experimental study of online information seeking on policy judgments." General Online Research Conference 2023 (GOR 23), University of Kassel, 2023-09-20.

Schmidt, Felix, Frank Mangold, Sebastian Stier, and Roberto Ulloa. 2023. "Facebook as an avenue to news: A comparison and validation of approaches to identify Facebook referrals." 73rd Conference of the International Communication Association (ICA), Toronto, Canada, 2023-05-27.

Ulloa, Roberto, Mykola Makhortykh, and Aleksandra Urman. 2022. "Algorithm Auditing at a Large-Scale: Insights From Search Engine Audits." 72nd Annual International Communication Association Conference (ICA), Online, 2022-05-26.

Makhortykh, Mykola, Aleksandra Urman, Roberto Ulloa, and Juhi Kulshrestha. 2022. "Comparative Algorithm Audit of Representation of Mass Atrocities on Web Search Engines." Poster on 72nd Annual International Communication Association Conference (ICA), 2022-05-26.

Kuznetsova, Elizaveta D., Mykola Makhortykh, Aleksandra Urman, and Roberto Ulloa. 2022. "Media Representations and Bias in Search Engines: Framing of the Russian COVID-19 Vaccine." 72nd Annual International Communication Association Conference (ICA), Paris, 2022-05-26.

Maier, Michaela, Viktor Aigenseer, Mykola Makhortykh, Roberto Ulloa, Aleksandra Urman, Clara Christner, and Teresa Gil-Lopez. 2022. "One Does Not Simply Analyze Tracking Data: Challenges of Utilizing Large-Scale Tracking Collections for Communication Research." 72nd Annual International Communication Association Conference (ICA), 2022-05-26.

Sen, Indira, Roberto Ulloa, Aleksandra Urman, Mykola Makhortykh, and Celina Kacperski. 2022. "Refugee or Expat, Hero or Threat: Migrant Queries in Google News Search Results." AoIR2022 , 2022-11-02.

Ulloa, Roberto, Ana Carolina Richter, Mykola Makhortykh, Aleksandra Urman, and Celina Kacperski. 2022. "Representativeness and Face-Ism: Gender Bias in Image Search." 72nd Annual International Communication Association Conference (ICA), 2022-05-27.

Urman, Aleksandra, Mykola Makhortykh, Roberto Ulloa, and Juhi Kulshrestha. 2022. "Where the Earth Is Flat and 9/11 Is an Inside Job: A Comparative Algorithm Audit of Conspiratorial Information in Web Search Results." 72nd Annual International Communication Association Conference (ICA), 2022-05-26.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2021. "Auditing algorithmic content curation on search engines using virtual agents." Virtual 71st Annual International Communication Association Conference (ICA), 2021-05-27.

Urman, Aleksandra, Mykola Makhortykh, and Roberto Ulloa. 2021. "Auditing source diversity bias in video search results using virtual agents ." 3rd International Workshop on Algorithmic Bias in Search and Recommendation (FATES 2021), 2021-04-12.

Urman, Aleksandra, Mykola Makhortykh, and Roberto Ulloa. 2021. "Detecting race and gender bias in visual representation of AI on web search engines." 2nd International Workshop on Algorithmic Bias in Search and Recommendation (BIAS 2021), Online, 2021-04-01. doi: https://doi.org/10.1177/08944393211006863.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2021. "Hey, Google, is it what the Holocaust really looked like? Auditing biases in visual representation of the Holocaust on web search engines." Virtual 71st Annual International Communication Association Conference (ICA), 2021-05-27.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2021. "Hey, Google, tell me what the Holocaust looked like: Visual framing of mass atrocities by search engines." Three-Country Conference on Communication Science (DACH 21), Online, 2021-04-08.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2021. "Scaling virtual agent-based testing for cross-platform analysis of algorithmic content curation." Three-Country Conference on Communication Science (DACH 21), Online, 2021-04-07.

Urman, Aleksandra, Mykola Makhortykh, and Roberto Ulloa. 2021. "Visual representation of migrants in Web search results." Virtual 71st Annual International Communication Association Conference (ICA), 2021-05-27.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2020. " The more knowledge, the more grief: Auditing how search engine algorithms structure memories about mass atrocities." Virtual Digital Humanities Benelux 2020, 2020-06-03.

Urman, Aleksandra, Mykola Makhortykh, and Roberto Ulloa. 2020. "Auditing search engines: The case of the 2020 US presidential election campaign." Virtual: Poster on Socinfo 2020, 2020-10-06.

Makhortykh, Mykola, Aleksandra Urman, and Roberto Ulloa. 2020. "This is what pandemic looks like: Visual framing of COVID-19 on search engines." 14th General Conference (ECPR 2020), 2020-08-24.

Ulloa, Roberto. 2020. "To track or not to track: Examining perceptions of online tracking in the context of information behaviour research." Virtual 70th Annual International Communication Association Conference (ICA), 2020-05-21.

Urman, Aleksandra, Mykola Makhortykh, Teresa Gil-Lopez, and Roberto Ulloa. 2020. "Would you kindly let us track you? Examining perceptions of online tracking in the context of information behaviour research." 9th biennial Surveillance & Society conference of the Surveillance Studies Network, Erasmus University Rotterdam, Rotterdam, Netherlands, 2020-06-07.

Ulloa, Roberto. 2016. "Nobility-targeting raids among the Classic Maya: Cooperation in scale-free networks persists under tournament attack when population size fluctuates." Poster on XV Conference on the Synthesis and Simulation of Living Systems.

Kacperski, Celina, Roberto Ulloa, and Craig Hall. 2014. "A qualitative analysis of construal levels in sport imagery." ASP - Performing under Pressure.

Suárez, Juan Luis, Javier de la Rosa, and Roberto Ulloa. 2013. "Not Exactly Prima Facie: Understanding the Representation of the Human Through the Analysis of Faces in World Painting." Poster on Digital Humanities 2013 Conference.

Ulloa, Roberto. 2012. "Snapshot of the position of a site in the Internet: Vector space model applied to external pages." XXXVIII Latin American Informatics Conference, 2013-03-05.

Ulloa, Roberto. 2009. "Parallel opponent modelling with knowledge domain: Direct search of strategies in multiagent systems." XXXV Latin American Informatics Conference.

Ulloa, Roberto. 2008. "Building a scenario for the simulation of attacks on systems of agents." Jornadas Chilenas de Computación. doi: https://doi.org/10.13140/RG.2.1.4785.4324.

Presentation not at a conference

Ulloa, Roberto. 2021. "Auditing algorithms: How platform technologies shape our digital environment." Meet the Experts: GESIS online talks on Social Science Methods and Research Data, Online, 2021-10-07. https://www.gesis.org/en/services/sharing-knowledge/consulting-and-guidelines/meet-the-experts.

Ulloa, Roberto. 2021. "Introduction to online data acquisition." Meet the Experts: GESIS online talks on Social Science Methods and Research Data, Online, 2021-09-30. https://www.gesis.org/en/services/sharing-knowledge/consulting-and-guidelines/meet-the-experts.

Ulloa, Roberto. 2016. "CulSim: Emergence and Resilience of Cultural Diversity in a Computer Simulator." Poster on 3rd GESIS Computational Social Science Winter Symposium.

Seminar

Samory, Mattia, Roberto Ulloa, Arnim Bleier, Indira Sen, Olga Zagovora, and Leon Froehling. 2021. "Web Data Collection and Natural Language Processing in Python." GESIS - Leibniz-Institut für Sozialwissenschaften, Köln.

Bleier, Arnim, Indira Sen, Mattia Samory, Roberto Ulloa, and Olga Zagovora. 2021. "Web Data Collection and Natural Language Processing in Python." GESIS - Leibniz-Institut für Sozialwissenschaften, Köln.

Other

Flöck, Fabian, Kenan Erdogan, Roberto Ulloa, and Maribel Acosta. 2015. "WikiWho Web Service : Extraction of Wikipedia Editor Interactions and Change Tracking." http://www.wikiwho.net.