The D.A.R.E. Workshop

as a part of the AAAI ICWSM 2023 conference at Limassol, Cyprus

Disrupt, Ally, Resist, Embrace (DARE):
Action Items for Computational Social Scientists in a Changing World

A gathering to discuss the emerging dilemmas around the principles and practice of computational social science research in a changing technological landscape.

Motivation

In the past decade, many sophisticated AI-powered tools have been developed and released to the scientific community and the public at large. At the same time, the socio-technical platforms that are at the center of our observations have transformed in unanticipated ways. Many of these developments have occurred against a backdrop of political and social polarization, and, public health and macroeconomic crises, which offer multiple lenses to contextualize (or distort) scientific reflexivity. To computational social scientists who study computer-mediated human behavior, these on- and offline changes have real implications on whom they study, and how they study them. How, then, should the ICWSM community members act in such a changing world? Which disruptions should they embrace and which ones should they resist? Whom do they ally with, and for what purpose? In this workshop, we invite experience-based perspectives on these issues, aimed at debating and drafting a future research agenda that we want to pursue together. The goal of this full-day workshop is to facilitate collaboration on position papers among its attendees, each of which must propose an actionable item for future computational science research.

Participate

In order to participate in the workshop, we invite two types of submissions:

  1. Short, 200-word statements of interest that express a desire to participate in the workshop discussion, by positioning oneself with respect to the issues discussed below.
  2. Slightly longer, 2-page (in AAAI format) extended abstracts that detail one’s position on one of the issues discussed below. Accepted abstracts will appear in the workshop proceedings.

Both types of submissions can be made at the submission portal on Easychair. Given limited space, preference will be given to those who submitted extended abstracts.

The position papers should, at a high level, address concerns with processes of and the principles underlying computational social science research, and how they are often, and repeatedly, disrupted by platform politics, new technologies, their implications, and their unknowables. By problems of process, we refer to, for example, the fact that the increasing availability of proprietary AI tools has created challenges for the research process (e.g., is the peer review process ready to handle papers co-written by LLMs?) as well as the researchers themselves (e.g., who can use these tools, and what would this imply for those who can not or may not?). We invite submissions that address problems in either the research process or impacts on yourself or others as a researcher.

With respect to issues of principle, we refer to the fact that recent events related to questionable technology takeovers and layoffs, exposes on techno-political alliances, and questionable labor practices at large technology companies create new dilemmas for researchers collecting, annotating, and analyzing online data. New technologies and tools may also spell unfavorable consequences for those who lack the computational resources to use them, who may even consider pivoting their research directions and methods. For instance, what does it mean to make privately-owned LLMs central to one’s research? Do they create dilemmas for “conscientious objectors”, who might refuse to use such tools on ethical grounds? Finally, note that position papers may be anonymous, in which case they will be (if accepted) shared anonymously with workshop participants.

Position papers should be grounded in evidence, prior published work, and ideally, also personal experiences. Examples of the position papers we seek can be found here, here, here, or they can be responses to news stories like this one. Ideally, position papers should respond to the provided prompts below (i.e., concerns about processess and principles), although we will also consider papers that do not explicitly respond to a prompt, but discuss an interesting and relevant problem pertinent to this discussion.

Prompts

1. What are the key principles that should guide the development and use of large language models, and how can we ensure that they are adhered to in practice? 2. Do the model cards speak to the generalizability of large language models across different domains and languages? Are they sufficiently detailed? 3. How can we ensure the reproducibility and replicability, particularly in light of their complexity and resource requirements? 4. How can we ensure that output evaluation is appropriate and meaningful? 5. How can we promote responsible and ethical research practices in the development and use of large language models, particularly given their potential impact on society? 6.What are the most promising approaches for mitigating bias and improving fairness in large language models, and how can we ensure that they are implemented in practice?

1. How can we ensure that the use of AI and LLMs in peer reviewing is transparent, fair, and accurate, and how can we address concerns about the potential impact on human reviewers and the quality of research? 2. What are the biggest challenges faced by peer reviewers and editors in the current research landscape, and how have these challenges been exacerbated by the pandemic? 3. What are the potential benefits and drawbacks of using AI and LLMs for peer reviewing and research assessment, and what are the ethical considerations that need to be taken into account? 4. What are the most effective strategies for improving the peer review process, such as breaking down the reviewing process into multiple stages or implementing incentives for reviewers? 5. What are the most effective strategies for improving the peer review process, such as breaking down the reviewing process into multiple stages or implementing incentives for reviewers? 6. What are the most effective strategies for improving the peer review process, such as breaking down the reviewing process into multiple stages or implementing incentives for reviewers?

1. What are the most promising technological solutions for enabling secure and privacy-preserving data sharing, and how can we ensure that these solutions are accessible and scalable for researchers? 2. What are potential new paradigms to promote collaboration and knowledge-sharing among researchers working on large datasets across different disciplines and institutions? 3. How can we develop paradigms of mutual benefit between researchers and companies/platforms to encourage data sharing for research purposes, and what are the key components of such paradigms? 4. What are the potential impacts of sharing intellectual property on innovation and creativity, and how can we ensure that the incentives for innovation and creativity are maintained in the context of shared intellectual property? 5. How can we promote responsible and ethical research practices in the development and use of new technology in industry, particularly given their potential impact on society? 6. What are the challenges that companies/platforms face in sharing data for research purposes, and how can researchers and policymakers work to address these challenges?

Submit
Clicking here will take you to our EasyChair page.

Important Dates

Program

This is a full-day workshop. Our goal is to provide a venue in which participants will engage with the listed issues through various formats: keynote speakers and panels, collaborative debate-style breakouts, and lightning sessions.

Depending on whether participants (those who are invited or who will apply to participate) are able to present in person or remotely, we will create a program to take into account how to best serve both audiences. Therefore, more details on the program will be posted in the future weeks.

Speakers

Saiph Savage

Organizers

Kokil Jaidka

National University of Singapore

Eni Mustafaraj

Wellesley College

David Schoch

GESIS

Kenny Joseph

University of Buffalo