![Twitter: @annettemarkham
Web: annettemarkham.com
Annette Markham, PhD
.
Professor MSO
Information Studies & Digital Design
Aarhus University
[Reverse engineering,
reflexivity, and other
useful words for enacting
ethical methods]
Methods
as Ethics](https://image.slidesharecdn.com/dro3talk-170311153523/85/Methods-as-Ethics-1-320.jpg)
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (17)
Semelhante a Methods as Ethics
Semelhante a Methods as Ethics (20)
Último
Último (20)
Methods as Ethics
- 1. Twitter: @annettemarkham Web: annettemarkham.com Annette Markham, PhD . Professor MSO Information Studies & Digital Design Aarhus University [Reverse engineering, reflexivity, and other useful words for enacting ethical methods] Methods as Ethics
- 2. 1) How we move 2) Why it matters 3) How to rethink and reconfigure
- 4. AoIR Ethics Committee Microsoft Research Labs “Creating Future Memories” http://futuremaking.space Shifting from statements to questions over a ten-year revision Making abstract concepts like “ethics” more concrete like “avoiding the creepy factor” Building experimental frameworks For citizens to control their own future memory possibilities Creating more conscious responses and accounts
- 7. methods Conditions for inquiry Everyday practices of inquiry
- 8. methods Conditions for inquiry Everyday practices of inquiry How are our research sensibilities being framed? What frames are we teaching/training others to see? ethics
- 9. Ethics as general principles Ethics as regulated norms Ethics as a mindset or vision Ethics as everyday practice
- 10. Ethics as general principles Ethics as regulated norms Ethics as a mindset or vision Ethics as everyday practice
- 11. Respect (for a person’s autonomy and rights) Justice (fair distribution of benefits and risks; equitable treatment of all) Beneficence (action for the good of others, do no harm)
- 12. Human Subjects Informed Consent Privacy (and data) Protection Vulnerability Risk / Benefit Ratio *Institutional Research Boards
- 13. What is a Human Subject? How do we (should we) get Informed Consent? It’s (almost) impossible to protect privacy (PII) Vulnerability often occurs after the fact A Benefit/risks ratio is not a universal perspective
- 14. Shifting perspectives Regulation [error]-driven approaches Concept [regulated]-driven approaches Process-driven approaches
- 15. AOIR Ethics Guidelines 2012
- 16. AOIR Ethics Guidelines 2012
- 17. Regulation (error)-driven approaches Concept (regulated)-driven approaches Process (question)-driven approaches How can we avoid the creepy factor? How can we modify the concepts in order to match new complexities? What does the context require? What is the goal of research in the first place? Shifting perspectives
- 18. Regulation (error)-driven approaches Concept (regulated)-driven approaches Process (question)-driven approaches Future (impact)-oriented approaches Shifting perspectives
- 19. methods Conditions for inquiry Everyday practices of inquiry How are our research sensibilities being framed? What frames are we teaching others to see? ethics impact Future harms and possibilities based on our research practices Motivation for doing research in the first place
- 20. Impact framework for ethics and methods Impact Arena 1: Treatment of people (beyond human subject or participant) Impact Arena 2: Use of data to make categorizations, inferences and conclusions. Impact Arena 3: Unintended side effects of technology design, prototype testing, or research design Impact Arena 4: Future possibilities and harms related to production and deployment, or dissemination
- 21. Impact framework for ethics and methods Impact Arena 1: Treatment of people (beyond human subject or participant) Impact Arena 2: Use of data to make categorizations, inferences and conclusions. Impact Arena 3: Unintended side effects of technology design, prototype testing, or research design Impact Arena 4: Future possibilities and harms related to production and deployment, or dissemination
- 22. We must take to task the myth that method purifies subjectivity. Methods mold subjectivity, not into patterns that erase all emotions from the researchers’ sensing body but into patterns that produce emotions of a different order, and also into attitudes that too often privilege cognitively driven procedures and social research. (remixed from James Davies, 2015, p. 13)
- 24. Decision Points Critical Movements Generating Questions Determining case or field boundaries Accessing Participants or Materials Sorting. Filtering, and selecting’what counts’ Collecting Information Using particular analytical tools Representing self and other in reports Identifying objects of analysis Sorting, thematizing, categorizing Discarding information Interpreting findings Framing Knowledge for the audience .
- 25. Granularity Reverse Engineering Remix or Bricolage Reflexivity Layered Accounts Crystalization
Notas do Editor
- I wrap up these four keynote sessions by focusing our attention to how we study stuff in the world, to share some of my own thinking around ethics and methods. The image of an old typewriter here is meant to hint that in this talk, I want to invoke thoughts of how we, as scholars, writing up our research, as we often do, enact our scholarship in ways that matter, in that we are having impact, making futures, and otherwise producing ethics even as we use them. I want to think through some ideas around three points.
- One point is about how we ‘move’ as we do our research. To emphasize and revisit the relation between the researcher and researched. The second point is why it matters, in terms of our impact as as researchers, which is a matter of doing the right thing, vis a vis producing future values, knowledge, and privacy and potential vulnerabilities in a digital era. This is a very brief point since i believe many of us already invest a lot of time thinking about issues of privacy and data protection. The third is a discussion about methods as choices at critical junctures, which shifts the accountability for method to the researcher rather than the tools we use or the regulatory disciplinary models we follow.
- WHY ME? I suppose I’m here because I read and think a lot about methods and think about the sensitivities of studying when we are vulnerable or when participants are vulnerable. I think most situations are precarious, so I think a lot about the relationship of self to other, and how our role as researchers has so much more power than those we study. These images of some of my titles say something about my own layers. I do research and publish in scholarly venues to try to do many things at once: ethnography, intervention, teaching, and challenging frames for “what counts” as valuable and valid research across social science, data science and humanities research settings. A lot of this work conceptualizes methods as ethical stances, and ethics as methodological stances, so there’s significant blurring between these terms.
- The stuff in this talk comes generally from my work in these four contexts.
- I can’t hope to speak to all of the disciplines or research situations represented in this room, but perhaps and I certainly gloss everything for purposes of this short provocation. Still, I hope you find some of this useful in some way to your own research practice. What I’m addressing in this talk spans several of my own papers as well as some key works I that inspire my thinking. Most directly, I’m drawing on: Emotions in the field, edited by Spencer & Davies Tales of the field, by John van Maanen And two pieces by me about methods and ethics in digital research contexts
- I also draw inspiration from the work of Postcolonial feminist and cultural theorist Sara Ahmed Poet and Naturalist Diane Ackerman Feminist qualitative research theorist Laura Ellingson And feminist theorist Karen Barad known particularly for her theory of agential realism.
- I use this model to help me get at some of the underlying or hidden processes or practices within which we find ourselves making ethical decisions, or decisions with ethical consequence, On one side of method, or here, depicted ABOVE METHOD, we might consider the epistemological conditions within which we find ourselves doing social researching in the 21st Century. What are some of the political, economic, disciplinary, or cultural situations delineating our projects? How are predominant frameworks facilitating or constraining the way we SEE problems or address them? How are we –and our everyday practices, in relation to technology, considered as part of the research process, or more critically, distant from the practice of everyday knowing? BELOW METHOD means taking a closer critical look at what we actually DO when we’re doing qualitative methods. practices and procedures of inquiry. Gardening. Crime scene investigation. Surveys of social media use among teens, ethnographies of health technologies in practice. I also mean practical and logistic activities of making decision about who to study, when, where, and how. What questions we ask in interviews, whether we do interviews or observations or have participants keep their own diaries of everyday use of technologies. I also mean those things we don’t generally think of as “method,” in that it is not directly validated as data collection or data analysis. Like finding books randomly on the library shelves that influence your conceptualization of a problem. Like “cleaning up” materials so they don’t contain the stuff that seems like “noise” or “irrelevant information.” Like doodling, drawing maps or connecting concepts visually on scratch paper. Like changing your mind in the middle of a study. By talking about the stuff AROUND method, I’m trying to help us identify the underlying frames that influence our studies.
- Relating this more specifically to ethics, we can add to this image, to consider how our ethical sensibilities are being framed… And also, what frames are we teaching others to see, through our own work, or in the way we teach or train?
- Ethical practice comes from general principles about what is good and bad, right and wrong. It’s about doing the right thing, essentially and most simply. As they become codified and standardized in practice, they are seen as regulated norms. These top two are the most common ways of thinking about ethics. Probably because the word “ethics” prompts us to make it into something complicated. At more basic everyday levels, we can think of ethics as sets of attitudes, mindsets, or visions, And even more basically, we engage in ethics all the time, when we offer our seat to someone, or when we decide to help someone on an exam. The magnification on this image typifies (generally) the emphasis given to these different depictions, in academic research environments.
- But we’re seeing a bit of a flip, where the specifics of the situations we face demand unique responses. While we are likely still guided by general principles, many of the regulations that functioned adequately in the 1950s simply don’t hold up any more. So in the past decade or so, we’ve seen a shift to bottom up rather than top down thinking about ethics, or a case by case judgment about what’s the best thing to do, which may or may not be in accordance with what is prescribed by regulators, traditions, or disciplinary norms. Let me walk through this a bit more closely, focusing mostly on the american regulatory system of ethics, since it is the most widely used.
- Principles of research ethics and ethical treatment of persons are codified in a number of policies and accepted documents, such as the UN Declaration of Human Rights, the Nuremberg Code, the Declaration of Helsinki, and the Belmont Report. At their core, the basic tenets shared by these policies include the fundamental rights of human dignity, autonomy, protection, safety, maximization of benefits and minimization of harms, or, in the most recent accepted phrasing, respect for persons, justice, and beneficence. While originally stemming from the biomedical contexts, these principles have been adapted beyond these early contexts and rise above disciplines and methodologies.
- This gets roughly translated into IRB regulatory framework as protection of human subjects, using the standard of getting informed consent, making sure that the procedures protect privacy, and carefully assessing whether the research benefits outweigh any risks. (NOT OFFICIAL, but my way of sketching this issue out) IT does NOT work well for digital contexts. At all: None of these concepts are easily identified in internet situations.
- For example, what is a human subject? What does ‘consent mean’? More importantly, but less discussed, what does “informed” mean? It’s (almost) impossible to protect privacy (PII), and PII is often thought about as the form of data that ought to be protected Vulnerability and risk is--as often as not-- something that can only be identified in retrospect because it is after the fact, not apriori And the strategy of balancing benefits and risks, is certainly universal perspective. I’m not going to dwell here, there are really good reviews of this elsewhere (partic the AOIR doc, 2012)
- So what we notice is a shift over the past 20 years or so, from a top down regulatory approach to more process driven approaches.
- This is an image from our AOIR document, which illustrates how the type and venue/context of information is not as clearcut as we might pre-suppose, meaning that many questions cut across the type of contexts we’re studying. If we go to the questions that ethical regulations supposedly address or answer, we can begin to see how contextual these question are. We can then consider each question within the context of an individual study, giving value to different answers. This is a more effective way to address the common ethics motto: “it all depends”
- Close-up of the aoir 2012 document
- Over the past decade, we’ve seen a fairly rapid shift in perspectives. Across all levels (except perhaps middle management and IRB committees). Here, I identify typical question that might undergird each approach. I think this process or practice step has been fruitful, taken up in many places. It emphasizes the CASE and the practice of asking questions at critical junctures. But I want to take this one step further. To wonder if we’re still missing something when we ask the final question on this slide: What is the goal of research in the first place? This question implies a new approach.
- To this list then, I add a deliberately future oriented approach that focuses ethical attention on IMPACT. This impact model is useful for including aspects of ethics that otherwise might be hidden behind concepts or a buried motive within the regulations emerging around ethics.
- We can return to this slide to add a third layer: impact All research has impact. We generally focus on potential harms, but this is only one of many words we should consider when thinking about doing the right thing. Harm is one side of a coin. Good is the other. What ‘good’ are we doing can help counteract the paralysis caused by focusing on harm. Building a better future, future speculating, future making, these are phrases that can also help shift research, design, and policymaking in directions that are less about minimizing harm or creating benefits in the short term, and more about thinking to the near and distant future and considering what is possible, what would be good, what would we want for our children. Impact is inevitable and is certainly not always-–or even often--predictable. Even as we accept this, we need to make sure we don’t stop designing, making policies, and doing science. But we need to NOT ignore that our actions and thoughts and knowledge building have consequences. Always. In ways we cannot anticipate. And in ways that could cause harm. What will we do about it? Think more, make informed decisions, and try to find projects that contribute good or combat the potential harm of our technologies.
- Details of this Impact model are derived in collaboration with Microsoft privacy officer Janice Tsai and Microsoft machine learning researcher Sumit Basu, Four here (and there might be more) arenas or ways we can talk about the impact of our actions. These arenas are meant as provocations for further conversation among data scientists, IT developers, and technology designers, who face different ethical dilemmas than the typical researcher. Notably, what follows is not a typology, but a conversation starter. Whether these are categories, scales, or levels is unclear. Also, while these focus on possible harm, this is just one angle of gaze on the topic. Caveat: This current iteration focuses on--or builds from a model of harm, which is only part of the story and therefore an inadequate depiction of impact. Impact Arena 1: Treatment of people We interact with people all the time as we People could be human subjects, but not necessarily. They could be subjects, volunteers, turkers/workers, participants. They could be researchers. What is the potential harm of manipulating the environment to test the impact of certain system inputs? Changing ‘facts’ to test learning software in a MOOC Embedding code to prompt turkers to work faster Obvious (well studied) deception and manipulation experiments (milgram) What is the potential harm of psychological interventions? Triggers without debriefing, and even with debriefing Giving a survey on depression to teens, dor example What is the impact (on humans) of seemingly innocuous research design: e.g. 1: sensors in the home e.g. 2: camera in my face e.g. 3: triggers in photo elicitation or feature-finding in machine learning What is the potential harm of the research project to the self—the researcher?
- Impact Arena 2: Categorizing with data Here, use of data refers to how we use data to make inferences/conclusions. It could be through using existing data sets, scraping/mining data, creating categories within data sets, clustering and classifying in data analysis, or designing particular ways that others access or use data sets. a) 23 and me. The option to “share findings” yields a divorce. b) Havasupai. Using archived data to produce new findings ruined their cultural heritage. c) categorizing genetic sequence data into race categories. This constructs a particular categorization of humans, reifies, standardized, and reifies a scientific categorization choice into potential ‘truth’ about humanity. Also potentially changes individual worldviews. d) public shaiming? Does this fit here? Impact Arena 3. impact of technology design, research design, or production and deployment. a) The impact could be immediate, as when designs are implemented with particular and immediate effects on users. e.g. 1: when a new platform design makes the user feel stupid e.g. 2: when search engine auto-results trigger negative emotions in users. e.g. 3: when search engines show a person’s latest twitter activity or hashtags, exposing individuals who never thought their communication would be remediated in that way b) representation/report. How does our published findings represent the facts, truth, individuals, etc. not protecting privacy through accounts in reports making generalizations about groups of people c) impact on users. e.g., Logging all teen activity e.g., Sensors in the home e.g., Paying for female gender in games but male avatar is free Impact Arena 4: Future shifting. Here, the impact is less obvious, longer term, possibly unavoidable. What impact does our research have on socio-political, cultural, and other spheres e.g., ubicomp e.g., The nuclear bomb e.g., The rise of public shaming facilitated by design of twitter e.g., anonymity disrupted by demographic requirements in TOS
- I link this conversation back to methods by reminding us that although we might know better when we consider it closely, there’s a persistent myth that somehow our methods will help erase the subjectivity of the researcher, and that if we follow prescribed techniques or guidelines, we will somehow also fulfill ethical needs of the situation. As Davies reminds us (2015), “We must take to task the myth that method purifies subjectivity.” “Methods mold subjectivity, not into patterns that erase all emotions from the researchers’ sensing body but into patterns that produce emotions of a different order, and also into attitudes that too often privilege cognitively driven procedures and social research.”
- For me, finding what Helen Nissenbaum has referred to as “contextual integrity” and what in 2002, we talked about as “context sensitivity” (at the Ethics conference in Trondheim) requires situating the self, being reflexive about what we do when we do what we’re calling research. Here, go back to the basic question. What is the process of research? One way to answer the question is to say, research is a series of choices. Choices at critical junctures. Every time we make a choice, we erase other possibilities. Rather than ignoring or diminishing this fact, embracing it can help free the researcher from the false constraints associated with believing that the process is clean or neat, or that the outcome will be a complete vision or explanation of the whole thing. Inquiry has never been about that. This also highlights the concept that WE move. We are being moved by our subject, moving with it. Moving through situations. Identifying what is moving. We might ask: what is circulating? Is it the self? Is it affect? Communication? Objects? Even asking these questions highlights multiple bodies in motion and interaction, making the scene we’re trying to make sense of. If we think about this, if only for a moment, it seems obvious and sensible. But researched, respectively, are in continual movement, why do we (scholars, i mean), persist (at least sometimes if not a lot), in privileging the static, ? giving fact-value to those moments and utterances that have been frozen, through our lenses? To take this to a more concrete or textbook-ish level, the question becomes, how are we choosing our techniques and tools for analyzing the experience, the ephemeral, the elusive? How much or little do we erase or include our own bodily knowing, our own instincts about what is meaningful or not? One way to invoke more of the self is to deconstruct terms we use for research, reconstructing them creatively to include more ways of knowing. Then, translating it back to vocabularies that can be heard/understood by others.
- When you think about all the practical as well as methodological choices you make, it gets a bit dizzying. CLICK Each of these “critical junctures” utilizes a unique method or tool, whether or not this methodology is articulated or deliberate. Each decision also invokes and produces an ethic. How would we each draw a map like this to articulate some of our own critical junctures or decision points? What is the experience of research? What is it to do research of precariousness, vulnerability? What does it mean for us, as researchers? What does it mean to others, the ones we study? Or the others, the ones we write for, speak to, and otherwise influence with our writing? What does it mean, to , as a sociologist, for example, to take some Other and transform their lived experience into something estranged and abstracted from the lived experience itself? What happens to the very lived and visceral experience when we label it a “phenomenon”? What happens when we write in third person, abstracting ourselves from the text or position ourselves as an observer with some or more level of objectivity? How do we deal with the tension of competing values? On the one hand, we strive, if we are academics/scholars, to be “rigorous.” No matter how we define that, it often involves using external criteria to evaluate our methods. On the other hand, we are bodies encountering the world, making sense, and representing that knowledge to others. How can we simultaneously use external criteria to evaluate our inquiry practices and hold tight to our own values about what is involved to know something about others In studying precarity, what happens to us? A realist would tell us we should try to reproduce experience as accurately and completely as possible. An interpretivist would say this is impossible. An artist would find a way to convey meaning through sensation, an appeal to our senses. As a methodologist i think about this all the time.
- In my search for alternate vocabularies, we can return to the 1990s as well as As many have already mentioned yesterday and this morning, the body is central to sensemaking, in that it is intimately involved with creating and enacting the imagination required to transform and translate from the field to the page. So this talk goes back to the heyday of the 1980s and 1990s, to underline the interpretie turn, the linguistic turn, and the undergirding premises of postmodernism. Specifically, to focus on the relationship of the researcher to the researched. To remindus of the way that our senses matter.
- I want to conclude by mentioning the underlying goals of our research within the future making research consortium. Our ideologies and habits around what counts as “Method” has a profound impact on our perceptual apparatuses. As we encounter the conditions of our fields, the phenomenon, the self, and the others of our research gaze, we are naturally disoriented. This disorientation might take the form of dissonance, dis-anchoring, unmooring. These sensations (and our responding sensations and actions) carry not only personal meaning, but analytical power. Yet, as Davies and Spencer’s authors poignantly articulate, we are generally not trained to pay attention to and hone these sensibilities. In the Future Making Research Consortium (and the Digital Living Research Commons here at AU), we’re developing innovative approaches, technologies, and vocabularies to think differently about the conduct of research, witihn the impact model of ethics I mentioned earlier- - to critically examine how we might otherwise impact possible futures.