Main-01.png Read-01.png Recipes-01.png Tactics-01.png Cases-01.png Tools-01.png

Bias Detection: Research

Team members

Erik Borra, Nils Markusson, David Moats, Lonneke van der Velden, Tommaso Venturini


These are the notes of a one day workshop exploring methodological avenues of bias detection and the characterization of types of bias. Our group first shortly discussed terms associated with bias, then moved on to methods of 'charging' and started to characterize different types of informational bias, both static and through time.

What the bias?

Some of the possible terms to denote what we are actually measuring are bias, partisanship or actor alignment. Partisanship can be considered too conceptually charged for our purposes. Also, through its use in (mainly American) politics it often has a negative connotation. Somebody who is considered strongly partisan does not recognize the viewpoints held by other parties. Actor alignment leaves little room for notions of informational bias.

In what follows we assume that bias is always a performative action; we thus treat bias as boundary work and bias detection as tracing those boundaries. Two implications of this working definition are that we assume that there are threads which we can trace (this means that we cannot detect the bias of the strategically silent) and that one of the traps of measuring bias might be a preference for big voices.

As the ACTA data set was 'charged' into a pro-camp, an anti-camp and a neutral camp we shortly explored the term neutral but deemed it difficult a term to work with. What does it mean exactly? Neutrality is constructed by people (census, balanced, …) or it can mean that is just not charged yet. The term is inherently ambiguous and should be avoided. Rather, we could speak of shared, uncertain or ambiguous.


As we assumed that bias is a performative action we sought to inquire into methods which can bring forward this performativity. We take an empirical associational approach and ask ourselves questions such as How often are terms used by specific opposing camps? Are some terms used more by one camp than the other? Conversely, if a specific term is used, can we say with some confidence that it is used most likely by an actor of one specific camp? In other words, we seek to 'charge' (Marres, 2012) terms by injecting the relative frequency of actors' positions into words. The reverse is also possible, infusing actors with the relative use of (framing) words they use.

In order to get at bias we require camps performing boundary work. We discussed two ways of getting the starting points to do so:
  • Dictionary based. Ask issue experts to identify camps (e.g. climate change skeptics versus alarmists) OR framing key words (e.g. freedom fighter vs terrorist). (This is the approach taken by Borra and Weber, 2012).
  • Link based. Computationally derive clusters from a disagreement graph (e.g. an article's revert graph in Wikipedia) OR from a co-word graph (of words frequently co-occuring together). If the network falls apart into clear clusters we might find differing camps / positions; if the network is homogeneous there might be consensus. This method is more mirky but praised for its capacity to dynamically detect camps instead of pre-defining them. (See also Shwed and Bearman, 2010)

In a simple polar opposition, charging actors or terms by camp becomes an exercise in polarization which can be operationalized as follows:
  • Given a set of actors defining a pro-camp and and a set of actors defining a con-camp terms can be 'charged' according to the fraction of times they are used by actors of a specific camp. (This is the approach taken by Borra and Weber, 2012).
  • Given a set of terms defining pro-language and a set of terms defining con-language actors can be 'charged' according to the fraction of times they use words from the pro-set and the fraction of times they use words from the con-set. Actors can thus be aligned based on the words they use.

By ranking actors or terms per camp, by the highest fractions in respective camps, the most polarized actors or terms surface. When there are multiple camps involved a set can be 'multi-polarized'.

Types of links between camps

It is possible to distinguish between at least two types of polarization. One depends on the notion that actors are into relation with each other by acknowledgement (they do not necessarily agree). One thus carves out a set of camps by ignoring others (the Issuecrawler approach, see also Rogers, 2012). Currently, however we don't have many measures for detection the performance of a controversy (accusation, disagreement, bias).

After camp-forming, after boundary work we thus sought to look into types of camp making as a way of qualifying connections. Although difficult to operationalize, we distinguished between positive and negative interactions and between within-group (intra) and between-group (inter) interactions. The following table sums it up:

  Intra-camp inter-camp
positive back-slapping (+) acknowledging (-)
negative in-fighting (-) accusation (+)

  • Back-slapping: if positive interaction happens within a group
  • In-fighting: if negative interaction happens within a group
  • Acknowledgement: positive interaction across groups
  • Accusation: negative interaction across groups

A controversy can be considered open (+) when there is in-fighting and acknowledgement and closed (-) when there is back-slapping and accusations.

Interesting types of links over time

Previous description of the types of links between camps is about (the practices leading to) a static state of separation. E.g. people accusing each other with derogatory terms. We consequently also discussed how terms can be(come) appropriated.

A word that used to co-occur predominantly with other words in one camp and that starts to co-occur with words in another camp could be called frame-washing. It is an appropriation of terms, a reframing. They are shifts in word associations, moving (or bridging) camps. One could think of this as the biased version of issue-liveliness (Marres and Weltevrede, forthcoming). E.g. the term queer was reappropriated from its use an anti-gay epithet.

A word that used to be used predominantly by actors in one camp and start being used also by actors in another camp could be said to lose charge or expand E.g. obamacare was used by the right to connote government takeover and death panels. Later, Obama made it loose charge by saying “They’re right. I do care.”

Changes in composition (structure of camps)

  • merger: camps get together
  • split: camps dissolve
  • internal moves: actors re-allign or words re-align (liveliness)
  • defection: part of a camp moves to another camp

Interesting exceptions

Mainly in cross-camp associations.
  • Actor in one camp mentions an actor in another camp. If positive it is an acknowledgement. If negative it is an accusation (partisan)
  • Actor in one camp using a word in another camp. If word is used positive whilst it was meant accusationally it is a frame-washing effort. If the word is used negative it is critiquing the other sites.


  • How to find secret lobby?
  • ACTA: unclear whether they are alliances or linking to critics
  • A possible tactic to overcome missing actors or words is to query for negatives.

  • We did not take into account (yet) whether medium specificity changes what we measure.


  • Borra, E. & Weber, I., 2012. Political insights: exploring partisanship in Web search queries. First Monday, 17(7). Tool available online at Last accessed, 4 November 2012
  • Marres, N., 2012. The environmental teapot and other loaded household objects. Re-connecting the politics of technology, issues and things. In Penelope Harvey et al., ed. Objects and Materials: A Routledge Companion. London: Routledge.
  • Marres, N. & Weltevrede, E., 2012. 'Scraping the Social? Issues in Real-time Research', submitted to the Journal of Cultural Economy.
  • Rogers, R., 2012. Mapping and the Politics of Web Space. Theory, Culture & Society, 29(4-5), pp.193–219.
  • Shwed, U. & Bearman, P.S., 2010. The Temporal Structure of Scientific Consensus Formation. American Sociological Review, 75(6), pp.817–840.

Whiteboard notes

These images display some intermediate stages of note taking during our discussion.

The final write-up, used for our presentation:

Writing and re-writing:
Topic revision: r4 - 06 Nov 2012 - 10:53:09 - ErikBorra