FOLLOWING THE I: THE BUSINESS OF BEING WATCHED

Living in an inbox_installation_Kin | Cultura Plasmic INC
Living in an inbox, installation, Kin | Cultura Plasmic INC.

Kin | Cultura Plasmic INC examines surveillance through moving image practices beyond the realm of CCTV.



When Barbara Kruger wrote ‘Surveillance is your busywork’ on posters aboard metro carriages within New York’s subway system, the year was 1983, well before the advent of a social media infrastructure that saps our time and attention to feed the ever-expanding appetites of engagement-driven algorithms. Although understandings of surveillance have shifted since the 1980s, the basic premise of Kruger’s statement is to link it with the appearance of keeping busy, a way to fill time, and work for the sake of work. It’s a slogan that gets to the heart of how ubiquitous surveillance is, and perhaps how historically unchallenged it has been: surveillance as a necessity and an accepted norm. It’s also a slogan that pinpoints a degree of complicity, using Kruger’s tried and tested technique of playing with pronouns to directly address the viewer.

It’s tempting to feel disempowered when considering surveillance and the seemingly all-powerful matrix of state and corporations behind it. Surveillance IS a technique of control. It runs counter to efforts that strive for liberation and autonomy and that fight against pressures to conform to set standards or norms. That’s why such appeals to agency continue to find their way into my work: we must feel like we have a choice in order to resist the apathy that such vast systems of control thrive on.

It was 2017 when I started making work that explores surveillance, and not just as a subject area but as a methodology with new mediums and tools to work with. Being a tool of moving image, CCTV systems have been a common device for artists working in this area. Not to say that this work isn’t important (actually some of my favourite works by artists exploring surveillance incorporate CCTV) but in considering how embedded contemporary surveillance is, it perhaps overlooks all of the other moving image-related surveillance tools there are:

  • machine-learning systems that process video to categorise different behaviours;
  • emotion-classification tools that measure facial responses like brow furrowing and degrees of smiling;
  • eye-tracking to gauge attention and the effectiveness of filmed content;
  • networked home gadgets like the Ring doorbell;
  • tracking features embedded into AR glasses, and so on.

My first video installation in this area was Watchtower, a work in which I used a drone to gather footage from above to construct a commentary on how constructions of threat and safety are intricately connected. In doing so, it looked at common justifications for surveillance, and through the narrative arc of the work, it alluded to its impact on behaviour and how prediction and anticipation leant towards conformity.


Living in an inbox_Kin | Cultura Plasmic INC
Living in an inbox, still, Kin | Cultura Plasmic INC.

In 2018, I developed Living in an Inbox, another video installation that positions social media, and in particular applications with integrated messaging functions and accompanying notifications, in the same frame of reference as advertising. To experience the installation, people sit within a comfy living room set-up to watch an advert promoting 24-hour engagement with digital platforms accessed largely via mobile phones. Throughout the film are slogans like ‘Be connected’, ‘Express yourself’ and ‘Deep-fried digi-diet: made for the ones you love’. Digital devices are presented as comfort food and such use of catchphrases that borrows from advertising cynically frames a diet of excess as a valuable social experience. On first impression it might not appear that this work is related to surveillance. But it absolutely is. One way of understanding this is through reference to the links between advertising, surveillance, consumerism, complicity and everydayness that Kruger points to.

In Kruger’s piece, the slogan is overlaid on an image of a man peering through a loupe, a small magnifying glass used by jewellers, watchmakers and photographers to inspect the details of the materials they are working with. In the image, the loupe forms a cage-like shadow around the eye that is doing the examining, the other eye scrunched tightly closed without a lens to peer through.

The magnifying glass may have been an apt sign for surveillance in decades past, but does it provide the same depth in a field that now offers, for instance, eye-tracking, (proposed) emotion-tracking and facial recognition? This was one of my considerations when designing Living in an Inbox.

Contemporary surveillance systems have pushed violations of privacy far beyond the global adoption of CCTV to monitor public spaces. Whereas in the 1980s, watchful security cameras transformed spaces like public transport systems in the name of safety, today’s monitoring systems transform the home, domestic contexts, and personal devices – often closer than within arm’s reach – in the name of user benefit: enhanced personalised services, better recommendations and tailored content. The eye, and a whole range of biometric data, have focused the hungry gaze of advertisers, agencies and businesses that profit from the trading of data, zooming in ever closer and personalised to the observed subject. Companies use eye-tracking to deduce value from users’ responses to content, so this becomes the method of surveillance and the commodity.

As computer scientist Jaron Lanier has previously summed up,

With old-fashioned advertising, you could measure whether a product did better after an ad was run, but now companies are measuring whether individuals change their behaviours as they browse, and the feeds for each person are constantly tweaked to get the desired result. In short, your behaviour has been turned into a product – and corporate and political clients are lining up to modify it.

If the eye is now trapped in this metaphorical cage and disciplined by being watched, much like Kruger’s image, how might artists address that? Perhaps one way would be to make visible the tracked eye movement, therefore opening an opportunity for user control. This is the route I went down when developing Living in an Inbox, making an interactive element crucial to this video installation.


Living in an inbox__Kin | Cultura Plasmic INC_still 3
Living in an inbox, still, Kin | Cultura Plasmic INC

When watching the advert, people can see how their eye movement is triggering bursts of ‘likes’ that spread across the screen. The power dynamic of control has shifted as viewers can now witness how their input is being fed into the machine. It’s about recognising agency; being conscious of it is the first step towards using it. (Side note: this is a conversation I’d like to return to another time. Does interactivity guarantee enhanced agency when at times defined choices can act as a smokescreen for how limited choice might actually be?)

In Baudrillard’s The System of Objects, he writes that every object transforms something else, giving the example that the process of transformation may be obvious with a coffee-grinder but less so with a mirror. Looking at the magnifying glass, it transforms both the subject being examined (e.g. it reveals previously hidden details) but also the eye that examines it (e.g. a perception change through technological prosthesis, whilst also making the eye look physically bigger through the other side of the glass). There is also a change in how the two relate and the implied power dynamic: one is being observed by the other. Eye-tracking technology transforms the material being viewed into a route of attention hotspots, and it transforms your attention into a quantifiable value to be maximised and manipulated. By making eye-tracking technology visible, I mean to say your attention is powerful. Where you direct it matters.

Maybe it also transforms what moving image is. When we are being watched watching a moving image, in a sense the viewing goes both ways. Our eye movement, or whatever facial movement is being measured, becomes the moving image for the machine (I’m using the word machine loosely here but you get the picture). I’m reminded of Hito Steyerl’s writing on how imagery mediated through digital technology is all but a series of numbers. Creating image data for machines to ‘see’ and interpret is a process of quantification.

I’ve largely focused on eye-tracking and attention-tracking to illustrate my explorations of surveillance within video installations but thinking about how my practice might develop going forward, I can’t help but wonder how things like the Metaverse, AR glasses, VR headsets, and so on will build on the existing demand for audience insights gathered via surveillance and spawn new subsets of data to be collected and aggregated.

It seems that the ideology and values of advertising have become almost inseparable from how we communicate with each other through interfaces. An edifice of surveillance, communications, and advertising has been built and is ever-expanding.

As we entered a period of increased remote working in 2020, the demand for video conferencing soared. Zoom was a popular choice and initially offered attendee attention tracking (a feature they have since removed due to privacy concerns voiced by the public and campaigners).1 Just this morning I was listening to a spokesperson for Affectiva2 – a company specialising in what they term ‘Artificial Emotional Intelligence’ – discuss ideas to use quantified facial expressions, vocal intonations, laughter and body language to provide a digitally mediated version of ‘reading the room’ in such video call contexts. 70% of the biggest advertisers use Affectiva’s technology to track and measure people’s emotional responses to ads as they’re watching them. One strand of this technology is being developed to judge responses to characters in films and TV, and although they don’t mention this, of course it could also be applied to people outside the realm of fiction and entertainment.1 With each of these developments, however, take note of the various ingenious attempts to circumnavigate such surveillance, e.g. schoolkids setting up cameras with images of their faces peering back at their webcams to create the illusion of paying attention.

In the past year, Affectiva has been acquired by Smart Eye, a company that lays claim to ‘eye tracking technology that understands, supports and predicts a person’s intentions and actions’. They write that ‘by carefully studying eye, facial and head movement, our technology can draw conclusions about a person’s awareness and mental state’. A lot of this segues into the tracking of workers in workplace environments (except now the separation of work and non-work is more than a little blurry, another area that Living in an Inbox hinted towards). It’s not enough to just be present; your presence must also be quantified in order to be bought, sold, manipulated, and distracted.


Living in an inbox_Kin | Cultura Plasmic INC_still 1
Living in an inbox, still, Kin | Cultura Plasmic INC

Surveillance has always been an attempt to control, a technique to consolidate power, but claims that AI-assisted surveillance can deduce our mental state (in a society that makes health-based discriminations and has historically excluded people based on set notions of ‘sanity’) takes this control to another level. The spaces free from these manipulative systems shrink to miniscule proportions. When those profiting from the technology claim to be able to predict anyone’s future actions, and to present those insights as irrefutable fact, it’s clear that the association between surveillance and public safety needs to be dismantled.

The data pulled via moving image and screen-based mediums has enhanced the power of these tools to manipulate emotion and decision-making. In doing so, surveillance becomes so much more than merely watching and observing. I think ultimately that’s the driving force for me in terms of incorporating interactive technologies and processes when exploring this area. We need to be able to see, feel, witness, and make tangible how our actions, inputs, voices, movements, and traces are feeding the surveillance-communications matrix that acts to determine so many areas of our lives.


More about Kin | Cultura Plasmic INC.

1 – With each of these developments, however, take note of the various ingenious attempts to circumnavigate such surveillance, e.g. schoolkids setting up cameras with images of their faces peering back at their webcams to create the illusion of paying attention.

2 – Affectiva’s technology is also used by 28% of the Fortune Global 500 companies to ‘test consumer engagement with ads, videos and TV programming’.

Bibliography

London, Barbara. 2020. Video Art: The First Fifty Years. Phaidon.

Baudrillard, Jean. 2020. The System of Objects. Verso.

Podcast: In Machines We Trust. 2022. ‘Encore: What’s Behind a Smile’. MIT Technology Review.

Steyerl, Hito. 2017. Duty Free Art: Art in the Age of Planetary Civil War. Verso.

Artspace.com, viewed 13 January 2021, <https://www.artspace.com/barbara_kruger/surveillance-1>

Affectiva.com, ‘Smart Eye Completes Acquisition of Affectiva’, viewed 13 January 2021, <https://www.affectiva.com/news-item/smart-eye-completes-acquisition-of-affectiva/>

Rider, Shawn. 1999. ‘Barbara Kruger: Signs of Postmodernity’, viewed 13 January 2021, <http://www.wdog.com/rider/writings/real_kruger.htm>



Click for more articles in this issue: