Demystify the Algorithmic Recommendation

The interview with Sam Lavigne took place two weeks before the realization of his project Other Orders as part of the web residencies by Solitude & ZKM. During this interview, his work was still in progress. We spoke with Lavigne about his proposal for the web residency »Rigged Systems« curated by Jonas Lund and why humor and language are important elements of his work.

Schlosspost: Your project proposal Other Orders for the »Rigged Systems« web residency is about computational sorting and offers alternative models for how sorting is done online. Can you elaborate on your interest in computational sorting?

Sam Lavigne: The interest comes from how content is presented to internet users. Sorting touches everything we do online, even if we rarely recognize it as such. Most of the web is made of sorted lists. Google search results, YouTube video recommendations, Twitter and Facebook timelines – all can be understood as the output of sorting mechanisms, which to an increasing degree mediate our relationships with each other and with the world. The concern over what content we see online could be be understood as a concern about sorting itself. It is not immediately understandable which sorting systems are used. Because online communication and knowledge systems are so commodified, we rarely have control of how things are sorted, or even a clear understanding that most of what we are looking at online is nothing more than a sorted list.

»In Other Orders I want to first demystify algorithmic recommendation engines by reframing these systems as sorting mechanisms, and second, I want to find ways to remystify sorting itself.«

There are growing concerns around recommendation engines like YouTube’s, which appears to radicalize towards the right, or Facebook timelines, which appear to spread misinformation. Facebook, for example, could easily fix their problems by sorting their content chronologically rather than algorithmically. But they won’t because it would break their revenue model. The questions that I am interested in touch upon relevance, and what makes up this kind of business model.

In Other Orders I want to first demystify algorithmic recommendation engines by reframing these systems as sorting mechanisms, and second, I want to find ways to remystify sorting itself. One goal is to try to figure out if you’re not optimizing for engagement (which is also a very funny word), what other things could you optimize for; what are the other logics? If you are for example concerned about pure communication, critique or analysis, what would a system look like?

SP: How did you start your research for this proposed project?

SL: The beginning of my research was to explore what techniques are used for sorting data. A timeline can be chronological, but doesn’t have to be. I am looking at what the systems are looking at. I’m really interested in sentiment analysis, for example, which allows a computer to tag sentences as being »positive« or »negative« but is extremely problematic because of the training data that is typically used. One could imagine training your own sentiment analyzer based on personal or political criteria of what is »positive« and »negative« and then using that system as the basis for a search engine or timeline.

SP: Have you thought about what the output might eventually look like?

SL: There are two ways that came up for me. I know that the output will somehow be related to different sorting methods. I am currently asking – how can we sort? That’s what I want to present. The thing that’s being sorted, could for example be the Twitter stream, but it could also be a novel, a philosophical text, or a YouTube video. On Twitter, for example, it would be interesting to look at tweets in alphabetical order. Or by how Marxist they are. There’s also the question of legibility. If items are sorted by some very esoteric system, how understandable or legible will it to be to the audience. For example, sorting tweets into alphabetical order is easy to understand, but sorting by »radicality,« or »despair,« might not be super legible. But maybe that’s okay, too.

SP: It sounds that your work is not only about aesthetics. It also offers a complex overview of the data, while also being critical. What layers might you like to include for the project you are proposing?

SL: I am still in the exploring phase. My work starts with a lot of experiments and then I am trying out new things, to see what works. I might throw a lot of things away in the end. So, one idea for this project is to re-imagine Twitter’s HTML, and put everything into an HTML table. The first layer is the joke of Twitter appearing as an HTML table, and that with this approach, one could solve online radicalism. Then, there could be many ways of sorting the tweets in the table, on different drop-down menus. Each sorting method, each drop-down, offers a whole world of data behind it, each could be a work in itself. I’m also interested in taking these different layers and applying them to texts in general, not just tweets. This way one could read novels or stories in as strangely ordered lists. Here, for example, is my software sorting »The Judgement« by Kafka in order of how close each sentence is to a motivational self-help speech.

1) Everything will be moved over with you.
2) We must set up a different life style for you.
3) If you think about it, you must remember.
4) For you there won’t be any change.
5) That won’t happen.
6) But old age demands its due.
7) Since the death of our dear mother certain nasty things have been going on.
8) Perhaps the time to talk about them has come and perhaps sooner than we think.
9) But what if you’re wrong!
10) For he knows everything, you stupid boy, he already knows everything!

I have this desire to think of sorting language through a poetic logic rather than purely scientific. That is, to apply a poetic logic to what are typically considered more or less scientific methods. And I’m interested in building a series of systems that I can apply to multiple projects going forward.

Data itself is a political fact, but a bunch of these tweets are just texts. That’s probably where I am headed. It’s an example of what I could do with this technology, and hopefully, it can be a form that I would like to explore further.

SP: The author and new media professor Lev Manovich wrote in one of his books that when one talks about the great successes of AI in recent years, the examples used are the same tasks that were defined at the beginning of the field many decades earlier: natural speech understanding, automated translation, and recognition of objects in photos. But what he sees differently is that AI today plays an equally important role in our cultural life and behavior, with the processes of aesthetic creation and choices becoming increasingly automated. This development raises questions in the development of culture. Manovich wonders whether such automation leads to a decline in aesthetic diversity over time. Do you agree and if so, what are your thoughts on how computational sorting can offer more varied items and thus diversity?

SL: It appears to be because everyone is using the same automatic tools. We produce the same kind of output with the same kind of cultural input. It is also a broader question of communication itself. Language becomes marginalized. That process is also a political process in a sense.In my own work I make automatic systems that produce aesthetic artifacts that are about how these systems are being made. I think that the question is to produce work in which the system itself is aesthetic.

»A lot of my work is language in some way, it is about the computer to be able to make sense.«

SP: With your project, you also look at the poetic potential, which I found out is closely connected to your interest in natural language processing. What does the power of words mean in relation to your project?

SL: I am not a trained artist or programmer; my background is in literature. For me, the interest is always in language and I find it exciting to work with programming and texts. A lot of my work involves language in some way, or is about the computer’s ability to make sense.

I have done a few projects that are directly related to this. One of them is called Are you ready. It is about language and immigration. I start with a video guide to the United States naturalization process, which offers viewers a series of practice questions for the citizenship test. Over time the video becomes increasingly difficult to understand, garbled. The technical part of this project is making use of understanding the sounds of each word or phonemes. The video slowly sorts them and interchanges them until the original becomes multi-voiced, garbled, and obscure.

SP: Your previous works also involve scandals, crimes, surveillance; and highlighting the role that technology can play in those things. You often give the users and viewers full access to information, providing a transparent view of reality. For example, in your works The Good Life (Enron Simulator) and White Collar Crime Risk Zones. What importance do you see in offering this model of transparency?

SL: The Good Life (Enron Simulator) is a project I collaborated on with Tega Brain. It hilariously emerged from another project I did, called Slow Hot Computer, where I experimented with the idea of a computer going very slowly, so that you can’t really do any work. It was a gesture and an experiment in computational strikes, and what it means to be on strike, or how one could even be on strike. I wanted to do something with emails or self-sabotage. I thought it would be great if you get like thousands of emails a day and you just couldn’t use your email account. So I was looking for an email datasets that I could use to receive thousands of emails a day and I happen to stumble upon the Enron Archive. Working with Tega, we realized it could exist on its own and should be separated from the idea of the Slow Hot Computer. This Enron database is very rich. It is fascinating. It was one of the most successful energy companies in America. At the end of 2001, it was revealed that Enron’s reported financial condition was sustained by institutionalized, systematic, and creatively planned accounting fraud. They were basically lying. It fell apart and went bankrupt. The US government investigated, and as part of the investigation, a lot of the data of the company was released. Part of this data was this archive of more than a million emails. Because it was one of the few available email data sets, a lot of machine learning systems have been trained one way or another with Enron data set, which is fascinating. And as a kind of interesting anecdote, one of the first Siris, before it was called Siri, is trained on Enron. The dataset has this incredible history as a cultural document. The story of United States companies and corruption and of computer science and machine learning.

The project now consists of more than 500,000 emails from the Enron archive. You can choose if you want 7 years of 196 emails sent to you each day, 14 years and 98 emails a day, or 28 years and 49 emails. One way or the other it will take over your email account completely. I think there are also valid concerns about privacy; there is stuff in this email that should not really be in there. There is this real need to see and access this stuff for what happened, and how it happened. But some of the emails are private and should not be in there. It has nothing to do with the fraud but these are of course the most fascinating emails as well.

The issue of transparency is also part of White Collar Crime Risk Zones, a machine learning tool to predict crimes. This is a collaboration between me, Brian Clifton and Frances Tseng. It’s built on the concept of predictive policing, a technique for predicting where and when crime will happen. Predictive policing systems are deployed and used throughout the United States. These are systems that are run by private companies – they take historic data from police and then use that data to predict where crime will happen. And then they use these systems to send police to possible crime scenes.

»I like the work to have many different outcomes. The totality of it is the piece and each thing is contributing in some way to this whole. You just keep adding more and more. That is the work that I like the most to do.«

There are lots of problems with this. One of the main issues is those police departments have racist tendencies. Predictive policing reproduces these tendencies. What it also does is that it creates an excuse for the police to not to be accountable. They could say »We did this thing because the machine told us to do it.« It is used as an objective scientific tool without any self-reflection. There is no accountability, and there is no transparency because it is made by private companies, and made to sell. For the project, we decided to use the same methodologies that predictive policing systems use but instead of using street crime as our data, we used white collar crime.

SP: What decisions do you make on how to present your work?

SL: Once I find something interesting in a topic, I build it to be as real as possible. With the project White Color Crime Risk Zones there is the website, the map, the app. I do prints of it, and even a video piece. It has many levels. It is also a performance, in a sense. When we launched the project, I sent this mass email to more than a thousand United States mayors of small cities. I wrote, “I am a professor at NYU and doing this research on crime and wanted to share this project with you.” This became part of the project as well. I like the work to have many different outcomes. The totality of it is the piece and each thing is contributing in some way to this whole. You just keep adding more and more. That is the work that I like the most to do.

SP: You have taught at the ITP/NYU, The New School, and the School for Poetic Computation. How does your teaching practice relate to your artistic projects?

SL: The things that I teach are my practice. I do a lot of automatic content production, for example I work with found material, or data in a broad sense, and the questions I am raising while working with these materials are: how do I acquire data and what do I do when I have it? How do I speak through that material with my voice? How do I reveal something that is hidden in this mass of data? How can I manipulate data in a presentable way? I think that the answers to that question are simple answers. I am looking for the humor and the contradiction. Those are the sensibilities that I also try to carry on in the students. I want to empower them around how they can use computation and collectively foster their imagination of what is possible.

»Again, it will be kind of a joke that I take too seriously, and hopefully some depth will emerge from it.«

SP: What other projects are you aiming to develop in the near future?

SL: I know that this project is going to take longer than this residency. I don’t know the outcome yet, but I know that it will take more than this one-month residency, because I won’t be done with it. I probably want to work on a search engine in the future, which is going to take a lot of time. Again, it will be kind of a joke that I take too seriously, and hopefully some depth will emerge from it.

SP: What does your desktop or workspace look like?

SL: I actually just moved, so my workspace doesn’t exist at the moment. Here’s my desktop background though…

Here’s a photo of me working on a project in my old studio:

SP: What are some links, book titles, sound files, videos – as further reading or background – that might be related to your artistic practices and research topics to share with our readers?

SL: Here’s my are.na channel that has all my inspiration on it!

https://www.are.na/sam-lavigne/screenshots-1518658460

 

The interview was conducted by Sarie Nijboer.