links
the context around the
tarot machine
unsupervised algorithm

Example Software Compas:

“Northpointe Suite: Automated Decision Support”, Northpointe
(Website, via Internet Archive), online:
https://web.archive.org/web/20160307002839/http://www.northpoin teinc.com/

Bias:

Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types
https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0231189&type=printable

Fallbeispiele:
https://www.bertelsmann-stiftung.de/fileadmin/files/BSt/Publikationen/GrauePublikationen/ADM_Fallstudien.pdf

Kissinger:
https://www.nzz.ch/feuilleton/die-neue-freiheit-im-cyberspace-ist-ein-leeres-versprechen-die-algorithmen-schaffen-ein-system-das-nur-sich-selber-stabilisiert-ld.1443344

Algorithmen decision making -
https://algorithmenethik.de/wp-content/uploads/sites/10/2018/02/ZweigADMFehlerquellen_final_ohneCover.pdf

AI stereotype catcher:
https://science.sciencemag.org/node/692879.full

Exploring Latent Semantic Information for Textual Emotion Recognition in Blog Articles:
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7833266

Human-like bias:
https://science.sciencemag.org/content/356/6334/183

Examples of bias:

Amazon:
https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G

AMS Oesterreich:
https://www.derstandard.at/story/2000108890110/warum-das-ams-keine-ki-auf-oesterreichische-buerger-loslassen-sollte
https://netzpolitik.org/2019/streit-um-den-ams-algorithmus-geht-in-die-naechste-runde/

Facebook sucidal:
https://www.nytimes.com/2018/12/31/technology/facebook-suicide-screening-algorithm.html
https://www.sciencemediacenter.de/alle-angebote/research-in-context/details/news/ethische-aspekte-bei-facebooks-suizid-praevention/
https://www.welt.de/gesundheit/article188741417/Facebook-Algorithmen-mit-KI-erkennen-Suizid-Gefahr.html

Is facial recognition technology racist?
https://www.theweek.co.uk/95383/is-facial-recognition-racist

James Vincent, “Twitter taught Microsoft’s friendly AI chatbot to be a racist asshole in less than a day”, The Verge (24 March 2016),
https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist

Interesting:

“Abgestuerzt“ Black Mirror Staffel 3 Folge 1

“schnelles Denken, langsames Denken“ von Daniel Kahneman

“Futurzwei“ Nr 7 /2019 Schwerpunkt “Kuenstliche Dummheit“

“Homo Deus“ Yuval Noah Harari

“ada heute das Morgen verstehen“
https://join-ada.com/

About the Tarot Maschine:

Code on GitHub:
https://github.com/TransparencyTango/tangorithm

GloVe:
https://translate.google.com/translate?hl=de&sl=en&tl=de&u=https%3A%2F%2Fen.m.wikipedia.org%2Fwiki%2FGloVe_(machine_learning)
https://epb.bibl.th-koeln.de/frontdoor/deliver/index/docId/1251/file/Bachelorarbeit+Alasdair+Maclachlan+Ver%c3%b6ffentlichung.pdf
https://github.com/stanfordnlp/GloVe

other sources:


Caliskan, A., Bryson, J. J., and Narayanan, A. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183{186.

Flask. http://flask.pocoo.org/.
[3] React. https://reactjs.org/.

abstract

the technical structure has been altered slightly, from an exhibition piece to now a web application. there is, for example, no longer a separation between input and output device.

the alg. tarot is an interactive exhibition piece, that metaphorically communicates the influence of historical bias in data on machine learning algorithms. the interaction is designed to be as simple as possible, to ensure comprehensibility for everyman. to begin the user types in words such as the name of their future kids. next the user will be shown a tarot card with a 3D symbol on the screen. the 3D symbol comes from a pool of predefined 3D symbols and words, each with an individual set of tags, describing the word. the tags are created by the machine’s algorithm. the user will be shown the model with the most fitting description according to the previously given input.

for similarity calculation i converted the words to vectors using word embeddings, that can be understood as representations of natural language words in form of vectors of real numbers. specifically, i used a set of publicly available pre trained word embeddings, learned by a statistical machine learning model, named the „global vectors for word representation“ (GloVe) on a corpus of web texts. since the similarity of input to output words and tags is determined through GloVe‘s learned word embeddings the matching reflects existing biases in the embeddings.

introduction and motivation

a common problem in machine learning (ML) is the existence of historical bias in the data used to train the algorithms. i understand historical bias as patterns in the data, that reflect the social environment in which the data was created and collected. if no precautions are taken, in counteracting the bias, these patterns are unintentionally learned together. this might skew the algorithm’s results to be unequal and biased. this issue is becoming increasingly problematic, since the use of ML algorithms in socially critical domains, like the law, is steadily increasing.

my goal in this project was therefore to raise awareness for the issue and demonstrate the possible impact of algorithmic decision making systems replicating human prejudices. to achieve this i built an interactive exhibition piece. the user stands in front of a display and enters their future kids possible name as an input into the input device. the result is a tarot card with (predefined) words that best match the input of the user.

since i consider the issue to be of relevance for all members of society, the exhibition pieces is targeted at the general public. two of the most important goals within the concept are to spark related discussions and try to lower the barriers of entry for those discussions.

word embeddings are mathematical representations of natural language words in the form of real number vectors. GloVe’s word embeddings were picked as they are considered standard ‘off-the-shelf machine learning components’. subsequently their use is wide spread in all sorts of ML models, especially within the field of natural language processing (NLP). explicitly those word embeddings are used to generate vectors that form the basis for out matching algorithm, from the user’s input. furthermore the existence and replicability of human-like biases in those embeddings are proven by Caliskan, Bryson and Narayanan. as the choice of possible output words (as representations of various fields in which algorithms are used) are selected by myself, it is important to mention that the existence of my personal bias, to a certain degree, cannot be denied. the tags of those words are given by the GloVe algorithm.

implementation

i developed a system that matches input words to predetermined sets of tags (words) based on GloVe. it then shows those associated words and a 3D symbol of the chosen tag set to the user.

architecture

the system consists of three main components: input, server and result screen. the user interface accepts multiple text inputs and additional information, if required, on the result screen (the tarot card).

it sends the user inputs to the server. the server consists of a pre-trained data set of word embeddings and it holds a representation for each possible output in this vector space. it is able to match a set of input words to the most similar word and 3 d model. the result screen shows the current output word and 3D model (based on the last input words). it can show additional information including the closest ‘neighbours’ of the input words in the vector space and the similarity to predefined words.

the server is the center of information in the application. the user interface forwards the user input to the server, this includes the set of words and whether the similarities and nearest neighbours are requested. the server is doing the matching and stores the result state: the current matched word and 3D symbol, and the state and content of similarities and nearest neighbours. the results screen repeatedly polls the state, to decide which information is shown.

input

the user input is a simple website, built with HTML, CSS, javascript. after the user clicks ‘generate’, an ajax request (i.e. without reloading the page) is sent to the backend. the data returned in json format is then processed in javascript. the user can see the opening credits video running while the new videos are loaded in the background (background video + tarot card video), which will be displayed after the end of the opening video.
a click on ‘interpretation’ triggers another server request. all other data, can be faded in via the buttons, has already been received with the first request.

server

the server is the center of the application, it serves two main purposes: the management of the application state and the matching algorithm between user input and output. the whole server application is written in python. it uses a REST API provided with the help of the framework Flask.

GloveExplorer

the GloveExplorer is a python class that holds the pre-trained GloVe word embeddings and provides all the necessary functionalities needed for matching the user input to a corresponding output word and 3D model. after the user submits their input, a matching request is sent to the server, which then executes all the steps shown in figure 3 under runtime and returns a string of the name of the corresponding output words and 3D symbol.

diagram diagram diagram figure 1: activity diagrams for preprocessing and runtime
 3D symbol inference


tarot state

the instance of this python class is used to store the current state of the tarot screen. it stores the name of the most recently matched output word and 3 symbols with its nearest neighbours and similarities to predetermined words and whether this additional information to the 3 symbols should be visible. to keep polling from the server simple, it continuously returns all of its stored attributes on requests.

unsuperveised Algorithm:

„...The methods of Machine Learning

There are two main ways of implementing ML: supervised and unsupervised learning.

Supervised Learning

Supervised learning is a form of machine learning where a correct answer is provided to the machine at the training stage. For example, an image could be provided together with a label to specify whether the image is that of a dog or a cat. Or for a real estate application, a number of properties of a house could be provided, together with the price of the house. The algorithm would ultimately try to predict this label with the properties available to it.
All machine learning algorithms follow a similar process:
Data : The programmer has to provide the algorithm with a dataset. This could be, for example, a set of one million house listings and their price. The price, in this example, would be the target that the algorithm attempts to predict. The more data, the better the algorithm can become. In fact, using more data with a “stupid” algorithm will usually beat a better algorithm with less data. A big advantage of the large firms in machine learning is the amount of data that they hold. Google, for example, holds and uses enormous data-sets in training their models. For example, Google generates data from the log-in process to its various services. To verify that they are not bots, users are asked to click on images containing certain elements, such as cars or signposts. This human-generated interpretation data can then be used to train AI systems. Beyond this, large tech companies employ thousands of workers that manually go through and label pictures for self-driving cars;
Features : the computer, at this stage, does not know how to deal with this data. It has to be turned

Unsupervised learning

Unsupervised learning is a class of machine learning where no labels are provided. Instead, the computer itself tries to figure out what distinguishes one piece of data from another. In our example of cats and dogs, this would be the engineer providing the algorithm with images of both cats and dogs, and the computer itself realizing that there are two different animals in the dataset, and what distinguishes them. Unsupervised learning does not perform as well as supervised learning. However, it is an active area of research and has several advantages over supervised learning. A big advantage is that the data does not have to be labelled, making enormous troves of unstructured data accessible to analysis. Therefore, many see unsupervised learning as the approach of the future. One important use of unsupervised learning is that of anomaly detection. Here, a network is trained to learn the structure and general appearance of a stream of data. It is then able to tell if one data point looks different from the rest. This can be used, for example, to detect problems in production lines or possible cyber fraud attempts in a large number of financial transactions.

Reinforcement learning

There are some other types of ML that are starting to surface. One is reinforcement learning, which sets an agent loose in an environment and tries to get it to achieve a certain goal, such as driving a car or playing a game. At first the algorithm starts out randomly. However, if by chance it achieves a winning condition, this behavior is reinforced. This is done until the algorithm reliably learns how to achieve the set goal. Reinforcement learning has been instrumental in learning to play everything from board games to computer games.

Generation

A relatively recent AI technique is the one associated with generative adversarial networks. It is a technique that uses artificial intelligence to not just classify, but also generate data. In technical terms, this means that one network tries to trick 22 another one into believing its images are real, and not fake. The two networks evolve together until they both get very good at their jobs. At this point, the generator is able to output data that almost looks real. There are also other architectures that perform well in generating data, such as Recursive Neural Networks. Generative Adversarial Networks and other types have been used to create images of faces, compose music and produce extremely realistic sounding speech. There are also methods for transferring one piece of generated (fake) content into another that is a true representation of reality. This can be used, for example, to turn any image into the style of a famous artist51 or to generate realistic videos of celebrities doing or saying things they have never said or done.

Benoit Dupont, Yuan Stevens, Hannes Westermann, Michael Joyce, „Artificial Intelligence in the Context of Crime and Criminal Justice
A REPORT FOR THE KOREAN INSTITUTE OF CRIMINOLOGY“ (December,2018)

Bias:

Discrimination

Another risk that has already manifested itself is that of discrimination. Artificial Intelligence is very good at learning from data. However, if this data is biased, these biases will be reproduced by the AI. For example, an automated analysis tool for job applications in the technology sector might spot a historical trend to prefer men over women and therefore value traits associated with men higher than those associated with women. Word embeddings, which try to learn the semantic meaning of words, often associate certain terms with women, and others with men, reproducing gender stereotypes.
For example, nurse might be associated with women while doctor is associated with men. Further, facial recognition software might fail to detect people of certain ethnic groups if the data used at the learning stage was exclusively drawn from another group.62 Bots replicating conversations between users might be taught to make racist remarks and adopt a discriminatory set of values in its interactions with other users. Men might be showed ads for jobs that attract a higher salary than those shown to women, reflecting the wage gap in many occupations and reproducing inequality in professional opportunities.

Benoit Dupont, Yuan Stevens, Hannes Westermann, Michael Joyce, „Artificial Intelligence in the Context of Crime and Criminal Justice
A REPORT FOR THE KOREAN INSTITUTE OF CRIMINOLOGY“ (December,2018)