OASI.
Technically, an algorithm is a set of instructions designed to perform a specific task or, in other words, a list of rules to follow in order to solve a problem or produce a desired outcome. A cooking recipe is an algorithm, for example. However, when we talk about algorithms in the context of computer applications, data and the internet, normally we are referring to a computer programme including an algorithmic system designed for a particular purpose.
Because algorithms can deal with any kind of data or content, and because they are much faster than humans at dealing with big amounts of information, more and more algorithms are being developed and implemented these days by all kinds of entities and organisations. Spotify and Netflix use algorithms to recommend you automatically what music to listen to or which TV show or film to watch. So do Amazon to tell you what you may want to buy next, Google to try to predict what you want to search, and Facebook to decide what to show you in your feed. Public institutions also increasingly use algorithms to decide what taxes you must pay, who has access to financial aid and social services, and which school a child will attend, among many other examples. And security forces and agencies also rely on algorithms to try and predict crime and to identify people through face recognition or by following their online activities, for example.
Machine Learning Algorithms
Traditional algorithms are sequences of steps programmed to achieve a particular goal. If you want to improve the algorithm’s performance, you need to modify those steps until you get the newly desired outcome. However, these days digital technology has made that traditional approach obsolete, and currently most algorithms being developed and implemented are machine-learning algorithms, in which the programme updates itself as it tries to improve its performance automatically and without human intervention: it’s as if the machine were trying to learn.
Machine-learning algorithms often consist on statistical models that, based on particular datasets, try to find out the most relevant information to enhance the algorithm’s performance as more data gets into the algorithmic system. In other words: machine-learning algorithms are statistical models design to predict something or to classify information according to some particular categories, and those algorithmic systems try to improve the accuracy of their prediction or classification independently of human intervention by iteratively updating their statistical models by processing new data.
The fact that machine-learning algorithms are able to modify or even develop new rules and steps for their programme on their own, without human intervention, means that we need to have procedures in place to make sure we can understand how these algorithms work: we need to be able to audit them.
Machine Learning in practice
Imagine a group of scientists that want to develop an algorithm (i.e. a set of rules) that accurately identify and save images of cats in the “Cats” folder and images of dogs in the “Dogs” folder. If they decide to develop a traditional algorithm, then they need to specify themselves a huge set of rules like these:
– If the animal is bigger than “X”, then probably it’s a dog.
– If the animal is smaller than “X”, then probably it’s a cat.
– If the animal’s snout is longer than “Y”, then probably it’s a dog.
– If the animal’s snout is shorter than “Y”, then probably it’s a cat.
The problem is that there are dogs that are much smaller than cats, and there are dogs that have shorter snouts than cats; so the scientists would need to produce and constantly revise a very long list of rules to increase the probability that the programme will correctly identify the images.
Instead, if they decide to develop a machine-learning algorithm, then the scientists can train the algorithm’s statistical model on many pre-identified images of cats and dogs, and then leave the algorithm to develop on its own a set of rules that will allow it to accurately identify new images as cats of dogs. In principle, the more pre-identified data the algorithm is trained on, and then the more new data the algorithm is fed, the more accurate its prediction or classification ability should become. This video offers a good illustration of how machine learning works.
However, the way machine learning works also has limitations and carries its own set of challenges. If the data used to train the system are biased, if for example the data don’t contain enough images of a particular type of cat or dog (or, as in real-life cases, if they don’t contain enough images of women or people of colour or any other minority or disadvantaged group), then the algorithm may develop a wrong set of rules that will make it commit systematic mistakes. In addition to that, the fact that a machine-learning algorithm can write its own sets of rules without human intervention or even knowledge means that it may be very hard to understand how the algorithms works and to make it accountable.
Often, algorithmic systems are what’s known as “black boxes” because it’s not publicly known how they work: we don’t know what goes on inside the “box”. In the case of machine-learning algorithms, the systems may become black boxes even to the people who designed them, because the algorithms can rewrite their own rules. When such algorithms affect public life, then the public should be able to know how those algorithmic systems work and to make them accountable: we should be able to audit those algorithms.
The speed with which algorithms are being developed and implemented in all kinds of fields by public entities and private organisations, the lack of public awareness about such technologies and often also the lack of adequate regulation, make it hard for a properly informed public debate about the use of algorithmic systems in our societies. That is especially worrying as many algorithms are opaque and unaccountable and have proved to be systematic biased against women and minority groups, and also because often algorithmic systems fail to deliver the expected or promised results.
The OASI Register compiles information about algorithms with the aim of increasing public awareness and providing the necessary knowledge for an informed public conversation, and of making possible for experts and the public to search and analyse the use and social impact of algorithmic systems across the world.
Browsing the OASI Register
-You can browse the OASI Register or search its content by clicking the magnifying glass icon on the top right corner of the table and typing any words into the search field.
-You can also select which Register categories to show and to hide by clicking on “Hide fields” on the top left corner of the table, and you can use the “Filter”, “Group” and “Sort” options to cross-reference values from the different categories in the OASI Register. For example, you could make the Register show you all the entries that have “Germany” or “France” in “Location” and “social services” in “Domain”, but not “Threat to privacy” in “Social impact”.
-If you refresh the webpage, all the filters and options get reset and the default view gets restored. There are many possibilities to explore the OASI Register, and we invite you to try different options.
Disclaimer: Because the field of algorithmic systems is constantly and quickly developing, the OASI Register is necessarily a work in progress that will be regularly updated. By using a particular set of categories to catalogue algorithms, we have tried to be as comprehensive as possible while still keeping the OASI Register manageable, but we cannot claim to have included every little bit of relevant information about a particular algorithm.
As the field of algorithms develops and as we gather more information, we may modify the OASI Register categories or add new ones. While all possible effort has been made to verify all the information at the time that each was entry was last updated, we cannot claim 100% accuracy.
The OASI Register is a collaborative effort: if you see any mistakes or think there are some data missing, or if you would like to contribute content or to know more about OASI, please get in touch with us. You can read more about our methodology here.
The field of algorithmic systems, where different kinds of public and private organisations and entities develop, test and implement algorithms for all kinds of different purposes, has been developing very quickly in the last years. In part because of that reason, and also due to the fact that some technical expertise may be needed to understand the way algorithmic systems work, regulation about the use of algorithms lags behind their actual implementation, and the public debate about algorithms is not sufficiently informed or sometimes simply missing.
The OASI Register compiles information about algorithms with the aim of increasing public awareness and providing the necessary knowledge for an informed public conversation, and of making possible for experts and the public to search and analyse the use and social impact of algorithmic systems across the world.
To find out information about algorithms, we keep track of reports published in the specialised and also mainstream press as well as in academia, and we proactively reach out to people and organisations working in the field to gather data and information about algorithmic system and their functioning. We aim at registering algorithms that have been proved or shown to have some kind of negative social impact. Because we can’t track every single algorithmic system that’s being developed, we try to add algorithms that are representative of the different domains, aims and social impacts in the whole field; and we make an effort to include algorithms being used in different world regions. Each entry in the OASI Register lists the sources of information we consulted about that algorithm. Where the information is not available, we have written “N/A”.
Categories
Because the field of algorithmic systems is constantly and quickly developing, the OASI Register is necessarily a work in progress that will be regularly updated.
At this point, any effort to compile and classify information about algorithms will have to rely on a conventional set of categories: in the OASI Register, we have tried to be as comprehensive as possible regarding the categories while still keeping the list of them and the whole register manageable. These are the categories and definitions we are currently using:
| Algorithm |
The name of the algorithm if it has one and we know it, or a short description of what the algorithm does. |
| Developed by |
The name/s of the organisations, companies or other institutions that have developed the algorithm. |
| Adoption stage |
Whether the algorithm is being developed, in use or no longer in use. |
| Implemented by |
The name/s of the organisations, companies, public bodies or other institutions that have implemented or are implementing the algorithm. |
| Location |
The jurisdiction/s where the algorithmic system is being or has been implemented. By “jurisdiction” we mean a city, region, state, transnational body or any other territory over which a legal or normative authority extends. |
| Implemented since |
The date when the algorithm started being implemented if that has been the case. |
| Implemented until |
The date when the algorithm stopped being used if that has been the case. |
| Domain |
The area of society or the economy, or a sphere of activity, where the algorithm is being or has been implemented. We have adapted the list of domains used by the European Commission in its proposed regulatory framework on AI:
|
| Aim |
The purpose, intention or desired outcome of the algorithmic system:
|
| Social impact |
The particular ways, fields, issues or areas of social or private life affected by the implementation of the algorithm:
|
| Has it been audited? |
Whether the algorithmic system has been reviewed by an organisation that is financially, politically and otherwise independent from the organisations, companies or institutions that developed the algorithm. |
| Jurisprudence |
Whether there are court cases that have discussed and passed judgement (or are expected to do so) about how the algorithm was developed, how it works, and/or what kind of impact it has on social and private life. |
| Links and sources |
Links to and references about the available primary and secondary sources of information regarding the algorithm. |
As the field of algorithms develops and as we gather more information about existing and new algorithmic systems, we may modify the OASI Register categories or add new ones. We will explain on this website any modification we do to the categories and definitions in the OASI Register.
Because of how fast the field is developing and the many actors involved worldwide, and while all possible effort has been made to get all the pertinent information about a particular algorithm and to verify it before adding it to the OASI Register, we cannot claim 100% accuracy or to have included every little bit of relevant data about every algorithm.
We envision the OASI Register is a collaborative effort: if you see any mistakes or think there are some data missing, or if you would like to contribute content or to know more about OASI, please get in touch with us (eticas@eticasfoundation.org). You can also submit information about an algorithm yourself through our online form.
The implementation and use of algorithms by public entities and private organisations always have some kind of consequences on social life, as they are often based on data about the public and deal with the distribution of public resources and the delivery of public services. If the data are biased or if the algorithms don’t work as intended or are designed to discriminate among people in an unjust way, then the use of such algorithms will have a negative side and harmful effects, and that’s what we mean here by social impact: threat to people’s privacy; unjust discrimination based on gender, race, religion, socioeconomic status or other issue; reproduction of existing inequality; weakening of democratic practices; state surveillance…
There are different sources of potential bias that can statistically affect algorithm outcomes throughout their life-cycle, but the social impact will be constrained by the socially-given meaning of the resulting discrimination or harmful effect.
It is due to those reasons that we need to keep in mind that, despite their seemingly neutral mathematical nature, an algorithm developed for a concrete service and taking all reasonable steps in its design, may still produce and reproduce biases that unjustly discriminate against women, people of colour, minorities, the poor and other traditionally excluded groups.
The Social Impact of Algorithms
Having processed more than a hundred algorithms of different kinds and aiming to tackle algorithmic bias rigorously and systematically, the Eticas Foundation team has defined the following discrimination taxonomy.
Racial discrimination
This refers to discrimination against individuals or groups on the basis of race, colour, descent, national origin or ethnic or immigrant status. An example would be some recidivism-predicting systems, which have been proven to be inefficient and racially biased against some populations.
Socioeconomic discrimination
This is the prejudice against individuals based on their income, level of education, professional status and/or social class. An example could be when insurance companies use machine-learning algorithms to mine data –such as shopping history– to label some customers as high-risk to charge them more for their insurance packages.
Religious discrimination
This consists in treating a person or group differently because of the beliefs they hold. As an example, research has shown that some machine-learning algorithms were using words related to the Muslim community to search for misconduct and potentially risky behaviour on social media.
State surveillance
Some algorithmic systems may contribute to practices of surveillance of individuals or groups by state bodies or by private organisations that are not the result of due process, which haven’t been properly sanctioned or audited, or which aren’t transparent and respectful of people’s rights.
Threat to people’s privacy
Automated decisions made without human judgment may affect the right of individuals to their own private sphere. This threat is boosted by the large amounts of data processed by algorithmic systems (for example, data from people’s social media use), which allows public and private organisations to infer highly sensitive and private information about individuals .
Generating addiction
Some algorithms may contribute to make people addicted to using or relying on particular products or activities in an unhealthy or otherwise harming way. That can happen, for instance, when gaming apps, social media or broadcasting services behave strategically to keep users in the app for as long as possible.
Social polarisation / radicalisation
The implementation of some algorithms may result in the production or distribution of online content that contributes to push individuals or groups towards extreme attitudes or behaviour. For example, algorithms used in social media may promote extreme and even violent content because it gets more clicks.
Manipulation / behavioural change
In some cases, algorithms may purposely or inadvertently contribute to modify people’s thinking, beliefs or way of behaving or acting without their awareness or in an unhealthy or otherwise harming way. That can be the case when algorithms produce highly targeted and personalised propaganda or advertising that manipulate the way people normally behave.
Disseminating misinformation
The use of algorithms may result in the production or distribution of online content that’s purposely untrue, wrong, partial or that in other way contributes to make people think or believe something that’s not true. That has been the case, for instance, regarding the climate crisis or the use of vaccines, about which there is scientific consensus.
The field of algorithms is quickly and constantly developing, and we know we may be missing information about interesting and relevant algorithmic systems. Please help us make the OASI Register better by telling us about algorithms that we haven’t found ourselves. Thank you!