Bias in Online Marketplaces

In the last section, we review literature on biases in online marketplaces.

In an ideal world, online marketplaces would be efficient, fair, and accessible to all. Unfortunately, empirical evidence shows real world online marketplaces exhibit biases, favoring certain groups of users over others. In this section, we review some known empirical studies showing biases in online marketplaces with regards to race, gender, status (both contextual and socio-economic). Finally, we also show recent advances in Artificial Intelligence (AI) and how AI can exhibit even subtler algorithmic biases with regards to texts and photos.

Bias is an inherent part of human nature. One of the most fundamental principles of social interaction is that similarity breeds connection -- the principle of homophily, which is defined as "a contact between similar people occurs at a higher rate than among dissimilar people" [1]. In a review published in 2001, McPherson et al. showed that the homophily principle "structures network ties of every type, including marriage, friendship, work, advice, support, information transfer, exchange, comembership, and other types of relationship" [1].

Online marketplaces are influenced by the principle of homophily as well. The bias of homophily was observed in the before-mentioned online experiment using Airbnb users [2]. It was shown that this bias can be offset with reputation systems, and we have inspected literature on reputation systems extensively in the last chapter. However, reputation systems themselves also suffer unique biases as we showed before, e.g. reputation inflation.

With the underlying principle of homophily, we now review studies showing biases in marketplaces regarding different factors, including race, gender, and status.

1. Racial bias

Before the popularity of online marketplaces, researchers have observed racial bias in labor markets. In 2004, Lavergne and Mullainathan published the paper "Are Emily and Greg More Employable than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination" [3]. In the study, researchers manipulated perceived race with resumes randomly assigning African-American or White-sounding names. As a result, white names received 50 percent more callbacks for interviews and the racial gap is uniformly found across occupation, industry and employer size. Following exactly the same field experiment setup, in 2015, Edelman et al. conducted the similar field experiment on Airbnb, manipulating perceived race by varying profile names. The field experiment on Airbnb showed that ethnic discrimination persists in this prominent online marketplace for lodging [4]. In addition to names eliciting racial bias, another experiment on eBay [5] used photos as stimuli. The experiment showed local online classified advertisements in U.S., each featuring photographs including a dark or light-skinned hand -- it was found that black sellers receive fewer and lower offers than white sellers, and the correspondence with black sellers indicates lower levels of trust [5].

Finally, in addition to field experiments, there was also additional evidence of racial discrimination through data analysis of observed data from other freelance marketplaces (TaskRabbit, Friverr) [6]. In the study, researchers collected dataset of TaskRabbit and had crowdworkers label the profile images with race and showed workers who are perceived to be black received significantly fewer reviews and worse ratings.

2. Gender

Gender gap is exhibited both in representation, and in performance. In what pre-dates online marketplaces for physical resources exchange, gender representation gap has been observed in online knowledge sharing and collaboration platforms. In 2012, Vasilescu et al. provided empirical evidence on StackOverflow on the representation and participation gap on gender, showing that that "men represent the vast majority of contributors to Stack Overflow" and that "men participate more, earn more reputation, and engage in the 'game' more than women do" [7]. Similar gender representation gap was reported in editors of Wikipedia, a survey in 2010 found that fewer than 13% of Wikipedia contributors were women. However, the representation gender gap is more nuanced once taking the activity level and contribution size into consideration, as Antin et al. reported in 2011, e.g., "among the most active Wikipedians men tended to make many more revisions than women" but "most active women tended to make larger revisions than the most active men" [8].

In addition to the representation bias, women sellers were found to receive a smaller number of bids and lower final prices on eBay auctions compared to men counterparts [9].

3. Status

The final source of bias we discuss is status, both contextual and socio-economic. By contextual status, I refer to the power imbalance induced by a specific situation, e.g. the case of Couchsurfing. Couchsurfing was founded in 2003, and is an unpaid platform for exchange of online lodging. There are two roles on Courchsurfing, a host who provides the lodging, and a surfer, who stays with the host for free. Thus the host would have power over a surfer, which creates a contextual status. In a study published in 2016, researchers used Courchsurfing mutual rating dataset between hosts and surfers and showed an asymmetry in ratings (meaning the hosts gave lower ratings than surfers did), potentially due to the mechanism of status-giving, according to to power dependence theory [10].

In addition to contextual status, several studies on online exchange platforms observed geographically induced bias in socio-economic status. A study published in 2017 highlighted the geographic bias of sharing economy platforms, showing that "sharing economy is significantly more effective in dense, high socioeconomic status (SES) areas than in low-SES areas and the suburbs" [11]. Researchers provided two empirical evidence for such geographic bias: first, on UberX, lower socio-economic status areas have higher wait times; second, on TaskRabbit, workers are less willing to travel to lower socio-economic areas and charge higher prices for tasks in those neighborhoods as a result.

We discussed empirical studies on biases on online marketplaces with regard to specific factors, such as race, gender, and status above. Below we shift gears to discussing biases with regard to more subtle factors, as a result of the fast developing Artificial Intelligence (AI).

During the same time online marketplaces gained popularity, AI, especially applied AI, has been making rapid advancements. The core of AI is machine learning algorithms and massive amount of training data. As these online marketplaces centralize data at scale, machine learning algorithms are gradually being implemented on these platforms as features, including recommendation, search optimization, and decision making assistance (e.g., systems used to rank teachers, decide who gets a loan or parole).

Recommendation systems are being adopted to address the problem of scale of online marketplaces. As the number of potential exchange partners exceed what an user could possibly screen trough manually, recommendation systems can surface potential matches more efficiently. Horton showed that when employers were offered algorithmic recommendations, a large portion followed and that "experimentally induced recruits were highly positively selected and were statistically indistinguishable from the kinds of workers employers recruit 'on their own.'" [12].

However, machine learning algorithms that power recommendation systems are opaque, and different features might work together to "construct" a feature that is highly similar to race. Further, many training data that these algorithms based on are biased, as shown in our review in various platforms above. As a result, algorithms trained on these biased data can also pickup these biases. For example, potentially due to the representation bias in training data, one study showed that Google's automatic speech recognition systems didn't recognize women's voices as well as men's. As we speak, the algorithms' capabilities are only expanding. In 2017, Ma et al. published a paper developing algorithms that can predict whether the self-description of a host on Airbnb will be perceived as more or less trustworthy [13]. A recent study using online dating profiles also developed an AI can detect sexuality based on profile pictures [14].

This brings us to the question: what can we do to reduce the bias in online marketplaces? One approach is algorithmic. Previous studies on correcting machine learning algorithms [15], giving people the tools to "debug" the data through visualization before training the algorithms from Google People+AI Research Initiative (PAIR). Another approach is policy-driven. Levy and Barocas analyzed "ten categories of design and policy choices through which platforms may make themselves more or less conducive to discrimination by users", divided to three groups: setting policies, structuring interactions, and monitoring and evaluating [16]. As AI bias is a complicated problem, the solution might be complex as well. It might require a combination of many approaches (algorithmic, policy-driven, and other) to understand and address the bias problem efficiently for the future of large scale online marketplaces.

References