Study: Airbnb’s reputational system builds trust & overcomes biases

By Alok Gupta 

Data Science, Airbnb

As a data scientist, the vast majority of my work is, by its nature, objective.

Yet, at Airbnb, our business is grounded in one of the most subjective concepts there is: trust. In order for home sharing to work, you have to trust another person and trust that Airbnb will be there for you if anything goes wrong.

It’s no easy feat, and this Olympic-sized trust we’re building that extends into both the online and offline world can be tricky. But, tonight, approximately 2 million people are staying at an Airbnb, and we currently have 4 million listings available in over 191 countriesmore than the five biggest hotel chains combined.

This global growth cannot happen without a strong foundation in trust, which is why I set outalong with my co-authors at Stanfordto better understand the effects of what I believed to be one of Airbnb’s core drivers of trust: our robust reputational system comprised of our authentic user-generated reviews and star ratings.

Specifically, we wanted to look at how this trust through reputation building is scaling, particularly among people from dissimilar backgrounds and geographies. It’s human nature to trust people like yourself and be more skeptical of those who aren’t, a common bias referred to as “homophily.” But, given the rapid growth of the Airbnb community, we wanted to know whether Airbnb is creating connections that reinforced our existing social predispositions or if these mechanisms for trust are helping users connect with a broader community beyond those similar to themselves.

To that end, we ran this study in two parts. First, we split nearly 9,000 real Airbnb users into two groups and showed them a series of simulated user profiles that we developed on a fake online platform. We asked each group to invest “credits” in each profile as a measure of how much they would trust that person. One group only saw user profiles that were somewhat similar to themselves demographically – i.e., profiles of similar age, gender, marital status, and location. Meanwhile, another group saw profiles similar to themselves in addition to the profiles of users different than themselves, but with impressive star ratings and a high number of positive reviews.

The results are striking. While the first group invested more in the profiles that were most similar to themselves, confirming “homophily”, the second group put more stock in users who were wildly different from themselves but had better reputations, as noted by the reviews and ratings.

Time and again, these reputational mechanisms embedded in user profiles helped outweigh the subjects’ biases to help create strong connections with very different peopleconnections that, unfortunately, are much less likely to occur unaided out in the world.

Based on this initial experiment using fictional profiles on a fictional platform, we then took a look at what’s actually happening on Airbnb. We dug into 1 million real interactions between hosts and guests and found that hosts with better reputations were attracting more demographically diverse guestsmirroring the findings of the first experiment.

For those of you inclined to dig into the details of the study, you can find all of the information here in the Proceedings of the National Academy of Sciences. But, to cut to the chase, what does this mean and why does it matter?

Ultimately, this research shows that in an increasingly global and diversified world, we can design digital tools to successfully foster trust by developing a framework and method for reputation development. Not only does this reputation and trust improve the efficacy of the sharing economy, our work bears out that we can actually use online reputation to counter and overcome deeply ingrained biases in society. In fact, we already are.

Here at Airbnb, our carefully crafted review and rating system is central to our entire community. Our founders intentionally designed it for trust from day one, so that our hosts and guests can only review each other until after a reservation is completemeaning you can be confident that the feedback you’re seeing is informed, unbiased, and real. If you’re curious what previous guests have thought about your potential host or home, or if you want to know what another host’s experience has been with a prospective guest, all you need to do is take a look at their profile and reputation as a part of our global community. Just ask the 2 million people staying at an Airbnb tonight.

There’s much more work and research to be done to better understand how to fight bias and discrimination (this particular study didn’t examine race), but we believe that investing more in our community reviews and ratings system is an important way to achieve this critical goal. We’re also engaged in a series of other projects that help fight discrimination and make our community fair for everyone, and you can learn more about that work here.

As subjective and personal as trust is, it’s the fundamental currency of the sharing economy, and the implications of these findings could be widely felt. If more companies and organizations can leverage digital mechanisms to build up individuals’ reputations and cultivate trust more efficiently and authentically online between people of varying backgroundsthe possibilities are endless.