Detecting Risk in Real Time: Introduction

@dalexeenko|March 20, 2023 (3y ago)140 views

This is the first in a series of blog posts on building risk detection and mitigation systems.

#Introduction

Over the years, a number of friends running startups have asked me for advice on dealing with fraud, risk, and building teams equipped to handle it. After countless chats, I figured it might be helpful to write down some thoughts on the subject.

When starting a business, fraud prevention probably isn't at the top of your list. You're focused on growth, which is 100% the right thing to do (startup = growth). However, as a company scales, it starts to attract bad actors who are eager to exploit its success. E.g., as soon as you have a substantial enough online presence, you'll start seeing disputes pile in. There's nothing special about fraudsters targeting your company specifically. Most of the time fraudsters simply diversify their portfolio and attack as many companies as they can. They play the numbers game. It's easy not to be prepared for this. I first ran into this problem years ago at Microsoft when we were fighting email spam. Later, at Airbnb, I saw how crucial it was to build trust in a marketplace where strangers rent their homes to each other.

Here's the thing about doing business online: unlike a physical store, you can't see your customers. When someone buys something from your website, you don't know much about them beyond maybe their credit card info. If you're running a platform for online stores, you might not even know if those stores are real. This invisibility makes it easier for people to do things like order products and never pay, or set up fake stores that only fulfill half their orders. It gets even trickier with marketplaces where buyers and sellers could work together to scam the system.

At first, you might think the solution is to just block any account that looks remotely suspicious. But that's not always the best move. You could end up turning away quite a few good customers. The real challenge is figuring out how to let the good users in while keeping the bad ones out. It turns out, that's a really hard problem. In this post, I'll share some ideas on how to build systems that can help protect your business from risk as it grows.

#What is risk

To manage risk, you first need to know what you're up against. It's not just one thing — risk can show up in many different ways depending on your business. Let's look at some common examples:

Financial fraud. At some point your customers will dispute some of your charges. Sometimes it can be that they are unhappy with the product or service, or they couldn't find how to cancel a subscription. Other times or it could be that a fraudster used a stolen credit card to buy your product, prompting the original card holder to issue a dispute. If too many customers dispute your charges, it can quickly erode margins.
Bots and bad or inappropriate content. These can make users lose trust in your platform.
Account takeovers. When someone hacks into a user's account, it can cause a lot of damage and undermine user trust.
Misuse of free trials, promotions, and coupons. Some people might use your free trial for things like mining Bitcoin or hosting banned content.

At Airbnb, we grouped the risks that we faced into four levels (in no particular order):

Tier 1. Financial harm to the platform. Things like stolen credit cards or abusing promotions or coupons.
Tier 2. Financial harm to users. Fake accounts, fake listings, phishing, spam, account hijacking.
Tier 3. Property damage. E.g., parties that trashed someone's home.
Tier 4. Personal safety. Anything from arguments to more serious incidents.

The most common fraud schemes we saw were:

Hacking host accounts to steal payouts. When legitimate guests made reservations, the payout went to the bank account controlled by a fraudster.
Hacking guest accounts to make fake bookings. E.g., use the credit card saved on the guest's profile to book a fake listing created by the bad actor.
Creating fake listings to trick guests into paying outside the platform.

At Stripe, we looked at fraud differently, given the different nature of the product. We constantly thought about the following four scenarios:

Good seller [in bad circumstances] + good buyer combination was typically tied to credit risk. We saw a lot of this at the beginning of COVID-19: quite a few merchants (especially in lodging, restaurants, entertainment industries) went out of business and couldn't deliver their services.
Good seller + bad buyer meant fraudulent transactions on good merchants. The most common example of this is bad actors leveraging stolen credit card data to buy (and resell) high-value goods (say, Apple products).
Bad seller + good buyer was fraud risk. We called it a predatory or fraudulent business. This happened when people ordered things online and never received it. A more sophisticated version of it was when a merchant sold some hot new item at a discount but fulfilled only 50% of orders. This was much harder to detect.
Bad seller + bad buyer was fraud risk. We called it card cashing, or credit card collusion. This happened when a bad actor set up both merchant and buyer accounts and tried "buying" imaginary things with aforementioned stolen credit cards, only to cash out the proceeds.

Stripe has a great introduction to risk management that goes into these types of risk in a lot more detail.

Understanding how exactly bad actors try to exploit your product will help you figure out how to better detect, diffuse, and deter them. Each type of fraud requires its own approach. Remember, the goal isn't to create a perfect system that works against everything. The aim is to make it difficult enough for bad actors that they find it not economically viable and decide to go elsewhere.

#How to approach risk

To stop any potential bad actors at Airbnb and Stripe we collaborated across product, engineering, operations, customer support, sales and data science, and employed a number of defensive systems:

Machine learning to detect anomalous behavior. We leveraged everything from logistic regressions, to random forest classifiers, XGBoost, and deep neural networks.
Friction flows to stop bad actors and to allow good users to prove their trustworthiness. Examples include 3D Secure, micro-authorization, billing statement verification, ID verification.
Integrations with many different sources of information to build a complete profile of user's risk: from integrations with Crunchbase, D&B, Trustpilot, to scraping user websites and social media. This information enabled us to make split second decisions.

The two primary questions that we had to continuously reevaluate were:

What was the right level of optimization between (a) financial losses, (b) good user experience impact and (c) operational expenditure (human reviews, vendor spend, third-party data and tooling).
How much we would invest into the prevention of account takeovers, fake accounts, chargebacks, credit risk, etc. The level of investment changed for each of these areas as some defenses became stronger and bad actors shifted their efforts.

In part two, we talk about building the strong risk system architecture and highlight top learnings about building defensive systems: from machine learning, to operational reviews, and third-party integrations.

Thanks to Dasha Cherepennikova, Eugene Shapiro, Simon Hachey, Steve Kirkham, and Tara Sandhu for reading drafts of this.