article key points writing

The Journal of Business Leadership, vol. XX, Number X

AI AND ALGORITHMIC FAIRNESS IN CONSUMER

LENDING: THE CASE OF “UPSTART”

Naveen Gudigantala, University of Portland

ABSTRACT

It is estimated that approximately 45 million Americans don’t have access to bank quality

credit. This work discusses the case of a fin-tech company called Upstart, which specializes in

using AI/ML based platform to provide credit to traditionally underserved populations. Upstart’s

AI platform uses alternative data in addition to the traditional FICO scores in its algorithms.

This alternative data includes borrowers’ educational data and occupational data. Upstart’s

data shows that a majority of traditionally underserved populations was able to obtain more

credit and at better terms using their credit scoring system. There is clearly a promise of using

AI/ML systems for expanding credit to underserved populations, but under the promise lurks

dangers of misuses of such technologies. This work also discusses the issues of algorithmic

fairness, policy status, algorithmic transparency and compliance, and offers directions for future

research.

INTRODUCTION

Issues surrounding the fairness of algorithms are attracting much attention from the

researchers (Saxena et al., 2019). The goal of this paper is to discuss the opportunities and

challenges in using alternative data for credit scoring modeling. The case study uses a fin-tech

company “Upstart Network, Inc.” (called “Upstart” from here on) and an analysis of Upstart’s AI

practices in lending to address the following questions:

- How does fin-tech business model work in consumer lending?

- How are AI & ML used in fin-tech industry with regards to consumer lending?

- How do different approaches to the development of ML models help or hinder fairness in lending?

- What benefits can consumers get when AI models are used for underwriting?

- What are the limitations of AI models?

- How do transparency and model compliance efforts benefit fin-tech companies?

- What are the policy issues concerning credit scoring models using alternative data?

- Is the alternative data used in this context really generating fair AI algorithms?

- Further issues in the use of AI/ML systems in the fin-tech industry regarding consumer credit

This case study is intended for researchers in AI and financial services, students learning

analytics/AI, and for practitioners doing AI/Data science work. The issues discussed in this case

will help students better evaluate the implications of models they learn to create as part of

analytics curriculum; for the researchers to continue investigating the problems raised in this

study; and for data science practitioners to reflect on issues of algorithmic fairness.

The Journal of Business Leadership, vol. XX, Number X

UPSTART’S BUSINESS MODEL

Upstart is an online lending platform, launched by ex-Google employees in 2014, with an

aim to provide credit to people with limited credit or work history. Figure 1 illustrates the

business model of Upstart. Consumers in need of credit approach Upstart, a website embedded

with Artificial Intelligence (AI)/ Machine Learning (ML) technologies, to inquire and create a

loan application. Upstart automated the underwriting technology for credit scoring, meaning,

given the information provided by the consumer, a model will decide whether to give or reject

loan and loan terms. The use of a model – as opposed to a human – for decision-making refers to

AI and the model itself may be developed using one or many machine learning (ML) algorithms.

The Upstart’s website is cloud-based, meaning the consumer data and underwriting technologies

operate on the Internet by providing online services to the consumers. This type of cloud service

is an example of software-as-a-service (SaaS). Upstart is essentially a middleman between Cross

River Bank, a bank which issues loans, and the consumer. Because Upstart’s underwriting

technology is used by Cross River Bank, we regard Upstart as a financial technology company

(“fin-tech” in short). Accredited investors and institutions are given an opportunity to invest in

the loans thus disbursed by Cross River Bank and Upstart to make money (Upstart, 2017).

Figure 1. Business Model of Upstart

PROBLEM ADDRESSED BY UPSTART FOR CONSUMER LENDING

Upstart learned from an early study that although 83% of Americans have never actually

defaulted on a loan, only 45% have access to bank-quality credit. Upstart notes this “45% vs.

83% gap” as unfair and sets out to create an AI platform that can make ingenious use of

alternative data in expanding credit to underserved groups (Girouard, 2019).

Girouard (2019), the CEO of Upstart, suggests that FICO score - a measure of credit risk

available through credit reporting agencies such as Equifax, Experian, and Transunion – is

limited in its predictive ability of consumer risk because it focuses exclusively on a consumer’s

past credit history. Therefore, traditional lenders who rely almost exclusively on FICO score and

traditional modeling techniques ignore some important predictive information about potential

borrowers. This is one of the reasons contributing to the “45% vs. 83% gap” (Girourad, 2019).

To overcome this problem, Upstart’s underwriting model, in addition to using FICO scores, uses

The Journal of Business Leadership, vol. XX, Number X

alternative data for borrowers, such as educational attainment and work history as predictors.

Using the model with alternative data, Upstart claimed that 27% more loans are approved which

also lowered interest rates by an average of 3.57% (Girourad, 2019). Although Upstart used

education and work history as alternative data, other pieces of data such as payment history

concerning rent, electricity, gas and telecom bills, repayments to payday lenders can be

considered as alternative data. The major U.S. credit reporting agencies are initiating attempts to

include alternative data in their credit scoring system, but they face several hurdles in fully

capturing this information (Malik, 2019). Therefore, opportunities emerge for companies such as

Upstart to ascertain creditworthy individuals with near prime FICO scores and create a business

model around such customers.

A BRIEF OVERVIEW OF UPSTART’S UNDERWRITING

METHODOLOGY FROM ML PERSPECTIVE

The business problem in this context is to assess the creditworthiness of a consumer.

When a consumer applies, two decisions need to be made: (1) whether to give a loan or not? and,

(2) if a loan is given, what are the terms? The terms refer to amount of loan, annual percentage

rate (APR,) etc. A machine learning (ML) model is developed which is essentially a

mathematical model for decision-making purposes. Table 2 shows some examples of elements

that go into the model development process: the target variables, traditional predictors, predictors

including alternative data, and possible modeling techniques. Upstart predictors include

traditional variable such as, FICO score and length of credit in years, but also alternative data

such as highest degree earned, area of study, and job history details (Upstart, 2017).

Table 1

ELEMENTS OF ML MODES FOR CREDIT RISK MODELING

Target variable Traditional

predictors

Predictors including

alternative data

Possible modeling techniques

Give a loan/Reject a

loan

FICO score, length

of credit in years

FICO score, length

of credit in years,

highest degree

earned, area of

study, job history

Logistic regression, K-nearest neighbors,

Classification Trees, Neural Networks, etc.

The Annual

Percentage Rate

(APR)

FICO score, length

of credit in years

FICO score, length

of credit in years,

highest degree

earned, area of

study, job history

Multiple regression, K-nearest neighbors,

Regression Trees, Neural Networks, etc.

HOW DO DIFFERENT APPROACHES TO THE DEVELOPMENT OF ML MODELS

HELP OR HINDER FAIRNESS IN LENDING?

In credit risk modeling, an important and universally used predictor is FICO score/credit

score. The FICO scores range from 300 to 850. Many lenders consider borrowers with FICO

scores of at least 720 to be “prime”. The next classification, “near-prime” generally falls in an

interval of mid-to-high 600s to the low 700s. The third classification of “sub-prime” includes

borrowers whose scores fall below 620 (Andriotis, 2016). The FICO score distribution of U.S.

population as of April 2018 is shown in Table 2. As per this data, the individuals with credit

scores between 300-600 don’t qualify for bank-quality credit (Dornhelm, 2018). Individuals with

The Journal of Business Leadership, vol. XX, Number X

credit scores above 700 (58.2% of population) usually qualify for best possible terms. The “near

prime” from this table can be loosely categorized as the percentage of people between scores 600

and 700, and they stand at 22.6% of U.S. population. This segment of population can be

considered as “traditionally underserved” in terms of credit.

Table 2

DISTRIBUTION OF FICO SCORES OF U.S. POPULATION (DORNHELM, 2018)

FICO Score Percentage of U.S. population

300-600 19.1%

600-649 9.6%

650-699 13%

700-749 16.2%

750 and above 42%

What is the problem with FICO score as an important predictor? The FICO credit scores

are unduly impacted by the length of credit history of an individual. Even the other components

that go into the calculation of FICO scores such as payment history, new credit, credit mix, and

credit utilization also favor individuals with longer credit history. In conclusion, FICO scores

inherently create bias against younger borrowers or recent immigrants with fewer accounts

(called “thin files”), lower credit limits, and fewer years of making payments. These types of

borrowers have substantially lower credit scores compared to older borrowers. Another

interesting point noted by Upstart is that a majority of traditional lenders use the length and

breadth of borrowers’ credit files as independent criteria in making determination of loans

(Upstart, 2017). Please see the data in table 3 and a scatterplot in figure 2 showing the positive

linear relationship between Age and FICO score (Dornhelm, 2018).

Table 3

DATA OF AGE AND CREDIT SCORE AS OF APRIL 2018 (DORNHELM, 2018)

Age Range of U.S. Individual Average FICO Score

18-29 659

30-39 677

40-49 690

50-59 713

60+ 747

So what happens if a model predominantly uses FICO score to assess the

creditworthiness of an individual? Upstart conducted a study in September 2016 with a random

sample of their borrowers during the years 2014-16. It used two models to do the comparison: a

limited model with no alternative data (used FICO score and length of credit history) and an

Upstart’s model with alternative data. The results are presented in table 4 and show that the use

of alternative data in credit modeling results in better credit terms and also improves the

predictive accuracy of the model (Upstart, 2017).

The Journal of Business Leadership, vol. XX, Number X

Figure 2. Relationship between Age and Credit Score as of April 2018 (Reference:

Dornhelm, 2018)

Table 4

RESULTS FROM COMPARISON OF CREDIT MODELS BY UPSTART

Limited Model (No alternative data; use of traditional

variables)

Upstart Model (traditional variables plus alternative

data)

Model recommended average APR of 23.5% Model recommended average APR of 16.7%

Model has lower R^2 for predicting default rate Model has higher R^2 for predicting default rate

WHAT BENEFITS DID UPSTART CONSUMERS GET WHEN AI MODELS ARE

USED FOR UNDERWRITING?

Dave Girouard (2019), CEO of Upstart, presented the following benefits of using AI system

to the House Committee Taskforce:

1. Upstart’s models approved 27% more consumers and lowered interest rate by an average

of 3.57% compared to traditional models.

2. For a near-prime consumers (620-660 FICO), Upstart’s models approved 95% more

consumers and reduced interest rates by an average of 5.42% compared to traditional

models.

3. Upstart’s models provided higher approval rates and lower interest rates for every

traditionally underserved demographic.

Upstart reported to have facilitated 80,000 loans totaling over $1 billion. The loans typically

fall in the range of $1,000 to $50,000 with repayment periods between 3 and 5 years. The

average age of the borrower is 28 years with the APR rates ranging between 4% and 25.9%

(Upstart, 2017).

An interesting aspect of these statistics is that the average age of borrower for Upstart’s

services is 28 years. What do Upstart’s consumers do with this money? A majority of Upstart’s

borrowers paydown higher interest credit card balances, use them to consolidate payday loans,

reduce student loans, or to pay tuition for graduate education (Upstart, 2017).

The Journal of Business Leadership, vol. XX, Number X

WHAT ARE THE LIMITATIONS OF AI MODELS USED BY UPSTART?

Any AI model that is developed within ‘certain constraints’ will not work well outside of

that specific environment. In this instance, Upstart appears to focus on relatively young

borrowers with limited credit history but good educational background and work history.

Looking at it from another perspective, Upstart’s models focus more on the future financial

potential of their borrowers – mostly appearing to be students and recent graduates – than the

traditional models which look at the past credit history of borrowers. Therefore, Upstart (2017)

acknowledges that their underwriting models may not be equally predictive across all

demographic groups, meaning that the benefits similar to those offered by Upstart to their “thin

file” consumers may not be as attractive to older borrowers.

HOW DO TRANSPARENCY/MODEL COMPLIANCE EFFORTS BENEFIT FIN-TECH

COMPANIES?

This section highlights how proactively working with regulators and good faith efforts in

maintaining transparency can yield good results, which further boost the competitive advantage

of companies. The government regulator in the field of consumer lending is the Consumer

Financial Protection Bureau (CFPB). Prior to launching their automated lending platform,

Upstart proactively approached CFPB and held discussions on appropriate ways to measure bias.

Upstart considered CFPB’s feedback in developing automated tests and shared the results of how

their credit decisions have positively benefited underserved groups on a routine basis. Upstart

also created a model compliance program which performs regular fair lending testing of its

underwriting model. This is important because AI models may change frequently. Upstart agreed

to notify CFPB prior to adding any new variables to their model. Upstarts’ compliance program

is monitored through internal and external processes and audits. They also train employees on

maintaining compliance. Further, if any of the automated results show bias, Upstart agreed to

take appropriate corrective action as necessary (Upstart, 2017).

Years of good faith efforts by Upstart have resulted in CFPB awarding a “no-action”

letter to Upstart, a first such letter issued by the regulator (Noto, 2017). Companies launching

new financial products can request for such letters and granting of “no action” letter in this

context means that CFPB will not initiate any enforcement or supervisory action against

Upstart’s automated model in regard to the Equal Credit Opportunity Act (ECOA) and its

implementing regulation B (Noto, 2017).

A “no-action” letter helps Upstart to continue the use of their automated underwriting AI

model without any regulatory uncertainty. More than that, it will serve as a signal for their fair

business practices, which will boost their competitive positioning in the market.

POLICY ISSUES CONCERNING CREDIT SCORING MODELS USING

ALTERNATIVE DATA

Where do lawmakers stand on the use of alternative data in credit scoring? Recently, the

United States House launched a bipartisan taskforce on financial technology headed by

Congressman Steven Lynch. The Congressman Lynch said, “the lives of consumers are changing

with user-friendly financial service apps but these emerging technologies come with

vulnerabilities and the need to reevaluate our consumer protection standards.” (U.S. House

The Journal of Business Leadership, vol. XX, Number X

Committee on Financial Services, 2019). The committee held its first hearing on June 25, 2019

and is expected to guide Congress in its policymaking. The Wall Street Journal reported that “the

Government Accountability Office, the independent watchdog arm of Congress, recommended

in December that financial regulators provide clearer guidance to lenders about how to use

alternative data in the underwriting process, something that hasn’t yet happened” (Hayashi,

2019). Therefore, in the absence of regulatory policy making pertaining to the use of alternative

data, the burden is on the financial services industry to prove that they have not violated any

provisions of the Equal Credit Opportunity Act (ECOA) and its implementing regulation B. This

may be a huge risk, which partly explains why many of the major lenders are hesitant to innovate

with alternative data (Girouard, 2019).

IS THE ALTERNATIVE DATA USED IN THIS CONTEXT REALLY GENERATING

FAIR AI ALGORITHMS?

In assessing fairness of AI algorithms, it is important to use the provisions of the Equal

Credit Opportunity Act (ECOA) and its implementing regulation B. As per ECOA, certain

variables are prohibited from being used by creditors to make decisions on credit. These

variables are: race, religion, sex, age, color, national origin, marital status, receipt of public

assistance, and exercise of certain legal rights (CFPB consumer laws, 2013). However, the use of

education and occupation in making a credit decision is debatable (Hayashi, 2019). The ECOA

uses two principal theories when assessing liability: disparate treatment and disparate impact.

Disparate treatment refers to a creditor’s use of prohibited variables such as race or sex in

treating an applicant differently. Disparate impact refers to the adverse impact generated by the

use of creditors’ practices on members of protected class (CFPB consumer laws, 2013).

To assess the fairness of Upstarts’ algorithms using alternative data, we could examine if

education of a borrower is really a fair variable and not a prohibited variable. As per the U.S.

census data, the percentage of people with at least Bachelor’s degree among different race

categories is: 33% among whites, 54% among Asians, 25% among African Americans, and 16%

among Hispanics (Hayashi, 2019). This work used the United States census data and Bayes’

theorem to compute probabilities of race of an applicant given they have at least a Bachelor’s

degree (United States Census Bureau, 2019; National Center for Educational Statistics, 2017).

These probabilities are:

𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 ( 𝑊ℎ𝑖𝑡𝑒

𝐵𝑎𝑐ℎ𝑒𝑙𝑜𝑟𝑠 ) = 67.19%

𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 ( 𝐴𝑠𝑖𝑎𝑛

𝐵𝑎𝑐ℎ𝑒𝑙𝑜𝑟𝑠 ) = 9.36%

𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 ( 𝐴𝑓𝑟𝑖𝑐𝑎𝑛 𝐴𝑚𝑒𝑟𝑖𝑐𝑎𝑛

𝐵𝑎𝑐ℎ𝑒𝑙𝑜𝑟𝑠 ) = 9.76%

𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 ( 𝐻𝑖𝑠𝑝𝑎𝑛𝑖𝑐

𝐵𝑎𝑐ℎ𝑒𝑙𝑜𝑟𝑠 ) = 8.57%

A fin-tech company, even after not asking for racial information (a prohibited variable),

if it knows that a borrower has at least a bachelor’s degree, can very well predict how likely the

borrower belongs to a race at an aggregate level. From the probabilities seen above, one can

make a case that “education” may serve as “proxy for race” in the sense that using information

The Journal of Business Leadership, vol. XX, Number X

on education may disproportionately favor some race group over others. Given the number of

borrowers run into several hundred thousands, the above probabilities hold up really well, and

thus creating question marks regarding the fairness of using education as a predictor variable.

Similarly, many variables in alternative data may show strong correlations with prohibited

variables under ECOA and thus creating regulatory risk factors for companies.

How about the use of social media data (photos and text posted in Facebook, Twitter, and

Instagram), consumer purchase history, web browsing history as predictors in consumer lending

models? Are they predictive of consumers’ ability to repay? Do they introduce any bias which is

indicative of disparate treatment and disparate impact? These issues need to be further

investigated.

FURTHER ISSUES IN THE USE OF AI/ML SYSTEMS IN THE FIN-TECH INDUSTRY

REGARDING CONSUMER CREDIT

Peter Waynard, Senior VP of Data Analytics at Equifax points to three issues that are of

interest when using alternative data for making credit decisions: (1) The AI/ML systems used for

credit scoring must be transparent and explainable, (2) alternative data used for modeling must

meet the same high standards of data completeness and accuracy needed for data elements used

in consumer reports, and (3) the fin-tech companies must demonstrate that the use of their AI

algorithms do not create disparate impact prior to their platforms being used in the marketplace

(Waynard, 2018).

The issue of algorithmic transparency and explainability is very important in credit

scoring systems. Some algorithms such as classification and regression trees and multiple

regression are inherently good at explaining the rationale behind the model recommendation

whereas others such as Artificial neural networks are not. For some deep learning algorithms,

even the modelers themselves may be unsure of the underlying variable structure that produced

these results. Therefore, in the context of consumer credit, wherein the burden of proof lies with

financial services companies to show no disparate impact, the use of explainable algorithms

becomes important. Further research is needed to show which algorithms work better in this

context.

The issues of data accuracy and completeness is very important for successful

implementation of AI algorithms. However, government policies must first specify what are the

acceptable variables in regards to the use of alternative data. If such alternative data doesn’t have

the same fidelity as the traditional data, then the use of alternative data introduces bias into the

models. Further research is needed to assess which alternative variables are both fair and

increase predictive accuracy of the models and also the best practices in managing such non-

traditional data.

Third, to encourage innovation, financial services companies should be given an

opportunity to illustrate the effectiveness of their new AI models in an artificial setting – called

product sandboxes – prior to their release in the market place. To establish the accuracy of the

models – and their consequent promise of granting bank-quality credit to underserved

communities requires some beta testing. This testing may need the use of prohibited variables to

show how the new models with alternative data don’t discriminate against the protected classes.

To drive innovation, policy makers have to provide certain protections to companies working on

such innovations. The CFPB is conceptualizing guidelines in instituting the product sandbox

The Journal of Business Leadership, vol. XX, Number X

mechanism, but much more research is needed to ensure that companies don’t misuse such

innovations.

In conclusion, the use of alternative data offers much promise in our efforts offer bank

quality credit to millions of underserved Americans. Such promise is possible because of Big

Data and the use of AI and ML technologies. However, there is also a great danger that lurks in

the corner if companies don’t exercise due diligence in employing this new generation of tools

and technologies. This work attempts to show the efforts of an innovative company, Upstart, in

making strides in the use of AI/ML to expand credit, and also given the challenges concerning

this nascent phenomenon, calls for further research.

REFERENCES Andriotis, Annamaria (2016). Banks Have a New Phrase for Risky Customers: ‘Near Prime’, Wall Street

Journal Blogs. Retrieved from https://blogs.wsj.com/moneybeat/2016/07/26/banks-have-a-new-phrase-for-risky-

customers-near-prime/ (Current August 15, 2019).

CFPB consumer laws (2013). Equal Credit Opportunity Act (ECOA). Retrieved from

https://files.consumerfinance.gov/f/201306_cfpb_laws-and-regulations_ecoa-combined-june-2013.pdf current

August 15, 2019.

Dornhelm, Ethan (2018). Average U.S. FICO Score Hits New High. FICO/Blog. Retrieved from

https://www.fico.com/blogs/average-u-s-fico-score-hits-new-high (Current August 15, 2019).

Girouard (2019). Examining the Use of Alternative Data in Underwriting and Credit Scoring to Expand

Credit Access. Testimony of Dave Girouard, CEO and Co-Founder, upstart Nework, Inc. Before the Taskforce on

Fintech, United States House Committee on Financial Services, Retrieved from:

https://financialservices.house.gov/uploadedfiles/hhrg-116-ba00-wstate-girouardd-20190725.pdf (Current

December 19, 2019).

Hayashi, Yuka (2019). Where You Went to College May Matter on Your Loan Application. The Wall

Street Journal. Retrieved from: https://www.wsj.com/articles/where-you-went-to-college-may-matter-on-your-loan-

application-11565258402 (current August 15, 2019).

Malik, Sanjay (2019). Alternative Data: The Great Equalizer To Lending Inequalities? Forbes. Retrieved

from: https://www.forbes.com/sites/forbestechcouncil/2019/08/14/alternative-data-the-great-equalizer-to-lending-

inequalities/#4d8b55db2449 (Current August 15, 2019).

National Center for Educational Statistics (2017). Digest of Educational Statistics. Retrieved from

https://nces.ed.gov/programs/digest/d17/tables/dt17_104.10.asp (Current August 15, 2019)

Noto, Grace (2017). CFPB Reassures Fintechs with ‘No-Action’ Letter for Upstart Network. Bank

Innovation.net. Retrieved from https://bankinnovation.net/allposts/operations/comp-reg/cfpb-reassures-fintechs-

with-upstart-network-no-action-letter/ (Current August 15, 2019).

Saxena, N. A., Huang, K., DeFilippis, E., Radanovic, G., Parkes, D. C., and Liu, Y. (2019). How Do

Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness. In

Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 99-106.

U.S. House Committee on Financial Services (2019). Waters Announces Committee Task Forces on

Financial Technology and Artificial Intelligence. Retrieved from

https://financialservices.house.gov/news/documentsingle.aspx?DocumentID=403738 current August 15, 2019.

United States Census Bureau (2019). Quick Facts United States. Retrieved from

https://www.census.gov/quickfacts/fact/table/US/PST045218 (Current August 15, 2019).

Upstart (2017). Request for No-action letter, Consumer Financial Protection Bureau. Retrieved from:

https://files.consumerfinance.gov/f/documents/201709_cfpb_upstart-no-action-letter-request.pdf (Current December

19, 2019).

Waynard, Peter (2018). Equifax Comment Letter to Policy on No-Action Letters and BCFP Product

Sandbox. Regulations.gov. Retrieved from: https://www.regulations.gov/document?D=CFPB-2018-0042-0021

current, August 15, 2019.