Case Study 1: The Algorithm and the Election

The Platform in the Center of the Storm

In 2016, Facebook had approximately 1.8 billion monthly active users. More than 40% of American adults reported getting news from the platform. Its News Feed algorithm — a proprietary system that determined which posts, articles, and updates each user saw — was among the most consequential editorial systems in the history of media. And unlike the editorial systems of newspapers and broadcasters, it had no editor accountable for its decisions. It had objectives and parameters, adjusted by engineers and product managers in response to engagement metrics.

The 2016 U.S. presidential election brought a cascade of revelations about how the platform had been used, and how the platform's own choices had shaped the information environment in which the election occurred.

What We Know: The Facts

The engagement optimization problem: Facebook's News Feed algorithm prioritized content that generated engagement — likes, shares, comments, reactions. The internal research, portions of which have since become public through whistleblower Frances Haugen and through investigative journalism, showed that the content most likely to generate engagement was emotionally activating content: content that provoked outrage, fear, moral indignation, or tribalism. The algorithm was not designed to maximize truth or democratic deliberation; it was designed to maximize time on platform, and it turned out that emotionally inflammatory content was more effective at doing so than accurate, nuanced content.

The misinformation problem: During the 2016 election, the platform was extensively used to spread false information: fabricated news stories about candidates, manipulated videos, and coordinated disinformation campaigns. A BuzzFeed News analysis found that in the final three months of the 2016 campaign, the top twenty fake election news stories generated more total engagement on Facebook than the top twenty election stories from major news outlets combined. Facebook's internal research, revealed by Haugen, showed that company employees had identified the problem and that remediation efforts were repeatedly scaled back out of concern they would reduce engagement.

The Cambridge Analytica incident: A political data firm called Cambridge Analytica obtained data on approximately 87 million Facebook users through a quiz application that harvested not just the quiz-takers' data but the data of all their friends. Cambridge Analytica used this data to build detailed psychological profiles of voters, which it then used on behalf of political campaigns. Facebook's platform architecture had, for years, permitted third-party applications to harvest friends' data in this way; the company was aware the practice existed and had not prohibited it.

The internal research on harmful effects: Haugen's disclosures included internal research showing that Facebook's own researchers had documented significant harmful effects of the platform on teenage girls' mental health, correlating Instagram use with increases in eating disorders, depression, and suicidal ideation. The company's public statements contradicted its internal findings.

The actions not taken: Throughout this period, Facebook had significant technical capability to reduce the spread of misinformation and to limit algorithmic amplification of inflammatory content. Internally, proposed changes were frequently rejected or scaled back. The "integrity" team's proposals were evaluated partly by their effect on engagement metrics, and changes that reduced engagement were systematically deprioritized.

Applying the Frameworks

Consequentialist Analysis

The consequentialist question is: what were the aggregate effects of Facebook's decisions during this period, and were those decisions defensible by reference to consequences?

On the benefit side, Facebook provides real value to billions of users: connection with friends and family, community formation, information sharing, organizing capacity for civic and charitable activities. These are genuine goods, and a fair consequentialist accounting must include them.

On the harm side: the scale of the harms identified in the public record is significant. The algorithmic amplification of inflammatory content contributed to a degraded information environment in which accurate information and deliberate misinformation competed on terms the misinformation often won. The Cambridge Analytica incident involved the unauthorized use of tens of millions of people's personal data for political manipulation — a harm to democratic processes that is difficult to quantify but not difficult to identify.

The most important consequentialist point is about what Facebook knew and when. The internal research Haugen disclosed showed that company employees had identified the harmful effects of the algorithm's engagement optimization. The question "were these decisions defensible by reference to consequences?" cannot be answered without acknowledging that the company had evidence about consequences and chose not to change course.

There is a harder consequentialist question here about aggregation: is the harm to democratic deliberation, teenage mental health, and the 87 million people whose data was misappropriated outweighed by the benefits Facebook provides to its user base? This question resists clean calculation, but the consequentialist framework at minimum demands that we ask it honestly, with the full accounting — including the harms — visible, rather than looking only at the benefits.

Kantian Analysis

The Kantian analysis of Facebook's situation is, in important respects, more damning than the consequentialist one.

Facebook's core business model involves treating users' attention, desires, and psychological vulnerabilities as resources to be extracted and monetized. The platform is designed not to help users achieve their own ends — connection, information, entertainment — but to maximize the time users spend on the platform, because time on platform generates advertising revenue. Users are, in the language of the digital economy, the product rather than the customer.

This is not a trivial concern recast in philosophical language. It is a fundamental structural fact about the relationship between the platform and the people who use it. The platform deploys sophisticated behavioral science — variable reward schedules, social comparison mechanisms, notifications designed to trigger anxiety — in service of its own revenue goals. From a Kantian perspective, this is treating users as means rather than ends: using their psychological vulnerabilities as instruments for the platform's objectives.

The Cambridge Analytica case is especially clear in Kantian terms. The people whose data was harvested did not consent to having their psychological profiles built and sold to political campaigns. Their personal information — their innermost beliefs, fears, and desires — was treated as raw material for others' purposes. This is as direct a violation of the requirement to treat persons as ends in themselves as one is likely to find in the contemporary data economy.

The internal research question has a Kantian dimension as well. If you know that your product is harming young people's mental health, and you choose not to act on that knowledge because acting would reduce engagement, you are treating those young people as instruments — as a resource whose suffering is acceptable so long as it does not disrupt the revenue model.

Virtue Ethics Analysis

What does Facebook's behavior during this period reveal about the institution's character?

The virtue ethics lens is illuminating here because it shifts attention from the analysis of individual decisions to the question of what kind of organization makes these patterns of decisions over time.

An organization of genuine integrity, confronted with internal research showing that its product was harming teenagers and degrading democratic discourse, would face a moment of reckoning. The virtuous response would require courage — the courage to accept reduced engagement metrics in exchange for reduced harm — and justice — the commitment to acting fairly toward all affected parties, not just shareholders.

What happened, according to available evidence, was something different: proposals to reduce harmful algorithmic amplification were repeatedly evaluated in terms of their effect on engagement, and changes that reduced engagement were deprioritized. This is not a single bad decision; it is a pattern that reveals institutional character. It reveals an organization that, when forced to choose between its own revenue objectives and the wellbeing of the people it serves, consistently chose revenue.

This is not a claim that the people at Facebook are individually bad. Many of the employees who identified these problems and proposed solutions were acting with integrity. The virtue ethics analysis points to institutional structure: organizations, like individuals, can have characters — patterns of response to difficult situations — and the character Facebook displayed during this period was one of systematic prioritization of engagement over welfare.

Care Ethics Analysis

Care ethics asks: who is actually being cared for here, and whose needs are being centered?

The answer that emerges from the available evidence is troubling. The needs of advertisers — for reach, for engagement, for demographic targeting — were carefully attended to. The needs of large shareholders — for growing revenue and user base — were systematically prioritized. The needs of the 87 million people whose data was misappropriated, of the teenage girls whose mental health was being damaged, of the American voters whose information environment was being polluted: these were attended to when and to the degree that doing so did not cost engagement.

Care ethics also asks about responsibility in a relational sense. Facebook occupied a position of extraordinary trust: billions of people shared their private thoughts, relationships, and moments on the platform. That trust creates obligations. An institution that exploits the data people share with it for purposes those people did not understand or consent to — even if technically permitted under a terms-of-service agreement buried in legal fine print — is violating the trust relationship that gave it the data in the first place.

Justice Analysis

The justice framework raises questions about distribution and procedural fairness that the other frameworks tend to address less directly.

Rawls's veil of ignorance: if you did not know whether you would be a Facebook user, a Facebook shareholder, an advertiser, a politician whose election was influenced by the information environment Facebook created, or a parent of a teenage daughter whose mental health was affected by Instagram — what rules would you choose for how the platform operates?

From behind the veil, you would almost certainly demand more transparency about how the algorithm works, more user control over the content you see, stronger protections against data misappropriation, and meaningful accountability when internal research identifies harms. You would probably not choose a system in which the platform's decisions about what 1.8 billion people see each day are made entirely in accordance with the platform's private revenue objectives, with no external accountability.

The justice analysis also raises the question of who was harmed most. The people most vulnerable to misinformation, most affected by the algorithmic amplification of inflammatory content, and most likely to lack the media literacy to distinguish reliable from unreliable information were not uniformly distributed across the population. The distributional effects of Facebook's algorithmic choices fell differently on different groups — a pattern that any justice analysis must take seriously.

The Harder Questions

The Regulator's Dilemma

One response to this case is: if Facebook was behaving badly, the answer is regulation. But regulation of technology platforms is genuinely difficult. The speed of technological change tends to outrun regulatory response. Platforms operate globally but regulations are jurisdictional. The technical complexity of algorithmic systems makes it hard to write rules that are both specific enough to be enforceable and general enough to remain relevant as technology changes.

None of this means regulation is impossible or inappropriate. It means the question of what kind of regulation is both achievable and effective is itself a serious policy and philosophical question.

The User's Responsibility

Is the user entirely a victim in this story? Most users consented to Facebook's terms of service — or at least clicked "I agree." Most users continued to use the platform despite growing awareness of its problems.

The Kantian analysis is that consent under conditions of manipulation and information asymmetry is not fully valid consent — you cannot meaningfully agree to something you don't understand. But there is a real question about user responsibility here. At what point does a user's continued use of a platform they know is problematic become a form of complicity?

The Moral Limits of "Move Fast and Break Things"

Facebook's unofficial motto for many years was "move fast and break things." In product development, this meant: ship features quickly, accept some failure, iterate. The ethos served Facebook's growth. But applied to a platform that affects democratic processes, mental health, and the information environment of billions of people, "move fast and break things" is an ethical catastrophe waiting to happen — and arguably did happen.

The virtue ethics question is: is "move fast and break things" a compatible with institutional virtue? Or is it, structurally, the kind of ethos that makes the Facebook story predictable in retrospect?

Discussion Questions

1. Frances Haugen, the Facebook whistleblower, is a former employee who chose to copy and share internal documents in violation of her employment agreement. Apply the framework from Chapter 8 (when is it ethical to break the rules?) to her decision. Was she justified? What considerations should have governed her choice?

2. Facebook's CEO Mark Zuckerberg testified before Congress that Facebook was "a platform, not a publisher" — arguing that it should not be held responsible for the content it hosts, the way a publisher would be. Evaluate this distinction. Does it hold philosophically? What follows from it if it does, and what follows from it if it doesn't?

3. Suppose you are a mid-level engineer at Facebook in 2015, working on the News Feed algorithm. You are aware of internal research showing that the algorithm amplifies inflammatory content. Your proposals for changes have been rejected twice. What do you do? Apply at least two frameworks.

4. The case for Facebook having some accountability for the 2016 information environment is strong. But Facebook was not the only actor: there were also foreign governments engaged in active disinformation campaigns, media organizations that amplified false stories, and individual users who chose to spread misinformation. How do you assign responsibility across this ecosystem of actors?