1. Introduction

1.1 The Problem of Separability

Modern intervention systems are built upon a foundational promise: that primary objectives can be pursued while collateral effects remain secondary, accidental, or correctable. Operational language across domains preserves a distinction between target and collateral domains, between intended outcomes and unintended spillovers.

Yet in many contexts this distinction proves structurally unstable.

Effects classified as “collateral” do not merely accompany action; they emerge from the same structural logic that makes the action possible. They are not external disturbances of an otherwise precise intervention, but co-generated outcomes. The persistence of such effects across diverse domains suggests a structural pattern rather than a set of contingent failures (Merton 1936; Perrow 1984).

1.2 Definition

Structural collateralism designates a structural property of complex intervention systems whereby non-primary effects are co-generated by the same architectural logic that produces primary outcomes, and cannot be eliminated without altering the system’s underlying configuration.

A system exhibits structural collateralism when:

  1. Non-primary effects arise from the same structural mechanisms as primary objectives.
  2. These effects occur under normal operating conditions rather than exceptional breakdowns.
  3. Their elimination would require modifying the structural logic of the system, not merely correcting contingent errors.

1.3 Diagnostic Criterion

Structural collateralism does not imply deliberate intent, negligence, or moral failure. It identifies a structural configuration in which collateral effects are intrinsic to the mode of operation.

Operationally, the claim that an effect is structural is counterfactual. Holding the system’s capacity to achieve its primary objective within an acceptable tolerance, local mitigations should fail to remove the non-primary effect to a material degree; a significant reduction should require changes to the system’s architecture (e.g., doctrine, mechanism of action, optimization objective, or governance constraints) (Woodward 2003).

1.4 The Structural Pattern

Across heterogeneous domains, a recurring configuration can be identified: (a) a complex, interconnected system; (b) an intervention optimized toward a defined primary objective; (c) a linguistic or formal distinction between “intended” and “collateral”; (d) recurrent non-primary effects structurally inseparable from the intervention logic; (e) persistent moral or epistemic tension resulting from that inseparability.

Structural collateralism describes the internal tension between operational precision and structural co-production. Complex systems characterized by tight coupling and interactive complexity tend to generate such configurations as a normal property of their architecture (Perrow 1984), and not as a symptom of exceptional breakdown. Science and technology studies uses the idiom of co-production to describe how knowledge and social order are constituted together (Jasanoff 2004). The same idiom is useful here: primary and non-primary effects arise from the same generative architecture. A related point appears in Luhmann’s account of functional differentiation: the operational closure of subsystems generates effects on their environment as a structural property of differentiation (1995).

2. Diagnostic Framework

2.1 Conditions for Structural Collateralism

The conditions introduced in section 1.2 as definitional can be read operationally as a diagnostic instrument. A system exhibits structural collateralism when…A system exhibits structural collateralism when three conditions jointly obtain. First, non-primary effects are co-generated by the same mechanisms that produce primary outcomes: they are not external disturbances but intrinsic outputs of the same generative logic. Second, these effects arise under normal operating conditions, not as symptoms of exceptional breakdown or implementation failure. Third, their material reduction would require modifying the system’s underlying configuration, not merely correcting individual outputs or adjusting parameters within the existing architecture.

These three conditions define the diagnostic space. Meadows’s (2008) vocabulary of stocks, flows, and leverage points provides a useful grammar for locating where structural change would be required: interventions at the level of parameters or rules operate on the surface of the system; interventions capable of altering structural collateralism must reach the level of goals, information flows, or the rules of the system itself. Perrow (1984) makes a complementary point: in tightly coupled systems, local interventions tend to displace rather than eliminate recurrent effects, because the generative architecture remains intact. The distinction between these levels of intervention is the operational core of the counterfactual criterion introduced in section 1.3, and grounds the separation from trade-offs and externalities developed in the next section.

2.2 Distinguishing from Trade-offs and Externalities

Structural collateralism departs from both trade-offs and externalities along a specific axis: whether non-primary effects are represented inside the system’s objective function, i.e., whether they count as something the system is explicitly optimizing over.

In a genuine trade-off, competing effects are not only acknowledged as outcomes; they are explicitly priced into the decision. The agent evaluates them ex ante and performs a deliberate exchange. Chemotherapy is a clean illustration: therapeutic benefit and cellular toxicity are co-generated by the same cytotoxic mechanism, yet both enter the treatment choice, because the clinician balances expected harm against expected benefit within a shared evaluative frame. The trade-off is structural (one mechanism, multiple effects) and deliberate (those effects are jointly represented in the objective function).

In structural collateralism, non-primary effects are likewise co-generated by the same mechanisms that produce the primary outcome, but they remain external to what the system optimizes. The architecture is tuned to maximize a primary target (e.g., a military objective, engagement, a therapeutic endpoint) while treating the collateral effect as “real” in production but irrelevant in optimization. In principle, the design could be altered to internalize the collateral dimension (through constrained objectives, multi-objective formulations, or explicit weights; see Meadows (2008) on leverage points); in practice, it systematically is not. What is optimized and what is produced thus diverge, not episodically, but by design.

An externality, by contrast, denotes effects that fall outside the agent’s cost-benefit calculus and typically land on parties who are not part of the decision (Coase 1960). Structural collateralism can certainly generate externalities, but the two notions are not interchangeable: a collateral effect can be foreseeable, recurrent, and tightly coupled to the system’s normal operation while still being excluded from the objective function that governs optimization.

The operational diagnostic remains straightforward. If non-primary effects can be materially reduced through local mitigations (parameter tuning, safeguards, procedural corrections) without degrading primary performance, the case is better captured as a trade-off (explicit or implicit) or as an implementation failure. If meaningful reduction requires rewriting what the system is optimizing for (i.e., modifying the objective function, the coupling architecture, or the governance constraints that bind them), the case exhibits structural collateralism.

3. Comparative Illustrations

3.1 Industrial Warfare

In high-altitude saturation bombing, operational doctrine defines discrete targets. The moral and legal framework governing targeting rests on the principle of discrimination and the doctrine of double effect, which formally separates intended from foreseen but unintended harm (Walzer 1977; Nagel 1972). In practice, geometric contiguity, visibility degradation, formation behavior, and technological constraints render urban spillover structurally predictable.

Civilian harm becomes recurrent not because it is necessarily intended, but because its elimination would require abandoning or fundamentally redesigning the operational doctrine itself.

The language of “collateral damage” remains formally operative, while operational boundaries between target and collateral space become porous. Butler (2009) suggests that framing practices shape which losses are publicly recognized and which remain backgrounded; in this sense, framing affects how recurrent non-primary harm is described and acknowledged.

3.2 Pharmacological Systems

In pharmacology, therapeutic and adverse effects frequently arise from the same molecular pathway. Biological interconnectivity limits absolute selectivity. Network pharmacology has demonstrated that drugs act not on isolated targets but on molecular networks, making off-target effects an intrinsic consequence of systemic connectivity rather than a failure of design (Hopkins 2008).

Side effects are not external intrusions into an otherwise isolated therapeutic mechanism. They are co-produced by the same structural interaction between molecule and organism that generates clinical benefit.

Eliminating such effects often requires redesigning the molecule, altering dosage regimes, or rethinking the therapeutic paradigm itself.

3.3 Generative Algorithmic Systems

In large language models, the primary objective is the maximization of the conditional probability of the next token: a system optimized not to track the state of the world, but to reproduce what is statistically frequent, locally coherent, and contextually expected (Bender et al. 2021). This optimization logic is not an external “source of bias” acting upon an otherwise neutral engine; it is the generative mechanism that co-produces both operational usefulness and systematic epistemic distortion. The two outcomes are therefore not separable, because they arise from the same architectural configuration (Weidinger et al. 2021).

The alignment layer introduced through human feedback does not remove this tension; it re-specifies the reward landscape in ways that tend to displace it (Ouyang et al. 2022). Formulations that minimize conflict, attenuate uncertainty, and conform to dominant framings are more readily reinforced; formulations that foreground disagreement, expose unresolved trade-offs, or question consolidated arrangements are more likely to be treated as unhelpful, confusing, or unsafe. What is optimized (plausibility aligned with expectations) and what is produced (a systematic skew toward overrepresented actors, narratives, and sources) thus diverge by design rather than by error (Gebru et al. 2021).

In this domain, structural collateralism is further stabilized by feedback loops that entangle production and reception: model outputs influence what gets read, cited, and published; these downstream selections re-enter corpora and evaluation regimes, amplifying the initial asymmetries. A material reduction of such effects would therefore require architectural intervention at multiple leverage points: (a) data composition and filtering; (b) learning objectives and loss functions; (c) alignment procedures and reward models; (d) retrieval and ranking architectures; and (e) the coupling between system outputs, institutional uptake, and business incentives. Correcting individual responses addresses the surface manifestation rather than the generative configuration. Floridi and Cowls (2019) propose five principles for AI in society (including non-maleficence and explicability); structural collateralism raises a specific question for such frameworks: whether these constraints can be satisfied without altering the underlying optimization logic. In this sense, structural collateralism assumes an epistemic form: the non-primary effect is not damage to bodies or infrastructures, but a recurrent distortion of what becomes visible, credible, and sayable within the mediated public record.In large language models, optimization toward plausibility, fluency, and expectation alignment generates systematic epistemic distortions.

4. Rationality and Intervention

4.1 Precision and Modern Technical Rationality

Structural collateralism is consistent with accounts of modern technical rationality in which operational claims of precision exceed the separability achievable under structural interdependence. Beck (1992) argues that modern industrial societies generate risks endogenously, as manufactured by-products of the same rationality that generates prosperity, rather than encountering them as external hazards.

Operational discourse often emphasizes targeting and control, whereas system architectures operate through networks, constraints, and systemic interconnections. Winner (1980) argues that technical artifacts embed structural configurations that generate political and social effects independently of the intentions of designers or operators. Ellul (1964) describes the expansion of technical logic as a defining feature of modern civilization, with effects that may exceed the purposes for which particular techniques are deployed. Scott (1998) extends this diagnosis to state-sponsored high-modernist interventions: schemes designed to improve the human condition systematically produce non-primary effects because they simplify complex systems in ways that destroy the very features that made those systems functional.

The gap between linguistic separability and structural inseparability is associated with persistent moral and epistemic tensions.

4.2 Analytical Status of the Concept

Structural collateralism is analytically descriptive rather than normatively accusatory.

It does not attribute intent. It does not presuppose moral wrongdoing. It identifies a property of systemic organization.

However, once recognized, structural collateralism carries normative implications: if collateral effects are intrinsic to structure, responsibility cannot be reduced to isolated errors or deviations. Young’s (2011) social connection model offers a relevant framework here: responsibility is not backward-looking and blame-attributing, but forward-looking and structurally distributed among those who participate in the systems that generate harm.

4.3 Toward a Comparative Framework

Structural collateralism is not domain-specific. It appears wherever: (a) interventions operate within highly interconnected systems; (b) selectivity is structurally constrained; (c) optimization logics shape system behavior; (d) primary objectives coexist with recurrent non-primary effects.

The concept provides a framework for rethinking design, responsibility, and governance in modern technological societies. It extends and formalizes a pattern that sociological theory has approached through adjacent concepts, unintended consequences (Merton 1936), normal accidents (Perrow 1984), manufactured risk (Beck 1992), while identifying the structural co-production of primary and non-primary effects as a distinct and analyzable property.

5. Implications

5.1 Responsibility for Configurations

If collateral effects are structurally co-generated, responsibility cannot be treated as a residual category, activated only when something “goes wrong.” The familiar moral grammar of deviation (fault, negligence, error) is often inadequate because the relevant mechanism is not episodic but architectural: the system produces its primary outputs through the very same generative pathways that recurrently produce its non-primary effects. The concept therefore invites a shift from responsibility-for-acts to responsibility-for-configurations, i.e., from ex post attribution to ex ante design and governance duties.

This does not collapse into a claim of ubiquitous guilt. Rather, it supports a more distributed and forward-looking account, close in spirit to Young’s social connection model: participation in a structural process that predictably generates harm grounds obligations to modify that process, even when no single agent can be identified as the proximate cause of a specific instance (2011). In this sense, structural collateralism clarifies why “lack of intent” may be morally relevant yet institutionally insufficient: intent can be absent, and nevertheless the architecture can remain a stable generator of foreseeable harm.

A practical implication is that responsibility becomes stratified across roles and temporal horizons: (a) epistemic responsibility (mapping the generative architecture, stating what is known and what is merely assumed, and maintaining institutional memory of recurrent effects); (b) design responsibility (altering objectives, coupling, constraints, and feedback channels that co-produce non-primary effects); (c) governance responsibility (instituting oversight, incentives, and enforceable constraints capable of acting on the architecture rather than merely on surface outputs). What structural collateralism resists, almost by definition, is the comforting reduction of responsibility to local corrections: “training,” “discipline,” or “better execution,” when the recurrent effect is generated under normal operation.

5.2 Physical vs Epistemic Collateralism

Structural collateralism can manifest as physical harm, epistemic harm, or (more often) as an entanglement of the two. Physical collateralism concerns recurrent damage to bodies, infrastructures, and environments, co-generated by the operational architecture of an intervention. Epistemic collateralism concerns recurrent distortions in what a system makes visible, credible, and actionable, co-generated by the same optimization logic that makes the system operationally effective.

The distinction matters because epistemic collateralism can be both an effect and a mediator. As an effect, it includes systematic misclassification, predictable misrecognition, and the stabilization of narratives that privilege the system’s target language over lived experience (or, in algorithmic contexts, plausibility over truth-conditions) (Fricker 2007). As a mediator, epistemic collateralism can translate into physical harm by shaping decisions, legitimations, and resource allocations; conversely, physical collateralism can retroactively reshape epistemic landscapes by producing enduring interpretive conflicts, contested frames, and institutionalized ambiguity.

This is why the war case and the AI case are not merely analogical but structurally adjacent: both involve a formal separation between “target” and “collateral” that remains linguistically stable while becoming operationally porous; both involve a recurrent mismatch between what is claimed (precision, discrimination, explicability) and what the architecture can reliably deliver under real conditions. The difference is not in the presence of collateralism but in the medium through which it propagates: kinetic effects in one domain; epistemic effects (sometimes with downstream physical consequences) in the other.

5.3 Governance Implications

If a collateral effect is structural, then governance must be structural as well. The main governance error, in such cases, is to treat recurrent non-primary effects as an “add-on” problem, solvable by peripheral mitigation while leaving the generative architecture intact. Structural collateralism implies that governance must operate at the level of objectives, coupling, and feedback, i.e., at the level of what Meadows (2008) would call leverage points in a system.

Three governance orientations follow. First, design for constrained objectives: where the optimization target is too narrow, non-primary effects will often be the predictable by-product of success; governance therefore has to shape what “success” means (e.g., constraints, penalties, multi-objective criteria, hard limits on permissible envelopes of harm). Second, reduce tight coupling and interactive complexity where feasible: systems that operate through tightly coupled chains and opaque interactions tend to generate collateral effects as a normal property of operation (Perrow 1984); governance should therefore privilege modularity, inspection, and controllable interfaces over end-to-end opacity. Third, institutionalize accountability for recurrent effects: not as moral theatre, but as a mechanism for forcing architectural change when surface-level fixes merely relocate the problem (Ostrom 1990).

In practical terms, governance becomes a disciplined sequence: specifying the primary objective; mapping recurrent co-generated effects and their pathways; declaring an acceptable tolerance envelope (if any) and the conditions under which the system’s deployment becomes illegitimate; and, crucially, identifying which architectural modifications are necessary for a material reduction of non-primary effects without losing the primary function. Where such modifications are not feasible, governance must be able to articulate the hard conclusion: that the system’s current mode of operation is incompatible with its declared normative constraints (non-maleficence, discrimination, explicability), not by accident but by structure (Floridi and Cowls 2019).

6. Two Extensions

6.1 Collateralism laundering

The concept of structural collateralism is descriptive: it identifies configurations in which non-primary effects are co-generated by the same structural logic that produces primary outcomes. A second, analytically distinct phenomenon concerns not the production of collateral effects, but their discursive and procedural stabilization. I refer to this as collateralism laundering.

By collateralism laundering I mean the instrumental use of “collateral” language, metrics, and compliance routines to reclassify structurally co-generated harms as marginal, accidental, or already accounted for, thereby rendering them administratively acceptable and politically sustainable. Laundering does not require denial of harm; it operates through a triad of ordinary moves: framing (how harm is described; (Goffman 1974)); counting (how harm is rendered legible and commensurable (Espeland and Stevens 1998)); and closing (how justification is procedurally terminated (Luhmann 1969)).

Collateralism laundering is typically legible when the following features co-occur: (a) foreseeability is acknowledged while responsibility is diffused, so that harm is treated as structurally predictable yet accountability is dispersed across chains of delegation, procedures, and organizational boundaries (a “no single point of responsibility” configuration) (Bovens 1998); (b) procedural substitution, whereby compliance with a protocol is offered as a surrogate for justification (the formula “we followed the rules” functions as closure rather than as a premise for revision) (Luhmann 1969); (c) metric shielding, in which harm is translated into indicators or thresholds that are systematically insensitive to lived impact, distributional asymmetry, and cumulative effects (Espeland and Stevens 1998); (d) asymmetric countability, so that some losses are made legible and commensurable while others are backgrounded or excluded by definitional choices (what counts as harm, whose harm counts) (Espeland and Stevens 1998; Scott 1998); and (e) post-hoc stabilization, whereby explanations are produced primarily to stabilize legitimacy after the fact, rather than to revise the architecture that generates the collateral effects (Suchman 1995).

This extension is compatible with, but distinct from, structural collateralism. Structural collateralism concerns the architecture of co-generation; laundering concerns the institutional management of that co-generation, i.e., the practices through which recurring collateral effects are rendered narratively, legally, and administratively “normal.” The analytic point is not that institutions are uniquely cynical, but that recurring harms require recurring mechanisms of normalization; where co-generated harm is structurally persistent, normalization becomes (almost inevitably) structurally recurrent (Vaughan 1996).

6.2 Laundering empirical instantiations

Three domains illustrate, in different idioms, the mechanics of collateralism laundering.

Warfare doctrine. In contemporary air operations, “acceptable civilian casualty” thresholds operate as metric shielding: civilian harm is translated into numerical estimates (e.g., collateral damage estimates, CDEs) that make losses commensurable, calculable, and therefore administratively governable (Chamayou 2015). The threshold then functions as a closure device: once the estimate falls below the stipulated bound, justification is procedurally terminated, even when the generative architecture of harm (e.g., signature strikes anchored to pattern-of-life inference) remains unchanged. Here foreseeability is not denied but routinized: civilian harm is incorporated into planning as an expected accompaniment of “normal” operation; responsibility is simultaneously dispersed along the kill chain (operators, targeteers, commanders, legal advisers), producing a recognizable “no single point of responsibility” configuration. The language of proportionality and military necessity, finally, performs a stabilizing reframing: recurrent harm is rendered legible as legal compliance rather than as structurally co-generated output, thereby sustaining practices that would otherwise call for architectural revision.

Pharmaceutical regulation. Adverse-event reporting systems (e.g., FDA’s FAERS) exemplify metric shielding and asymmetric countability in tandem: side effects are counted as discrete events and encoded as reportable units, often abstracted from cumulative burden, quality-of-life degradation, and distributional asymmetries in who bears the harm (with women, older adults, and marginalized groups frequently underrepresented in trials and therefore weakly represented in evidentiary baselines) (Epstein 2007). The reporting protocol also enables procedural closure: once adverse events are logged, classified, and judged “within acceptable limits” relative to therapeutic benefit, contestation is displaced into routine channels rather than translated into architectural revision (molecular mechanism, dosing regime, approval pathway). Recurrent harm is normalized as the “price of innovation,” and what is structurally co-generated (therapeutic effect and toxicity arising from the same pathway) is discursively stabilized as a rational trade-off, even when the populations most affected had limited voice in defining the very thresholds of “acceptability.”

Algorithmic systems. In large-scale content moderation and recommendation infrastructures, “fairness metrics” can become a form of asymmetric countability: bias is rendered visible primarily in aggregate statistical form (e.g., demographic parity, equalized odds), while lived experiences of epistemic harm, reputational damage, exclusion, and downstream vulnerability remain weakly captured (or entirely outside the measurement frame) (Noble 2018; Buolamwini and Gebru 2018). In laundering terms, the critical move is not the existence of metrics per se, but their institutional use as procedural closure: compliance with benchmarked indicators is presented as a substitute for justification (or redesign), while the optimization logic that systematically produces differential exposure and harm (engagement maximization, virality amplification, profit-driven ranking) remains structurally intact. Post-hoc explanations (“our models meet industry fairness standards”) stabilize legitimacy ex post without addressing the co-generation mechanism (Selbst et al. 2019); when harm recurs, it is attributed to “edge cases,” “adversarial inputs,” or “user behavior,” rather than to the coupling between business incentives and epistemic collateralism.

6.3 Systemic collateralism

Structural collateralism is formulated at the level of a single intervention system: non-primary effects are co-generated by the same architectural logic that produces primary outcomes, and resist elimination under local mitigation. In many contemporary settings, however, the relevant unit is not a single system but a coupled ecology of systems. This motivates a second extension: systemic collateralism.

By systemic collateralism I mean an emergent configuration in which multiple systems, each governed by its own optimization logic, mutually reinforce the production and persistence of non-primary effects through coupling, feedback loops, and distributed accountability. In such cases, collateral effects are not merely inseparable from one system’s architecture; they become stabilized at the level of the ecosystem, so that even well-intentioned modifications of individual components fail to materially alter the overall harm profile.

Systemic collateralism is characterized by a small set of mechanisms that tend to appear in combination: (a) feedback loops and datafied stabilization, whereby outputs from one system become inputs for another (or for the same system at a later time), transforming collateral effects into “evidence,” baselines, or priors for subsequent operations (Meadows 2008); (b) cross-domain coupling, through which technical, economic, legal, and cultural subsystems interact so that local mitigation in one layer is offset or neutralized by pressures in another (Perrow 1984; Luhmann 1995); (c) normalization through routine, as what begins as an exceptional cost becomes a stable expectation embedded in operational planning, budgeting, risk management, and user-facing narratives (Vaughan 1996); and (d) structural responsibility gaps, whereby accountability is distributed across actors and modules in a way that makes collateral effects predictable yet difficult to attribute, contest, or repair at any single point (Bovens 1998; Young 2011).

This extension clarifies a practical implication: in system-of-systems environments, remedies that address only one component (a model patch, a doctrine refinement, a procedural safeguard) may reduce visible harm locally while leaving the broader collateral configuration intact. The persistence of collateral effects under partial reforms is therefore not necessarily a sign of bad faith or poor implementation; it can be a structural property of coupled systems, especially where coupling generates path dependence (David 1985) and where incentives are misaligned across layers.

Systemic collateralism thus generalizes the core idea of structural collateralism from intra-system co-generation to inter-system stabilization. It provides a vocabulary for describing how collateral effects persist and reproduce in modern infrastructures whose defining feature is not merely complexity within systems, but tight coupling between them (Perrow 1984).

7. Limits, Objections, and Research Agenda

A first objection is inflationary: if every intervention produces side effects, then structural collateralism risks becoming a synonym for “the world is complex.” The answer is methodological rather than rhetorical. The concept is not meant to cover any non-primary effect, but a specific class: recurrent effects that (a) appear under normal operating conditions; (b) are predictably co-generated by the same mechanisms that produce primary outcomes; and (c) resist local mitigation such that material reduction requires architectural change. Without these conditions, the more familiar notions of trade-off, externality, or implementation failure remain sufficient.

A second objection concerns overlap with established frameworks (unintended consequences, normal accidents, manufactured risk). Structural collateralism does not compete with them; it reframes a subset of cases by foregrounding co-generation rather than surprise, and architecture rather than episodic breakdown. It also clarifies a recurring rhetorical asymmetry: operational discourse tends to preserve separability (target versus collateral), while structural analysis shows inseparability (primary and non-primary effects as outputs of a single generative configuration).

A third objection is operational: how can one empirically diagnose that an effect is structural, rather than merely persistent? The counterfactual criterion proposed earlier is demanding, but it is precisely the point. Structural claims should be hard to make. Where plausible, diagnostic work can compare mitigation regimes: if successive local interventions (parameter tuning, procedural safeguards, training) fail to produce material reduction while preserving primary performance, and if only architectural redesign changes the harm profile, the structural hypothesis gains support (Woodward 2003). In domains where experimentation is constrained (e.g., warfare, governance of large infrastructures), the evidence will often be indirect; in such cases, the concept functions as a disciplined interpretive claim rather than a measurement claim.

Finally, there is the risk of fatalism: if collateral effects are structural, does that mean they are inevitable? No. The word “structural” should not be read as metaphysical necessity, but as a description of what must be changed for a different outcome to be possible. Structural collateralism therefore operates as a sobriety test for modern rationality: it asks whether the promised separability between primary aims and collateral costs is compatible with the architectures we actually deploy; and if not, whether we are willing to redesign objectives, constraints, and institutions rather than merely refining the rhetoric of precision.

References

Beck, Ulrich. 1992. Risk Society: Towards a New Modernity. Sage.
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–23. https://doi.org/10.1145/3442188.3445922.
Bovens, Mark. 1998. The Quest for Responsibility: Accountability and Citizenship in Complex Organisations. Cambridge University Press.
Buolamwini, Joy, and Timnit Gebru. 2018. “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.” Proceedings of Machine Learning Research 81: 1–15.
Butler, Judith. 2009. Frames of War: When Is Life Grievable? Verso.
Chamayou, Grégoire. 2015. A Theory of the Drone. The New Press.
Coase, Ronald H. 1960. “The Problem of Social Cost.” Journal of Law and Economics 3: 1–44.
David, Paul A. 1985. “Clio and the Economics of QWERTY.” American Economic Review 75 (2): 332–37.
Ellul, Jacques. 1964. The Technological Society. Knopf.
Epstein, Steven. 2007. Inclusion: The Politics of Difference in Medical Research. University of Chicago Press.
Espeland, Wendy Nelson, and Mitchell L. Stevens. 1998. “Commensuration as a Social Process.” Annual Review of Sociology 24: 313–43. https://doi.org/10.1146/annurev.soc.24.1.313.
Floridi, Luciano, and Josh Cowls. 2019. “A Unified Framework of Five Principles for AI in Society.” Harvard Data Science Review 1 (1). https://doi.org/10.1162/99608f92.8cd550d1.
Fricker, Miranda. 2007. Epistemic Injustice: Power and the Ethics of Knowing. Oxford University Press.
Gebru, Timnit, Jamie Morgenstern, Briana Vecchione, et al. 2021. “Datasheets for Datasets.” Communications of the ACM 64 (12): 86–92. https://doi.org/10.1145/3458723.
Goffman, Erving. 1974. Frame Analysis: An Essay on the Organization of Experience. Harvard University Press.
Hopkins, Andrew L. 2008. “Network Pharmacology: The Next Paradigm in Drug Discovery.” Nature Chemical Biology 4 (11): 682–90. https://doi.org/10.1038/nchembio.118.
Jasanoff, Sheila, ed. 2004. States of Knowledge: The Co-Production of Science and the Social Order. Routledge.
Luhmann, Niklas. 1969. Legitimation Durch Verfahren. Luchterhand.
Luhmann, Niklas. 1995. Social Systems. Stanford University Press.
Meadows, Donella H. 2008. Thinking in Systems: A Primer. Edited by Diana Wright. Chelsea Green Publishing.
Merton, Robert K. 1936. “The Unanticipated Consequences of Purposive Social Action.” American Sociological Review 1 (6): 894–904. https://doi.org/10.2307/2084615.
Nagel, Thomas. 1972. “War and Massacre.” Philosophy & Public Affairs 1 (3): 123–44.
Noble, Safiya Umoja. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. NYU Press.
Ostrom, Elinor. 1990. Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge University Press.
Ouyang, Long, Jeffrey Wu, Xu Jiang, et al. 2022. “Training Language Models to Follow Instructions with Human Feedback.” Advances in Neural Information Processing Systems 35: 27730–44.
Perrow, Charles. 1984. Normal Accidents: Living with High-Risk Technologies. Basic Books.
Scott, James C. 1998. Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. Yale University Press.
Selbst, Andrew D., Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. “Fairness and Abstraction in Sociotechnical Systems.” Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68. https://doi.org/10.1145/3287560.3287598.
Suchman, Mark C. 1995. “Managing Legitimacy: Strategic and Institutional Approaches.” Academy of Management Review 20 (3): 571–610. https://doi.org/10.2307/258788.
Vaughan, Diane. 1996. The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA. University of Chicago Press.
Walzer, Michael. 1977. Just and Unjust Wars: A Moral Argument with Historical Illustrations. Basic Books.
Weidinger, Laura, John Mellor, Maribeth Rauh, Conor Griffin, and Jonathan Uesato. 2021. Ethical and Social Risks of Harm from Language Models. DeepMind.
Winner, Langdon. 1980. “Do Artifacts Have Politics?” Daedalus 109 (1): 121–36.
Woodward, James. 2003. Making Things Happen: A Theory of Causal Explanation. Oxford University Press.
Young, Iris Marion. 2011. Responsibility for Justice. Oxford University Press.