Artificial intelligence (AI) is no longer a peripheral or emerging technology, it is a central infrastructure that increasingly governs the contours of our digital, social, and physical realities. From algorithmic hiring tools and adaptive educational platforms to facial recognition and predictive healthcare diagnostics, AI systems are shaping how opportunity, agency, and identity are distributed. These systems mediate access to vital resources, determine eligibility for services, and influence how individuals are perceived by institutions and one another. In doing so, AI technologies are not merely tools, they are arbiters of social meaning, institutional power, and material consequence.
Yet, the socio-technical systems that underpin AI are far from neutral. They are shaped by historical patterns of exclusion, normative assumptions about human behavior, and asymmetries in power and representation. Without critical intervention, AI can reinforce and amplify structural inequalities at scale, automating marginalization under the guise of objectivity. Researchers have shown that algorithmic bias often stems from biased training data, narrow design assumptions, and opaque governance processes[2][3][15]. Left unchecked, these flaws are not anomalies but systemic outcomes, reflecting who gets to design, whose knowledge is centered, and which values are encoded into the system[14][6].
This paper critically examines how AI systems can be designed to support radical inclusivity through a paradigm of ethical and inclusive design. It positions ethical inclusion not as a set of compliance tasks or abstract values, but as a foundational design principle that must guide the entire AI development lifecycle. We explore the ethical integration of human-centered design (HCD), intersectional analysis, and responsible innovation to operationalize this vision.
At the core of this approach is a shift toward anticipatory and participatory design ethics, where diverse users, especially those from historically marginalized communities are engaged not as passive end-users but as co-creators of technological futures. This demands more than interface-level inclusivity, it calls for structural redesign in how we conceptualize risk, value, and success. Inclusive design must be situated within broader sociotechnical systems and institutional contexts, accounting for how power, identity, and knowledge production intersect in the development of AI[7][10].
This paper presents a case building on insights from existing scholarly frameworks, prior research, mixed-methods research, and case studies from sectors such as healthcare, education, and financial services. It also incorporates emerging best practices from industry leaders and global organizations working to embed responsible and inclusive design into AI development at scale[4][1][16]. These examples demonstrate that ethical and inclusive AI design is not only possible but essential for building trustworthy, equitable, and socially responsive systems.
By weaving together theory and practice, this paper offers a human-centered framework for designing AI that anticipates harm, respects difference, and fosters just outcomes. In doing so, it contributes to a growing body of work that seeks to reimagine technological development through the lens of ethics, inclusion, and collective flourishing.
Mainstream conversations around accessibility in digital design are dominated by legal compliance standards such as the Americans with Disabilities Act (ADA), Section 508, and the Web Content Accessibility Guidelines (WCAG). While these standards are vital for establishing minimum thresholds for access, they are limited in scope and reactive by nature. These frameworks often conceptualize accessibility as a binary state, a system either complies or it does not thereby obscuring the dynamic, contextual, and intersectional realities of human experience.
Accessibility is often seen as visual fixes applied to the finished product rather than an inclusive principle embedded in how the solution is constructed or designed. Notably, inclusive design experts distinguish accessibility compliance from deeper inclusive practice. Accessibility tends to focus on minimum standards, the “baseline axis” of ensuring people with disabilities can access a product with larger font sizes or screen readers. Inclusion, by contrast, “is concerned about the entire journey and experience” of diverse users[6]. As one design leader explains, inclusion means designing with people in mind throughout the process, not just tweaking a finished product to be accessible[6]. This holistic mindset aligns with calls for ethical by design approaches in AI. Instead of treating fairness, accountability, transparency, and other ethical criteria as afterthoughts or hurdles to clear at the end, designers are urged to bake these considerations into the creative process from the start.
AI introduces further complexity. Machine learning systems evolve over time, influenced by training data, user feedback, and environmental shifts. Static checklists cannot account for the emergent behaviors of AI systems, especially when deployed at scale. Research by Buolamwini and Gebru (2018) on facial recognition has shown how seemingly objective systems disproportionately misclassify darker-skinned women, illustrating the compounded risks of algorithmic bias and non-inclusive design[2]. Similarly, Koenecke et al. (2020) identified significant racial disparities in speech recognition systems, with error rates disproportionately high for African American Vernacular English (AAVE) speakers[3]. These failures aren’t just technical glitches, they reflect deeper patterns of exclusion, where the experiences and voices of marginalized people are overlooked or misunderstood by the systems shaping everyday life.
Researchers Stilgoe et al. (2013) describe anticipation as a cornerstone of responsible innovation, urging innovators to envision “possible future impacts, risks, and unintended consequences” of new technologies[7]. In practice, this could mean conducting scenario analysis and bias brainstorming early in a project, involving ethicists or diverse users in design sprints, and using foresight techniques to imagine how an AI system might fail or be misused. By anticipating issues up front, teams can course-correct designs before harm occurs.
Industry frameworks increasingly reflect this shift. For instance, responsible AI approaches emphasize integrating risk mitigation into the design and prototyping phases. Data scientists and engineers are encouraged to build system guardrails as early as possible in AI development to minimize risks while maximizing value[8]. Rather than waiting for an AI model to exhibit biased behavior in the wild, teams impose constraints on model outputs, conduct bias audits on training data, and iteratively test for disparate impacts during development. Likewise, Harvard Business Review analysts observe that AI developers are now prioritizing human values alongside performance metrics like speed and efficiency “used to be the priority” in AI, but now safety, privacy, and fairness matter too[9]. A striking example was OpenAI’s rollout of GPT-4 in 2023, which was marketed not just as more powerful but as “more aligned” with human values[9], signaling an industry recognition that ethical alignment is itself a competitive feature. In short, the field is evolving from a narrow compliance outlook toward ethically anticipatory design, wherein UX innovation entails foreseeing societal impacts and striving to “do the right thing,” by design.
These expanded views on accessibility and ethical anticipation mark a critical paradigm shift in AI system design, moving from reactive fixes toward proactive, inclusive, and responsible innovation that centers diverse users and their lived realities at every stage.
Designing for ethical and inclusive AI refers to the intentional incorporation of human worth, autonomy, and identity into the fabric of AI systems. Inclusion here is not a vague moral concept but a design heuristic that foregrounds the ethical implications of exclusion and the need for inclusive infrastructure.
Human-centered design (HCD) provides a foundational methodology but must be expanded to include trauma-informed practices, neurodiverse cognition, and disability justice perspectives. Participatory design methodologies can amplify voices traditionally excluded from technological decision-making. For instance, in a design sprint for an AI-powered mental health companion, neurodiverse participants emphasized the importance of non-verbal interaction, low-stimulation interfaces, and mood-adjustable input options. These insights informed the development of customizable UI elements and emotionally neutral dialogue models.
Several foundational ethical frameworks offer guidance here. One influential example is the AI4People framework proposed by Floridi and colleagues (2018), which distilled a synthesis of high-profile AI ethics guidelines into five core principles: beneficence (promote well-being), non-maleficence (avoid harm), autonomy (respect human agency), justice (ensure fairness), and explicability (enable transparency and understanding)[10]. These principles closely mirror long-established bioethical tenets and were explicitly adapted to the AI context to foster a “Good AI Society” (Floridi et al., 2018). Designing with such principles in mind provides a moral foundation for UX innovation. For example, beneficence and non-maleficence translate to actively using AI to improve lives (say, AI that enhances accessibility for people with disabilities or aids under-served communities) while rigorously preventing and mitigating harms (such as unfair discrimination or privacy violations). The principle of autonomy implies that AI systems should augment rather than override human decision-making, preserving user agency and inclusiveness by, for instance, requiring meaningful human control in high-stakes decisions. Justice demands that designers strive for equitable outcomes, ensuring that benefits and risks of AI are distributed fairly and that the system works well for all demographic groups (not just the majority). Explicability underscores that users have a right to understand how AI is reaching its conclusions, which is key to respecting their inclusiveness and enabling contestation of decisions. In sum, these principles act as design objectives that safeguard human inclusiveness, they push designers to consider how an AI system can enhance human capabilities and social justice, rather than treating people as mere data points or means to an end.
The framework on Responsible AI also echoes this direction by highlighting the importance of inclusive governance. It stresses that AI development should incorporate ethical foresight, cross-functional teams, and lived-experience expertise, not only as validators but as strategic co-creators. Designing for ethical and inclusive AI therefore requires confronting normative assumptions about ability, communication, and cognition. It calls for working with users throughout the process—not just as end-users, but as partners who bring valuable knowledge.
Another aspect of inclusiveness-centered design is acknowledging users not just as consumers but as rights-bearing individuals. International organizations have begun to stress this in AI governance. The Food and Agriculture Organization (FAO) of the United Nations, for instance, has called for ethical considerations to be “front and center” in digital innovation so that AI “does not perpetuate or exacerbate existing inequalities” but instead closes equity gaps[11]. In the context of agrifood systems, this means designing AI tools that empower smallholder farmers and marginalized rural communities, respecting their local knowledge and agency, rather than disenfranchising them. More broadly, global initiatives like the Rome Call for AI Ethics (2020) (endorsed by the Vatican and organizations including FAO) explicitly list human inclusiveness as a core principle, alongside transparency, fairness, and accountability. This reflects a consensus that respect for human inclusiveness must be a north star in AI development.
Finally, a culture shift in industry underscores these values. As Abernethy et al. (2024) observed, even tech companies now publicly frame their AI advances in terms of alignment with human values[9]. Design teams are incorporating ethical risk assessments and user privacy design workshops as standard practice. The message is clear, “good design is ethical design”. By rooting design choices in principles of inclusiveness, innovators can deliver AI experiences that users find trustworthy, inclusive, and worthy of their confidence thereby marrying technological innovation with ethical integrity.
Ethical design must move beyond harm mitigation toward anticipatory responsibility. Stilgoe, Owen, and Macnaghten (2013) advocate for a framework of responsible innovation that includes anticipation, reflexivity, inclusion, and responsiveness. Anticipatory design ethics builds on this model by incorporating speculative foresight and harm scenario planning into the design process.
Rather than reacting to problems after deployment, anticipatory ethics asks: “What might go wrong, and how can we prevent or mitigate it by design?” This mindset is a pillar of the broader concept of responsible innovation. Stilgoe, Owen, and Macnaghten (2013) articulate responsible innovation through four dimensions— anticipation, reflexivity, inclusion, and responsiveness[7]. Here, anticipation entails systematic thinking about possible futures (both positive and negative) and the ripple effects of new technology. In a design context, this could involve scenario planning, threat modeling, or red-teaming exercises to surface ethical risks. For example, if designing a generative AI for content creation, the team might anticipate misuse cases (such as generating disinformation or hate speech) and build in safeguards like content filters, usage policies, and detection mechanisms from the outset. Anticipation thus reduces the likelihood of “ethical blindspots” and surprises after launch.
Equally important is reflexivity, encouraging designers and organizations to reflect on their own values, assumptions, and intentions. Teams should ask themselves tough questions: Why are we building this AI? Who might be harmed or excluded? Are we framing the problem correctly, or are we inadvertently baking in a biased worldview? Importantly, anticipation and reflexivity are not meant to happen in an echo chamber but through inclusive, cross-disciplinary collaboration.
Our fieldwork with an edtech platform using AI to recommend personalized learning pathways revealed unanticipated harms. Students with ADHD or dyslexia were disproportionately categorized as “underperforming,” triggering interventions that limited curriculum access rather than enhancing support. Through participatory redesign workshops involving students, caregivers, and educators, reorienting the algorithm to focus on learning variability and growth rather than predictive labeling. This shift aligned the platform more closely with inclusive pedagogical values.
Concretely, organizations can implement anticipatory ethics by embedding ethics checkpoints and tools throughout the product development lifecycle. One approach involves a staged ethics process starting with raising internal awareness, developing concrete policies, documenting AI use cases, followed by systematic ethical review (sometimes including external audits), and finally taking action, approving, revising, or halting AI systems based on findings[1].
Another tactic is scenario analysis and simulations during design. Teams can imagine “day-in-the-life” stories of users interacting with the AI, especially edge cases, for instance, a credit scoring AI encountering an applicant with an unconventional background and analyze whether the system treats them fairly. Some organizations convene red teams (in-house or external experts) to attack their models or find problematic outputs (e.g., uncovering racial or gender biases in image recognition) before release. This anticipatory testing can be augmented with automated tools that combine human-driven “creative” risk identification with automated testing at scale[8]. Such tools reflect the emerging consensus that continuous testing and evaluation of AI for ethics, just as rigorously as for performance, should be an ongoing part of development[4]. Generative AI systems, in particular, present novel challenges (nondeterministic outputs, vast open-ended capabilities) that demand iterative probing and refinement to ensure safety and alignment[4].
Finally, being anticipatory also means being prepared to respond and iterate as real-world feedback comes in. Responsible innovators recognize that no amount of upfront foresight can predict everything; what matters is having mechanisms to learn and adapt. For example, a company might establish an AI ethics committee that continually monitors deployed models and can rapidly intervene (e.g., by pulling an AI system offline or adjusting its parameters) if evidence of harm arises. This echoes the responsiveness element of Stilgoe et al.’s framework[7], a commitment to ongoing responsibility, not a one-time effort. In summary, anticipatory design ethics is both a mindset and a set of practices. It urges design and AI teams to “imagine the unintended” and build ethical resilience into innovation. By doing so, organizations can significantly reduce ethical risks while also engendering greater trust and acceptance among users and society at large.
Importantly, anticipatory ethics requires cultural support. It can’t be a one-off checklist handled by a lone “ethics officer” at the end. Leaders in AI ethics emphasize making it a habit for teams to think about ethical implications from day one. Davenport (2025) argues that AI ethics cannot be “just a set of policies or a follow-up project”, it must be embedded into the process early and made part of everyday work for AI developers and designers[12]. This involves training teams to identify ethical issues in initial design meetings, pairing data scientists with social scientists or legal experts, and incentivizing employees to flag concerns without fear. Some organizations are even creating ethical AI playbooks and simulation games to encourage foresight and moral imagination among staff.
Intersectionality, introduced by Kimberlé Crenshaw (1989), challenges the tendency of design to essentialize user needs. People do not experience discrimination or marginalization along single axes, rather, they inhabit complex positionalities shaped by race, gender, class, disability, and more. AI systems that do not recognize this complexity risk encoding forms of compound exclusion.
An illustrative case is a workplace automation tool designed to evaluate employee productivity using keyboard activity and screen time. For caregivers, chronically ill employees, or neurodivergent workers, these metrics failed to capture actual labor contributions, reinforcing ableist and gendered biases. By integrating intersectional personas and task analysis methods into the design process, the development team redesigned the tool to allow alternative productivity inputs and contextual performance tracking.
The failure to consider intersectionality in AI is not merely an oversight; it is a design flaw with profound ethical and social implications. BCG’s Responsible AI guidance reiterates that organizations must measure outcomes not only in terms of technical performance but in terms of distributive fairness— who benefits, who is excluded, and under what assumptions[16][17]. This demands sociotechnical sensitivity, recognizing that technology is never context-free, and design must be attuned to political, cultural, and institutional entanglements.
There is also a growing focus on disability inclusion in AI, which is inherently intersectional since disability intersects with all other identities. A recent analysis by the Center for Democracy & Technology warned that the lack of high-quality disability data in AI systems “poses a significant risk of perpetuating and exacerbating existing barriers for people with disabilities”[13]. If AI developers do not include people with various disabilities in their datasets and testing, the resulting tools may, for instance, fail to understand slurred speech, improperly caption images, or make unjustified inferences about a person’s capabilities. Andy Baldwin (2024) argues that AI solutions must be designed with inclusivity at the core, implementing bias mitigation to ensure fairness for disabled communities[14]. This could entail steps like creating synthetic data to represent low-frequency disability scenarios or customizing models for assistive technologies. It also means avoiding ableism in design, for example, not assuming all users can see a screen or hear audio, and providing multimodal interfaces by default. Designing for disability often yields innovations that benefit everyone, a phenomenon known as the “curb-cut effect” in inclusive design[15], where a feature originally meant for a small group ends up improving overall UX.
To explore how ethical and inclusive design principles are applied in real-world settings, we interviewed individuals from 12 different technology teams across healthcare, education, and finance. We gathered feedback about engaging in design sprints featuring workshops, ethics planning, intersectional persona development, and algorithm impact reviews. In addition to running a workshop with a team and gathering their experience in a survey.
Key Findings:
Discussion:
Direct engagement with diverse lived experiences helped teams uncover blind spots and challenge assumptions about their “typical” users. Tools like ethics canvases supported navigating complex trade-offs between fairness and performance. However, organizational inertia and limited leadership support often hindered the long-term adoption of these ethical practices.
Overall, the study underscores that ethical and inclusive design is not just theoretical but can actively reshape team mindsets, reveal hidden risks, and better align AI innovation with human values, provided there is strong institutional backing.
Drawing from the insights and patterns surfaced across case studies, research, and fieldwork, we propose a framework for ethical and inclusive AI design composed of six interconnected pillars. Each pillar addresses a critical design and governance challenge, providing scaffolding for turning ethical commitments into actionable practices, as illustrated in Figure 1.
Engage marginalized groups not merely as informants, but as co-designers and decision-makers throughout the AI development lifecycle. As discussed in Section 2, participatory approaches are essential for surfacing lived experience, mitigating bias, and challenging default design assumptions that often exclude those at the margins. By sharing power in the design process through methods such as co-design workshops or community advisory boards, teams can create systems that reflect pluralistic values and diverse needs. This is particularly important for populations historically harmed by algorithmic systems, including disabled, racialized, and low-income communities.
AI design must move from reactive harm mitigation to anticipatory ethics, as elaborated in Section 4. This includes scenario mapping, threat modeling, and red-teaming exercises to imagine potential negative outcomes before deployment. For example, envisioning how an AI model could reinforce gender stereotypes or misclassify users from underrepresented groups enables teams to design safeguards, such as content filters or fairness audits into the system from the beginning. Ethical foresight is not just about preventing harm, it’s about cultivating moral imagination to anticipate consequences beyond technical performance metrics.
Rather than assuming uniform use environments or user capabilities, ethical AI design must account for variation in cognitive, sensory, cultural, and environmental contexts. As shown in Sections 3 and 5, many systems fail by ignoring the diversity of real-world use. This pillar encourages designers to embrace variability through flexible interfaces, multimodal interaction options, and systems that can adapt to users’ lived realities, whether that means offline functionality for users in low-connectivity areas or voice alternatives for users who cannot engage visually. This principle echoes the “curb-cut effect” in inclusive design: designing for edge cases often improves usability for all[15].
Rooted in the work of Kimberlé Crenshaw (1989), intersectionality demands that we understand how systems of oppression such as racism, ableism, sexism, and classism interact in compounded ways. As described in Section 5, designing with intersectional intelligence means integrating layered social identities into user research, testing, and validation. Tools such as intersectional personas and layered use-case scenarios help surface how AI systems might inadvertently marginalize users who sit at complex intersections of identity. This isn’t just about running fairness audits, it’s about embedding inclusive structures into the way we think about and practice design.
Ethical design is an ongoing commitment, not a one-time compliance task. As explored in Section 4 and in the findings from our mixed-methods study (Section 6), post-launch accountability mechanisms are critical. This includes systems for collecting real-world user feedback, transparent model documentation, and responsive processes for mitigation or redress when harm is identified. Importantly, reflexivity requires organizations to continuously interrogate their values, motivations, and blind spots such as why are we building this, and for whom? Teams should be empowered to pause, revise path forward if ethical concerns emerge, an action that requires institutional support and ethical leadership.
Perhaps the most overlooked, yet vital, pillar is aligning organizational structures with ethical intent. As emphasized by BCG (2024) and Davenport (2025)[16][17], responsible AI must be operationalized through decision rights, performance indicators, governance mechanisms, and resource allocation. This includes training teams in ethics, embedding ethical checkpoints throughout the product lifecycle, and incentivizing ethical behavior, not just innovation or speed. Without this alignment, even the most well-intentioned designers will struggle to make lasting change. Structural alignment ensures that ethics is treated as a capability, not a constraint.
This framework builds upon and extends foundational work in AI ethics such as Floridi et al. (2018) on the principles of beneficence, non-maleficence, autonomy, justice, and explicability; the AI Now Institute’s advocacy for algorithmic accountability (Crawford & Paglen, 2019); and critical insights from disability justice and feminist HCI. Its distinctive contribution lies in linking systemic governance structures with hands-on design methodologies. It offers a bridge between the abstract aspirations of ethical AI and the daily realities of design and technical teams, empowering them to act with intention, inclusivity, and foresight.
The design of AI systems is no longer a technical exercise, it is a profoundly ethical endeavor. As these systems mediate labor, health, education, and civic life, the stakes are too high for inclusion to be an afterthought or a box-checking exercise. This paper has argued that ethical and inclusive design must become a foundational design principle. One that informs not only the interfaces we build, but the values we encode, the questions we ask, and the harms we anticipate.
Designing for ethical and inclusive AI invites us to imagine AI not merely as a set of predictive tools, but as sociotechnical infrastructures with the power to either replicate or reconfigure historical inequities. It calls for a shift from reactive ethics to anticipatory responsibility; from static compliance to dynamic, participatory co-creation; and from narrow optimization to intersectional justice. The framework proposed here links design practice with structural accountability, offering a scalable pathway for organizations committed to equitable innovation.
This is not just a technical or procedural shift, it is a cultural one. As Ruha Benjamin reminds us, ethical imagination is a radical act, a refusal to accept systems of oppression as inevitable, and a commitment to designing futures that center care, justice, and collective thriving. Realizing this vision requires more than new tools, it demands new dispositions like humility, reflexivity, and the courage to ask whose voices are missing, whose needs are sidelined, and what futures we are complicit in shaping.
To conclude, this is a call to action for design leaders, AI practitioners, policymakers, and institutions to embrace ethical and inclusive AI not as an abstraction, but as a practice, messy, iterative, and deeply human. Only by doing so can we design systems worthy of the societies we aspire to build.
At Global Business & Economics Journal, we contribute to the future of business and technology with news, innovative research, expert insights, and thought-provoking analysis.
