Important Issues in Responsible Data Science

AU Winter Institute in Data Science

Richard Ressler

2024-01-09

Introduction

Privacy, security, and ethics all fall within the realm of Responsible Data Science.

We will discuss Responsible Data Science through four topics:

I. Legal, Professional, and Ethical Considerations
II. Ethical Frameworks
III. Identifying Ethical Issues
IV. What Can You Do? What Should You Do?

The goal is to help you practice responsible data science from legal, professional, and ethical perspectives.

I. Legal, Professional, and Ethical Considerations

Data Science Follows a Life Cycle

A Data Science Life Cycle
Responsible Data Science Affects the Whole Life Cycle

The DS Life Cycle is about answering a question.

Responsible Data Science asks questions across the life cycle.

Data Scientists Make Choices

What kinds of projects do I work on and What questions do I analyze?
Who are the Stakeholders?
Towards what goals do I optimize my models? Accuracy, Fairness, Equity, Equality, …?
How do I get my data? How do I protect my data?
Is my data representative of the population?
What do I do about “bad” or missing data or “outliers” that “mess up” my results?
What variables/features/attributes do I use?
How much effort do I put into checking my results? Are they repeatable?
How do I leverage/credit other people’s work?
How do I report my results - what is intellectual property and what should be public?
…
Choices can involve Legal, Professional, or Ethical Considerations

Choices have consequences, with benefits and risks.

Legal Considerations

Legal considerations address the criminal/civil risks for violating laws.

Laws (statutes) specify permissible and/or impermissible activities as well as potential punishments (judicial, e.g., imprisonment).
Policies (regulations) provide guidance on implementing activities (within legal constraints) along with adjudication procedures and potential punishments (non-judicial, e.g., debarment).
Legal issues in big data include how you gather, protect, and share data, and increasingly, how you use it.

Laws are “local” not universal: Pirates or Privateers

Laws do not keep pace with technology (generative AI anyone?) and can be difficult to interpret.

When in doubt, ask a lawyer competent in the issue.

Laws Proscribe Some Choices

Health Insurance Portability and Accountability Act (HIPAA) (US DHHS 2015). See Big Data Analytics under HIPAA (Coy and Hoffman 2016)
Family Educational Rights and Privacy Act (FERPA) (US DOEd 2017)
Fair Credit Reporting Act (FTC 2013b). See FTC Credit Reporting Information (FTC 2018).
Fair Housing Act (DHUD 2021)
Equal Employment Opportunity Act (EEOC 2021b)
Children’s Online Privacy Protect Act (COPPA) and Rule (FTC 2013a)
Genetic Information Nondiscrimination Act (GINA) (EEOC 2021a)
European General Data Protection Regulation (GDPR) (Wikipedia 2022)
- See What is GDPR (Burgess 2020)
California Consumer Protection Act (Edelman 2020)
And many more … with the expectation of more to come …
- Congressional interest in Section 230 of the Communications Decency Act of 1996.
- In 2022 alone, 17 states and DC introduced bills or resolutions concerning AI and several states adopted legislation including Vermont, Colorado, Illinois, and Washington. (Loeb et al. 2023)
- In December 2023, the European Union reached agreement on the Artificial Intelligence Act. (Union 2023)

Ignorantia juris non excusat

Professional Considerations

Professional considerations address the risk of activities related to organizations with which you affiliate.
- Professional Organizations
- Employer or Volunteer Organizations

Organizational bylaws or policies identify and manage risk to the institution.
These may include professional “Codes of Conduct” or “Codes of Ethics”.
Organizational behavior and ethics may conflict with individual ethics
- Choices can include trying to change the organization or offending individuals; or leaving or being forced to leave the organization, with potential legal issues as well.

When in doubt, ask a mentor or manager you trust.

Ethical Considerations

Ethical considerations arise when asking
- What should I do?
- What is the “right” or “moral” thing to do?

Ethical Choices can be hard, especially when choices may require violating a law, regulation, and/or professional guideline.
Individual principles and cultural norms shape options and guide choices in complex situations.
Often, there is no universally-accepted or even a good “right answer”.
May have to choose between two bad outcomes.
- The Trolley Problem variations. (“Trolley Problem” 2024)

May have to choose between individual and group outcomes.
Ethical choices can lead to feelings of guilt, group reprobation, civil action (torts), or criminal charges.

Many, many, frameworks attempt to guide Ethical Choices.

II. Ethical Frameworks

Three (of many) Ethical Frameworks ¹

Consequentialist or Utilitarianism: greatest balance of good over harm (groups/individual).
- Choose the future outcomes that produce the most good.
- Compromise is expected as the end justifies the means.
Duty: Do your Duty, Respect Rights, Be Fair, Follow Divine Guidance:
- Do what is “right” regardless of the consequences or emotions.
- Everyone has the same duties and ethical obligations at all times.
Virtues: Live a Virtuous Life by developing the proper character traits.
- Ethical behavior is whatever a virtuous person would do.
- Tends to reinforce local cultural norms as the standard of ethical behavior.

Frameworks can conflict with each other or are “wrong” in the extreme.

No single or simple right answer!

Using Frameworks in Ethical Decisions

Recognize There May Be an Ethical Issue.
- Assess the underlying definitions, facts, and assumptions and constraints
Consider the Parties Involved, the stakeholders.
- What individuals or groups might be harmed or benefit, and by when.
Gather all Relevant Information. Are you missing key facts? Are they knowable?
Formulate Actions and Evaluate under Alternative Frameworks.
- What will produce the most good and do the least harm? (Utilitarian)
- What respects the rights of everyone affected by the decision? (Rights)
- What treats people equally or proportionately? (Justice)
- What serves the entire community, not just some members? (Common Good)
- What leads me to act as the sort of person I want to be? (Virtues)
Examine Alternatives and Make a Decision.
Act and Observe.
Assess and Reflect on the Outcomes.

If you don’t like the outcomes, restart the process.

III. Identifying Ethical Issues

Our Biased Brains Helped Us Survive

Our brains have evolved mechanisms to make quick decisions.

These are the source of “Unconscious Biases” or “Implicit Biases”.
- Unconscious stereotypes, prejudices, or attitudes that affect our decision-making, perception, and social behavior (“Implicit Bias Vs Unconscious Bias: Types Ways to Prevent Them” 2022).
- Friend or Threat, Familiar or Strange, Us versus Them
- Learn more and test yourself at Project Implicit (Implicit 2022).
Under stress, we tend to bypass the higher-level cognitive centers that evolved later and take more time to reason.
Humans get comfortable with patterns which can lead to systematic deviations from making rational judgments.
- 12 Major Types of Cognitive Bias Explained (Psychology 2017)
- Many potential biases could affect our decisions (Desjardins 2017)
Ethical Challenges can arise from our own implicit biases or the implicit biases of others affecting our data, thoughts, and actions.

Short-term choices under stress may be bad in the long run ….

Given Bias in Data and Algorithms are DS Systems Ethical?

Not a new issue - goes back decades. However, the explosive growth of AI systems to support and even make decisions is generating concerns.

Bias in the (training) data (historical, sampling, …) can drive biased outcomes.
Are algorithms really less biased than people? It depends …
- How do you measure fairness? See Tutorial: 21 Definitions of Fairness …
- How can you tell with “black box” models? See Machine learning transparency and … A Toolkit for Transparency in Dataset Documentation
- Algorithms find “hidden” relationships among proxy variables that can distort the results and interpretations.
- What are the trade offs among accuracy, explainability, and transparency?
  - Increasing “fairness” will generally decrease accuracy.
Active area for research and publication.
- MIT AI Blindspot Cards (MIT-Media-Lab 2019)
- Understand, Manage, and Prevent Algorithmic Bias (Baer 2019)
- University of Chicago’s Algorithmic Bias Playbook (UChicago n.d.)

Three Articles for Consideration

Discussion Questions
- Is there an ethical issue or more than one? What is it?
- Who is affected and who is responsible?
- Pick on of the Professional Codes of Conduct or Guidelines. How would it apply?
- What would you do differently or recommend?

More Examples of Ethical Issues

Contradictions and competition among legal, professional, and ethical guidelines.
Using biased data (even unknowingly)
Eliminating extreme values or combining small groups.
Using Proxies for “Protected” Attributes (even unknowingly).
Protection of Intellectual Property versus Explainability, Transparency and Accountability
Law of Unintended Consequences - people will use your products and solutions in “creative” ways that
- May not align with your principles, or,
- Be technically inappropriate.

So what can you do? What should you do?

IV. Practicing Responsible Data Science

What Can You Do? What Should You Do?

Consider Ethical Choices Across the DS Life Cycle

Responsible Data Science Life Cycle: Ask a question to Observe Outcomes

Ask a question: Equity or equality? Stakeholders and Trade offs? What are our interests? Recency or Confirmation Bias?

Frame the Analysis: What is the population? Role of proxy variables? How are metrics for “fairness” affecting groups/individuals? Do we need an IRB (APA 2022)?

Get Data: How was it collected? Was informed consent required/given? Is there balanced representation? Selection Bias? Availability Bias? Survivorship Bias?
Shape Data: Are we aggregating distinct groups? How do we treat missing data? Are we separating training and testing data?
Model and Analyze: How are we documenting assumptions, treating extreme values, or checking over-fitting? Are we checking multiple fairness and performance metrics?
Communicate Results: Are the graphs misleading? Did we cherry pick or data snoop? Are we reporting \(p\)-values and hyper-parameters?
Deploy/Implement: Is the deployment accessible to all?
Observe Outcomes: Can we check assumptions and analyze outcomes for bias?

Are we following professional guidelines from ASA, ACM, INFORMS, …?

Consider Other Frameworks for Responsible Data Science

UK Government
US DoD
Google
IBM
The 5 C’s Framework

Published by the Royal Statistical Society and the Institute and Faculty of Actuaries in A Guide for Ethical Data Science

Start with clear user need and public benefit
Be aware of relevant legislation and codes of practice
Use data that is proportionate to the user need
Understand the limitations of the data
Ensure robust practices and work within your skill set
Make your work transparent and be accountable
Embed data use responsibly

(RSS-IFA 2021)

Department of Defense AI Capabilities shall be:

Responsible … exercise appropriate levels of judgment and care, while remaining responsible for the development, deployment, and use….
Equitable … take deliberate steps to minimize unintended bias in AI capabilities.
Traceable … develop and deploy AI capabilities such that relevant personnel have an appropriate understanding …, including with transparent and auditable methodologies, data sources, and design procedures and documentation.
Reliable … AI capabilities will have explicit, well-defined uses, and the safety, security, and effectiveness … will be subject to testing and assurance within those defined uses ….
Governable … design and engineer AI capabilities to fulfill their intended functions while possessing the ability to detect and avoid unintended consequences, and the ability to disengage or deactivate deployed systems that demonstrate unintended behavior.

(DoD 2020)

AI Principles

Be socially beneficial.
Avoid creating or reinforcing unfair bias.
Be built and tested for safety.
Be accountable to people.
Incorporate privacy design principles.
Uphold high standards of scientific excellence.
Be made available for uses that accord with these principles.

We will not design or deploy AI in the following application areas: Weapons, Surveillance, …

(GoogleAI 2022)

IBM Principles for Trust and Transparency

The purpose of AI is to augment human intelligence. The purpose of AI and cognitive systems developed and applied by IBM is to augment – not replace – human intelligence.
Data and insights belong to their creator.

IBM clients’ data is their data, and their insights are their insights. Client data and the insights produced on IBM’s cloud or from IBM’s AI are owned by IBM’s clients. We believe that government data policies should be fair and equitable and prioritize openness.

New technology, including AI systems, must be transparent and explainable.

For the public to trust AI, it must be transparent. Technology companies must be clear about who trains their AI systems, what data was used in that training and, most importantly, what went into their algorithm’s recommendations. If we are to use AI to help make important decisions, it must be explainable.

(IBM 2019)

A guide to Building “Trustworthy” Data Products

Based on the golden rule: Treat others’ data as you would have them treat your data

Consent - Get permission from the owners or subjects of the data before …
Clarity - Ensure permission is based on a clear understanding of the extent of your intended usage
Consistency - Build trust by ensuring third parties adhere to your standards/agreements
Control (and Transparency) - Respond to data subject requests for access/modification/deletion, e.g., the right to be forgotten
Consequences (and Harm) - Consider how your usage may affect others in society and potential unintended applications.

(Loukides, Mason, and Patil 2018)

To Be an Ethically Responsible Data Scientist …

Integrate ethical decision making into your environment.

Recognize legal, professional, and ethical issues surround your work.
- See Weapons of Math Destruction for multiple scenarios. (O’Neil 2017)
Ask Questions of Peers, Managers, Mentors, and Experts to get guidance.
Document your sources and approach: assumptions, data, code, and references.
- Use Literate Programming. e.g., with Quarto. (Posit 2022)
Protect data: especially HIPPA or Personal Identifiable Info (PII). (Us Dept of Labor 2024)
- Consider Differential Privacy (Devaux 2022) à la the Census Bureau. (Bureau 2023)
Create reproducible analysis and solicit appropriate reviews and feedback.
- Avoid “Spurious Correlations”. (Vigen 2021), produce results that Reveal, Don’t Conceal. (Weissgerber Tracey L. 2019) and don’t create Examples of Misleading Statistics.(Calzon 2021)
Follow Publishing and Conference Ethical Guidelines e.g., AutoML’s. (AutoML 2022)
Consider your own biases. (Acton 2022)

As Davy Crockett might say, “Try to be sure you are right, then Go Ahead!”

“Davy Crockett” (2024)

Stay Current on Emerging Ideas

Stanford ’s Human-Centered AI Index for 2022 Report (HCAI 2022)
Review articles on Big Data/Data Science/AI Ethics, e.g.,
- Google’s Project Nightingale (Schneble, Elger, and Shaw 2020)
- Aspects of Data Ethics in a Changing World: Where Are We Now? (Hand 2018)
Review the Inventory of Awesome AI Guidelines (IEAI-ML 2021)
Investigate Academic Centers, e.g.,
- Georgetown Center for Digital Ethics (Fleisher 2024)
- Markkula Center at SCU Intro Course in DS Ethics (SCU 2022)
Consider popular press articles
Analyze Case Studies, e.g., Princeton University’s Case Studies (University 2018)

Try to stay on the fast moving train that is Responsible Data Science.

Module Summary

After completing this module you should now be able to demonstrate the LOs:

Differentiate legal, professional, and ethical considerations.
Identify and interpret professional Codes of Conduct.
Apply Philosophical Frameworks for ethical reasoning.
Identify sources of potential ethical issues in Data Science.
Practice responsible data science from legal, professional, and ethical perspectives.

You should also have greater competence in considering choices that practice and promote responsible data science today, and in the future.

Finally, when it comes to responsible Data Science, Jane Addams reminds us that just thinking about ethics is not enough, …

“Action indeed is the sole medium of expression for ethics.”

V. References

ACM. 2021. “The Code for Computing Professionals.” https://www.acm.org/code-of-ethics.

Acton, Carmen. 2022. “Are You Aware of Your Biases?” Harvard Business Review, February. https://hbr.org/2022/02/are-you-aware-of-your-biases.

APA. 2022. “FAQs about IRBs.” Https://Www.apa.org. https://www.apa.org/advocacy/research/defending-research/review-boards.

ASA. 2021. “Ethical Guidelines for Statistical Practice.” https://www.amstat.org/ASA/Your-Career/Ethical-Guidelines-for-Statistical-Practice.aspx.

Association, DS. 2021. “Code of Conduct.” https://www.datascienceassn.org/code-of-conduct.html.

AutoML. 2022. “Ethics and Accessibility Guidelines.” https://2023.automl.cc/ethics/.

Baer, Tobais. 2019. Understand, Manage, and Prevent Algorithmic Bias. 1st ed. Aprees.

Bracey, Kali, and Marguerite Moeller. 2018. “Legal Considerations When Using Big Data and Artificial Intelligence to Make Credit Decisions.” https://github.com/AU-datascience/data/blob/main/413-613/Bracey%20Moeller%20Unregulated%20e-scores%20March%202018.pdf.

Brown University, Science and Technology Studies. 2021. “A Framework for Making Ethical Decisions.” https://www.brown.edu/academics/science-and-technology-studies/framework-making-ethical-decisions.

Bureau, US Census. 2023. “Understanding Differential Privacy.” Census.gov. https://www.census.gov/programs-surveys/decennial-census/decade/2020/planning-management/process/disclosure-avoidance/differential-privacy.html.

Burgess, Matt. 2020. “What Is GDPR? A Summary Guide for the UK.” Wired UK, March. https://www.wired.co.uk/article/what-is-gdpr-uk-eu-legislation-compliance-summary-fines-2018.

Calzon, Bernadita. 2021. “Misleading Statistics Real World Examples For Misuse of Data.” BI Blog | Data Visualization & Analytics Blog | Datapine. https://www.datapine.com/blog/misleading-statistics-and-data/.

Coy, Kevin, and Neil Hoffman. 2016. “Big Data Analytics Under HIPAA.” https://www.jdsupra.com/legalnews/big-data-analytics-under-hipaa-80678/.

“Davy Crockett.” 2024. Wikipedia, January. https://en.wikipedia.org/w/index.php?title=Davy_Crockett&oldid=1193687251.

Desjardins, Jeff. 2017. “Every Single Cognitive Bias in One Infographic.” https://www.visualcapitalist.com/every-single-cognitive-bias/.

Devaux, Elise. 2022. “What Is Differential Privacy: Definition, Mechanisms, and Examples - Statice.” https://www.statice.ai/post/what-is-differential-privacy-definition-mechanisms-examples.

DHUD, US. 2021. “Fair Housing Act.” https://www.hud.gov/program_offices/fair_housing_equal_opp/fair_housing_act_overview.

DoD, US. 2020. “DOD Adopts 5 Principles of Artificial Intelligence Ethics.” https://www.defense.gov/Explore/News/Article/Article/2094085/dod-adopts-5-principles-of-artificial-intelligence-ethics/.

Edelman, Gilead. 2020. “Everything You Need to Know About the CCPA.” https://www.wired.com/story/ccpa-guide-california-privacy-law-takes-effect/.

EEOC, US. 2021a. “Genetic Information Nondiscrimination Act of 2008.” https://www.eeoc.gov/laws/statutes/gina.cfm.

———. 2021b. “Laws Enforced by EEOC.” https://www.eeoc.gov/statutes/laws-enforced-eeoc.

Fleisher, Will. 2024. “AI Ethics.” Center for Digital Ethics. https://digitalethics.georgetown.edu/ai-artificial-intelligence-ethics/.

FTC. 2013a. “Children’s Online Privacy Protection Rule ("COPPA").” https://www.ftc.gov/enforcement/rules/rulemaking-regulatory-reform-proceedings/childrens-online-privacy-protection-rule.

FTC, US. 2013b. “Fair Credit Reporting Act.” https://www.ftc.gov/enforcement/statutes/fair-credit-reporting-act.

———. 2018. “Credit Reporting Information.” https://www.ftc.gov/news-events/media-resources/consumer-finance/credit-reporting.

GoogleAI. 2022. “Our Principles.” Google AI. https://ai.google/principles/.

Gordon, Cindy. 2022. “2023 Will Be The Year Of AI Ethics Legislation Acceleration.” Forbes. https://www.forbes.com/sites/cindygordon/2022/12/28/2023-will-be-the-year-of-ai-ethics-legislation-acceleration/.

Hand, David J. 2018. “Aspects of Data Ethics in a Changing World: Where Are We Now?” Big Data 6 (3): 176–90. https://doi.org/10.1089/big.2018.0083.

HCAI, Standford. 2022. “The AI Index Report Artificial Intelligence Index.” Welcome to the 2022 AI Index Report. https://aiindex.stanford.edu/report/.

IBM. 2019. “IBM’S Principles for Data Trust and Transparency.” IBM Policy. https://www.ibm.com/policy/trust-principles/.

IEAI-ML. 2021. “Awesome AI Guidelines.” https://github.com/EthicalML/awesome-artificial-intelligence-guidelines.

“Implicit Bias Vs Unconscious Bias: Types Ways to Prevent Them.” 2022. https://getimpactly.com//post/implicit-bias-vs-unconscious-bias.

Implicit, Project. 2022. “Testing.” https://implicit.harvard.edu/implicit/.

INFORMS. 2021. “INFORMS Ethics Guidelines.” https://www.informs.org/About-INFORMS/Governance/INFORMS-Ethics-Guidelines.

Loeb, Emily, Adam Unikowsky, Caroline Cease, and Benjamin Hand. 2023. “Client Alert: Emerging AI Regulation: A Global Patchwork Quilt.” Jenner - Client Alert: Emerging AI Regulation: A Global Patchwork Quilt. https://www.jenner.com/en/news-insights/publications/client-alert-emerging-ai-regulation-a-global-patchwork-quilt.

Loukides, Mike, Hilary Mason, and DJ Patil. 2018. Ethics and Data Science. 1st ed. O’Reilly Media, Inc. https://www.oreilly.com/library/view/ethics-and-data/9781492043898/.

Lynch, Shana. 2023. “2023 State of AI in 14 Charts.” Stanford HAI. https://hai.stanford.edu/news/2023-state-ai-14-charts.

MIT-Media-Lab. 2019. “AI Blindspot: A Discovery Process for Preventing, Detecting, and Mitigating Bias in AI Systems.” https://aiblindspot.media.mit.edu/.

O’Brien, Matt. 2019. “MIT Researcher Exposing Bias in Facial Recognition Tech ...” https://www.insurancejournal.com/news/national/2019/04/08/523153.htm.

O’Neil, Cathy. 2017. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. https://weaponsofmathdestructionbook.com/.

Pomeroy, Robin, and Simon Torkington. 2023. “How Can We Create Responsible AI? Industry Leaders Share Their Insights.” World Economic Forum. https://www.weforum.org/agenda/2023/06/responsible-generative-ai-industry-regulation/.

Posit. 2022. “Quarto.” Quarto Overview. https://quarto.org/.

Psychology, Practical. 2017. “12 Cognitive Biases Explained -.” https://www.youtube.com/watch?v=wEwGBIr_RIw.

Raths, David. 2018. “When Learning Analytics Violate Student Privacy.” https://campustechnology.com/articles/2018/05/02/when-learning-analytics-violate-student-privacy.aspx.

RSS-IFA. 2021. “Data Science Ethics Guidelines UK.” https://www.actuaries.org.uk/upholding-standards/data-science-ethics.

Schneble, Christophe Olivier, Bernice Simone Elger, and David Martin Shaw. 2020. “Google’s Project Nightingale Highlights the Necessity of Data Science Ethics Review.” EMBO Molecular Medicine 12 (3): e12053. https://doi.org/10.15252/emmm.202012053.

SCU, Santa Clara. 2022. “An Introduction to Data Ethics.” https://www.scu.edu/ethics/focus-areas/technology-ethics/resources/an-introduction-to-data-ethics/.

“Trolley Problem.” 2024. Wikipedia, January. https://en.wikipedia.org/w/index.php?title=Trolley_problem&oldid=1193140026.

UChicago. n.d. “Center for Applied Artificial Intelligence.” The University of Chicago Booth School of Business. Accessed December 26, 2022. https://www.chicagobooth.edu/research/center-for-applied-artificial-intelligence.

Union, European. 2023. “Artificial Intelligence Act: Deal on Comprehensive Rules for Trustworthy AI.” https://www.europarl.europa.eu/news/en/press-room/20231206IPR15699/artificial-intelligence-act-deal-on-comprehensive-rules-for-trustworthy-ai.

University, Princeton. 2018. “Dialogues on AI and Ethics Case Studies.” https://aiethics.princeton.edu/case-studies/.

Us Dept of Labor. 2024. “Guidance on the Protection of Personal Identifiable Information.” DOL. http://www.dol.gov/general/ppii.

US DHHS, Office for Civil. 2015. “HIPAA for Professionals.” Text. https://www.hhs.gov/hipaa/for-professionals/index.html.

US DOEd, Privay Technical Assistance Center. 2017. “Integrated Data Systems and Student Privacy.”

Vigen, Tyler. 2021. “15 Insane Things That Correlate With Each Other.” http://tylervigen.com/spurious-correlations.

Weissgerber Tracey L., Tracey et al. 2019. “Reveal, Don’t Conceal.” Circulation 140 (18): 1506–18. https://doi.org/10.1161/CIRCULATIONAHA.118.037777.

Wikipedia. 2022. “General Data Protection Regulation.” Wikipedia, December. https://en.wikipedia.org/w/index.php?title=General_Data_Protection_Regulation&oldid=1128013242.