12  Considerations for Responsible Data Science

Keywords

responsible data science, data science life cycle, law, policy, ethics, ethcial frameworks

12.1 Introduction

12.1.1 Learning Outcomes

• Differentiate legal, professional, and ethical considerations.
• Identify and interpret professional Codes of Conduct.
• Apply Philosophical Frameworks for ethical reasoning.
• Identify sources of potential ethical issues in Data Science.
• Practice responsible data science from legal, professional, and ethical perspectives.

12.1.2 References:

• See references.

12.1.2.1 Other References

• See links in the notes.

Warning

These notes provide general background to facilitate educational discussion.

They do not offer any form of legal advice or recommendations.

12.2 Legal, Professional, and Ethical Considerations

12.2.1 Data Science Follows a Life Cycle

A Data Science Life Cycle

The DS Life Cycle is about answering a question.

Responsible Data Science Affects the Whole Life Cycle

Responsible Data Science asks questions across the life cycle.

12.2.2 Data Scientists Make Choices When Answering Questions

• What kinds of projects do I work on and What questions do I analyze?
• Who are the Stakeholders?
• Towards what goals do I optimize my models? Accuracy, Fairness, Equity, Equality, …?
• How do I get my data? How do I protectmy data?
• Is my data representative of the population?
• What do I do about “bad” or missing data or “outliers” that “mess up” my results?
• What variables/features/attributes do I use?
• How much effort do I put into checking my results? Are they repeatable?
• How do I leverage/credit other people’s work?
• How do I report my results - what is intellectual propertyand what should be public?
• …
  
• Choices can involve Legal, Professional, or Ethical Considerations

Choices have consequences, with benefits and risks.

12.2.3 Legal Considerations

Legal considerations address the criminal/civil risks for violating laws.

• Laws (statutes) specify permissible and/or impermissible activities as well as potential punishments (judicial, e.g., imprisonment).

• Policies (regulations) provide guidance on implementing activities (within legal constraints) along with adjudication procedures and potential punishments (non-judicial, e.g., debarment).

Legal issues in big data include how you gather, protect, and share data, and increasingly, how you use it.

Laws are “local” not universal: Pirates or Privateers

Laws do not keep pace with technology (generative AI anyone?) and can be difficult to interpret.

When in doubt, ask a lawyer competent in the issue.

12.2.4 Laws Proscribe Some Choices

• Health Insurance Portability and Accountability Act (HIPAA) (US DHHS 2015). See Big Data Analytics under HIPAA (Coy and Hoffman 2016)
• Family Educational Rights and Privacy Act (FERPA) (US DOEd 2017)
• Fair Credit Reporting Act (FTC 2013b). See FTC Credit Reporting Information (FTC 2018).
• Fair Housing Act (DHUD 2021)
• Equal Employment Opportunity Act (EEOC 2021b)
• Childrens Online Privacy Protect Act (COPPA) and Rule (FTC 2013a)
• Genetic Information Nondiscrimination Act (GINA) (EEOC 2021a)
• European General Data Protection Regulation (GDPR) (Wikipedia 2022)
  • See What is GDPR (Burgess 2020)
• California Consumer Protection Act (Edelman 2020)
• And many more … with the expectation of more to come …
  • Consider recent Congressional interest in Section 230 of the Communications Decency Act of 1996.

Ignorantia juris non excusat

12.2.5 Professional Considerations

Professional considerations address the risk of activities related to organizations with which you affiliate.

• Professional Organizations
• Employer Organizations Organizational bylaws or policies identify and manage risk to the institution.

These may include professional “Codes of Conduct” or “Codes of Ethics”.

• American Statistical Association (ASA 2021)
• Association of Computing Machinery (ACM 2021)
• INFORMS Ethical Guidelines (INFORMS 2021)
• Data Science Association (Association 2021)
• AMA Journal of Ethics addresses big data in health care (AMA 2021)

Organizational behavior and ethics may conflict with individual ethics.

• Choices can include trying to change the organization or offending individuals; or leaving or being forced to leave the organization, with potential legal issues as well.

When in doubt, ask a mentor or manager you trust.

12.2.6 Ethical Considerations

Ethical considerations arise when asking

• “What should I do?”
• “What is Right”?

Ethical Choices can be hard, especially when choices may require violating a law, regulation, or professional guideline.

Individual principles and cultural norms shape options and guide choices in complex situations.

Often, there is no universally-accepted or even a good “right answer”.

• May have to choose between two bad outcomes as in the The Trolley Problem which has many variations and analogs. (Merriam-Webster 2021)

May have to choose between individual and group outcomes.

Ethical choices can lead to feelings of guilt, group reprobation, civil action (torts), or criminal charges.

Many, many, frameworks attempt to guide Ethical Choices.

12.3 Ethical Frameworks

12.3.1 Some (of many) Ethical Frameworks ¹

Consequentialist or Utilitarianism: greatest balance of good over harm (groups/individual).

• Focus is on the consequences.
• Choose the future outcomes that produce the most good.
• Compromise is expected as the end justifies the means.

Duty or Deontology: Do your Duty , Respect the Rules, Be Fair, Follow Divine Guidance.

• Focus is on the rules and whether the act violated the rules regardless of consequences.
• Do what is “right” regardless of the consequences or emotions.
• Everyone has the same duties and ethical obligations at all times.

Virtues: Live a Virtuous Life by developing the proper character traits.

• Focus is on what society would say what a virtuous person would do.
• Ethical behavior is whatever a virtuous person would do.
• Tends to reinforce local cultural norms as the standard of ethical behavior.

Rights

• Focus on the rights of the affected stakeholders, both individuals and groups.
• Ethical behavior does not result in a violation of someone’s rights.
• How do we know agree on is a “right” versus a “privilege” or “benefit”?

Frameworks can conflict with each other or are “wrong” in the extreme.

No single or simple right answer!

12.3.2 Using Frameworks in Ethical Decisions

1. Recognize There May Be an Ethical Issue.
   • Assess the underlying definitions, facts, and assumptions and constraints
2. Consider the Parties Involved, the stakeholders.
   • What individuals or groups might be harmed or benefit, and by when.
3. Gather all Relevant Information. Are you missing key facts? Are they knowable?
4. Formulate Actions and Evaluate under Alternative Frameworks.
   • What will produce the most good and do the least harm? ( Utilitarian)
   • What will be in accordance with existing rules or principles? (Duty)
   • What leads me to act as the sort of person I want to be? (Virtues)
   • What respects the rights of everyone affected by the decision? (Rights)
   • What treats people equally, equitably, or proportionately? ( Justice)
   • What serves the entire community, not just some members? (Common Good)
5. Examine Alternatives and Make a Decision.
6. Act and Observe.
7. Assess and Reflect on the Outcomes.

If you don’t like the outcomes, restart the process.

12.4 Identifying Ethical Issues

12.4.1 Our Biased Brains Helped Us Survive

Our brains have evolved mechanisms to make quick decisions.

These are the source of “Unconscious Biases” or “Implicit Biases”.

• Unconscious stereotypes, prejudices, or attitudes that affect our decision-making, perception, and social behavior (“Implicit Bias Vs Unconscious Bias: Types Ways to Prevent Them” 2022).
• Friend or Threat, Familiar or Strange, Us versus Them
• Learn more and test yourself at Project Implicit (Implicit 2022).
• Under stress, we tend to bypass the higher-level cognitive centers that evolved later and take more time to reason.

Humans get comfortable with patterns which can lead to systematic deviations from making rational judgments.

• 12 Major Types of Cognitive Bias Explained. (Psychology 2017)
• Many potential biases could affect our decisions. (Desjardins 2017)

Ethical Challenges can arise from our own implicit biases or the implicit biases of others affecting our data, thoughts, and actions.

Short-term choices under stress may be bad in the long run ….

12.4.2 Given Bias in Data and Algorithms are DS Systems Ethical?

Not a new issue - goes back decades. However, the explosive growth of AI systems to support and even make decisions is generating concerns.

• Bias in the (training) data (historical, sampling, …) can drive biased outcomes.
• Are algorithms really less biased than people? It depends …
  • How do you measure fairness? See Tutorial: 21 Definitions of Fairness …
  • How can you tell with “black box” models? See Machine learning transparency and … A Toolkit for Transparency in Dataset Documentation
  • Algorithms find “hidden” relationships among proxy variablesthat can distort the interpretations.
  • What are the trade offs among accuracy, explainability, and transparency?
    • Increasing “fairness” can decrease accuracy.

Active area for research and publication

• MIT AI Blindspot Cards (MIT-Media-Lab 2019)
• Understand, Manage, and Prevent Algorithmic Bias(Baer 2019)
• University of Chicago’s Algorithmic Bias Playbook(UChicago n.d.)

12.4.3 Three Articles for Consideration

1. Higher error rates in classifying the gender of darker-skinned women than for lighter-skinned men (O’Brien 2019)

2. Big Data used to generate unregulated e-scores in lieu of FICO scores for Credit in Lending] is on Canvas. (Bracey and Moeller 2018)

3. Learning Analytics Can Violate Student Privacy (Raths 2018)

Discussion Questions

• Is there an ethical issue or more than one? What is it?
• Who is affected and who is responsible?
• What would you do differently or recommend?

12.4.4 More Examples of Ethical Issues

• Contradictions and competition among legal, professional, and ethical guidelines.
• Using biased data ( even unknowingly) or eliminating extreme values or small groups.
• Using Proxies for “Protected” Attributes (even unknowingly).
• Protection of Intellectual Property versus Explainability, Transparency and Accountability
• Law of Unintended Consequences - people will use your products and solutions in “creative” ways that - May not align with your principles, or, - Be technically inappropriate.

So what can you do? What should you do?

12.5 Practicing Responsible Data Science.

What Can You Do? What Should You Do?

12.5.1 Consider Ethical Choices Across the DS Life Cycle

Ask a question: Equity or equality ? Stakeholders and Trade offs ? What are our interests? Recency or Confirmation Bias?

Frame the Analysis: What is the population? Role of proxy variables? How are metrics for “fairness”affecting groups/individuals? Do we need an IRB(APA 2022)?

Get Data: How was it collected? Was informed consent required/given? Is there balanced representation ? Selection Bias? Availability Bias? Survivorship Bias?

Shape Data: Are we aggregating distinct groups? How do we treat missing data? Are we separating training and testing data ?

Model and Analyze: How are we documenting assumptions, treating extreme values, or checking over-fitting? Are we checking multiple fairness and performance metrics?

Communicate Results: Are the graphs misleading? Did we cherry pick or data snoop? Are we reporting \(p\)-values and hyper-parameters?

Deploy/Implement: Is the deployment accessible to all?

Observe Outcomes: Can we check assumptions and analyze outcomes for bias?

Are we following professional guidelines from ASA, ACM, INFORMS, …?

12.5.2 Consider Frameworks for Responsible Data Science

UK Government

Published by the Royal Statistical Society and the Institute and Faculty of Actuaries in A Guide for Ethical Data Science

1. Start with clear user need and public benefit
   
2. Be aware of relevant legislation and codes of practice
3. Use data that is proportionate to the user need
4. Understand the limitations of the data
5. Ensure robust practices and work within your skill set
6. Make your work transparent and be accountable
7. Embed data use responsibly

(RSS-IFA 2021)

US DoD

Department of Defense AI Capabilities shall be:

• Responsible. … exercise appropriate levels of judgment and care, while remaining responsible for the development, deployment, and use….

• Equitable. … take deliberate steps to minimize unintended bias in AI capabilities.

• Traceable. … develop and deploy AI capabilities such that relevant personnel have an appropriate understanding …, including with transparent and auditable methodologies, data sources, and design procedures and documentation.

• Reliable. … AI capabilities will have explicit, well-defined uses, and the safety, security, and effectiveness … will be subject to testing and assurance within those defined uses ….

• Governable. … design and engineer AI capabilities to fulfill their intended functions while possessing the ability to detect and avoid unintended consequences, and the ability to disengage or deactivate deployed systems that demonstrate unintended behavior.

(DoD 2020)

Google

AI Principles

1. Be socially beneficial.
2. Avoid creating or reinforcing unfair bias.
3. Be built and tested for safety.
4. Be accountable to people.
5. Incorporate privacy design principles.
6. Uphold high standards of scientific excellence.
7. Be made available for uses that accord with these principles.

We will not design or deploy AI in the following application areas: Weapons, Surveillance, …

(GoogleAI 2022)

IBM

IBM Principles for Trust and Transparency

1. The purpose of AI is to augment human intelligence. The purpose of AI and cognitive systems developed and applied by IBM is to augment – not replace – human intelligence.
2. Data and insights belong to their creator.
   

• IBM clients’ data is their data, and their insights are their insights. Client data and the insights produced on IBM’s cloud or from IBM’s AI are owned by IBM’s clients. We believe that government data policies should be fair and equitable and prioritize openness.

3. New technology, including AI systems, must be transparent and explainable.
   

• For the public to trust AI, it must be transparent. Technology companies must be clear about who trains their AI systems, what data was used in that training and, most importantly, what went into their algorithm’s recommendations. If we are to use AI to help make important decisions, it must be explainable.

(IBM 2019)

The 5 C’s Framework

A guide to Building “Trustworthy” Data Products

Based on the golden rule: Treat others’ data as you would have them treat your data

• Consent - Get permission from the owners or subjects of the data before …

• Clarity - Ensure permission is based on a clear understanding of the extent of your intended usage

• Consistency - Build trust by ensuring third parties adhere to your standards/agreements

• Control (and Transparency) - Respond to data subject requests for access/modification/deletion, e.g., the right to be forgotten

• Consequences (and Harm) - Consider how your usage may affect others in society and potential unintended applications.

(Loukides, Mason, and Patil 2018)

12.5.3 To Be an Ethically Responsible Data Scientist …

Integrate ethical decision making into your environment.

• Recognize legal, professional, and ethical issues surround your work - See Weapons of Math Destruction (O’Neil 2017)
• Ask Questions of Peers, Managers, Mentors, and Experts to get guidance.
• Document your sources and approach: assumptions, data, code, and references. - Use Literate Programming. Quarto(Posit 2022)
• Protect your data: especially Personally Identifiable Information (PII) or HIPAA data. - Consider Differential Privacy a la the Census Bureau.(ETI 2022)
• Create reproducible analysis and solicit appropriate reviews and feedback.
  • Avoid “Spurious Correlations”. (Vigen 2021)
  • Produce Results that Reveal, Don’t Conceal. (Weissgerber Tracey L. 2019)
  • Don’t create Examples of Misleading Statistics.(Calzon 2021)
• Follow Publishing and Conference Guidelines e.g., AutoML’s. (AutoML 2022)
• Consider your own biases (Acton 2022)

As Davy Crockett might say, “Try to be sure you are right, then Go Ahead!”

12.5.4 Stay Current on Emerging Ideas

• Stanford ’s Human-Centered AI Index for 2022 Report(HCAI 2022)
• Review articles on Big Data/Data Science/AI Ethics, e.g.,
  • Google’s Project Nightingale (Schneble, Elger, and Shaw 2020)
  • Aspects of Data Ethics in a Changing World: Where Are We Now? (Hand 2018)
• Review the Inventory of Awesome AI Guidelines (IEAI-ML 2021)
• Investigate Academic Centers
  • Markkula Center at SCU Intro Course in DS Ethics (SCU 2022)
• Consider popular press articles
  • Self-restraint and regulation’ - how the tech companies transforming the world view responsible AI Pomeroy and Torkington (2023)
  • 2023 Will Be The Year Of AI Ethics Legislation Acceleration Gordon (2022)
  • 2023 State of AI in 14 Charts Lynch (2023)
• Analyze Case Studies, e.g., Princeton University’s Case Studies (University 2018)

Try to stay on the fast moving train that is Responsible Data Science.

12.6 Considerations for Responsible Data Science Summary

After completing this section, you should now be able to demonstrate the LOs:

• Differentiate legal, professional, and ethical considerations.
• Identify and interpret professional Codes of Conduct.
• Apply Philosophical Frameworks for ethical reasoning.
• Identify sources of potential ethical issues in Data Science.
• Practice responsible data science from legal, professional, and ethical perspectives.

You should also have greater competence in considering choices that practice and promote responsible data science today, and in the future.

Finally, when it comes to responsible Data Science, Jane Adams reminds us that thinking about ethics is not enough, …

“Action indeed is the sole medium of expression for ethics.”

ACM. 2021. “The Code for Computing Professionals.” https://www.acm.org/code-of-ethics.

Acton, Carmen. 2022. “Are You Aware of Your Biases?” Harvard Business Review, February. https://hbr.org/2022/02/are-you-aware-of-your-biases.

AMA. 2021. “AMA Journal of Ethics.” AMA Journal of Ethics, February. https://journalofethics.ama-assn.org/.

APA. 2022. “FAQs about IRBs.” Https://Www.apa.org. https://www.apa.org/advocacy/research/defending-research/review-boards.

ASA. 2021. “Ethical Guidelines for Statistical Practice.” https://www.amstat.org/ASA/Your-Career/Ethical-Guidelines-for-Statistical-Practice.aspx.

Association, DS. 2021. “Code of Conduct.” https://www.datascienceassn.org/code-of-conduct.html.

AutoML. 2022. “Ethics and Accessibility Guidelines.” https://2023.automl.cc/ethics/.

Baer, Tobais. 2019. Understand, Manage, and Prevent Algorithmic Bias. First. Aprees.

Bracey, Kali, and Marguerite Moeller. 2018. “Legal Considerations When Using Big Data and Artificial Intelligence to Make Credit Decisions.” https://jenner.com/library/posts/17946.

Brown University, Science and Technology Studies. 2021. “A Framework for Making Ethical Decisions.” https://www.brown.edu/academics/science-and-technology-studies/framework-making-ethical-decisions.

Burgess, Matt. 2020. “What Is GDPR? A Summary Guide for the UK.” Wired UK, March. https://www.wired.co.uk/article/what-is-gdpr-uk-eu-legislation-compliance-summary-fines-2018.

Calzon, Bernadita. 2021. “Misleading Statistics Real World Examples For Misuse of Data.” BI Blog | Data Visualization & Analytics Blog | Datapine. https://www.datapine.com/blog/misleading-statistics-and-data/.

Coy, Kevin, and Neil Hoffman. 2016. “Big Data Analytics Under HIPAA.” https://www.jdsupra.com/legalnews/big-data-analytics-under-hipaa-80678/.

Desjardins, Jeff. 2017. “Every Single Cognitive Bias in One Infographic.” https://www.visualcapitalist.com/every-single-cognitive-bias/.

DHUD, US. 2021. “Fair Housing Act.” https://www.hud.gov/program_offices/fair_housing_equal_opp/fair_housing_act_overview.

DoD, US. 2020. “DOD Adopts 5 Principles of Artificial Intelligence Ethics.” https://www.defense.gov/Explore/News/Article/Article/2094085/dod-adopts-5-principles-of-artificial-intelligence-ethics/.

Edelman, Gilead. 2020. “Everything You Need to Know About the CCPA.” https://www.wired.com/story/ccpa-guide-california-privacy-law-takes-effect/.

EEOC, US. 2021a. “Genetic Information Nondiscrimination Act of 2008.” https://www.eeoc.gov/laws/statutes/gina.cfm.

———. 2021b. “Laws Enforced by EEOC.” https://www.eeoc.gov/statutes/laws-enforced-eeoc.

ETI, MIT. 2022. “What Is Differential Privacy?” What Is Differential Privacy? http://eti.mit.edu/what-is-differential-privacy/.

FTC. 2013a. “Children’s Online Privacy Protection Rule ("COPPA").” https://www.ftc.gov/enforcement/rules/rulemaking-regulatory-reform-proceedings/childrens-online-privacy-protection-rule.

FTC, US. 2013b. “Fair Credit Reporting Act.” https://www.ftc.gov/enforcement/statutes/fair-credit-reporting-act.

———. 2018. “Credit Reporting Information.” https://www.ftc.gov/news-events/media-resources/consumer-finance/credit-reporting.

GoogleAI. 2022. “Our Principles.” Google AI. https://ai.google/principles/.

Gordon, Cindy. 2022. “2023 Will Be The Year Of AI Ethics Legislation Acceleration.” Forbes. https://www.forbes.com/sites/cindygordon/2022/12/28/2023-will-be-the-year-of-ai-ethics-legislation-acceleration/.

Hand, David J. 2018. “Aspects of Data Ethics in a Changing World: Where Are We Now?” Big Data 6 (3): 176–90. https://doi.org/10.1089/big.2018.0083.

HCAI, Standford. 2022. “The AI Index Report Artificial Intelligence Index.” Welcome to the 2022 AI Index Report. https://aiindex.stanford.edu/report/.

IBM. 2019. “IBM’S Principles for Data Trust and Transparency.” IBM Policy. https://www.ibm.com/policy/trust-principles/.

IEAI-ML. 2021. “Awesome AI Guidelines.” https://github.com/EthicalML/awesome-artificial-intelligence-guidelines.

“Implicit Bias Vs Unconscious Bias: Types Ways to Prevent Them.” 2022. https://getimpactly.com//post/implicit-bias-vs-unconscious-bias.

Implicit, Project. 2022. “Testing.” https://implicit.harvard.edu/implicit/.

INFORMS. 2021. “INFORMS Ethics Guidelines.” https://www.informs.org/About-INFORMS/Governance/INFORMS-Ethics-Guidelines.

Loukides, Mike, Hilary Mason, and DJ Patil. 2018. Ethics and Data Science. First. O’Reilly Media, Inc. https://www.oreilly.com/library/view/ethics-and-data/9781492043898/.

Lynch, Shana. 2023. “2023 State of AI in 14 Charts.” Stanford HAI. https://hai.stanford.edu/news/2023-state-ai-14-charts.

Merriam-Webster. 2021. “Next Stop: ‘Trolley Problem’.” https://www.merriam-webster.com/words-at-play/trolley-problem-moral-philosophy-ethics.

MIT-Media-Lab. 2019. “AI Blindspot: A Discovery Process for Preventing, Detecting, and Mitigating Bias in AI Systems.” https://aiblindspot.media.mit.edu/.

O’Brien, Matt. 2019. “MIT Researcher Exposing Bias in Facial Recognition Tech ...” https://www.insurancejournal.com/news/national/2019/04/08/523153.htm.

O’Neil, Cathy. 2017. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. https://weaponsofmathdestructionbook.com/.

Pomeroy, Robin, and Simon Torkington. 2023. “How Can We Create Responsible AI? Industry Leaders Share Their Insights.” World Economic Forum. https://www.weforum.org/agenda/2023/06/responsible-generative-ai-industry-regulation/.

Posit. 2022. “Quarto.” Quarto Overview. https://quarto.org/.

Psychology, Practical. 2017. “12 Cognitive Biases Explained -.” https://www.youtube.com/watch?v=wEwGBIr_RIw.

Raths, David. 2018. “When Learning Analytics Violate Student Privacy.” https://campustechnology.com/articles/2018/05/02/when-learning-analytics-violate-student-privacy.aspx.

RSS-IFA. 2021. “Data Science Ethics Guidelines UK.” https://www.actuaries.org.uk/upholding-standards/data-science-ethics.

Schneble, Christophe Olivier, Bernice Simone Elger, and David Martin Shaw. 2020. “Google’s Project Nightingale Highlights the Necessity of Data Science Ethics Review.” EMBO Molecular Medicine 12 (3): e12053. https://doi.org/10.15252/emmm.202012053.

SCU, Santa Clara. 2022. “An Introduction to Data Ethics.” https://www.scu.edu/ethics/focus-areas/technology-ethics/resources/an-introduction-to-data-ethics/.

UChicago. n.d. “Center for Applied Artificial Intelligence.” The University of Chicago Booth School of Business. Accessed December 26, 2022. https://www.chicagobooth.edu/research/center-for-applied-artificial-intelligence.

University, Princeton. 2018. “Dialogues on AI and Ethics Case Studies.” https://aiethics.princeton.edu/case-studies/.

US DHHS, Office for Civil. 2015. “HIPAA for Professionals.” Text. https://www.hhs.gov/hipaa/for-professionals/index.html.

US DOEd, Privay Technical Assistance Center. 2017. “Integrated Data Systems and Student Privacy.”

Vigen, Tyler. 2021. “15 Insane Things That Correlate With Each Other.” http://tylervigen.com/spurious-correlations.

Weissgerber Tracey L., Tracey et al. 2019. “Reveal, Don’t Conceal.” Circulation 140 (18): 1506–18. https://doi.org/10.1161/CIRCULATIONAHA.118.037777.

Wikipedia. 2022. “General Data Protection Regulation.” Wikipedia, December. https://en.wikipedia.org/w/index.php?title=General_Data_Protection_Regulation&oldid=1128013242.

---

1. Brown University (2021)