From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
From Design to Impact Assessment
Prof. Rafael A. Calvo
@Rafael_A_Calvo
1
Go to http://menti.com
use the code 11 35 66
What is your background?
Agenda
• HISTORY – From unbridled progress to human impact
• APPROACH – How do we assess human impact?
• PROCESS – A process for responsible technology
• METHODS – Values sensitive design and other methods for understanding impact
• CASE STUDIES – Examples in practice
3
HUMAN IMPACT ASSESSMENT
HISTORY
7
9
Industrial Revolution
INPUTS (depletion)
ENVIRONMENT ECONOMY
NATURAL RESOURCES TECHNOLOGY
land, water, air
OUTPUTS (impacts)
Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).Circular Economy
INPUTS (depletion)
ENVIRONMENT ECONOMY
NATURAL RESOURCES
TECHNOLOGY
land, water, air
OUTPUTS (impacts)
Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).OUTPUTS + WASTE (impacts) Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).
Infographic: Wikimedia Commons, image credit: RawPixel 13
"Data is the new oil"
- Clive Humby
1415
INPUTS (depletion)
ENVIRONMENT ECONOMY
NATURAL RESOURCES HUMAN TECHNOLOGY
land, water, air attention,
behavior, data
OUTPUTS (impacts)
Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).HUMAN IMPACT ASSESSMENT APPROACH
Human Impact Assessment for Technology - HIAT
Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).Differences between traditional EIA and HIAT
EIA HIAT
• Slow or static once installed • Dynamic, Self-learns
• has boundaries, regional • Unbounded
• Largely anticipatory • Continuous & Iterative
• Nature as resource • Humans as resource
• Long history • New to technologyGo to http://menti.com
use the code 11 35 66
What values do you think a
Human Impact Assessment should protect?Go to http://menti.com
use the code 11 35 66
What would you measure to judge if the values are being
satisfied?Technologists & Users Doctors and Patients
• Do no harm
• Wellbeing
• Human Autonomy
• Fairness
Engineering Ethics Biomedical Ethics
22“All Technology
should be designed to support POSITIVE
COMPUTING
psychological wellbeing” Technology for Wellbeing
and Human Potential
Rafael A. Calvo
Dorian Peters
R.A. Calvo and D. Peters. (2014) “Positive Computing: Technology for wellbeing and human potential” MIT Press.Speed-dating with key psychological theories
MODELS OF PSYCHOLOGICAL WELLBEING
1. Clinical DSM Global Assessment of Functioning (e.g. CES-D < 16)
2. Positive Psychology (e.g. Fredrickson, Seligman, Lyubimorski,
Huppert, etc. )
3. Socio-emotional intelligence (e.g. Salovey, Mayer & Caruso)
4. Self-determination Theory (Ryan & Deci)
5. Subjective well-being (e.g. Diener, Kahneman)
6. Neuroscience and Physiology (e.g., Panksepp, Davidson)Basic Psychological Needs & SDT
1. Autonomy - feeling willingness and volition in action
2. Competence - feeling able and effective
3. Relatedness - feeling connected to othersThe Spheres
of Technology
Experience
Credit: Dorian Peters, Rafael A. Calvo, Richard, M. Ryan, 2019
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience.
Frontiers in Psychology, 9(MAY). @Rafael_A_CalvoAdapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience. Frontiers in Psychology, 9(MAY).
METUX
SPHERE PSYCHOLOGICAL NEEDS IN CONTEXT DESIRABLE OUTCOMES MEASURES
Adoption • To what extent is adoption of the technology autonomously • Adoption of the technology (purchase, • ACTA
The decision-making experience motivated? download, use)
between becoming aware of a new • To what extent does a potential user anticipate need satisfaction
technology and acquiring it. in relation to technology
Interface • To what extent does direct interaction with the technology (i.e., • Sustained engagement with technology, • TENS-Interface
The experience of interacting with a via the user interface) support psychological need satisfaction? • High usability scores,
technology via its interface during • User satisfaction
use.
Tasks • To what extent does engagement in technology-specific tasks • Sustained engagement with tasks and • TENS-Task
The experience of engaging in a support psychological need satisfaction? (e.g., tracking, text technology,
technology-specific task. chatting, etc.) • User satisfaction
Behaviour • To what extent does the technology improve psychological need • Engagement • Assessments of psychological
The experience of engaging in a satisfaction with respect to the behavior that the technology is (with behavior) need satisfactions in relation
behaviour (that a technology is intended to support? (e.g., exercise, managing a chronic disease, • Satisfaction to behavior (e.g., PNSESi)
intended to support). communicating with friends, speaking a second language.) (with the behavior) • Assessments of behaviour-
• Behavior-specific outcomes (e.g., weight- specific outcomes (e.g.,
loss, symptom control, learning weight measure)
outcomes)
• Greater wellbeing during behavior.
Life • To what extent does the technology influence the user’s • Greater life satisfaction, • BPNSii,
An individual’s overall experience of experience of psychological need satisfaction in their life overall? • Increased wellbeing, • Other validated measures of
life including all that is outside or • Thriving/flourishing flourishing
beyond the technology.
Society • To what extent does the introduction of the technology impact • Increased measures of societal wellbeing, Population measures such as the
The experiences of all members of a on societal wellbeing? • Societal flourishing FSiIi or SLWS iv
society beyond the users of a
technology.
Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience.
Frontiers in Psychology, 9(MAY).METUX Providing evidence for the utility of the model in practice is ongoing.
HUMAN IMPACT ASSESSMENT
PROCESSGo to http://menti.com
use the code 11 35 66
What development process do you follow?A design process to support wellbeing, and ethics
38A design process to support wellbeing, and ethics
Research: investigate the needs, preferences, contexts,
and lives of the people who will be served or, otherwise,
impacted by a technology.
Direct and Indirect Stakeholders
39A design process to support wellbeing, and ethics
Insights: analysis of the data from the
research phase, and synthesis into specific
insights for design.
• Wellbeing theory & opp for design
• Ethics: identifying potential biases,
ethical risks, and tensions.
40A design process to support wellbeing, and ethics
Ideation: divergent generation of
ideas for design solutions.
Ethical reflection can be integrated into
the ideation phase through framing.
41A design process to support wellbeing, and ethics
Prototypes: Converge and build various design
solutions.
Responsible impact analysis involves collaborative
speculation on the wellbeing and ethical impacts
(good and bad) to which a particular design concept
may lead.
This will ideally involve a wide range of stakeholders
including end users.
42A design process to support wellbeing, and ethics
Evaluation: during actual use
Wellbeing impact evaluation involves evaluating the impact of
technology use on a user’s psychological experience during and
after use.
Ethical impact evaluation involves evaluating the ethical impacts of
a technology’s use, not just on its users, but often, also on those
indirectly affected, such as their friends and families, communities,
society as a whole, and the planet.
43Environmental Impact Statements Think before you do it.
EIA Process – a comparisson 1. Screening. Does the proposal require an EIA? 2. Scoping. What environmental impacts need to be examined? 3. Prediction. What is the size & magnitude of the likely impact? 4. Assessment. Is the impact significant? 5. Mitigation. What can be done to address the adverse impact? 6. Review. Is the assessment adequate? 7. Approval decision. Should it be approved? 8. Follow up. What occurred since the implementation?
Software ‘implementations’ Which on we do we
regulate?
Adaptation
Enterprise
to vertical
user
market
e.G Gov A
e.g security
AI algorithms
and framework
e.g. facial Adaptation
to vertical Enterprise
recognition user
market
e.g health e.g Hospital X
Adaptation
to vertical Enterprise
market User
e.g Corp Z
marketingS: software project
Innovation costs I: Infrastructure project
Core Tech Market Client
Adaptation
Cost € Enterprise
to vertical
user
market
e.G Gov A
S e.g security
AI algorithms
and framework
I
Adaptation
e.g. facial
recognition S
to vertical Which
Enterpriseon we do we
user
market
I
e.g health regulate?
e.g Hospital X
Adaptation
to vertical
I Enterprise
market S User
e.g Corp Z
marketing
Time / phaseAlgorithmic Impact Assessment (AINow)
Public agencies urgently need a practical framework to assess automated
decision systems and to ensure public accountability
Its focus is on procurement, and the process:
• Pre-acquisition review: allows the agency and the public to identify.
• Initial agency disclosure requirements. evaluate potential issues of
inaccuracy, bias, and harms to affected communities, and address these
potential impacts
• Comment period. Opportunity for the public to engage
• Due process challenge period. path for the public to challenge an agency
• Renewing AIAs. renew AIAs on a regular schedule
https://ainowinstitute.org/aiareport2018.pdfDigital Catapult - Machine Intelligence Ethics 1. Be clear about the benefits of your product or service 2. Know and manage your risks 3. Use data responsibly 4. Be worthy of trust 5. Promote diversity, equality and inclusion 6. Be open and understandable in communications 7. Consider your business model
Screening and Inclusion Lists • Technology that promises to support health and wellbeing. • Technology for which there is academic evidence that it may cause harm to humans or the environment • Technology targeted to vulnerable populations Positive: easy to apply Negative: • New tech might not be included • Perceived as adding bureaucracy • Salami slicing – “Make a rule and people will find ways to break it”
HUMAN IMPACT ASSESSMENT
METHODSLearning from Environmental Protection
Lessons from EPA:
The process matters: it elicits behavioural change by
proponents and other stakeholders
See Jones, M., & Morrison-Saunders, A. (2017). Understanding the long-term influence of EIA on
organisational learning and transformation. Environmental Impact Assessment Review, 64, 131-138.Impact on direct stakeholders
Lessons from health:
Measuring impact is difficult and expensive,
but necessary.Impact Measures and stakeholders Value sensitive Design provides useful methods to identify and work with stakeholders Direct. Those who will interact with the technology Indirect. Others (including organisations, society)
HUMAN IMPACT ASSESSMENT CASE STUDIES
Case Study: Ada Lovelace Institute & Covid19 report
Ada Lovelace Institute (20 April 2020) Exit through the App Store? A rapid evidence review of the
technical considerations and societal implications of using technology to transition from the COVID-
19 crisis.
Examines the potential development and implementation of technical solutions to support
• symptom tracking,
• contact tracing and
• immunity certification
Evidence from experts for:
• Government
• Parliament
• Technology Provider and Developers
56Ada Lovelace Institute (20 April 2020) Exit through the App Store…
Contact Tracing
“The technical limitations, barriers to effective deployment and social impacts demand more
consideration before digital contact tracing is deployed.” ~Screening
Recommendation (Mitigation):
• Government must establish an independent Group of Advisors on Technology in
Emergencies to oversee the development and testing of any prospective digital contact
tracing app.
“The effectiveness of a digital contact tracing app will be contingent on widespread public trust
and confidence, which must translate into broad adoption of the app. “
• In order to increase public trust and confidence, and guard against abuse and scope creep,
Government should advance primary legislation ….
58Case Study: Headgear
Mental fitness in male-dominated workplaces
USER RESEARCH WIREFRAMES MOCKUPS PROTOTYPE LAUNCH, ITERATE
Participatory Interaction design. Look & feel, User User testing Pilot, clinical trial
workshops Wellbeing design feedbackParticipatory process
• Co-design Workshops with 62 Firefighters & Transport
workers in urban, suburban and rural locations.
• 106 additional workers filled out a questionnaire
• Focus groups held with postal workers
• Ongoing feedback from organization representatives.
Peters D, Deady M, Glozier N, Harvey S, Calvo RA “Worker Preferences for a Mental Health App Within Male-Dominated Industries: Participatory Study” JMIR Ment
Health 2018;5(2):e30
M. Deady, D. Peters, H. Lang, R. Calvo, N. Glozier, H. Christensen, S. B. Harvey; Designing smartphone mental health applications for emergency service workers. Occup
Med (Lond) Volume 67, Issue 6, 1 August 2017, Pages 425–428
@Rafael_A_CalvoImpact: RCT results
2,271 participants – Largest RCT of a workplace mental health app
Cases of Depression (PHQ)Study 2: Youth Mental Health NLP, Social Media Classification & Conversational systems
Project: Moderator Assistant (版主助手)
Helping those who help others
Liu, Calvo, Davenport, Hickie “Moderator Assistant: helping those who help via online mental health support groups”.
Social Technologies for Health and Wellbeing’ workshop at OzChi 2013.Some results • The algorithm achieved 84% f-measure in identifying content that required a moderator response. • Between prelaunch and post-training periods, response ratios increased by 0.9, 4.4, and 10.5 percentage points for messages labelled as crisis, red, and green, respectively, but decreased by 5.0 percentage points for amber messages. • Response latency was significantly reduced (P
Global Initiatives
An enormous list
• AI Now Institute
• IEEE Ethically Aligned Design (IEEE Standards P7000, P7010, P7005 etc)
• Z-inspection: Towards a process to assess Ethical AI (Frankfurt Big Data Lab)
67Failures in responsibility
CannyAI VDR; Face Replacement as a Service or “Fake News as a service”
https://www.fxguide.com/quicktakes/cannyai-vdr-face-replacement-as-a-service/
https://www.youtube.com/watch?v=y_hthFKMxgw
68How can we prevent irresponsible design
and engineering from happening?
• Regulation,
• Technology Push &
• Market Pull
69The importance of organizational capabilities In addition: Firms need the organisational, technological and marketing capabilities to respond to regulatory pressures and customer demand related to environment with innovation whilst Others fail in the face of external pressures (regulation and customer demand). Demirel, P., & Kesidou, E. (2019). Sustainability-oriented capabilities for eco-innovation: Meeting the regulatory, technology, and market demands. Business Strategy and the Environment, 28(5), 847-857.
You can also read
