Does the world need another face filter? What we learned prototyping augmented reality in the classroom - MODE Summit
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Introduction
Augmented Reality: What is it? Who can use it? Should I make it?
Working in any emerging technology presents unique challenges in the design
Does the world need another face filter? classroom. Students must familiarize themselves with new communication
media and navigate access to new hardware and software. In addition, they need
What we learned prototyping augmented to develop awareness of the moral ramifications of their work. As educators,
Hodara and Rettig start with the basics—by asking the three questions above.
reality in the classroom. From these questions, they have developed a pedagogical framework called
Play, Make, Speculate.
While these are not new concepts in the design classroom, this paper presents
Martha Rettig Play, Make, and Speculate as a unified framework to address issues that arise
Massachusetts College of Art and Design, Boston MA, United States when teaching emerging technology. They attribute this pedagogy to their
experience as students at the Dynamic Media Institute (DMI), a graduate
Sofie Hodara design program that was founded by Jan Kubasiewicz in 2000 at Massachusetts
Massachusetts College of Art and Design, Boston MA + School of the College of Art and Design (MassArt). At DMI, through classes like Elements of
Museum of Fine Arts at Tufts University, Boston MA, United States Media and Design Seminar, students are asked to develop thesis topics that
explore digital media through freeform play, rapid making, and speculation. In
this paper, Hodara and Rettig draw on the works of Mitchell Resnik and Dunne
ABSTRACT Play, Make, Speculate is a pedagogical framework for the and Raby, and on their experiences teaching augmented reality (AR) in the
design classroom, developed by Hodara and Rettig. This paper demonstrates design classroom to demonstrate how Play, Make, and Speculate, is a useful
how freeform play, rapid making, and speculation can be used to address framework for teaching emergent media.
the visual and technical components of augmented reality, and challenges
students to address the ethical concerns that are critical to their work. These This paper is divided into three sections. Play, the first section, answers “what
include issues of who has access and the repercussions of adding new media is it?” and presents the classroom as a place where students can explore and
to our society. develop a personal connection to the new medium. Make responds to “who can
use it?”. In this section, the question highlights accessibility challenges when
Hodara and Rettig draw on original curricula and student work from the designing with emerging technologies and offers strategies for students to
following courses: Augmented and Virtual Reality for Designers & Storytellers, overcome these barriers by “just making.” The Speculate section asks “should
a graduate seminar at The Dynamic Media Institute at Massachusetts College I make it?” and reflects on the ethical and moral responsibilities students have
of Art and Design (Boston MA); Disobedient Design, an interdisciplinary as designers. The conclusion responds to the question: “does the world need
undergraduate elective in the Communication Design Department at another face filter?”.
Massachusetts College of Art and Design; and Augment My Type, a workshop
in Type@Cooper at The Cooper Union (New York, NY). Hodara and Rettig’s interest in emerging technologies began in 2012, when
they met in graduate school at the DMI. Five years later, as colleagues at
Keywords: augmented reality, pedagogy, prototyping, communication design, MassArt, they became curious about bringing AR and virtual reality (VR) into
emerging technology, ethics, accessibility, play, make, speculate the classroom and wanted students to leave school with strategies for working
with emerging technologies. Since 2017 they have designed and implemented
multiple courses in the communication design curricula at MassArt and School
of Museum of Fine Arts at Tufts University (SMFA) in Boston, MA, as well as at
workshops at Cooper Union in New York and the Interaction Design Education
Summit in Milan, Italy.
102 MODE 2021 Edited Conference Proceedings
What are emerging technologies? The Problem: What is AR?
Hodara and Rettig define “emerging technologies” as new communication Though students may be familiar with the term, many do not know the difference
media for which hardware and software workflows have yet to be standardized. between AR and VR. AR overlays digital content onto a physical environment,
Today, there are few platforms for prototyping AR—think the iPhone before usually through a screen, like glasses or smartphones. It blurs the lines between
Sketch and Figma. Today, Adobe’s AR project, Adobe Aero, is still in beta. Lens the digital and physical. In contrast, VR is a fully digital immersive environment,
Studio and SparkAR, from Snapchat and Facebook respectively, are the most often experienced through a headset and controllers with audio, visual, and
accessible platforms for designers but still have intimidating interfaces and haptic feedback (Milgram et al.). To add to the confusion, terms in industry are
require publishing to their respective platforms. Unity and UnReal Engine, constantly morphing, with the introduction of extended reality (XR) and mixed
gaming engines, require programming skills and high-powered computers reality (MR).
that many 2D designers don’t have.
Though AR contact lenses are not yet a thing, our mobile phones have made
Along with hardware and software, emerging technologies need new design it possible to integrate AR into our daily life: Google Maps’s LiveView feature
standards. Guidelines for AR designers are only beginning to be developed – overlays directions onto the street; Ikea and Wayfair have dedicated apps for
this is akin to the early days of iOS Human Centered Guidelines or Google previewing to-scale furniture in one’s home; and the New York Times has
Material Design. Though both Apple and Google have published AR guidelines, transformed the way they report, with a dedicated immersive section in their
research is just starting to emerge. At the end of 2020, Aurora Harley, Senior application. Beyond consumer use, companies like Augmedics xvision use AR
User Experience Specialist with Nielsen Norman Group, published UX to assist in spinal surgery. There is an industry of custom hardware and software
Guidelines for Augmented Reality Shopping Tools, which describes a remote for manufacturing, from companies like Skylight, Quinso, and PTC. AR, whether
study with 10 participants and stresses the importance of discoverability for used for entertainment or as a functional tool, is a promising emerging
this unfamiliar technology. There is also the work of Andrew Johnson, Product technology. Yet students struggle to come up with ideas when asked to design
Design Prototyper at Oculus. He has conducted a series of rigorous typography in an unfamiliar medium. This is where Play comes in.
experiments published on his blog (aetherpoint.com). The most recent paper,
Approaching Spatially Adaptive Type (September 2019), draws on work he’s Pedagogy: Play In the Classroom
done on legibility and responsive typography in AR with top type designers like Play is already a part of the designer’s process, especially with new materials
DJR and CJ Dunn. This nascent field is ripe for discovery and experimentation. and foreign deliverable forms. For example: take the shiny new technology out of
It also poses challenges for the classroom specific to working with emerging the picture and imagine designing a billboard for the first time. There is a lot at
technologies similar to those that design educators experienced during the stake. Any little mistake could become a public, pricey, and dangerous mishap
evolution of responsive web design and mobile app design. AR is neither the (the wrong type size or color combination could cause a traffic accident!). What
first nor the last emerging technology that designers will encounter. does a designer do? Get in the car, go for a drive, and notice. What makes a
billboard captivating in rush hour traffic? Or at 60mph? How does it change
Play when it’s lit up at night? Which ones are barely noticeable? Which ones bring
When students are asked to design a poster, they get excited. They know what joy, and why?
a poster is. There is probably one on the wall next to them. When asked to
design an augmented experience, their minds go blank. They panic. This is Designers begin by observing. No matter the medium, freeform no-expectations
foreign terrain. observation is vital to creativity. However, this type of observation can be diffi-
cult for achievement-oriented students. And simply telling a student to observe
A common challenge when introducing new technologies in the classroom is doesn’t always work. Instead, allotting class time for exploration, without the
students’ lack of experience. Using classroom time for unstructured play gives pressure of doing, encourages students to observe. That’s why it is important to
them the chance to become familiar with a new medium. build Play into the classroom.
Does the world need another face filter? 103
What is Play? The Oxford English Dictionary defines play as engaging “in an “I really liked the design of this app, that was what initially drew me to download
activity for enjoyment and recreation rather than a serious or practical it. The color palettes are very pleasing, and the low poly style of the blocks is
purpose.” Play as pedagogy is used by many educators. Most notably, Mitchell nice to look at.” Or student Rose Flynn downloaded and tested Tattoo Hunter:
Resnik, draws on the work of early education specialists like Maria Montessori “It piqued my interest because I don’t [sic] tattoos out of fear of them not looking
and Friedrich Froebel in his book Lifelong Kindergarten to argue that “the rest good on me, so I thought this would give me a better idea of what I would look
of school (indeed, the rest of life) should become more like Kindergarten.” He like with tattoos!”
views play as integral to the kindergarten experience, allowing children to be “...
constantly exploring, experimenting, and testing the boundaries—and develop- Despite the informal nature of the assignment, the students were discerning. Of
ing as creative thinkers.” Hodara and Rettig use Play in the classroom to Angry Birds AR, Kathyleen Martin wrote, “To be honest... I assumed that angry
validate the process of exploration and emphasize the value of experimentation. birds was just trying to cash in using newer technology. However, I was delight-
They tell their students: Dive in. Have no expectations. What do you see? fully surprised! The AR levels were constructed to be explored and find pockets
What do you feel? You can’t fail. Play. to throw your birds into! It was fun getting up close to the characters and seeing
the animations they do!”
To introduce AR in the classroom, Rettig and Hodara start by asking “what is it?”
and answer with play. This helps students feel comfortable with the medium and They made their own observations about the affordances of AR. Saleenah
expand their perception of it. For some, that means learning that a face filter is St. Louis wrote about AR Dragon: “It could be a good way to [sic] people to
AR; for others it’s about envisioning the operating room of the future. decompress, or fun for kids who may be allergic or not allowed to have pets.”
And they were critical, as Nandini Srinivasan describes Tattoo Hunter, “I’m
Student Examples actually now realizing that you can basically do this with stickers....so this
Disobedient Design, Fall 2020 might not need to be AR at all.” Play led to insightful reflection about what
In Disobedient Design, a class Hodara and Rettig co-wrote and co-taught, AR does well.
students worked on a series of shorter projects in response to social and
political issues that they cared about. With each new project, students were
asked to use a wide range of making materials: lino and ink, sidewalk chalk,
cross stitch, cardboard, etc. The course ended with an AR project. Moving from
tactile making methods to the unfamiliar territory of AR was frustrating for
students. In response to their resistance, we assigned students a one-week
warm-up activity:
Play. Download, install, and choose an AR application. Document your
experience with a screenshot or short screen recording (~10–15 seconds).
Post to the course website with a ~100 word, open-ended reflection.
As a starting point, Hodara and Rettig provided a list of vetted AR experiences
(high-end, editorial, immersive content). We also encouraged them to pick their
own applications.
Student responses to the warm-up were noteworthy because many didn’t
use the vetted experiences provided, they found their own. This was a Figure 1: (l to r) Play screenshots from students Kathyleen Martin, Saleenah St. Louis, Nandini
learning moment for us as educators: our students didn’t want to visualize Srinivasan, and Rose Flynn. Source: Sofie Hodara and Martha Rettig, 2020.
the pollution in their living room, as seen in a beautiful NYTimes immersive
experience (Popovich). They wanted to play games, like Angry Birds AR and
PokemonGO, and put animals in their rooms with applications like AR Dragon
or Eugene’s Pets. For example student Anna Shobe picked Stack AR because
104 MODE 2021 Edited Conference Proceedings
Takeaways
In any emerging technology, lack of familiarity is a challenge that can be
addressed through expectation-free Play. Play sets aside time for students to,
as Resnik writes, explore, experiment, and test the boundaries of this subject.
In doing so, they learn to trust their responses to the medium.
Hodara and Rettig discovered that their curriculum content, from class discus-
sions to critiques, was driven by students’ personal experiences and insights
as opposed to those of the instructors. Because students picked their own AR
applications to explore, it was their examples, not those of the instructors, that
answered the questions: “What is AR?”. And when students drive course content
in this way, content can easily adapt from one semester to another. Whether it’s Figure 2: Hodara and Rettig in Magic Leap and AR for Everyone. Source: Martha Rettig, 2019
the technology or student interests that evolve, content remains relevant.
So, is PokemonGO fun because it’s AR? Maybe. It’s also fun because, as one to implement AR on the devices that 90% of Americans have in their pockets
student from Disobedient Design, Jameson Noonan, put it, “it lets every kid (O'Dea), that does not mean it is easy for non-programmers to create in this
play out their dream of having a pokemon in their room and you can walk around space. Designing in AR is a new frontier where hardware is expensive, software
it, take pictures with it, play with it, and give it treats that all trigger visual emerges as quickly as it disappears, and the skills needed to navigate 3D space
responses.” Through Play, answering the question “what is AR?” expands into are complex.
answering “what is AR good for?”
These problems are exacerbated when teaching emerging technology at
Make a public college like MassArt, where budgets are tight and rarely allow for
Play introduces students to AR and paves the way for them to create their new hardware or software. In the case of AR, the two most popular software
own experiences. However, familiarity is only the first challenge when teaching platforms for creators, Unity and UnReal Engine, require high-powered
emergent technologies. This section addresses accessibility issues, specifically computers, which the school can’t provide and most students don’t have.
expensive hardware and complex software. In order to overcome these Additionally, design students don’t have programming or 3D skills.
obstacles, Hodara and Rettig move away from learning complex, industry-
standard software and offer students a variety of open-source tools so they These accessibility issues are illustrated by a VR component Rettig introduced
can “just make a thing.” in her MassArt elective Programming for Designers (Spring 2018). Even before
the course began, there were challenges. Due to the high price point of VR
Today, the promise of AR conjures images of fancy eyeglasses and headsets that headsets, she had to use affordable Google Cardboards, which many don’t
can transform our living rooms into magical games and digital workspaces. This consider true VR. To get started, students needed to assemble the headsets.
high-end AR is a rapidly expanding industry. In March 2021, The Verge reported This was comical: straps were taped inside out, lenses installed backwards
that 20% of Facebook employees were working on mixed reality devices, a figure (there may have been tears). Then students needed to find and download
that quadrupled from 5% in 2017 (Byford). However tempting it is to stare myo- specific applications on their smartphones. This was painful: many applications
pically into a bright future, this excitement doesn’t reflect the way most people weren’t available for Android or older iPhones, phones didn’t have enough stor-
experience AR. Headsets are still reserved primarily for industry settings. Most age for the large applications, and slow wifi made downloads nearly impossible.
users of AR experience it through their smartphones. By the end of class, batteries were drained, and many students were unable to
experience VR. As they began to create their VR experiences in Unity, the pain
Though mobile AR has been around for awhile, it rapidly grew in popularity after didn’t end: they struggled moving from 2D design into a 3D space, and their
2018, when Apple and Google released AR-specific SDKs (software develop- computers could barely handle the intensive platform. So Rettig had to get
ment kits) that enabled programmers to access smartphone hardware—like scrappy. She introduced an open-source, browser-based VR platform, A-Frame,
cameras and accelerometers— necessary for AR experiences. Though it is easy which allowed every student to build a 360 VR experience.
Does the world need another face filter? 105
If competent undergraduates and digital natives could barely do this, who
could? Classroom discussions ensued about “who can use it?” and audience
accessibility and eventually morphed into “who can make it?”. Bringing VR into
the classroom taught Rettig to center courses around who has access to the
technology. So how do we surpass barriers to making in AR? By providing a
variety of tools and asking students to “just make a thing.”
Pedagogy: Make in the Classroom
What is Make? In Hodara and Rettig’s classroom, Make asks students to create
rapidly. They often say, “just make a thing.” Make needs to be timeboxed and
[3]
not focus on a single software platform—it should emphasize the “thing,” as
opposed to how the “thing” is made.
In Fall 2019, Hodara and Fish McGill co-taught Human-Centered Design
Research in the Communication Department at MassArt. Knowing that soph-
omore students were nervous about making a brief video for their final project,
McGill and Hodara could have demo-ed Adobe Premiere or iMovie. Instead they
threw a birthday party, complete with decorations and a cake. Students were
given different roles: party-guest or videographer. Videographers were assigned
specific shots: blowing out candles or cutting cake. Including cake, the party
took an hour, and for the remainder of class, students made a video document-
ing the party.
There were questions about software, but not many. The biggest technical
hurdle was sharing video footage over poor wifi. Because the workshop
demanded students to “make” rapidly, they were forced to rely on their limited
knowledge of video-making. Instead of trying to learn new software, they [4]
refined their storytelling skills. As a byproduct, they learned how to create
a video. Though video is not an emerging technology. But this workshop high-
lights the key components of “make”: to work rapidly, focus on concept not
software, and have fun (and cake). To summarize, Make as a methodology,
overcomes issues of access to hardware, software, and skills by taking the
emphasis off learning software, and puts it on “making a thing.”
Student Examples
Augmented Cities, Fall 2019 & Fall 2020—Martha Rettig
Augmented Cities is a senior level course in the Communication Design depart-
ment at MassArt. Despite expertise across many platforms (AfterEffects, Figma,
[5]
InDesign, etc.), these advanced and talented makers doubt their abilities when
faced with the prospect of making an augmented experience. To overcome
this paralysis, in a mini in-class assignment students are asked to make “AR Figure 3: Students in Programming for Designers assembling headsets. Source: Martha Rettig, 2018
Playground/Demos”: Figure 4: Still from the Birthday Party Workshop. Source: Sofie Hodara, 2019
Figure 5: Pictured L to R. AR Platform Demo by Alyssa Kirk, Emily St. Laurent, and Aidan Wright.
Source: Martha Rettig, 2020
106 MODE 2021 Edited Conference Proceedings
Pick a free AR studio from the following list: LensStudio, SparkAR, Aero, Takeaways
ZappAR, Wiarframe, theRoar/ In response to accessibility challenges, Hodara and Rettig teach students
Make a thing in 1 hour. about the importance of figuring out how to “just make a thing.” Making in both
Share to #homework. classes helped students build confidence in their technical abilities and learn
how to teach themselves new skills.
Not only did students have fun as they built forests and added poetry lyrics in
their rooms, they became experts in various platforms. Peer to peer learning In addition, when students are given the freedom to choose which platform they
took place as they shared their knowledge with others. This removed the burden want to explore, as they were in Augmenting Cities, they become class experts.
from the instructor of having to know everything about ever-changing tools. This builds confidence and creates a more lateral classroom, without a single
authority holding the firehose of knowledge. Further, it allows for a flexible
Creative Activism For Emerging Technologies, Spring 2021—Sofie Hodara curriculum, one that can evolve as platforms and workflows do.
The SMFA doesn’t have majors or sequential tracks of study. Therefore, each
student's expertise is unique and their conceptual and technical skills vary. In Emerging Technologies for Creative Activism, students were asked to
In this class, some students had already worked in Unity, while others didn’t approach AR by making a face filter, which is easy in Lens Studio. By starting
know how to use Photoshop. To prepare for working in AR and in new software, in the familiar territory of face filters, students gained confidence in their ability
Hodara designed an in-class face filter activity: Glitch Face Filter Demo. as creators and made something they could share with their friends.
Before introducing AR, students read excerpts from Glitch Feminism: A
Manifesto (2020), by Legacy Russell. In her book, Russell dismantles the binary
divide between the digital and physical worlds by looking at contemporary new
media artists interested in race, gender, technology, and identity. The reading
prepared students to critically consider the implications of blurring the digital
and physical, as we do with AR.
In the activity, the majority of class time was spent using a text editor to glitch
images of students choosing. However, in the last twenty minutes of class, stu-
dents opened Snapchat’s free AR studio, Lens Studio, for the first time. In a few
simple steps, they completed their face filters, without any technical difficulties.
Instead of spending class time learning software, students focused on concep-
tual development. Anh-Tuan Le was interested in the mutability of identity. Using
his own glitching process, he bitmapped a series of portraits to wear over Figure 6: Pictured L to R: Glitch face filters by Anh-Tuan Le, Maya Gangan, Emrys Schweber, and
his face. Maya Gangan glitched a map of redlining in Boston. By putting the Chacha Cha. Source: Sofie Hodara, 2021
decimated map on her face, she articulated her contemporary presence within
a complex history community and racism. Emrys Schweber used a glitched
camo image, and Chacha Cha transformed her image so completely that her
face resembled an obliterated screen. By limiting students to one quick action
on one platform and focusing on the concept, they successfully “made”.
Does the world need another face filter? 107
Speculate There is no doubt that misinformation is dangerous. But does Facebook have
This section explores the ethical ramifications of working with emerging technol- the right to govern what one overlays on their face, in the privacy of their own
ogies. In addition to Play and Make, Hodara and Rettig ask students to home? Facebook, like Twitter booting 45, has the freedom to decide what gets
Speculate about what the world could be, in response to the question: “should published on its platform, but who governs these corporations to ensure
I make it?”. This section explores the ethical ramifications of working with free speech and equal rights? These questions don’t have easy answers. By
emerging technologies. In addition to Play and Make, Hodara and Rettig ask integrating examples and assignments inspired by speculative design and
students to Speculate about the question “should I make it,” and if they did, future studies disciplines into their classrooms, Hodara and Rettig present
what the world could be. these questions to their students and ask them to Speculate—to consider
the implications of what they make and add to the world, for better or worse.
The Problem: Should I make it? Does the world need another face filter?
In 2017, Snapchat released its AR platform by installing geotagged Jeff Koons Pedagogy: Speculate in the Classroom
sculptures around the world. The sculptures were available at specific locations, Though exuberant digital natives, contemporary students are acutely aware of
like Central Park. This was met with excitement and backlash. While Snapchat the effect social media and other platforms have on their mental health. They
devotees posted selfies with Koons’ Balloon Dog, artist and designer Sebastian often discuss deleting or taking social media “detoxes”. However, there is a
Errazuriz (and his team from CrossLab) were questioning the corporation’s right disconnect: When asked to critique a platform, like Instagram, they make notes
to digital public space. Within 24 hours, they created an identical 3D AR Balloon on typography, color, and layout – they do not critique how the technology is
Dog, geotagged it with the same coordinates, and vandalized it with graffiti. As being used. It isn’t until students learn about dark UX patterns (Brignull) and
Errazuriz explains on his instagram: “websites designed to be addictive” (Schwär), their perceptions shift. They start
to question the role of the designer.
For a company to have the freedom to GPS tag whatever they want is an
enormous luxury that we should not be giving out for free. The virtual As Schwär writes in “Instagram and Facebook are intentionally conditioning you
public space belongs to us. We should charge them rent. to treat your phone like a drug”:
Though the project received abundant press, it’s impossible to tell how many Several ex-employees of Apple, Google and Facebook have warned that
people actually saw it. Errazuriz and Crosslab had to release their own applica- large tech companies deliberately design apps to be addictive. The logic
tion, ARNYC, to install the dog in Central Park (they submitted to Snapchat but is simple: the more time you spend on the app, the more profit it
got no response). Along with a host of projects and artists (Hyperreality, generates.
Manifest.AR, and Nancy BakerCahill), this piece raises important questions
about ownership and governance in digital space pertinent to AR. Today, it is crucial that students shift their ideas from how something looks to
how it works. In the classroom, educators can encourage this by asking stu-
Who has the right to create in digital space? Who owns and governs it? Is that dents to reimagine, or speculate about, their relationships to emerging technol-
space public or private? What about the digital space in my living room? These ogies, for better or worse.
questions aren’t easy. Take for example, what happened on Facebook in March
2020 regarding a wave of COVID-related AR filters with false information. In What is Speculate? In Speculative Everything, Anthuny Dunne and Fiona Raby
response Facebook took action. On March 13, they released this statement: define speculative design as design that is “free from market pressures and
available to explore ideas and issues… We are more interested in designing for
Across Facebook we’re taking steps to limit misinformation and how things could be… ” (12). Borrowing from this, in our classroom Speculate
harmful content… means: Even if you can’t build it, imagine “how things could be.” Imagine the bad
First, we’ve removed previously-published effects and are rejecting all and the good.
new effects…Secondly, we won’t allow people to search for COVID-19
related AR effects on Instagram...
108 MODE 2021 Edited Conference Proceedings
Further, Dunne and Raby position speculative design as a form of critique that Does The World Need Another Face Filter?
isn’t just critical, but reflective and hopeful: Classrooms are about asking questions. In the design classroom, answers come
in the form of doing. Hodara and Rettig take their three questions—what is it?,
Design as critique can do many things—pose questions, encourage who can use it?, and should I make it?—and turn them into verbs: Play, Make,
thought, expose assumptions, provoke action, spark debate, raise and Speculate. Whether it’s a poster, AR, or some yet-to-be invented medium,
awareness, offer new perspectives, and inspire. (43). these verbs inspire a classroom where students can envision the futures they
want to see.
To demonstrate this in the classroom, we show examples from artistic activism,
afrofuturism, and from the popular TV show Black Mirror. What happens when students make another face filter? Yes, it is entertain-
ing to add dog ears to one’s face. Yes, it is fun to test out a new hair color. But
Student Examples what is the potential power of a face filter? Can it be used as a form of critique
Write Your Own Black Mirror Episode, Class Assignment or resistance? And does the world need it? No. But students should make face
To introduce Speculate, students are assigned to watch specific episodes of filters anyway. By Playing with them, they learn to be critical of the medium. By
Black Mirror. Each episode examines the unintended consequences of Making them, they learn about the limitations and opportunities of the medium,
humanity’s growing technological dependence. Though the episodes are usually and they develop confidence in their abilities. And without those lessons, they
positioned in dystopian futures, they reflect the interpersonal or social trends wouldn’t be able to Speculate.
of today. In response, students are assigned to write their own episodes:
Besides, students make really interesting face filters! Have you ever seen an
Using basic storytelling structures and tools, demonstrate either the artist-statement face filter? Or a face filter serve as an ignorant-people-check?
positive or negative repercussions of technology through a main action,
with a clearly defined beginning, middle, and end.
Each time this project runs, the student responses demonstrate their innate
ability of being future-orientated in the design field: They offer insightful critique
of the present and hope for a better future. The examples below explore scenari-
os where the health of individuals and communities are prioritized.
Takeaways
If you can’t build it, tell it. When students are asked to Speculate, they dream big.
As a result, many projects can’t be built. Students need to rely on storytelling
skills, which is both a hard skill, with tools and strategies that can be learned,
and a method that allows them to engage in social critique. This way students
can focus on presenting their ideas.
From making a thing to making a thing that matters. Emerging media is, and
will be, hard to build for design students. They are not engineers or developers.
Speculation allows them to see design as a form of critique. By imagining future
worlds, they become reflective of the world around them and the role they
play in it. They reframe and expand what it means, for them, to be a designer.
Speculation allows them to move from making a thing to making a thing
that matters.
Does the world need another face filter? 109
[7]
[9]
Figure 7: The SEED, Tomasz Vav’s Write Your Own Black Mirror Episode. Source: Tomasz Vav, 2021
Figure 8: Selected images from Deep Breathing, Tatiania Baughman’s Write Your Own Black Mirror Episode
Figure 9: The Range, Trevor Collin’s Write Your Own Black Mirror Episode. Source: Trevor Collins, 2020
Figure 10: Pictured L to R: Disobedient Design face filters by Runming Dai, Nandini Srinivasan,
and Nick DiPaola. Source: Sofie Hodara and Martha Rettig, 2020
Figure 11: (L to R) Artist Statement Face Filter by Nori Needle, Revelation by Maya Gangan, [8]
Live Balanced by Isabella Joeng. Source: Sofie Hodara, 2021
110 MODE 2021 Edited Conference Proceedings
[10]
[11]
Teenage Engineering’s OP-Z and its Topology of Sound and Visuals 111Ikea UK. “Say Hej to IKEA Place.” YouTube, YouTube, 12 Sept. 2017, www.youtube.
WORKS CITED
com/watch?v=r0ViFTEb8aQ.
Apple Developer. “Human Interface Guidelines: Augmented Reality.” developer.
Johnson, Andrew. “Approaching Spatially Adaptive Type.” Aetherpoint, 9 Sept.
apple.com/design/human-interface-guidelines/ios/system-capabilities/aug-
2019, www.aetherpoint.com/blogpost/approaching-spatially-adaptive-type/.
mented-reality.
Melnick, Kyle, and Kyle Melnick. How an Outbreak of Coronavirus Ar Filters Is
Boland, Mike. “2021 Predictions: Mobile AR Exceeds 800 Million Users.” AR
Contaminating Instagram with Misinformation. 15 Mar. 2020, vrscout.com/
Insider, 11 Jan. 2021, arinsider.co/2020/12/16/2021-predictions-mobile-ar-ex-
news/coronavirus-instagram-ar-filters-misinformation/.
ceeds-800-million-users/.
Milgram, Paul, et al. “Augmented Reality: A Class of Displays on the Reality-Vir-
Brignull, Harry. “Dark Patterns: inside the Interfaces Designed to Trick You.” The
tuality Continuum.” Telemanipulator and Telepresence Technologies, SPIE
Verge, 29 Aug. 2013, www.theverge.com/2013/8/29/4640308/dark-patterns-
VOL. 2351, 1994, pp. 181–191.
inside-the-interfaces-designed-to-trick-you.
McGill, Fish and Sofie Hodara. “Birthday Party Workshop.” Human-Centered De-
BusinessWire. “Augmedics Announces FDA 510K Clearance and U S Launch
sign Research. Massachusetts College of Art and Design. Fall 2019. In-class
of Xvision™.” YouTube, BusinessWire, 23 Dec. 2019, www.youtube.com/
activity.
watch?v=zQZZozY3XVo.
“The New York Times Special Edition.” Steve Lambert, 15 June 2020, visitsteve.
Byford, Sam. Almost a Fifth of Facebook Employees Are Now Working on VR
com/made/the-ny-times-special-edition/.
and AR: Report. 12 Mar. 2021, www.theverge.com/2021/3/12/22326875/face-
book-reality-labs-ar-vr-headcount-report. O’Dea, S.. “Cell Phone Sales Worldwide 2007-2021.” Statista, 11 Mar. 2021, www.
statista.com/statistics/263437/global-smartphone-sales-to-end-users-
Cantave, Glenn, et al. “Our Work.” Movers and Shakers NYC, www.moversand-
since-2007/.
shakersnyc.com/our-work.
PLAY: Definition of Play by Oxford Dictionary on LEXICO.COM Also Meaning of
Codrea-rado, Anna. “Virtual Vandalism: Jeff Koons's 'Balloon Dog' Is Graffi-
Play. www.lexico.com/en/definition/play.
ti-Bombed.” The New York Times, The New York Times, 10 Oct. 2017, www.
nytimes.com/2017/10/10/arts/design/augmented-reality-jeff-koons.html. Poetker, Bridget. “A Brief History of Augmented Reality (+Future Trends & Im-
pact).” G2 Learning Hub, learn.g2.com/history-of-augmented-reality.
Dunne, Anthony, and Fiona Raby. Speculative Everything. The MIT Press, 2013.
Popovich, Nadja, et al. “See How the World’s Most Polluted Air Compares With
Errazuriz, Sebastian. “On corporate Geo-Tagging and the upcoming Augmented
Your City’s.” The New York Times, 2 Dec. 2019, www.nytimes.com/interac-
Reality.” Instagram, October 6, 2017, www.instagram.com/p/BZ7Bng4jk-
tive/2019/12/02/climate/air-pollution-compare-ar-ul.html.
0C/?utm_source=ig_web_copy_link.
Quinso Solutions. Augmented Reality Warehouse - Extended Warehouse Man-
Google. “See the Way with Live View in Google Maps.” Youtube, Google Maps, 12
agement. 16 June 2016, www.youtube.com/watch?v=5E6qambiCo0. Ac-
Aug. 2020, www.youtube.com/watch?v=ip_SV70WFO0.
cessed 26 Mar. 2021.
Google Developers. “Augmented Reality Design Guidelines|ARCore|Google
Rettig, Martha, and Sofie Hodara. Augmented Reality for the Classroom: A Peda-
Developers.” https://developers.google.com/ar/design.
gogy, Interaction Design Education Summit, 9 Mar. 2020, medium.com/ixda/
Harley, Aurora.“UX Guidelines for Augmented-Reality Shopping Tools.” Nielsen augmented-reality-for-the-classroom-a-pedagogy-f998067f7354.
Norman Group, 6 Dec. 2020, www.nngroup.com/articles/augmented-reali-
Rettig, Martha and Sofie Hodara “Looking into the Black Mirror.” Augmented
ty-ecommerce-guidelines/.
and Virtual Reality for Designers and Storytellers. Massachusetts College of
Hodara, Sofie. “Glitch Face Filter.” Creative Activism for Emerging Technologies. Art and Design. Summer 2018, Fall 2019, Spring 2021. Class project. https://
School of The Museum of Visual Arts. Spring 2021. In-class activity. https:// docs.google.com/document/d/1iDiwvCRxyRL6H7hiUBlw4yT9EYN4aU-
docs.google.com/document/d/1pDAIu5X3OMUYKCvZEud6kqujZp-ElHGDt- V11sZs6Lih81s/edit#
mPGFEStu34/edit#heading=h.pcdtwuflexm8.
Rettig, Martha and Sofie Hodara. “Play.” Disobedient Design. Massachusetts
Hyundai. “Head Up Display Explained | Palisade | Hyundai.” Youtube, HyundaiU- College of Art and Design. Fall 2020. In-class activity.
SA, 11 July 2019, www.youtube.com/watch?v=A5lSkierJsU&t=38s.
Rettig, Martha. “AR Playground/Demos: Make a Thing + Share”.” Augmenting
Cities. Massachusetts College of Art and Design. Fall 2019, Fall 2020. In-class
activity.
112 MODE 2021 Edited Conference ProceedingsRettig, Martha. “360 Journalism.” Programming for Designers. Massachusetts
Software Applications
College of Art and Design. Spring 2018. Assignment.
A-frame 1.1.0. aframe.io.
Schwär, Hannah. “Instagram and Facebook Are Intentionally Conditioning
You to Treat Your Phone like a Drug.” Business Insider, Business Insider, 5 Adobe Aero. www.adobe.com/products/aero.html.
Apr. 2020, www.businessinsider.com/facebook-has-been-deliberately-de- Lens Studio. lensstudio.snapchat.com.
signed-to-mimic-addictive-painkillers-2018-12. Roar. theroar.io.
“Spark AR Community : # **COVID-19 Spark AR Update**.” Facebook, Spark AR Studio. sparkar.facebook.com/ar-studio.
www.alpha.facebook.com/groups/SparkARcommunity/perma-
link/848237828921594/. Wiarframe. www.wiarframe.com.
“Upskill and Boeing Use Skylight to Reinvent Wire Harness Assembly.” Upskill, Zappar: Augmented, Virtual & Mixed Reality Solutions. www.zappar.com.
YouTube, 15 Jan. 2017, www.youtube.com/watch?v=qTblKJjTadQ.
Vailshery, Lionel Sujay. AR/VR Headset Shipments WORLDWIDE 2020-2025. 16
Mar. 2021, www.statista.com/statistics/653390/worldwide-virtual-and-aug-
mented-reality-headset-shipments/.
AUTHOR(S) BIOGRAPHYS
Martha Rettig is a designer, experimenter, and immersive artist whose work Sofie Hodara is a Boston-based multimedia artist and educator. Often working
focuses on merging traditional mediums with emerging technologies. Her collaboratively, she explores the intersection between traditional and emerging
experience crosses many disciplines, including design concept, visual design, media in order to create beautiful, non-utilitarian experiences with technology.
interactive design, interface design, data visualization, experience design, and The results range in form from paper weavings, to letterpress prints, to interac-
creative direction. She co-founded an interactive design agency, Cykod, in 2 tive installations, and to mixed reality. Hodara has exhibited her work across the
006 and helped build digital solutions for over two hundred companies. Martha country at spaces including Icebox Project Space in Philadelphia PA, UC San
currently is an Associate Professor at Massachusetts College of Art + Design Diego’s Calit2 Theater, and the Bromfield Gallery in Boston MA. Her work
teaching in the undergraduate Communication Design department and the has been featured in the Boston Globe, Fresh Paint Magazine, Made in Mind
co-director of the college's Dynamic Media Institute MFA program. For more Magazine, and the Journal of the New Media Caucus. She has presented on
information: martha.rettig.org augmented reality at conferences internationally, including the AR in Action
Leadership Summit at the MIT Media Lab and the IXDA Education Summit in
Milan, Italy. She has taught undergraduate and graduate courses and work-
shops in design, printmaking, typography, augmented and virtual reality, and
creative activism at Massachusetts College of Art and Design, SMFA at Tufts
University, Emmanuel College, University of Massachusetts Boston, and The
Cooper Union for the Advancement of Science and Art. For more Information:
https://sofie.space.
Does the world need another face filter? 113You can also read
