User Involvement, Social Media, and Service Evolution: The Case of Habbo
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
User Involvement, Social Media, and Service Evolution:
The Case of Habbo
Mikael Johnson
National Consumer Research Centre
mikael.johnson@ncrc.fi
Abstract the service, and after the research project the author
Understanding user needs and involving users are has informally followed the case.
key success factors for service development. However, In this article, the focus is not on the minute details
it is unclear how to apply old wisdoms from traditional of a particular user involvement method, but more on a
product development settings in a social media context. strategic level. During ten years, Sulake developers
This article analyses user involvement practices in a have explored various user involvement methods
longitudinal case study of the virtual world and social ranging from informal observation, focus groups,
networking service called Habbo. The focus is on how market surveys, and new social media possibilities.
particular social media aspects, such as persistent The aim of the case study was to answer the question
user-created content and developers’ easy access to of what practices social media developers have
online user action, shaped user involvement and how it regarding user involvement. In this article, we focus on
evolved over time. In line with previous research, the two aspects in more detail. First, how the social media
findings indicate that user involvement becomes more context influences the possibilities for user involve-
formal as the user community grows and the service- ment, considering especially persistent content created
market combination matures. In addition, user involve- by users and developers’ easy access to online user
ment regarding social media appears to be an issue of
action. Second, how user involvement practices change
timing, trans-nationality, and governance. The results
as the service evolves, with particular interest in the
suggest that the usability practitioner’s ‘toolbox’ needs
tuning regarding social media. increased complexity, signified by the increasing
numbers of both users and developers.
1. Introduction 2. User Involvement Approaches
Involving users in service development is a key There are many arguments for user involvement:
strategy to build better services and democratize the reducing risks of failure in technological innovations
innovation process [1-3]. While there are mixed results [1], democratizing the innovation process [2], building
on the cost-effectiveness of user involvement [4], there better products, etc. Despite a consensus that under-
is an abundance of methods available to bridge the gap standing user needs is a key factor in commercial
between developers and users [5-9]. Many user success [3], this is not easily translated into action in a
involvement methods have been created and applied in particular innovation context. There is an abundance of
contexts like commercial development for offices, methods to solve the ‘sticky information’ problem:
government health care, but few consider the emerging information about users’ needs and developers’ capabi-
social media. Is the standard textbook advice ‘study lities is highly contextual, tacit and difficult to transfer
first, design second’ applicable in social media [2]. However, which user involvement approach is
development, or is it the other way around [10]? applicable in which particular design situation is still
This article gives an overview of user involvement an open question.
practices from a longitudinal case study: the virtual In the same way as using a particular research
world and social networking service called Habbo that method implies a particular role for a researcher, any
was launched in 2000. The Habbo producer, Sulake user involvement approach imply a particular user-
Corporation, was involved in a research project in developer relationship. A researcher can be anything
2003-2006, during which the author followed the user between a detached and uninvolved observer to a fully
involvement approaches used. In retrospective inter- engaged participant in a group or organisation under
views developers shed light on the first few years of investigation. Developers’ information about users
978-0-7695-3869-3/10 $26.00 © 2010 IEEE 1Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
Table 1: Pros and cons of user involvement methods [12].
Approach to Pros Cons
learning about
end-users
Surveys and Systematic approach to data collection May not always feed into programme design
interviews Surveys provide the possibility to poll Surveys may be designed to confirm existing
representative samples preconceptions, may fail to bring up new insights
Conducting good research may be expensive and
require specialized skills
Prior research, Sound theoretical base can guide observations Strong commitment to prior findings or theories may
particular theoretical and help to make sense of energy-related lead to overlooking contextual particularities
perspectives behaviour Overly theoretical background can lead to complex
and confusing designs
Experience from prior Sound experience creates confidence and ‘Competence trap’: overconfidence and failure to
projects and similar practical skills/solutions that are difficult to learn new skills in new contexts
examples codify
User-driven project End-users know about their needs and End-users may not be fully aware of their behaviour
(or pilot project) circumstances and can contribute to context- and all the factors underlying it
tailored and user-friendly designs ‘Upscaling’ from small user-driven pilots to broader
End-users are motivated and engaged from the group of end-users can be difficult
start, thus ‘less work’ is left for the programme
manager.
Familiarity and Informal interactions allow for a rich exchange It can take a lot of time and commitment to build up
informal interaction of information (including non-verbal the level of familiarity needed to execute a
with the target group communication) successful programme
Familiarity creates trust and mutual confidence Contacts may be biased: some end-users are more
familiar than others.
originates in similar relations: statistics of user standing of the culture and practices of users [12-14].
practices can easily be detached from a particular However, while the initiatives recognise the complex-
situation, aggregated and analysed from a distance, ity and diversity of user settings, they seem to have had
where as participant observation of situated use implies modest influence over system design overall [15].
that actual developers and users meet. In the tradition Stewart & Williams [16] warn against a pitfall they call
of designing for usability, developers refer to these the design fallacy: the assumption that from the diver-
situations as usability evaluation with or without users, sity of user needs follows that extensive knowledge
and methods have emerged that are tuned to diverse about user contexts must be built in the product. In
design situations [11,5,6]. Table 1 lists pros and cons contrast, they report that generic products that can be
of particular approaches to user involvement, taken customised and configured seem to have had more
from experiences in energy demand-side programmes. impact on system design than user-centred design [15].
User involvement methods for established markets There is no consensus on why potential initiatives
differ from the methods that are developed for new such as user-centred design have had little impact.
markets. Leonard-Barton [3] found that traditional Recent debates suggest that standard methods such as
market research methods (latent needs analysis, lead ethnographic field studies or usability evaluations can
users, surveys, focus groups, mall studies) fit well with do more harm than good if not applied carefully
an established market and a high maturity of [17,18]. This reflective debate can be interpreted as a
technology design. For new markets, more used were sign of maturity in the human-computer interaction
methods like market experimentation, future scenarios, field as it highlights specific problems that must be
and trends exploration. In the grey area between these addressed when mixing particular research disciplines
typical situations, Leonard-Burton argued for less with product development. Another suggestion is that
traditional market research methods under the rubric of the boundary conditions of product development
empathic design: studying actual observed customer projects have not been addressed thoroughly. Svanæs
behaviour, direct interaction between developers and and Gulliksen [19] suggest that the design context
users, and considering existing technological capabi- needs greater attention, that is, the internal structure of
lities. the developer and the client organizations, contractual
Empathic, participatory, and other user-centred and tender issues, software engineering tools, and
design approaches emerged because user requirements stakeholder agendas and relations.
engineering techniques at the time lacked an under-
2Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
2.1. Social Media the time of writing, there are Habbo hotels in more
than thirty countries, and 13 million players visit
New online web services often referred to as digital Habbo each month [23]. Instead of an entrance or a
media, web 2.0, and social media have changed the monthly fee, the profit model is based on micro-
possibilities for user involvement. On the one hand, payments in the hotel. Virtual furniture, mini-games,
people discuss particular products and brands in online and membership in the Habbo club are bought with
discussion forums and communities. Product deve- Habbo credits. These credits can be purchased
lopers can more or less openly visit these forums to get (depending on the country) with pre-paid cards, bank
customer feedback. On the other hand, when people transactions, credit cards, or special text messages that
use web 2.0 services, they leave an electronic trail. add a specified amount of money to the customer’s
What people click on, what they search for, what they mobile phone bill.
select, and how they navigate on a web site can all be The social interaction in Habbo is truly diverse. In
collected and aggregated. Service developers can use the design of Habbo, clear winning conditions and
this log statistics to fine-tune the service. Furthermore, gameplay rules have been avoided, and instead, players
in some services it is customary for users to openly are encouraged to create their own objectives beyond
share their preferences, bookmarks, opinions, and other chatting, room decoration, and meeting friends. The
content with other users. This participative web and provided environment for these activities is a hotel
user-created content [20] sometimes plays a major role consisting of public and private rooms, where the
in the service and can also be used by service virtual hotel visitors, called Habbos, chat, buy virtual
developers as inspiration for new features. furniture, decorate rooms, and arrange social and game
Social media is clearly not only win-win situations, events. Most of the teenage players log on after school,
since many discussions on privacy, advertising, and according to Sulake, the developer company, on
intellectual property rights, and other conflicts between average they spend around forty-five minutes per day
commercial actors and consumers have emerged. in the hotel or on its related discussion forums.
Lehmuskallio et al. [21] observed that these discus-
sions relate to communicational changes in access to, 3.1. Data and Method
scale, distribution, and persistence of computer-
mediated communication. For instance, web 2.0 The Habbo producers, Sulake, was established in
business models and privacy are often difficult to 2000 and focuses on virtual worlds and social network-
combine, as private information can be used to target ing. Sulake’s strategy is to be a leader in community
ads, be aggregated and sold to third parties, as well as based entertainment with a portfolio of properties
get into unexpected hands when big commercial addressing a wide range of target audiences. During
players buy web 2.0 start-up companies. 2003-2006, Sulake, together with ten other commercial
As social media change user-producer relation- actors of the Finnish gaming industry, participated in a
ships, new research questions related to user involve- research project coordinated by the Helsinki Institute
ment emerge. Do the short release cycles frequently of Information Technology [24]. This research context
associated with social media – the infamous ’forever made different design research studies and collabo-
beta’ approach – give more or less room for user ration with Sulake’s product development possible.
feedback? How do social media developers apply user The focus of the Habbo study was user involvement
involvement methods as the service evolves over time? strategies in the design and management of social
In a discussion of community building, Kim [22] media. Since its launch, the user communities have
suggests that many online communities start with the participated actively in the shaping of Habbo [25,26].
developers as central figures in the community, but as Some as active playmakers [27] and others as equally
the community grows, the developers’ roles and important participants and audience.
influence decrease. Is such a change in the developer- The author had the opportunity to conduct both
user relationship reflected in the use of user quantitative and qualitative research to understand the
involvement methods? Finnish Habbo communities. The project started with
participant observation in Habbo, pilot interviews, a
3. Case Habbo community manager survey (N=4), as well as an
explorative survey (June 2004) on the visitor profiles
Habbo is a virtual environment where children and (N=10000) as Sulake’s first global youth survey was
teenagers meet, socialize, and play many types of published two years later. The author participated in
games. It was first launched in August 2000 in Finland Habbo, explored the features and their affordances,
as Hotelli Kultakala (‘Hotel Goldfish’) and it was analysed default values and users’ degrees of freedom
based on the developers’ two earlier online services. At as a routine when new features were launched. The
3Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
visits to Habbo occurred on the average about twice The data analysis has proceeded in multiple waves
per month throughout the first three years of research, over the years, including quantitative and qualitative
and a visit every second month from 2007 onwards. clustering of Habbo user profiles [29], coding and
Fansites became an important source of knowledge examining interviews with regard to participation and
about the user communities. In 2004, Finnish Habbo development histories in Habbo, analysis of fansite
fansites were analyzed in detail [28]. Whereas the contents and developer views to them [25,28], and
survey findings provided background statistics, the here, constructing the changes in Habbo user-developer
fansites and forum discussions allowed an insight into relations over time.
active user groups, popular activities, and hotel history.
Since its very beginning Habbo fans have kept several 3.2. Habbo Service Evolution
sites devoted to documenting changes in the hotel, for
instance in kinds and appearance of furniture, charac- Like many other digital startups, Habbo is a
ters, mini-games, and bugs. moving target for many reasons. In 2003 there were
In 2005, the author conducted ten thematic 2-3 hour four hotels and 1 million users, but at the time of
interviews with ten Habbo developers, or about two writing, these numbers are more than tenfold. The
thirds of the game development organization at the producer company has grown from a small group to a
time. Six of the interviewed developers (graphical 300 people corporation, with local offices in many
designers, and client and server developers) had been hotel countries. To aid the analysis process, a division
in the organization since the beginnings, five years of Habbo service evolution into four stages will be
earlier, while four developers had about one year of outlined below.
Habbo experience. This was followed by 2-3 hour Habbo was based on two previous launches,
individual, pair, and group interviews with 11-16 years Mobiles Disco (Oct 1999) and a snowball game called
old users (N=6), and 30+ users (N=6) to focus on their Lumisota (Feb 2000). Mobiles Disco provided the
participation histories, motivations, meanings they give basic hotel infrastructure in the pixelated retrostyle,
to Habbo, and groups they participate in. whereas Lumisota featured payments through mobile
Habbo has also been a topic of collaboration phone messages, which at the time was a practical
between the author and other Habbo researchers. A solution for the Finnish market. Starting with the first
Sulake employee’s master’s thesis on communication international step to UK, other payment systems were
and action in Habbo provided important secondary implemented to suit markets with less wide use of
sources to Habbo data, as did several youth work mobile phones among teens. Phone call systems, youth
studies1. Usability research students at Helsinki cash cards made of paper, check systems came first,
University of Technology also provided secondary but soon followed payment by credit cards, Paypal, and
data, as part of to their efforts on both a usability test bank transfers.
on new 10-12 year old Habbo users (N=8) in 2004, and With the UK hotel also followed the teen invasion
a software engineering project in 2004-2005 to develop of Habbo [30]. Habbo was not designed for teens at
a fansite starter kit for active Habbo users. first and the large number of teen visitors also meant a
After these activities, the author had the large number of concerned parents. To keep all this
opportunity to take part in an intervention study with together, the community management received a lot of
Sulake. For release 9 of Habbo in 2006, a set of user attention. A set of guidelines for good behaviour was
feedback methods were explored with different established, called Habbo Way, and tools for governing
stakeholders inside Sulake. This included confidential it were developed.
data sets: database statistics and surveys from two During 2002-2003 Sulake developed its business,
countries on the use of a new feature. In addition to the hotel and the community simultaneously. Strategic
these research activities, Sulake representants partnerships were made and brands like Mountain Dew
participated in project partner seminars arranged every and Britney Spears entered Habbo. Later these brands
6 months, two workshops on virtual economy, and have disappeared as Sulake tried another approach with
many project meetings arranged by the research Coca-Cola and Warner Cinema. The brands got
project. These meetings made informal discussions and customised versions of the technical game engine
a continuous dialogue with Sulake possible. behind Habbo, which helped fund the development of
Taken together these bodies of data provide an the hotel. The hotel’s technical architecture was
excellent view to the varying forms of dialogue stabilized and security improvements were made to
between the users and developers of this virtual world. package the hotel into a product that could be more
easily rolled out in new countries.
1
These are only available in Finnish, but noted here for further During 2004-2005 around ten new hotels and local
inquiries: Pietiläinen 2004, Sihvola 2005, Koskinen 2006, Merikivi
2007.
offices were established in different parts of the world.
4Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
Habbo actively wanted to become the largest teen avatars and their descriptions reveal what kind of
brand in the world. The English-speaking UK hotel Habbo crowd is present. By studying the arrangement
was split into separate hotels for the US, Canada, and of Habbo furniture and guidelines written on virtual
Australia. Also game development recruited more post-it notes, the developer learns about how particular
developers in 2004, and new features were packaged pieces of furniture are used in specific user activities.
into simultaneous releases in all hotel countries. Developers as users. In the early stages of concept
Sometimes during 2006 and 2007, the strategies development and beta testing, the developers were
changed again as Sulake entered the social networking developing the service for themselves, their friends and
market. Not only did Habbo get new social networking their new media colleagues. The first prototype was a
features, but Sulake acquired IRC-Galleria, the most hobby project for a friend’s band. The original group
popular social networking service in Finland. As IRC- of developers started out as insiders in the user
Galleria users were a little bit older, average age community, but as the service become more popular,
around 20, the strategy to concentrate on teens was other insider groups started to form. After a while, the
opened up to a variety of age groups. developers became more distanced to the user
Based on this information, the author has grouped community as it became more difficult for them to
the service evolution into four stages. spend time in Habbo. Each time a Habbo developer’s
avatar was online, it did not take long before Habbo
Table 2. Habbo Service Evolution Stages newcomers gathered around and wanted to become
Habbo friends, or even asked for freebies. Habbo
Concept Beta Expansion Complexity developers soon invented their own tactics to partici-
Development Testing Management pate in cognito: they created new avatars, whose real
1999-2000 2001-2003 2004-2005 2006-
identity they only told trusted friends.
Informal evaluations. Various informal evaluation
Concept Design refers to the first prototypes in practices guided the design early on. In interviews, the
1999 and 2000: Mobiles Disco, Lumisota, and Hotelli
developers talked about slogans such as ”easy access,
Kultakala, while Beta Testing refers to the time period
easy play” and ”where else”. A developer described the
between 2001-2003, when much of the basic latter one as ”We won’t make obvious choices, but
functionality was completed. Expansion refers to 2004- rather something personal that gives the Habbo world
2005 when the product was packaged so that it made a something odd and own, which creates an own
roll out possible in more than 10 new countries during
persona.” (Interview 19 Apr 2005). The “easy access,
one year. Complexity Management refers to 2006 and
easy play” slogan was not only a slogan, but among the
onwards when the product was extended to a social
developers it also had a visual form called the Habbo
networking service. Ladder that showed the first steps a new user takes.
The Habbo Ladder was made to contrast Habbo
4. Habbo User Involvement Methods with the comparably higher threshold to start playing
massively multiplayer online games at the time. The
The user involvement methods reported here idea was that during the first few minutes that anyone
attempts to be an overview of the methods used by is willing to spend on a new web service, one should
Sulake from 1999 to 2009. The criteria for what counts be able to login and create an avatar easily, learn the
as a method is a more or less organized practice where basic navigation, and have a chat with someone else.
developers learn about the users, which then shapes the Each step on the ladder makes more reasons to return
service. The user involvement methods are organized to the service, as a developer explained:
according to the service evolution stages. ”Especially if one gets the first friend on the list of
friends, then that it is a reason to return, that you have
4.1. Concept Development really got to know someone with whom you might
have had an interesting discussion or of whom an
Avatar activities. Avatar activities refer to what interesting image has been conveyed. Then further on,
users do online in the service. Developers have easy when you have your own room, well that is of course a
access to users in the sense that the developers can log real investment, even the notion that you have
on to Habbo and check what is going on. In the same something own going on there, then that is already a
way as users find out what happens where in the hotel, good reason to return, especially if you have decorated
developers can also use the list of most popular rooms the room, really purchased something.” (Interview 19
or tags to find the trends. In the room, a developer can Apr 2005).
choose to participate in the ongoing activity and take E-mail feedback. During the first year or so,
note of which users are present. The names of the user developers received a lot of e-mail feedback by users,
5Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
who suggested changes and new features. This e-mail homepages, and some created totally Habbo-themed
feedback became a handy storage for design inspiration websites. Being a Habbo journalist was one way of
for the developers, who used to return to it later and becoming famous, besides participating in various and
browse for good ideas. As the user community grew in creating own online Habbo activities. Some fansites
size, this direct e-mail feedback became impractical as featured discussion forums, which also were an
the number of e-mails became too large. Users not only important source for developer inspiration [25,28].
sent feature suggestions per e-mail, but all kinds of Official Web Fanzine. For two years, Sulake
support requests, which then as the organisation grew, published their own official Hotel Magazine in Finland
became the responsibility of a small but growing (www.kultakalankuvalehti.com), but due to limited
number of volunteer community moderators and resources they closed it down. It started out with
customer service officers. writings from Habbo developers, and interviews with
Volunteers. Already before Habbo, in Mobiles Habbo users, but soon enough some Habbo users
Disco, there was a need to keep the shared meeting featured as guest authors. In 2004 Sulake launched
places in order, in case a user did not follow the shared local competitions in each country to give a few
conventions of appropriate online interaction — the elected large fansites the status of being official Habbo
netiquette. Trusted friends of the developers or other fansites. While these fansites complemented Sulake’s
insider users received special powers, for instance to community management, they also helped Sulake in
remove (kick) a disturbing user avatar from a room, regulating sites that were out to scam the users.
and the responsibility to keep the online place neat. Summer Meetings. With the responsibility for
These volunteers followed along from the first proto- teenagers online, Sulake had to develop a policy that
types to Hotelli Kultakala and Habbo. Later it became discouraged sharing of contact information online and
possible to apply to become a volunteer, so called meetings in person. To support this policy, Sulake
Hobba. However handy early on, and applicable in arranged summer meetings where users could meet
many hotel countries, this arrangement involved more each other in person, as part of an organised event. In
than 1500 volunteers during the expansion phase in the first meetings in Helsinki active users and
2005. Sulake changed the volunteering practice in volunteers got to meet the community manager and a
2005, as the special powers were given to paid few developers. Later on these meetings have been
moderators only, and the notion of volunteer arranged by active users, without much involvement of
moderators transformed into volunteer guides, who developers.
could guide online newcomers. Sales Statistics. In contrast to other Habbo stake-
holders, developers have an additional perspective to
4.2. Beta Testing the online activities in Habbo. They have a back-end
service that keeps track of what pieces of furniture
Volunteer forum. To share experiences and mode- have been sold and how many in which hotel. This
ration policies, the volunteers created an online forum means that developers can compare Habbo features on
for themselves. Along with the internationalisation and the basis of their economical behaviour, not only based
more organised volunteer management, Sulake started on functional or aesthetic properties. Even though the
hosting a local volunteer forum per hotel country. The use of Habbo is anonymous, the registration process
volunteers soon got an important role as mediators of asks users to fill in age and gender, which can then be
user opinions: the developers knew that as the connected to sales statistics on an aggregated level.
volunteers spent the most time in the hotel, they were Customer service. The need for dedicated custo-
always the first to know about the current user mer service emerged as the user communities grew and
concerns, wishes and emergent activities. the developers could not find time to handle all support
Weekly newsletter and polls. In an attempt to requests. Each hotel country got their own country
organise user feedback and create a rhythm of new office, where a local manager managed the volunteer
content, the hotel community management started moderators and customer service officers sorted out
creating weekly polls and newsletters. The newsletter user requests.
contained information about new features and events
organised by the hotel manager (a Sulake employee). 4.3. Expansion
With the newsletter was usually a gallup poll or a
competition that the users could participate in. These Market survey. With the international expansion
weekly polls were important to gather immediate user emerged a need to know whether the user communities
feedback regarding new features. were similar or different in different hotel countries.
Fansites. Right from the start, Habbo users started An outsourced market survey in 2004 generated
writing about and debating Habbo on their own customer segments and their regional distribution. The
6Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
named customer segments and their demographics pilot country acted as beta-testers and could comment
were important in the communication about Habbo. on the features in the release before they were
Even though demographics seemed less important than finalized. The process of selecting the pilot country has
user practices from a game development perspective, changed several times. In 2005, countries with a
customer demographics played an important role in relatively small number of users acted as pilot
business negotiations, regarding advertisement for countries. However, translation and language become
instance. an issue, which is why piloting returned to Finland
Focus groups. To prepare for the rapid inter- sometimes around 2007. However, at the time of
national expansion that happened during 2004-2005, writing, UK is the pilot country.
focus groups were conducted. The applicability of
Habbo pixel style graphics and use of colours was eva- 4.4. Complexity Management
luated for the Asian market. In addition to traditional
market research focus groups, two variants of the focus Online user panel. In an effort to gather systematic
group method from different research traditions were feedback before the implementation of new features,
used, usability evaluation and playability testing. Sulake recruited 200 volunteers in one country to form
Usability evaluation with users is more like a an online panel. The online panel was given a weekly
controlled lab experiment in psychology, where users task consisting of a set of questions regarding design
try out the software with the help of scenarios and sketches and an opportunity to share opinions
evaluators measure issues like usefulness, ease-of-use, regarding the sketches in a forum. The online panel
learnability, and memorability [31]. was popular among developers, who queued to be the
Usability Evaluation. The first formal usability one who got to get design-time feedback from the
evaluation in the fall of 2004 was targeted at checking users.
the usability of service registration and those services Global youth survey. As both the users and
in Habbo that were subject to a fee, from the viewpoint competitors change, in 2006 it was again time for a
of 10-14 year olds. The evaluation was outsourced and new market survey (GHYS’06). This time the
conducted by usability students at the local university customer segmentation resulted in certain Habbo
of technology (TKK). After the first usability evalua- lifestyles: achievers, creatives, loners, rebels, and
tion, Sulake established an in-house usability process traditionals. The report featured extensive country-
through a pilot project. The pilot project was related to specific information on teenagers’ favourite brands and
a product extension for the mobile phone market, and media usage patterns.
was influenced by usability standards in the field, such User, group homepages, tags. In early 2007,
as the J2ME guidelines made by Idean Research [32]. Sulake launched social networking features in Habbo
Playability testing. While the usability evaluation as every Habbo avatar got an automatic and
method was appropriate for features such as login, cha- customisable Habbo homepage. It was also possible to
racter creation, furniture purchases, usability methods form Habbo groups, which meant a logo, a group
in general are not tuned to assess playability. For these homepage and a discussion forum. In addition, users
reasons, Sulake’s R&D lab used playability testing could ”tag” their avatars, which meant that users could
methods developed at Tampere University’s Hyper- attach a set of clickable one-word descriptors to their
media Lab [33] to assess various playability aspects, avatar. When a user clicks on a tag, the service
such as gameplay, game mechanics, appearance, generates a dynamic index of all the users and groups
sound, and social playability. with that particular tag.
CRM system. As some hotel communities grew User experience testing. Following current
larger, pressure emerged for customer service to research terminology, in 2008 Sulake talked about user
automate their responses. For instance, in a country experience testing in addition to usability or playability
with several hundreds of thousands of users, a new testing. While the usability testing has evolved from a
feature might spawn several thousands of inquiries per more stand-alone practice, to a tight integration with
day. In 2005, a new customer relationship management agile software development, Sulake conducts user
system was introduced. It featured a set of standard experience evaluations with both new and old users,
questions and responses, which reportedly reduced internally called ”live tests”, in one country for every
inquiries with 90 percent. major release, which makes about once a year
Release pilots. As the organic beta testing phase (Personal communication with User and Market
changed into a more controlled release management Insight Director, 26 Nov 2008).
process, Sulake started piloting the release for one Personas. As part of further developing user-
month in one hotel country, before diffusing the release centred design processes, during Spring of 2009,
to other hotel countries. This means that users in the Sulake applied the Persona method. Six user
7Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
Table 3. Overview of Habbo user involvement methods.
Concept Development Beta Testing Expansion Complexity Management
Avatar activities Volunteer Forum Market survey Online user panel
Developers as users Weekly newsletter and polls Focus groups Global youth survey
Informal evaluations Fansites Usability evaluation User and Group homepages, tags
E-mail feedback Official Web Fanzine Playability testing User Experience Testing
Volunteers Summer meetings CRM system Personas
Sales statistics Release pilots
Customer service
archetypes have been constructed from data to knowledge, which was provided by the use of more
represent the users. The idea is that developers have an formal methods.
updated reference to the goals and needs of Habbo So far the results from this case study is much in
users at hand, which can inform design solutions and line with previous research. However, the user
evaluations. (Personal communication with User and involvement practices in this case study stands in
Market Insight Specialist, 2 Jun 2009). contrast with the strategy of developing a method and
then using it in the same repeatable way over and over.
5. Discussion In this case it has been evident that the history of
method usage shapes the succeeding use of methods.
For instance, as the service as a whole had been
5.1. User Involvement and Service Evolution
usability evaluated, the usability specialists at Sulake
found no sense in repeating the same evaluation again.
Based on the Habbo case it seems evident that very
Usability evaluations turned towards smaller details of
few service characteristics relevant to user involvement
the service after the overall evaluation. The same
methods remain invariant over time. Some charac-
finding was found in the market survey practices. After
teristics remained stable: the service concept and the
the first overall customer segmentation, the following
profit model (after small experiments during the first
surveys could fine-tune specific issues that remained
few years), the technical access to online user
open in the previous surveys. This practice can be
activities, as well as the service domain and (life-)
interpreted as a sensible ongoing tailoring of methods
criticality of the interface. However, a lot has changed
that becomes necessary when dealing with as complex
over ten years: at least the number of users, the number
and changing phenomena as in this case.
of developers and resources available, the amount of
money involved, competition on the market, the stage
of design, the skills and experience of the developers. 5.2. User Involvement and Social Media
As the maturity of the market changed, so did the
relationship between developers and users, much like It seems that the developers have benefited from
in previous research by Leonard-Barton [3] and Kim particular user involvement opportunities related to
[22]. In the Habbo case, like with most social media social media. User created fansites have been an
applications, the developers started developing the important source for developer inspiration, and the
service for themselves and their friends. While starting online user panel is an interesting way of making
out as insiders in the user community, with commercial prolonged contact with a particular user group, which
success, the developers inevitably end up as excluded complemented both the more informal feedback that
from the user community, if for no other reason, just developers get in the service itself and the more formal
by the sheer number of users and their practices. surveys. The social networking features – user and
The first usability study and succeeding studies group homepages as well as tagging – provided Habbo
marked two changes in user-developer relations. First, users with new possibilities to express themselves
as the number of users increased, the margins for errors within the service and find other interesting users and
in the service were decreased, thus more formal groups. On the other hand, these social networking
methods were valued. Second, developers’ gut feeling, features made it easier for Sulake to get an overview of
that is, undocumented previous development expe- group activities in Habbo as well as search and
rience and participation in the user community, carried aggregate data on users.
less weight, as not everyone in the growing company Let us return to one question posed in the begin-
could be involved in the user communities. Business ning. What happens with user feedback in a “forever
and development decisions needed more documented beta” approach, is there more or less room for it
compared to traditional product development?
8Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
In the Habbo case, it seems that user feedback can 6. Conclusions
shape the service most when the design space is open.
How open the design space is, depends on what is In addition to the themes discussed in research
going on in development. During concept development literature on user involvement, user involvement
and early beta testing Habbo was fairly open for fea- regarding social media appears to be an issue of
ture suggestions, but it did not take many months for timing, trans-nationality, and governance. It is a timing
the developers to get more ideas than they could handle issue, because user feedback influences most the
for the foreseeable future. Then the question was no features under development at a particular point in
longer about getting good ideas, but rather about find- time. It is a question of trans-nationality, because user
ing projects and funding to implement the ideas and feedback from different countries is influenced by both
stabilize the service. And, until the service was stabi- language and market size. It is a question of
lized and packaged enough for manageable duplication governance, because user feedback might also address
in the expansion phase, the design space was fairly issues that are not considered open for discussion by
closed. From a Finnish user point of view, the next the developers.
window of opportunity to shape the service was Sulake’s user involvement practices reveal a bias in
probably the implementation of the social networking common classifications of user involvement methods.
features in early 2007, which again changed the rules Traditional formal methods and usability evaluations
of the game. Then as the new features were tried out, are only one small part of the big picture regarding
user feedback again helped fine-tune the service. learning about social media users. The 23 [!] different
User feedback varies between Habbo hotel coun- modes of user involvement reported in this study
tries. Like with any trans-national service, Habbo suggest that the usability practitioner’s ‘toolbox’ needs
developers experience the dilemmas of language tuning regarding social media.
regions and differently sized markets. On the one hand, This case also reveals how the nature of user
one could argue that, because of Habbo’s Finnish involvement has changed with social media. Before
origins, hotels with close linguistic ties to strong social media, ‘involving users’ has been a much more
languages in Finland would shape the service most. explicit effort, as developers and researchers have had
However, there are two factors counteracting this to go where the users are and ask permission to
simple argument. First, language skills and effective observe and get feedback. While some ways of
communication among Sulake employees can mediate involving social media users remain as indirect as
remote user needs. Second, since some user needs are before, feedback from social media users is at the same
readily communicated through sales statistics, the time genuinely direct. The very thing people do in and
market size of a particular hotel country might be more return to the service for, can be interpreted as user
significant than the language in that country. feedback. The users might not think that they are being
In a broader perspective, ”what happens with user ‘involved’ or formally ‘invited’ – they will just not
feedback in social media?” is also a question about return if the service does not meet their needs.
governance, that is what users get to decide on and the However, previously it took longer before a decision
form of their influence. Typically users have less by a user to stop using a product or service reached the
influence on generic products and more influence on producers.
customisable products. The impact of user feedback is This ‘direct feedback’ from social media users, in
especially big if the users themselves take the initiative terms of frequent data points of user actions, has a
to new development. In contrast, social media is not major consequence to user studies. The techniques
usually developed as a contract between a developer commonly denoted as evaluation ‘with’ and ‘without’
and a client organisation, but rather on the initiative of users need to be supplemented with ‘evaluation with
a handful of developers. Habbo users have had a big database access to users’. As both use and non-use is
influence on early feature development and issues like logged in the server database, alarms can be set to
community moderation (as volunteers), online discus- trigger if use practices change. These alarms can then
sion about Habbo (as fansite authors), but have never be used as starting points for succeeding user studies to
been invited to decide on pricing, advertisements, or explain the changes in use practices. How strange, or
feature roadmapping. This is not to say that they perhaps disciplinary inconvenient, it may sound, the
should be, it is just a fact that needs to be stated when experience from this case study suggests that
outlining the Habbo user-developer relationship. E- acquisition and analysis of database logs should take
democracy in Habbo has not yet reached the extent of place before embarking on either quantitative or
the governance experiments in virtual worlds such as qualitative studies on social media use.
EVE Online or A Tale in the Desert.
9Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010
7. References Human factors in computing systems, Florence, Italy:
ACM, 2008, pp. 111-120.
[19] D. Svanæs and J. Gulliksen, “Understanding the context
[1] A.L. Friedman and D.S. Cornford, Computer Systems
Development: History Organization and of design: towards tactical user centered design,”
Implementation, John Wiley & Sons, 1989. Proceedings of the 5th Nordic conference on Human-
computer interaction: building bridges, Lund,
[2] E.V. Hippel, Democratizing Innovation, The MIT Press,
Sweden: ACM, 2008, pp. 353-362.
2005.
[3] D. Leonard-Barton, Wellsprings of Knowledge, Harvard [20] OECD, Participative Web and User-Created Content:
Business School Press, 1995. Web 2.0, Wikis and Social Networking, 2007.
[21] A. Lehmuskallio, S. Tamminen, and M. Johnson,
[4] S. Kujala, “User Studies: A Practical Approach to User
“Managing Privacy on Social Network Sites,”
Involvement for Gathering User Needs and
Requirements,” Doctoral Thesis, Helsinki University Medienamateure: Wie verändern Laien unsere
of Technology, 2002. visuelle Kultur?, Siegen, Germany: 2008.
[5] J.T. Hackos and J.C. Redish, User and Task Analysis [22] A.J. Kim, Community Building on the Web: Secret
Strategies for Successful Online Communities,
for Interface Design, Wiley, 1998.
[6] M. Kuniavsky, Observing the User Experience: A Berkeley, CA, USA: Peachpit Press, 2000.
[23] Sulake, “Quick Habbo facts,” July 2009,
Practitioner's Guide to User Research, Morgan
http://www.sulake.com/habbo
Kaufmann, 2003.
[24] M. Turpeinen and K. Kuikkaniemi, Mobile content
[7] J. Hom, “The Usability Methods Toolbox,” 1998,
communities final report, Helsinki, Finland: Helsinki
http://jthom.best.vwh.net/usability/index.htm
Institute for Information Technology, 2007.
[8] U.S. Dept. of Health & Human Services,
[25] M. Johnson, “Unscrambling the “Average User” of
“Usability.gov: Your guide for developing usable &
Habbo Hotel,” Human Technology, vol. 3, May 2007,
useful Web sites,” 2009, http://usability.gov
pp. 127-153.
[9] UsabilityNet, “UsabilityNet: usability resources for prac-
[26] M. Johnson and T. Sihvonen, “On the Dark Side:
titioners and managers,” 2003, http://usabilitynet.org
Gothic Play and Performance in a Virtual World,”
[10] D. Norman, “Why doing user observations first is
Journal of Virtual Worlds Research, vol. 1, 2009.
wrong,” Interactions, vol. 13, 2006, pp. 50-ff.
[27] A. Salovaara, M. Johnson, K. Toiskallio, S. Tiitta, and
[11] H. Beyer and K. Holtzblatt, Contextual Design: A
M. Turpeinen, “Playmakers in multiplayer game
Customer-Centered Approach to Systems Designs,
communities: their importance and motivations for
Morgan Kaufmann, 1997.
participation,” Proceedings of ACE 2005, Valencia,
[12] P. Ehn, Work-Oriented Design of Computer Artifacts,
Spain: ACM, 2005, pp. 334-337.
Stockholm, Sweden: Arbetslivscentrum, 1988.
[28] M. Johnson and K. Toiskallio, “Fansites as sources for
[13] J. Greenbaum and M. Kyng, Design at Work:
user research: Case Habbo Hotel.,” 28th Information
Cooperative Design of Computer Systems, Hillsdale,
Systems Research Seminar in Scandinavia,
NJ, USA: Lawrence Erlbaum Associates, 1991.
Kristiansand, Norge: 2005.
[14] D. Schuler and A. Namioka, Participatory Design:
[29] M. Johnson and K. Toiskallio, “Who are the Users of
Principles and Practices, Mahwah, NJ, USA:
Habbo Hotel?,” Mobile Content Communities, HIIT
Lawrence Erlbaum Associates, 1993.
Publications 2007-1, 2007, pp. 89-107.
[15] R. Williams, J. Stewart, and R. Slack, Social learning in
[30] M. Johnson, “How The Teenage Invasion Changed
technological innovation: Experimenting with infor-
Habbo or Age-Sensitive Design In Inclusive Society,”
mation and communication technologies, Cheltenham,
The Good, The Bad, The Challenging: The user and
UK: Edward Elgar, 2005.
the future of information and communication techno-
[16] J. Stewart and R. Williams, “The wrong trousers?
logies: a transdisciplinary conference. Organized by
Beyond the design fallacy: Social learning and the
COST Action 298, Participation in the Broadband
user,” Handbook of critical information systems
Society, B. Sapio, L. Haddon, E. Mante-Meijer, L.
research: Theory and application, D. Howcroft and E.
Fortunati, T. Turk, and E. Loos, eds, Copenhagen,
Trauth, eds., Cheltenham, UK: Edward Elgar, 2005,
Denmark: 2009.
pp. 195-221.
[31] J. Nielsen, Usability Engineering, Academic Press Inc,
[17] A. Crabtree, T. Rodden, P. Tolmie, and G. Button,
1993.
“Ethnography considered harmful,” Proceedings of
[32] Idean Research, “Series 60 Developer Platform 2.0:
the 27th international conference on Human factors in
Usability Guidelines For Enterprise Applications.”
computing systems, Boston, MA, USA: ACM, 2009,
[33] A. Järvinen, S. Heliö, and F. Mäyrä, Communication
pp. 879-888.
and Community in Digital Entertainment Services,
[18] S. Greenberg and B. Buxton, “Usability evaluation
Prestudy Research Report, Tampere, Finland:
considered harmful (some of the time),” Proceeding
Hypermedia Laboratory, University of Tampere,
of the twenty-sixth annual SIGCHI conference on
2002.
10You can also read
