Travel and Tourism The Art of the Pivot - Multilingual Magazine

Page created by Helen Chambers
 
CONTINUE READING
Travel and Tourism The Art of the Pivot - Multilingual Magazine
The Art of the Pivot       A Tale of Two Tourisms   You’re Talking to my Dad

                       Travel and Tourism
Travel and Tourism The Art of the Pivot - Multilingual Magazine
/

    •
Travel and Tourism The Art of the Pivot - Multilingual Magazine
Travel and Tourism The Art of the Pivot - Multilingual Magazine
Travel and Tourism The Art of the Pivot - Multilingual Magazine
Pos t Ed i t i n g

                                                               #195 Volume 34 Issue 3 May/June 2021
                                                               Publisher: Marjolein Groot Nibbelink
                                                               Editor-in-Chief: Katie Botkin
                                                               Layout and Design: Antonella Tiezzi
                                                               Subscription Management: Terri Jadick

  T  ourism may have been severely neutralized in the
past year — but it still managed to get me pregnant.
                                                               Assistant Editor: Michael Reid
                                                               Chief Marketing Officer: Nika Allahverdi
                                                               Marketing Coordinator: Evelyn Najarian
  In the course of 2020, Idaho tourism, was, ironically        Chief Information Officer: Aleksey Schipack
                                                               Staff Writers: Katie Botkin, Michael Reid, Andrew Warner
enough, hopping. People flooded into my resort
                                                               Chief Financial Officer: Kristen Glant
town from neighboring states, making the streets               Chairman of the Board: Renato Beninatto
busier than I’d ever seen them. Idaho standards were           Board Director: Tucker Johnson
comparatively lax, the local lake offered hundreds of
miles of shoreline, and the mountains had plenty of            Editorial Board
hiking trails. I figured out how to be an AirBnB host,         Games: Miguel Á. Bernal-Merino
and subsequently met a guest I fell in love with and           Standards: David Filip
started dating. Several months later, and, well… as a          Business: Aki Ito
result of COVID-era travel, I’m expecting my first child.      Marketing: Nataly Kelly
                                                               User Experience: Ultan Ó Broin
   The travel and tourism sector isn’t dying so much as        Interpreting: Barry Slaughter Olsen
it’s figuring out its new nascent qualities. This spring,      Technology: Jost Zetzsche
instead of taking my annual trip to Asia to escape the
snow, I headed south with my youngest brother on a             Advertising
US road trip. We stayed with friends who had been              advertising@multilingual.com
vaccinated; took side trips into deserted areas.
                                                               News and press releases
  Before, the annual trek to Asia coincided with               news@multilingual.com
work trips — for the 12 years prior to the pandemic,
MultiLingual gave me opportunities to travel and the           Subscriptions, back issues, customer service
freedom to explore after working. When the pandemic            subscriptions@multilingual.com
hit, of course, everything changed, and meeting
colleagues in the bars of Tokyo and Lisbon was no              Submissions
longer on the menu. Traveling was relegated more to            editor@multilingual.com
the deserts and mountains of my own country.                   Reprints and eprints
   MultiLingual, and all the cross-cultural exchange it        reprints@multilingual.com
represented, has had my love as long as I’ve known
it. The joke was, before my actual baby, MultiLingual          MultiLingual Media LLC
was my baby. I edited more than 100 issues, starting           319 North First Avenue, Suite 2
                                                               Sandpoint, Idaho 83864-1495 USA
with issue #95 on machine translation, and ending
with #195 on travel — a fitting note to conclude on as I       info@multilingual.com
move toward other things.                                      https://multilingual.com
  I have enormous affection for all the writers and
other talented people I’ve worked with over the years,
so thank you all. And I’m happy to offer loyal readers
this, my final issue, delving into a topic that’s sure to be
on our minds now more than ever.                               MultiLingual (ISSN 1523-0309), Copyright © 2021 by MultiLingual Media LLC, is published
                                                               bimonthly: Jan/Feb, Mar/Apr, May/Jun, Jul/Aug, Sep/Oct, Nov/Dec. by MultiLingual
                                                               Media LLC., 319 North 1st Avenue, Suite 2, Sandpoint, ID 83864-1495. Business and
                                                               Editorial Offices: 319 North 1st Avenue, Suite 2, Sandpoint, ID 83864-1495. Accounting
                                                               Offices: 515 30th Ave S, Seattle, WA 98144-2509. Circulation Offices: 319 North 1st
                                                               Avenue, Suite 2, Sandpoint, ID 83864-1495. Email subscriptions@multilingual.com to
                                                               subscribe. Periodicals postage paid at Sandpoint, ID and additional mailing offices.
                                                               POSTMASTER: Send address changes to MultiLingual, 319 North 1st Avenue, Suite 2,
                                                               Sandpoint, ID 83864-1495.

                                                                                        May | June 2021                                   5
Travel and Tourism The Art of the Pivot - Multilingual Magazine
#195 - TRAVEL AN D TOURI SM

               TABLE OF CONTENTS

05
Postediting
                              18                       20                   22
                              Review                   Column               Column

08                            An Introduction to
                              Religious Language
                                                       Managing
                                                       Employees as an HR
                                                                            AI and Technology
                                                                            in Healthcare
Featured Reader               Katie Botkin             Expert               Christophe Djaouani
                                                       Terena Bell

09
Recaps
                              24                       28                   37
17                            Column                   Focus                Focus
Calendar
                              Metrics, What are        Remote Working In    Reconstructing
                              they Good For?           2021                 Travel and Leisure
74                            Absolutely... Nothing?
                              John Tinsley
                                                       Molly Lipson         Arle Lommel and
                                                                            Rebecca Ray
Buyers Guide

  6                May | June 2021
Travel and Tourism The Art of the Pivot - Multilingual Magazine
T R AVE L A N D T O U R IS M # 1 9 5 -

                                            #195
                                            MAY/JUNE
                                            Travel and Tourism
                                           Whatever language we speak, we’re
                                           looking forward to saying “hello” again.
                                           From left to right: Arabic, Chinese,
                                           English, Greek, Portuguese, Guaraní.

40                       46                        52                             58
Focus                    Focus                      Focus                         Business

Tourism Inside a         The Art of the Pivot      A Tale of Two                  You’re Talking to my
COVID Haven              Jessica Roland            Tourisms                       Dad
Serena Puang                                       Michael Reid                   Pushpinder Lubana

64                       70                                                       81
Tech                     Special Report                                           Takeaway

The Present and Future   Nimdzi 100                                               The Connective
of Natural Language      Nimdzi                                                   Multilingual Internet
Processing                                                                        John Yunker
Babita Jain and
Hardik Dwivedi

                                                                      May | June 2021                        7
Travel and Tourism The Art of the Pivot - Multilingual Magazine
N EWS

        FEATURED READER

       With Adam Asnes
       CEO of Lingoport

Where do you live?
  Boulder, Colorado.

How did you get started in this industry?
  I joined a startup in 1998 that was globalization engineer-
ing focused. Besides being their first sales person, I got to lead
partnership efforts, which gave me an excellent introduction to
many influential industry leaders. I left and started Lingoport
in March of 2001.

What are you working on now?
   We just released new technology to make linguistic reviews
and updates for software localization fast and easy. That capa-
bility, along with enhancements to our continuous internation-
alization and localization suite, could have significant impacts
on how software localization is performed, I think.

What language(s) do you speak?                                       What industry organizations and activities do
  I’m embarrassed to say really just English. I have some            you participate in?
high school French, which gets better when I have to use it if         We run our own webinar series with sessions almost monthly,
traveling in the French countryside. I also spent three months       so that is my main activity. Other than that, there are LocWorld
in Yugoslavia in 1989, and got conversational in Serbian via         conferences, pop-up webinars and events from partners, and
immersion. It’s surprising how much of that can come back to         podcasts where I am asked to be a guest.
me when the opportunity occasionally arises.
                                                                     Do you have any social feeds of your own? Twit-
Whose industry social feeds (twitter, blog,                          ter handle, blog?
LinkedIn, Facebook) do you follow?                                     @adamasnes and @Lingoport on twitter. I’m somewhat
 The usual suspects: Women in Localization, Kathrin Buss-            active on LinkedIn too. There’s Facebook, but that ends up
mann, Renato Beninatto, Tucker Johnson, Anna Schlegel, and           being mostly friends and family with a touch of industry.
many more.
                                                                     Why do you read MultiLingual?
What do you like to do in your spare time?                             MultiLingual is the voice of the industry. It’s also changing
  I’m big into cycling and ride nearly every day. I started racing   with new management, and I’m interested in following how
at age 16 and still compete regionally and nationally.               the medium and message is developing these days.

   8                     May | June 2021
Travel and Tourism The Art of the Pivot - Multilingual Magazine
REC A PS

     NEWS
                       M E R G E R S A N D AC Q U I S I T I O N S

                       Women-led Language I/O raises $5 Million in its First
                       Funding Round

                                                                    following the funding round, owning just over 50% of the
                                                                    company.
                                                                      “Language I/O is disrupting how businesses communicate
                                                                    with their global customers,” said Davoli. “The combination
                                                                    of innovative technology, product market fit, and an excep-
                                                                    tionally strong team compelled me to invest. Language I/O is
                                                                    eminently poised for tremendous growth.”
                                                                      “I am extremely proud of our team,” said Shoemaker. “This
                                                                    investment will allow us to expand our sales and development
                                                                    teams so we can build out our conversational AI translation
                                                                    tech to meet our customer’s needs.”
                                                                      “We are thrilled to be one of the rare technology companies
                                                                    founded and led by women to receive venture capital fund-
                                                                    ing,” stated Kvaavik.

Heather Morgan Shoemaker

   Language focused customer support solutions provider
Language I/O announced on March 23 that it had raised over
$5 million in A round funding. The women-led, Wyoming-
based company had been bootstrapped since 2015, except for
a seed round of $500,000 in October 2020. The new funding
is particularly significant because according to data published
by Crunchbase, only 2% of global venture capital funding
goes to companies founded solely by women, and even less
in the tech space.
   Bob Davoli of Gutbrain Ventures and Bruce Clarke of
PBJ Capital co-led the A round and will join the Language
I/O board of directors along with the two co-founders of
Language I/O, CEO Heather Morgan Shoemaker and CBO
Kaarina Kvaavik. The founders are still majority shareholders       Kaarina Kvaavik

                                                                                         May | June 2021                   9
Travel and Tourism The Art of the Pivot - Multilingual Magazine
RECAPS

                      M E R G E R S A N D AC Q U I S I T I O N S

                      Ten LSPs form translate5 Consortium

                                                                      The development of translate5 has been led by MittagQI,
                                                                   and is well-known for its use in proofreading and post-edit-
                                                                   ing. One of the translate5 Consortium’s major goals is to
                                                                   improve the system’s machine translation and post-editing
                                                                   capabilities by adding more analysis options. Because trans-
                                                                   late5 is an open-source software, it is both adaptable and
                                                                   affordable, which has led to the system’s popularity among
                                                                   many LSPs. “It’s an intuitively designed browser-based
                                                                   translation software, developed by a team that excels in col-
                                                                   laboration,” said Rémy Blättler, the chief technology officer
                                                                   (CTO) at Supertext, another founding member company in
                                                                   the consortium.
                                                                      As the use of technology in the industry becomes more
                                                                   and more ubiquitous, members of the consortium see
                                                                   translate5 as particularly useful because of its reliability and
                                                                   flexibility. The translate5 Consortium currently consists of
                                                                   10 member companies, but welcomes other LSPs to join the
                                                                   consortium.
Marion Randelshofer

  Ten European language services providers (LSPs) have
come together to form the translate5 Consortium, a group
that’s investing in the open-source translation management
system and editor, translate5, in the hopes of developing an
alternative translation solution for LSPs across the world.
The project includes a slew of additions to the translate5
system, such as a switch to cloud infrastructure based on
Kubernetes and Docker and improvements to its usability
and the translation editor and project management features.
  “Translate5 is a modern tool that can be used to solve
review and other processes elegantly and securely,” said
Marion Randelshofer, the chief operating officer (COO)
at World Translation A/S, a Denmark-based company that
joined the translate5 Consortium. The consortium is made
up of a group of companies that want to “actively shape,
support, and use translate5 as a basic and next-generation
technology,” according to a blog post on the platform’s
website.
                                                                   Remy Blättler

 10                    May | June 2021
REC A PS

                       M E R G E R S A N D AC Q U I S I T I O N S

                       KUDO closes $21 Million in oversubscribed Series A Round

                                                                       KUDO, established in 2017 as a remote interpretation solu-
                                                                    tions provider, saw significant growth in 2020 as the COVID-19
                                                                    pandemic shifted global business to a largely virtual format. In
                                                                    early 2021 they also launched KUDO Marketplace, a schedul-
                                                                    ing and automated interpreter booking platform. Speaking to
                                                                    MultiLingual, KUDO CEO Fardad Zabetian said that investors
                                                                    reacted positively to the launch of the Marketplace platform, and
                                                                    believe that it will “stimulate a whole new level of growth.”
                                                                       Asked for specific initiatives the company is planning to
                                                                    pursue, Zabetian said that KUDO plans to use the funding to
                                                                    “broaden access to multilingual meetings, opening up use cases
                                                                    that hadn’t previously considered interpretation.”

Fardad Zabetian

   KUDO, Inc., creator of the eponymous cloud-based video
conferencing platform that incorporates real-time multilingual
interpretation supporting over 100 spoken languages and 147
signed languages, announced on March 30 the close of its $21
million series A funding round. Combined investments bring
the total round of funding to nearly $27 million. According
to KUDO, this new capital will be used for talent attraction,
product engineering, marketing, and business expansion.
                                                                    Session console in Spanish - www.kudoway.com

                                                                                          May | June 2021                      11
RECAPS

              I N T E RV I E WS

              Conversations with Character(s)

                                       Of the world’s approximately 7,000 languages, around
                                    4,000 are written. Though many of us, certainly those of us
                                    from secure linguistic communities, take the act of writing
                                    for granted, it takes on a deeper meaning for those of us from
                                    linguistic communities that are under threat or for whom
                                    writing is tied to a more integral part of our sociocultural
                                    identity. Jost Zetzsche’s just-published Characters with Char-
                                    acter pays homage to some of these endangered scripts, for
                                    example.
                                       The internet exposes us everyday to the more famous chil-
                                    dren of the Egyptian hieroglyphs: languages that use some
                                    form of the Latin alphabet — English chief among them —
                                    form the majority of written internet content, with Cyrillic,
                                    Greek, and Arabic and their variants also well represented.
                                       Less well represented, by several orders of magnitude, is the
                                    bichig, the traditional Mongolian script largely supplanted
                                    since the post-WWII era by the Cyrillic script in the Republic
                                    of Mongolia.
                                       It’s in this context that Tim Brookes, founder of the Endan-
                                    gered Alphabets Project, set out to create a game that would
                                    help preserve and expand knowledge of this unusual and
                                    ornate script. I spoke with Tim about Ulus, the strategy card
                                    game he and a team of dedicated linguists, calligraphers, and
                                    game designers are developing. It’s a fascinating and wide-
                                    ranging discussion that touches on language, game design,
                                    localization, and social responsibility. See the full interview
                                    on our website, multilingual.com.

                                      Michael Reid: What is the Endangered Alphabets Proj-
Tim Brookes                         ect and how did it come to be?
                                      Tim Brookes: I’m going to answer that question back-
                                    wards… in 2009 I started carving pieces of text in indigenous
                                    and minority writing systems. I discovered to my amazement
                                    that I was pretty much the only person who was doing this,
                                    which was sort of a horrifying thought because I have no back-
                                    ground as a linguist or anthropologist. I exhibited [my carv-
                                    ings] in May 2010 and the response I got was extraordinary.
                                      The reason I started the Endangered Alphabets Project as
                                    a nonprofit organization was because I began collaborating

 12            May | June 2021
REC A PS

with someone who was a member of an indigenous minor-               that Cyrillic be used. In the Russian province of Buryatia,
ity in Bangladesh and was trying to reintroduce teaching in         which is a Mongol region, virtually no people there can read
indigenous mother languages In the Chittagong Hill tribes           and write the Mongol script, even if they speak the Mongol
of Bangladesh, and it was clear that at any moment he could         language, because they’re so used to reading and writing in
have been arrested or disappeared. So I thought I wanted to         Cyrillic. So it’s a perfect example of the danger that could
collaborate with him and help in any way I could, and that          befall even such a substantial culture as the Mongols, when
wound up being a project that is still going on today.              a more powerful neighboring or regional culture dominates
                                                                    them. In the country of Mongolia, they too have been using
   MR: What is Ulus and how do you play it?                         the Cyrillic script since the Second World War, roughly. Ironi-
   TB: There are two to six players. Each player blind chooses      cally the only place where the vertical traditional bichig script
a god card. This is based on the notion of the Greek gods,          has been safe up until now was in southern/Inner Mongolia.
on the idea that the dramas of the gods were played out on          Interestingly enough it gained from its relationship with
the human chess board. Each of these gods has a different           Chinese writing and calligraphy practices, and also Tibet was
vision for the future of the Mongol lands. They’re in competi-      very influential because of course this was an area that was
tion to gather the assets and strength necessary to create that     very influenced by Tibetan Buddhism.
particular vision of the ulus, the Mongol lands. Each player
then chooses a champion, this is the human representative              MR: Who did you work with while developing the game?
of the chosen god’s ambition. Some of these champions are              TB: I was really fortunate in three respects. One is that for
historical figures, some are quasi-historical, some of them are     11 years I taught at Champlain College in Burlington, Ver-
mythological features.                                              mont, which had one of the more developed game programs
   Each of the champions has certain abilities that you choose      in the country, so I had already met a number of people who
to play at different points in the game. The champions travel       had practical experience with game design, game art, things
in a caravan around the Mongol lands, going from one sacred         like that. I was the advisor for one student’s MFA thesis, an
site to another. When they’re at each sacred site they have the     iPad drawing game which was meant to teach Mongolian cal-
chance to battle a monster for an asset. Each of these asset        ligraphy. And because he knew about my interest in endan-
cards are valuable either to any of the ulus ambitions, like the    gered alphabets I was an advisor for him.
horse card, or are suitable only to one specific quest. So you’re      Additionally, my friend in Poland [whom I’m working with
trying to collect these cards, you’re trading them, fighting for    on the game] has a number of contacts, including a friend
them, and having completed the circuit around the Mongol            who is a professor of Mongolian language and Mongolian
lands you’re learning about real places in Mongolian history.       studies in Warsaw and whose husband is a preeminent Mon-
At the end the champion who has the most assets cards plus          golian calligrapher. They were really helpful in terms of being
the most strength is the winner.                                    a cultural resource and they’ll be translating the rules. They
                                                                    put me in touch with a couple of other people, including a
  MR: Why did you make the game?                                    Polish woman who’s one of our game artists, as well as people
  TB: About four years ago I thought, if you’re going to            who are involved in Mongolian teaching and calligraphy.
revitalize a language or a script you have to begin with chil-         The third thing I was lucky with was Twitter. So many
dren. And if you’re going to begin with children you’re going       of my contacts have come out of me just reaching out on
to have to begin with games. I started creating a number of         Twitter. I even have someone who has promised me that the
games for various other contexts, and that’s been going ever        women in his village will sew the game mats for us.
since.                                                                 In that sense, I don’t have a conventional development
  The Chinese government announced that starting in Sep-            team. One of them did his MFA thesis on a game that
tember 2020 certain lessons in Inner Mongolia — Mongolians          involved studying Mongolian culture to a very considerable
call it southern Mongolia to show that it’s part of their region;   degree, and the other one has been to Mongolia, his father
the Chinese call it the Inner Mongolia Autonomous Region            was a diplomat there. He teaches Chinese at a university in
— would henceforth be taught in [Mandarin] Chinese.                 Warsaw, so he knows this landscape extremely well. At the
  It was clear that this was a situation for the Endangered         heart of all of this is me, and I’m totally aware of how little I
Alphabets Project, because you have a clear and present dan-        know. I lead with my naivete and ignorance.
ger, a culture that is well established, and one that is closely
identified with its language and its unique, vertical script.         MR: Were there any challenges you faced while creating
  The degree to which the Mongol people are identified with         the game?
their script can be seen by the fact that, when the Russians          TB: I hope that what we’re doing is not creating an ongoing
took over a slab of western Mongol territory, they insisted         business enterprise, but creating a resource that can then be

                                                                                          May | June 2021                      13
RECAPS

taken over by other entrepreneurs, especially in Mongolia. It
would be great to have a Mongolian publisher for this game.
The government of Mongolia has made a pledge to reintro-                 “From the point of view of the dominant
duce the traditional bichig script within the next four years.           culture localization is “how can I sell more
There are actually these great Youtube videos being made by
                                                                         Mercedes in Indonesia?” But for a more
the president of Mongolia, who is learning the script and has
the humility to be able to show what he’s learned each week on           sophisticated understanding of localization,
a blackboard. I’ve been hoping we can make connections there             especially if you want to actually be successful
but we haven’t been able to.                                             selling Mercedes in Indonesia, you’ve really
   As for other challenges…when we were doing the Kickstarter
it became really clear that we’re being supported by two very
                                                                         got to understand what Indonesia is. 17,000
different communities who overlapped a little. One was the               islands, six major religions, at least 400
community that typically supports the Endangered Alphabets               different languages and 15 different writing
Project; people who are interested in language and social jus-           systems, and so when you start drilling down
tice and similar issues. The other community was the gaming
community, and the gaming community uses Kickstarter in a                to that level and saying “oh, this is what
very different way to the way in which I’d been using it.                localization means” then you’re there with the
   Another difficulty, especially thinking in terms of what              Endangered Alphabets Project.”
MultiLingual readers want to know, is that this is a game that
has been developed almost entirely by people who cannot
speak Mongolian or Chinese or Russian, and can’t even read
the script that we’re working with. I’m very good at carving            MR: This reminds me of Lebanon, where there’s a move-
this stuff and producing artwork out of it, but that doesn’t          ment, which can admittedly go in odd directions in other
mean that I can read it. That means that the language part,           areas, to recognize the multiethnic character of the country.
the translation of the rules, figuring out how much informa-          There are a lot of people who don’t like writing Lebanese
tion we need to give to players in the west about the language        Arabic in the Arabic script because, for them, it’s the script of
or the culture, is still an interesting issue. Some people have       the oppressor, and they prefer the Latin alphabet because it’s
contacted me and asked if they can learn Mongolian by play-           more neutral for them.
ing this game, and the answer is no. You can learn a lot about          TB: [Companies sometimes] try to create these symbols of
Mongolian culture and history and tradition and mythology,            peace or unity, but in doing they wind up doing what I call
and you can pick up the basics of the script and my hope is it        a [US] State Department thing, which is to say “who’s ‘in
will fire people’s interest to learn it, but the game that involves   charge’ in this country? OK, we’ll deal with them.”
learning Mongolian is the card game that my German friend
is still working on.                                                    MR: Thereby stripping the agency or recognition from
   Another one of the phrases I use about the Latin alphabet          indigenous or minority populations.
— and also the English language — is that they developed                TB: Yes, and of course this also touches on the issue of
supremacy not because of any inherent qualities of the lan-           localization. From the point of view of the dominant culture
guage or the script, but because at crucial moments in history        localization is “how can I sell more Mercedes in Indonesia?”
we had more lawyers, guns, and money than anybody else. It’s          But for a more sophisticated understanding of localization,
amazing how standard it was, and even to some extent still            especially if you want to actually be successful selling Mer-
is, in linguistics to talk about the Latin alphabet as being the      cedes in Indonesia, you’ve really got to understand what
most efficient alphabet, and it’s total hogwash. There was this       Indonesia is. 17,000 islands, six major religions, at least 400
notion that was very popular in the 19th century among Brit-          different languages and 15 different writing systems, and so
ish missionaries, and is currently popular among American             when you start drilling down to that level and saying “oh,
missionaries, that we should create orthographies for people          this is what localization means” then you’re there with the
that are based on the Latin alphabet. But the Latin alphabet          Endangered Alphabets Project.
is the alphabet of their oppressors. It’s the alphabet of the
conquerors. But there are also instances where people have              MR: What has the reception to the concept of the game
adopted the Latin alphabet because it just so happened that           been so far?
the people who their oppressors were not European. So it’s like         TB: It’s really the Endangered Alphabets Project in minia-
no, we’re going to go with the Latin alphabet because those are       ture, in some respects. I constantly have ethical and linguistic
not the bad guys from our point of view.                              questions about everything that I do, and others do as well.

 14                      May | June 2021
REC A PS

May | June 2021          15
RECAPS

But oftentimes I’ll do a carving and post it on Facebook and        great. I like the idea that this is going to change the view
somebody will say, “thank you for showing and respecting            of Mongolian culture to gamers who are only used to seeing
our beautiful writing.” The response that I’m getting from          Mongols as being enormous brutes with those wonderfully
people in Mongolia and from the Mongolian diaspora is that          complex weapons that game artists come up with. Mongolia
they are just super enthusiastic that somebody isn’t trying to      is a Buddhist country with a wide range of deities. The word
silence or demonize them. The danger there is that I take that      ulus means empire or nation or land. When Genghis Khan
as sort of a carte blanche, and I don’t realize that one of the     created this enormous empire that went from Bulgaria to
characters I created is not actually a god, or is not Mongolian,    Korea, they actually stopped and said, “what do we want to
or is insulting to the Mongolian people. Talk about a localiza-     do with this? What do we want to be, as a people?” There
tion issue! And of course people in the gaming industry know        was a division between people who thought we are at heart a
this really, really well. The more characters you have, the more    nomadic people, and people who had been to the city states of
active the game is and the more it moves around, the more           Europe and western Asia and said “that’s what civilization is.”
localization you have to think about. Because every one of          So the game is based on this competition for the Mongolian
those steps needs to have some authenticity or at least not be      soul, this question of “who do we want to be”? Mongolia is
obnoxious.                                                          still the most sparsely populated major country in the world.
                                                                    And that’s why that wide open sky, that we in the US would
   MR: Who do you hope shows an interest in the game?               think of as part of Wyoming or Montana, is a spiritual thing
   TB: I would love it if the government of Mongolia got            for them. The question is, is that who we are, or are we the
interested in this. They face a huge task; they’re trying to        major urban center of Ulaanbaatar?
teach and use a script in an entire country that has not used
it essentially within living memory, so the resistance is going       [ML]Ulus is scheduled to be released in June of 2021. You
to be enormous. Anything they can do to make this a more            can find out more about the Endangered Alphabets Project at
fun, domestic, intergenerational thing I think that would be        endangeredalphabets.com.

                                          FOR MORE NEWS, UPDATED DAILY, VISIT
                                            w w w. m u l t i l i n g u a l .c o m /n ews

 16                     May | June 2021
C A LENDA R

                                     UPCOMING
                                   INDUSTRY EVENTS

MAY
05-26                                               1 2 -14
El Congreso Fragmentado para empresas y             ITI Conference 2021: An unprecedented
profesionales de traducción                         opportunity
  Gespoint, RWS Trados, Estudio Sampere               Institute of Translation and Interpreting
  Online                                              Online
  aneti.es/congreso-online                            iti.org.ukߜ

06                                                  14 -1 6
GlobalSaké The PARLAmINT: Multilingual              NAJIT 42nd Annual Conference
content strategy: transcreation, culturalization,     NAJIT
regionalization and local content origination         Las Vegas, NV, USA
  GlobalSaké                                          najit.org/najit2021
  Online
  globalsakegrowth.com/the-parlamint
                                                    27
The Future of the Localization Industry - Young     #LocFromHome: the online localization
Professionals Speak Out                             conference
  GALA Global                                         SmartCat
  Live Webinar                                        Online
  www.gala-global.org/events                          smartcat.com/locfromhome-conference

0 6 - 07
Elia’s Focus on Executives                           All events are subject to change.
  Elia
  Elysium Resort & Spa | 5 star hotel in Rhodes,
Leoforos Kallitheas, Faliraki, Greece
  elia-association.org/our-events

                                                                       May | June 2021                17
REVIEW

An Introduction to
Religious Language
Textbook-style exploration of the
language used to appeal to religious
sensibilities

                                                  Katie Botkin
                                                  Katie Botkin is the editor-in-chief of MultiLingual magazine.
                                                  She has edited for the localization industry since 2008, has
                                                  a master’s in linguistics, and grew up in a deeply religious
                                                  US microculture.

                                                    An Introduction to Religious Language is exactly
                                                  what it purports to be: a text introducing readers to
                                                  theolinguistics, or the study of religious language.
                                                  Written more for students of linguistics than laypeople,
                                                  it is dense, with some sentences packing a particular
                                                  punch. “All humans participate in sacred-making,”
                                                  writes Valerie Hobbs in summary.

                                                     Hobbs, a senior lecturer in applied linguistics at the UK’s
                                                  University of Sheffield, has spent many years studying the
                                                  topic and interacting with religious texts in various capaci-
                                                  ties — including some that made her a target of harassment.
                                                  In 2016, for example, she made headlines in the Evangelical
                                                  world when, along with blogger Rachel Miller, she uncovered
                                                  plagiarism in a book authored by Randy Booth and Douglas
                                                  Wilson, an already-controversial pastor. The publishing house
                                                  that had put out the book, affiliated with Wilson’s church,
                                                  contacted Hobbs's dean and accused her of “piracy” for hav-
                                                  ing run the book through plagiarism software. Both the dean
                                                  and the software company defended Hobbs's use of the tool.
                                                  “For as long as I have been an academic I have focused my
                                                  work on examining the ways in which seemingly inclusive large
                                                  institutions, both religious and seemingly secular, establish
                                                  community boundaries,” Hobbs writes in her preface.
                                                     Hobbs has a background in writing novels for young read-
                                                  ers, so her writing tends to be more relatable than many of
                                                  the linguistic textbooks I’ve come across in the past. An Intro-
                                                  duction to Religious Language itself focuses on how religious
      An Introduction to Religious Language:      language works. For example, the phrase “Tilda Swinton is
      Exploring Theolinguistics in Contemporary   divine” contains religious language despite the fact that it is
      Contexts, by Valerie Hobbs. Bloomsbury,     not about a religious topic. Religious language can be familiar,
      2021. $36.95. Paperback, 240 pages.
                                                  and even comforting. “Numerous studies have found that not
                                                  only is religion a means to cope with mystery, but religion and
                                                  religious language are also a powerful anti-stress mechanism,”

 18                    May | June 2021
R EV IEW

Hobbs says. “Articulating beliefs about what we hold sacred is
powerful means by which we cope existentially.”
   Hobbs points out that overt religious language is regularly
used in Alcoholics Anonymous, for example, even though
the organization is secular and not attached to any particular
religion.
   Primarily, however, Hobbs focuses on religious language
that is more intentional, considering specific examples span-
ning soap advertisements to religious letters, beginning each
exploratory chapter with a text illustrating the chapter theme.
   For example, chapter 3 focuses on the function of religious
language and begins with a 2006 billboard in Atlanta, Georgia,
declaring “JesUSAves: HE WILL HEAL OUR LAND.” Hobbs                      Figure 1: Religious language in UK politics.
breaks down the underlying messages present in this brief text,
from the implication of the pronoun “our” to the question                politics isn’t relegated to the United States — Hobbs points
of what exactly the land is supposed to “heal” from. And, of             to a 2019 Boris Johnson Brexit quote demanding “LET MY
course, there’s the literal blending of Jesus and the USA.               PEOPLE GO” (Figure 1). The implication is that Johnson is
   “Religious language in politics in the United States is well-         Moses, and the political opposition is Pharaoh. In the UK,
established and linked to the notion of civil religion, a set of         says Hobbs, religious language is creeping more overtly into
shared beliefs, symbols and rituals which a society uses to make         politics in ways it did not in years prior — and in ways that are
sense of its past in light of a deeper reality,” writes Hobbs. This      persuasive in exactly the same ways that other forms of political
particular message connects “feelings of nationalism and Chris-          speech can be persuasive.
tian religious conviction… combining the two and imbuing
them with a subtle sense of dread. Even more troubling is the
billboard’s veiled xenophobic meaning which privileges not
just Americans but Americans of a particular religion. This
billboard seems to be saying, ultimately, that Jesus will heal the
United States from the blight of non-Christians.” Perhaps more
accurately, as someone who grew up in US Christianity of this
nature, the blight of unsanctioned beliefs and actions — which
would mean everything from swearing to gun control, critical
race theory, and promoting trans rights. The strongest part of
the message is the idea of an in-group — a specific religious
group that provides social cohesion.
   Hobbs contrasts this approach and private spirituality, using
the Burning Man community as an example. “The community
itself is founded on diversity of practice. It is the diversity itself
which brings people together.” Just because something is spiri-
tual does not make it religious, Hobbs argues.
   Religious language does a few things, Hobbs says. It can
point us to “what is important and what is not.” As a discourse
strategy, it may reveal a “dichotomous worldview,” meaning
that it splits things into groups of what is good and what is
bad, and thus “enforces sacred boundaries.” Additionally, and
perhaps most significantly for so many public uses of religious
language, speaking in religious ways “signals legitimacy” to an
in-group.
   This is why religious language is everywhere, says Hobbs,
from sports to ads to political discourse. She often touches on
politics, mentioning former President Donald Trump’s 2019
assertion that “I am the chosen one” as an example of “mes-
sianic imagery” that played to his religious base. Religion in

                                                                                                      May | June 2021                 19
COLUMN

 Behind the Scenes

 Managing Employees as an HR Expert
 Terena Bell

               Who are the people who make interpreting happen? Ask people on the street — or in our own
             industry, even — and the first response you’ll get likely would be interpreters. But the answer doesn’t
             end there. From scheduling managers to quality assurance, linguistically, an entire team backs each
             interpreter, handling the language and logistical needs that get that professional on the line. So what
             about all those other people? From marketers to office managers, interpreting companies are filled
             with monolingual and multilingual professionals who make the industry go.

                                   In this issue’s edition of “Behind the Scenes,” a MultiLingual column profiling these very
                                essential folks, meet Elizabeth Garvin, director of human resources (HR) for Portland, Oregon-
                                based interpreting giant Certified Languages International (CLI).
                                   So what does an HR director do? Process interpreter applications? Not exactly. “Our inter-
                                preter services department works directly with our independent contract interpreters on things
                                like recruitment, testing, and general inquiries,” says Garvin. “As an HR director, I oversee a
                                wide variety of things that have a great impact on the company, such as salary and benefits, com-
                                munication, employee relations, training, employment law, strategic planning, recruitment,
                                and employee engagement, to name a few.”
Terena Bell                         The HR team at CLI only works with its employees. As far as what makes those groups
Terena Bell is a reporter       different, in the United States, interpreters are typically freelance, independent contractors as
covering the language           opposed to employees. Contractors can interpret for multiple providers; employees typically
industry for MultiLingual,      work full- or part-time for only one. Contractors also control their own schedules — they’re
The Atlantic, The               allowed to turn down any given assignment, whereas employees cannot. Taxes are also processed
Guardian, and others. In        differently and contractors do not receive insurance or other benefits. The difference may sound
a past life, she owned In       negligible to those outside the country, but confuse the two and a company faces hefty govern-
Every Language, an LSP,         ment fines.
and served on the GALA              How is HR for the language industry different than other industries? “The one difference is
and ALC boards.
                                the variety of cultures I get to interact with and learn about. The actual issues that I deal with
                                on a day-to-day basis are very similar to other industries. Most [employee concerns] within
                                the workplace are not unique to the language industry.” No matter what they do, says Garvin,

 20                    May | June 2021
CO LU MN

                                                                  ELIZABETH GARVIN

people want to feel valued and know that the work they do is      until I worked for CLI,” says Garvin. “I wanted to work for a
meaningful.                                                       company that made a difference and I found that in CLI and
   As for why the language industry even needs HR profes-         in the language industry. Knowing that each call or video ses-
sionals, Garvin says “Every industry needs HR professionals!      sion is touching someone’s life, helping people connect to one
As a company grows, HR becomes much more than employee            another, or delivering vital information is very rewarding. We
paperwork. HR helps guide and grow the business by looking        are not selling widgets; we are helping people and businesses
at all aspects of the organization from people to finances. HR    connect.”
handles what just might be your largest expense — payroll and         CLI’s work has also made a difference for the language
benefits — and employee retention.”                               industry itself. Before his death in 2016, founder William
   In addition, HR can be “the heart and soul of a company,”      Graeper lobbied US federal and state governments for lan-
says Garvin. “Having an individual or team who can support        guage company rights, played a pivotal role in ASTM F43
managers and team members through challenges and celebra-         standards discussions, and mentored countless translation
tions on the job or outside of work is only going to make         company owners, including this reporter. CLI was also a
your company stronger. The more meaningful engagement             founding member of the Association of Language Companies
you have with your employees, the more they are going to          (ALC), a US-based association that offers a service award in
know that they are valued and needed. This may just sound         Graeper’s name today — most recently given to Bill Rivers,
like feel-good stuff but in actuality, when employees are taken   our January/February profilee.
care of, are valued, and know that the work they do has mean-       Like so many in the industry, Garvin encounters her share
ing, they are more productive and tend to become longer-term      of surprise. The biggest misperception when non-industry
employees. This all translates to a stronger bottom line.”        people find out where she works has to do with language.
   CLI is the only language company Garvin has worked for,        “The first thing that I’m always asked is how many languages
and she’s done so for almost 20 years in a few different roles.   do I speak. It surprises people when I tell them I only speak
She began in the accounting department when CLI only had          English.”
about ten employees. At the time of interview, there were 212.      As for those inside the industry, people are surprised to learn
“Over the years, I took on more responsibilities and helped       how many facets of the business HR touches. “We work with
wherever a hand was needed. I found that my passion was           every employee in each department. We help guide and shape
within the HR field.” She became HR certified by the Society      the culture of our organization, balancing both employee and
for Human Resource Management in 2015.                            company needs. We are an integral part of everything from
   “I didn’t know I wanted to work in the language industry       party planning to disciplinary actions,” says Garvin.

                                                                                        May | June 2021                      21
COLUMN

 Rules of the Trade

 AI and Technology in Healthcare
 Christophe Djaouani

               The previous edition of “Rules of the Trade” examined the role of patient centricity in clinical trials.
             I’d like to flip the coin and delve further into the growing role of AI and technology in healthcare.

                                   One element that will drive the success of all up-and-coming developments in healthcare is
                                 the involvement of the language industry — it has a lot to offer to the healthcare sector by way
                                 of technology to speed up and enhance many of the processes where language is involved (hint:
                                 everywhere). This unrealized potential is on its way, especially with the events of the past 12
                                 months resulting in the rapid shift towards digital in all industries. Adoption of AI and technol-
                                 ogy is understandably accelerating.

                                 Barriers to the adoption of AI and tech
                                   One could say that the life sciences and the healthcare services sectors are notoriously slow
Christophe Djaouani              on the uptake of new ways of working, but this is not without good reason. To safeguard their
Christophe Djaouani is an        patients, these sectors are heavily regulated, often weighed down with onerous regulations and
executive and international      processes.
strategist specializing            Healthcare service providers and insurers have come under increased regulatory pressure in
in communication and             recent years to provide language-appropriate materials and services for individuals who do not
localization for regulated       understand the native language of the country they are in. This requirement further amplifies an
industries. Prior to this he     already major pain point — time-to-market.
joined SDL in 2018 with the        Thanks to the transition to digital communications (web, social, video, print, and so on), we
acquisition of Donnelley         have witnessed an unprecedented explosion of content volumes over the past decade, leaving
Language Solutions,              many organizations struggling to keep up. And in this new context, manual translation alone is
where his teams provided         simply no longer an appropriate response.
multilingual communication
solutions to the finance,
                                 The post-COVID landscape
investor relations, legal, and
                                   In an unexpected twist of events, the barriers listed above were upended when COVID-19
life sciences industries.
                                 forced the life sciences and healthcare sectors to embrace new ways of working. Adoption of
                                 medical technology and pharmaceutical services was boosted by the COVID-19 crisis through
                                 the sudden increase in production of diagnostic testing supplies, personal protective equipment
                                 (PPE), and acute care devices such as ventilators. Over the last year, the entire world has been
                                 holding its breath waiting for the successful development of vaccines, which have thankfully
                                 arrived and are being administered at varying rates around the world.

 22                     May | June 2021
CO LU MN

  In addition, the acceleration towards telemedicine and tele-
health models means we are arriving at a new state, known                      GLOSSARY AND TERMS FOR
as the new health economy, earlier than predicted. Language                       FURTHER LEARNING
technology can be a major part of that transformation.
                                                                           To familiarize yourself with the possible applications
                                                                           of language AI and technology in content manage-
Applications of language AI and technology                                 ment for life sciences, here are some key terms.
   Patients justifiably prefer to receive information in their
native language, which means companies must get their                      Machine translation: The field of computational
content translated into multiple languages — for example,                  linguistics that automates translating text from one
                                                                           language to another.
outreach materials to assist with patient recruitment and reten-
tion. Recruitment has seen an increase in the use of e-consent             Statistical machine translation (SMT): This approach
software to facilitate and enhance the enrollment process, so              replaced rule-based translation (RBT) and example-
this is an obvious place for large scale gains to be made with             based translation (EBMT). SMT generates translations
                                                                           based on statistical models that are trained over time
technology.                                                                by comparing parallel texts of a source language and
   Another area is clinical trial design and optimization. Par-            a human-translated target language.
ticularly in the last year or so due to COVID-19, virtual clini-
cal trials have become an alternative in response to limitations           Neural machine translation (NMT): NMT can
                                                                           be applied to any language pair and adapted to
on physical interactions. Service providers in this area are using         particular writing styles and formats, such as those
operational data to drive AI-enabled clinical trial analytics and          applied in pharmaceuticals, medical devices,
language technology has a critical role to play in ensuring the            diagnostics, and other regulated documents. One of
correct interpretation of captured findings.                               its disadvantages is lack of consistency, which is a real
                                                                           issue in healthcare where accuracy is a must.
   A third example is the use of language technology to optimize
patient monitoring and medication adherence, especially with               Post editing: An additional step undertaken by a
the transition to provision of healthcare via digital platforms.           human translator to review and correct any errors
Rapidly delivering accurate multilingual content here would                in a piece of machine translation by comparing the
                                                                           original and the machine output.
directly improve the ongoing health management of individu-
als through more accurate diagnoses of their conditions. This              Natural language processing (NLP): NLP is a branch
impact cannot be overstated.                                               of AI that enables machines to understand human
                                                                           language. This is what allows us to create chatbots or
                                                                           apply spellchecks to our work.
What’s in it for the end customer?
  LSPs are pouring huge resources into the R&D of language                 Reinforcement learning: Useful when dealing with
technology with the aim of alleviating several key pain points             enormous volumes of data, RL is an effective way to
for customers:                                                             train an NMT system. RL is considered the best way
                                                                           of achieving true intelligence.
• Patient experience. The experience for a patient to whom
   materials are presented in their native language is incompa-            Conversational AI: The use of chatbots is increasing
   rable to that of an individual forced to wrestle with content           as a way for businesses to stay in touch with their
   in another language. Again, the positive impact on trust,               customers 24/7. More progress is needed to human-
                                                                           ize interactions with them, with the way language
   reputation, and ultimately loyalty and retention from this              is formulated being one of the pain points — users
   consideration is hard to measure.                                       often report that the repetitive responses make it
• Enhanced security. Adherence to critical data security regu-             clear the experience is very much automated.
   lations like HIPAA have become table stakes to consumers,
   while providers fully appreciate the risk to their reputation
   and trust from the threat of patient data being compromised.
   Secure language technology has a major role to play in the
   overall security ecosystem by which patient data is protected.      turnaround time. This can in some cases be alleviated by
• Cost reduction. Where healthcare is a paid-for service,              using machine translation with postediting, or if using an
   efficiencies through implementation of technology can be            interpreter, over the phone interpreting (OPI).
   passed onto patients and also have the added benefit of           • Increased accuracy. Currently, machine translation does not
   enabling the healthcare provider to be more competitive in          provide accurate enough results to allow it to be deployed
   the market.                                                         independently of humans, but this can be offset by adding
• Turnaround time reduction. Using a human translator or               pre- and post-editing steps, resulting in overall efficiency
   interpreter can be time-consuming and significantly increase        gains.

                                                                                            May | June 2021                            23
COLUMN

 Unpacking the Black Box

 Metrics, what are they good for?
 Absolutely...nothing?
 John Tinsley

               Edwin Starr’s 1970 Number 1 hit “War” repeatedly asks the question, “War, what is it good for?”
             and the response is always “nothing.” You could ask the same question of many things — garlic
             crushers, Roombas, automatic evaluation metrics for machine translation (MT) — apply the same
             answer, and have the majority of people agree with you.

                                   There has been a lot written about evaluation metrics and BLEU scores, and indeed this
                                 column is now another to add to the pile. However, maybe this one can serve to draw somewhat
                                 of a line under the topic. That would be a fine thing! What it will hopefully do, at least, is act as
                                 a reasonable go-to guide to explain what metrics are out there, what exactly they do, how they
                                 differ, and offer some light at the end of the tunnel of alternatives.

                                 The fundamentals
                                    You probably already have a general idea about how these metrics work, but let’s be explicit
                                 for the avoidance of doubt. Essentially, they take some output from an MT engine and compare
John Tinsley                     it to a “correct” version of the translation. This correct version is known as a reference transla-
John Tinsley is co-managing      tion and has typically (ideally) been produced by a human translator.
director of Iconic Translation      The comparison is based on some algorithm that looks at the differences between the MT
Machines, which joined           output and the reference, i.e. how well the MT correlates with a human translation. These
the RWS Group in 2020.           algorithms can range from simple ones that look at how many words the two translations have
He has worked with               in common, to more complex ones which look at character-based n-gram similarity.
machine translation over            The benefits of these metrics are clear — they are quick to run and allow us to carry out rapid,
the last 16 years in both        iterative assessments of multiple MT engines. The drawbacks of these metrics, however, require
academia and industry. In        their own section.
2009, he received a PhD in
Computing from Dublin City       The fundamental flaws
University, after which he
                                   The single biggest flaw with these metrics is the concept that MT output is compared to and
co-founded Iconic. He now
                                 judged based on a “correct” reference translation. As I’m sure most readers will agree, there’s
leads Iconic as a dedicated
                                 no such thing as a single correct translation. There are often very many ways to adequately
technology division of RWS
                                 translate a given source text. Therefore, even if it is correct, if the MT differs from the reference
Group.
                                 translation, it will be penalized. As we’ll see shortly, some metrics have tried to account for this
                                 in their algorithms with varying degrees of success. Another suggested workaround for this issue

 24                     May | June 2021
CO LU MN

is to have multiple reference translations, but that is not very   tells us that if a score is too high, the more suspicious it is, and
practical in most scenarios and is rarely used.                    maybe there was some mistake made during testing!
   The next biggest flaw with these metrics is that the scores       These flaws are openly acknowledged by developers and
they produce have very little meaning in absolute terms. So        users, and there has been a concerted effort to refine and
your Russian to English MT engine has a BLEU score of 45           improve them over the past 15 years. This has resulted in a
— what does that mean? Absolutely nothing, literally.              large number of different metrics in the field with varying
   Scores might be informative for a relative comparison, like     degrees of adoption.
if another Russian to English MT engine tested on the exact          What are all these metrics and what is the difference between
same documents had a BLEU score of 41, we could say with           them? Table 1 offers a non-exhaustive (but not far from it)
some degree of certainty that the first engine is probably bet-    attempt to explain; questionable acronyms included.
ter than the second one. But we cannot say if either of them         Other metrics that exist but are not detailed here include
are actually any good. Beware anyone trying to tell (sell) you     GTM, AMBER, BEER, GLEU, MP4IBM1, YiSi, MTeR-
otherwise!                                                         ATER, and PER.
   Scores generally aren’t comparable across different languages
or on different test documents either, so their application is     The present and future of automated metrics
quite narrow. Even when you try to use common sense, they             Despite their known flaws, these metrics are still used exten-
can confound. For example, you might think the higher the          sively. Particularly the BLEU score, which was even acknowl-
score the better, and you’d be correct. However, experience        edged by its developers to be limited. The reason for this is really

          METRIC                                MEANING                                           DESCRIPTION

                                                                                 Calculates the n-gram precision of the MT, i.e.
                                                                                 how many (groups of) words does it match
 BLEU                        BiLingual Evaluation Understudy                     from the reference in the right place, with a
                                                                                 “brevity penalty” for translations that are too
                                                                                 short.

                                                                                 Variation on BLEU that accounts for
 SacreBLEU
                                                                                 differences in tokenization and normalization.

                             National Institute of Standards and                 Same as BLEU, except certain n-grams are
 NIST
                             Technology Metric                                   given more importance than others.

                                                                                 Calculates the harmonic mean of unigram
                             Metric for Evaluation of Translation with           precision and recall, with more importance
 METEOR
                             Explicit Ordering                                   given to recall. Option to look at word stems
                                                                                 and synonyms to account for differences.

                                                                                 Calculates the number of word edits needed
                                                                                 to make the translation match the reference.
 TER                         Translation Edit Rate
                                                                                 Proportional to the length of the sentence.
                                                                                 Designed to mimic post-edit effort.

                                                                                 Variation of TER to account for edits of
 TERp                        Translation Edit Rate — phrases                     phrases, as well as features of METEOR like
                                                                                 stemming and synonyms.

                                                                                          May | June 2021                        25
COLUMN

                                                                                      Variation on TER that counts edits on the
 CharacTER                      Character-based TER
                                                                                      character level, as opposed to words.

                                                                                      Basic algorithm to calculate the edit distance
 WER                            Word Error Rate                                       of a translation, popularized first in speech
                                                                                      recognition.

                                                                                      Calculates the F-score (harmonic mean of
 chrF                           Character n-gram F-score                              precision and recall) of character-based
                                                                                      n-grams up to 6-grams.

                                                                                      Variation on chrF to allow limits on the value
 chrF++
                                                                                      of n.

                                                                                      Uses many features in other metrics (listed in
                                Length Penalty, Precision, n-gram Position
 LEPOR                                                                                the name) but makes them tunable so they
                                difference Penalty and Recall.
                                                                                      can be adjusted for different languages.

                                                                                      Calculates similarity based on longest
                                Recall-Oriented Understudy for Gisting
 ROUGE                                                                                common subsequence of words with
                                Evaluation
                                                                                      adjustments possible for skipped words.

                                                                                      A neural framework for evaluation using
                                                                                      models trained on Direct Assessments,
                                Crosslingual Optimized Metric for Evaluation
 COMET                                                                                Human-mediated Translation Edit Rate and
                                of Translation.
                                                                                      Multidimensional Quality Metrics to predict
                                                                                      quality.
Table 1

a case of sticking with the familiar in the absence of a silver bullet   themselves, by seeing how well they correlate with human
alternative. Not to be hypocritical, I used them back in my own          assessments.
research days and we still use them at RWS Iconic. Typically,              The oracle of evaluation, however, is quality estimation. A
practitioners will tend not to rely on a single metric, but rather       perpetual hot topic in MT, quality estimation aims to predict
calculate scores from a variety of different metrics, as a kind of       the accuracy of machine translations without needing to com-
sanity check. However, important decisions, particularly when            pare them to reference translations. Not only is this very useful
it comes to buyers choosing whether to use MT (and/or which              as a product feature, it would be ideal as an absolute evaluation
MT), will additionally rely on human assessments.                        metric.
   Fortunately, all is not lost when it comes to automated met-            This will no doubt be an upcoming topic in these pages, and
rics. There is a lot of R&D effort spent on trying to create new         research in this area is ongoing to make this a reality. Until
and better metrics. As part of the annual WMT Conference,                then, use automated metrics in the wild with caution! Correctly
there has been a shared task related to new metric development           interpreting what they are telling you takes a dollop of experi-
running since 2008, with the next edition in November this               ence along with a serving of intuition. Without this, a little
year. This task focuses on evaluating the evaluation metrics             knowledge can be a dangerous thing.

 26                       May | June 2021
You can also read