TRENDS AND TECHNOLOGIES IN DIGITAL MEDIA - #WEK NOWHOW - FRAUNHOFER ALLIANZ DIGITAL MEDIA
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
F R A U N H O F E R D I G I TA L M E D I A A L L I A N C E
TRENDS AND TECHNOLOGIES
IN DIGITAL MEDIA
#WeKnowHow
PREFACE We cannot change the wind, but we can set
the sails differently”. (Aristoteles)
6 Audio forensics: detecting fakes
The Coronavirus pandemic in 2020 set new quality as well as VVC (Versatile Video Codec),
8 Automatic analysis of TV and radio broadcasts priorities in our business, public and private the successor of HEVC, for HD/UHD content
lives. In addition, it teaches us to rethink our streaming. Creating free viewpoint volumetric
10 Artificial intelligence and its drawbacks usual workflows, adapting them to new re- video from real-life scenes is one of the future
quirements and making them more flexible. solutions to go beyond fixed display presenta-
12 Mining Platform: automatic multimedia content analysis at scale tion with static viewpoint.
In the media industry an unprecedented de-
16 AI-Powered Encoding and Optimized Streaming mand for audiovisual content and communi- AI (artificial intelligence) and its power for
cation over the Internet calls for new technol- streaming analytics, coding, and especially for
20 JPEG XS – Lowest latency for multiple en-/coding ogies, codecs, analytics and solutions for crisp data mining and audiovisual quality control,
images and sounds to enhance our experiences fake detection and so on, is among the key
22 JPEG XS – Advantages in brief when we chat, discuss or exchange topics or technologies that will influence the way of
share content. And even more with virtual content production and transmission.
24 VVC brings video compression to new levels trade shows, we test how to present new
developments to a broader audience. IBC2020 will show many ways to enhance
26 Deceptively lifelike: realistic virtual people! media workflows even under changing condi-
Fraunhofer Digital Media Alliance accepts the tions. We hope to see you in our virtual booth
28 LC3/LC3plus: A plus in audio quality and transmission robustness challenge: We provide technological insights at www.ibc2020.digitalmedia.fraunhofer.de
for wireless accessories via the IBC 365 platform and set up our own and as usual, please enjoy reading our digital
virtual booth for IBC 2020. Major topics here trend brochure.
30 360 Reality Audio Immersive Music Services and are MPEG-H authoring tools to realize immer-
Devices Announced with MPEG-H sive and personalized sound adaptable for Sincerely, yours
every surrounding and device. Video codecs
32 ABOUT US and implementation as JPEG XS for transfer- Dr. Siegfried Foessel
33 Fraunhofer Digital Media Alliance ring up to 8k video over IP in production Spokesman Fraunhofer Digital Media Alliance
2 3
– Half of the world‘s digi-
tal broadcast TV sound and
– Every second bit of the – Nearly 50 percent of the most digital radio systems – xHE-AAC is natively
worldwide data traffic bits transported via the are enabled by Fraunhofer integrated in Android
can be sent thanks to Internet are compressed using IIS audio codecs. and standard codec
Fraunhofer technologies. the video coding standard – The Apple and for Digital Radio
– mp3, AAC and HE-AAC H.264/AVC, co-developed by Android ecosystems Mondiale.
are today in all consumer Fraunhofer HHI. are based on AAC,
electronic devices, PCs for instance music
and smartphones. distribution, Facetime,
Airplay, or Carplay.
FRAUNHOFER DIGITAL MEDIA ALLIANCE
SOLUTIONS THAT CREATE THE FUTURE OF MEDIA
– SpatialSound Wave:
– MPEG-H Audio is First to market in
used in Korean UHDTV object-based audio.
since May 2017.
– Almost every enter-
tainment device uses – The easyDCP soft- – AudioID: – The 3GPP communica-
audio and video codecs ware suite for movie Pioneer in audio tion codec EVS is in use – World leading in speech
of Fraunhofer. content mastering and matching since 2002. worldwide, for example recognition transcribing
packaging is one of in Europe, USA, China, audio and video files in real
the most reliable tools Japan and South Korea. time with a vocabulary
in the industry. of over 2 million words,
processing any voice/dialect.
4 5
Fraunhofer IDMT
AUDIO FORENSICS: DETECTING FAKES
The CEO’s voice came through loud and clear: his instructions were to
transfer a large sum of money to a certain account. This order, how-
ever, was faked – something that was only discovered once it was too
late. At Deutsche Welle, the German public international broadcaster,
falsifications are set to be exposed more quickly in the future: Within
the project “Digger”, Fraunhofer IDMT is integrating its audio forensics
technology into DW’s content verification platform “Truly Media”.
“That’s a great quote! But is it real?”– it traces that can be detected with forensic
is not easy for journalists to expose forge- tools,” according to Patrick Aichroth,
ries, as fake audio and video material is Group Manager at Fraunhofer IDMT.
becoming increasingly credible. Fakes can
be created by means of skilled editing, In the joint project “Digger”, funded by
but also by imitating the speech and into- Google DNI, Fraunhofer IDMT is integra-
nation of a speaker very convincingly ting their audio forensics technologies into
using artificial intelligence. the web-based content verification plat-
form “Truly Media”, which is developed by
Researchers at Fraunhofer IDMT are Deutsche Welle and Athens Technology
working on uncovering forgeries of both Center. From 2021, with the help of this
kinds: By analyzing “footprints” that are Fraunhofer IDMT “a detective’s toolkit for
added during the recording – such as cha- fakes,” journalists will be able to carry out
racteristic traces of microphones – or by analyses of audio material and detect Contact:
identifying inconsistencies in the material manipulations. Patrick Aichroth
that result from editing. This is because patrick.aichroth@idmt.fraunhofer.de
“every content processing step leaves +49 3677 467-121
6 7
© stock.adobe.com
AUTOMATIC ANALYSIS OF TV AND
RADIO BROADCASTS
We are all used to matching apps that can help us find out which
songs are playing on the radio, but researchers are also working on
a similar technology that can help broadcasters and media compa-
nies to track metadata and find partial overlaps and duplicates —
it’s known as partial matching. Patrick Aichroth, Group Manager at
Fraunhofer IDMT, explains what it’s about.
What does the term “partial mat- What can this technology do that What does the future hold for partial
ching” mean? matching cannot? matching?
With conventional matching, the goal is While matching enables the analysis of We are currently running further tests and
typically to identify content, such as a when and how often a piece of content will then put the technology into a first
song or a video, using a snippet or sam- has been broadcast, partial matching can product that can be used to detect and lo-
ple of that material. With partial mat- analyze entire TV or radio programs or da- calize differences between pre-produced
ching, on the other hand, your aim is to tasets and identify partial overlaps. This material and broadcasted productions,
find any partial overlaps within a data- can be used for de-duplication, to auto- which should be available by early 2021.
set, without knowing beforehand whe- matically propagate metadata within ar- Afterwards, we plan to provide further
ther or where such overlaps exist. Both chives, or find out which parts of a pre- product versions for de-duplication and
cases of matching may sound similar, but produced item were actually broadcast, metadata / rights tracking.
there are somewhat different technolo- which is useful to create program cue
gies involved. sheets and clear rights. We have already Contact:
carried out a few test runs with broadcas- Patrick Aichroth
ters and the results have been encoura- patrick.aichroth@idmt.fraunhofer.de
ging. +49 3677 467-121
8 9
© stock.adobe.com
ARTIFICIAL INTELLIGENCE AND
ITS DRAWBACKS
Does the future belong to artificial intelligence? Can artificial
intelligence solve pressing problems facing humanity? It can cer-
tainly help – if we recognize the limits of the technology and apply
its strengths correctly. The trick is knowing how.
Analyzing radiological images and assis- with different recording quality and di- must also be applied correctly. The fact everything can be solved by adapting the AI
ting physicians in evaluating them, recog- verse content – this often leads to unex- that it’s not always obvious what the AI components to the use case. Or have them
nizing cars in traffic scenarios, even distin- pected effects.” model is used for makes the situation adapted: the researchers at Fraunhofer IDMT
guishing dogs from cats – artificial more difficult. “The most important thing will be happy to help.”
intelligence can do all of this brilliantly. So is it best to stay away from artificial is that the system is trained with data
But successes such as these are rapidly intelligence? that is representative for the applica-
raising expectations for this technology. “Certainly not! However, it is essential to tion,” says Lukashevich. „If, say, a model
define the appropriate model in advance has only been trained with high-quality
“Many people see intelligence as a system – e.g. distinguishing between dogs and audio data, it will later struggle with tele-
that thinks for itself and is very smart,” cats – and to train the model accordingly. phone-quality audio data.“
says Hanna Lukashevich, group leader at If the developer did not include zebras in
Fraunhofer IDMT. “But they’re wrong: ar- the definition of the model, the artificial What if the artificial intelligence just
tificial intelligence can only do what it has intelligence will not be able to recognize won´t do what you want?
been taught in advance – and therefore zebras either,” explains Lukashevich. Then it’s time to adapt the training data.
performs particularly well in well-cont- After all, artificial intelligence is one thing “AI isn’t magic; it’s mostly mathematics,” Contact:
rolled environments. If, on the other hand, above all else: a tool. And just as nobody Lukashevich assures us. “To put it ano- Hanna Lukashevich
they are applied to versatile content – would think of drilling a hole in a pane of ther way, there‘s always a reason why hanna.lukashevich@idmt.fraunhofer.de
such as analyzing any kind of audio files glass with a wall drill, artificial intelligence something won‘t work. By and large, +49 3677 467-224
10 11
© Fraunhofer IAIS
MINING PLATFORM: AUTOMATIC MULTIMEDIA
Step-by-step analyses thanks to The metadata – the people recognized in
CONTENT ANALYSIS AT SCALE
workflow components the image, and the speakers recognized –
Many valuable data treasures are hidden in media archives. Extracting these The individual steps involved in analyzing a can then be linked together.
treasures is still difficult – manually annotating text, images, and audio and media file are controlled by a workflow
video files with content-related metadata is very laborious, time consuming component. This has several benefits: the The system can, say, detect when a person
and can only be created for a limited amount of media files. architecture allows failed analysis steps to is visible on screen while also talking. In
Now, however, the Fraunhofer IAIS Mining Platform is capable to analyze be repeated and enables the prioritization another step downstream, the transcript
almost unlimited quantities of multimedia documents with high precision of the scheduled analyses. This is helpful can be further analyzed using text mining
and flexibility. The automatically generated metadata makes valuable when certain media data has priority over methods. This means that not only can the
content much more accessible to journalists and other media creators, as other data in the analysis, for example in system determine when which person can
well as allowing searching by topic. the case of live events. be seen or heard in a video, but also what
topics, people, places, or institutions are
A modular and extensible system The Fraunhofer team is currently develo- The workflow component can also be mentioned.
The Mining Platform has a modular struc- ping and integrating additional analysis used to model more complex processing
ture, allowing the integration of almost services, such as for detecting cuts and sequences, for example when the output To allow the Mining Platform to scale ac-
any metadata extraction method. scenes, key frame extraction, and object of an analysis service serves as an input for cording to the quantity of media data to
Components for named entity recognition, recognition. Methods developed by subsequent services. be processed, it uses a microservice-based
keyword extraction, topic modeling, and third-party providers can also be integra- architecture and is operated in a Kuberne-
semantic tagging are already integrated ted easily on request. It is also possible to For example, a video can be analyzed tes cluster. Thus, the number of instances
for analyzing text documents. train customer-specific models for the using facial recognition, while audio mi- of the individual analysis services can be
Spoken material can be automatically existing analysis services, thus replacing ning can transcribe the speech captured adapted to required workload.
transcribed with the high performance the ready-made models – an advantage on the audio track and even recognize the
Fraunhofer IAIS AudioMining solution. not offered by other systems so far. separate speakers.
Furthermore, visual related metadata
such as faces can be detected in images
and videos.
12 13
Operating the Mining Platform in a Kuberne-
tes cluster also has the advantage that it can
be operated in a private cloud or directly on-
premises. Cyber sovereignty is guaranteed
at all times. Connection via REST interfaces
means that the Mining Platform can be ea-
sily integrated into other broadcast systems.
Cross-media searching: the Mining Plat-
form successfully developed with and
deployed by ARD
The Mining Platform is being developed in a
strategic partnership with Germany’s public
broadcasting network, ARD, where it is
being used for applications such as a
cross-media search system.
Contact:
Dr. Jens Fisseler
jens.fisseler@iais.fraunhofer.de
+49 2241 141974 Icons made by Freepik from www.flaticon.com
14 15
AI-POWERED ENCODING AND
OPTIMIZED STREAMING
Nowadays, the world‘s Internet traffic is During a live playout, we analyze the video
predominantly made up of video strea- and collect existing playback metrics in or-
ming. In order to support the Internet‘s der to predict the upcoming optimal lad-
limited transmission capacities, videos are der. With the prediction, encoding
compressed during encoding. Reducing settings are then adapted accordingly.
video file sizes comes at an expense, how-
ever, when disseminating large volumes Video analytics is conducted on a per-
of streaming video via the Internet, resul- scene basis in order to adapt settings to
ting in a loss in quality. the current scene. In comparison to traditi-
Live streaming, in particular, has special onal encoding solutions, average storage
requirements with regards to the video and transfer volumes are reduced by 30%.
transmission process, and can benefit This large decrease leads to significant cost Key Benefits:
from intelligent approaches in compres- savings in the long-run, and improves the – Supports VoD and live linear content
sing content in a more efficient manner. overall Quality of Experience for the end
user. This FAMIUM Deep Encode solution – Encoding and streaming workflow is
Our FAMIUM Deep Encode solution utili- is codec and format agnostic, and the qua- dynamically optimized by collecting
zes artificial intelligence methods for au- lity prediction (VMAF) is based on several metrics with a feedback loop Format-
tomating per-title encoding for Video on extracted unique video characteristics. agnostic with established video codecs
Demand and live streaming.
– Prediction of video quality metrics Contact:
(VMAF) without the need of test encodes Robert Seeliger
robert.seeliger@fokus.fraunhofer.de
+49 30 3463 7262
16 17
Automatic analysis, classification and
metric calculation of video content by
Deep Encode
Predicted Encoding Settings
and optimal Bitrate Ladder
18 19© stock.adobe.com
LOWEST LATENCY FOR VIDEO OVER
IP TRANSMISSION JPEG XS
The standardized JPEG XS codec was developed to handle media workflows Integration of JPEG XS into cameras JPEG XS already exists as transport and file
from acquisition to distribution by using Ethernet settings and infrastructure and image sensors formats, like RTP, MPEG2-TS, JXS, MP4, and
only. Until very recently, digital image transmission for production and con- The current standardization activity is a HEIF. The standardization of JPEG XS inside
tribution could be done only by using specific interfaces such as SDI, big step forward to enable JPEG-XS for the MXF file container is under progress in
IEEE1394, or CameraLink. However, with the availability of higher bandwidth compression of RAW Bayer image data. SMPTE under the item ST 2124. With these
of Ethernet interfaces, the handling of highest-quality images over internet During this JPEG XS development phase, activities, a complete suite of formats is now
protocol (IP) in local and wide area networks was required and JPEG XS is a a PSNR gain of 5 dB in coding efficiency available for JPEG XS allowing the transport
codec enabling these requirements. could be achieved and will be included in and storage of this format in the postpro-
a new amendment. duction workflow.
An update to the new video produc- The core coding system of JPEG XS was
tion codec for professional video standardized in ISO at the end of 2018 This allows the industry to integrate the JPEG XS SDK available
A low compression of up to 10:1 allows as ISO/IEC 21122-1, the remaining parts codec into today’s cameras and image Fraunhofer IIS offers development kits for
near-transparent transmission. in 2019. sensors. It offers the use of the codec in CPU and GPU usage, as well as consulting
JPEG XS – developed to offer lowest l the complete production pipeline – from projects for integration into products to
atency for multiple encoding-decoding What is available for industry applications the image capturing to the distribution the industry. Initial implementations for
cycles and moderate computational today are the compression of RGB and encoder. It facilitates the use of the codec JPEG XS were carried out successfully,
resource requirements while preserving YCbCr images in 444 and 422 sampling in other use cases, like integration in even in 8k.
image quality at the highest level – fulfills formats with up to 12 bits per component cameras for machine vision, automotive,
these demands to facilitate production / for broadcast and prosumer use cases. or high quality surveillance, too.
contribution settings, even for 4 and 8k Some smaller extensions, like compression
images. of 420 sampling formats and lossless com- Dr. Siegfried Foessel
pression, are under development. siegfried.foessel@iis.fraunhofer.de
+49 9131 776 5140
20 21JPEG XS – Advantages in brief Transport/
Container
Type Description – Main purpose Extension
RTP RTP Payload Format for JPEG XS IP based transport
Professional formats: Support of RGB/444, RGBA/4444, YCbCr 444/422, YCbCrA (IETF draft)
4444/42224 and YCbCr 420 image formats of up to 12 bits per color component sample
MPEG2-TS ISO/IEC 13818-1:2019 AMD1:2020 Carriage of associated CMAF boxes for
precision with the option to extend it to 16 bits in the future audio-visual elementary streams and
JPEG XS in MPEG-2 TS
Highest fidelity: Visually lossless, i.e. no visible degradation, even over multiple compression/
Video over IP SMPTE 2110-22:2019 Encapsulation of compressed video
decompression cycles. The latter is known under the term “multi-generation robustness”. streams in SMPTE 2110 as RTP stream
JXS JPEG XS file format For storing of single images .jxs
Low latency: Maximum 32 lines end-to-end (compression-decompression) latency in
hardware, in special modes even subline-latency. (defined in ISO/IEC 21122-3 Annex A) JPEG 2000 syntax based
MP4 ISO Base Media File format (ISOBMFF) For storing of video .mp4
Bitrate: Exact bitrate allocation per frame slice avoiding data overshooting
(defined in ISO/IEC 21122-3 Annex B) ISOBMFF syntax based
Parallelism: Support for multiple platforms e.g. FPGA, ASIC, GPU and CPU, by high degree of HEIF High Efficiency Image File Format For storing of mixed image and video .heif
content
parallelism (defined in ISO/IEC 21122-3 Annex C)
High Performance: Real-time software implementation capability for 4k/60p formats on MXF SMPTE 2124 (FCD) For storing of video .mxf
today’s standard computers. MXF syntax based
Notes: Status is as of 01.June.2020
Low complexity: defined as a maximum percentage of a specific low-cost FPGA.
No external frame buffer required in embedded applications. In particular, individual frames Transport
Transportandand
File File
formats for JPEG
formats forXSJPEG XS
shall be decoded independently
Typical compression ratios: 2:1 – 10:1
22 23© Fraunhofer HHI
VVC BRINGS VIDEO COMPRESSION
TO NEW LEVELS
Is it possible to stream high quality video despite having a poor inter-
net connection? With the newly released (July 2020) Versatile Video
Coding (VVC) standard, now it is! Benjamin Bross, head of the Video
Coding Systems group at Fraunhofer HHI and editor of VVC, gives us
the inside scoop.
Fraunhofer HHI played a key role in So, the main benefit of VVC is the ef- What is the current status of the
developing the new H.266/VVC stan- ficiency at compressing data? standard?
dard. This is an important benefit, but not the In July, the standard was finalized by
Mr. Bross, what is special about this only one. VVC also offers flexibility, as re- ITU-T and ISO/IEC. We plan to present
new standard? flected in its name (“versatile”). It is the the first efficient implementations in
Compared to its predecessor (H.265/HEVC), first standard that is suitable for a range software at the IBC trade fair.
VVC reduces the required bit rate by of applications: from game videos or vir- The necessary hardware (e.g., chips that
50%. In other words, it is much more tual reality to video conferencing and reduce battery consumption on smart-
efficient at compressing data. Therefore, teaching. The last two have been recently phones) should be available in 2021.
even with a slow Internet connection, proven to be of critical importance in
VVC enables the streaming and viewing face of a pandemic. Each application has
of high-definition video. VVC also bene- specific requirements for the compression
fits smartphone users by reducing the algorithms, which were taken into ac- Maria Ott
consumption of data volume when count in the development of VVC. maria.ott@hhi.fraunhofer.de
streaming video. +49 30 31002-862
Benjamin Bross
benjamin.bross@hhi.fraunhofer.de
24 25DECEPTIVELY LIFELIKE:
REALISTIC VIRTUAL PEOPLE!
Your favorite sports star comes into your living room to give you pri-
vate lessons? Mixed Reality makes it possible: The chosen person is in-
serted into the real environment through mixed-reality glasses.
Dr. Oliver Schreer, Wieland Morgenstern, and Dr. Cornelius Hellge from
Fraunhofer HHI explain the tricks that make virtual humans look lifelike.
Why don’t you let us in on the secret Are special end devices needed?
of making avatars look realistic? Dr. Cornelius Hellge: If these photorealistic
Dr. Oliver Schreer: Until now, avatars have volumetric videos are rendered on the end
been created using computer animations – device, the bit rates and the demands on
but these often seem artificial. So we’ve ta- the graphics cards are too high. Together
ken a different approach. We place an actor with Deutsche Telekom, we are therefore
or an actress in the middle of a rotunda and moving the rendering to the cloud – so only
film him or her with 32 cameras, calculate a normal 2D video stream is transmitted to
the 3D information, and use this to create a the end device. This 2D video stream runs
3D model. The result is a dynamic mesh on all standard devices and on all browsers.
sequence, known as a volumetric video.
Wieland Morgenstern: In the next step, we
animate this volumetric video with a high Maria Ott,
level of detail – we use model-based anima- +49 30 31002-862
tion to change the actor or actress so that, maria.ott@hhi.fraunhofer.de
say, his or her gaze can follow the user. Oliver Schreer
This makes the contact more personal. oliver.schreer@hhi.fraunhofer.de
© Fraunhofer HHI
26 27© nyul – stock.adobe.com
LC3/LC3PLUS: A PLUS IN AUDIO QUALITY
dition to applications and devices based VoIP channels, the redundant transmission
AND TRANSMISSION ROBUSTNESS FOR
on the 2019 DECT and ETSI TS 103 634 of LC3plus voice data ensures verifiably
WIRELESS ACCESSORIES standards, LC3plus is also suitable for more stable phone calls. For DECT tele-
LC3 / LC3plus (short for Low Complexity Communication Codec) was Bluetooth. This enables manufacturers to phones, the inherent tools for forward er-
developed by Fraunhofer IIS and Ericsson. The new audio codec resol- deliver the benefits of LC3plus on Blue- ror correction in LC3plus were again spe-
ves fundamental shortcomings present in today’s wireless communica- tooth-based wireless accessories, such as cially adapted to exploit typical
tion platforms, such as Bluetooth and Digital Enhanced Cordless Tele- headsets, headphones and earbuds. characteristics of DECT links. This signifi-
communications (DECT). Its operation modes range from medium bit cantly improves call quality compared to
rates for optimal voice transmission to high bit rates for high-resolu- A plus in user experience previous codecs: uninterrupted calls can be
tion music streaming services. At the same time, the codec operates at With the introduction of the 3GPP En- made even when the handset is far away
low latency, low computational complexity and low memory footprint. hanced Voice Services (EVS) audio codec from the base station.
for VoLTE came super wideband (SWB)
audio quality for mobile phones. Users It’s not only for cordless telephony that
While LC3 is the audio codec for Blue- the required bit rate compared to other feel as though they are in the same room LC3plus offers significant improvements –
tooth LE Audio, a new audio architecture state-of-the-art technologies. The latter as the person they are talking to. Now, thanks to its high-resolution audio mode,
designed to boost the performance of helps prolong battery life and paves the LC3plus makes the equivalent of EVS users can now enjoy music streaming with
Bluetooth Audio, the superset LC3plus way for smaller products. Low latency, available for the landline network, en- high sampling rates and wide dynamic
was standardized in 2019 as ETSI TS 103 low complexity and low memory require- abling users of VoIP applications and range via wireless accessories such as
634 and is included in the 2019 DECT ments are also characteristics of both co- DECT telephones to also share in the headphones or speakers without any loss
standard. This makes LC3plus the only dec variants. feeling that the conversation is really of quality. This makes LC3plus the ideal
open standardized high-resolution audio taking place face to face. codec for these applications.
codec, reducing dependencies on specific What puts the “plus” in LC3plus?
proprietary vendor solutions. LC3 and Among other things, higher robustness LC3plus can also minimize disruptions
LC3plus cover the same bases, as in high against transmission errors, even lower during phone calls: the codec is ex- Mandy Garcia
speech and audio quality for wireless au- encoding delay, and the ability to play tremely robust when it comes to voice amm-info@iis.fraunhofer.de
dio accessories, as well as a reduction in back high-resolution audio quality. In ad- packet loss and bit errors. In overloaded +49 9131 776-6178
28 29© Adrian Murtaza – Fraunhofer IIS
360 REALITY AUDIO IMMERSIVE MUSIC Thanks to a cooperation between
Fraunhofer IIS and Sony the format will be
SERVICES AND DEVICES ANNOUNCED
compliant with MPEG-H 3D Audio, an in-
WITH MPEG-H ternational open audio standard, and opti-
The MPEG-H immersive audio format, which powers 360 Reality Audio, mized for music streaming.
makes it possible for artists and music creators to produce a spatial musical As licensors, Sony and Fraunhofer IIS will
experience. When listeners hear content produced in the 360 Reality Audio continue working to expand the available
music format, they experience an immersion into sound that transports content library, participating streaming
them onstage with their favorite artists. You can hear the music as the artist services and compatible audio devices
intended it during production. moving forward.
Playback of 360 Reality Audio content are breathtaking and we are proud to be About MPEG-H Audio
can now be enjoyed on mobile devices part of it,” said Dr. Bernhard Grill, Direc- MPEG-H, substantially developed by Fraun-
on the go and at home. Being able to en- tor of Fraunhofer IIS. hofer IIS, is the industry’s most advanced
joy popular recording artists’ latest im- audio system, supporting both immersive
mersive music mixes in any environment “The future of music is here and immer- sound and the ability for users to adjust el-
and on many 3D audio enabled sive. With Fraunhofer, we found an in- ements in the audio to their preferences.
devices will provide audiences with a dustry-leading innovation partner. The MPEG-H has been on the air since 2017 on
seamless immersive experience. open standardized MPEG-H audio codec all TV networks in South Korea under the
enables us to offer 360 Reality Audio new ATSC 3.0 standard, and it has been
“We are very pleased that Sony has cho- music on a growing number of streaming selected for new broadcast standards to
sen MPEG-H Audio as the distribution services and many playback platforms be launched in China and Brazil. Fraun-
format for 360 Reality Audio content. from various manufacturers,” said Yoshi- hofer offers MPEG-H software implemen-
The creative possibilities this offers the nori Matsumoto, Director and Deputy tations for many popular CPU, SoC and
music industry and the new level of mu- President, Sony Home Entertainment and DSP platforms. It is widely deployed today Mandy Garcia
sic experience this provides to consumers Sound Products Inc. in TV sets, premium soundbars and high- amm-info@iis.fraunhofer.de
end smart speakers. +49 9131 776-6178
30 31ABOUT US
FRAUNHOFER DIGITAL MEDIA ALLIANCE
As an one-stop competence center for as well as new cinematography, audio, and Publication Information Photo acknowledgements
digital media we provide for our custom- projection technologies, post-production, Cover picture: Fraunhofer
ers scientific know-how and the develop- distribution, and archiving. The goal of the Fraunhofer Digital Media Alliance Page 7: Fraunhofer IDMT
ment of solutions that can be integrated Fraunhofer Digital Media Alliance is to c/o Fraunhofer Institute for Page 9: © stock.adobe.com
in workflows and optimize process steps. quickly and easily help find the right con- Integrated Circuits IIS Page 11: © stock.adobe.com
tacts, partners, and suitable technology. Page 13: Fraunhofer IAIS
The members of the Digital Media network Am Wolfsmantel 33 Page 17-19: Fraunhofer Fokus
are actively working in renowned organiza- The Fraunhofer Institute members are 91058 Erlangen, Germany Page 22-23: © stock.adobe.com
tions and bodies like International Stan- – Digital Media Technologie IDMT, Page 25: Fraunhofer HHI
dardization Organization ISO, ISDCF (In- Ilmenau Concept and Editor Page 27: Fraunhofer HHI
ter-Society Digital Cinema Forum), SMPTE – Integrated Circuits IIS, Erlangen Angela Raguse Page 29: © nyul – stock.adobe.com
(Society for Motion Picture and Television – Telecommunications, Fraunhofer Digital Media Alliance Page 31: © Adrian Murtaza , Fraunhofer IIS
Engineers), FKTG (German Society for Heinrich-Hertz-Institut HHI, Berlin
Broadcast and Motion Picture), and in the – Open Communication Systems Editors © Fraunhofer-Gesellschaft
EDCF (European Digital Cinema Forum). FOKUS, Berlin Dr. Janine van Ackeren,
– Guest: Intelligent Analysis and Informa- Romana Redtenbacher,
Fraunhofer institutes in the Digital Media tion Systems IAIS, St. Augustin Robert Seeliger
Alliance jointly offer innovative solutions
and products for the transition to the digi- Contact Layout and production
tal movie and media world of tomorrow. Fraunhofer Digital Media Alliance Ariane Ritter
The Institutes in the Alliance are available Angela Raguse M.A.
as renowned contacts and partners for all Phone +49 9131 776-5105
of the digital topics connected to digital alliance-dc@iis.fraunhofer.de
media, digital movies, and standardization, www.digitalmedia.fraunhofer.de
32 33You can also read
