Conceptual Closure Elicited by Event Boundary Transitions Affects Commercial Communication Effectiveness - Frontiers
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
ORIGINAL RESEARCH published: 07 April 2020 doi: 10.3389/fnins.2020.00292 Conceptual Closure Elicited by Event Boundary Transitions Affects Commercial Communication Effectiveness Richard Silberstein 1,2* , Shaun Seixas 2 and Geoffrey Nield 2 1 Centre for Human Psychopharmacology, Swinburne University of Technology, Hawthorn, VIC, Australia, 2 Neuro-Insight Pty Ltd., Hawthorn, VIC, Australia While our experience of the world may appear continuous, recent evidence suggests that our experience is automatically segmented and encoded into long-term memory as a set of discrete events. Event segmentation is an important process in long- term memory encoding with evidence pointing to experiences occurring around event boundaries being better recognized subsequently. Neuroimaging studies have shown increased activity in the hippocampus and other nodes of the default mode network Edited by: (DMN) when encountering an event boundary. We have previously demonstrated that Thomas Zoëga Ramsøy, Neurons Inc, Denmark the steady state topography (SST) measure of brain activity at a left inferior frontal scalp Reviewed by: sites is correlated with the strength of long-term memory encoding. More recently, we Steven Bellman, have noted that event boundaries occurring in naturalistic stimuli such as television University of South Australia, Australia Kathleen J. Kennedy, advertising trigger a transient drop in activity at the inferior frontal scalp sites, an effect University of Arizona, United States we have termed Conceptual Closure. In this study, SST measures of brain activity *Correspondence: were recorded in 50 male participants as they viewed a first-person journey through a Richard Silberstein 10-room virtual art gallery. We hypothesized that the transition from one room to another rsilberstein@swin.edu.au; rsilberstein@neuro-insight.com.au would serve as an event boundary which would triggers increased hippocampal and DMN activity while correspondingly decreasing activity in task positive networks in the Specialty section: vicinity of the inferior frontal cortex thus eliciting Conceptual Closure. A permutation test This article was submitted to Decision Neuroscience, confirmed the hypothesis in that the appearance of the door between gallery rooms a section of the journal was associated with Conceptual Closure in that we observed a significant drop in brain Frontiers in Neuroscience activity at the left hemisphere inferior frontal scalp site at this point in time. Finally, we Received: 16 December 2019 Accepted: 13 March 2020 illustrate the real-world impact of Conceptual Closure by considering the commercial Published: 07 April 2020 effectiveness of a television advertisement that exhibited Conceptual Closure at points Citation: of branding. The television advertisement was broadcast before and after it was re- Silberstein R, Seixas S and edited to minimize Conceptual Closure at the time the advertising brand was being Nield G (2020) Conceptual Closure Elicited by Event Boundary Transitions featured. Minimizing Conceptual Closure at the time of branding and key message was Affects Commercial Communication associated with significant increased commercial effectiveness of the advertisement. Effectiveness. Front. Neurosci. 14:292. Keywords: event boundary, long-term memory encoding, steady state visual evoked potential, prefrontal cortex, doi: 10.3389/fnins.2020.00292 conceptual closure Frontiers in Neuroscience | www.frontiersin.org 1 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding INTRODUCTION external perception and the engagement of internally oriented networks (Nakano, 2015). While our experience of the world may appear continuous, Earlier observations of enhanced long-term memory encoding recent evidence suggests that our experience is segmented for events occurring around the times of event boundaries led and encoded into long-term memory as a set of discrete researchers to examine the relationship between hippocampal events (Zacks and Swallow, 2007; Kurby and Zacks, 2008). activity and event boundaries (Ben-Yakov et al., 2013, 2014; Such segmentation appears to be automatic and operating on Ben-Yakov and Henson, 2018). In the most recent study (Ben- multiple timescales with finer temporal components constituting Yakov and Henson, 2018), brain activity was measured while subcomponents of larger scale segments (Swallow et al., 2009). participants viewed either an 8.5 min unfamiliar film or a In the current study, we examine the impact of event boundaries 120 min film they had first viewed some years previously. on long-term memory encoding. The motivation for this study Five independent observers identified the occurrence of event emerged from our observation of the brain activity correlates boundaries in the films and the event boundary strength of of long-term memory encoding when viewing naturalistic time an event boundary was defined as the number of observers extended stimuli such as television advertisements or television (more than one) that identified the specific event boundary. programs. We found that event boundaries in the viewed material The findings were consistent with the stated hypothesis in were frequently associated with transient drops in long-term that only hippocampal activity increased monotonically with memory encoding and the current study aims to examine event boundary strength. Furthermore, a data-driven approach this effect using a naturalistic stimulus with repeated, well- indicated that the times of hippocampal activity peaks were defined and consistent event boundaries. We conclude our associated with the occurrence of event boundaries. report with an illustrative example of the potential impact of While much of the research concerning the processing of event such event boundaries on the commercial effectiveness of a boundaries have focused on memory processes and especially television advertisement. hippocampal activity, a recent study has revealed a temporal Evidence for segmentation of experience was originally hierarchy of event boundary processing in a number of cortical suggested by the inter-subject consistency of the location of regions. Using a data driven approach, researchers observed event boundaries in video content (Newtson, 1973; Newtson transitions in stable patterns of brain activity as participants et al., 1977; Speer et al., 2003). Event segmentation also appears viewed an episode of a television serial (Baldassano et al., 2017). to play a role in long-term memory encoding with evidence These transitions formed a temporal nested hierarchy with the pointing to material occurring around an event boundary being most rapid transitions occurring in the primary visual and better recognized subsequently (Swallow et al., 2009). While auditory cortices, longer transitions occurring in the multimodal events occurring around an event boundary appear to be better cortices and the longest transitions observed in the hippocampus. recognized, there is evidence that working memory is updated at The longest hippocampal transitions also coincided with human an event boundary. For example, participants walking through labeled event boundaries. a doorway in a virtual reality experience exhibit reduced recall In summary, both behavioral and brain imaging studies of for content and events prior to walking through the doorway, event boundaries processing suggests that event boundaries presumably an event boundary (Radvansky and Copeland, 2006). trigger transitions in brain activity in a range of cortical regions Neuroimaging studies also indicated changes in brain activity that occur on multiple time scales and form a nested temporal when encountering an event boundary. For example, higher hierarchy with DMN and hippocampal transitions occurring on levels of brain activity were observed when encountering an event the longest time scale. At the level of the DMN and hippocampus, boundary in either film or reading (Kurby and Zacks, 2008; event boundaries trigger subsequent increased activity and a Zacks et al., 2010; Ben-Yakov et al., 2013). Furthermore, while simultaneous reduction in task positive networks such as the increases in brain activity have been observed to coincide with the dorsal attentional network. occurrence of consciously identified event boundaries, there may Along with the hippocampus, the ventro-lateral prefrontal be other events that are processed as event boundaries but are cortex (vlPFC) is also known to play an important role in not consciously recognized as such. Nakano et al. (2013), Nakano long-term memory encoding. During a memory task, targets (2015) examined the brain activity correlates of blinks. They associated with higher levels of activity at the vlPFC are better found that blinks tend to occur at what the authors describe as recognized subsequently, an effect known as the ‘differential ‘breakpoints’ or points in time coinciding with event boundaries neural activity based on memory’ and referred to as Dm (Buckner such as a pause in speaking or the end of a sentence while et al., 1999; Paller and Wagner, 2002; Blumenfeld and Ranganath, reading (Nakano et al., 2013). The blink related changes in 2007). We have previously demonstrated the Dm effect at an brain activity reported by Nakano et al. (2013), Nakano (2015) inferior frontal site using steady state topography (SST) an reveal richer detail in the changes in brain activity associated evoked potential methodology (Silberstein et al., 2000a). The SST with blink related event boundaries. Specifically, brain activity methodology makes use of the steady state visual evoked potential increases immediately following blinks were primarily restricted (SSVEP) which is elicited by a diffuse peripheral 13 Hz sinusoidal to the default mode network (DMN), including the hippocampus visual flicker presented at the same time that participants view while activity in the dorsal attention network, a task positive centrally presented video content The SSVEP phase variations in network, was reduced (Nakano et al., 2013, Nakano, 2015). This turn indicate SSVEP latency changes that we interpret in terms was interpreted as a transient disengagement from sensory or of regional brain activity changes with SSVEP latency decreases Frontiers in Neuroscience | www.frontiersin.org 2 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding or phase advance indicating increased activity and vice versa task positive networks (Buckner et al., 2008) suggests that the (Silberstein et al., 2000b; Kemp et al., 2004). Depending on the increases in DMN activity associated with the experience of event context and ease of understanding, we will use the terms SSVEP boundaries should be associated with a transient reduction in latency and activity interchangeably (Silberstein, 1995; Silberstein activity at the left hemisphere site FC5. et al., 2016, 2017, 2019). Our choice of the SST methodology for this study was based on a number of factors. Firstly, unlike transient evoked potentials MATERIALS AND METHODS such as N200 or P3b, SST is well suited to the study of time extended cognitive tasks such as the ones described in the current Participants study. In addition, the SST methodology is highly resistant to The sample comprised 50 male participants, mean age 34.1 years the common sources of EEG artifacts such as muscle activity, (SD = 12.6 years, range 18–59 years). All were screened for blinks and eye movements (Silberstein, 1995; Gray et al., 2003). the presence of pre-existing medical, neurological or psychiatric Finally, the SST measures of brain activity appear to be suitably conditions, including epilepsy. Participants were recruited via sensitive to cognitive processes such as visual attention and a recruitment agency and all testing was conducted on the long-term memory encoding (Silberstein et al., 1990, 2000a,b; premises of Neuro-Insight Pty Ltd. The study was approved by Silberstein and Nield, 2008). the Swinburne University Human Research Ethics Committee as In the above mentioned Dm study (Silberstein et al., 2000a), application SUHREC2014/029. we measured variations in 13 Hz SSVEP phase while 35 participants viewed naturalistic television content including Procedure unfamiliar advertisements sourced overseas. Advertising images Participants viewed a first-person journey through a 10-room coinciding with SSVEP latency minima or peaks in activity virtual art gallery that was created by the multi-media software at the left hemisphere lateral frontal scalp site FC5 located company Opaque Media. Each room of the gallery exhibited three between C3 and F7 (overlaying the vlPFC) were better recognized well known paintings and was entered through a hinged door that 7 days later than advertising images coinciding with times opened on approach. Participants spent 35 s in each of the 10 of minimum activity at this site (66.0% vs. 37.9%, d.f. = 34, rooms and the entire viewing period occupied 380 s. The timing t = 7.1, p < 5 × 10−8 ). As hypothesized, the Dm effect was of the appearance of each painting as well as the appearance of the largest at FC5 and the correlation between the 7-day recognition door to the next room after entering a room was identical for all of score and the SST measure of activity was only statistically the 10 rooms. Views of one of the gallery rooms in 5 s increments significant at FC5 after correcting for multiple comparisons are illustrated in Figure 1. (Silberstein et al., 2000a). The task was presented on a video monitor subtending 24◦ In the current study, we use SST to examine variations in vertically and 46◦ horizontally. The stimulus used to evoke brain activity while participants encountered event boundaries in the steady state visually evoked potential (SSVEP) was a white a virtual first-person journey through a 10-room art gallery. Each spatially diffuse 13-Hz sinusoidal flicker with a modulation depth of the rooms displays three paintings that were always viewed at of 45% and subtending a horizontal angle of 160◦ and a vertical the same pace and followed by an opening door leading to the angle of 90◦ , which was superimposed on the visual fields. next gallery room. We hypothesize that the first-person journey This flicker was present throughout the task and special goggles through the open door to the next gallery room constitutes an enabled subjects to simultaneously view the cognitive task and the event boundary and that such event boundary will be associated sinusoidal flicker. with a transient reduction in activity at FC5, the left hemisphere site we have previously shown to demonstrate a robust Dm Data Acquisition effect. Our choice of the doors separating the gallery rooms as Brain electrical activity was recorded from 20 scalp sites the event boundaries was based not only on the fact that the illustrated in Figure 2 and these were positioned and labeled doors were the actual boundaries between participants viewing a according to the International 10-10 system (Acharya et al., different trio of paintings but also published reports that passing 2016). Electrode locations FC5 and FC6 are located closest to through doorways triggered reductions in access to memory of the left and right VLPFC respectively. The average potential of items in the preceding room (Radvansky and Copeland, 2006; both mastoids served as a reference while an electrode located Radvansky et al., 2011). These findings suggest that doorway at FPz served as a ground. Brain electrical activity was amplified passages are processed as event boundaries. In addition, we base and bandpass filtered (3 dB down at 0.1 Hz and 30 Hz) before our hypothesis of a transient reduction in activity at this site on digitization to 16-bit accuracy at a rate of 400 Hz. The major accumulated evidence that the left and right vlPFCs are nodes features of the signal processing have been described (Silberstein of task positive cortical networks. Specifically, the right vlPFC et al., 2016). Briefly, the SSVEP was determined from the 13- is considered a node of the ventral attentional network (Vossel Hz single-cycle cosine and sine (or real and imaginary) Fourier et al., 2014) while the left vlPFC plays an important role in speech coefficients that were then smoothed using a 10 stimulus-cycle as well as the cognitive control of memory processes (Badre and cosine weighted smoothing window. At the stimulus frequency Wagner, 2007; Blumenfeld and Ranganath, 2007). As both left of 13 Hz, this yields a temporal resolution of 380 ms or half and right vlPFCs are components of task positive networks, the the 10-cycle window width because of the cosine weighting. The widely observed negative correlation between DMN activity and cosine smoothing window was then shifted 1 stimulus cycle and Frontiers in Neuroscience | www.frontiersin.org 3 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding FIGURE 1 | The timing of events in a room in the virtual gallery are illustrated. During the 35 s period in each room, the exit door comes into view at the 30 s mark while the passage through the doorway to the next room occurs at the 35 s mark. the coefficients were recalculated for this overlapping period. SSVEP phase difference time series are multiplied by −1 and a This process was continued until the entire 380 s of activity was new average determined. This multiplication by −1 is equivalent analyzed. An identical procedure was applied to data recorded to interchanging the SSVEP mean of the first 5 s and the SSVEP from all recording sites. value at a given point in time and is consistent with the NULL For each participant, the SSVEP data at each electrode hypothesis where it is assumed that the SSVEP phase is as likely to associated with the 35 s epoch following the entrance to the next exceed the reference level as it is to be exceeded by the reference room was averaged across all of the 10 rooms. level. This process is repeated 100,000 times and the number of For each participant, the mean phase of the SSVEP at each times where the permuted average of the SSVEP phase difference electrode evaluated over the first 5 s epoch following the entrance is equal to or less than the observed value at the 30 s point in to the next room was set to zero before separately averaging the time (the time of the appearance of the door) is determined. This real and imaginary components of the SSVEP time-series across number when divided by the number of permutations (100,000) the 10 rooms. The SSVEP time series data at each electrode was yields the exact one-sided p-value of the observed SSVEP phase then averaged across all participants to create the pooled SSVEP difference occurring under the NULL hypothesis. This process phase time-series data. was repeated for all 20 recording sites. As the hypothesis of this study is that the appearance of Statistical Analysis the exit door will be associated with a transient reduction in We used a permutation test to evaluate the hypothesis that the activity at electrode site FC5, we only consider the statistical SSVEP phase at the FC5 site will exhibit a transient reduction at significance of the activity on the appearance of the exit door at the time the exit door of the gallery room is viewed. We used a the 30 s point in time. permutation test rather than a parametric test such as Student’s t for two main reasons. Firstly, parametric tests are based on the assumption that the test parameter is distributed normally. RESULTS As the SSVEP phase data is not distributed normally, we used the permutation test which does not depend on any assumption The SST based measures of activity during the 35 s epoch are of normality. In addition, the permutation test provides exact illustrated in Figure 2 and the first appearance of the door at the probabilities (Blair and Karniski, 1993). These features have 30 s mark is associated with a transient activity reduction that is led biomedical researchers to consider permutation tests as the most prominent at left frontal sites FC5 and C3. preferred statistical methodology (Ludbrook and Dudley, 1998). Table 1 lists the exact probability of falsely rejecting the NULL Our NULL hypothesis is that there is no statistical difference hypothesis at the 30 s point in time for all electrode sites. While in SSVEP phase at the time the exit door is viewed compared our hypothesis only refers to effects at electrode site FC5 we will to the mean SSVEP phase during the first 5.0 s of the trial. The extend our consideration beyond this single site and thus adjust mean of the first 5 s of the trial are used as the reference level as the experiment wide p level to account for the 20 comparisons during this interval no paintings have been viewed in the current considered. After applying a Bonferroni correction for the 20 room. The mean SSVEP phase evaluated over the first 5.0 s of the comparisons, only sites FC5 and C3 demonstrate an experiment trial is subtracted from all the SSVEP phase values in the 35 s wide effect significant at the corrected p < 0.01 level. Applying a epoch yielding an SSVEP phase difference time series for each less conservative criterion of an experiment wide effect significant electrode. To conduct the permutation test, a random subset of at the corrected p < 0.05 level indicates a significant effect at Frontiers in Neuroscience | www.frontiersin.org 4 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding FIGURE 2 | Pooled SSVEP phase averaged over the all gallery rooms for the 35 s period spent in each room. Positive values indicate phase advance or increased activity and negative values, phase lag or reduced activity. The vertical line indicates the 30 s mark coinciding with the appearance of the exit door as illustrated above. The associated p-values for the NULL hypothesis that the observed SSVEP phase at the 30 s mark is not less than the mean value during the first 5 s in the room are listed in Table 1. Frontiers in Neuroscience | www.frontiersin.org 5 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding TABLE 1 | Exact probability of rejecting the NULL hypothesis that activity at the 20 are associated with blinking, not all blinks are associated with recording sites is not less than the mean of the initial 5 s value. event boundaries. For example, lapses or reductions in attention Electrode p Electrode p Electrode p also cause increased blinking (McIntire et al., 2014). The possible contribution of factors other than event boundaries to the Fp1 0.0045 FP2 0.0270 Nakano et al. (2013), Nakano (2015) right vlPFC findings is thus F3 0.0062 Fz 0.0103 F4 0.0043 consistent with the right vlPFC comprising a node of the ventral FC5 0.0002** FC6 0.0783 attentional network (Vossel et al., 2014) and such attention effects T7 0.0134 T8 0.0840 are observed most consistently in the right hemisphere rather C3 0.0002** Cz 0.0178 C4 0.0385 than the left (McIntire et al., 2014). P3 0.0011* Pz 0.0198 P4 0.0542 Another possible contributing factor is that participants in the PO7 0.0201 PO8 0.1972 Nakano studies were told that their recollection of the material O1 0.3129 Oz 0.1697 O2 0.6308 viewed while in the scanner would be tested after the scan. By One asterisk indicates that the effect is significant at the experiment wide p < 0.05 making the task one of memory recall, it is possible that this level after correcting for multiple comparisons, that is p < 0.05/20. The two reduced the impact of event boundaries on long-term memory asterisks indicate that the effect is significant at the experiment wide p < 0.01 or p < 0.01/20. encoding and hence attenuated the event boundary related drops at the left vlPFC. Our findings are also consistent with those of Ben-Yakov et al. the left parietal site P3. Statistically, the strongest effect was (2014), Ben-Yakov and Henson (2018) who reported increased observed at FC5 and C3 where both are associated with a value hippocampal activity immediately following the occurrence of of p = 2 × 10−4 . Using the requivalent approach suggested by event boundaries in a film viewed by participants. Increased Rosenthal and Rubin (2003) to determine experimental effect size activity in the hippocampus and other DMN nodes when of this SSVEP reduction, we see that the effect size is r = 0.52, perceiving an event boundary would result in transient decreased which is considered a large effect (Cohen, 1988). activity in task-oriented networks including the vlPFC (Buckner In summary, we note that the strongest effects associated with et al., 2008). This would in turn be associated with a transient the appearance of the door at the 30 s mark are restricted to the reduction in attention and long term memory encoding. As the left frontal sites. transient reduction in long-term memory encoding following an event boundary occurs in conjunction with increased hippocampal activity, this suggests that this transient reduction DISCUSSION coincides with better encoding of the experience preceding the event boundary. Our hypothesis was confirmed in that the appearance of the We observed a frontal left hemisphere bias in the strength of event boundary represented by the ‘virtual gallery door’ was the event boundary related drop in SSVEP phase. This was the associated with a robust transient drop in activity at FC5 the case even though the viewing material was primarily pictorial left frontal scalp site overlaying the ventro lateral prefrontal with no speech and text restricted to the painting labels. This cortex (vlPFC). Given the recognized role of the vlPFC in long- left hemisphere bias is consistent with our earlier report on the term memory encoding (Buckner et al., 1999; Blumenfeld and SSVEP Dm effect (Silberstein et al., 2000a) as well as numerous Ranganath, 2007) and our previous work demonstrating a Dm studies pointing to a preferential role for the left vlPFC in the effect for the SSVEP phase at FC5 the scalp site (Silberstein control of both encoding and retrieval of long-term memory et al., 2000a), our current findings suggest that an event boundary (Badre and Wagner, 2007; Blumenfeld and Ranganath, 2007). triggers a transient reduction in long-term memory encoding. Furthermore, our findings suggest that the left prefrontal and Such a reduction is consistent with the fMRI findings of Nakano frontal cortex play an important role in not only encoding et al. (2013), Nakano (2015) who interpreted their findings information into long-term memory but also in the segmentation of blink related activity reduction in nodes of the dorsal and of time extended events and the identification of high-level ventral attentional networks in terms of event boundaries causing event boundaries. In summary, our findings point to the left a ‘transient disengagement from external perception and the frontal cortex playing an important role in interposing a transient engagement of internally oriented network,’ presumably such as drop in long-term memory encoding on the perception of the default mode network. Interestingly, Nakano et al. (2013) an event boundary. observed a blink related activity reduction at the right vlPFC While our hypothesis is confirmed in that activity at the left while we observed the event boundary related reduction at FC5 frontal site FC5 showed a sharp drop at the event boundary, located in the vicinity of the left vlPFC. We suggest two possible the findings of Ben-Yakov and Henson (2018) call into question factors may have contributed to this difference. One could be the underlying mechanism we proposed for the event boundary the different events used to initiate data acquisition in our effect. We proposed that the drop in vlPFC activity was driven respective studies. We used explicit events (doorways) that are by an increase in DMN and hippocampal activity at the event widely considered to comprise event boundaries (Radvansky and boundary. However, Ben-Yakov and Henson (2018) reported Copeland, 2006; Radvansky et al., 2011, 2015). By contrast, the bilateral and approximately equal increases in hippocampal Nakano et al. (2013), Nakano (2015) studies focused exclusively activity on encountering an event boundary. If the vlPFC drop on blink related changes in brain activity. While event boundaries was determined by increased ipsilateral hippocampal activity, Frontiers in Neuroscience | www.frontiersin.org 6 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding then we might expect to see a drop in both left and right vlPFC boundary is presented verbally, such as in the conclusion of a activity or at scalp sites FC5 and FC6. We suggest that two narrative or a verbal joke. possible explanations may account for the inconsistency between A point which we believe warrants comment is the relatively the neural mechanism proposed to account for the vlPFC event strong effect size of the observed event boundary related boundary effect and our observation of an overwhelmingly left drop in SSVEP phase at FC5 (r = 0.52). We observed a hemisphere effect. One possibility is that the interaction between comparably large effect size in our original report describing hippocampus and vlPFC may differ between left and right the Dm effect evaluated with the SSVEP (Silberstein et al., hemispheres. This would be consistent with the differing roles of 2000a). While signal to noise considerations may play a role the left and right vlPFC. The left vlPFC is known to play a crucial in contributing to the experimental effect size, we suggest that role in the control in both encoding and retrieval of information more fundamental aspects of neural information processing from long-term memory (Badre and Wagner, 2007). By contrast in the cortex may play an important role. Specifically, we the right vlPFC is involved in non-memory processes such as suggest that the 13 Hz frequency used to elicit the SSVEP response inhibition, etc. (Levy and Wagner, 2011). may be an important factor influencing the effect size. This Another possibility is suggested by a consideration of is suggested in the light of recent findings concerning the Event Segmentation Theory which proposes that an ‘event neural mechanisms mediating sensory-driven or ‘bottom-up’ comprehension systems’ make predictions about upcoming and cortical control or ‘top down’ information flows within the events and an event boundary is identified when a prediction cortex. While these are briefly described here, readers seeking of upcoming events is in error (Swallow et al., 2009). Such a more detailed description are referred to Silberstein et al. prediction errors are associated with increased hippocampal (2016). It is now recognized that both ‘bottom-up’ and ‘top- theta activity suggesting increased encoding in long-term down’ intracortical communication processes are mediated by memory (Paller and Wagner, 2002; Garrido et al., 2015). synchronous oscillations. Recent work examining the difference A magnetoencephalography study of brain activity responses to between the information flows reveals that they are mediated by a sequence of pictorial stimuli that violate expectations indicated oscillations in different frequency ranges. Specifically, bottom- that an EEG theta frequency mismatch signal is generated up communication appears to be mediated predominantly in a network comprising the hippocampus and ventro-medial by synchronous oscillations in the EEG gamma frequency prefrontal cortex (vmPFC) (Garrido et al., 2015). The authors range (40–80 Hz) while top-down communication is mediated report that it is the vmPFC that drives hippocampal activity when predominantly synchronous oscillations in the EEG high alpha- expectations are violated and that this effect is restricted to the low beta range (10–20 Hz) (Von Stein et al., 2000; Markov et al., left vmPFC and hippocampus. Such a left hemisphere bias is also 2014; Fries, 2015). Our choice of 13 Hz as the frequency used consistent with our findings. to elicit the SSVEP means that our responses (measurements, One possibility that must be considered is the extent to which recorded findings) are likely to be driven predominantly by top- our findings are due to visual sensory effects rather than the down processes that among other things would modulate long- appearance of an event boundary. If sensory driven changes in term memory encoding. In other words, our SST methodology visual attention were responsible for our observations, we would may be particularly well suited to measuring top-down processes expect to see such changes reflected in activity at occipital sites and this may contribute to the strong effect size we have observed. (Silberstein et al., 1990). We consider this possibility unlikely We have termed event boundary related transient drop in the for the following reason. While the appearance of the ‘door’ SST based measure of activity at FC5 and C3 Conceptual Closure. was associated with a robust SSVEP phase reduction at left We have frequently observed what we take to be Conceptual frontal sites FC5 and C3, none of the occipital sites demonstrated Closure when recording SST brain activity in naturalistic time statistically significant changes or differences with respect to the extended stimuli such as television advertisements (Silberstein reference level at this time. However, while the SSVEP phase and Nield, 2008). While Conceptual Closure appears at obvious values at occipital sites are consistent with our suggestion that the event boundaries such as the screen changing color or music effects at FC5 and C3 are unlikely to be triggered by visual sensory ending, we have also observed it in more subtle indications of effects, our experimental design and methodology does not allow event boundaries such as two people standing to shake hands us to completely eliminate this possibility. For example, it may be or a car driving into the distance. Consistent with our finding possible that changes in activity at some components of the visual that long-term memory encoding at branding is associated processing system in either the dorsal or ventral visual processing with television advertising effectiveness (Silberstein and Nield, systems may account for our observations at FC5 and C3. One 2008) we have found that Conceptual Closure can have a possibility suggested by a reviewer is that the Limited Capacity significant negative impact of the commercial effectiveness of Model of Motivated Mediated Message Processing (LC4MP) may an advertisement if it coincides with the sole appearance of provide an alternative interpretation for our findings (Lang, product branding or key message. This is typically the case 2006). Specifically, changes in the visual environment in moving where a television advertisement is structured as a narrative that from one gallery room to the next engages increased visual describes the issue or problem to be addressed in the absence of processing resources which in turn decreases performance in an explicit or implicit representation of the brand or product in a ‘secondary task’ such as long-term memory encoding. One question. Frequently, the narrative concludes immediately before interesting approach suggested by the reviewer is a study where the appearance of the product or brand and thus the narrative the visual environment remains unchanged while the event conclusion is processed by the brain as an event boundary Frontiers in Neuroscience | www.frontiersin.org 7 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding triggering Conceptual Closure at the time the product or brand participants in the first study (six females) also participated in the first appears in the advertisement. We have found this situation subsequent study. is more likely to be associated with consumers associating a In both studies, participants viewed the client television competing brand with the advertisement. advertisement in the context of a typical television viewing experience. The advertisement was viewed as one of a group of Illustrative Case Study four advertisements in an advertising break occurring in a typical The following case study is provided to illustrate the manner prime time television program. in which Conceptual Closure can be managed so as to A total of three groups of advertisements or advertising enhance an advertisement’s commercial effectiveness. It should ‘breaks’ were present in both studies with the presentation of the be stressed that the following case study was undertaken client advertisement counterbalanced in either the first or second as a standard commercial study for a client as advertiser. advertisement break. It is primarily presented here to illustrate how findings The advertisement was viewed in one of two groups of concerning Conceptual Closure are dealt with in a commercially advertisements or advertising breaks. In both cases, the client relevant manner. advertisement occurred as the second advertisement in the break Neuro-Insight Pty Ltd. was commissioned to conduct a and was preceded by a different advertisement in each break. two-part advertising research study examining an advertising The order in which the advertisement was viewed was evenly campaign by the client, NAB Ltd. a major Australian financial rotated across the entire sample. All details of the procedure, data services provider for its superannuation fund, MLC. The design acquisition and analysis are the same as those described in the of the advertising campaign, named ‘Save Retirement’ was to multi room gallery study described above. encourage likely clients to seek retirement related financial advice Both versions of the advertisement featured a young boy and from MLC. The campaign was launched in early 2014 and his grandfather visiting a futuristic ‘museum’ where a display had been on air for approximately 3 months with independent features a happy older couple going for a drive in the country tracking indicating the campaign was suffering from poor brand in their sports car, and by inference enjoying their retirement. As linkage in that an unacceptably high proportion of viewers were they look at the display, the young boy asks ‘what are they doing’ inclined to identify a competing brand as the advertiser. and the grandfather wistfully replies ‘they are in retirement’ and In the first part of the study, Neuro-Insight evaluated the in retirement ‘they can do anything they want.’ In the original effectiveness of the original 30 s television advertisement on the advertisement, a voice-over then suggests people speak to their basis of SST measures of long-term memory encoding (scalp site financial advisor and subsequently the boy pulls his grandfather FC5) at the point of branding in the final 5 s of the advertisement. away and they both leave the display. Recommendations on how to address identified deficiencies were Analysis of the original advertisement indicated two transient made and implemented by the creative agency with a re-edited reductions in long-term memory encoding in the final 7 s of the 30 s advertisement broadcast in the last quarter of 2014. The advertisement. The first at the 22.5 s mark coincides with the end re-edited advertisement can be viewed at https://www.youtube. of the story narrative and a verbal call to action. The second at com/watch?v=ZbzKcTaOlXg. The original advertisement was the 24 s mark coincided with the boy pulling his grandfather on air for 3 months and the re-edited advertisement for a from the ‘retirement display.’ We consider both drops in long- following 2 months. An independent market research firm term memory encoding to be examples of Conceptual Closure was commissioned by the client to track the performance of as they occurred at clear event boundaries in the advertisement. both the original and revised advertisement. As this was a The likely impact of Conceptual Closure on the effectiveness of standard commercial study conducted by Neuro-Insight it was the advertisement is indicated by the level of long-term memory undertaken in accordance with the ESOMAR International Code encoding during branding during the final 3.5 s. The brand on Market, Opinion and Social Research and Data Analytics and is only featured in the last 3.5 s of the original advertisement the Neuromarketing Science and Business Association (NMSBA) (final branding) and during this period, long-term memory Code of Ethics. It should also be noted that the data recording encoding is low and in the lowest 8% of our database of financial procedures and equipment used in this case study are identical services advertisements. to those used in the preceding study and approved by the The main changes recommended for the revised Swinburne University Human Research Ethics Committee as advertisement were aimed at reducing the impact of Conceptual application SUHREC2014/029. Closure at the time of branding. These included introducing Studies of the original and re-edited advertisement were the brand verbally and visually earlier to suggest a continuation both conducted in Melbourne at the Neuro-Insight offices. Fifty of the original narrative while the grandfather and boy remain participants (25 females) with a mean age of 54 years (range at the display. Furthermore, the scene triggering Conceptual 45–64 years) participated in the first study undertaken in July Closure was edited out so that the boy and grandfather remained 2014. Another group of 50 participants (25 males) with a mean stationary throughout the advertisement. To accommodate these age of 52 years (range 45–61 years) took part in the subsequent changes, the end of the narrative and call to action occurred study undertaken in September 2014. Participants in both studies 2 s earlier. The impact of these changes on long-term memory were recruited through a recruitment agency and had to satisfy encoding at the times of branding was significant in that the level a range of criteria such as age, employment status and financial of long-term memory encoding at time of the newly branded decision-making responsibility as specified by the client. Nine scene peaked at the 96th percentile of our database of financial Frontiers in Neuroscience | www.frontiersin.org 8 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding FIGURE 3 | Activity at FC5 indicating levels of long-term memory encoding for the last 15 s of the original and re-edited advertisements. The dashed line illustrates long-term memory encoding for the original advertisement while the solid line is the equivalent finding for the re-edited advertisement. The 84% confidence interval are included for both time series and when these do not overlap, the statistical significance of the difference between time series corresponds to p < 0.05 (Austin and Hux, 2002). The points in time where the narrative ends and the call to action starts in the original advertisement is indicated by the dashed arrows labeled ‘b’ while the dashed arrow labeled ‘c’ indicates when the child starts pulling his grandfather from the display cabinet. The solid arrow labeled ‘a’ indicates the point in time where the narrative end and the call to action starts in the re-edited advertisement. The brown bar labeled ‘d’ indicates the duration of the logo on the screen and final branding. The removal of the segment where the boy pulls his grandfather away from the cabinet appears to reduce the impact of Conceptual Closure and lead to higher long-term memory encoding during the final 3 s of the advertisement. services advertisements while the final branding final visual Limitations and Future Research branding was now at the 72nd percentile of our database of We acknowledge that one of the limitations of this study financial services advertisements. These differences in long-term is that the experimental design does not enable us to memory encoding at the time of branding were significant at include behavioral data to confirm the drop in long-term the p < 0.05 level and this is explained in more detail in the memory encoding on the appearance of the door. We have Figure 3 legend. relied on extensive evidence from other laboratories on the During the 3 months period that the original advertisement role of the left ventrolateral prefrontal cortex and long- was aired and the following 1 month where the re-edited term memory encoding as well as our previous findings advertisement was being broadcast, an independent market demonstrating the link between activity at FC5 long-term research firm conducted advertising tracking studies for the memory encoding to infer a drop in long-term memory client. It was found that in the 3-month period when the original encoding on the appearance of the event boundary. Nevertheless, advertisement was being aired, the proportion of viewers who the incorporation of a behavioral test of long-term memory had seen the advertisement and correctly identify the brand encoding at the event boundary is a recommendation (brand linkage) was considered low by the market research for future studies. firm conducting the tracking study. However, in the first Another limitation concerns the limited amount of month that the re-edited advertisement was being broadcast, information that could be included in the illustrative case we were advised by the client that the market research firm study concerning the effectiveness of the first and then conducting the tracking study reported that brand linkage had the revised advertising phases. As this was a commercial more than doubled, increasing by 120%. While the tracking study, the client has the final say on what, or if any findings are consistent with the SST indications of improved commercially sensitive information concerning the study advertising effectiveness, we cannot exclude the possibility that may be published. We are grateful to MLC for their other factors external to those determined by the client may have permission to include the percentage increase in the brand also contributed. However, the client is unaware of any such linkage and advise that brand linkage data for the first external factors that may have contributed to the advertisement’s and revised advertising phases has not been made available improved performance. for publication. Frontiers in Neuroscience | www.frontiersin.org 9 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding CONCLUSION AUTHOR CONTRIBUTIONS Event boundaries can trigger a brief drop in long-term memory RS and GN conceived the project. SS and GN supervised encoding or Conceptual Closure. While Conceptual Closure, development of video first person journey. RS and SS contributed in general may not compromise advertising effectiveness, it to the experimental design. SS supervised the project. RS may do so if it occurs at the time that a key message developed statistical analysis software for the study and or branding is presented. In the case study, we show how analyzed the data. minor changes to a finished advertisement can modify the impact of Conceptual Closure and in turn significantly enhance advertising effectiveness. FUNDING Neuro-Insight Pty Ltd. provided facilities for the study at no DATA AVAILABILITY STATEMENT charge to the authors. Neuro-Insight Pty Ltd. also paid for the recruitment and reimbursement of participants. Neuro- The datasets generated for this study are available on request to Insight Pty Ltd. provided the services of SS to supervise the the corresponding author. recording at no charge. ETHICS STATEMENT ACKNOWLEDGMENTS The studies involving human participants were reviewed and approved by the Swinburne University Human Research The authors acknowledge the assistance of Neuro-Insight Pty Ltd. Ethics Committee as application SUHREC2014/029. The in funding the study. The audio-visual material depicting the patients/participants provided their written informed consent first-person journey through the virtual art gallery was developed to participate in this study. by Opaque Media Pty Ltd. REFERENCES Fries, P. (2015). Rhythms for cognition: communication through coherence. Neuron 88, 220–235. doi: 10.1016/j.neuron.2015.09.034 Acharya, J. N., Hani, A. J., Cheek, J., Thirumala, P., and Tsuchida, T. N. (2016). Garrido, M. I., Barnes, G. R., Kumaran, D., Maguire, E. A., and Dolan, R. J. American Clinical Neurophysiology Society guideline 2: guidelines for standard (2015). Ventromedial prefrontal cortex drives hippocampal theta oscillations electrode position nomenclature. Neurodiagn. J. 56, 245–252. doi: 10.1080/ induced by mismatch computations. Neuroimage 120, 362–370. doi: 10.1016/j. 21646821.2016.1245558 neuroimage.2015.07.016 Austin, P. C., and Hux, J. E. (2002). A brief note on overlapping confidence Gray, M., Kemp, A. H., Silberstein, R. B., and Nathan, P. J. (2003). Cortical intervals. J. Vasc. Surg. 36, 194–195. doi: 10.1067/mva.2002.125015 neurophysiology of anticipatory anxiety: an investigation utilizing steady state Badre, D., and Wagner, A. D. (2007). Left ventrolateral prefrontal cortex and the probe topography (SSPT). Neuroimage 20, 975–986. doi: 10.1016/s1053- cognitive control of memory. Neuropsychologia 45, 2883–2901. doi: 10.1016/j. 8119(03)00401-4 neuropsychologia.2007.06.015 Kemp, A. H., Silberstein, R. B., Armstrong, S., and Nathan, P. J. (2004). Gender Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., and Norman, differences in the cortical electrophysiological processing of visual emotional K. A. (2017). Discovering event structure in continuous narrative perception stimuli. Neuroimage 2004, 632–646. doi: 10.1016/j.neuroimage.2003.09.055 and memory. Neuron 95, 709–721. Kurby, C. A., and Zacks, J. M. (2008). Segmentation in the perception and memory Ben-Yakov, A., Eshel, N., and Dudai, Y. (2013). Hippocampal immediate of events. Trends Cogn. Sci. 12, 72–79. doi: 10.1016/j.tics.2007.11.004 poststimulus activity in the encoding of consecutive naturalistic episodes. J. Exp. Lang, A. (2006). Using the limited capacity model of motivated mediated message Psychol. Gen. 142:1255. doi: 10.1037/a0033558 processing to design effective cancer communication messages. J. Commun. 56, Ben-Yakov, A., and Henson, R. N. (2018). The hippocampal film-editor: sensitivity S57–S80. and specificity to event boundaries in continuous experience. J. Neurosci. 38, Levy, B. J., and Wagner, A. D. (2011). Cognitive control and right ventrolateral 10057–10068. doi: 10.1523/jneurosci.0524-18.2018 prefrontal cortex: reflexive reorienting, motor inhibition, and action updating. Ben-Yakov, A., Rubinson, M., and Dudai, Y. (2014). Shifting gears in hippocampus: Ann. N. Y. Acad. Sci. 1224, 40–62. doi: 10.1111/j.1749-6632.2011.05958.x temporal dissociation between familiarity and novelty signatures in a Ludbrook, J., and Dudley, H. (1998). Why permutation tests are superior to t and single event. J. Neurosci. 34, 12973–12981. doi: 10.1523/jneurosci.1892-14. F tests in biomedical research. Am. Statist. 52, 127–132. doi: 10.1080/00031305. 2014 1998.10480551 Blair, R. C., and Karniski, W. (1993). An alternative method for significance testing Markov, N. T., Vezoli, J., Chameau, P., Falchier, A., Quilodran, R., Huissoud, C., of waveform difference potentials. Psychophysiology 30, 518–524. doi: 10.1111/ et al. (2014). Anatomy of hierarchy: feedforward and feedback pathways in j.1469-8986.1993.tb02075.x macaque visual cortex. J. Comp. Neurol. 522, 225–259. doi: 10.1002/cne.23458 Blumenfeld, R. S., and Ranganath, C. (2007). Prefrontal cortex and long-term McIntire, L. K., McKinley, R. A., Goodyear, C., and McIntire, J. P. (2014). Detection memory encoding: an integrative review of findings from neuropsychology and of vigilance performance using eye blinks. Appl. Ergon. 45, 354–362. doi: neuroimaging. Neuroscientist 13, 280–291. doi: 10.1177/1073858407299290 10.1016/j.apergo.2013.04.020 Buckner, R. L., Andrews-Hanna, J. R., and Schacter, D. L. (2008). The brain’s default Nakano, T. (2015). Blink-related dynamic switching between internal and external network. Ann. N. Y. Acad. Sci. 1124, 1–38. orienting networks while viewing videos. Neurosci. Res. 96, 54–58. doi: 10.1016/ Buckner, R. L., Kelley, W. M., and Petersen, S. E. (1999). Frontal cortex contributes j.neures.2015.02.010 to human memory formation. Nat. Neurosci. 2, 311–314. doi: 10.1038/7221 Nakano, T., Kato, M., Morito, Y., Itoi, S., and Kitazawa, S. (2013). Blink-related Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Edn. momentary activation of the default mode network while viewing videos. Proc. Hillsdale, NJ: Erlbaum. Natl. Acad. Sci. U.S.A. 110, 702–706. doi: 10.1073/pnas.1214804110 Frontiers in Neuroscience | www.frontiersin.org 10 April 2020 | Volume 14 | Article 292
Silberstein et al. Event Boundaries and Memory Encoding Newtson, D. (1973). Attribution and the unit of perception of ongoing behavior. Silberstein, R. B., and Nield, G. E. (2008). Brain activity correlates of consumer J. Pers. Soc. Psychol. 28, 28–38. doi: 10.1037/h0035584 brand choice shift associated with television advertising. Int. J. Advert. 27, Newtson, D., Engquist, G. A., and Bois, J. (1977). The objective basis of behavior 359–380. doi: 10.2501/s0265048708080025 units. J. Pers. Soc. Psychol. 35, 847–862. doi: 10.1037/0022-3514.35.12.847 Silberstein, R. B., Pipingas, A., Farrow, M., Levy, F., and Stough, C. K. Paller, K. A., and Wagner, A. D. (2002). Observing the transformation of experience (2016). Dopaminergic modulation of default mode network brain functional into memory. Trends Cogn. Sci. 6, 93–102. doi: 10.1016/s1364-6613(00) connectivity in attention deficit hyperactivity disorder. Brain Behav. 6:e00582. 01845-3 doi: 10.1002/brb3.582 Radvansky, G. A., and Copeland, D. E. (2006). Walking through doorways causes Silberstein, R. B., Schier, M. A., Pipingas, A., Ciorciari, J., Wood, S. R., and Simpson, forgetting: situation models and experienced space. Mem. Cogn. 34, 1150–1156. D. G. (1990). Steady state visually evoked potential topography associated with doi: 10.3758/bf03193261 a visual vigilance task. Brain Topogr. 3, 337–347. doi: 10.1007/bf01135443 Radvansky, G. A., Krawietz, S. A., and Tamplin, A. K. (2011). Walking through Speer, N. K., Swallow, K. M., and Zacks, J. M. (2003). Activation of human motion doorways causes forgetting: further explorations. Q. J. Exp. Psychol. 64, 1632– processing areas during event perception. Cogn. Affect. Behav. Neurosci. 3, 1645. doi: 10.1080/17470218.2011.571267 335–345. doi: 10.3758/cabn.3.4.335 Radvansky, G. A., Pettijohn, K. A., and Kim, J. (2015). Walking through doorways Swallow, K. M., Zacks, J. M., and Abrams, R. A. (2009). Event boundaries in causes forgetting: younger and older adults. Psychol. Aging 30, 259–265. doi: perception affect memory encoding and updating. J. Exp. Psychol. Gen. 138:236. 10.1037/a0039259 doi: 10.1037/a0015631 Rosenthal, R., and Rubin, D. B. (2003). r_equivalent: a simple effect size indicator. Von Stein, A., Chiang, C., and König, P. (2000). Top-down processing mediated Psychol. Methods 8, 492–496. doi: 10.1037/1082-989x.8.4.492 by interareal synchronization. Proc. Natl. Acad. Sci. U.S.A. 97, 14748–14753. Silberstein, R. B. (1995). “Steady state visually evoked potentials, brain resonances doi: 10.1073/pnas.97.26.14748 and cognitive processes,” in Neocortical Dynamics and Human EEG Rhythms, Vossel, S., Geng, J. J., and Fink, G. R. (2014). Dorsal and ventral attention systems: ed. P. L. Nunez (New York, NY: Oxford University Press), 272–303. distinct neural circuits but collaborative roles. Neuroscientist 20, 150–159. doi: Silberstein, R. B., Camfield, D. A., Nield, G., and Stough, C. (2019). Gender 10.1177/1073858413494269 differences in parieto-frontal brain functional connectivity correlates of Zacks, J. M., Speer, N. K., Swallow, K. M., and Maley, C. J. (2010). The brain’s creativity. Brain Behav. 9:e01196. doi: 10.1002/brb3.1196 cutting-room floor: segmentation of narrative cinema. Front. Hum. Neurosci. Silberstein, R. B., Harris, P. G., Nield, G. A., and Pipingas, A. (2000a). 4:168. doi: 10.3389/fnhum.2010.00168 Frontal steady-state potential changes predict long term recognition memory Zacks, J. M., and Swallow, K. M. (2007). Event segmentation. Curr. Dir. Psychol. performance. Int. J. Psychophysiol. 2000, 79–85. doi: 10.1016/s0167-8760(00) Sci. 16, 80–84. 00118-5 Silberstein, R. B., Line, P., Pipingas, A., Copolov, D., and Harris, P. (2000b). Steady- Conflict of Interest: RS, SS, and GN are employed by Neuro-Insight Pty Ltd. and state visually evoked potential topography during the continuous performance have a financial interest in Neuro-Insight Pty Ltd. task in normal controls and schizophrenia. Clin. Neurophysiol. 111, 850–857. doi: 10.1016/s1388-2457(99)00324-7 Copyright © 2020 Silberstein, Seixas and Nield. This is an open-access article Silberstein, R. B., Levy, F., Pipingas, A., and Farrow, M. (2017). First-dose distributed under the terms of the Creative Commons Attribution License (CC BY). methylphenidate–induced changes in brain functional connectivity are The use, distribution or reproduction in other forums is permitted, provided the correlated with 3-month attention-deficit/hyperactivity disorder symptom original author(s) and the copyright owner(s) are credited and that the original response. Biol. Psychiatry 82, 679–686. doi: 10.1016/j.biopsych.2017. publication in this journal is cited, in accordance with accepted academic practice. No 03.022 use, distribution or reproduction is permitted which does not comply with these terms. Frontiers in Neuroscience | www.frontiersin.org 11 April 2020 | Volume 14 | Article 292
You can also read