THINK BIG: BRITAIN'S DATA OPPORTUNITY - WANdisco
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
4 FOREWORD 8 CHAPTER 01 DATA IS THE 14 THE DATA EXPLOSION CHAPTER 02 NEW OIL THE OPPORTUNITY WITHIN 20 CHAPTER 03 LAYING THE FOUNDATIONS 24 CHAPTER 04 CLIVE HUMBY THE FUTURE OF DATA CHIEF DATA SCIENTIST, STARCOUNT 30 CONCLUSION INVENTOR TESCO CLUBCARD 32 PREDICTIONS & RECOMMENDATIONS 34 ABOUT US
THINK BIG: BRITAIN’S DATA OPPORTUNITY FOREWORD Data rules the world. This isn’t just what The authors suggested that Big Data’s potential they say out in Silicon Valley – it’s a simple has been overhyped; analysts stand accused of making leaps of logic when interpreting matter of fact. the data, identifying causation where there We create more data every 20 minutes than is none, and making recommendations is currently held by the Library of Congress. based on spurious correlations. More than 90 per cent of all the data in the While there may be some truth in this, such world was generated in the last two years criticism does not diminish the increasing alone. Today, every company – from Aston value of Big Data in various fields, from Martin to Zoopla – is to a greater or lesser business to healthcare, from sport to space extent a data company that churns out exploration. The backlash is a typical response colossal levels of information by the second. to ambitious claims about potential, even if This has led to the rise of what has become the technology does end up transforming known as ‘Big Data’: information sets so the world. The Economist perhaps said it large and complex that to understand them best: “It happened with the internet, and using traditional methods would be impossible. television, radio, motion pictures and the An increasing number of organisations hope telegraph before it. Now it is simply Big that using new technology to store, query and Data’s turn to face the grumblers.” analyse this data in real time will help them The truth is that with more information at our better understand their customers’ behaviour. fingertips we stand to learn more about the Information that was previously thrown away is world around us – and act with greater transforming every facet of human existence. precision, speed and efficiency. In Think Big: And what was once dismissed as useless is Britain’s Data Opportunity, we outline the Big essential to devising data-driven solutions to Data opportunity to show how organisations universal problems, offering the potential big and small are capitalising on its potential. to make us safer, happier and healthier. We Companies of the future will live or die by shouldn’t think about Big Data as simply being their analytical skills and the Chief Data an endless bundle of information. Instead we Officer could well be the future king of the should think of it as providing the opportunity boardroom. The business heroes of the of competitive advantage across every industry. coming years may be more scientific than A study by McKinsey estimated that data- artistic, able to make insightful commercial driven strategies could save up to $100 billion conclusions based on empirical evidence. annually across the US healthcare system Here at WANdisco we believe in the power alone, helping to optimise innovation, improve of Big Data, but our view is the UK isn’t yet research and ultimately build new life-saving doing enough to harness the opportunity. tools for medical staff. SAS, a software What’s lacking is an approach that will ensure provider, predicts the UK government could we have enough data scientists, as well as save up to £2 billion by implementing Big Data enough managers and analysts with the solutions to detect fraud1. know-how to make effective decisions. The UK government has named Big Data as Our education system is not producing the one of ‘eight great technologies’ for the future, graduates with the skills to get Britain ready predicting that it will generate £216 billion for its data destiny. and provide 58,000 jobs for the economy by It’s our belief that every conceivable area of 2017. Almost £50 million has been set aside to life where things need to be organised and fund The Alan Turing Institute and provide a decisions made will transform to become British base for Big Data and algorithm leaner, more efficient, more powerful and research. more lasting. Welcome to a new era: the age Despite this momentum, Big Data has its of Big Data. critics. In March 2014, FT Weekend published a four-page feature with the title ‘Big Data: are we making a big mistake?’, while in June DAVID RICHARDS the same year The Guardian ran with CEO, PRESIDENT & CO-FOUNDER, ‘Big Data: saviour or sham?’. WANDISCO 4 Data equity: Unlocking the value of big data, SAS & CEBR, 2012. 1 www.wandisco.com 5
BY 2020 THE AVERAGE BY 2017 THERE WILL THE MARKET FOR BIG BUSINESS WILL HAVE BE A TOTAL OF 19 DATA TECHNOLOGY TO MANAGE 50 TIMES BILLION INTERNET AND SERVICES WILL MORE INFORMATION CONNECTED DEVICES REACH $16.9 BILLION THAN IT DOES TODAY IN THE WORLD BY 2015 ECONOMIST INTELLIGENCE UNIT WEF 2012 IDC 2012 & CAPGEMINI 2012 BIG DATA IN 2014 THERE WILL OVER THE LAST 10 BE AN EXPECTED 77 YEARS THE DIGITAL BY NUMBERS BILLION APPLICATION SHARE OF THE DOWNLOADS, WORLD’S STORED COMPARED TO 10 INFORMATION HAS BILLION IN 2010 INCREASED FROM THERE WILL BE CISCO 2013 25% TO OVER 98% OVER 40 TRILLION IBM 2013 GIGABYTES ON EARTH BY 2020 – THAT'S 5,200 GIGABYTES 64% OF ENTERPRISES FOR EVERY PERSON ARE PLANNING BIG BIG DATA JOBS IDC & EMC 2012 DATA PROJECTS IN ARE FORECAST TO THE COMING YEAR INCREASE 92% BY 2017 GARTNER 2013 SAS 2013 6 7
THINK BIG: BRITAIN’S DATA OPPORTUNITY We live in a world dominated by data. The main reason for this data explosion is Every shopping trip, search queried on the rise of connected devices and software capable of creating files of text, images and Google, Uber-cab hailed and change video. At the end of 2013, one in every five in the traffic-lights is in some way a people in the world had a smartphone, while data transaction. one in 17 owned a tablet, according to CHAPTER 01 research by Business Insider. 01 The phenomenon has also been fuelled THE DATA because web users are increasingly comfortable with sharing personal details about themselves when they interact It’s not necessarily true that there’s more EXPLOSION with websites. data out there than before; rather that, instead of disappearing into the ether, details Meanwhile, even more data is created by of how we conduct our daily lives are mobile phones automatically through global increasingly being captured, stored and used positioning, Bluetooth, Wifi and other kinds (in most cases) to make our lives better. of mobile data that phones produce without users necessarily knowing about it. In 2012, the International Data Corporation (IDC) calculated that there was a total of In 2012, technology analysts at Gartner 1.8 zettabytes of data stored in one shape or concluded that between 10% and 15% of another across the world, nearly 50% more organisations were making proper use of Big than in 2010. It said that the global stock of Data and added that those that did would see data was therefore doubling, at least, every a 20% increase in revenue as a result. two years. Google, Facebook and Amazon are the For context, an average email might be giants of the internet age, but above all they 50 kilobytes (50,000 bytes) in size – attach are data companies. Demography, buying a picture and it becomes five megabytes. trends, times, dates, where people come A large computer hard drive might be able to from and where they go are all logged and store a terabyte – or 1,000 bytes cubed – crunched in a bid to understand users better, of data, while even an average phone give them more of what they want, and contains 16 gigabytes. make them easier to target for marketeers. 8 www.wandisco.com 9
CHAPTER 01 THE DATA EXPLOSION THINK BIG: BRITAIN’S DATA OPPORTUNITY 235 TERABYTES OF DATA SITS IN THE US LIBRARY OF INTRODUCING BIG DATA CONGRESS – FACEBOOK ALONE STORES, ACCESSES AND ANALYSES OVER 30,000 TERABYTES. MCKINSEY HADOOP CLOUDERA HORTONWORKS GOOGLE PROCESSES MORE THAN Apache Hadoop is the enterprise Cloudera was founded in 2008 to Hortonworks does one thing: 24,000 TERABYTES OF DATA A DAY. framework of Big Data that is used provide the first enterprise-ready building, managing and both to store and process implementation of Apache implementing Hadoop. The ACM – COMPUTING SOCIETY incredibly large data sets. Hadoop, set up by three engineers company has devoted itself to from Google, Yahoo and Facebook working within the open source The underlying technology was THE BIG DATA MARKET IS FORECAST TO BE WORTH originally invented by Google to – three businesses that were space, reaching out to customers among the trailblazers of Hadoop. through the existing products of $32.1 BILLION IN 2015 – RISING BY 49.5% TO $48 BILLION index all the rich textual and its partners such as Microsoft, structural information they were Based in Palo Alto, California, IN 2016 AND 66% TO $53.4 BILLION BY 2017. collating, and present results to Cloudera has enjoyed rapid Teradata and SAP. It houses the WIKIBON largest collection of Hadoop users in a meaningful way. growth since first receiving some “committers” – the name given to $5 million worth of funding in Yahoo developed Apache Hadoop those who add code to the THE MOST ADVANCED FLOPPY-DISKS STORED 200 as an enterprise platform 2009. The company has since Hadoop mainframe. expanded to become one of the MEGABYTES OF DATA – THE AVERAGE HOME COMPUTER incorporating this technology, leading providers of Big Data Founded in June 2011, the firm’s allowing companies to run STORES MORE THAN 5,200 TIMES THIS AMOUNT. extensive analytics of both solutions, used by a diverse range dedication to open source has of companies and organisations seen it quickly established as one structured and unstructured data including Expedia, BT, Western of the major players within the – information that doesn’t fit nicely Union, Nokia and eBay. Hadoop community. into tables. In the same way Google indexed users’ search In March 2014, the firm received a The company was set up by 24 Advertisers were some of the earliest adopters With every year that passes more and more behaviour, Hadoop lets $740 million investment from Intel, engineers who were involved in the of the Big Data approach. Rather than hit and ideas transfer from the realm of science organisations learn more about with its 15% stake valuing the firm original development of Hadoop at hope, they could target customers who fiction to that of science fact. customers and consumers. at $4.1 billion. Cloudera is one of a Yahoo, and backed by Benchmark wanted to buy from them. No longer, in theory, handful of Silicon Valley start-ups Capital the firm raised $25 million in For businesses, Big Data arguably presents The system is designed to run on would retirees see ads for 18-to-30 holidays to receive multi-billion dollar November of that year. the biggest opportunity since the birth of a large number of machines that nor would students be forced to sit through valuations from investors pre IPO. the internet itself. Big Data is not just for don’t share any memory or disks. Named after Horton the Elephant details of generous new pension plans. sophisticated technology firms and it is being When an organisation loads its By replacing its own Big Data of the Horton Hears a Who! book, But there are thousands of other applications put to use by all organisations in all industries. data into Hadoop, the software project with Cloudera’s solutions, the company was described by of Big Data that are at various stages of separates that information into Intel provided a strong indication Forrester Research as the Everyone can learn about the world they development. Some are being used widely pieces that it then spreads across that Hadoop is the future of “technology leader and ecosystem inhabit in microscopic detail and use the right now; some are evolving into usable different servers – meaning there Big Data. builder of the entire Hadoop information to their advantage. From products and yet more are mere concepts of is no one place where you industry”. pharmaceuticals to sports science, how things could look in the future. Take manage all of your data. Because businesses that utilise Big Data will create GRITIT, a UK-based firm that helps to clear there are multiple copy stores, competitive advantage and open up new roads during periods of heavy snow. It uses data stored on a server that market opportunities. Met Office data to organise and allocate jobs goes offline or dies can be to regional digger drivers and log work Big Data will not only help businesses make automatically replicated from carried out. smarter decisions, it will also create a new a known good copy. line of smart products and services that use Sending jobs to gritters via smartphone, the information to perform at their peak while company tracks their progress while creating cutting out waste. One thing is for sure, the reports in real time. By analysing the many data explosion is only going to get bigger. variables in the data, the management team can gauge the success of the job and teams can investigate if something goes wrong – even on nights when there are thousands of site visits. 10 www.wandisco.com 11
THINK BIG: BRITAIN’S DATA OPPORTUNITY EXPERT VIEW CLIVE HUMBY CHIEF DATA SCIENTIST, STARCOUNT INVENTOR TESCO CLUBCARD Clive Humby is chief data scientist at Starcount, and merging of data sets. This is Big Data, it is the fan science company, and one half of the different to lots of data you find in a single source such as phone records. husband-and-wife duo behind dunnhumby, the firm credited with the invention of the Taking the retail example further, if we now compare what people buy with their TV Tesco Clubcard – one of the first instances viewing habits or changes in the weather, of Big Data’s early success. we can provide unique and highly specific insights which are already improving the way that businesses serve their customers and I once said that what we are witnessing with how efficiently they run. Big Data is a revolution reminiscent of that seen with the discovery of crude oil. That Business processes, HR and the customer remains true today, as we continue to go will all enjoy a massive payback from this through a process of sourcing and refining revolution. data, to obtain its true value. And this isn’t just beneficial for business. As oil became more plentiful, we developed Big Data has the potential to be the number technology to turn it into plastics, food and one game-changer in healthcare provision other powerful applications and as data is globally. From identifying patterns in today becoming more plentiful, we’re behaviour and illness, to pre-empting getting much better at using it in new and reactions to specific medicines, data has the innovative ways. potential to save millions of lives. The data already exists – we just need to combine it Big Data, just like crude oil, has the potential and analyse it. to become a revolutionary powerhouse of not only industry but society as well. As we become increasingly reliant on data as a major driver of economic and social Yet its value will only be found in a careful development, it is vital we treat it with the application of analysis and refinement. careful analysis and application it requires. At my former company, dunnhumby, we first We must also nurture it. Consumer privacy learned to understand customers by and fair use of data become important analysing what they were buying, where they principles if we want the well to keep flowing. were shopping and when – establishing the Downtime is no longer an option and the effect Tesco Clubcard and transforming Tesco’s of poor analysis will become increasingly business model. Today, this is an established damaging. We must ensure our data is safe process and works well for many retailers. and that the next generation are fully As our experience and ability to analyse data prepared with the skills required to transform improves, it is clear that the true benefits of the new crude oil into the enormously data will come with the intricate mapping valuable asset it has the potential to be. “BIG DATA HAS THE POTENTIAL TO BE THE NUMBER ONE GAME-CHANGER” 12 www.wandisco.com 13
THINK BIG: BRITAIN’S DATA OPPORTUNITY THE FUTURE IS NOW Big Data is often talked about as an asset Data of the sort mined by Tesco is part of a of the future, but for many organisations it much broader set that has sparked particular interest – the “found data” that includes the represents the here and now. Businesses digital exhaust of web searches, credit card across the world are using the information at payments and mobiles pinging the nearest their fingertips to work smarter by creating phone mast. While found data is cheap to products and services that are more relevant to collect, it is in essence a messy collage of datapoints, each collected for disparate their customer base. purposes that is constantly being updated in CHAPTER 02 real time. 02 THE OPPORTUNITY Retailers were among the early adopters of Harnessing found data was one of the early Big Data analytics, perhaps most notably challenges encountered by data scientists, Tesco, whose Clubcard provided the template with a number of critics pointing the finger at for others to emulate. Back in the early 1990s misguided conclusions as reason to be WITHIN Tesco was lagging behind both Marks & Spencer and Sainsbury’s. Fast-forward a few years and it was the UK’s No.1 retailer. sceptical of data’s potential. The UK Meteorological Office is making the weather a Big Data success story, developing The story of Tesco’s rise to dominance is the a revolutionary forecasting service. Events in stuff of legend, but it was rooted in the space, such as solar flares and solar wind, premise that if any business is to be can impact the performance of the electricity successful it has to listen. And as former CEO grid, satellites, GPS systems, aviation and Sir Terry Leahy said: “The best place to find mobile communications used by satellite the truth is to listen to your customer.” operators, electricity and aviation industries – even the armed forces. By analysing the data gleaned from the company’s flagship loyalty card, Tesco was able The project enables the Met Office to analyse to extend tailor-made offers to their customers large amounts of different types of data, with the aim of increasing the return rate. It’s including solar flare imagery from NASA, and a scheme that has since been imitated the provide warnings of space weather events so world over, but it is Tesco’s Clubcard that that the government and businesses can take continues to set the gold standard. appropriate action to minimise its impacts. The Met Office aims to deliver public space weather forecasts, providing stakeholders and interested parties with access to real-time space weather information and predictions. MET OFFICE The Met Office receives about 100 million ‘observation messages’ every day (2014), using an IBM supercomputer that can do more than 100 trillion calculations per second and produce over 20 terabytes of data every day. TESCO CLUBCARD Clubcard now has over 43 million holders globally, processing 6 million transactions a day (2009) and the value of points redeemed was £780 million in 2010/11. 14 www.wandisco.com 15
CHAPTER 02 THE OPPORTUNITY WITHIN THINK BIG: BRITAIN’S DATA OPPORTUNITY THE DATA DISRUPTORS McLaren’s cars are fitted with more than 100 sensors that record thousands of different Timesaving strategies are becoming readings about race conditions, as well as the increasingly common, helping institutions car and driver’s performance. It is said that by make the shift to digital while increasing the the end of an average race enough data has level of service afforded to customers. been logged to fill several telephone books. For more than 150 years Western Union The sensors are not only deployed across has helped make the world a small place, several different cars on race day, but also in EXPERT VIEW originally delivering telegrams and now testing, simulations and practice laps. The data processing money transfers that stretch from is interpreted to find out whether performance BARBARA HOLZAPFEL one corner to the globe to the other. is optimal or if more tweaking is required. It FORMER MD, SAP LABS NORTH AMERICA Today it is one of the world’s largest financial is compared to previous races and different services companies, one that processes drivers to get the best possible picture. $79 billion and generates more than 200 Barbara Holzapfel is former head of SAP Labs, Data is already having massive implications But this technology developed for F1 has terabytes of data annually. for how people work and live and how been revolutionary elsewhere. McLaren the world-leading provider of enterprise The firm is applying Big Data analytics to Electronic Systems is heavily involved business is conducted. It will create new software and software-related services. opportunities in the development of new eliminate wire transfer fraud, speeding up in advanced monitoring for healthcare a process that takes a fraction of the time applications. The tech that was developed to applications and services; anything from demanded by traditional methods. Every day monitor human physiology on the racetrack network infrastructure, to databases, to the the company taps massive databases to sort is the underlying framework for what is used carrier capacity, to the software tools on top through customer information and calculate by doctors in hospitals. And at McLaren Big Data is a huge driving force in the wider of it all. All the existing models for those will the risk that a particular transaction might be Applied Technologies, the company has context of the Internet of Things. Companies be rethought. the result of a scam. When they find a risky been applying F1-tested manufacturing are now collecting, aggregating and analysing To illustrate one such opportunity we could transaction, they block it. technology to new products that would data at a much higher speed than previously look at a case in San Francisco, where 30 per benefit from similar structural techniques – possible. It has opened up a completely new To protect consumers Western Union cent of the traffic in the city comes from drivers such as aerodynamics. dynamic in how businesses are run and how analyses huge streams of data for anomalies, trying to find parking spaces. That has a major they interact with their ecosystem. using Hadoop to deal with the complexity McLaren CIO Stuart Birrell says that Big Data impact on the local and global environment. of its data queries. Not only is this a lot less is underpinning everything the company is But what we are seeing now is just the tip Through innovations in Big Data, you can expensive and more capable than past data working on. The firm says that during racing of the iceberg and innovation will accelerate now find and reserve a clear parking spot warehouse solutions, the fraud prevention season the car is modified every 20 minutes at an unprecedented pace over the next near your destination. You reserve and pay rate is higher and more comprehensive than based on new information gleaned from few years. for this through your mobile. The benefit to ever before. analytics. But the insight provided by Big Data The value proposition is important and it’s up a local coffee company is a tie in to offer is only so good as the queries made by its to companies like SAP to be clear on what the drinks promotion at the same time. It’s a The underlying inspiration behind Big Data engineers. The challenge, as Birrell says, is to user benefits are: what people can do now completely new scenario combining Big applications is competition – the endeavour “give these guys who are naturally inquisitive that they couldn’t before and how valuable Data with the cloud, mobile payments and to gain an advantage over your rival. Perhaps the ability to ask questions that we in IT that is to them. the Internet of Things. nowhere is this better demonstrated than in would have never predicted.” the cutthroat sport of Formula One racing. If anything this shows that Big Data does not erase the need for vision or human insight. On the contrary, we must have business leaders who can spot a great opportunity, understand how a market is developing, WESTERN UNION think creatively and propose truly novel Western Union has 70 million customers in 200 countries, enabling offerings. The successful companies of the 650 million transactions every year, which is around 29 transactions per next decade will be the ones whose leaders second. This has contributed to what has been described as can do all that while changing the way their “one of the world’s largest enterprise data sets”. organisations make many decisions. “WHAT WE ARE SEEING It has more than 200 terabytes of data, and that data is now growing at a rate of 100 terabytes a year. The data comes in from 174 sources NOW IS JUST THE TIP OF including digital platforms, retail locations, and banking partners. THE ICEBERG” 16 www.wandisco.com 17
THINK BIG: BRITAIN’S DATA OPPORTUNITY CASE STUDY JAMIE TURNER CO-FOUNDER AND CHIEF TECHNOLOGY OFFICER, POSTCODE ANYWHERE Jamie Turner is co-founder of Postcode We are comparatively a ‘little’ Big Data Anywhere – an address capture company company. We are not like Google or Facebook, which generate terabytes of data that helps organisations obtain better quality every second. data and use it more efficiently. On a website But we have more access to data than people typically start to input their address the average business and it’s a massive with a postcode and then they pick their house opportunity for us. We treat it as a unique number off a list – Postcode Anywhere does asset that’s very precious – it’s our eyes and that on a very large scale across the world. ears, and is vital to our future success. When you rely on automation in this way, it always has to work, and we need 100% We have been going for 14 years and on a busy uptime. From a simplistic point of view if day we process about 10 million transactions. things don’t work, not only do you not get Our software powers the Royal Mail’s postcode paid, but you also lose credibility and a finder and the Canadian postal service’s positive reputation with your customers that postcode finder and we’re in talks with several has taken many years to build. European data owners to do the same thing. Our example is even more unusual We started collecting search terms and the way because our software is used not only by customers interacted with the service – more our customers, but also by our customers’ for audit purposes than anything else – but customers, so the knock on impact of we realised the type of service we could offer downtime, or lack of access, will affect an customers became more useful as we grew. awful lot of people very quickly. Soon we found ourselves with two major Our postcode software is used in retail pots of data: all the user history going back checkouts and the impact in those situations now almost 15 years – several billion records could mean major losses in revenues for our – and also how our customers interact customers, who simply wouldn’t be able to with us, what pages they visit and what’s process orders for the period of downtime. happened on their account. To the end user, our business is very simple. That search history lets us know exactly what You put a postcode in and you get an people get out of the service, and we use address out the other end. We have built that that information to improve the accuracy business with a reliance on Big Data so quite of searches. We take search engine-type simply, it has to work all the time, no matter technologies and apply them to address and what external issues might put that service location data sets. at risk. “WE HAVE MORE ACCESS TO DATA THAN THE AVERAGE BUSINESS AND IT’S A MASSIVE OPPORTUNITY FOR US” 18 www.wandisco.com 19
THINK BIG: BRITAIN’S DATA OPPORTUNITY HEALTH SCARE “Better information means better care”. That was A BRITISH MIT the message on millions of leaflets distributed Higher education is where the UK must start, to by the UK National Health Service (NHS) in ensure a pipeline of talent that can fuel a British February 2014, offering an opt-out to the new Big Data explosion. care.data central database of medical records. In many respects UK universities lead the world, evidenced by students who come in their droves from across the world to study at Devised as a singular body of information, to institutions such as Oxford, Cambridge, LSE CHAPTER 03 give the health service access to information and Edinburgh. 03 which is otherwise scattered among the UK’s But even the best universities are well behind LAYING THE 10,981 GP surgeries, care.data was to provide the game when it comes to training the next an unparalleled national picture to the generation of tech talent. There isn’t a single medical profession. UK institution that could hold a candle to MIT, FOUNDATIONS “I would say that we are running the health Stanford, Caltech or Berkeley in the States. service blind without it,” was the argument of The difference is partially one of investment, one former GP and director of the Health and but above all one of curriculum and Social Care Information Centre. connectivity. Take Stanford, the umbilical cord Yet no sooner had the public information of the Silicon Valley tech ecosystem, providing campaign landed on the doormats of a ready stream of bright and highly trained households across the UK, than it came graduates ready to staff Google and Apple, or under attack from privacy campaigners. in many cases start their own company. Criticised as a rushed job which neither the In the Valley, universities work in perfect public nor doctors properly understood, it harmony with tech companies, and there is was shelved for a minimum six-month period. an inter-dependence that creates an ongoing The episode highlighted an inherent virtuous circle between them. It’s no different conservatism in the public attitude towards when you look at the fast-emerging clusters the sharing and usage of data. No matter how in India and Israel, to take just two examples. tangible the benefits — in Scotland, for What the UK must address is its own instance, amputations as a result of diabetes fundamental disconnect between universities have fallen 40% since a central database of and business. In both cases, many of the sufferers was created2 — there will always be a institutions are outstanding, but the dialogue kickback from those fearful about the use and and partnership is not. And that has meant our abuse of personal information. higher education lags a step behind the rest of The fate of care.data also suggested that the world, teaching the skills that were all the Britain has some way to go to embrace the rage a decade ago, but not the technical full potential of Big Data. And it’s not just the applications needed by businesses today. scare stories and popular aversion standing in the way of progress. There are serious deficits to be addressed in the skills base and commercial infrastructure before Britain can truly “out-compete, out-smart and out-do the rest of the world” in Big Data, as Chancellor George Osborne called for in his 2014 Budget. 20 2 The Economist, 22.02.14, “Caring and Sharing”. www.wandisco.com 21
CHAPTER 03 LAYING THE FOUNDATIONS THINK BIG: BRITAIN’S DATA OPPORTUNITY CASE STUDY UCI Charles Boicey is the Informatics Solutions They are both useful databases, certainly, Architect for UCI Health, which comprises but they have their restrictions. the clinical, medical education and research The EMR has to be manually updated, with enterprises of the University of California at critical data usually not uploaded until 24 Plumbing in a better interface between In December 2013, the government-owned hours after a medical event has taken place. business and higher education should be seen Royal Bank of Scotland saw thousands take Irvine. US News and World Report listed UCI As a result, they can only present findings as a matter of urgency if Britain is to take to social media after their cards were refused Irvine Medical Center among America’s Best retrospectively, no doubt useful for when a advantage of the Big Data opportunity. The at shopping tills. The previous year, sister Hospitals for the 13th consecutive year in 2013. clinician checks a patient’s medical records experience of a great many UK tech companies company NatWest saw a failed software but not the groundbreaking revolution is that the graduates on offer are not being update lead to chaos, as payments were Earlier this year, the hospital became one of the promised. suitably prepared by their university courses. unable to process from millions of accounts. world’s first medical institutions to embrace the opportunity provided by Big Data. Earlier this year, UCI Medical Center became “The graduates that we have here and in the Entering into the Big Data game is no small one of the first hospitals in the world to EU, unfortunately, are just not as good as undertaking for companies of all sizes. embrace Big Data. We now use Hadoop- those from universities in countries like the That means investing in a battle-ready The medical profession has, as with nearly based technology to process accurate US and India,” is the view of Andrew infrastructure for handling unprecedented every other service and industry, been pattern-set recognitions, use algorithms Humphries, co-founder of UK start-up levels of information. transformed by technology. Where we work to monitor patient recovery for non-linear accelerator The Bakery.3 and the tools we use on a daily basis are a far complications, and build predictive-modeling Without it, disaster can strike, with even a It is fast-growing businesses that do the most short period of downtime potentially leading cry from the hospitals we trained in, let alone systems to minimize deaths caused by to lead innovation in the field, and they do to disenchanted customers, complaints and the hospitals in which we were born. medical error. not have the time or resources to train new even compensation payouts. The future of medicine will be similarly This is drastically improving the level of care recruits. The imperative lies with universities informed by the revolution in Big Data, with we offer our patients, with doctors alerted as to up their game. ACTING ON GOOD INTENTIONS many seeing it as the next frontier in identifying soon as vital signs cross a key threshold. And Larger employers also have a role to play, new cures, minimizing the impact of infection, it has helped ease the burden on our doctors Big Data is well recognised within the UK co-investing in the training of the next or taking pharmaceuticals to the next level. and nurses, whose heavy patient loads as a game-changing technology, officially generation of British software engineers. The prevent round-the-clock observation. recognised as such by the Department for Everything from heart monitors and UK has a proud tradition of developing Business, Innovation and Skills. ventilators, from medicine dispensers to Hadoop is helping UCI Medical create apprenticeship schemes, but we need big thermometers can now send out a seemingly predictive models that the EMR cannot, business to get involved. All the right noises are being made, from infinite amount of data. And it’s been posited especially around heart attack or pneumonia central government to businesses big and It is vitally important to involve businesses in that hugely significant trends can be found patients. We are now able to detect trends in small, about how Britain can take advantage the major training decisions, particularly for a within all this information, crunching the vast our patients that we weren’t able to before of the opportunities it creates. sector that will grow as rapidly as Big Data. amounts of electronic data that’s emitted and act accordingly to prevent hospital Employers are the people best placed to The Alan Turing Institute, announced and collated by the second to extract highly readmission. judge what training is worth investing in, in George Osborne’s 2014 Budget, is a valuable findings. Specifically, we can intercept infections at the providing students with real, on-the-job strong show of intent. But now a news But a major challenge has been managing earliest possible opportunity, minimizing the training and fast-tracking them into decent announcement must become a bricks-and- the data sets in a way that makes it possible risk to the patient. The positive implications jobs with promising progression prospects. mortar development, a hub that will enable to draw conclusions that are coherent and this has for patients are vast and I’ve no doubt Britain to take a leap forward and make up for useful. Some doctors and medical staff still more hospitals will follow our lead. lost time in the Big Data market. INFRASTRUCTURE need convincing that Big Data analytics has In order to commit to Big Data in this way, With the right talent and software relevance for the hospitals of the future, let The Big Data behemoth requires not just we needed to know that nothing will fail – infrastructure in place, there is no reason alone hospitals today. Until recently, we at a ready supply of people to sustain it, we need round-the-clock availability and Britain cannot claim status as one of the UCI Medical Center would have agreed. but the infrastructure to cope with levels reliability of this information every second of world leaders in showing the transformative of information that until recently were We found that the existing data systems did every day. power of Big Data. considered unmanageable. not allow us to manage the flow of data as it Our healthcare system is on the threshold of happened in real time, not to mention store Companies increasingly need constant a radical overhaul as Big Data starts to make the information in its native form. availability of data, and the costs of losing this its first tangible mark and shift our approach access – downtime – can be catastrophic. Most digitally enabled hospitals will be in a most revolutionary way. Soon we will not Large organisations running legacy systems familiar with the current data systems, most be focused on curing people when they are can quickly find themselves in the eye of a notably the Electronic Medical Record (EMR) sick, but will be able to identify what can be storm, when software glitches strike. or Enterprise Data Warehouse (EDW). done to make people well. 3 Computer Weekly, 03.09.13, “UK tech startups face 22 hiring roadblock due to immigration policy”. www.wandisco.com 23
THINK BIG: BRITAIN’S DATA OPPORTUNITY In many ways the term Big Data will soon be As more and more business activity is obsolete. As the data explosion grows, today’s digitised, new sources of information and ever-cheaper equipment combine to bring us big will become tomorrow’s small. into a new era – one in which large amounts of digital information will exist on virtually any topic. The challenge will be to identify the CHAPTER 04 Research conducted by data storage competitive advantage amid all the data. 04 company EMC found that more data is now The financial services sector has begun to THE FUTURE transferred across the internet every second recognise the value, with the University of than was stored across the entire internet Oxford last year reporting that more than just 20 years ago. The same study reported 71% of the industry already uses Big Data that the digital universe will be 40 times its OF DATA analytics. Today’s banks are using new current size by the end of the decade, such is data-driven models in an attempt to return the rate of its expansion. to profit following the financial crisis that While not everyone is embracing Big Data, began in 2008. there is a strong indication that data-driven Most investment houses used to rely on approaches correlate with higher levels of overnight batch to make trading decisions, productivity and increased profitability. meaning their risk management models were The Harvard Business Review found that the forced to use information that soon became more companies identified themselves as out of date. Real time analytics are enabling data-driven, the better they performed better trading and risk decisions, safeguarding on objective measures of financial and them against the threat of collapse. operational results. Cross sector analysis Given the rate at which financial services reported that firms in the top third of their firms are embracing Big Data, it will be vital to industry in the use of data-driven decision ensure appropriate safeguards are in place to making were, on average, 5% more productive protect against technical failure. When and profitable than their competitors. Hurricane Sandy hit the east coast of the US, Elsewhere, a survey conducted by it ripped right through the heart of New York’s Capgemini and the Economist Intelligence financial district. While data was backed up on Unit found respondents said the use of Big servers in New Jersey, the neighbouring state Data has improved business performance on was in the path of the same hurricane leaving average by 26%. Almost 60% of the same the financial markets in disarray. As Big Data respondents said they planned to make a becomes ubiquitous, events like these will no bigger investment in Big Data over the next doubt become increasingly frequent without three years. the appropriate protection. 24 www.wandisco.com 25
CHAPTER 04 THE FUTURE OF DATA THINK BIG: BRITAIN’S DATA OPPORTUNITY As with any sector in its infancy, a wave TV viewing figures have always provided CASE STUDY of new companies will no doubt spring something of a snapshot of how popular up to provide services that deal with problems arising from Big Data – issues and a programme is. For years, TV executives placed great weight in the data gleaned THE ALAN TURING INSTITUTE complications that were not anticipated. Just from transmitters that monitor which shows think about the range of different business are being watched and when. Extrapolated types to have emerged for the internet. from the data gleaned from a few thousand In 2014 the British government announced that Government expects Big Data technologies households, shows are renewed or axed based Big Data was to have a permanent home in the will create some 58,000 more jobs in the UK Not only will the volume of data change, but on the reception given by this small sample. by 2017, contributing some £216 billion to the so too will the variety of sources. Many of the UK. Named after the computer pioneer and UK economy. most important sources of data are relatively But instead of a rough estimate of how Bletchley Park code-breaker, The Alan Turing new, taking the form of messages, updates many people watch a programme, Big Data Science minister David Willetts MP said a and images posted to social networks; affords a pinpoint-accurate representation Institute will focus on new ways of collecting, focus on Big Data would be crucial to ensuring readings from sensors; GPS signals from of how long they watched, when they fast- organising and analysing data. the UK is able to compete with the world's mobile phones and more. forwarded, what they tweeted. It’s claimed most technologically advanced countries. that Netflix pushed ahead with its House of Although healthcare and retail have been "Making the most of large and complex data Cards series because their data found that The government committed to funding among the most successful early adopters, is a huge priority for government as it has the BBC original, programmes featuring some £42 million over the next five years for the public sector will benefit hugely from the potential to transform public and private Kevin Spacey, and films directed by David the project, with universities and other better data management. sector organisations, drive research and Fincher to be in high demand among interested parties encouraged to bid to development, increase productivity and Tom Heath from the Open Data Institute has its users. host the institute. innovation, and enable market-changing said that Big Data can help government cut What’s beyond doubt is that companies Speaking in his Budget speech, Chancellor products and services," he said. "The new costs, be more effective and better serve won’t reap the full benefits of a transition George Osborne MP said: "I am determined data research centres will help the UK grasp their citizens. During bad weather, a local to Big Data unless they’re able to manage that our country is going to out-compete, these opportunities and get ahead in the authority can blend existing data about change effectively. Rather than accepting out-smart and out-do the rest of the world." global race." services, such as road-gritting, with those correlations blindly, data scientists will need for at-risk groups, such as ‘meals on wheels’, The announcement was hailed as having the theory to explain why the patterns are the ensuring that different providers no longer potential to marry the needs of business with way they are. operate in silo. the ability of UK academia. Companies will succeed using Big Data not Public services are an often overlooked simply because they have more or better area for innovation. Emergency services data, but because they have leadership teams have already begun to look to Twitter to that set clear goals, define what success aid their efforts, but a more formal social looks like, and ask the right questions. media strategy can provide early warning mechanisms after a major disaster has taken place. While newspaper reports are divided about Big Data’s potential, the wider media has already started to look to audience data to cut through an ever-expanding marketplace. 26 www.wandisco.com 27
THINK BIG: BRITAIN’S DATA OPPORTUNITY CASE STUDY CHANNEL 4 GILL WHITEHEAD DIRECTOR OF AUDIENCE, TECHNOLOGY AND INSIGHT AT CHANNEL 4 SANJEEVAN BALA HEAD OF DATA PLANNING AND ANALYTICS AT CHANNEL 4 Gill Whitehead and Sanjeevan Bala lead the In January, we announced that 10 million team at Channel 4 that applied Big Data to viewers had agreed to share information about how they used the service. This was better understand its audience and reinvent revolutionary within our industry and the way in which people consume television. something we achieved by creating a value exchange, so when we asked them to give us information we gave them something back. Across our online offering, we’re looking to We started our Big Data journey three years enhance the user experience, both in terms ago and at that time our team had two main of the viewer journey and at what points we priorities. The first, and our main focus, was intervene to make it personal. to use Big Data to build and strengthen direct Big Data enables a collaborative relationship relationships with our audience. with clients and end users. We now offer This is something that was never possible access to our complete archive, exclusive before in linear television, but as TV became areas and the opportunity to watch premiers connected to the internet we suddenly had – the value exchange must be right. a vital return path to the consumer. Amongst all this, we know that to customers Our other priority was to deliver a return back Big Data is still new and many are concerned to the business via Big Data which we did by about privacy – this is a challenge to many creating new advertising products that delivered industries and not just our own. So we more targeted advertising opportunities to created our viewer promise, fronted by Alan clients which allowed us to charge a Carr, who could explain exactly why we ask premium and deliver greater returns. for each piece of data and what we use it for. We use Big Data across the business and we As part of our viewer promise, we don’t want now have more direct marketing opportunities to experiment with anything that could be that we didn’t have before. We’re also looking perceived as being invasive. Our relationship at how Big Data can complement existing with our viewers remains our number one research sources to support commissioning focus and Big Data must benefit that, rather and scheduling of TV programmes. than risk putting it in jeopardy. 28 www.wandisco.com 29
THINK BIG: BRITAIN’S DATA OPPORTUNITY British industry will need 1.25 million new graduates in science, technology, engineering CONCLUSION and maths by 2020 to maintain current employment markets “First came the binge, then came the hangover,” Britain must now capitalise on this ROYAL ACADEMY OF ENGINEERING was the apt summary of a Bloomberg special opportunity. A recent study by EMC found that while the average business will be report on Big Data from June 2014.4 managing 50 times more information than it £42 million committed to The Alan Turing Institute does today, the number of IT staff will only rise by 50%. If that holds, then the UK won’t GEORGE OSBORNE MP MARCH 2014 The desire to scrutinise Big Data’s big be able to fill the 58,000 Big Data jobs that promises has led to a fierce debate over the will be created by 2017. extent of what it can achieve, with critics £73 million additional funding for public sector We have to ensure training programmes are emerging to claim sighting of a new Icarus in place that will provide the skills needed analytics projects flying too close to the sun. to cope. This means having universities with DAVID WILLETS MP FEBRUARY 2014 A Science magazine report on the much- the facilities and courses to train the next heralded Google Flu Trends project accused generation of data scientists and analysts. the search giant of “big data hubris”, The Alan Turing Institute promises to be a 58,000 more Big Data jobs to be created in the suggesting that the flu-tracking service was delivering consistently wayward estimates of huge step in the right direction, but further UK by 2017, contributing £216 billion to the details remain scarce. We need reassurances actual infections. that plans are in place to follow up on the UK economy That followed a Harvard Business Review Chancellor’s promises, particularly with an DAVID WILLETS MP FEBRUARY 2014 article from December 2013, entitled “You election looming in 2015. may not need Big Data after all”, while a Creating Big Data apprenticeship schemes Fortune magazine column the previous will help address this skills shortage in UK Government could save £2 billion in fraud month argued that the tools deployed to analyse the datasets threaten to undermine the short term, and we are delighted that detection, create 2,000 new jobs and generate WANdisco is the first UK firm to offer these. their credibility. This is a move we hope will provide the £3.6 billion in savings But as the dust settles, there is little doubt we blueprint for other companies to follow. SAS/CENTRE FOR ECONOMIC & BUSINESS RESEARCH 2013 will see a new wave of companies emerge, It’s a choice we made because we designed to help manage the realities of a believe that the Big Data promise must world built around data-driven solutions – be substantiated by measureable action. strategies built on intelligent and considered Good intentions will not produce the skilled analysis rather than number-crunching alone. graduate workforce needed to make Britain’s It would be churlish to deny the challenges, Big Data dream a reality, nor will they but lessons will no doubt be learned from bring about the required changes in how early failures. Despite the recent scepticism, companies use and think about data. The smart businesses will be mindful of the time for talking has passed, and we must opportunity presented by Big Data. It is now enter a phase of fiercely urgent action providing an unprecedented insight into to place Britain at the heart of the worldwide customers, sectors and trends. And even if phenomenon that is Big Data. it cannot yet track the spread of disease, it is helping hospitals save lives and offer greater levels of care. 30 4 Bloomberg, “Buried in Big Data”. www.wandisco.com 31
THINK BIG: BRITAIN’S DATA OPPORTUNITY 03 Both government and business need to get better at explaining the possibilities Big Data presents. 01 The information explosion fuelling 04 the Big Data movement will only The UK faces a technical skills continue to grow as the size of PREDICTIONS & shortage and requires both the digital universe expands. long-term and short-term solutions to ensure a steady RECOMMENDATIONS 02 stream of talent to help harness Britain’s Big Data opportunity. Big Data represents this generation’s defining moment of competitive 05 advantage – with the potential to WANdisco believes that Big Data is good for business. It is transforming UK government needs to follow disrupt every industry and sector. the way companies understand their customers, disrupting markets, up its commitment to The and has the potential to eliminate discovery by chance. Alan Turing Institute by making a considerable investment in Britain has an enviable reputation for technological innovation and further education and Big Data with the right focus and investment, Big Data will become one of our apprenticeships. greatest success stories. In Think Big: Britain’s Data Opportunity we have mapped out what needs to be done for the UK to capitalise – here are our key findings. 32 www.wandisco.com 33
ABOUT WANDISCO WANdisco harnesses the power of 100% to realise the possibilities of Big Data. Its unique software helps the world’s most admired and influential organisations to become stronger, more agile and more competitive, and allows innovators in every field to make the new and important discoveries that will shape the future of society. WANdisco believes that, in an era of ubiquitous information, an ability to store and query data is the defining factor of success. Our patented replication technology provides organisations with 100% reliable, real-time access to data with no downtime, data loss or latency – making Big Data invincible. By harnessing the strength, security and stability of 100% certainty, WANdisco unlocks the power of the possible, empowering our customers to push the boundaries of their ambition and benefit from the transformative power of Big Data. Whether to enhance efficiency or protect profits, boost revenue or save lives, with WANdisco one thing is certain: anything is possible. 34
WWW.WANDISCO.COM
You can also read