Big Data Diary

I was recently asked by Figaro Digital to keep a record of all my data interactions over the course of a single day and to explain their significance.

Here’s my story behind my daily big data trail:

Monday 8 April 2013:

I’m heading to Chicago today for a business trip. I have a long flight ahead of me so I’ve prepared several downloads from BBC iPlayer onto my iPad. I’ve also downloaded several audio books onto my iPhone to listen to on the plane whilst working, and for my training runs between meetings up the Lake Michigan coastline.

The data day

I wake early and book a taxi to take me to the airport via my iPhone application. The app knows my home address, it has my credit card details saved and can send a clean, wifi-enabled cab within minutes of ordering. Rather than the driver ringing the doorbell and waking the kids up, an SMS arrives to tell me that the driver is waiting outside. I sneak out of the house and jump into the cab.

As usual I’m bored straight away so I check Facebook, Twitter and LinkedIn, which is something I regularly do first thing every morning. I have received a couple of requests from LinkedIn but post nothing myself. We come into traffic on the north circular and I’m worried about flight times as I always leave it to the last minute. I look at Apple maps to check traffic conditions. Luckily there are no major problems so I should get to the airport on time.

Next I decide to check in for my flight. I downloaded my favourite airline app some months ago which means my boarding card is already waiting and my loyalty card is input so I have access to the lounge and have not missed the valuable points I collect from each flight. It shows me some hotel options with good discounts and at the usual standard of quality which my company affords me, so I select one and book it.

My flight is shown as departing on time but I’m flying out of T5 at Heathrow, famous for its three satellite terminals and I want to know which gate I’m leaving from to give myself enough time. The airline app doesn’t give me the gate details but TripIt does. It’s a neat little app that stores all my travel plans, from flights, hotels and car hire, and keeps track of everything from delays to reward points, constantly alerting me about my travel plans. It also keeps my family in the loop about my whereabouts and allows my colleagues to book the same flights and hotel as me if we’re travelling together.

I arrive at the airport and in less than 40 minutes after leaving my house, big data has entered my world, harvested valuable information and provided me with equally rewarding data.

Harvest and reward

The entertainment downloads I made last night have all been logged and tracked against my profile and they allow me to be targeted with more personalised content in the future. Equally, they know where I’ve downloaded the content from and to, so they know which devices I prefer These are then logged as interest types on my profile for future push personalisation. For the organisations, logging this information in the Enterprise Resource Planning (ERP) platforms enables them to source further content to reach my demographic.

For organisations to do this effectively they need to harvest my data through clever apps and utilities, such as reward programmes. They then need to join the dots and build my profile around all of the interactive touch-points I have with that organisation. Algorithms are then used to extract and understand the data meaning. Finally, all this needs to be applied back to the user through apps and websites.

This enables the user to feel rewarded and ultimately, helps ensure that the organisation gets more custom from that individual. As consumers get more tech savvy, they enjoy utility, such as not worrying about collecting boarding passes, which ensure that the organisations they use the most know they are valued customers and will be rewarded.

Into the cloud

There’s no form of big data in the queue through the security lane. I hope the new security scanners aren’t sending images to the cloud, but something in me suspects that they are.

I often wonder about data privacy and who gets access to what. I run to the lounge, fly through the reception with my app (it is great that there is no need for a boarding card and loyalty card to get in). I grab a coffee just before my TripIt app makes the airport announcement noise – big data has pushed a live feed of departure and boarding announcements to my phone. I take one look and head to the appropriate gate. Jumping on the flight, I get settled and take off for the real clouds. I know for the time being that I’m disconnected, but this will become limited as most airlines are now investing in wifi on planes.

Seven and a half hours later I land at Chicago O’Hare International Airport. Data from the immigration queue, fingerprints, photos and passport details goes straight to the US government’s private cloud. Goodness only knows what happens with that data!

Virtual data – real utility

Once I’ve made my way through security and collected my luggage, it is time to make a decision on how to get to the hotel. I have quite a few meetings throughout the week in various locations. Is it cheaper to use public transport all week, rely on cabs or hire a car? I opt for the car hire. Out comes the iPhone again and I fire up one of my favourite apps provided by Zipcar. I’m already a member of the car club so they have my driving license details and credit card details saved. I type in my location and see that there is a car free for three days and that it’s located in a nearby short-stay car park.

I head to the car and using my iPhone app, I click the ‘car unlock’ button. It sends a message to the cloud, checks my details then sends a message back to the car’s little black box, unlocking the car as if I had pressed the key fob. I hop in and drive to the hotel; the black box keeps a record of my mileage and charges my credit card accordingly. All the while, big data is recording mine and the car’s whereabouts. Now that’s real utility; the ability to hop in a car parked at the bottom of your street and pay as you go without having forms to fill in.

Dropping my bags off at the hotel, I drive to an office complex to get some work done. Each user pays a subscription to use the space (including a hot desk and meeting rooms) and uses their own technology to conduct their day-to-day business. The cloud and big data have a big presence here; they are being used to network, share ideas, innovate, collaborate and work on projects in a more flexible manner. This use of big data is not only about companies harvesting data, it is about using big data and the cloud in new ways to create a virtual, more globalised, connected world.

Easy access

With most of today’s examples, the biggest benefit of big data is to make my life easier. We still have to be vigilant about privacy and companies must continue to only use data to provide an enhanced service, giving consumers something back rather than selfishly marketing at them. All in all, it is about making our worlds much easier.

So, where next for big data? For me it is clear: on an average day I need to see the dots being joined around my touch-points with a particular brand. Whether it is an airline, car hire service or booking a hotel, I don’t want to keep retyping. I want convenience, quick access, intelligence and utility. Organisations that get this formula right and continue to innovate their service culture using big data will benefit in the long-term.

2013 Predictions

To download the podcast to go with this article click here.

The sky’s are getting greyer and the nights drawing in, the year starts to draw to a close and its time again to stick my neck out and put some technology predictions together for 2013.

So I’m firing these out with no particular research other than gut feel and what I’m seeing in our sector.  Here goes…

Mobile will continue (again) to play part of 2013

In the UK and much of the developed world, 4G will be upon us this year.  This means quicker speeds and more capability from our smart phones.  Mobile browsing will be come even more of a requirement for most of the websites out there.  However mobile apps need to be thought through before investing.  We are getting to the tipping point in the app stores now where your apps simply won’t get found.  Brochureware apps are no longer novelty, so we really need to think about real application and utility. Saying all this if you get the utility right or the entertainment factor right then consumers will make the download.  When setting out on your app journey in 2013 think about two things.  How close are you to your customers and would they value a utility or an entertaining app from you on their precious phone real estate? If you get this wrong then you are going to be throwing the investment away.

On the other hand mobile web and mobile web browsing will be major.  If your websites not mobiled up for smart phones then you are going to be singled out and branded slow to adopt.  We know most smartphones can zoom browse and therefore full screen browsing works, however consumers are expecting a fat finger touch experience as the first point of entry, they will switch to your classic site after that.

Also for 2013, here in London, the tube is starting to get wifi-ed up, again increased connectivity such as wifi on planes, the underground and trains will create a pull for mobile users to consume content and that content will often be video. From a developers perspective we need to keep embracing HTML5 and progressive enhancement.  The industry will need to keep pushing the boundaries,  2012 has seen the launch of more html5 compliant websites with richer assets all of which are putting pressure on current website performance however we need to persevere because bandwidth will keep up.  It won’t be quite like moores law but it will improve, so keep innovating.

Tablet Revolution will continue

This is such an obvious one but will continue into 2013. I will explain later how Windows 8 will help buoy this.  Apple launched iPad mini yesterday, Amazon will follow with the fire and I’m left explaining to my colleagues why its all a good idea.  Surely two different sizes of tablet, how can than be a benefit, how will they sell, surely they are cannibalising their other markets in the case of Apple? The answer is handbags, all different sizes but they are all needed and we(well not me) will have more than one. I think Apple have got it right and I’m determined to have all three sizes.

“Connected Big Data”

I’m going to claim this term first before the rest of the tech world gets it.  We have seen big data growing in significance, like cloud computing in 2011, its a buzz word that means something to tech consultancy firms, but what does it really mean to our customers.  My view on this, is we are really starting to see the emergence of connected big data.  i.e. a lots of our customers are starting to join the dots between silos of data and systems with a single purpose to unify around their customer or consumer at the web layer.  The silos of data may be getting joined up globally or just with the sole purpose of providing unified information. We are seeing unification of product data, consumer data, crm data, analytical data and general content all of which require connected specialist with the ability to consult across all levels within an organisation. We typically see four core  data hubs occurring within most global companies;-

1) Product data and enriched product data

Here we are seeing platforms like SAP or Oracle working in collaboration with enrichment tools such as digital asset management platforms and content management platforms.

2) Customer and CRM data

Here we a seeing a unified customer view where we are connecting the customer with common data sources and we are joining the dots between data silos.  Starting with a single identity for a customer.  i.e. how do we create a identity passport and then map the users CRM footprint to that passport as they interact with your brand either via your website or on in the social and mobile worlds.

3) Transaction Data and Analytics

We are seeing even more sophisticated data mining techniques with business intelligence technology fed back to the web layer to optimise user experience and customer engagement.

4) Centralised and localised content

Traditionally the home of the enterprise content management platforms.  However we are seeing architecture and approaches challenging the dominance of these platforms.

So connected big data compared with just big data, sees the joining of dots at a global level between systems with big data silos such as SAP to surface that data to the web layer and allow web layer users to contribute and participate in that data as opposed to just surfacing that data for analytical purposes.


Will the growth in social continue?

Yes most definitely but are we seeing increased demand to play in this area?  I believe we are seeing a plateaux in innovation, which is starting to slow down social network innovation, this means there is less opportunity for growth.  I think it will be still a major part of any digital agency’s portfolio but I think there is a level of maturity beginning to emerge.  Saying this I think we are ahead of the curve and there is still significant motion in this sector with lots of organisation now getting it and starting to put money into social for customer engagement, marketing and application.  So it will still be a big part of 2013 and certainly a time for agency’s to capitalise on the hype.  However innovation is required to lead the pack.  Facebook will continue to grow, but more slowly.  Verticalised social networks like pinterest will also see growth however there will be common sharing standards and approaches starting to emerge such a sign-on protocols like facebook connect.

The biggest advance in innovation will be context sensitive social.  It will be used to drive likes, connect friends and customise information.  Organisation that develop good delivery platforms for this from both a technology and campaign perspective will be able to take most advantage of this market as it starts to mature. Finally watch out for revenue drive innovation from the both Twitter and Facebook they have to do something in 2013.

Will Windows 8 make an impact?

Is Microsoft really on the decline, is Apple’s position now dominant for the next 10 years? As we eagerly await the launch of windows 8, are we really expecting a fundamental relaunch of good old microsoft?  Well Microsoft are expecting to spend big to push this one out.  I think it will start to make an impact on businesses with I.T. departments still hanging onto a level of control and still trying to hold off the day when they can no longer resist their staff enjoying a dose of Apple. However it certainly won’t be a revolution as we saw in 95. We will see interest regenerated by some new sexy devices, even some nicely designed tablet hybrids that will catch on.  But don’t expect too much.

What it will do is start to standardise the tablet and touch.  I know Apple, Samsung and practically every other vendor supports gesture and touch capability and have been doing it for some time. But with Microsoft coming online will mean we will start to see the standard fully adopted. Navigation of apps and web browsing will need to ensure touch is at the core of their design from now on. This will mean innovation in design, html5 and our favourite, Javascript.

Open Source at the Server

Open source is continuing grow and its now proving itself with organisations and in big projects.  One winner in this is Drupal which is rapidly becoming enterprise capable.  This will continue and Drupal could start to rival some big content management vendors. Watch this one closely for developments. We certainly are.

Content Management Platforms

This leads me on to platforms and CMS platforms in particular.  Our favourite has been Tridion for many years.  We feel it will be a push for Tridion to be knocked from its well earned enterprise content management leader. However the ones to watch and not ignore in 2013 are;-

  • Sitecore
  • Drupal
  • Adobe CQ
  • EpiServer

Amaze are developing strategies and approaches to ensure our customer can pick wisely.

E-Commerce Platforms

E-commerce continues to grow and grow.  Choice of platform remains critical and its becoming an increasing requirement as businesses start to look at second generation commerce capability.  Obviously depending on your size will depend on your choice of platform.  But if you are in the enterprise bracket i.e. you are operating a global platform or have substantial business being run through e-commerce then we still would recommend Hybris as number one.  Why? Because it is more complete than any of the other big vendors. i.e. you get more to start with; for example product information platform, e-commerce accelerators, customer support, content management and mobile.  This therefore leads to less complex software integration programmes and less risk.  It also means you can get to market quicker. We feel its still number one and one to embrace in 2013.

Content Delivery Networks

Is a big theme for infrastructure going into 2013.  There are still a lot of companies that have not even looked into this let alone deployed CDN.  With content increasing and globalisation becoming an ever increasing factor content delivery networks are becoming a necessity.  There are only a few key players that do this well;-

  • Akamai
  • MaxCDN
  • Edgecast
  • Amazon
  • Rackspace

So happened to?

Finally I just wanted to wrap up by revisiting last years buzz word and trends that I have not mentioned here.

HTML5 – adoption continues.  Yet still not a ratified standard but the industry is ploughing ahead. It must continue to be embraced.

Cloud – interest and adoption growing more slowly than the hype, however its here to stay and will continue to be a big factor in any big infrastructure project.  Microsoft Axure has been slowest to succeed whilst Amazon is trail blazing.  What we are seeing is growth in vendors starting to provide onDemand services to their traditional software license models.  This is good news for the industry.

Cloud Computing

Cloud computing has certainly gained momentum over the last 12 months.  It has no doubt struck accord with cash strapped businesses.  But our view is cloud computing is too low down the software stack and is predominantly concerned with vitualising platforms.  As more and more businesses compete in this space we see the value of cloud computing moving up the stack and unleashing its service orientated flexibility on the domain of traditional software as a service vendors.  Confusing?  They both operate in the cloud but for pure software as a service vendors to add even more benefits to business they need to reach into organisations.  Clouds will grow tentacles into businesses as the membrane between the traditional I.T. systems and the cloud gets thinner.  What does this mean in real terms?

Data will become the platform but it will extend with its applications into the cloud.  As with the web the data will reside in the cloud along with an app store approach to enterprise applications.  Internal private clouds and infrastructure will become agents to the cloud.  The cloud concept will increase up the value chain where cost savings are only half the reason why people will make the move.

The connected visitor

We are working on some interesting connections between user visits to a website and the development of that user’s digital profile.  What I mean is that a visitor to your website generally takes the form of a hit or visit.  But most users are now starting to connect with your website via some form of authentication or single sign on through their social network accounts. This enables us to build intelligence on that user so we can instantly respond to that user’s likes and needs.

Behind the scenes we can use intelligence to join the dots around that user and look to build a digital footprint that enables us to blend content and navigation around the user.  We can also use this intelligence to  understand the user(taking into account their consent) and segment according to their likes and dislikes.

The next generation website will use a combination of big data and connected intelligence to provide a more tailored browsing experience with content that is both more useful and intuitive.

This type of technology plays dividends when built into e-commerce solutions, both in terms of the templates that display product information but also the dashboard technology that enables e-commerce managers to keep track of the performance of their retail operations. It allows them to have a digital nerve centre monitoring all aspects of the commerce site.

The technology to do this exists in the identity layer of the site. We are seeing the development of SasS based identity management providers that can start to provide connected sign on, intelligence and reporting for your digital estate.  We will report more on this in the next few weeks.

Big data can we make sense of it all?

With Big Data becoming increasingly talked about, are there really solutions out there that can maximise intelligence within this data and make it clear enough for businesses to make sense of it and act on it?

My view is that its not as simple as just rolling out clever analytical inference tools like Autonomy as this just creates another layer of data. What I feel is needed is a set of communication skills that are inbuilt in most digital agencies.  Combining these skills with technology, business knowledge and your business requirements will enable us to produce clever and clear dashboards with your organisation’s data.  Whether its a set of analytical reports or e-commerce conversion statistics, we can design the tools and dashboard to manage your KPI’s, present the information to your team, collaborate on the actions and get moving with the necessary change. Technology alone won’t solve Big Data, but technology combined with a good set of communication and design tools can produce clear answers to your company’s big data challenge, more importantly it will help you and your colleagues make sense and make better decisions.