A recent IBM survey found that over half of business leaders today realize they don’t have access to the insights they need to do their jobs. They just aren’t harnessing the data.
Today, surrounded by sensors, apps and systems, we have ability to generate and store data cheaply like never before. Ironically, as the data piles up, the ranks of organizations able to process it is declining, and the talent shortage for managing it is increasing.
The ability to harness and leverage data is now a big competitive advantage. This doesn’t mean that experience, judgment and intuition are less relevant; on the contrary, you need those attributes to evaluate data.
You do not need a million dollar budget or a Ph.D.in Computer Science to be a Big Data Voodoo Daddy – one who knows where the data is, can harness it, and mines it daily to delight customers and boost revenue. Devise a basic plan or framework, select a few tools, apply them to your most important data, and you’re started. Just be sure you’re improving on existing data and processes; you can’t automate a vacuum.
Basic Framework
Ask: Where is your data? What does it look like? It generally has 3 characteristics:
- Volume – nobody is scared anymore by the prefix “giga” as in gigabytes (GB). The average hard drive stores hundreds of GB. So, the average U.S. household has a place for it. Of course, there is always the cloud computing and storage option, as long as you don’t mind internet and power outages bringing your business to its knees.
- Variety – here it gets a little hairy. Data is everywhere, but not organized. Some of it lies dormant in locked spreadsheets; some of it is in silos, like Point of Sale receipt reports, accounting ledgers, and customer records. Some of it is “unstructured” i.e. not contained in nice neat rows and columns, but rather in text records and notes. You need tools to organize and standardize data so it will fit nicely into your main database. This is tedious and messy, but worth doing. Fortunately, most major software providers understand this too and have enabled their products to interoperate with one another.
- Velocity – some data is historic; some is collected periodically and batched; some data is streaming live (example: your location in a GPS app, your favorite mobile shopping app). Does live streaming data matter to you? It should if your customers are mobile, and these days who isn’t?
Basic Tools
You need 3 things to harness the volume, variety and velocity of your data:
- a database / storage place (love that cloud storage option!)
- software to help you access, manage, report, cleanse, update, analyze and act on it;
- helper apps to get help standardize data from multiple “feeds” so they can enter your main storage place.
While some of this may sound geeky, be assured that the tooling and resources are becoming more usable. For some large organizations, however, expert talent and technology remain the best option.
One Big challenge Remains
The single biggest challenge of data these days is its quality. In that huge haystack of data, there are some gems and usually a bunch of stale (call it historical) data. Stale and historical data are okay, and even valuable for trend spotting and progress reporting. You can easily take steps to update it via customer surveys, sorting, web / email response forms, etc. Most people and related data sources will happily keep their info in your database current if it helps you stay relevant to one another. Just be sure to keep that process easy, and clearly spell out the benefits. Consider sharing your data discoveries with them, too – at least to the extent that you do not infringe on people’s privacy.
Best Uses for Big Data
According to McKinsey Global Institute, big data has five broad opportunity areas: increased transparency and use, improved performance management, better decisions, greater precision in meeting customer demand, and more targeted R&D. If you are in Sales and Marketing management, web and social media analytics, financial reporting, call center reporting, fraud and security detection, energy usage, safety management, risk and opportunity management, inventory, assets, logistics, agriculture or health care, you have the opportunity to excel and lead in your marketplace by leveraging available data.
You must use these powers only to do good. Concerns about privacy, access, security, intellectual property rights and liability must be factored into our thinking, policies and practice concerning the use of data.
Got data challenges? Drop a comment. I’ve listed a few helpful resources below.
Cheers, ~Ed
Additional Resources (see sidebar on this site for tool ideas)
Pluris Intro (Pluris Marketing) – OCDP (omni-channel dynamic profiling) so big orgs can treat people as individuals
Aryng (“A-ring”) – Analytics help, e-zine, training etc. for big data people
Big Data – What is It? (IBM)
Don’t Build a Database of Ruin (HBR)
Big Data: The Next Frontier for Competition (McKinsey Global Institute)
