Directions Hadoop is Moving In

Hadoop is a data system so big it is like a virtual jumbo where your PC is a flea. One of the developers named it after his kid?s toy elephant so there is no complicated acronym to stumble over. The system is actually conceptually simple. It has loads of storage capacity and an unusual way of processing data. It does not wait for big files to report in to its software. Instead, it takes the processing system to the data.

The next question is what to do with Hadoop. Perhaps the question would be better expressed as, what can we do with a wonderful opportunity that we could not do before. Certainly, Hadoop is not for storing videos when your laptop starts complaining. The interfaces are clumsy and Hadoop belongs in the realm of large organisations that have the money. Here are two examples to illustrate the point.

Hadoop in Healthcare

In the U.S., healthcare generates more than 150 gigabytes of data annually. Within this data there are important clues that online training provider DeZyre believes could lead to these solutions:

  • Personalised cancer treatments that relate to how individual genomes cause the disease to mutate uniquely
  • Intelligent online analysis of life signs (blood pressure, heart beat, breathing) in remote children?s hospitals treating multiple victims of catastrophes
  • Mining of patient information from health records, financial status and payroll data to understand how these variables impact on patient health
  • Understanding trends in healthcare claims to empower hospitals and health insurers to increase their competitive advantages.
  • New ways to prevent health insurance fraud by correlating it with claims histories, attorney costs and call centre notes.

Hadoop in Retail

The retail industry also generates a vast amount of data, due to consumer volumes and multiple touch points in the delivery funnel. Skillspeed business trainers report the following emerging trends:

  • Tracing individual consumers along the marketing trail to determine individual patterns for different demographics and understand consumers better.
  • Obtaining access to aggregated consumer feedback regarding advertising campaigns, product launches, competitor tactics and so on.
  • Staying with individual consumers as they move through retail outlets and personalising their experience by delivering contextual messages.
  • Understanding the routes that virtual shoppers follow, and adding handy popups with useful hints and tips to encourage them on.
  • Detecting trends in consumer preferences in order to forecast next season sales and stock up or down accordingly.

Where to From Here?

Big data mining is akin to deep space research in that we are exploring fresh frontiers and discovering new worlds of information. The future is as broad as our imagination.?

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Check our similar posts

A Business Case for Sharing

We blogged about sharing services in a decentralised business context recently, and explained why we think why these should be IT-Based for speedy delivery. This is not to say that all shared services projects worldwide have been resounding successes. This is often down to the lack of a solid business case up front. We decided to lay out the logic behind this process.

Management Overview ? The overview includes a clear definition of why the current situation is unacceptable, the anticipated benefits of sharing, and an implementation plan were it to go ahead. The project should not proceed until the stakeholders have considered and agreed on this.

Alternatives Considered ? The next stage is to get closer to the other options in order to determine whether an alternative might perhaps be preferable. Substitutes for shared services are often doing nothing, improving the current method, and outsourcing the service to a third party.

The Bottom Line in Business ? Sharing services comes at an initial cost of infrastructure changes, and the impact on human capital (the latter deserves its own blog). The following need careful consideration from the financial angle:

Numbers to Work Through

  • Manpower to design and roll the project out in parallel with the existing organisation.
  • Capital for creating facilities at the central point including civil works, furniture and equipment and IT infrastructure.
  • The costs of travel, feeding and accommodation. These can be significant depending on the time that implementation takes.
  • The opportunity loss of diverting key staff – and the cost of temporary replacements – if appointing line staff to the project team.
  • Crystal-clear project metrics including (a) the direct, realisable savings (b) the medium and long-term effects on profit and (c) where to deploy the savings

Risk Management

Shared services projects don’t go equally smoothly, although planning should reduce the risk to manageable levels. Nonetheless it is important to imagine potential snags, decide how to mitigate them and what the cost might be.

We believe in implementing shared services on a pilot basis in the business unit that eventually provides them. We recommend building these out to other branches only when new processes are working smoothly.

Moving On From a Decision

We recommend you revisit your management overview, the logic behind it, the assumptions you made, and the costs and benefits you envisage before deciding to go ahead

The final step in proving a business case is doable should be fleshing out your roadmap into a detailed operations plan with dependencies on a spreadsheet.

The General Data Protection Regulation & The Duty to use Encryption

The General Data Protection Regulation, abbreviated to GDPR, raised a storm when it arrived. In reality, it merely tightened up on existing good practice according to digital security specialists Gemalto. The right to withhold consent and to be forgotten has always been there, for example. However, the GDPR brings a free enforcement service for consumers, thus avoiding the need for third party, paid assistance.

The GDPR Bottom Lines for Data Security
Moreover, the GDPR has penalties it can apply, of the order that might have a judge choking on his wig. Under it, data security measures such as pseudonymisation (substitution of identifying fields) and encryption (encoding including password protection) have become mandatory. Businesses must further respect their client data by:

a) Storing it in a secure environment supported by robust services and systems

b) Having proven measures to restore availability and access after a breach

c) Being able to prove frequent effectiveness testing of these measures.

The General Data Protection Regulation places an onus on businesses to report any data breaches. This places us in a difficult situation. We must either face at least a wrist slap upon reporting failures. Alternatively, pay a fine of up to ?10 million, or 2% of total worldwide annual turnover.

The Engineered Weak Link in the System
Our greatest threat of breach is probably when the data leaves our secure environment, and travels across cyberspace to an employee, stakeholder, collaborator, or the client themselves. Since email became open to attack, businesses and individuals have turned to sharing platforms like Dropbox, Google Drive, Skydrive, and so on. While these do allow an additional layer of password protection, none of these has proved foolproof. The GDPR may still fine us heavily, whether or not we are to blame for the actual breach.

How Hacking is Approaching Being a Science
We may make a mistake we may regret, if we do not take hacking seriously. The 10 worst data hacks Identity Force lists are proof positive that spending lots of money does not guarantee security (any more than having the biggest stock of nuclear weapons). We have to be smart, and start thinking the way that hackers do.

Hacker heaven is finding an Experian or a Dun & Bradstreet that may have shielded 143 million, and 33 million consumer records respectively, behind a single, flimsy cyber-security door. Ignorance is no excuse for them. They should simply have known better. They should have rendered consumer data unreadable at individual record level. The hackers could have found this too demanding to unpick, and have looked elsewhere.

How Data Encryption Can Help Prevent Hackers Succeeding
Encrypting data is dashboard driven, and businesses need not concern themselves about it works. There are, however, a few basic decisions they must take:

a) Purge the database of all information held without explicit permission

b) Challenge the need for the remaining data and purge the nice-to-haves

c) Adopt a policy of encrypting access at business and customer interfaces

d) Register with three freemium encryption services that seem acceptable

e) After experimenting, sign up for a premium service and be prepared to pay

Factors to Consider When Reaching a Decision
Life Hacker?suggests the following criteria although the list is a one-size-fits-all

a) Is the system fast, simple, and easy to operate

b) Can you encrypt hidden volumes within volumes

c) Can you mass-encrypt a batch of files easily

d) Do all other files remain encrypted when you open one

e) Do files automatically re-encrypt when you close them

f) How confident are you with the vendor, on a scale of 1 to 10

It may be wise to encrypt all the files on your system, and not just your customer data. We are always open to a hack by the competition after our strategic planning. If we leave the decision up to IT, then IT, being human may take the easy way out, and encrypt as little as possible.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
Web Analytics

There’s a vast ocean of raw customer data on the Web. Ever thought of the implications if somehow you could harness all that data and transform it into useful information? Information that perhaps you can use in your SEO (Search Engine Optimisation) and conversion optimisation?

There are web analytics tools you can employ for these purposes. But using web analytics tools will only win you half the battle. You’ll have to be proficient in configuring these tools to generate insightful and actionable results out of them. A poorly configured tool can produce confusing or even misleading information.

Our web analysts possess the expertise to configure and use web analytics tools, as well as analyse results and leverage information obtained from them.

These are the things we can do to help you take advantage of web analytics.

  • Discuss with your managers to establish your specific goals, to determine what specific data we have to collect/analyse and to plan out how to go about with the entire process.
  • Help you select an appropriate tool, install it and set optimal configurations including page tags, filters, funnels, reports and others.
  • Wield the full force of your analytics tool(s) to make sound business decisions.
  • Monitor the entire web analytics system and implement adjustments when needed.

Ready to work with Denizon?

Contact Us Today