Directions Hadoop is Moving In

Hadoop is a data system so big it is like a virtual jumbo where your PC is a flea. One of the developers named it after his kid?s toy elephant so there is no complicated acronym to stumble over. The system is actually conceptually simple. It has loads of storage capacity and an unusual way of processing data. It does not wait for big files to report in to its software. Instead, it takes the processing system to the data.

The next question is what to do with Hadoop. Perhaps the question would be better expressed as, what can we do with a wonderful opportunity that we could not do before. Certainly, Hadoop is not for storing videos when your laptop starts complaining. The interfaces are clumsy and Hadoop belongs in the realm of large organisations that have the money. Here are two examples to illustrate the point.

Hadoop in Healthcare

In the U.S., healthcare generates more than 150 gigabytes of data annually. Within this data there are important clues that online training provider DeZyre believes could lead to these solutions:

  • Personalised cancer treatments that relate to how individual genomes cause the disease to mutate uniquely
  • Intelligent online analysis of life signs (blood pressure, heart beat, breathing) in remote children?s hospitals treating multiple victims of catastrophes
  • Mining of patient information from health records, financial status and payroll data to understand how these variables impact on patient health
  • Understanding trends in healthcare claims to empower hospitals and health insurers to increase their competitive advantages.
  • New ways to prevent health insurance fraud by correlating it with claims histories, attorney costs and call centre notes.

Hadoop in Retail

The retail industry also generates a vast amount of data, due to consumer volumes and multiple touch points in the delivery funnel. Skillspeed business trainers report the following emerging trends:

  • Tracing individual consumers along the marketing trail to determine individual patterns for different demographics and understand consumers better.
  • Obtaining access to aggregated consumer feedback regarding advertising campaigns, product launches, competitor tactics and so on.
  • Staying with individual consumers as they move through retail outlets and personalising their experience by delivering contextual messages.
  • Understanding the routes that virtual shoppers follow, and adding handy popups with useful hints and tips to encourage them on.
  • Detecting trends in consumer preferences in order to forecast next season sales and stock up or down accordingly.

Where to From Here?

Big data mining is akin to deep space research in that we are exploring fresh frontiers and discovering new worlds of information. The future is as broad as our imagination.?

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Check our similar posts

Is the GDPR Good or Bad News for Business

The European Union?s General Data Protection Act (GDPR) is a new data authority coming into force on 25 May 2018. It replaces the current Data Protection Directive 95/46/EC, while extending the remit to include the export of personal data outside the EU. It aims to give EU citizens and residents living there more control over their personal information. It also hopes to make regulatory compliance simpler for participating businesses.

The Broad Implications for Business
The GDPR puts another layer of accountability on businesses falling within its remit. It requires them to implement ?comprehensive but proportionate governance measures? including recording how they make decisions. The long-term goal is to reduce privacy infringements. In the short run, businesses without good governance may find themselves writing new policies and procedures.

Article 5 of the European Union?s General Data Protection Act lays down the following guidelines for managing personal data. This shall be ?
? Processed transparently, fairly, and lawfully
? Acquired for specific, legitimate purposes only
? Adequate, relevant and limited to essentials
? Not used for any other, incompatible purpose
? However it may be archived in the public interest
? Kept up to date with all inaccuracies corrected
? Ring-fenced when the information becomes irrelevant
? Adequately protected against unauthorised access
? Stored in a way that prevents accidental loss
Furthermore, affected businesses shall appoint a ?controller responsible for, and able to demonstrate, compliance with the principles.?

Implementing Accountability and Governance
The UK Information Commissioner?s Office has issued guidelines regarding provisions to assure governance and accountability. These are along the lines of the ?don’t tell me, show me? management approach the office has generally been following. In summary form, a business, and its controller must:
? Implement measures that assist it to ensure demonstrated compliance
? Maintain suitable, relevant records of personal data processing activities
? Appoint a dedicated data protection officer if scale makes this appropriate
? Implement technologies that ensure data protection by design
? Conduct data protection assessments and respond to results timeously

Implementing the General Data Protection Act in Ireland
The Irish Data Protection Commissioner has decided it is unnecessary to incorporate the GDPR into Irish law, since EU regulations have direct effect. The office of the Commissioner is working in tandem with data practitioners, and industry and professional bodies to raise awareness in business through 2017. It has produced a document detailing what it considers the essentials for business compliance. Briefly, these pre-requisites are:
? Ensure awareness among key personnel, and make sure they incorporate the GDPR into their planning
? Conduct an early assessment of quality management gaps, and budget for additional resources needed
? Do an audit of personal data held, to determine the origin, the necessity to hold it, and with whom shared
? Inform internal and external stakeholders of the current status, and your future plans to implement the GDPR
? Examine current procedures in the light of the new directive. Could you ?survive? a challenge from a data subject?
? Determine how you will process requests for access to the data in the future from within and outside your organization
? Assess how you currently obtain customer consent to store their data. Is this “freely given, specific, informed and unambiguous”?
? Find how you handle information from underage people. Do you have systems to verify ages and obtain guardian consent?
? Implement procedures to detect, investigate, and report data breaches to the Data Protection Commissioner within 72 hours
? Implement a culture of always assessing the effect on individual privacy before starting new initiatives

So Is the GDPR Good or Bad for Business
The GDPR should be good news for business customers. Their personal data will be more secure, and they should see their rate of spam marketing come down. The GDPR is also good news for businesses currently investing resources to protect their clients? interests. It could however, be bad news for businesses that have not been focussing on these matters. They may have a high mountain to climb to come in line with the GDPR.
Disclaimer: This article is for information only and not intended as a comprehensive guide.

Contact Us

  • (+353)(0)1-443-3807 (IRL)
  • (+44)(0)20-7193-9751 (UK)
Benefits Realisation Frameworks – A Useful Handle

One of the greatest challenges of project management is maintaining top-down support in the face of fluctuating priorities. If you elect to take on the role yourself and are peppered by other priorities, it can be a challenge to exactly remember why you are changing things and what your goals are. Sometimes you may not even notice you have reached your goal.

The Benefits Realisation Chart-room

The Benefits Realisation Model is a framework on which to hang key elements of any project. These traditionally include the following, although yours may not necessarily be the same:

  • Definition of the project goal
  • Quantification of intended benefits
  • Project plan versus actual progress
  • How you know you reached your goal
  • Quantification of actual benefits

Another way of describing Benefits Realisation Frameworks is they answer four fundamental questions that every project manager should know by heart:

  • What am I going to do?
  • How am I going to do it?
  • When will I know it’s done?
  • What exactly did I achieve?

The Benefits Realisation Promise

An astounding number of projects fail to reach completion, or miss their targets. It’s not for nothing that the expression ?after the project failed the non-participants were awarded medals? is often used in project rooms. We’re not saying that it is a panacea for success. However it can alert you to warnings that your project is beginning to falter in terms of delivering the over-arching benefits that justify the effort.

When Projects Wander Off-Target

Pinning blame on participants is pointless when project goals are flawed. For example, the goals may be entirely savings-focused and not follow through on what to do with the windfall. At other times realisation targets may be in place, but nobody appointed to recycle the benefits back into the organisation. This is why a Benefits Realisation Framework needs to look beyond the project manager?s role.

Realisation Management in Practice

If the project framework does not look beyond the project manager?s role, then it is over when it reaches its own targets ? and can even run the risk of being an event that feeds entirely off itself. In order to avoid a project being a means to its own end, this first phase must culminate with handover to a benefits realisation custodian.

An example of this might be a project to centralise facilities that is justified in terms of labour savings. The project manager?s job is to build the structure. Someone else needs to rationalise the organisation.

In conclusion, the Benefits Realisation Framework is a useful way of ensuring a project does not only achieve its internal goals, but also remains a focus of management attention because of its extended, tangible benefits.

Top 10 Disadvantages of Spreadsheets

Fraudulent manipulations in company Excel files have already resulted in Billion-Dollar losses. The main underlying reason behind this spreadsheet vulnerability is the inherent lack of controls, which makes it so easy to alter either formulas, values, or dependencies without being detected.


Disadvantages of Spreadsheets - Kindle

Disadvantages of Spreadsheets

Comprehensive information and data your organisation needs, to circumvent the threats posed by spreadsheets.


Buy Now

1. Vulnerable to Fraud

Of all the spreadsheet disadvantages listed here, this is perhaps the most damaging. Fraudulent manipulations in company Excel files have already resulted in Billion-Dollar losses. The main underlying reason behind this spreadsheet vulnerability is the inherent lack of controls, which makes it so easy to alter either formulas, values, or dependencies without being detected.

2. Susceptible to trivial human errors

While fraud will always be a threat to spreadsheet systems, there is a more significant threat that should make you seriously consider getting rid of these outdated systems. And that is its extreme susceptibility to even trivial human errors. Missed negative signs and misaligned rows may sound harmless.

But when they damage investor confidence or cause a considerable loss of opportunity amounting to millions of dollars (Are we serious? Google up ?spreadsheet horror stories? to find out), you should understand that it?s time to move on to better alternatives.

3. Difficult to troubleshoot or test

So how about testing spreadsheets to mitigate the risks of items 1 and 2? Good luck. Spreadsheets just aren?t built for that. It?s not uncommon to have interrelated spreadsheet data scattered across different folders, workstations, offices, or even geographical locations.

Worse, even if you are able pinpoint the locations of every related file, tracing the logic of formulas from one related cell to another can take ages. It?s pretty obvious now how you?ll also encounter a similar problem when troubleshooting questionable data.

4. Obstructive to regulatory compliance

Combine items 1, 2, and 3, and what do you get? A big headache impacting regulatory compliance. There are number of regulations that have a serious impact on the use of spreadsheets.

Some of the many regulations that impact spreadsheet systems include:

And to think it looks like regulatory bodies are just getting warmed up. Over the last two decades, we’ve seen a surge in regulations that directly affect spreadsheet-based systems. Now, you tell me that you haven?t wished there was a better way to beat regulatory compliance deadlines. Well, if you?re still using spreadsheets, then there certainly is a better way.

5. Unfit for agile business practices

We’re now in an age when major changes are shaping and reshaping the business landscape. Mergers and Acquisitions, Management Buyouts, earthquakes, tsunamis, hurricanes, uprisings, climate change, new technologies, and so on. If your business is not agile enough to adapt to such changes, it could easily be left behind or even face extinction.

Spreadsheets are normally created by individuals who have not the slightest know-how regarding software documentation. In the end, spreadsheet files become highly personalised user developed applications. So when it?s time for a new person to take over as part of a large scale business change, the newcomer may have to start from scratch.

Read further about Implementing Large-Scale Business Change

 

6. Not designed for collaborative work

Planning, forecasting, budgeting, and reporting are all collaborative activities. In other words, plans, forecasts, budgets, and reports typically require information from different individuals belonging to different departments. In addition, the final documents are a result of multiple exchanges of data, ideas, and files.

Now, if your company?s offices are scattered throughout the country or if certain team members are separated by large distances, the only way to exchange data stored in spreadsheets is through email.

Experience will tell you that such a method of exchange is susceptible to duplicate and even erroneous data. Team members will tend to find it hard to keep track of similar files going back and forth, and sometimes even end up sending the wrong version.

7. Hard to consolidate

When it comes to simple data entry and quick ad hoc data analysis tasks, spreadsheets are highly favoured by end users. This has made them one of the most ubiquitous office tools on the planet. But as a consequence, data in spreadsheet-based systems are distributed throughout the organisation.

So when it’s time to generate reports, you’ll really have to go through a slow consolidation process. In most cases, end users would have to collect data from different files, summarise them, and submit the same to their department heads through emails, portable storage media (e.g. CDs or USB flash-drives), or by copying to a commonly shared network folder.

Department heads would have to undergo a similar process before submitting them to their own superiors. This has to go on until all the information reaches their organisation’s top decision makers. Throughout the entire consolidation process, data is subjected to numerous error-prone activities such as copy-pasting, cell entry, and range specification.

8. Incapable of supporting quick decision making

In a spreadsheet-based environment, extracting data from different departments, consolidating them, and summarising the information so that it could aid the company’s top brass in making sound decisions can be very time consuming.

And because we know how susceptible spreadsheets are to errors, everyone involved in the information processing has to be ultra careful to keep the integrity of the data intact. Hence it would be prudent to enforce double-checking as much as possible.

This extra but necessary exercise can further delay the process. So, when the final information arrives at the hands of the top executive, he may not have much time to work with. (Read about Business Intelligence)

9. Unsuited for business continuity

As mentioned earlier, data in spreadsheet systems are never kept in a single place. In fact, it’s the exact opposite. The worse thing about it is that they’re always in the hands of non-IT personnel, who are understandably not familiar with storage and backup best practices.

Thus, if a major disaster strikes, full data recovery can be very difficult if not impossible. As a consequence, even if the company has financial reserves, the absence of data (e.g. accounts receivable records, customer records, and inventory) to work on can prevent the company from making a quick restart.

10. Scales poorly

As an organisation grows, data in spreadsheet-based systems get more distributed; subsequently compounding the issues outlined above. It is absolutely not advisable for a large organisation to keep using spreadsheets.

 

More Spreadsheet Blogs

Spreadsheet Risks in Banks

Top 10 Disadvantages of Spreadsheets

Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry

How Internal Auditors can win the War against Spreadsheet Fraud

Spreadsheet Reporting – No Room in your company in an age of Business Intelligence

Still looking for a Way to Consolidate Excel Spreadsheets?

Disadvantages of Spreadsheets

Spreadsheet woes – ill equipped for an Agile Business Environment

Spreadsheet Fraud

Spreadsheet Woes – Limited features for easy adoption of a control framework

Spreadsheet woes – Burden in SOX Compliance and other Regulations

Spreadsheet Risk Issues

Server Application Solutions – Don’t let Spreadsheets hold your Business back

Why Spreadsheets can send the pillars of Solvency II crashing down

 

Advert-Book-UK

amazon.co.uk

 

Advert-Book-USA

amazon.com

 

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Ready to work with Denizon?