Directions Hadoop is Moving In

Hadoop is a data system so big it is like a virtual jumbo where your PC is a flea. One of the developers named it after his kid?s toy elephant so there is no complicated acronym to stumble over. The system is actually conceptually simple. It has loads of storage capacity and an unusual way of processing data. It does not wait for big files to report in to its software. Instead, it takes the processing system to the data.

The next question is what to do with Hadoop. Perhaps the question would be better expressed as, what can we do with a wonderful opportunity that we could not do before. Certainly, Hadoop is not for storing videos when your laptop starts complaining. The interfaces are clumsy and Hadoop belongs in the realm of large organisations that have the money. Here are two examples to illustrate the point.

Hadoop in Healthcare

In the U.S., healthcare generates more than 150 gigabytes of data annually. Within this data there are important clues that online training provider DeZyre believes could lead to these solutions:

  • Personalised cancer treatments that relate to how individual genomes cause the disease to mutate uniquely
  • Intelligent online analysis of life signs (blood pressure, heart beat, breathing) in remote children?s hospitals treating multiple victims of catastrophes
  • Mining of patient information from health records, financial status and payroll data to understand how these variables impact on patient health
  • Understanding trends in healthcare claims to empower hospitals and health insurers to increase their competitive advantages.
  • New ways to prevent health insurance fraud by correlating it with claims histories, attorney costs and call centre notes.

Hadoop in Retail

The retail industry also generates a vast amount of data, due to consumer volumes and multiple touch points in the delivery funnel. Skillspeed business trainers report the following emerging trends:

  • Tracing individual consumers along the marketing trail to determine individual patterns for different demographics and understand consumers better.
  • Obtaining access to aggregated consumer feedback regarding advertising campaigns, product launches, competitor tactics and so on.
  • Staying with individual consumers as they move through retail outlets and personalising their experience by delivering contextual messages.
  • Understanding the routes that virtual shoppers follow, and adding handy popups with useful hints and tips to encourage them on.
  • Detecting trends in consumer preferences in order to forecast next season sales and stock up or down accordingly.

Where to From Here?

Big data mining is akin to deep space research in that we are exploring fresh frontiers and discovering new worlds of information. The future is as broad as our imagination.?

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Check our similar posts

Transformation to a process based organisation

Today’s global marketplace rewards nimble organisations that learn and reinvent themselves faster than their competition. Employees at all levels of these organisations see themselves as members of teams responsible for specific business processes, with performance measures tied to the success of the enterprise. As team members, they are “owners” of the process (or processes) to which they are assigned. They are responsible for both the day to day functioning of their process(s), and also for continuously seeking sustainable process improvements.

Transforming a traditionally designed “top down control” enterprise to a process-based organisation built around empowered teams actively engaged in business process re-engineering (BPR) has proven more difficult than many corporate leaders have expected. Poorly planned transformation efforts have resulted in both serious impacts to the bottom line, and even more serious damage to the organisation’s fabric of trust and confidence in leadership.

Tomislav Hernaus, in a publication titled “Generic Process Transformation Model: Transition to Process-based Organisation” has presented an overview of existing approaches to organisational transformation. From the sources reviewed, Heraus has synthesised a set of steps that collectively represent a framework for planning a successful organisational change effort. Key elements identified by Hernaus include:

Strategic Analysis:

The essential first step in any transformation effort must be development of a clear and practical vision of a future organisation that will be able to profitably compete under anticipated market conditions. That vision must be expected to flex and adjust as understanding of future market conditions change, but it must always be stated in terms that all organisational members can understand.

Identifying Core Business Processes:

With the strategic vision for the organisation in mind, the next step is to define the core business processes necessary for the future organisation to function. These processes may exist across the legacy organisation’s organisational structures.

Designing around Core Processes:

The next step is development of a schematic representation of the “end state” company, organised around the Core Business Processes defined in the previous step.

Transitional Organisational Forms/ Developing Support Systems:

In his transformation model, Hernaus recognises that information management systems designed for the legacy organisation may not be able to meet the needs of the process management teams in the new organisation. Interim management structures (that can function with currently available IT system outputs) may be required to allow IT professionals time to redesign the organisation’s information management system to be flexible enough to meet changing team needs.

Creating Awareness, Understanding, and Acceptance of the Process-based Organisation:

Starting immediately after the completion of the Strategic Analysis process described above, management must devote sufficient resources to assure that all organisation members, especially key managers, have a full understanding of how a process-based organisation functions. In addition, data based process management skills need to be provided to future process team members. It is not enough to schedule communication and training activities, and check them off the list as they are completed. It is critical that management set behavioural criteria for communication and training efforts that allow objective evaluation of the results of these efforts. Management must commit to continuing essential communication and training efforts until success criteria are achieved. During this effort, it may be determined that some members of the organisation are unlikely to ever accept the new roles they will be required to assume in a process-based organization. Replacement of these individuals should be seen as both an organisational necessity and a kindness to the employees affected.

Implementation of Process Teams:

After the completion of required training AND the completion of required IT system changes, process teams can be formally rolled out in a planned sequence. Providing new teams with part time support by qualified facilitators during the firsts weeks after start-up can pay valuable long term dividends.

Team Skill Development and Continuous Process Improvement:

Providing resources for on-going skill development and for providing timely and meaningful recognition of process team successes are two keys for success in a process-based organisation. Qualified individuals with responsibility for providing training and recognition must be clearly identified and provided with sufficient budgetary resources.

The Hernaus model for transformation to a process based organisation is both well thought out and clear. His paper provides an ample resource of references for further study.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
Server Application Solutions – Don’t Let Spreadsheets Hold Your Business Back

The problems and limitations of spreadsheet-based systems are well documented. That’s why we at Denizon have come up with ways to give you freedom from these UDAs (User Developed Applications). With the server application solutions we offer, your IT and financial system can be:

Totally devoid of spreadsheet risks

By getting rid of spreadsheets, you also get rid of broken links, incomplete range selections, accidental deletion of cells, incorrect copy-pasting and other spreadsheet-related slip-ups.

In their place, we offer a faster but more robust and reliable centralised system. Errors are substantially minimised by built-in controls, while inconsistencies are avoided because changes made by one user are automatically reflected on the data delivered to others.

Built-in business-critical controls

Some solutions are designed to add control features on spreadsheets. We believe that such features can only be truly effective in today?s fast-paced and dynamic business environment if they are already inherent in the design of the IT solution; not something that’s merely added as an afterthought.

For one, while these band-aid solutions may succeed in adding controls, they don’t get rid of the slow, tedious, and time-consuming processes that accompany spreadsheet systems.

Less prone to fraud

Weak controls and the absence of reliable audit trails are two factors that encourage fraudsters to prey on spreadsheet systems.

With our server-based applications solutions, your data is protected by user-based access controls that allow users to see only the information that they’re supposed to see and modify data which they have been granted sufficient access rights to.

Our solutions also produce clear audit trails for painless tracking, viewing and searching of user-entered changes. This will enable you to pinpoint who changed what, as well as where and when the changes were made.

Ready for regulatory compliance and beyond

When better controls are enforced, financial reports become more reliable. That should give your company the edge it needs to easily comply with SOX as well as other regulations and, as a consequence, build stakeholder confidence.

And because our solutions can churn out accurate reports for regulation compliance at shorter turnaround times than spreadsheet systems, you end up saving more man-hours. That should give your team more time to innovate, analyse information and deliver goods or services to your customers faster.

Designed for agility

Let’s face it. Spreadsheets, which used to serve as nifty ad-hoc business tools, are no longer suitable for agile organisations. When faced with the demands of rapidly changing markets and dynamic environments, spreadsheets can instead slow a business down.

Multi-dimensional reports, dashboards, report filters, drill-downs, collaboration and automated reporting, budgeting and forecasting capabilities are needed for gaining insights and making fast critical decisions.

Sad to say, your trusty spreadsheet application is not designed to provide these features. Hence, it’s time to move on to the type of solutions that are.

Our solutions can transform your IT and financial systems and make them better-equipped to meet the demands of today?s rapidly changing economic environment. With features designed for agile businesses, our solutions can help you tackle change with ease.

Automatic consolidation eliminates errors and wasted time caused by tedious copy-pasting of data and linking of cells.

Better collaboration capabilities allows team members to bring their heads together for planning, budgeting and reporting even while on the go.

Mobility support enables users to input data or retrieve information through their wireless mobile devices.

Superior sharing features ensures that everyone is exactly on the same page and viewing real-time information.

Dashboards provide insightful information at-a-glance through KPIs, graphs and various metrics.

Drill-downs enable users to investigate unusual figures and gain a better understanding of the details that contribute to the big picture.

Easy to learn interfaces allow your organisation to cope with fast personnel turnaround or Mergers & Acquisitions.

More Spreadsheet Blogs


Spreadsheet Risks in Banks


Top 10 Disadvantages of Spreadsheets


Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry


How Internal Auditors can win the War against Spreadsheet Fraud


Spreadsheet Reporting – No Room in your company in an age of Business Intelligence


Still looking for a Way to Consolidate Excel Spreadsheets?


Disadvantages of Spreadsheets


Spreadsheet woes – ill equipped for an Agile Business Environment


Spreadsheet Fraud


Spreadsheet Woes – Limited features for easy adoption of a control framework


Spreadsheet woes – Burden in SOX Compliance and other Regulations


Spreadsheet Risk Issues


Server Application Solutions – Don’t let Spreadsheets hold your Business back


Why Spreadsheets can send the pillars of Solvency II crashing down

?

Advert-Book-UK

amazon.co.uk

?

Advert-Book-USA

amazon.com

Data Leakage Prevention – Protecting Sensitive Information

When DuPont lost $400 million in intellectual property, it wasn’t because a hacker from the other side of the world infiltrated their system. The information was simply stolen by a former employee. Alarmingly, data loss incidents are not always caused by deliberate actions.

A file containing personal information accidentally attached to an email and sent to multiple recipients; financial data stored in a USB pen drive, accidentally left in a restaurant; or bank account data of colleagues, inadvertently posted on a company website – these are also some of the everyday causes of data loss.

A report done by research company Infowatch regarding global data leaks in 2010 showed that there were actually more accidental data leaks in that year compared to intentional ones. Accidental leaks comprised 53%, while intentional leaks comprised 42% (the rest were unidentified).

But even if they ?only? happened accidentally, breach incidents like these can still be very costly. The tens of thousands of dollars that you could sometimes end up paying in civil penalties (as in the case when you lose other people?s personal information) can just be the beginning. More costly than this is the loss of customer and investor confidence. Once you lose those, you could consequently lose a considerable portion of your business.

Confidential information that may already be leaking out right under your nose

With all the data you collect, process, exchange, and store electronically every day, your IT system has surely now become a storehouse of sensitive information. Some of them, you may be even taking for granted.

But imagine what would happen if any of the following trade secrets fell into the wrong hands: marketing plans, confidential customer information, pricing data, product development strategies, business plans, supplier information, source codes, and employee salaries.

These are not the only kind of data that you should be worried about. You could also get into trouble if your sloppy IT security fails to protect employee or client personal information such as their names; social security numbers; drivers license numbers; or bank account numbers and credit/debit card numbers along with their corresponding PINs.

In some countries, you could face onerous data breach notification requirements and heavy fines when these kind of data are involved.

There are now more holes to plug

It’s not just the different varieties of sensitive electronic information that you have to worry about. Because these data can take on different forms, i.e. data-at-rest, data-in-motion, and data-at-the-endpoints, you also need to take aim at different areas in your IT system.

Sensitive information can be found ?at rest? in each of your employees? hard disks, in your servers, storage disks, and in off-site backup disks. They can also be found ?in motion? in email, instant messaging, social networking messaging, P2P file sharing, ftp, http, and so on.

That’s not all. Your highly mobile workforce may have already introduced yet another high-risk area into your system: data-at-the-endpoints. This includes USB flash-disks, laptops, portable hard disks, CDs, and even smartphones.

The main challenge of data leak prevention

Having been made aware of the various aspects of data leakage, have you already come to grips with the extent of the task at hand?

There are two major things you need to do here to prevent data leakage.

One, you need to identify what data you have that can be considered as sensitive/confidential information. Of course you have financial information and employee salaries in your files. But do you also store personally identifiable information? Do you have trade secrets that are stored in electronic form?

Two, you need to pinpoint their locations. Are they only on your hard disks and laptops? Or have they made their way to flash drives, CDs/DVDs, or portable HDDs? Are they being transmitted through email or any other file transfer media?

The reason why you need to know what your sensitive data are as well as where they are is because you would like all efforts of securing them to be as efficient and unobtrusive as possible.

Let’s say, as a way of protecting your data, you decide to implement encryption. Since encryption can consume a lot of storage space and significantly reduce performance, it may be impractical to encrypt your entire database or all your files. For the same reason, you wouldn’t want to encrypt every single email that you send.

Thus, the best way would be to encrypt only the data that really need encryption. But again, you need to know what data needs to be encrypted and where those data can be found. That alone is no simple task.

Not only will you need to deal with the data you already have, you will also have to worry about the data that will go through your systems during the course of your day-to-day transactions.

Identifying sensitive data as it enters or leaves your system, goes through your network, or gets stored in your file system or database, and then applying the necessary security actions should be done automatically and intelligently. Otherwise, you could end up spending on a lot of man-hours or, worse, wasting them on a lot of false positives and negatives.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Ready to work with Denizon?

Contact Us Today