Directions Hadoop is Moving In

Hadoop is a data system so big it is like a virtual jumbo where your PC is a flea. One of the developers named it after his kid?s toy elephant so there is no complicated acronym to stumble over. The system is actually conceptually simple. It has loads of storage capacity and an unusual way of processing data. It does not wait for big files to report in to its software. Instead, it takes the processing system to the data.

The next question is what to do with Hadoop. Perhaps the question would be better expressed as, what can we do with a wonderful opportunity that we could not do before. Certainly, Hadoop is not for storing videos when your laptop starts complaining. The interfaces are clumsy and Hadoop belongs in the realm of large organisations that have the money. Here are two examples to illustrate the point.

Hadoop in Healthcare

In the U.S., healthcare generates more than 150 gigabytes of data annually. Within this data there are important clues that online training provider DeZyre believes could lead to these solutions:

  • Personalised cancer treatments that relate to how individual genomes cause the disease to mutate uniquely
  • Intelligent online analysis of life signs (blood pressure, heart beat, breathing) in remote children?s hospitals treating multiple victims of catastrophes
  • Mining of patient information from health records, financial status and payroll data to understand how these variables impact on patient health
  • Understanding trends in healthcare claims to empower hospitals and health insurers to increase their competitive advantages.
  • New ways to prevent health insurance fraud by correlating it with claims histories, attorney costs and call centre notes.

Hadoop in Retail

The retail industry also generates a vast amount of data, due to consumer volumes and multiple touch points in the delivery funnel. Skillspeed business trainers report the following emerging trends:

  • Tracing individual consumers along the marketing trail to determine individual patterns for different demographics and understand consumers better.
  • Obtaining access to aggregated consumer feedback regarding advertising campaigns, product launches, competitor tactics and so on.
  • Staying with individual consumers as they move through retail outlets and personalising their experience by delivering contextual messages.
  • Understanding the routes that virtual shoppers follow, and adding handy popups with useful hints and tips to encourage them on.
  • Detecting trends in consumer preferences in order to forecast next season sales and stock up or down accordingly.

Where to From Here?

Big data mining is akin to deep space research in that we are exploring fresh frontiers and discovering new worlds of information. The future is as broad as our imagination.?

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Check our similar posts

How DevOps Could Change Your Business

Henry Ford turned the U.S. auto industry on its head when he introduced the idea of prefabricating components at remote sites, and then putting them together on a production line. Despite many industries following suit, software lagged behind until 2008, when Andrew Clay Shafer and Patrick Debois told the Agile Conference there was a better way to develop code:
– Write the Code
– Test the Code
– Use the Code
– Evaluate, Schedule for Next Review

The term ?DevOps? is short for Development and Operations. It first appeared in Belgium, where developers refined Shafer and Depois? ideas. Since then, DevOps became a counter movement against the belief that software development is a linear process and has largely overwhelmed it.

DevOps – A Better Way

DevOps emerged at an exciting time in the IT industry, with new technology benefiting from a faster internet. However, the 2008 world recession was also beginning to bite. Developers scampered to lower their human resource costs and get to market sooner.

The DevOps method enabled them to colloborate across organizational boundaries and work together to write, quality assure and performance test each piece of code produced in parallel.
DevOps? greater time-efficiency got them to market sooner and helped them steal a march on the competition.

There are many advantages to DevOps when we work in this collaborative way. Cooperation improves relationships between developers, quality assurers and end users. This helps ensure a better understanding of the other drivers and a more time-effective product.

Summary of DevOps Objectives

DevOps spans the entire delivery pipeline, and increases the frequency with which progress is reviewed, and updates are deployed. The benefits of this include:

? Faster time to market and implementation

? Lower failure rate of new releases

? Shortened lead time for bug fixes and updates

The Psycho-Social Implications of DevOps

DevOps drills through organization borders and traditional work roles. Participants must welcome change and take on board new skills. Its interdepartmental approach requires closer collaboration across structural boundaries and greater focus on overarching business goals.

Outsourcing the detail to freelancers on the Internet adds a further layer of opportunity. Cultures and time zones vary, requiring advanced project management skills. Although cloud-based project management software provides adequate tools, it needs an astute mind to build teams that are never going to meet.

The DevOps movement is thus primarily a culture changer, where parties to a project accept the good intentions of their collaborators, while perhaps tactfully proposing alternatives. There is more to accepting a culture than using a new tool. We have to blend different ways of thinking together. We conclude by discussing three different methods to achieve this.

Three Ways to Deploy DevOps in your?Organisation

If you foresee regular DevOps-based projects, consider running your entire organisation through an awareness program to redirect thinking. This will help non-participants understand why DevOps members may be ?off limits? when they are occupied with project work. Outsourcing tasks to contracting freelancers can mitigate this effect.

There are three implementation models associated with DevOps although these are not mutually exclusive.

? Use systems thinking. Adopt DevOps as company culture and apply it to every change regardless of whether the process is digital, or not

? Drive the process via increased understanding and feedback from key receivers. Allow this to auto-generate participative DevOps projects

? Adopt a continuous improvement culture. DevOps is not only for mega upgrades. Feedback between role players is paramount for success everywhere we go.

You can use the DevOps concept everywhere you go and whenever you need a bridge to better understanding of new ideas. We diminish DevOps when we restrict its usefulness to the vital role it plays in software development. The philosophy behind it belongs in every business.

What is Business Intelligence?

How well do you know your customers? That is, can you actually pinpoint which among them are you most profitable with and which are making you spend more? Are you content with the accuracy of your forecasts and market predictions? Do you feel you’re spending more on legal costs and regulatory compliance than you should?

Your IT department may be handling these concerns pretty well but perhaps you’d like to know how you can further improve things.

What we’ve got is an IT solution wrapped in a fancy name called ‘Business Intelligence’ or BI. If you think that’s too strong a term, we invite you to read more below, then you be the judge.

Dashboards – Determine the health of your business at a glance

Most drivers rarely make use of their car’s dashboard. After all, you can still reach your destination by just using the steering wheel, pedals, gear stick and so on. But that’s not exactly the most efficient way to drive, right?

If you want to save on fuel, you’ll want to glance on the RPM and speedometer from time to time. You might also want to utilise the trip meter to determine which route is the shortest to a given destination. Other dashboard components like the fuel gauge, tire pressure gauge, engine temperature indicator, and volt meter can likewise provide information about your car’s health.

The same concept applies to business management. If you want to run your business intelligently, you can make use of BI dashboards. These are tools in a typical business intelligence package that will allow you to determine the health of your business via a set of smartly configured gauges and other intuitive graphical representations.

So that, literally, at a mere glance, you’ll already know whether various units in your company are working efficiently. A dashboard will also give you instant feedback of the strategies you’ve recently implemented; to let you know if things are working as planned.

If you want more information than a dashboard can provide, our BI packages also include highly customised reports.

Reports that help you decide faster

Dashboards are great for getting valuable information at a glance but they won’t tell you everything. For more details, you’ll need to view highly customised reports. Our reports are tailor made for each user. We see to it that, by default, each person gets the information he needs the most.

If you belong to the sales department, you normally won’t need a presentation of the data that is appropriate for people in accounting. That way, you don’t spend time filtering. Instead, you and your people can move on to making well-informed decisions.

Our BI systems make use of your vast collection of data to provide reports that will organise your regulatory requirements and call your attention to approaching deadlines. The same system will provide the right information for your people on the field. If your team members are equipped with smart phones and Pocket PCs, they can retrieve whatever it is they need to know to close deals, make sales, and serve clients faster than the competition.

Generating logical information from disparate sources of data scattered over an enterprise-wide organisation is no easy task. But we’ll make it look simple. That’s because we’ve got the expertise to bring it all together into a robust data warehouse and to extract them in the form of reports and dashboards through OLAP.

OLAP and Data Warehousing – Powering the generation of actionable information

Want to know how to generate reports with the highest degree of accuracy and reliability? In theory, what you need is a single repository or a data warehouse. That is, order receipts, sales invoices, as well as customer & supplier data is integrated with regulatory details, personnel data, and others. These are all specially organised for future reporting and analysis.

However, data, no matter how all-embracing, is useless until it is processed into actionable information. Through OLAP or Online Analytical Processing, you can seamlessly collect all relevant data from your vast repository to answer queries like “What is our company’s profitability for the 2nd quarter in all identified key cities for our top-of-the-line products?”.

The strength of OLAP lies in its inherent ability to perform data analysis and very complex calculations, thus enabling it to return complex queries much faster than other database technologies. It is therefore suitable for very large data sources, i.e., data warehouses.

Dashboards and reports will only give your organisation the edge if the information retrieved is reliable, fast, and accurate – exactly the kind OLAP is so good at.

Mobile BI – Step back and see the big picture anytime, anywhere

Spreadsheets are great for displaying detailed information. However, in today’s highly competitive market, retrieving information that matters the most in the shortest possible time is vital in maintaining a sizeable lead over the competition. To step back and see the big picture, you’ll need insightful tools like dashboards and automatically generated reports.

Reports can be beamed to mobile devices such as smart phones and Pocket PCs. They can also be viewed on eBook readers as well. You can also do the same tasks with spreadsheets. But imagine how you’d need to scroll over a large spreadsheet on any of these mobile devices just to know which customer in your current location has performed well over the last month.

If you really want to make quick, well-informed decisions, BI dashboards for mobile devices is the way to go. You can make use of various business objects such as drill-able charts, performance metrics, and metric trend graphs to make crucial decisions even when on you’re in the field.

Spreadsheet Fraud

To any company executive or business owner, the mere possibility of fraud can be enough to send alarm bells ringing – for good reason. In a prolonged recession, the last thing investors would want to discover is a huge, gaping hole where supposedly a neat profit should have been. Also to find out that such loss was brought about by deliberately falsified accounting and poor spreadsheet controls only makes the situation even more regrettable.

Why?

Because these losses would not have occurred had there been a stronger risk management program in place and more stringent quality control on critical data to begin with.

But given the nature of a spreadsheet system i.e. its sheer flexibility and easy accessibility, plus the fact that they were never intended to be enterprise-level tools, there are no hard and fast rules for auditing spreadsheets. Also because of the lack of internal controls for end user computing (EUC) applications, in this case spreadsheets, you can’t expect these systems to yield consistently accurate results.

In fact, most managers assume that major spreadsheet errors should result in figures that are blatantly out of touch with how things stand in the real world, making these errors easily detectable.

Well they assumed wrong. You’ll find cases where the losses ran to millions of dollars without anyone being the wiser.

In instances of fraud, the problem becomes more complicated as these errors are deliberately hidden and cleverly disguised, perhaps one erroneous cell at a time. Even if these cover-ups started out with smaller figures that may have had negligible impact on a company?s operation, the cumulative costs of these ?insignificant? errors multiply exponentially as the spreadsheets are reused and utilised as bases for other related reports.

While there is no generally accepted definition of the term ?spreadsheet fraud?, its quite easy to identify one when a case crops up. Fraud arising from spreadsheets are typically characterised by:

Fallacious inputs – correct figures are deliberately replaced with false values.

Erroneous outputs owing to data alteration – hyperlinks are linking to the wrong spreadsheets or cells; use of macros or special lines of code which are understandable only to the person who developed the code.

Concealment of critical information – can be done with easy ?tweaks? such as hidden rows and columns, using the same colour for both the font and the background, or hard coding additional values into a cell.

There is nothing really highly-sophisticated or technical in any of these methodologies. But without internal spreadsheet controls in place, it would take a discerning eye and a thorough review to catch the inconsistencies contained in a spreadsheet fraught with errors. Also, if these errors are knowingly placed there, the chances of finding them are close to nil.

Learn more about our server application solutions and discover a better way to protect your company from spreadsheet fraud.

More Spreadsheet Blogs


Spreadsheet Risks in Banks


Top 10 Disadvantages of Spreadsheets


Disadvantages of Spreadsheets – obstacles to compliance in the Healthcare Industry


How Internal Auditors can win the War against Spreadsheet Fraud


Spreadsheet Reporting – No Room in your company in an age of Business Intelligence


Still looking for a Way to Consolidate Excel Spreadsheets?


Disadvantages of Spreadsheets


Spreadsheet woes – ill equipped for an Agile Business Environment


Spreadsheet Fraud


Spreadsheet Woes – Limited features for easy adoption of a control framework


Spreadsheet woes – Burden in SOX Compliance and other Regulations


Spreadsheet Risk Issues


Server Application Solutions – Don’t let Spreadsheets hold your Business back


Why Spreadsheets can send the pillars of Solvency II crashing down

?

Advert-Book-UK

amazon.co.uk

?

Advert-Book-USA

amazon.com

Ready to work with Denizon?