Directions Hadoop is Moving In

Hadoop is a data system so big it is like a virtual jumbo where your PC is a flea. One of the developers named it after his kid?s toy elephant so there is no complicated acronym to stumble over. The system is actually conceptually simple. It has loads of storage capacity and an unusual way of processing data. It does not wait for big files to report in to its software. Instead, it takes the processing system to the data.

The next question is what to do with Hadoop. Perhaps the question would be better expressed as, what can we do with a wonderful opportunity that we could not do before. Certainly, Hadoop is not for storing videos when your laptop starts complaining. The interfaces are clumsy and Hadoop belongs in the realm of large organisations that have the money. Here are two examples to illustrate the point.

Hadoop in Healthcare

In the U.S., healthcare generates more than 150 gigabytes of data annually. Within this data there are important clues that online training provider DeZyre believes could lead to these solutions:

  • Personalised cancer treatments that relate to how individual genomes cause the disease to mutate uniquely
  • Intelligent online analysis of life signs (blood pressure, heart beat, breathing) in remote children?s hospitals treating multiple victims of catastrophes
  • Mining of patient information from health records, financial status and payroll data to understand how these variables impact on patient health
  • Understanding trends in healthcare claims to empower hospitals and health insurers to increase their competitive advantages.
  • New ways to prevent health insurance fraud by correlating it with claims histories, attorney costs and call centre notes.

Hadoop in Retail

The retail industry also generates a vast amount of data, due to consumer volumes and multiple touch points in the delivery funnel. Skillspeed business trainers report the following emerging trends:

  • Tracing individual consumers along the marketing trail to determine individual patterns for different demographics and understand consumers better.
  • Obtaining access to aggregated consumer feedback regarding advertising campaigns, product launches, competitor tactics and so on.
  • Staying with individual consumers as they move through retail outlets and personalising their experience by delivering contextual messages.
  • Understanding the routes that virtual shoppers follow, and adding handy popups with useful hints and tips to encourage them on.
  • Detecting trends in consumer preferences in order to forecast next season sales and stock up or down accordingly.

Where to From Here?

Big data mining is akin to deep space research in that we are exploring fresh frontiers and discovering new worlds of information. The future is as broad as our imagination.?

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Check our similar posts

The Matrix Management Structure

Organizations exploit matrix management in various ways. A company, for instance, that operates globally uses it at larger scale by giving consistent products to various countries internationally. A business entity, having many products, does not assign its people to each product full-time but assign those to different ones on a part time basis, instead. And when it comes to delivering high quality and low cost products, companies overcome industry pressures with the help of many overseeing managers. In a rapidly changing environment, organizations respond quickly by sharing information through a matrix model.

Understanding the Matrix Management Structure

A basic understanding of matrix management starts with the three key roles and responsibilities that applies in the structure.

  • Matrix Leader ? The common person above all the matrix bosses is the matrix leader. He ensures that the balance of power is maintained in the entire organization by delegating decisions and promoting collaboration among the people.
  • Matrix Managers ? The managers cooperate with each other by defining the respective activities that they are responsible for.
  • Matrix Employees – The employees have lesser direct authority but has more responsibilities. They resolve differing demands from more than one matrix managers while they work things out upwards. Their loyalty must be dual and their relationships with managers must be maintained.

Characteristics of a Matrix Structure

Here are some features that define the matrix management structure:

  • Hybrid Structure ?The matrix structure is a mix of functional and project organization. Since it is a combination of these two, matrix management is hybrid in nature.
  • Functional Manager ? When it comes to the technical phases of the project, the functional manager assumes responsibility. The manager decides on how to get the project done, delegates the tasks to the subordinates and oversees the operational parts of the organization.
  • Project Manager ? The project manager has full authority in the administrative phases, including the physical and financial resources needed to complete the project. The responsibilities of a project manager comprise deciding on what to do, scheduling the work, coordinating the activities to diverse functions and evaluating over-all project performance.
  • Specialization ?As the functional managers concentrate on the technical factors, the project managers focus on administrative ones. Thus, in matrix management, there is specialization.
  • Challenge in Unity of Command ? Companies that employs matrix management usually experience a problem when it comes to the unity of command. This is largely due to the conflicting orders from the functional and project managers.

Types of Matrix Structure

The matrix management structure can be classified according to the level of power of the project manager. Here are three distinct types of matrix structures that are widely used by organizations.

  • Weak Matrix ? The project manager has limited authority and power as the functional manager controls the budget of the project. His role is only part-time and more like a coordinator.
  • Strong Matrix ? Here, the project manager has almost all the authority and power. He controls the budget, holds the full time administrative project management and has a full time role.
  • Balanced Matrix ? In this structure type, both the project and functional managers control the budget of the project. The authority and power is shared by the two as well. Although the project manager has a full time role, he only has a part time authority for the administrative staff to report under his leadership.

Successful companies of today venture more on enhancing the abilities, skills, behavior and performances of their managers than the pursuit of finding the best physical structure. Indeed, learning the fundamentals of the matrix structure is essential to maximize its efficiency. A senior executive pointed out that one of the challenges in matrix management is not more of building a structure but in creating the matrix to the mind of the managers. This comes to say that matrix management is not just about the structure, it is a frame in the mind.

UK Hauliers Pull Together on ESOS

ESOS is what UK business needed, to encourage it to become more responsible for the environmental consequences of making money. Government has met with industry leaders to hammer out the finer details. Now there are heartening signs of intra-industry collaboration, for the example the FTA approach we discuss here.

The Freight Transport Association (FTA) is one of the UK?s biggest trade associations, and exists to represent the interests of companies moving goods by air, rail, sea and road. It is their representative at national, European and local level that advises them on legal compliance. In February 2015, it announced plans to help the industry comply with ESOS too.

The association has been active since the announcement of the UK?s Energy Saving Opportunity Scheme. It has engaged with government and membership through the portal of its Logistics Carbon Reduction Scheme (LCRS). The Environment Agency has singled this out as a benchmark other industries could follow.

FTA general manager for consultancy and tendering Karen Packham recently said, ?With our highly experienced and fully qualified team of transport auditors ?the FTA is best placed to offer practical advice and is able to provide specialist audits to ensure members are fully compliant ? and will gain all the benefits that the scheme has to offer.?

These co-audits with Environment Agency specialists advising, will focus on the full range of operational and supporting activities, and ensure that all haulage companies with over 250 employees do the following:

  • Assess energy use across their full spread of buildings, transport media and industrial activity
  • Examine energy-intensive pressure points and identify savings opportunities that provide financial benefit
  • Nominate an ESOS person to conduct future audits, or oversee and approve them independently
  • Report to the Environment Agency as scheme administrator per statutory intervals

Ecovaro has energy management software that turns metrics into high-level information that busy people understand. Give us a call if you are puzzling how best to present your data. We believe two heads can achieve so much more together.

How DevOps oils the Value Chain

DevOps ? a clipped compound of development and operations – is a way of working whereby software developers are in a team with project beneficiaries. A client centred approach extends the project plan to include the life cycle of the product or service, for which the software is developed.

We can then no longer speak of a software project for say Joe?s Accounting App. The software has no intrinsic value of its own. It follows that the software engineers are building an accounting app product. This is a small, crucially important distinction, because they are no longer in a silo with different business interests.

To take the analogy further, the developers are no longer contractors possibly trying to stretch out the process. They are members of Joe?s accounting company, and they are just as keen to get to market fast as Joe is to start earning income. DevOps uses this synergy to achieve the overarching business goal.

A Brief Introduction to OpsDev

You can skip this section if you already read this article. If not then you need to know that DevOps is a culture, not a working method. The three ?members? are the software developers, the beneficiaries, and a quality control mechanism. The developers break their task into smaller chunks instead of releasing the code to quality control as a single batch. As a result, the review process happens contiguously along these simplified lines.

Code QC Test ? ? ?
? Code QC Test ? ?
? ? Code QC Test ?
? ? ? Code QC Test
Colour Key Developers Quality Control Beneficiary

This is a marked improvement over the previously cumbersome method below.

Write the Code ? Test the Code ? Use the Code
? Evaluate, Schedule for Next Review ?

Working quickly and releasing smaller amounts of code means the OpsDev team learns quickly from mistakes, and should come to product release ahead of any competitor using the older, more linear method. The shared method of working releases huge resources in terms of user experience and in-line QC practices. Instead of being in a silo working on its own, development finds it has a richer brief and more support from being ?on the same side of the organisation?.

The Key Role that Application Program Interfaces Play

Application Program Interfaces, or API?s for short, are building blocks for software applications. Using proprietary software-bridges speeds this process up. A good example would be the PayPal applications that we find on so many websites today. API?s are not just for commercial sites, and they can reduce costs and improve efficiency considerably.

The following diagram courtesy of TIBCO illustrates how second-party applications integrate with PayPal architecture via an API fa?ade.

Working quickly and releasing smaller amounts of code means the OpsDev team learns quickly from mistakes, and should come to product release ahead of any competitor using the older, more linear method. The shared method of working releases huge resources in terms of user experience and in-line QC practices. Instead of being in a silo working on its own, development finds it has a richer brief and more support from being ?on the same side of the organisation?.

imgd2.jpg

The DevOps Revolution Continues ?

We close with some important insights from an interview with Jim Stoneham. He was general manager of the Yahoo Communities business unit, at the time Flickr became a part. ?Flickr was a codebase,? Jim recalls, ?that evolved to operate at high scale over 7 years – and continuing to scale while adding and refining features was no small challenge. During this transition, it was a huge advantage that there was such an integrated dev and ops team?

The ?maturity model? as engineers refer to DevOps status currently, enables developers to learn faster, and deploy upgrades ahead of their competitors. This means the client reaches and exceeds break-even sooner. DevOps lubricates the value chain so companies add value to a product faster. One reason it worked so well with Flickr, was the immense trust between Dev and Ops, and that is a lesson we should learn.

?We transformed from a team of employees to a team of owners. When you move at that speed, and are looking at the numbers and the results daily, your investment level radically changes. This just can’t happen in teams that release quarterly, and it’s difficult even with monthly cycles.? (Jim Stoneham)

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Ready to work with Denizon?

Contact Us Today