Directions Hadoop is Moving In

Hadoop is a data system so big it is like a virtual jumbo where your PC is a flea. One of the developers named it after his kid?s toy elephant so there is no complicated acronym to stumble over. The system is actually conceptually simple. It has loads of storage capacity and an unusual way of processing data. It does not wait for big files to report in to its software. Instead, it takes the processing system to the data.

The next question is what to do with Hadoop. Perhaps the question would be better expressed as, what can we do with a wonderful opportunity that we could not do before. Certainly, Hadoop is not for storing videos when your laptop starts complaining. The interfaces are clumsy and Hadoop belongs in the realm of large organisations that have the money. Here are two examples to illustrate the point.

Hadoop in Healthcare

In the U.S., healthcare generates more than 150 gigabytes of data annually. Within this data there are important clues that online training provider DeZyre believes could lead to these solutions:

  • Personalised cancer treatments that relate to how individual genomes cause the disease to mutate uniquely
  • Intelligent online analysis of life signs (blood pressure, heart beat, breathing) in remote children?s hospitals treating multiple victims of catastrophes
  • Mining of patient information from health records, financial status and payroll data to understand how these variables impact on patient health
  • Understanding trends in healthcare claims to empower hospitals and health insurers to increase their competitive advantages.
  • New ways to prevent health insurance fraud by correlating it with claims histories, attorney costs and call centre notes.

Hadoop in Retail

The retail industry also generates a vast amount of data, due to consumer volumes and multiple touch points in the delivery funnel. Skillspeed business trainers report the following emerging trends:

  • Tracing individual consumers along the marketing trail to determine individual patterns for different demographics and understand consumers better.
  • Obtaining access to aggregated consumer feedback regarding advertising campaigns, product launches, competitor tactics and so on.
  • Staying with individual consumers as they move through retail outlets and personalising their experience by delivering contextual messages.
  • Understanding the routes that virtual shoppers follow, and adding handy popups with useful hints and tips to encourage them on.
  • Detecting trends in consumer preferences in order to forecast next season sales and stock up or down accordingly.

Where to From Here?

Big data mining is akin to deep space research in that we are exploring fresh frontiers and discovering new worlds of information. The future is as broad as our imagination.?

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Check our similar posts

Reducing Your Carbon Footprint

Climate change creates a loud buzz across the globe. People are talking about how extreme the weather is, how polluted the environment has become or how devastating the results of carbon emissions are. While it is true that humans contribute a large impact to the worsening climate situations, people are also the most influential key towards making this world a better place. As much as the increase in carbon emissions results from what you do, the healthy change can also start in you.

Although it is a bit difficult to determine what you can do to help the society, do not be disheartened. The devastating forces may be massive for you to work through, but there are countless simple actions?you can take to reduce your carbon footprints day by day.

Home

While you are in the comfort of your home, you can start saving energy to reduce your carbon emission. You could’replace your standard light bulbs with compact fluorescent ones. A compact fluorescent bulb saves more than 2/3rds or up to 1,300 pounds of carbon dioxide in its lifetime. This bulb contains mercury, so make sure to choose a brand that has lower mercury than others.

Another thing, you can do to reduce your carbon footprint at home, is to mind your electronics. When you do not use your gadgets and appliances, make sure you unplug them. If you buy new ones, take time to look at the energy rating of the electronics to save you more energy in future use.

Alternative renewable energy is also a good thing to shift into. Try solar, hydro or wind power at home. Setting up your own residential solar panels and building your own turbines are excellent ways to choose green energy.

Food

The food industry is one of the largest contributors of carbon emissions. You may not have control over the food processing, but you can lower your carbon footprint by buying local products in the market. These local products are not transported from far off places, so the carbon dioxide released from them is lower compared to imported ones. Take a look at the packaging as well; less packaging means less waste.

If you have a big backyard, you could use your it to grow food. ?Eating food, either fruit or vegetable, which you grow at home is energy efficient. No more fuel combustion from transportation and other consequent food processing.

Travel

When you have your own car, accelerating it slowly and smoothly, as well as maintaining speed while driving will help lower your carbon emissions. If you drive a lot, it would be better to get a green car. As of now, you can consider using?public transportation and go for road travel rather than air travel when you take long distance trips. But when you need to take planes, better choose a non-stop flight instead of connecting ones.

Indeed, there are many ways you can combat global warming and climate change. The road to improved life quality through energy efficiency might be hard, but a transformed lifestyle can make a big difference. Start now ? lighten your carbon footprint and help save the world.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
How DevOps oils the Value Chain

DevOps ? a clipped compound of development and operations – is a way of working whereby software developers are in a team with project beneficiaries. A client centred approach extends the project plan to include the life cycle of the product or service, for which the software is developed.

We can then no longer speak of a software project for say Joe?s Accounting App. The software has no intrinsic value of its own. It follows that the software engineers are building an accounting app product. This is a small, crucially important distinction, because they are no longer in a silo with different business interests.

To take the analogy further, the developers are no longer contractors possibly trying to stretch out the process. They are members of Joe?s accounting company, and they are just as keen to get to market fast as Joe is to start earning income. DevOps uses this synergy to achieve the overarching business goal.

A Brief Introduction to OpsDev

You can skip this section if you already read this article. If not then you need to know that DevOps is a culture, not a working method. The three ?members? are the software developers, the beneficiaries, and a quality control mechanism. The developers break their task into smaller chunks instead of releasing the code to quality control as a single batch. As a result, the review process happens contiguously along these simplified lines.

Code QC Test ? ? ?
? Code QC Test ? ?
? ? Code QC Test ?
? ? ? Code QC Test
Colour Key Developers Quality Control Beneficiary

This is a marked improvement over the previously cumbersome method below.

Write the Code ? Test the Code ? Use the Code
? Evaluate, Schedule for Next Review ?

Working quickly and releasing smaller amounts of code means the OpsDev team learns quickly from mistakes, and should come to product release ahead of any competitor using the older, more linear method. The shared method of working releases huge resources in terms of user experience and in-line QC practices. Instead of being in a silo working on its own, development finds it has a richer brief and more support from being ?on the same side of the organisation?.

The Key Role that Application Program Interfaces Play

Application Program Interfaces, or API?s for short, are building blocks for software applications. Using proprietary software-bridges speeds this process up. A good example would be the PayPal applications that we find on so many websites today. API?s are not just for commercial sites, and they can reduce costs and improve efficiency considerably.

The following diagram courtesy of TIBCO illustrates how second-party applications integrate with PayPal architecture via an API fa?ade.

Working quickly and releasing smaller amounts of code means the OpsDev team learns quickly from mistakes, and should come to product release ahead of any competitor using the older, more linear method. The shared method of working releases huge resources in terms of user experience and in-line QC practices. Instead of being in a silo working on its own, development finds it has a richer brief and more support from being ?on the same side of the organisation?.

imgd2.jpg

The DevOps Revolution Continues ?

We close with some important insights from an interview with Jim Stoneham. He was general manager of the Yahoo Communities business unit, at the time Flickr became a part. ?Flickr was a codebase,? Jim recalls, ?that evolved to operate at high scale over 7 years – and continuing to scale while adding and refining features was no small challenge. During this transition, it was a huge advantage that there was such an integrated dev and ops team?

The ?maturity model? as engineers refer to DevOps status currently, enables developers to learn faster, and deploy upgrades ahead of their competitors. This means the client reaches and exceeds break-even sooner. DevOps lubricates the value chain so companies add value to a product faster. One reason it worked so well with Flickr, was the immense trust between Dev and Ops, and that is a lesson we should learn.

?We transformed from a team of employees to a team of owners. When you move at that speed, and are looking at the numbers and the results daily, your investment level radically changes. This just can’t happen in teams that release quarterly, and it’s difficult even with monthly cycles.? (Jim Stoneham)

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
Knowing the Caveats in Cloud Computing

Cloud computing has become such a buzzword in business circles today that many organisations both small and large, are quick to jump on the cloud bandwagon – sometimes a little too hastily.

Yes, the benefits of the cloud are numerous: reduced infrastructure costs, improved performance, faster time-to-market, capability to develop more applications, lower IT staff expenses; you get the picture. But contrary to what many may be expecting or have been led to believe, cloud computing is not without its share of drawbacks, especially for smaller organisations who have limited knowledge to go on with.

So before businesses move to the cloud, it pays to learn a little more about the caveats that could meet them along the way. Here are some tips to getting started with cloud computing as a small business consumer.

Know your cloud. As with anything else, knowledge is always key. Because it is a relatively new tool in IT, it’s not surprising that there is some confusion about the term cloud computing among many business owners and even CIOs. According to the document The NIST Definition of Cloud Computing, cloud computing has five essential characteristics, three basic service models (Saas, Paas and Iaas), and four deployment models (public, community, private and hybrid).

The first thing organisations should do is make a review of their operations and evaluate if they really need a cloud service. If they would indeed benefit from cloud computing, the next steps would be deciding on the service model that would best fit the organisation and choosing the right cloud service provider. These factors are particularly important when you consider data security and compliance issues.

Read the fine print. Before entering into a contract with a cloud provider, businesses should first ensure that the responsibilities for both parties are well-defined, and if the cloud vendor has the vital mechanisms in place for contingency measures. For instance, how does the provider intend to carry out backup and data retrieval operations? Is there assurance that the business’ critical data and systems will be accessible at all times? And if not, how soon can the data be available in case of a temporary shutdown of the cloud?

Also, what if either the company or the cloud provider stops operations or goes bankrupt? It should be clear from the get go that the data remains the sole property of the consumer or company subscribing to the cloud.

As you can see, there are various concerns that need to be addressed closely before any agreement is finalised. While these details are usually found in the Service Level Agreements (SLAs) of most outsourcing and servicing contracts, unfortunately, the same cannot be said of cloud contracts.

Be aware of possible unforeseen costs. The ability of smaller companies to avail of computing resources on a scalable, pay-as-you-go model is one of the biggest selling points of cloud computing. But there’s also an inherent risk here: the possibility of runaway costs. Rather than allowing significant cost savings, small businesses could end up with a bill that’s bound to blow a big hole in their budget.

Take for example the case of a software company cited on InformationWeek.com to illustrate this point. The 250-server cluster the company rented from a cloud provider was inadvertently left turned on by the testing team over the weekend. As a result, their usual $2,300 bill ballooned to a whopping $23,400 over the course of one weekend.

Of course, in all likelihood, this isn’t going to happen to every small and midsize enterprise that shifts to the cloud. However, this should alert business owners, finance executives, and CEOs to look beyond the perceived savings and identify potential sources of unexpected costs. What may start as a fixed rate scheme for on-demand computing resources, may end up becoming a complex pricing puzzle as the needs of the business grow, or simply because of human error as the example above shows.

The caveats we’ve listed here are among the most crucial ones that soon-to-be cloud adopters need to keep in mind. But should these be reasons enough for businesses to stop pursuing a cloud strategy? Most definitely not. Armed with the right information, cloud computing is still the fastest and most effective way for many small enterprises to get the business off the ground with the lowest start-up costs.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Ready to work with Denizon?

Contact Us Today