Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

  • Free to download, use and contribute to
  • Everything you need ?in the box? to get started
  • Distributed across multiple fire-walled computers
  • Fast processing of data held in efficient cluster nodes
  • Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

Computer Forensics

So you had a customer data security breach last weekend? Do you know you could be held liable in court for failing to implement required security procedures? That’s right. Due to the overwhelming surge in identity theft wherein nearly 20 million Americans have already been affected, most states have enacted laws to curtail this fast rising crime. Therefore, it is important to redefine how your company deals with customer data security.

  • First, you’ll want to know what your obligations are as dictated by law. Some places, for example, require the destruction or deletion of personal data through shredding, erasing, or by rendering them undecipherable.
  • Second, not only do you need to comply with the said requirements, you’ll also have to prove in court that you actually complied if ever a security breach does happen.
  • Third, you need to be aware of your post-breach duties to avoid being dealt additional penalties.

Obviously, such situations now call for individuals who are experts in both the legal and technical aspects regarding data security. Such individuals are practitioners of a relatively new discipline known as computer forensics.

Armed with our computer forensics specialists, we’ll be able to help you deal with the above concerns. As a result, you can be prevented from having to pay fines that can go up to hundreds of thousands of euros.

There are other equally important reasons why you would want to avail of computer forensics services. For example, you’ll need computer forensics specialists because you want to:

  • Catch a person involved in criminal activities such as child porn, stealing of personal data, and destroying intellectual property.
  • Investigate a computer, network, or even a mobile device for clues that may lead to the culprit.
  • Determine the extent and possible causes when you discover your digital data has been damaged.
  • Find and recover damaged, deleted or encrypted data regardless of whether the cause was intentional or not. If the data in question will be used as evidence in a legal action, there are certain procedures that need to be followed during recovery operations to retain the integrity of the data. Computer forensic specialists are highly qualified for such operations.
  • Implement security policies in your organisation. Such policies have to operate within legal bounds if you want to avoid possible sanctions in the future. These policies should also be designed such that future forensic operations can be conducted with a high likelihood of success.

That said, a company that integrates computer forensics into its IT security policies and practices will be better equipped to remedy the situation once data security has already been compromised than a company that doesn’t.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
How DevOps oils the Value Chain

DevOps ? a clipped compound of development and operations – is a way of working whereby software developers are in a team with project beneficiaries. A client centred approach extends the project plan to include the life cycle of the product or service, for which the software is developed.

We can then no longer speak of a software project for say Joe?s Accounting App. The software has no intrinsic value of its own. It follows that the software engineers are building an accounting app product. This is a small, crucially important distinction, because they are no longer in a silo with different business interests.

To take the analogy further, the developers are no longer contractors possibly trying to stretch out the process. They are members of Joe?s accounting company, and they are just as keen to get to market fast as Joe is to start earning income. DevOps uses this synergy to achieve the overarching business goal.

A Brief Introduction to OpsDev

You can skip this section if you already read this article. If not then you need to know that DevOps is a culture, not a working method. The three ?members? are the software developers, the beneficiaries, and a quality control mechanism. The developers break their task into smaller chunks instead of releasing the code to quality control as a single batch. As a result, the review process happens contiguously along these simplified lines.

Code QC Test ? ? ?
? Code QC Test ? ?
? ? Code QC Test ?
? ? ? Code QC Test
Colour Key Developers Quality Control Beneficiary

This is a marked improvement over the previously cumbersome method below.

Write the Code ? Test the Code ? Use the Code
? Evaluate, Schedule for Next Review ?

Working quickly and releasing smaller amounts of code means the OpsDev team learns quickly from mistakes, and should come to product release ahead of any competitor using the older, more linear method. The shared method of working releases huge resources in terms of user experience and in-line QC practices. Instead of being in a silo working on its own, development finds it has a richer brief and more support from being ?on the same side of the organisation?.

The Key Role that Application Program Interfaces Play

Application Program Interfaces, or API?s for short, are building blocks for software applications. Using proprietary software-bridges speeds this process up. A good example would be the PayPal applications that we find on so many websites today. API?s are not just for commercial sites, and they can reduce costs and improve efficiency considerably.

The following diagram courtesy of TIBCO illustrates how second-party applications integrate with PayPal architecture via an API fa?ade.

Working quickly and releasing smaller amounts of code means the OpsDev team learns quickly from mistakes, and should come to product release ahead of any competitor using the older, more linear method. The shared method of working releases huge resources in terms of user experience and in-line QC practices. Instead of being in a silo working on its own, development finds it has a richer brief and more support from being ?on the same side of the organisation?.

imgd2.jpg

The DevOps Revolution Continues ?

We close with some important insights from an interview with Jim Stoneham. He was general manager of the Yahoo Communities business unit, at the time Flickr became a part. ?Flickr was a codebase,? Jim recalls, ?that evolved to operate at high scale over 7 years – and continuing to scale while adding and refining features was no small challenge. During this transition, it was a huge advantage that there was such an integrated dev and ops team?

The ?maturity model? as engineers refer to DevOps status currently, enables developers to learn faster, and deploy upgrades ahead of their competitors. This means the client reaches and exceeds break-even sooner. DevOps lubricates the value chain so companies add value to a product faster. One reason it worked so well with Flickr, was the immense trust between Dev and Ops, and that is a lesson we should learn.

?We transformed from a team of employees to a team of owners. When you move at that speed, and are looking at the numbers and the results daily, your investment level radically changes. This just can’t happen in teams that release quarterly, and it’s difficult even with monthly cycles.? (Jim Stoneham)

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK
Shared Services ? Are They A Good Idea

Things happen fast in business and we need to stay on top. It does not seem long ago that some enterprises were still hands-on traders or artisans with a few youngsters to help out. People like that did not do admin and their accounting was a matter of making sure there was enough money in the jar.

When Wal-Mart’s Sam Walton took over his first shop in 1945 things had moved on from there, although he did still deal directly with his customers. When he died his legacy was 380,000 jobs, and a business larger than most economies. So there?s plenty we can learn from how he grew his business.

One of Sam?s secrets was his capacity to centralise what needed gathering together, while empowering store managers to think independently when it came to local conditions. His regional warehouses had individual outlets clustered around them within one day?s drive each. This shared service eliminated 90% of safety stock and released capital for expansion.

Wal-Mart took sharing services a step further in February 2006, when it centralised accounts payable, accounts receivable, general accounting and human resources administration at Wal-Mart Stores and Sam?s Clubs in the U.S. and Puerto Rico. The objective was to bring costs down, while allowing local managers more time to focus on their business plans and other initiatives. As a further spin-off, Wal-Mart was able to integrate its data on a single SAP platform and eliminate significant roadblocks.

This is an excellent example of sharing services by creating own centres of excellence.? Of course, this is not the only business possibility. Other corporates have successfully completely outsourced their support activities, and Wal-Mart has no doubt had a variety of similar offers too. But, is the Wal-Mart picture entirely rosy, or is there a catch?

The Association of Chartered Certified Accountants has indicated that top talent may be the loser globally. This is because the Wal-Mart model removes many challenges through standardisation, and offers less scope for internal promotion as a result. Language and cultural differences may also have a long-term detrimental effect on the way the departments work well together.

Local outsourcing ? this is the business model where several firms engage a shared service provider independently- may hence prove to be a more malleable option for smaller companies. It often makes more sense to hunt down made-to-order services. Offerings such as the professional support we offer on this site.

Ready to work with Denizon?

Contact Us Today