Are Master Data Management and Hadoop a Good Match?

Master Data is the critical electronic information about the company we cannot afford to lose. Accordingly, we should sanitise it, look after it, and store it safely in several separate places that are independent of each other. The advent of Big Data introduced the current era of huge repositories ?in the clouds?. They are not, of course but at least they are remote. This short article includes a discussion about Hadoop, and whether this is a good platform to back up your Master Data.

About Hadoop

Hadoop is an open-source Apache software framework built on the assumption that hardware failure is so common that backups are unavoidable. It comprises a storage area and a management part that distributes the data to smaller nodes where it processes faster and more efficiently. Prominent users include Yahoo! and Facebook. In fact more than half Fortune 50 companies were using Hadoop in 2013.

Hadoop – initially launched in December 2011 ? has survived its baptism of fire and became a respected, reliable option. But is this something the average business owner can tackle on their own? Bear in mind that open source software generally comes with little implementation support from the vendor.

The Hadoop Strong Suite

  • Free to download, use and contribute to
  • Everything you need ?in the box? to get started
  • Distributed across multiple fire-walled computers
  • Fast processing of data held in efficient cluster nodes
  • Massive scaleable storage you are unlikely to run out of

Practical Constraints

There is more to Hadoop than writing to WordPress. The most straightforward solutions are uploading using Java commands, obtaining an interface mechanism, or using third party vendor connectors such as ACCESS or SAS. The system does not replace the need for IT support, although it is cheap and exceptionally powerful.

The Not-Free Safer Option

Smaller companies without in-depth in-house support are wise to engage with a technical intermediary. There are companies providing commercial implementations followed by support. Microsoft, Amazon and Google among others all have commercial versions in their catalogues, and support teams at the end of the line.

Check our similar posts

How COBIT helps you achieve SOX Compliance

First released way back in 1996, COBIT has already been around for quite a while. One reason why it never took off was because companies were never compelled to use it ? until now. Today, many CEOs and CIOs are finding it to be a vital tool for achieving SOX compliance in IT.

Thanks to SOX, COBIT (Control Objectives for Information and related Technology) is now one of the most widely accepted source of guidance among companies who have IT integrated with their accounting/financial systems. It has also gained general acceptability with third parties and regulators. But how did this happen?

Role of control frameworks in SOX compliance

You see, the Sarbanes-Oxley Act, despite having clearly manifested the urgency of establishing effective internal controls, does not provide a road map for you to follow nor does it specify a yardstick to help you determine whether an acceptable mileage in the right direction has already been achieved.

In other words, if you were a CIO and you wanted to find guidance on what steps you had to take to achieve compliance, you wouldn’t be able to find the answers in the legislation itself.

That can be a big problem. Two of your main SOX compliance obligations as a CEO or CIO is to assume responsibility in establishing internal controls over financial reporting and to certify their effectiveness. After that, the external auditors are supposed to attest to your assertions. Obviously, there has to be a well-defined basis before you can make such assertions and auditors can attest to anything.

In the language of auditors, this ?well-defined basis? is known as a control framework. Simply put, once you certify the presence of adequate internal controls in your organisation, the external auditor will ask, ?What control framework did you use??

Knowing what control framework you employed will help external auditors determine how to proceed with their evaluations and tests. For your part, a control framework can serve as a guide to help you work towards specific objectives for achieving compliance. Both of you can use it as a common reference point before drawing any conclusions regarding your controls.

But there are many control frameworks out there. What should you use?

How SOX, COSO, and COBIT fit together

Fortunately, despite SOX?s silence regarding control frameworks, you aren’t left entirely to your own devices. You could actually take a hint from the SEC and PCAOB, two of the lead organisations responsible for implementing SOX. SEC and PCAOB point to the adoption of any widely accepted control framework.

In this regard, they both highly endorse COSO, a well-established internal control framework formulated by the Committee of Sponsoring Organisations of the Treadway Commission (COSO). Now, I must tell you, if you’re looking specifically for instructions pertaining to IT controls, you won’t find those in COSO either.

Although COSO is the most established control framework for enterprise governance and risk management you’ll ever find (and in fact, it’s what we recommend for your general accounting processes), it lacks many IT-related details. What is therefore needed for your IT processes is a framework that, in addition to being highly aligned with COSO, also provides more detailed considerations for IT.

This is where COBIT fits the bill.

How COBIT can contribute to your regulatory compliance endeavors

COBIT builds upon and adheres with COSO while providing a finer grain of detail focused on IT. You can even find a mapping between COBIT IT processes and COSO components within the COBIT document itself.

Designed with regulatory compliance in mind, COBIT lays down a clear path for developing policies and good practice for IT control, thus enabling you to bridge the gap between control requirements, technical issues, and business risks.

Some of the components you’ll find in COBIT include:

IT control objectives

These are statements defining specific desired results that, as a whole, characterise a well-managed IT process. They come in two forms for each COBIT-defined IT process: a high-level control objective and a number of detailed control objectives. These objectives will enable you to have a sense of direction by telling you exactly what you need to aim for.

Maturity models

These are used as benchmarks that give you a relative measurement stating where your level of management or control over an IT process or high-level control objective stands. It serves as a basis for setting as-is and to-be positions and enables support for gap analysis, which determines what needs to be done to achieve a chosen level. Basically, if a control objective points you to a direction, then its corresponding maturity model tells you how far in that direction you’ve gone.

RACI charts

These charts tell you who (e.g. CEO, CFO, Head of Operations, Head of IT Administration) should be Responsible, Accountable, Consulted, and Informed for each activity.

Goals and Metrics

These are sets of goals along with the corresponding metrics that allow you to measure against those goals. Goals and metrics are defined in three levels: IT goals and metrics, which define what business expects from IT; process goals and metrics, which define what the IT process should deliver to support It’s objectives; and activity goals and metrics, which measure how well the process is performing.

In addition to those, you’ll also find mappings of each process to the information criteria involved, IT resources that need to be leveraged, and the governance focus areas that are affected.

Everything is presented in a logical and manageable structure, so that you can easily draw connections between IT processes and business goals, which will in turn help you decide what appropriate governance and control is needed. Ultimately, COBIT can equip you with the right tools to maintain a cost-benefit balance as you work towards achieving SOX compliance.

Eck Industries Sheds Fresh Light

William Eck began his business in 1948 in a 650m2 garage building. The aluminium foundry prospered, and now has an 18,500m2 factory in Manitowoc, Wisconsin employing 250 people casting a variety of casings. Like high-tech industries around the globe it needs effective illumination. After it measured its carbon footprint, it realised it needed energy efficient lighting too.

When Eck Industries began its review it had around 360 high-pressure sodium lights throughout the plant. Their operating cost was substantial. After taking independent advice from an independent agency they realised they needed to replace these with more energy-efficient fluorescent lights that consume half as much energy.

The feasibility team conducted performance tests to determine the optimum solution. After selecting enclosed, gasketed and waterproof T8 fluorescents (available in G13 bipin, single pin and recessed double contacts) they collaborated with the supplier to calculate the best combination of 4 and 6 bulb fixtures.

The fittings they chose cost $60,000 plus $10,000 installation. However a $33,000 energy rebate wrote down 47% of this immediately. They achieved further energy savings by attaching motion sensors to lights over low-traffic walkways.

The retrofit was a huge success, with an 8 month payback via a direct operating saving of $55,000 a year. Over and above enhanced illumination Eck Industries slashed 674,000 kilowatt hours off its annual lighting bill. During the 20 year design life, this equates to a total 13.5 million kilowatt hours. Other quantifiable benefits include 443 tons less carbon, 2 tons less sulphur dioxide, and 1 ton less nitrogen oxide per year.

Many companies face similar opportunities but fail to capitalise on them for a number of reasons. These may include not being aware of what is available, lacking technical insight, being short of working capital and simply being too busy to focus on them.

Eck Industries got several things right. Firstly, they consulted an independent specialist; secondly they trusted their supplier to provide honest advice, and thirdly they accepted that any significant saving is worth chasing down. Other spin-offs were safer, more attractive working conditions and an opportunity to take their foot off the carbon pedal. This is an excellent example of what is possible when you try.

If you have measured your illumination cost and are concerned about it (but are unsure what the metric means within the bigger picture) then Ecovaro offers online reports comparing it with your industry average, and highlights the cost-benefits of alternative lighting. 

Scrumming Down to Complete Projects

Everybody knows about rugby union scrums. For our purposes, perhaps it is best to view them as mini projects where the goal is to get the ball back to the fly-half no matter what the opposition does. Some scrums are set pieces where players follow planned manoeuvres. Loose / rolling scrums develop on the fly where the team responds as best according to the situation. If that sounds to you like software project management then read on, because there are more similarities?.

Isn’t Scrum Project Management the Same as Agile?

No it’s not, because Scrum is disinterested in customer liaison or project planning, although the team members may be happy to receive the accolades following success. In the same way that rugby players let somebody else decide the rules and arrange the fixtures, a software Scrum team just wants the action.

Scrum does however align closely ? dare I say interchangeably with Agile?s sprints. Stripping it of all the other stages frees the observer up to analyse it more closely in the context of a rough and tumble project, where every morning can begin with a backlog of revised requirements to back fit.

The 3 Main Phases of a Scrum

A Scrum is a single day in the life of a project, building onto what went before and setting the stage for what will happen the following day. The desired output is a block of component software that can be tested separately and inserted later. Scrumming is also a useful technique for managing any project that can be broken into discreet phases. The construction industry is a good example.

Phase 1 – Define the Backlog. A Scrum Team?s day begins with a 15 minute planning meeting where team members agree individual to-do lists called ?backlogs?.

Phase 2 – Sprint Towards the Goal. The team separates to allow each member to complete their individual lines of code. Little or no discussion is needed as this stage.

Phase 3 – Review Meeting. At the end of each working day, the team reconvenes to walk down what has been achieved, and check the interconnected functionality.

The 3 Main Phases of a Scrum ? Conclusions and Thoughts

Scrum is a great way to liberate a competent project team from unnecessary constraints that liberate creativity. The question you need to ask yourself as manager is, are you comfortable enough to watch proceedings from the side lines without rushing onto the field to grab the ball.

Contact Us

  • (+353)(0)1-443-3807 – IRL
  • (+44)(0)20-7193-9751 – UK

Ready to work with Denizon?