Through my work meeting and discussing BI an Analytics with literally hundreds of clients and potential clients, a discrete but prevalent list of challenges continually emerges. This is not to say that these companies are experiencing challenges in all aspects but some common issues exist. Yes, these problems are common, fundamental even, but as easy as they are to understand, they are difficult to overcome. It takes dedicated effort, domain expertise and the most precious of resources, time.
5. Data Integration – Usually on the top of everyone’s list. I rank this as number 5 because if you achieve dominance over the next 4, integration will cease to be a problem. This issue is also complicated by over-reliance on tools. Integration tools only help you execute; they don’t help you decide what should be executed. This area has been exacerbated due to the rise self-service visualization tools such as Tableau. These types of tools will let you make whatever join you like, right or wrong. In terms of good BI, proceed at your own risk, your mileage may vary.
4. Data Quality – Yes I know, “Garbage in/Garbage out” as the saying goes. A fundamental issue to be sure but one that has whole areas of science and practitioners to help solve. There is also usually a clear success definition as well. Either the data conforms to an accepted standard or it doesn’t. The issue here is really one of execution. Still an arduous process but tools and talent abound.
3. Governance – One of the easiest concepts to understand but perhaps the hardest to implement. Literally who has responsibility for what. A good model, whether Light, Medium or Heavy will clearly delineate the roles, Owner, Steward etc. What usually happens is that most of the functions default to the IT side of the company as business sees anything data related being in that domain. In the best run models, IT is the execution arm and the business makes the decisions as to content, scrubbing, source, use and security. Abdicating this responsibility will ensure a flawed product.
2. Content Understanding – One cannot over emphasize the importance of knowing what your data means. It is the most fundamental of all BI concepts and without it no governance, quality or integration effort will succeed. Tools such as SAS, R and indeed Excel are dedicated to deriving meaning and relevance. This is the sole reason why Data Scientists as a role exist. And yet it does not land in my top position. Why?
1. Business/IT working relationship – The most critical issue in having a top notch BI effort is not technical. Technical challenges are relatively easy to overcome. A trusting, equal relationship must be present to foster effective BI. I’ve seen many BI efforts succeed with one or more of the above issue as problems—except for this one. This is the one issue that will derail the most self-service, automated and established effort. It can not only prevent a developing BI effort from being successful, it can kill an established one should the relationship change. This is the most important success factor and the first one to establish.
The above list represents what I’ve experienced in 30 years of BI and Data Warehousing across multiple industries. There is nothing new here, just an approach as to importance. Like any successful endeavor, it requires patience, trust, commitment and mutual respect. If you’re thinking this sounds like the recipe for a good marriage, my work here is done.