Building Analytic Components, Part 1

Business Intelligence seems to be on the tips of many IT executives tongues these days. After the Cloud, Big Data, and Master Data Management; Business Intelligence has a prominent place in the pantheon of IT buzz words. The intent of business intelligence to me is quite simple, leverage the information available to the business to improve performance. This can take many forms, but at its core BI isn’t about making the dashboards, cool charts, or data warehouses. It is about helping the organization make better decisions. With that said I am going to walk through the process of designing a single business intelligence analysis component focused on IT Portfolio Management in two articles. The first article will describe the high level process of developing the analytic, and the second article will be a detailed dive into the development of a specific analytic component that will be used to manage investments within the portfolio. I will be using data from the Federal IT Spending dashboard which is publically available at http://www.itdashboard.gov/. Just because the data is from the public sector doesn’t make this example less relevant to the private sector.  IT investment portfolio characteristics like deviation from cost and schedule are broadly applicable across both the public and private sector, and the complex missions and comprehensive IT portfolios ofthe public sector can provide a rich point of reference particularly for very large (Fortune 500) private sector organizations.


Caption: IT Spending Dashboard
I think that with any analysis or reporting component it is important to start by identifying the stakeholder community that is being served. In this case, the analysis component under development will be used by IT Investment Managers and the senior executive team tasked with managing the organizational IT investment portfolio for performance and risk. In both the public and private sectors IT portfolio management is an often talked about but often difficult to execute practice. For large organizationsthe ability to manage competing organizational requirements, disparate technologies, uneven performance data, and complex portfolio composition issimply too difficult.  The portfolio ends up becoming largely managed on the basis of organizational politics and personalities. In order to effectively manage the IT investment portfolio of a large and complex organization there needs to be a data driven approach and analytics that are available and agreed upon by the entire organization. Developing these analytics and the comprehensive suite of informational requirements necessary to manage a complex organization required more space than I have in this article. My intent here is really to focus more on the process of tailoring analytics to meet stakeholder requirements rather than to focus too deeply on a particular problem faced by investment managers.  However, I think there will be some value for people interested in that particular specialty as well.

Whose needs are being serviced?

In the development of any serious analytic component the first thing we need to understand is the stakeholder community weare serving. In our case this community is tasked with managing the IT investment portfolios of large complex organizations. In order to better understand their informational requirements one of the first things I try to do is get a grasp on the informational inputs that they are currently using tomake decisions. This organizational reporting archeology work is revealing because it will often uncover vast differences in the information executives are using to make similar decisions. This is important to surface because one important benefit a BI effort can bring is more standardization to decision-making approaches and a more open evaluation of decision criteria. Another finding is that similar information is being developed for different report, for different executives by different groups.  This results in enormous wasted effort. This homework process sets the stage for the first big exercise in developing a better decision support system for our investment managers – the facilitated session.

Caption: Remember that all your stakeholders are different.
As a consultant I believe that these facilitated sessions are critical to the requirements gathering process and enable frank and open discussion of competing requirements. One of the most consistent things that I have found is that the information and analytics that are currently in use by an organization reflect a much smaller subset of the stakeholders than intended, often only the person who made the buy decision.  The second is that the stakeholders often have very different opinions on what the informational components of the analysis should be. One of the things I often do for my facilitated sessions is to create the giant wall of reporting.  Depending on the meeting space and the volume of reports that I have on hand, I will create an entire wall dedicated to the information that is currently being used to drive the decision making process. In the case of the investment portfolio management team, this will include reports related to individual investments as well as any dashboards, spreadsheets or analytics that are being used to support portfolio-wide decisions.  I will also have one of our team members develop a “core data” sheet that maps the various informational inputs being used across the current reports and analytics, along with some associated meta data regarding authoritative information sources, the process that creates the information, data refresh requirements and any obvious differences inreporting semantics. Finally, we will set up a whiteboard or flip chart board with some basic categories including columns for information that is relevant in the aggregate, information that is relevant at the discrete level, questions that we are trying to answer, and sources of data. Depending on the analysis weare trying to facilitate we may have more spaces available, but those are the basics. This set up process usually takes about two weeks depending on the size of the organization and complexity of the analysis we are trying to facilitate.  A lot of this is working through the homework processes associated with the existing decision-making and reporting artifacts.  Depending on the scope of the engagement this activity may be coming from a business process re-engineering effort or in the specific case of IT Investment Portfolio Management, we mayhave been asked in to supplement or review the process as a whole, which has implications for this process as well.
Important Factors in Understanding the IT InvestmentPortfolio
In any case, once the set up is complete and our homework is done it is time to bring in the stakeholders. I find that one of the first things that is useful is to attempt to provide some structure and scope to the exercise by focusing on the questions that need to be answered and the use cases for the analysis. One of the great benefits of this is providing a focus on the results and outcomes we are trying to achieve rather than simply organizing and reporting on the information that is available. One of the great problems I have found with BI initiatives is that they are often in large part driven by the information that is available rather than the information that would be useful to make decisions. I have never sat in a facilitated session that did not uncover some informational gap in the current decisions support scenario.  The group may discover that some of the data currently being developed and managed is not truly necessary in order to support decision-making. It iscritical that this be recorded and that the thin layer of information required to support decision making be explicit.  This way the resources that are being expended in the as-is state to support the existing decision making process can be re-allocated to meet other organizational needs. Too often reports and analysis components are added but nothing is ever removed. This is not just wasteful of resources, but it is actually harmful to the quality of the decisions that will be made. The world is a complex place for modern executives. The ability of business intelligence systems to not just provide the decision maker with relevant information but to remove informational clutter is critical to the overall quality of the decisions. Many teams are hesitant to remove analytics and reports because “somebody might be using them” or because “we spent so much time developing them.” These are not good reasons to continue maintaining these resources. This is not to say that these exercises should result in an entirely new set of reports and analytics. Maintaining legacy reporting is great as long as there is a real use case, in fact some of the best reports and analysis that will support your organization are probably sitting in spreadsheets on a few executives’ desks right now. These are usually cobbled together from several existing reporting systems by an executive’s staff and have been used to drive organizational meetings for years. Unfortunately, these spreadsheets and PowerPoint are rarely the things that survive a new BI initiative.  Instead, the reports they were cobbled together from in the old system are re-built in the new system and the need to cobble together the spreadsheet that actually drives decision making continues to need to be built every week or month.  Identifying the questions and use cases is critical because it enables the development of criteria for what stays and what goes, as well as setting boundaries around data requirements.
Developing the burning questions
Once you have come upon an initial set of questions and use cases you can begin to design analytic components to meet those requirements. In our investment manager example, one of the use cases is to identify under performing components of the investment portfolio. This enables earlier engagement by management to either increase the performance or terminate depending on a more detailed analysis. The task then becomes selecting from the larger portfolio those investments that may be at risk. In next weeks article I will provide adetailed step-by-step walkthrough of developing a meaningful analytic component for our IT Investment Manager. 

Thanks as always for reading my blog, I hope you will join the conversation by commenting on this post.

If you liked this post, please consider subscribing to this blog and following me on twitter @jmillsapps. I regularly give talks via webinar and speak at events and other engagements. If you are interested in finding out where to see me next please look at the my events page on this blog. If you would interested in having me speak at your event please contact me at events@joshmillsapps.com.

If you are interested in consulting services please go to MB&A Online to learn more.

Be Sociable, Share!

Leave a Reply

Your email address will not be published.