News
eFOCUS - spring 2007
Performance Measurement - it's not rocket science is it?
Industry comment by Peter Ellis, Principal, Investit, www.investit.com
Research carried out last year by Investit found evidence that performance measurement and analysis are not scaleable business processes in most investment management firms. A lot of people who have worked in performance over the past 10 years or so would probably dispute this assertion. For example, the calculation of performance returns involves significantly less manual effort than was required in the 1990s. The explanation for this apparently counter-intuitive finding lies in the details of how performance practitioners operate and behave.
Performance teams have grown in stature within our industry over the past 10 to 15 years. This is because firms have come to recognise the importance of the function that measures and analyses the effectiveness of their revenue-generating processes. However, it is still an area that is poorly understood by people who do not work in it. Many people in investment management regard performance measurement and analysis as a highly technical and mathematical subject. In other words they regard it as the proverbial rocket science.
In reality, the theory behind performance measurement isn't really that difficult, and it certainly isn't rocket science. However, those who work in the area do seem to like engaging in technical discussions. Performance conferences and forums are often dominated by presentations and panel sessions that argue the merits of one calculation method over another. This creates the impression that the subject is a highly technical one. That impression is actually bad for performance teams, and for their firms, because it encourages the wrong type of behaviour, both inside the team and in the way it interacts with the other parts of the firm.
The real challenge facing performance teams is not the theory of performance measurement but the practice. Specifically, the key challenge is how to ensure the integrity of the source data used to calculate performance data. Collecting a complete and accurate set of source data can be extremely difficult because few people outside the performance function require the data to be at the same level of integrity. Index data and portfolio data that is perfectly acceptable for an order management system, may fall well short of the level of completeness and correctness required by performance systems.
Data issues have always impacted performance teams, but the issues have got much worse over the past 5 years. This is because performance data is now being calculated at much lower levels of detail. For example, the following are now common practice: stock level attribution, fixed income attribution, the use of daily values of data to calculate returns, and transaction-based attribution. Calculating performance at the total assets level and at sector/country level for standard time periods is a thing of the past. The modern trend is towards calculating at the lowest level and aggregating results as necessary over user-defined time periods.
Typically, performance teams rarely have a complete set of correct source data, across all time periods and across all clients and portfolios. Data collection and cleansing are iterative processes: calculate performance data, check it, investigate errors and anomalies, correct the source data, and recalculate performance data. What makes things worse is that the upstream systems and business processes on which performance teams are dependent are often fragmented, and the performance team has to provide the necessary level of business integration, usually with manual effort.
This is why performance processes are not as scaleable as they could be even though over the past 10 years the functionality of systems has improved significantly. These improvements have made it possible to centralise and automate the calculation of performance data, but systems have been implemented as departmental solutions. Performance teams tend to be insular, looking to solve their data issues internally, but data integrity is a business-wide issue requiring a business-wide solution.
So what should performance teams be doing?
Firstly, performance teams need to be more proactive in interacting with the other areas of their firms; they need to abandon the insular approach that has characterised them in the past. Performance data is usually calculated at the end of a reporting production line but calculating it correctly and on a timely basis relies on the integrity of all upstream business processes, systems and data sources. The volume and range of source data extends throughout the firm but, while errors can be introduced at any stage, many of them go undetected until performance data is calculated.
Despite all of this, most performance teams do not take a business-wide, strategic approach to systems and business processes. Some examples of ways in which performance teams can be more proactive in their interactions with their firms are:
- Agree SLAs with the teams that supply them with data.
- Design integrated business processes with other teams to ensure that effective end-to-end processes are established across the firm.
- Use performance data as an active component of data verification processes. For example, if performance data is calculated and checked daily or weekly, it can be used to identify data issues and anomalies during the month, instead of at the end of the month when there is deadline pressure to produce reports.
The second major change that performance teams should make is to become less preoccupied with the technical aspects of performance measurement and analysis. Performance is necessarily a technical discipline but people who work in the field focus too much intellectual energy on technical matters. Performance teams do not operate in an academic environment, performance is a technical discipline operating within a business environment. Intellectual discussions about, for example, the relative merits of money-weighted and time-weighted calculation methods are interesting. However, what really counts is not simply the views and opinions of performance teams as to what are the best techniques to be used, it is the requirements and needs of their clients both internal and external.