Research, results and recommendations
In 2018-2019 I undertook a research project on the causes of IT Project misalignment and contract disputes in collaboration with NZIAC. In this longer article, I am outlining in detail what the research involved and the key outcomes.
SUMMARY
To date, successful delivery of IT Projects globally have had average success rates. In fact, software procurement, development, implementation and support and maintenance engagements (referred to herein as an IT Project) have had distinctly mixed success rates over the last 25 years (Success being classified as delivered on time and on Budget, and – regardless of functionality ultimately used – with a satisfactory business and User experience). The credentialed USA based Standish Consultancy Group in its 2015 Global Chaos report surveyed 50000 IT Projects globally and concluded that only 29% of IT Projects were successful, 52% were challenged and 19% failed.
The nature of technology models and the availability of data in the last 10 years are making the need for successful delivery rates ever more critical.
Often IT Project misalignment is discovered and recognised as an IT Project contract dispute when the Delivery phase is well in flight. At this time key stakeholders at all levels usually have multiple demands on their time and turning their minds to conflict and its resolution is not desirable and often fatigue based, with limited understanding of key information, and a range of accountability pressures, all meaning their ability to stand back, unlock and resolve can be challenging .
In these circumstances we propose that IT Project Contract terms, which permit a time out review through early use of the above neutral facilitation methods, offer Customer and Supplier leadership (executive or delivery) an effective alternative involving renegotiating the IT Project Contract and recovering and resetting the delivery of an IT Project.
RESEARCH CONTEXT
The research project focused on IT Projects which, fundamentally, involve multifaceted elements, phases and various stakeholders. While procurement methods and project management methodologies continue to evolve, IT Projects are rapidly evolving around the aspects of self-learning Software, big data capture, distributed data centres, high speed networks, multiple device deployment, development methods, integration and cybersecurity complexity, and regulatory frameworks. These all elevate the risks and consequences arising from IT project misalignment and ensuing contractual disputes between Suppliers and Customers.
RESEARCH METHODOLOGY
The following methodology was adopted:
- A survey was issued that focused on the sequence of IT Project phases (ie Procurement, Analysis, Design, Build, Testing, Deployment and Production (Support and Maintenance). The survey asked quantitative and qualitative questions concerning key activities in each phase and the extent to which IT Project misalignment occurred in those phases, what that misalignment was caused by and the extent to which it contributed to an ensuing IT Project contract dispute (ie was it a primary or contributory cause?). The survey also raised questions relating to the behaviours observed when an IT Project contract dispute emerges and individuals’ experience of dispute resolution methods used.
- Senior industry IT Project stakeholders then responded (including CEO/CIO/PM/SA/Program/QA lead/Sales/Legal/other) from 8 countries (New Zealand, Australia, China, Singapore, United Kingdom, Germany, Italy, United States of America). Over 80% of survey respondents had extensive IT Project experience.
- The survey respondents represented a balanced spread of Supplier and/or Customer experience and in key verticals (financial services, utilities, FMCG, telecommunication providers, retail, and government agencies).
- Based on this rich data a draft report was prepared and, with a number of survey respondents and additional senior industry stakeholders, further feedback was sought.
RESEARCH OBJECTIVE
The objective of the research was to use the outcomes to review, assess, explore and develop tailored dispute resolution methods that will best assist Customers and Suppliers in improving IT Project success and maintaining Customer-Supplier relationships (the DR Research).
DR RESEARCH KEY OUTCOMES
These are summarised below:
A. WHEN IT PROJECT MISALIGNMENT MOST OFTEN OCCURS AND ITS PRIMARY CAUSES:
- IT Project delivery misalignment and the emergence of a contractual dispute most often occurs in the period after IT Project Contract execution and during the Solution delivery phases.
- The qualitative and quantitative survey responses and consultation identified two distinct IT Project phases, and primary causes within those, that lead to IT Project misalignment.
1. Procurement phase:
Various Customer and Supplier actions in the Analysis and scoping of transformational Customer Requirements[1] for a proposed Solution were the dominant inadequacy, as distinct from activities in other IT Project phases (Design, Build, Testing, Deployment and Production (Support and Maintenance). In particular these activities include:
a. Business case preparation – driven by Budget constraints and expeditious target business outcomes of a Sponsor, often there is a lack of time spent in reaching a full understanding of transformational scope (Current State to Future State), User needs, optimal Solution selection and the Business Objectives. In this context, Supplier bid or sales teams respond. The direction of travel then shifts to ensuring successful project progression of the selected Solution rather than a Solution delivering an optimal outcome based on fully understood business Requirements.
b. Internal Requirements scoping – inadequate analysis of User needs and reDesign scope together with deficiencies in internal Customer planning coordination, resourcing and contributory engagement of all relevant stakeholders. This leads to Supplier and Solution selection on this basis, with Suppliers responding to the “factual situation” and mitigating risk via high level Assumptions. Their actions are also often driven by competitive “sales pricing” pressures.
c. Known compared to be discovered requirements – there is also inadequacy in finding a mutual Customer – Supplier understanding or balance between scoping known Requirements and to be discovered transformation and the risk and costs associated with this. There is a call broadly for partnering engagement and a deeper pre-Solution selection due diligence (collaborative discovery based on a Customer’s willingness to pay some Supplier resourcing cost in what is a pre-selection time).
2. Delivery phase:
In relation to the areas of planning, people, development and change management, the following insights were discerned:
a. Project Scheduling – often over-ambitious in its shaping.
b. Resource capacity planning, role scope and performance quality:
- For Customer and Supplier personnel often inadequate.
- For the Supplier team, from the start, team resources need to know clearly role description and expected outputs. If Supplier personnel changes, there can be problems with continuity and quality of work and variances in skills.
- Customer management of their service providers (involved in an IT Project) can also be inadequate.
c. Agile or Waterfall development method?
- Cautious optimism that Agile de-risks and improves IT Project outcomes for smaller or medium sized IT Projects. For complex Deployments into Production, Waterfall including its traditional governance processes remains the preferred approach.
- For Agile, all stakeholders need better understanding and training for the roles undertaken. In particular, a Customer needs more effective resourcing, stakeholder engagement and decision making and both Customer and Supplier need to find a better understanding concerning Budget, Requirements prioritisation and the scale of Sprints.
d. Change Control Procedure:
- Materially problematic leading to disagreement as to what is in scope for an agreed change to the Solution and what is out of scope.
- When a dispute crystallises, the audit review of the Change Requests (documented project and contractual) can be incomplete and a source of conflict itself.
3. Governance and measuring success
These are the two main overarching elements in relation to the above specific activities:
a. Governance:
- A dominant theme is the need for it to be effective and active in both the IT project procurement and delivery phases.
- For both phases, from commencement, the nature of governance engagement (motivated and understanding context and what is required) and the skills and experience levels (communication, leading, resolving, guiding and ensuring correct and satisfactory business outcomes) are all critical to IT Project success.
- Specifically, in IT Project governance processes, it was highlighted that failures to escalate, or, when escalated, poor contribution or reaction based on inadequate or misleading information (eg between stakeholders in a customer – prime supplier–subcontractor context) contributed as a cause of material misalignment.
b. Measuring success:
- For IT Projects there is a need for evolution in how success is measured.
- Highlighted is a lack of continuous coordination with business stakeholders and Users, with reliance often placed on the limited traditional Iron Triangle measure.
- Advocacy, from commencement, for measurement of an IT Project’s success driven by consistent leadership and based on KPIs, accountability and fully scoped business outcomes and processes (continuously monitored) was identified as something that was required.
B. DOES THE NEGOTIATED IT PROJECT CONTRACT ALIGN WITH THE ENSUING IT PROJECT DELIVERY TRAJECTORY?
- There was a material response that IT Project Contracts, as negotiated, deviate from the ensuing IT Project delivery trajectory and are too rigidly negotiated. In addition, rigid and contested Change Control Procedures are usually the only process to deal with new or unforeseen circumstances discovered by a Customer or Supplier.
- A dominant number of survey respondents believe that an IT Project Contract should have terms that provide flexibility for the parties to address such circumstances contractually.
C. WHAT BEHAVIOURS ARE OBSERVED WHEN AN IT PROJECT MISALIGNMENT AND IT PROJECT CONTRACT DISPUTE AROSE?
- Initially, often a lack of awareness that a misalignment or dispute is occurring.
- Once it is known, fear, positional and conflict orientated behaviours are demonstrated.
D. WHAT WAS THEIR EXPERIENCE OF DISPUTE RESOLUTION PROCESSES USED?
- Dominant dispute resolution services used are Negotiation (ie re-negotiation) and (neutral) Mediation. There was general satisfaction with the outcomes these processes delivered.
- Litigation was largely viewed negatively, it being fault based, cost intensive and relationship ending.
- Negotiation and Mediation are optimally used as early as possible at the point where trust and respect (absent power leveraging) are present and the dispute resolver exhibits analytical and facilitative skills to unlock entrenched positions.
- It was clear a neutral dispute resolver is preferable to a Customer appointed audit reviewer.
E. PRIMA FACIE RECOMMENDATIONS ARISING FROM THE DR RESEARCH FOR ADOPTION IN IT PROJECT CONTRACTS ARE:
i. IT Project Contract negotiation:
- There is need for more effective alignment in the negotiation of an IT Project’s terms with the material IT Project risks highlighted above.
- Proposed is the inclusion of an overriding acknowledgement or guiding principle that: (i) the parties have used their reasonable efforts to properly scope customer Requirements and Supplier Solution (excluded here are represented and or warranted subject matters); and (ii) if it is discovered, by one party or both, that the scoping exercise was not accurate or sufficient then, for specific agreed areas, the parties may review, discuss and renegotiate these areas (on a without prejudice basis).The parties can renegotiate themselves or have recourse to a mediator or a dispute review board (as set out below). Effectively this would involve by agreement a short suspension of milestone delivery dates and a time out to review, resolve and unlock.
- These contract terms are an alternative to the historically rigid and often contested Change Control Procedure terms.
ii. Modified DR Services specifically for IT Projects:
a. Tailored mediation:
- In IT Project Contract governance, there is a specific agreement for the use of a Mediator to review, analyse and facilitate the resolution of the misalignment or dispute for specific IT Project phases and their activities, for example, Requirements Analysis. Of course, the parties can refer other subject matters to the Mediator.
- The Mediator should have an adequate understanding of IT Project procurement, development methodology (eg Waterfall or Agile or a hybrid or other) and the Solution.
- The Mediator’s costs would be equally shared between the Customer and the Supplier and be included in both the Budget and the Supplier’s Bid costs and/or Fees.
- There may also be a role for a neutral Mediator as part of the Bid selection procurement process or post selection in the IT Project Contract negotiation phase, whereby, in either case, recourse is available for a material subject matter that is subject to disagreement.
b. Dispute review board (DRB):
- For use in large, complex IT Projects as commonly used in the (physical) construction industry
- Like Mediation, the earlier in an IT Project cycle that a Customer and Supplier engage with the DRB, the better. A DRB referral can be utilised for specific IT Project phases or activities.
- The DRB may or may not include a determinative function as well as the facilitative function. This would ensure finality, with the parties able to obtain a decision in the event they were unable to reach a mutual agreement with the assistance of the DRB without having to go outside of the DRB process with the attendant cost and time implications.
These dispute resolution methods are recommended as alternatives to often escalating conflict orientated and resource and cost intensive arbitration or litigation processes. The latter leading to a mandatory and enforceable determination imposed on a party found to be at fault.
[i] Please refer to the Final Report Glossary for further information on defined terms used herein.
Be Amorgos 