Progress meetings are the most common during the design phase of the project.

K. J. Gutteridge BEng, CEng, FIET

Abstract 

Based on experience from several large-scale Mass Rapid Transit (MRT) projects in Asia, the author has found that most design organizations, whether they operate under an Engineering, Procurement & Construction (EPC), Turnkey or Project Management/Project Delivery contracting role, do not effectively manage the design document review process and the comments that this process generates. This element of design management is often poorly managed, yet the impacts of poor design comment management can be significant. Poorly managed design comment management can negatively affect a project in several ways - ranging from delay to design approval that, in turn, may impact the project design schedule and can effect the project financial performance when payment is linked to approvals. This paper describes methods that can be used to improve the management of design review comments.

Context

The MRT E&M systems design process requires design documentation to be submitted for review and approval by the Client. In developing the project design schedule, by default, most organizations do not consider that the design approval process may take longer than planned. The design schedule is usually based on optimistic submission, review and approval timescales. It is the author’s experience that the submission and review process generates many review comments - on large projects often measured in the thousands. In addition to optimistic schedules, project design teams often do not effectively manage design review comments received from the Client’s reviewers. This can lead to:

• Multiple document re-submissions;
• Accumulation of sets of individual reviewer comments that need to be collated, tracked and managed; and
• Impact on the project design schedule.

Where design document approval is linked to progress payments, there can be significant impact to the project cash flow.

Based on experience from several large-scale projects, design documents submitted for Client review and approval will typically generate many comments. Obtaining approval on the first submission of a design document submission is a rare occurrence. The most likely outcome is that each individual reviewer will generate many diverse comments. Once the permitted review time period has elapsed, the reviewer(s) will transmit their comments to the Designer. Prior to the commencement of the design phase, the Designer will prepare a standard form to record and collate review comments for each design submission. The form is a record of the reviewer’s comments for each individual submission - i.e. a Comment Review Sheet (CRS) [see example in Appendix 1]. Prior to any design submissions being made, the Designer should agree with the Client (and other reviewers) that all comments on each and every submission should be presented on the CRS. It is not uncommon for the Client’s reviewer(s) to each return an individual CRS. There may be no attempt to consolidate comments made by each individual reviewer, which can result in some comments overlapping or even contradicting each other. Once the Designer is in receipt of the CRS(s), each comment needs to be assessed and addressed by the author of each design document.

With comments addressed, the design submission document is revised and resubmitted for further Client review. This process becomes more difficult to manage when contracting arrangements involve multiple system or sub-system designers and multiple stakeholders. Consultation with several parties may be needed before a review comment can be addressed. When a project has a poor or ineffective comment management system in place, the comment review cycle can lead to the submission of 5, 6 or more revisions of the same document before approval is obtained. To avoid this situation, careful management of the submission/comment/re-submission review cycle is crucial to minimizing risk to the project design schedule. 

Typically, design team members are aware of the document/review/comment process but it is usually not well managed because it is not ‘owned’ by a senior staff member - e.g. the Design Manager. The ‘owner’ of the process should hold regular (weekly) meetings with the design team members to review the status of every submission - in order to identify and flag potential hold-ups or issues as early as possible. Typically, design team members have heavy workloads and in the absence of regular reviews, problems can go unnoticed until they become significant issues - e.g. non-approval of an element of the design.

Managing the Process and Avoiding Multiple Submission/Comment/Resubmission Cycles

At the time the project design schedule is prepared, it is almost certain that the project planners have not allowed for 5 or 6 submissions of a design document or design package. Typically, the planners will allow for a 30-day Client review period for the first submission and 15 days for re-submission. A period of 15 days is typically allowed for the Designer to incorporate the comments and resubmit the document for further review by the Client. Therefore, if a design document has to be submitted 5 times (to finally incorporate all of the Client’s comments), it could take some 150 days to gain Client approval for that element of the design. This may seem extreme but is not uncommon that the Designer encounters. In projects where there are multiple elements of the design - each requiring Client approval - the Designer’s workload can be great, which in turn exacerbates the problem (when a 15-day turn around for the resubmission of a document cannot be achieved).

Clearly, design approval periods of 150 days (or sometimes more) would impact even the most conservative design schedule. To put this into the context of a large fast-track project, where the project is scheduled to be completed in 3 years, it becomes even more concerning. Furthermore, for many projects, the system design typically consists of several related documents together with their interfaces with other parties. Overall design approval may only be achieved once each individual design document that makes up the design package is reviewed without objection (i.e. approved). Given this situation, it is not surprising to see discrete design phases (e.g. Concept Design and Detailed Design) overlap one another - with design approval becoming a lengthy drawn out process with delays to the design schedule. 

When meeting the design schedule is the priority, and in the absence of Client approval of design documents, the Designer’s alternative is to proceed without approval at its own risk. Depending on the Client’s reason for withholding approval, the risk to the Designer and potentially the project can range from minor to major. Another well-known strategy available to the Designer is to request that comments be resolved in subsequent design stages. However, without any formal agreement in place that clearly defines expectations, accompanied by a mutual agreement between Designer and reviewer, there is a risk that underlying issues may not be addressed - but are simply deferred to a later date.

There are ways to avoid, or at least minimize, the submission/comment/re-submission review cycle - to reduce any impact on the design schedule and in turn the overall project schedule. The following strategies contain obvious elements of good project management, but it is the author’s experience that these are often overlooked, or at best poorly considered by projects:

• Plan ahead;
• Define and establish the Process;
• Define the document submission sequence, scope and purpose;
• Hold pre-submission reviews;
• Undertake Technical & Quality checks before transmittal;
• Avoid comment ‘back and forth’;
• Seek comment arbitration when necessary;
• Consolidate individual review comments;
• Establish metrics to manage the comments; and
• Use a Database tool.

Plan ahead – The Designer should define in advance the design documentation that is to be prepared and submitted. Ideally, this should be prepared well before the commencement of the design phase. The document submission plan (which is typically a deliverable required by the Client) should be derived from the Client’s documents - the contract and technical specifications. However, from experience, it is prudent not to rely solely on the contract and technical specifications, since, in practice either by intention or otherwise, these documents may not comprehensively list everything that is required. Therefore, it is best to use the Client documents as a base and build-out the submission list with other items that are known to be required.

Define & Establish the Process – Design managers and/or project controls personnel should, in advance of the project’s design phase, define and document the design submission and comment review process. This can be in the form of a flow chart [see example in Appendix 2], with clearly defined responsibilities, or the creation of a project specific procedure with the flow chart embedded. Once defined, all design team staff must be briefed on its content and each person’s role within the process must be clearly defined. It should be noted that this process is closely linked to other project controls including, for example, document management, document numbering, document technical and quality reviews and document submission. These other project controls must be integrated with the design submission and comment review process and should not be carried out in isolation.

Central to the process is the comment recording system - for recording each comment and showing its status, and the associated workflows (activity, process, sequence, responsibility). For comment recording, as previously mentioned, most projects use a CRS. However, for the CRS to be an effective tool to record and track comments, other than recording the reviewer’s comments, there are several other important data that it should record. These are:

• Design document number, document revision and document date;
• Date of CRS;
• Comment response (from Designer);
• Comment status - open/closed;
• Reviewer’s statement of severity and impact, if any, on design work - e.g. design can or cannot proceed without Client approval, or no objection subject to incorporation of the Client’s comments; and
• Indicating document approval status e.g. no objection, approved subject to incorporation of comments, rejected etc.

It is important to note that re-submissions of the same document generate multiple CRSs and corresponding sets of individual comments. The ability to record, track and relate the status of each individual CRS and each individual comment is critical to effective management of the comment review process.

Typically, upon the completion of the review, the Client creates a formal project correspondence (letter) with the respective CRS included as an attachment to the letter. This is then sent as a transmittal to the Designer. With this process, great care should be taken to ensure that time from receipt of incoming letter (with attached CRS) to the CRS being available to the designer team member(s) is as short as possible. However, experience shows that this is often not the case, causing delays and increasing the Designer’s document re-submission turnaround time. Furthermore, in these situations it is not uncommon to see project staff resorting to exchanging emails in parallel, which potentially undermines the whole submission/comment/review process. If the project schedule is based on a 15-day turnaround, poor internal document distribution, which can result in delays, is not acceptable.

When establishing incoming document/CRS distribution workflow, it should be remembered that decentralization of design resource is common practice on large-scale projects. Therefore it is unlikely that the Designer’s entire staff is located together and some may not even be working in the same time zone as the Client and the Designer’s project office. In this situation, efficient communication and the speedy distribution of the document/CRS are essential for an effective design document review process. 

Use of on-line comment review, transmission and approval can improve efficiency and a database tool can facilitate this. Therefore, for large-scale projects a database is a better choice than a spreadsheet or Word document. For example, with a centralized database where both the Client and the Designer are connected to the database, review, comment and transmittals are exchanged in ‘real time’ (see also ‘Use a Database Tool’ section).

Document Sequence, Scope and Purpose – Using the submission list as the overall plan, the Designer should define the submission sequence of each document, its intended scope, its limits and relationship to other documents. This is sometimes referred to as a document ‘roadmap’. This is important, not only to guide the Designer but also to avoid reviewers’ comments that will surely arise if the sequence, scope and purpose are not clearly described in the design document. For example, design detail being presented before the design concepts are defined. The document ‘roadmap’ will also help prevent duplication of content between documents. Duplication (the creation of multiple instances of the same information in different documents) should be avoided. Documents that contain the same information must be maintained in parallel, and experience shows that this is extremely difficult, time consuming and prone to human error. Best-practice is to create a single source of information and to use cross-references where needed - by creating separate documents to describe interface details between systems and not repeating the interface details in each related design document. It is also a good idea to include a document hierarchy diagram [see example in Appendix 3] in each submission to illustrate the interrelationship of each related document. Such diagrams provide context and will aid the reviewer.

Pre-submission Reviews – Wherever possible and practical, for the first submission of a design document, the Designer should schedule a pre-submission review between authors of the design submission and the Client. Whilst it may not be practical to conduct pre-submission reviews for every design submission, it is recommended that pre-submission reviews be held for all major/critical design documents. This provides the Client with an understanding of what to expect and allows any issues to be discussed and addressed. It provides the Client some familiarity with the document before it is formally submitted and before comments are made. From a Designer’s point of view, it provides an opportunity to understand the Client’s expectations, helping to focus content and may ultimately reduce comments.

Technical AND Quality Review BEFORE Submission – All design documents should be reviewed by the Designer before they are submitted to the Client. For greatest benefit, a person independent (but experienced and familiar with the design) from the document author should conduct the reviews. The review should include both technical and quality aspects. A distinction is made between technical and quality reviews since they require different expertise and, as such, are typically undertaken by different sections of the Design organization. Technical reviews typically focus on the technical merits of the design and compliance with the Client’s requirements. The questions need to be asked include:- “do we have the right design and is it designed correctly?” Where traceability between the specification requirements and the design is necessary, the design document should also include a Verification & Validation report [see example in Appendix 4] that indicates how the design satisfies each requirement. 

Quality reviews typically focus on compliance with the project’s QA standard including styles, format, structure and correction of typographical errors. Where documents are translated from another language into English, in addition to Technical and Quality reviews, the translation should be checked to verify that the meanings and interpretation are consistent with the original.

Avoid Comment ‘back and forth’ – Experience from both Designer’s and reviewer’s perspectives indicates that it is very easy to unintentionally get into a situation where comments are bounced back and forth between each other (like a game of ‘ping pong’). In doing so, there is a tendency for each party to become even more deeply entrenched in their views. Furthermore, such a situation can also arise where the intent and meaning of a comment is misunderstood by the designer - as the CRS itself does not afford the opportunity to discuss comments with the reviewer. To avoid this situation and reach an early-as-possible approval of the submissions, the Designer and reviewer should meet and discuss any such ‘repetitive’ comments. Although this is a resource intensive method, experience shows that this approach saves at least one re-submission. Ideally, this should be done after the first submission, but definitely after the second submission. 

Comment Arbitration – During the submission review process, it is likely that there may be occasions when the reviewer and Designer cannot agree. If left unresolved, time can be lost and the situation may regress to comments being transmitted ‘back and forth’ (“ping-pong” as mentioned previously). The Designer should be alert to this possibility developing and be prepared to escalate to a 3rd party to arbitrate - the objectives of escalation are to avoid unnecessary delays, keep the design moving forward, and prevent relatively small issues spiraling out of context and out of control. The initial recourse to arbitration, for example, could be to a peer group (comprising both Client and Designer) to provide an independent perspective and opinion. If necessary, further escalation measures could be to higher-levels of technical management with decision-making responsibilities. If a compromise cannot be reached, the 3rd party arbitrator should be prepared to determine the action and direct the appropriate party accordingly.

Consolidate Reviewer Comments – This may seem an obvious task, but it is vital that the Designer and reviewers understand where comments overlap or conflict. Overlaps and conflicts should be resolved via face-to-face discussions between the Designer and reviewer(s) to reach mutual agreement. When faced with schedule pressure, care should be taken not to waste the resubmission of a document by not addressing ALL comments AND any overlaps of conflicts.

Establish Metrics to Measure Status – It is not unusual to find that project staff do not carefully track or measure document submission and approval status. Metrics can help identify problems that are negatively impacting the project schedule/financial performance. Clearly, understanding these enables resolution efforts to be focused. Typical metrics that can be used include, “planned versus actual submissions”, “number of comments”, “number of submissions”, “average number of submissions (to gain approval)”, “documents approved (as a percentage of submissions)”, “review classification (no objection, approved subject to incorporation of comments, rejected etc.)” with a breakdown for each system or sub-system. Once metrics like these are established and regularly recorded and reviewed, trends can be identified and used to correct and improve the process.

Use a Database Tool – Spreadsheets are by far the most common tool to record comments, approval status, record responses and open-closed status (and many other things). Clearly, spreadsheets are widely used to track comments (closely followed by Word documents). However, it is questionable whether these are the most efficient methods? Whilst they are quick, cheap and ubiquitous with little or no set up required, there are several shortcomings that are exacerbated as the number of items being tracked increases. These include:

• Version control - determining who has the current version of the spreadsheet, where even the slightest doubt undermines the credibility of its use;
• The use of coloured/highlighted cells to track updates and changes quickly becomes overwhelming; 
• Keeping spreadsheets up-to-date is often time-consuming requiring extensive manual manipulation of data; and
• Where design documents are submitted multiple times, keeping track of the comment trail also becomes difficult.

With the above points in mind, for large-scale projects, where a large quantity of comments and submissions is anticipated, use of a database is a better choice. In addition to solving the problems noted for spreadsheets, a common centralized database, accessible to both Client and the Designer, can facilitate on-line comment review, electronic transmission of comments and on-line approval. For example, when both Client and the Designer are connected to the same centralized database, the electronic CRS is available to the Designer immediately it is released from the Client’s user. The principles of transmittal and receipt of ‘incoming’ documents are replicated electronically by the database, therefore it is not necessary to mail and distribute hard copies. However, if required for commercial, contract or other purposes, hard copies can be created as needed. 

Use of a centralized database should not be confused with cloud-based file-sharing applications, such as “DropBox”, “Box”, “SharePoint” etc. These applications allow Excel, Word and other file types to be uploaded and shared between users and provide varying degrees of version control. However, cloud-based file-sharing applications do not include other important functions that are necessary to control and manage the comment review process. These include, for example, establishing rules, access privileges, process (workflow) and ensuring the process is followed. A database tool also provides built-in automatic generation and updating of metrics, trends and overall status. A database will do this repeatedly, consistently, efficiently and error free (once the database is configured and properly tested). Clearly, the database tool selected to carry out these functions must allow user configuration/customization.

Conclusions

Design documents submitted to Clients for review will inevitably generate comments from the reviewers. Throughout the duration of the project lifecycle, the Clients’ reviewers can usually be expected to generate large numbers of review comments - particularly if the preparation of design documents is not closely controlled before submission to the Client. The review of design comments needs to be carefully managed by the Designer to ensure that comments are properly considered and addressed and documents revised, resubmitted and the comments closed out. If not properly managed, the comment review process can impact the achievement of the design schedule, delay approval of the design and, where design document approval is linked to payments, impact project short-term financial performance (cash flow). There are established methods to manage review comments, and most design organizations will recognize them. However these methods/processes need to be planned in advance, before the commencement of the design and an experienced staff member must ‘own’ the process and closely manage them throughout the project duration.

Strategies for managing comments and reducing the number of design document submissions include: 

• Planning ahead;
• Definition and establishment of the Submission/Review Process;
• Defining the document submission sequence, scope and purpose;
• Conducting, pre-submission reviews;
• Undertaking technical & Quality Checks before transmittal;
• Avoiding comment ‘back and forth’;
• Seeking comment arbitration - if necessary;
• The consolidation of individual review comments;
• Establishment of metrics to manage the comments; and
• Use of a Database tool.

Adopting these strategies, and incorporating them in a pre-defined design comment management process convinces the author that it is possible to reduce the number of design re-submissions and their consequent impact on the design schedule and possible financial performance of the project.

Credits - Special thanks to David (Dai) Rees, FIET (retired) and Taranjit (Tom) Khella, FIET for their peer reviews, editing and advice.

Appendix 1 - Example Comment Response Sheet (CRS)

 Appendix 2a - Example Design Submission and Comment Review Flowcharts

Appendix 3 - Example Document Hierarchy Diagram

Appendix 4 - Example Document Verification & Validation Report 

Source - Taroko.org, Sigma

When preparing for the project the contractor begins the documentation process?

What is an RFI quizlet?