I had written a brief post on the seminal “The Mythical Man-Month” some months back. I am currently in between jobs and thought of using this period of lull to do a quick refresh on this timeless classic on project management. I work in an Indian IT services shop so this is purely from my point of view. The thoughts below intend to mesh together the seminal classic’s and my observations over a decade in the IT service sector.
Requirements and Estimation
Let’s say you are a project manager for a new development project. At the inception itself the first action item you would need to tackle is defining efforts in terms of resources and schedule for the completion of the project. It is extremely rare or to be blunt normally impossible to define requirements during the early days of the project. Usually at this point, we have what can be considered as very high level requirements whose sole purpose is to provide the project team with a general idea about what the final solution might turn out to be. Here the project manager with his experience and domain knowledge is expected to fill in the gaps in requirements and arrive at an estimate in terms of resources and schedule. The accuracy of this estimate is subject to the depth of the domain knowledge of the estimator and also his/her familiarity with the customer’s vagaries in terms of degree of change in requirements. The later part is useful in adding more efforts to act as contingency against scope creep or changing requirements. Note that I am assuming that the project is going to be a ‘fixed price’ project executing using the waterfall or the in vogue iterative development methodology. Indian IT customers have started experimenting with ‘agile’ methodology; however these are purely being pursued to validate if agile can be practiced with an offshore model and agile till today does not have a large scale adoption.
So how does the project manager put together his estimates? Based on requirements provided, he starts guessing the possible web screens the application will require. He might have details around the computational algorithms and business processing the application needs to do. Additionally if information needs to flow in a uni or bidirectional manner with external resources; the same will be considered. Once he has identified the system scope, its boundaries etc, he will use either the formal Function Point, Use Case estimation or Work Breakdown Structure estimation technique to arrive at an estimate. The choice of the estimation technique will be purely based on the project manager’s comfort level with the technique. Some techniques also use a factor called as productivity. The factor is meant to reflect the enhanced productivity gains achieved by deploying senior personnel on a project. Although this is great from a sales talk perspective, it is half truth. Productivity gains can be achieved only if the same personnel are deployed on the project for a long time. This is not a valid statement as personnel are regularly rotated or lost due to attrition or growth within the organization. Also if the same person is retained, he/she will be paid a higher salary as a part of the annual salary revision; productivity gains will be offset by higher salary costs. Today, the fact of the matter is that the average experience age in a project team is decreasing. This is due to increase in demand for experienced people. The total count of experienced personnel in India is a constant and will increase in a fixed fashion on a year on year basis. The demand on the other hand is steadfastly increasing. This is reflected in the productivity figures. Around 5 years back the Java Function Point development project productivity was around 0.9- 1.0; today it is around 0.7. Therefore, delivering projects faster is becoming a trying and not challenging experience. Coming back to our illustration.
The project manager prepares his estimates and bravely puts forth the schedule and estimates before the client. Today in 9 out of ten scenarios if the schedule is beyond six months, the client would revert back saying that the schedule is too long. Can the team come back with a leaner schedule which should be say around six months? Why six months because that is the maximum duration for a company can delay in getting a new product/application out in the market. Anything beyond that time and the company is losing its first mover or the early adopter advantage. Now you may be asking if this question depends on the project size. Meaning if the project size is 1000 person days or 15000 person days will the client still ask for a six month delivery? Well, the answer is yes. Size does not matter. Delivery in six months is seen as a business imperative. The actual delivery might not occur till a year or more has gone but the client expects a commitment from the IT service provider of six months. The project manager is now subjected to external forces namely the provider’s sales team, his superiors etc who are hell bent on getting this project. This is a typical Indian IT phenomenon. Do not refuse projects with unrealistic time lines. We will see about project delivery later. So what does the embattled project manager do? He uses the man-month interchangeability formula. The number of resources on the project are increased and the schedule reduced. I have seen this occur time and again. Time and again the schedule is crunched and time and again the project runs into schedule overrun.
What does Mythical Man-Month say?
So far we have an insight on how the project estimation takes place. So let’s now move on to what Frederick Brooks has to say @30 years back on this. He says that the following:
- Estimation techniques are poorly developed
- We tend to confuse efforts with progress
- We are uncertain about our estimates
- Schedule progress is poorly monitored
- In case of schedule slippage, the natural response is to add manpower
Let’s look at it point by point. I partially disagree with point 1. I do not think that the formal estimation techniques are poor. I think provided the estimator is sufficiently experienced and has got his/her requirements right the estimates are realistic with -+10% variance. Estimation techniques cannot compensate poorly written requirements. I believe what estimation methodologies lack is the ability to accept interim changes in requirements. Estimation techniques are adequate for measuring the entire size of the project, but they do not provide a way of recomputing a project size midway during project development. Point 2 I agree, we do display this tendency of confusing efforts with progress; although there is way around this. We break the large project into granular sub-modules. The degree of granularity can be decided by the project team, although the recommendation is to break each sub-module down to such a level that each sub-module size does not exceed 5 person days. Point 3 has more to do with changing requirements and that is inevitable. For point 4 I believe my thought process would be inline with the explanation mentioned for point 2. Point 5 I wholeheartedly agree and I have already elaborated on this in my previous post.
Now let’s talk about man month interchangeability. As per Brooks, man month interchangeability is no longer valid in case a task cannot be further partitioned or where partitioning is possible but it would involve communication(training and intercommunication) between the sub-tasks. Although the interchangeability is regularly used to reduce project schedules, one should be giving due deference to the communication costs. Unfortunately there are no formal methodologies to estimate these costs and also to provide project viability for a particular resource and schedule combination. It all boils down to the project manager’s experience to determine if it is wise to pursue a project of a given size with a specific resource and schedule combination.
Here’s the final word on man month:
Good cooking takes time. If you are made to wait, it is to serve you better, and to please you.
Requirements & technical specification document
OK. The project manager did the estimates, came up with the resource count, schedule and translated those inputs into project costs. The client approves the cost and the project begins. The team sends across a single or more than one business analyst(s) to the client side to start requirements elicitation, drill down and documentation. The final deliverable of this exercise is a requirements specification document. Typically in my experience this exercise is normally done by a single person and in some cases two people. As the document serves as the single source of truth in case of contradictions etc, it is vital the document be written in precise fashion leaving no room for ambiguity. Typically in Indian IT this exercise is done by people with 5 years of technical expertise. Their oral and written communication skills could be lacking leading to poor quality of requirements elicitation and documentation which has severe implications on the subsequent downstream activities. Post completion or when the requirements document is nearly complete, the technical architecture document is initiated. The technical architecture document provides information of the technical frameworks/components selected, their justification for selection, coverage of all functional and non-functional requirements and any specific aspects the design team needs to take into consideration during the high level design.
What does Mythical Man-Month say?
I doubt Brooks at his time faced acute resource crunch that Indian IT faces today, so his thoughts do not cover this aspect, although he does mention that it is vital to maintain conceptual integrity of the architecture. To achieve this he maintains that the architecture documentation be undertaken by preferably one person or a very small number of people who share the same thought process. In Indian IT context, this is true by default as most of the times, the requirements/architecture document is produced by a single person however I believe this has far reaching implications in case of product development where the desired features will be large in numbers, hence there might be some thought process of having many hands making the work light.
Project requirements analysis is over and they have been clearly articulated in the requirements document. Post requirements, the application architecture and high level design is complete and now we move on to the project development phase. To ensure better project monitoring, the development effort gets sub-divided into tasks and a collection/set of tasks is provided with a clearly defined milestone. The milestone defines a date by which those tasks need to be completed. Let’s say we have a project with schedule of 1 year starting on 1st Jan and has four milestones each ending every quarter i.e. 31st Mar, 30th Jun, 30th Sep and 31st Dec. The project has started on Jan 1st and on reaching 31st March the project manager observes that the tasks associated with this milestone has not been completed. What does he do? He either asks his existing project team to start putting in long hours so as to make up for the lost time, alternatively in case the slippage is caused by requirement creep or other external factors, he will increase his team size so that the subsequent milestones do not suffer. He will rarely think of renegotiating with the customer for a new deadline.
What does Mythical Man-Month say?
In a similar situation, Fredrick Brooks envisages that the project manager has the following alternatives:
- Assume that the first milestone was estimated incorrectly, add manpower to cover the difference and leave the project schedule untouched
- Assume that estimates for the entire project was incorrect, re-estimate and add appropriate manpower, leaving schedule unchanged.
- Reschedule the project milestones and ensure that there are no misses in requirements
- Trim the project requirements to leave resources and schedule untouched.
As per Brooks options one and two are recipes for disaster. Yeah, that’s been my experience too. Options 3 and 4 are the way to go. Although I agree with Brooks, on the ground due to external influences these two approaches stand very little chances of ever being put before the client.
To sum as Brooks says
Adding manpower to a late software project makes it later.
Man learns from his experience. He tends to apply his learning to his subsequent work. The same holds true in software. A person who has used a specific technology continues to promote the same for time immortal even when there are better alternatives available in the market. This is what I called psychological inertia. A person who has developed web applications using Struts framework may continue to promote same even if there are other viable alternatives. It is important to learn new skills and it is vitally important to unlearn new skills. This would allow you to make infallible technology choices which can affect project team’s ability to deliver.
What does Mythical Man-Month say?
Brooks does not make any reference to psychological inertia. He makes a reference to ‘Second-System Effect’. Brooks says that an architect builds his first system carefully and is constantly on guard against over engineering in turn delivering an optimum system. During the first system’s development, he maintains a log of all the nice nuggets or features he desired but was not able to incorporate in the first system. When it is time for the second system to be designed, the architect tends to go overboard by incorporating those missing nice to haves and thus bloating his second system. Brooks says that architects should consciously guard themselves against this tendency.
These are my thoughts for the moment. Will augment this post as I finish the book.