The 12-Step Program for Successful Software Development
© 1997 Frederick Gault
Frederick Gault & Associates, Inc.
195 Vernon Street
San Francisco, CA 94132
(415) 239 – 7681
fgault@pacbell.net
www.fgault.com
The 12-Step Program for Successful Software Development
*I. Step One – Plan Properly
*Planning For Software & Hardware Products
*The Author’s Experience
*The Benefits of Advance Work, And How to Do It
*The Dangers of Poor Planning
*A War Story
*II. Step Two – Design the Product Properly
*Define What You Want to Build
*The Necessary Parts of a Written Technical Specification.
*III. Step Three – Determine Project Constraints
*What Are the Limits on Your Project
*IV. Step Four – Know Your Customer
*Put Everything in Writing
*Build in Controls
*V. Step Five - Modularize Your Project
*Everything Is Late
*How to Buy Insurance
*VI. Step Six – Make a Proper Schedule
*The Necessary Parts of a Written Schedule
*Events That Destroy Schedules
*VII. Step Seven – Remember Testing and Quality
*Why Testing is Essential
*Testing is Iterative
*How Testing Can Balloon Out of Control
*How to Limit Testing to the Possible
*Testing Pitfalls
*VIII. Step Eight – Make Sure You Are Financed Properly
*Prevent Budget Overruns
*Project Starvation
*IX. Step Nine - Problems and How to Avoid Them
*Management
*Engineers
*Testing
*Artists and Other Creative Types
*Another War Story (The Mad Artist)
*Customers
*X. Step Ten – Consider the Legal Issues
*Never Use Unlicensed Software
*XI. Step Eleven – Don’t do This
*The Story of a Spectacular Failure!
*XII. Step Twelve – Do This
*The Story of a Stunning Success!
*Frederick Gault managed or helped Produce the following Software Projects *:
*
Planning For Software & Hardware Products
Many software development problems are avoidable. Proper planning can prevent staffing problems, budget problems and schedule overruns. This paper discusses how to pre-handle project problems in detail, but by far the most important piece of advice is to plan thoroughly.
The reader is excused from asking, "why should I believe this guy?" I’ve been fortunate to have developed scores of software products over the last Twenty years. And they’ve been of all types, some products are on sale at software outlets, one system I built is used to prevent a type of credit card fraud and yet another system manages trucks that carry wine from a major winery. In all, I estimate that I’ve been asked to build over 100 software systems and products. To be sure I didn’t personally write every line of code, but many of these products would not exist without my participation. I’ve included a list of products at the end of this article for the reader’s amusement.
The Benefits of Advance Work, And How to Do It
Somewhere in the organization there is a person or persons who have the vision of what the new project should produce. It is wise to remember that these people are the individuals who will determine if a project has been successful. It is not enough to have a pleased engineering staff because they used a cool new technology when completing the project. It is most important to produce a product that solves the business problem at hand.
Talk to the visionaries and the ultimate users of the system. Keep the technical staff involved in this process to provide an anchor on the users’ wild-eyed ideas. It is a good idea to schedule several different interviews with each player. As you discuss the project with one group questions will arise that can only be addressed by another group.
Ask questions about the business problem that will be solved by this new system. Also be sure to ask about the user interface design, the budget expectations, the date the system is needed and who is going to support the finished product. It’s always good to ask directly, "how will this system make your life better?"
It’s always fun to run off to beach on a whim. In that case planning would ruin the experience. However, you probably wouldn’t want to plan a trip to another country quite so impetuously, and a trip to the moon would require extensive planning. Software projects are in the "Trip To the Moon" Category.
My favorite cartoon is the one where the manager is looking back over his shoulder at his programming staff as they sit at their desks. He says, "you start coding, I’ll go up and see what they want!" Lets just enumerate what you won’t know if you don’t plan. You won’t know:
Poor planning means you haven’t adequately investigated these issues. Remember Murphy’s Law, that states ‘if something can go wrong, it will’. Human nature being what it is, people will assume the rosiest scenario if you don’t give them realistic estimates. That means that management will assume a short development time on a meager budget.
A design firm came to me to help them complete a screen saver for a major computer hardware company. They had designed and built the screen saver to run in the computer stores so customers could learn of the benefits of the Hardware Company’s system. I was asked to have my team fix a few "minor" problems with their system. The design firm claimed that the system was pretty much done but they lacked my experience and knowledge to "tighten up" the code.
You can probably tell even with that brief description what my error was. Yes, instead of investigating exactly what it was that I was to do, I simply agreed to accept the job. The engineers quickly determined that the program had been hacked together by a rank amateur and needed to be completely re-written. The "minor" problems included one that seemed to cause a catastrophic system failure that prevented a re-boot. Needless to say we were looking at financial ruin. Fortunately I was able to work with my customer and work out a compromise. We identified the problems that needed fixing and then agreed to an daily rate to work through the list of problems.
The point is, I could have avoided the confusion in the first place by taking the time to understand what was needed and develop detailed estimates.
II. Step Two – Design the Product Properly
You must define the product you want to build. For such an obvious statement you’d be surprised at how seldom it is done. The more detailed the design the fewer surprises will be encountered during the project. I have literally been involved in projects where someone felt that a verbal discussion was a design. Insist on a written document with lots of engineering and User Interface details.
The Necessary Parts of a Written Technical Specification.
A complete technical specification should consist of the following sections:
III. Step Three – Determine Project Constraints
What Are the Limits on Your Project
Determine the boundaries of the project. The obvious limits are time and money, but there are many others as well. As much as we might like to design and build the ultimate computer system, the reality of modern business is one of limitations. Keep your design within the boundaries of the limitations. Here is a checklist of project limitations:
IV. Step Four – Know Your Customer
You may be thinking, "I work in a large company, I don’t have customers". Think of the people who are going to use the new system you build as your customers. Also think of the managers who sanction a project as customers.
It is human nature to want something for nothing. Everyone looks for the best price. And whenever we spend money there is always the nagging suspicion that it’s too much. Your customers are that way about you. Whatever you propose, your customers will want more for less money and in less time! Don’t be upset, that’s just the way people are. To avoid running afoul of these attitudes, be prepared to give your customers realistic estimates and specific plans in writing. Then, if necessary, the document is available to remind everyone of the agreement.
It is almost inevitable that during the life of a project your customer will demand a change to even the most carefully researched plan. This is never a problem as long as the customer understands that changes to the design mean extra costs in time, money and resources. Consequently it is wise to say so, in writing. If you must craft a contract with a third party then attach the design specification and have a provision in the contract that states that deviations from the design mean extra costs.
V. Step Five - Modularize Your Project
Parkinson’s Law states that the work will expand to fit the time and materials allotted. In other words, people will look at the schedule and then proceed to use all the time allotted. That means they will most likely overestimate their skills and underestimate the problems they will encounter. So, delivery will be late. This is something of a catch-22 – how do you avoid it?
The best way to prevent late delivery is to build in a safety factor on the schedule. However, it must be in such a way that the engineers don’t see the time as available to them. An example would be to build in a time period for Beta Testing. The schedule should clearly indicate that the Beta Delivery means code complete. Beta Testing is defined as testing the completed product, so the engineers think they must be done by Beta. However, the Beta Testing Cycle is really about testing and fixing the product, so unfinished parts of the product can fall into the Beta Testing time slot.
Another insurance policy is Milestone Deliveries. A milestone is a delivery during the middle of the development cycle. There can be several milestones. If you clearly define what tasks should be finished at each milestone, and when that milestone is, then you can gauge the performance of your team. It also helps a team to have shorter-term goals. This is in addition to the tasks assigned to each engineer. A milestone is most effective when several different subsystems must start working together. No engineer wants to be the odd man out when the milestone is due.
When you decide the schedule, be careful to schedule tasks at a point when it is least likely they will have to be done over. For example, if you schedule the completion of the User Interface before the database design is finished, it might be necessary to re-do some screens based on database design revisions. Look carefully at the dependencies of tasks to avoid this type of problem.
Last, if you understand all the tasks in the project and the dependencies between them, you will see parts of the final product that can be removed. It is always good to see the project in terms of separate functional areas. In that way it will be possible to assign one functional area to another team if necessary, or remove an entire part of the project for implementation in a later release. It helps to understand the relative importance of the product features. If you must drop features, make sure they are the ones least needed to solve the business problem.
VI. Step Six – Make a Proper Schedule
The Necessary Parts of a Written Schedule
It is wise to use a 3rd party scheduling package. A schedule is most useful when the tasks are known in great detail. The schedule must contain the following information:
You’ve probably seen schedules that are dead on arrival. I know I’ve seen schedules that were little more than wishful thinking. How does this happen? How do we prevent it?
Don’t forget the deep-seated human need for a demonstration. Usually the more important the person demanding the demo, the more disruptive it will be. Ask your customers during the planning phase, "will you need to see a demo?" Let them know when you will be ready to give a demo. There is nothing worse than having the system in pieces and getting word that the President wants a sneak preview. Sometimes there might be an industry trade-show that your customer feels must include a demo of your work. Make sure your schedule reflects this.
Bad assumptions are the most likely source of schedule trouble. If you don’t have a good feel for how long it takes to do a given task, your schedule will begin to suffer from compounded bad estimates. Be sure to ask engineers how long they think it will take to do things. Usually they will be generous to themselves and give a task plenty of time. Don’t be afraid to challenge them. Remember, knowing in detail what tasks must occur and what tasks are dependent upon each other will go a long way towards building a tight schedule.
The most serious schedule problem, and the most frequent is what I call "Schedule as Fiction". This schedule is one that is built by an individual without enough backbone to stand up to the customer’s wishful thinking. Even the most powerful executive can’t suspend the laws of physics. If your management wants a six-month project in a month, it won’t happen. Don’t be afraid to say so, believe me they are going to find out sooner or later! I believe that you will gain more respect by telling it like it is. But be prepared to back-up your assertions with research. Don’t just say you don’t think something is possible, research it and find out for sure.
VII. Step Seven – Remember Testing and Quality
Once again we have a self-evident concept - testing - that is often shortchanged. Quality Assurance is a discipline in its own right. Find someone trained in the science of Quality Assurance and heed that person’s advice. Quality is cost effective. Plan for the time to test your product.
A common misconception about testing is that you do it and its over. A moment of reflection tells us that once you identify a bug in your new product it must be fixed. Once the bug is fixed then the entire product must be re-tested. So, plan for many iterations of test cycles.
How Testing Can Balloon Out of Control
If testing is not approached realistically it can quickly escalate into the realm of the impossible. If you think of the hundreds of features in a typical software product, the number of permutations between them is astronomical. It is often literally impossible to test all combinations of user activity in the new system. And remember that testing must be re-done every time the code is changed.
How to Limit Testing to the Possible
There is an entire science to quality. However, here are a few ideas that can help you as you consider testing:
Develop a test suite. A test suite is a script for testing. If possible automate it. This can insure that each test cycle is complete. The wrong way to test is to have someone "fool around" with the system.
Use realistic data. Setup test data sets that are actual sized. If your system will need to deal with millions of records, it is not enough to test with a small test database. Also try and use a database populated with realistic values. This can expose problems that might not be evident with a small test file.
Have the end users work with you in the testing process. The users can not only expose functional problems with your product but also user interface issues that make life difficult for them.
Develop a Sparse Matrix. This is a statistically significant subset of all known environmental variables. For example, if your product needs to work on all known makes and models of desktop computers, it would be impossible to purchase one of each and test on it. However, you can identify a set of desktop computers that will most likely expose any platform dependent problems.
Be sure to consider the following when testing your system or product:
VIII. Step Eight – Make Sure You Are Financed Properly
If you work in an organization where you are given a budget, make sure you scale your expectations to that budget. If you are responsible for the budget, make sure its enough. Even though you may please your customers in the short run with a nice price, eventually it will become clear how that price limits the results. Its best to match the budget and expectations up front. If this is out of your control, make your views known and be sure to support your assertions with facts.
You should always be prepared to answer the question "Where have all the dollars gone?" But it’s best not to find yourself in the position where you are called upon to do so. Money feeds your project. When you run out the project starves and eventually dies. Consequently it is wise to budget realistically and plan for problems. Consider what the costs will be for typical problems. What will it cost if a key engineer quits or becomes disabled? What will it cost if you need to buy equipment?
IX. Step Nine - Problems and How to Avoid Them
Most project problems are caused by people. Here are some of the personality types and the problems they might cause:
Managers are often referred to as "clueless". In fact management is focused on business issues which are not always visible to people working on a project. Add to this the fact that managers are not paid to be engineers, so don’t scorn them for not understanding arcane technical issues. It’s your job to inform management so they can make good decisions. You will gain more respect in the long run by being professional and honest at all times. If you develop a bad attitude towards management trouble is more likely to occur.
Engineers are smart people and often have strong opinions. Its common to meet engineers who are convinced they know best how to run the entire business. The truth of the matter is that engineers seldom understand the business issues that provide them with jobs. It is important to keep them focused on the problems they are paid to solve. This can be a challenge. The best strategy is to keep them well informed about the business issues and set specific goals. Sometimes it will be necessary to tell an engineer that although a decision is "wrong" it is the way that management has decided to proceed and there is nothing that can change it. This is not to say that engineers who have complaints about safety or moral issues should be ignored.
The people who test your product have some control over the development cycle. If a tester won’t agree with you that a bug is really a "feature" you may have a problem. The testers should understand that the product will be finished even if it isn’t 100% bug free. The types of errors in the product should be ranked in order of importance. Before testing begins it would be wise to have a management referee who can resolve disputes between the engineers and the testers.
Artists and Other Creative Types
In recent years, with more advanced User Interfaces, it is common to have designers working with you. These people can be very frustrating to work with. Often they will have a more relaxed view of deadlines. It is best to work with commercial artists. Fine artists will not fit into the high-pressure environment of the typical software project. In fact many artists pride themselves on their counter culture point of view. Commercial artists, on the other hand, are more aware of the needs of their clients. A commercial artist will understand that (s)he may need to create something (s)he doesn’t find asthetically pleasing.
Another War Story (The Mad Artist)
During the development of a multimedia product we needed a design for the main screen. We retained a fine artist of some renoun. His design sketches looked fantastic and we had high hopes. However, once production began we didn’t receive any art from the artist. The artist spent a great deal of time working on some sort of "textures" that had no place in the product. After a few weeks we gave up and hired a commercial artist to complete the work. Meanwhile, the fine artist didn’t seem to understand that he had been fired and kept contacting us about buying a new high quality scanner so he can digitize his new textures!
Always communicate professionally and honestly with your customers. You should be resigned to the fact that no matter what you do the customer will be unhappy about something. Resist the temptation to tell the customer what (s)he wants to hear. Here are some ways to prevent problems:
Contract control. Make sure the contract is detailed and clear. The point of a good contract is not to have a document that helps you in court. The point is not to have to go to court.
Testing - who controls it. Agree on what constitutes being finished. It helps to agree that there may be some minor bugs allowed in the final product.
Finance. Make sure there is a provision in your agreement for changes. If there is no consequence to the customer for changing his mind, then (s)he will make budget busting design revisions.
X. Step Ten – Consider the Legal Issues
First and foremost, work with an attorney. In modern society everything around you is owned by someone. Photographs, music, logos, even some words. Most importantly the computer code is owned by someone. Make sure that as you and your team create the product its ownership is clear.
If you are working inside a large business, then it is likely that corporate council has asked you to place a copyright notice within the code. But if some of your staff are on contract, then legal probably has asked for an assignment from those contractors.
If you are independent and working with a client, then your client will need to understand who owns the final product. Depending upon the deal, your team may or may not retain some rights.
Again, work with an attorney. Ask the question "Who owns what?" Make sure the ownership of the following is not in doubt:
Make sure your staff understands that using unlicensed software will not be tolerated. The potential legal consequences far outweigh any possible savings from using stolen software.
XI. Step Eleven – Don’t do This
The Story of a Spectacular Failure!
I was contracted by a large multimedia development company to build a children’s product. After careful analysis I developed a budget and fielded a team to do it. The client agreed to the budget and the contract and we began to work. Early in the development the customer demanded that we prepare a demo for an upcoming trade show. The client was unyielding about the need to show a prototype at the trade show. Unfortunately we weren’t anywhere near ready to demo, and the contract didn’t cover such a situation. Instead of telling the customer it wasn’t possible, I tried to appease them by diverting the development effort to create a demo. None of the demo development effort was usable in the final product. The final result was that we missed the final delivery date by two months.
The mistake was:
Believe me this customer was not at all pleased when the final delivery date was missed.
The Story of a Stunning Success!
While working at a major California bank I was asked to take on an impossible project. The credit card clearing authority had given the bank a lengthy period of time to institute a process that would determine if an individual was attempting to apply for many credit cards at once. In order to secure the cooperation of each member bank, there was a daily fine for failure to have the system up and running on time. Because of a pressing backlog the bank had not assigned anyone to build this process.
For political reasons this certain-to-fail project had been dumped into the lap of a woman manager. When she approached me, she knew I could work fast. I accepted even though the project had to be completed in 6 weeks! Working 8 hour days and even taking a two week vacation, I was finished on time. How was this possible in a world of constipated software schedules? I attribute my success to several factors:
Surprisingly these kinds of stories of not uncommon. Often entrepreneurial first cuts of software are built very rapidly. This is because the task is clear, the staff is motivated and the team is small. In my experience big backlogs seem to correlate with big teams with fuzzy goals.
Frederick Gault managed or helped Produce the following Software Projects *:
Title |
Publisher |
Mfg |
Work Canyon Did |
QuickTime for Windows 1.0 |
Apple |
Apple |
Windows C & Asm code/design |
QuickTime for Windows 1.1 |
Apple |
Apple |
Windows C & Asm code |
QuickTime for Windows 1.5 |
Apple |
Apple |
Windows C & Asm code |
QuickTime for Windows VR |
Apple |
Apple |
Windows C & Asm code |
Worldview Analyzer |
Intel |
Intel |
Windows C++ design & code |
Video Capture for ISVR card |
Intel |
Intel |
Windows C & Asm code |
Video over Oracle Network |
Oracle |
Oracle |
Design and prototype code |
Adobe Type Manager |
Adobe |
Adobe |
OS/2 coding in C |
OS/2 C++ Compiler |
Borland |
Borland |
OS/2 coding in C |
How Your Body Works |
Mindscape |
Mindscape |
Win & Mac Programming C++ |
Mavis Beacon Typing for Kids |
Mindscape |
Mindscape |
Macintosh Programming C++ |
DNA Autosequencer |
Li-Core |
Li-Core |
OS/2 Design and Code C |
Peter Rabbit Stories |
Mindscape |
Mindscape |
Win95 & Mac Prog C++ |
Cooperation |
NCR |
NCR |
OS/2 coding in C |
StarTrek Tech Manual |
Simon&Sch |
Imergy |
Dir Xobject - VR Technology |
Gettysburg |
Turner Inter. |
Turner |
Digitize Video |
How To Digitize Video |
John Wiley |
Canyon |
Book CD Win & Mac |
NFL SuperBowls |
DiscUs Sports |
DiscUs |
Windows Programming C++ |
Penthouse Interactive 1 |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Interactive 2 |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Interactive 3 |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Interactive 4 |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Interactive 5 |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Interactive 6 |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Kama Sutra |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Select-A-Pet 1 |
General Media |
ICFX |
Win & Mac Programming C++ |
Penthouse Select-A-Pet 2 |
General Media |
ICFX |
Win & Mac Programming C++ |
Shoot Video Like A Pro |
Zelos |
Redhill |
Win & Mac Programming C++ |
Wrath of the Gods |
Luminaria |
Luminaria |
Windows Installation |
Critical Path |
Media Vision |
Machaedus |
Consulting on QTW |
Oracle Annual Report |
Oracle |
Oracle |
Installation & Digitizing |
Sony Demo CD Comdex |
Sony |
Sony |
Windows programming C++ |
Peak Performance |
Media Vision |
C-Wave |
Windows Programming C++ |
Forever Growing Garden |
MediaVision |
C-Wave |
Windows fix AMT |
VideoServer Client QTW |
Starlight Netwrks |
Starlight |
Windows C++ code |
VideoServer Client MPEG |
Starlight Netwrks |
Starlight |
Windows C++ code |
Liquid Chromatography |
Spectra Physics |
Spectra Physics |
OS/2 coding in C |
Canyon Clipz |
Canyon |
Canyon |
Win & Mac & Digitizing |
* Incomplete list, does not include internationalized versions.