Enterprise Focused Development

This article introduces Enterprise Focused Development methodology, EFD. EFD methodology has also been published in the November 2008 issue the Journal of Enterprise Architecture

The purpose of the methodology is to cut government's software development expenses and create higher quality computer systems.

Enterprise Focused Development uses relational data modeling to focus software development teams in the most effective way possible while building the organization's enterprise architecture.

It is a method of creating computer software systems by modeling the whole system, then building the code in very small increments and having the client test the small increments before making additions to the system. This makes the client an involved partner throughout system design and implementation.

Enterprise Focused Development's flexible method contrasts sharply with conventional methods, such as waterfall methodology, where clients' specs are obtained in the early phase of system design, then designers and programmers follow rigidly planned building steps until the system is completed and presented to the client for testing as a finished product.

The power of Enterprise Focused Development's method stems from its ability to handle change as it happens. Change is inevitable and comes in many forms:

• Clients often envision better design ideas after they see actual working components in action. If they are unable to see parts of the system in action early on, and influence design on an ongoing basis, then many improvements are permanently lost.
• Legislation mandates change.
• The data environment changes such as when related systems are introduced or undergo changes.

Enterprise Focused Development resolves change related problems by addressing them as they occur.

Here's a typical example. Suppose the creation of an appointment scheduling system for a governor's office is requested. During initial interviews it is determined that the system should work with time units that are a minimum of 15 minutes. With Enterprise Focused Development, a simple menu would be available to the governor's staff within a few weeks just for adding a couple of test records. Nothing else has been coded or committed to, just rough data entry programming code for adding meetings. During input of actual data the governor's staff realizes that 15 minutes is too large of a time frame and requests that the minimum unit be reduced to 1 minute.

Analyze the difference in the way the methodologies handle change. With Enterprise Focused Development, the correction is made early. With conventional methods, the problem is not caught until the whole project is completed and handed to the client for testing months or years later, but now the error is embedded in thousands of places throughout system code and data files.

Multiply this example with many other problems and oversights in a very large project and you can see the vast advantages of Enterprise Focused Development.

If this was a big IT project, there could be several scenarios with conventional methodology:

• Best case scenario: The change regarding 15 to 1 minute units goes to a change control board and is approved and problem is repaired, raising the cost of the system.
• Bad case scenario: The change control board views this as scope creep and denies the change.
• Worst case scenario: Contract lawyers become involved as the client and project manager battle in court over whether the system requirements were met under the contract's definition.

Enterprise Focused Development on the other hand, guarantees client satisfaction because the client is an ongoing partner in the development of the system. Even if you start off in the wrong direction, Enterprise Focused Development corrects the project early on.

Enterprise Focused Development methodology flushes out problems in the system during each step of the process. Problems that get flushed out are:

1. Misunderstandings between client and the project team
2. Missed requirements
3. Incorrect requirements
4. Design weaknesses
5. Incorrectly configured hardware and system environment.

Conventional methodologies accrue problems until the first finished version of the project is handed to the client for testing. The complexity of the problems can overwhelm project managers who then fulfill project management's first rule: most projects fail.

Enterprise Focused Development works best for business systems where prototyping software is easy, and is least effective where blueprints must be rigidly followed such as when building a submarine.

I named the methodology Enterprise Focused Development because a data modeler uses Enterprise Architecture-mindful table design to focus the whole project's development effort.

Enterprise Focused Development has some similarities to Agile software development methodology.

I replace the traditional Agile concept of self-organizing teams with data modeler-organized teams. This puts the development team in sync with:

1. Its members when the project is very large
2. Project-wide integration
3. Enterprise-wide data integration/Enterprise Architecture
4. The organization's mission.

Step by step guide to using Enterprise Focused Development

Critical to the success of the project are (a) strict normalization of the data as it changes and (b) having the data modeler be the project designer, requirements gatherer and project leader.

The data modeler must be the requirements gatherer for these reasons:

• The data modeler can visualize data connections with obscure regions of the organization, or other organizations, down to the field level, possibly eliminating whole processes that the conventional project manager or requirements gatherer cannot imagine or mistakenly believes are requirements. Questions to clients can then include connectivity and enterprise architecture related areas early on instead of as an afterthought. This saves the organization from veering off into silo territory and strengthens the enterprise architecture. A data modeler can ask questions that reveal subtle requirements that are difficult for non-modelers to imagine.
• The data modeler can see SOA opportunities for internal use or for external organizations in the current project that other disciplines cannot imagine and begin researching this area so that potential stakeholders receive an early heads up regarding collaboration potential.
• The data modeler is less likely to confuse a process with a true requirement.
• The data modeler is more capable of envisioning ways to re-engineer a process through their understanding of the data and ask opportunity-creating questions during requirements gathering. To receive requirements secondhand would be a loss of this opportunity, increase the time and cost of the project and make project impact analysis superficial, missing deeper effects on the organization's operation.

A data modeler must be the project leader because controlling the project via tables is the key to keeping thousands of components in sync with each other and the project's mission. This is especially valuable in controlling code building by large programmer teams in huge projects. The same is true for the system designer. If instead of a data modeler, a process oriented project designer was used, serious weakness would be introduced into project design because complex or changing requirements cannot be addressed rigorously with a process viewpoint.

Roadmap to a successful project:

1. Gather all of the requirements. Be prepared to flexibly modify requirements throughout the process, even towards the end of the project.
2. Model the data. Be prepared to flexibly re-model data on a daily basis. The whole system from beginning to end should be modeled. This is only the first iteration of modeling. No program code has been written yet but the complete system workings are revealed by the data design. Data tables document requirements better than paper format, however business rules notes should accompany each table's design where clarity is needed. Enterprise Architecture and SOA should be designed in during this phase and into every iteration as described here. As each table is created it should be:
   
   1. registered in a database defining its sharable potential such as Web Services or simple file exchange potential
      
   2. evaluated for security considerations
      
   3. evaluated for use in BI, including executive and legislative summaries
      
   4. evaluated for possible metrics use and loaded to a metrics registry
   5. a complete checklist is located here: http://gov-ideas.com/opportunities-checklist-for-new-software-projects.htm
      
3. Create user interface (menu screen) based upon client's requirements and ghost out everything except for data entry, find, view, edit and delete options for the very first table only. It is not necessary to make the menu extremely detailed on the first day. Start with a simple rough overview.
4. Build programming code for the first find, view, add, delete and edit screens for the first table
5. Ask the client to add the first few records and start providing feedback
6. Add another feature based upon client feedback. Each time a feature is added, repeat the above process. The more frequent the iterations, daily is the best, the more efficiently will the project proceed. Make sure to personally visit or get the client feedback process fine-tuned daily so that a comfortable working relationship exists between the modeler and the client. Keep the daily meetings short. Now client trust has been built by (1) visible quick progress (2) the establishment of a repetitive communication pattern with the modeler. The goal is to bring client interaction with the modeler to where it is a daily habit and to encourage frank communications on complex issues. At mid point, clients see so many new features and unexpected capabilities created by relational table flexibility that their enthusiasm builds into strong project acceptance.
7. The menu layout and data schema need to change quickly, daily, to perfectly capture the client's goals, which will evolve during the process in a manner most likely to make the project succeed. Never vary from strict third normal form. This will allow the whole system, no matter how large, to turn on a dime and align itself to the client's ongoing requirements changes throughout the whole project. This is the heart of Enterprise Focused Development. Continuous data re-modeling is the agile component of this methodology. Should anyone besides a data modeler be in charge of the project, agile table modeling that aligns the project to improving requirements would be lost during bureaucratic delays, draining the lifeblood of the project's success and adding expenses tied to delays in correcting requirements errors. A data modeler agilely tuning the project to requirements is the core process responsible for Enterprise Focused Development's exponential cost savings. This process creates a steady rhythm of feature completions that builds client trust and support for the project.
8. Programmers, especially contractors, should not be allowed to create tables (or arrays, which should be banned from all projects) but they should be encouraged to discuss and recommend table design to the data modeler who will correctly design them to fit into the enterprise-wide picture. In essence, the data modeler is directing the project through the table design and programmers, DBAs, server teams, webmasters, etc. are filling in the details. This personnel organization is less dependent upon highly skilled programmers since the database design drives the project towards robust stability.
9. After the project is put into production, stay with the client for monitoring purposes and remain prepared to change system design radically to accommodate improvements arising from the success of Enterprise Focused Development in stimulating deeper levels of insight. It is these last minute changes that are often responsible for making the system the longest lasting success.
10. Other than the above concepts, Enterprise Focused Development projects should use traditional PMBOK methodology. Some PMBOK processes become simplified or change with Enterprise Focused Development, but there is good fundamental effectiveness in 99% of PMBOK techniques. Currently, Enterprise Architecture and Enterprise Focused Development (or other iterative methodologies) are missing from PMBOK and they should be added to it. Instead of a work breakdown structure (WBS), a "feature breakdown structure" should work better with Enterprise Focused Development because it measures progress better and makes it easier for the client to communicate about the project.

Progress using Enterprise Focused Development is very fast, while at the same time client satisfaction is ensured. The client is never out of touch with the project.

The larger and more complex the project is, the more likely Enterprise Focused Development is to out perform conventional methodologies or Agile.

Risks:

• Not having an experienced, effective data modeler that can translate business requirements into data tables
• Culture change barriers on the part of programmers and their managers
• Management interference

Real life example of Enterprise Focused Development implementation

In 1992 I was asked to write a personnel system for a state agency. Since this should have been a centralized system, I recommended that they use the state's centralized system, but in those days, the centralized system seemed to be the most disliked app in the state. I contacted many other state agencies hoping that they could share theirs with me. The other agencies intensely disliked the centralized system so much that they also tried to write their own apps but the system requirements were so complex that they had to abandon theirs and use the state's centralized system. One agency paid a private company $500,000 to write their system but it also failed due to requirement complexity. During my conversations with the other agencies I saw the same, unhappy attitude about the centralized system and anger about being forced to use it. The strong emotions just poured over the phone line into my ears.

I was given no choice but to proceed, so I interviewed the clients and gathered all documents to obtain the requirements.

Within a few days I completely modeled the data. Not having any modeling tools, I typed the table designs into text files and printed them out onto about 300 small pieces of paper and spread them out onto a large table where I could move them around and document how they worked together. The tables and their relationship to each other represented a fully working model of the system.

Next I built a simple menu screen with only a few working entries: Add An Exam, Edit An Exam, Delete An Exam, Find Exams, View Exams. Ghosted out were main branches of the system menu: Print Applicant Rankings, Schedule Exams, Print Certification Letters, etc. There was no code behind the ghosted out options. This allowed me to freely rearrange the menu items as more efficient menuing configurations came into focus.

I asked the client to add a few test personnel exams. Immediately, it was discovered that what the client described as immutable business rules of how exams were named, were not so immutable. This forced me to change the table design early. At this point I realized how effectively Enterprise Focused Development flushed out problems early in the design phase and saved me from having them compounded at the end of the project.

The menu screen was a perfect communication tool, allowing the client to communicate with me as to how to best fine-tune the requirements, and for me to obtain confirmation that the project was on the right track. Seeing actual menu options greatly facilitated joint analysis of complex processes.

Physically sitting alongside the client allowed me to study minor reactions regarding interface ease of use. This revealed opportunities to make the interface more intuitive or put online documentation regarding difficult issues at locations where future, untrained users would need it the most. The system ended up being so intuitive that only short online instructions were required and no user manual had to be created.

Each iteration was a quick refocusing of the entire project to the continually refined client requirements eventually making the finished system highly likely to satisfy the client

Many false directions were nipped in the bud and many unexpected improvements resulted from bringing the client in as a partner during project development. Repeating the above process for every new feature, the personnel system was vetted every step of the way and completed very quickly. It was a complete success.

How many large projects can you name, have gone into production fulfilling every detailed requirement, leaving the client completely satisfied? If government managers were forced to choose one thing to keep them awake thinking at night, this should be their choice. It is simply a waste of money not to include Enterprise Focused Development in IT development.

Comparison of Waterfall methodology and Enterprise Focused Development performance:

1. Time
(a) Enterprise Focused Development - one Enterprise Focused developer completed the personnel system in two years.
(b) Waterfall - The only other contestant that finished the project was the centralized state department team that had six programmers working on it for twenty years.

2. Scope
(a) Enterprise Focused developer completed a fully automated personnel system.
(b) The centralized state department team system was not fully automated.

3. Budget
(a) Enterprise Focused Development - $50,000 a year for one developer's salary for two years.
(b) Waterfall - difficult to estimate, but at least ten times as much.
(c) Private industry - $500,000, but project failed.

4. Quality
(a) Enterprise Focused Development - A quality project was delivered without a single flaw.
(b) Waterfall - difficult to use.

5. Risk
(a) Enterprise Focused Development - no risk because client tested and approved every new feature daily. Financial expenditure was minimal.
(b) Waterfall - very risky as all projects except one failed, wasting large sums of taxpayer money.

6. Client satisfaction
(a) Enterprise Focused Development - clients were extremely satisfied.
(b) Waterfall - all projects failed, except the centralized state app, which all clients disliked so much, they tried to build their own.

It is a stunning example of how one Enterprise Focused developer out performed many government teams, private industry and a whole state department. It is revealing to have a head-to-head contest where all teams have the exact same requirements.

Criticism of Traditional Agile

I have been using this methodology since the 1980's but did not hear the term "Agile" until 2008 when reading about how it could have saved an expensive IT project from failure. I was interested to see descriptions of others using Agile, and some of the techniques and benefits closely matched my experience. I also found that there were criticisms with some actual Agile implementations that I hope my methods described above prevent. I can't find details of how Agile projects failed, but I suspect that there were scalability problems and that there was a random quality to the programming that is prevented by the method I have described where a master data modeler maintains highly integrated data design, and the programmers merely flesh out the design. The agility in the methodology comes from strict and ongoing data normalization from a centralized source that responds instantly to any changes at any time in a way that most closely and successfully aligns the project to the client's mission. Updated data modeling by the project leader is the control center that keeps the project focused correctly.

Remove waterfall methodology requirements from Requests for Proposals

Government currently issues requests for proposals (RFPs) that require the project be developed using waterfall methodology, the most inefficient way imaginable. Especially for budget authorities, waterfall creates an illusion of control.

RFPs must be written so that requirements are not complete until prototyping with Enterprise Focused Development. Requirements should be delivered multiple times, after feature implementation, each time bringing the updated system into clearer focus. Budget authorities should remove waterfall from their requirements.

If your agency contracts out most IT projects, write the contract so that vendors use Enterprise Focused Development, that is, it includes iterative code builds in the requirements gathering stage so that vendors produce requirements multiple times. Traditional project managers have been trained to refrain from coding until requirements are written into stone, however as described above, this method causes projects to fail. In the case of contracted projects, vendors lose money because by following waterfall methodology, clients end up with software they don't want and will not pay for.

Conclusion

In one of the first definitions of the scientific method, Leonardo da Vinci said But first I shall appeal to experience before proceeding further.

The simple implementation guide in this article is basically all that is needed. For large, complex projects, it simplifies the process down to manageable chunks focused to work together by a data modeler.

Enterprise Focused Development has broad implications for reducing government IT development costs. It should be a best practice, the standard for developing IT business systems.

Do not alter or customize these rules. Leaving out any one of them dramatically increases chances of project failure.