Introduction To Rational Unified Process (RUP)

This post is an introduction to the Rational Unified Process (RUP), a method driving software application implementation. It is an iterative framework, which puts it in the set of Agile methods.

Concepts

• RUP is based on UP (Unified Process) which defines 50 optional artifacts. These are organized within disciplines.
• Each RUP project is customized with a set of practices and artifacts.
• RUP defines 4 phases:
• Inception - Lasts few days, usually no iteration. Includes: Business requirement definition, scope definition, 10% of requirements are described in detail, identification of key risks, estimation of effort requirements, creation of risk list, definition of UML use cases, prototyping.
• Elaboration - Small iterations start: coding and testing of system parts, identification and mitigation of main risks, 80% of requirements are analyzed in detail.
• Construction - System is completed with more iteration and made ready for deployment (integrated and tested baseline). Includes: stakeholder evaluation, little new changes or requirements, performance tuning, documentation.
• Transmission - System is deployed, starting with a release candidate. There may be multiple releases until the final release. Training is performed.
• A Vision is the summary of stakeholder's needs and features.
• A Design Model is a set of objects (not OO) describing how the different parts of the system to be implemented are going to work together.
• The Iteration Plan is the set of task and objectives for the current/next iteration.
• A Risk List with priorities, and mitigation plans.
• The RUP roles are:
• Stakeholders on the customer side.
• On the implementation team: implementer, tester, architect, analyst, database designer, user interface designer.
• The implementation is lead by a project manager, assisted by process engineers.

Practice

• Short iterations 2 to 6 weeks.
• Focus on high-risk and high-value elements first.
• Reuse of existing component.
• Accommodate changes early in the process.
• Deliver value to the customer.
• Team size can be 3 to several hundred.
• Continuous verification of quality.

Conclusion

RUP is well suited-for many types of projects, though the overhead will be higher than other Agile methods for small projects. XP and Scrum are more suited for projects created from scratch or involving many new user requirements or features. RUP is efficient for migration projects, ERP deployments or large scale projects. 

Introduction To Extreme Programming (XP)

This post is an introduction to Extreme Programming (XP), another of the Agile methods for software application implementation. Reading this post is a prerequisite to this post.

The underlying philosophy of Extreme Programming is to take the best elements of software engineering and bring them to an "extreme" level. If a little is good, more will necessarily be better. XP believes that changes are unavoidable in project implementation. Hence, it does not make sense to define a stable set of requirements.

Core Practices & Values

Extreme Programming operates on 12 core principles:

1. Planning Game - It is the planning process happening at the meeting happening at the beginning of an iteration. It includes release planning and iteration planning.
2. Small Frequent Release - The iteration cycle is 1-3 weeks.
3. System Metaphors - A user description of what the system should do. It helps drive the direction of the project and expectations. In the UML paradigm, this would be the set of use cases.
4. Simple Design - This is Occam Razor's principle applied to project implementation. The code should not do more than what satisfies requirements. If something is not needed, do not implement it.
5. Testing - XP favors test driven development, where unit tests are executed frequently to check for any regressions. Users also define acceptation tests, to make sure required functionalities meet expectations. These should be automated and not require human intervention.
6. Frequent Refactoring - Refactoring is either about producing clearer and clearer code, or modifying it to respond to new or revised requirements.
7. Pair Programming - All code is created by two people working at the same working station (one creative mind and one pragmatic mind). It is not about mentoring. Pairs are not fixed and can be reorganized. One purpose is to increase productivity, the other is to spread the knowledge of the implemented project.
8. Team Code Ownership - Everyone is responsible for all the code.
9. Continuous Integration - All developers work on the same code base, to prevent integration issues.
10. Sustainable Pace - It is about avoiding excessive stress which often leads to low quality or buggy code. Overall, speeding up the process too much leads to increased cost in debugging and improving bad quality code. An optimum speed has to be found. XP does not promote overtime.
11. Whole Team Together - The team should operate from the same room. It is not only about the development team, the customer should be on hand at all times and available for questions.
12. Coding Standards - The code that looks the same all over the application is easier to read and to understand.

The 4 values of XP are: communication, simplicity, feedback & courage.

Concepts

• Stories are written by the customer who describes what he does as part of his job. The closest concept in UML would be a use case.
• There are 6 main roles in XP:
• The customer writes stories and acceptance tests. He picks for them for product releases and iterations.
• Programmers write code, tests, refactors code, identifies tasks to be performed and estimates time to perform them.
• Testers help customer write acceptance tests.
• The Coach has the role of a teacher and checks the evolution of the process.
• The Tracker collects metrics, monitors progress.
• Consultant is usually any technical consultant.
• Release Planning Game is the process of defining which features should be part of the next software application release. Which stories should be included? Programmers estimate the difficulty of each task.
• Iteration Planning Game is the process of choosing stories to implement by the next iteration.
• Task lists are the set of task each developer must perform by the next iteration.

Practice

• XP does not promote the creation of an overall design specification document.
• Keep documentation to a minimum.
• Teams should be made of maximum 10 developers.
• Project should not last more than one year.
• Code reviews should be frequent.
• XP projects life cycles are made of several phases:
• Exploration - Prepare enough story cards for the first release, make sure the project is feasible (prototypes).
• Planning - Agree on date and stories for first release, perform release planning game, create more story cards if necessary.
• Iterations to 1st release - Implement tested system, perform iteration planning game, task definitions...
• Productionizing - Operational deployment, documentation, training...
• Maintenance - Enhancement, bug fixing, major releases (incremental or not)... 

Conclusion

Extreme Programming is assuming that the customer is highly available and capable of expressing all his needs. Stories are not as precise as UML cases. Refusing to make detailed analysis when necessary can be counter-productive in many projects. Letting the customer make request changes informally does not help setting a sustainable pace too.

XP maximizes the division of work responsibilities and roles. It is consuming a lot of resources. It tends to focus more on efficiency than effectiveness. Pair programming is also not adapted to everyone's psychology, and making everyone responsible for all the code is overkill in most project. People's time is not always consumed efficiently.

To be efficient, XP requires an environment having a lot of resources and willing to live on the edge (like Formula 1 racing). It is not adapted to migration projects, or projects implementing complex systems including high security for example. Waterfall or Scrum will be more appropriate.

Introduction To Scrum

This post is an introduction to Scrum, one of the Agile methods to drive software application implementation. Reading this post is a prerequisite to this post.

Concepts

• Scrum projects deliver software application features iteratively.
• Each iteration is called a sprint.
• Scrum projects have 4 stages:
  • Planning - Definition of the vision, budget, and expectations. The first version of the product backlog should containing enough implementation items for the first sprint.
  • Staging - This is the first iteration where the requirements and product backlog created in the planning are refined.
  • Development - It is the set of sprints required to implement the project fully. It ends when the product backlog is empty.
  • Release - The final product is deployed, training is performed, documentation is finalized, etc... The release backlog can be used as the product backlog for the next release of the product.
• There are 4 main roles in scrum projects: team member, product owner, stakeholders and scrum master:
• A product owner is a customer representative acting as a single point of contact on the customer side.
• A stakeholder is anyone having a vested interest in the project, providing information about requirements and participating to the decision process of features to be implemented.
• A scrum master is a facilitator and single point of contact on the development team side. It is a support function, not a 'controlling chief' role. It is a mix between a team leader and a software engineering role, which does not include people management or profit & lost responsibilities.
• A team member is a software engineer.
• Scrum of scrum can be implemented when multiple scrum teams are working on the same large project. Scrum master and product representatives get together.
• The product backlog is a list of features, use cases, enhancement, defect, etc... to be implemented in the project's forthcoming sprints.
• The release backlog is the list of features, uses cases, enhancement, defect, etc... which are postponed to the next version of the project (not next sprint).

Practice

• Spring teams typically do not have more than 7 members.
• A sprint duration is 30 days.
• Scrum projects are client-driven. They select the features to be implemented.
• Each sprint begins with two meetings:
• The stakeholder meeting, with the scrum master and customer representative to re-prioritize the product backlog and update the release backlog.
• The product owner and team meeting where tasks are created from the product backlog.
• Each task must require between 4 and 16 hours of work. Bigger tasks must be subdivided into smaller tasks.
• The scrum master and product owner check whether there are enough resources to support the efforts required to achieve the sprint and adjust the workload accordingly.
• Team member pick their tasks and work on their implementations.
• Each spring finishes with a presentation of implemented features to the stakeholders.
• Every day, a stand-up meeting (15-20 mins) with the team members and eventually the product owner is organized in front of a white board. The following is discussed:
• Which tasks have been achieved since the last meeting?
• Are there new tasks? New requirements?
• What is blocking? Any impediments?
• If decisions have to be taken, the Scrum Master should take them quickly, within an hour if possible
• The Scrum Master should deal quickly with any impediments and communication issues
• Scrum teams should preferably operate from the same room

Conclusion

As for all Agile methods, the Scrum approach is best suited for new projects created from scratch. Over time, when successive application releases are implemented, the project can slowly transform into a continuous integration or even maintenance project, where less stakeholders input or daily supervision is required. The cogs are well oiled and operate naturally.

Introduction To Agile Principles

This post is a reminder about Agile principles. It is also an introduction for those who want to learn about it. The Agile principles were initially proposed in 2001 in the Manifesto for Agile Software Development which defines 4 values and 12 principles.

Several project implementations methods were already available then, but many found these inflexible, cumbersome or unadapted. Some tried to apply them with too much orthodoxy and rigidity, went far beyond planned budgets and did not manage to produce operational software. Those who did deliver a product often saw end users refusing to use it. The end result was far from their real needs and expectations.

Agile is a reaction to the think & plan first → design everything → implement everything → test everything → deliver everything mentality (waterfall). Secluding each step of the software process development proved to be a failure in many cases, especially when developing new software from scratch.

By focusing on iterative development, (i.e., successive deliveries of software parts and functionalities with frequent customer feedback), Agile is aiming at being rapid and flexible in response to changes. Project are kept small to maximize success rates. Iterative methods were existing long before Agile. Agile is extending the concept and principles.

Agile Manifesto Values

• Individuals & Interactions over processes and tools - When reading maps, some people tend to try to confirm that they are indeed where they think they are on the map, rather than taking indications on the ground and use these to find their position on the map. The map should only be a support to human interactions, not the other way round.
• Working software over comprehensive documentation - Excessive planning and design used to plague many projects. The process was too intellectualized and disconnected from the final product and user needs.
• Customer collaboration over contract negotiation - Customers know best about their needs (what the product should do), not the designer and software engineer. By getting the customer engaged, one also gets their approval on the delivered product.
• Responding to change over following a plan - Setting rigid project milestones, with a predefined set of tasks, fails on many occasions. The customer rarely has a fully detailed picture of the product he wants or needs. Design and gathering of needs is an ongoing process.

Agile Manifesto Principles

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software - Customer likes to see where the money is going. The more they see something substantial, the more they trust the process, and the more they get engaged.
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage - Devil is in the details, and sometimes, these lie under several layers of analysis and design. For new software, some real practical requirements are only discovered late in the process. These are essential and must be accommodated for.
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale - This practice guarantees that expectations are met and that the project does not go in the wrong direction. Frequent customer feedback is the lighthouse.
4. Business people and developers must work together daily throughout the project - Developers usually have little managerial decision power, yet their daily work depends on managerial decisions which must be made for the project to flow.
5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done - This is about empowerment and people management. It is not specific to Agile only.
6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation - Self-explanatory.
7. Working software is the primary measure of progress - Again, this principle helps avoiding never-ending projects.
8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely - This is achieved with face-to-face communication and frequent deliveries of product features, plus frequent customer feedback.
9. Continuous attention to technical excellence and good design enhances agility - This is a tautology.
10. Simplicity--the art of maximizing the amount of work not done--is essential - It is complicated to keep things simple, to resist the temptation of including too many technologies or frameworks. Overkill is common in software implementation. The purpose is to satisfy requirements, not optimizing the last lemon drop. The investment is often not worth the return.
11. The best architectures, requirements, and designs emerge from self-organizing teams - Again, this is about resisting implementing large strategic plans or theories, and trust the process and people.
12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly - This nothing but debriefing in action.

Implications

By following Agile principles and values:

• Implementation teams practice iterative development, by collecting customer needs, implementing them progressively, with frequent releases (a.k.a. incremental delivery)
• This implies some kind of evolution of user requirements over time
• Programming and testing starts early, alongside analysis
• At each iteration, the client chooses the next features to implement
• Each team member takes ownership of the tasks they will achieved and decide how to implement them until the next iteration
• The riskier and most complex requirements are dealt with first
• Each iteration has a fixed duration (1-4 weeks), which cannot be changed
• Once features have been selected for an iteration, no changes are allowed
• Complete features before moving to the next one
• All this helps reducing the cone of uncertainty of requirements (Mc Connell)

Key To Success

• Identify what the customer values
• Identify stakeholders and make sure they have a vested interest in the project's success
• Make sure there is a customer representative (single point of contact)
• Favor consensus in team decisions, whenever possible
• Break large tasks into smaller tasks
• Focus on delivery, not process compliance
• Master technical details and skills required to succeed
• Facilitate communication channel between all involved parties
• Have people put their ego in their pocket
• Practice open  information as much as possible and necessary
• Use feedback, both from customer and development team to improve
• Avoid long term planning

Conclusion

Agile methods are not silver bullets for all situations. They are best when developing a new software application from scratch or for continuous integration projects. However, waterfall remains more appropriate when upgrading an ERP or when converting an existing application from one technology to another.

The most famous Agile methods applying Agile principles and values are Scrum and Extreme Programming (XP). There is also Adapative Software Development, Crystal methods, Dynamic Solutions Delivery Model (DSDM), Feature-Driven Developpment (FDD), Lean Development and Pragmatic Programming.