PURPOSE
The purpose of this document is to provide the reader with a high level understanding of the software product development process. Specifically, the phases that make up the process, the stakeholders/tasks/decisions that are involved at each stage, and the tools/processes used to manage tasks within the stages. One caveat to call out is that this document provides just one generic example of the process, many variations are found, and there is not necessarily one right way. In fact, some of the biggest tech companies avoid anything nearly as process heavy as the workflow described below.
OVERVIEW
The fundamental purpose of the software product development lifecycle is to employ human resources in the task of building software in order to achieve organizational objectives. Aside from the core task of engineers writing code, the lifecycle involves groups of cross-functional stakeholders planning and coordinating to ensure that resources are being deployed on the most important tasks and in the most efficient manner. These planning activities take place at varying levels of granularity and frequency depending on the phase/scope of the development lifecycle, and involve different stakeholders at each stage. A high level simplified overview the process is as follows: 1.) executives decide on long term objectives or product investment themes and allocate headcount (i.e. software engineers) to teams to work on those areas, 2.) product managers & engineering managers from each team decide which features to build using their given resources, and 3.) engineering teams build the prioritized features in discrete chunks over a number of two week cycles called sprints. These stages are discussed in more detail below.
ANNUAL PLAN
In the highest level stage of the development lifecycle, executives are tasked with deciding on the long term (1-3 year) strategic objectives for the organization, and allocating the fixed headcount resources to the relevant teams/organizations within their purview. Typically this is done on an annual basis, but could also be done semi-annually. Teams will each present their recommendations/plans for the upcoming year along with proposed goals, and may include proposals for large new initiatives that require funding of incremental resources (i.e. software engineers/product managers/marketing budget etc). Once the executives have reviewed all of the plans and made their decisions about which initiatives/teams to fund, then the goals for the following year are finalized, and the teams may begin hiring any new headcount along with planning how to best achieve their new goals.
PORTFOLIO MANAGEMENT
At the portfolio management stage, a senior manager may manage several discrete teams that each own a product or products in a related space, and then this person is tasked with thinking holistically across the portfolio of these products about how to best achieve the business/organizational goal(s) that they are responsible for delivering. In some ways this is done upfront as an input to the annual planning process, but resources ultimately need to be allocated amongst the various product teams, and prioritization decisions can be made to try and improve the synergy between products in a given portfolio. The portfolio management process will also help product managers prioritize which “epics” should be worked on next (discussed in the next phase). There are also aspects of ongoing management of the portfolio such as monthly/quarterly reviews to monitor progress/track status updates.
PRODUCT/PROGRAM MANAGEMENT
As we move one layer down from portfolio management to product/program management we begin to transition from strategic planning processes into tactical executional processes (but elements of both remain). At this layer, product managers and engineering managers on a given team are responsible for one or several products comprised of numerous features. The decisions/outputs made at this stage are determining what should be prioritized to be worked on by the engineering team during upcoming sprints in the next 1-3 months (i.e. what is going in the “roadmap”), along with a groomed/prioritized backlog of stories to be worked on. At this level, the goals/objectives are known, and the items being prioritized can range in size from large “epics” to small features/stories. These also need to be balanced against ongoing operational work and “tech debt”.
In order to do this, the team needs to have reasonably accurate estimates of the “cost” of implementing a given item, along with the impact/value it will deliver toward the team’s goals. Cost is meant to represent the estimated developer time needed to accomplish the project, and is often expressed in relative terms, for example t-shirt sizing (S, M, L, XL) or Fibonacci numbers (1, 2, 3, 5, 8, 13), but can also come in terms of developer days/weeks/months (or even calendar time, which can vary widely from development time). Once the roadmap is aligned amongst the team from a strategic perspective, the team needs to figure out how to execute against it. Typically the product manager will write what is called a “user story” which describes the desired outcome for a given feature and is written in the format of: “As a user_type I want to functionality_desired so that I can outcome_desired”. This would also include “acceptance criteria” for the story to be considered complete. The engineering team can then review this artifact in a recurring meeting called a “grooming session” and assign a more accurate estimate of implementation time called “story points”, along with breaking into smaller tasks/sub-tasks that can be worked on by a single engineer during a sprint. Groomed stories are then moved into a groomed/prioritized backlog so they can be picked up in a sprint (discussed in next section).
SPRINT EXECUTION
The final stage of the product development lifecycle is the actual execution of the software engineering work to deliver the items planned in the preceding phases. In this phase the engineering managers together with the software engineers (and sometimes PMs) decide what stories/tasks will be worked on in the upcoming sprint in order to best meet the roadmap delivery expectations, and then execute upon them. A sprint is a two week block during which the engineers will only work on tasks decided upon during the sprint planning meeting at the beginning of the sprint (i.e. no unplanned work), and the objective of the sprint is to fully complete all of the tasks (i.e. no carry over). The tasks are chosen from the prioritized backlog, and the number of tasks assigned to the sprint is based on the available capacity of the team expressed in story points. At the end of the sprint there is a sprint retrospective meeting where the team discusses what went well, didn’t go well, needs to be improved, etc, and individuals may share demos of the work that they completed during the past sprint. At this point, value has been delivered to the customer and the sprint cycle repeats itself again and again to burn down the tasks in the backlog, thus delivering upon the larger “epics” and organizational goals.
Appendix A: Visualization of the Traditional/Legacy Agile Workflow
Appendix B: Variation in Methodologies Across Companies
