Friday, May 30, 2014

Release and Code Branching Process

1.     Terminologies

Main Dev Branch:  this is the main development branch. All feature development for user case will be carried out on this branch. In TFS server the version control path for it will like  $/TeamProject/Main.  This branch will be maintained and worked on during the project life cycle.  All other branches are created from the main branch

Release Branch:  this is the branch associated with a specific iteration. (e.g. Iteration 1)  At any point of time, there could be none, one or a few more release branches.  It is to our interest to keep the number of it to minimum. (Not more than 2). The life span of release branches is limited. Normally it is created on the official deployment of the iteration at the end of the iteration or the starting time of the following iteration. It ends at the time when the subsequence iteration is released.  The decision point on if we need to have a branch vs. if we simply create a label on the main code branch is: do we need to fix bugs, defects in the release branch without including the new development effort in the subsequence iteration.  From overall manpower cost of view, you want to touch the release branch as less as possible. However, in the software development life cycle point of view, sometime it is necessary to provide bug fix on the release branch, especially when the system is in production. So the common practices are creating it at the time of release, and try not to touch as much as possible.

Replication: when work has been done on the release branch (e.g. bug fix), it is common expectation that those fixes need to be included in the subsequence release of the main development branch.  At the same time, because the main development branch is also an active branch, it is more than often simple merge from release branch to the main development branch would not work.  The file touched in release branch may be moved or updated in the main development branch; maybe the changes made in release branch are no longer needed in the main development branch.   The solution of the same bug fix in release branch and in main development branch might be different due to the priority and timing difference. It is common practices to provide quick fix in release branch and get the testing or production operation continue and provide more involved design over haul in the main development branch.  However, the bug fix needs to be included in the main development branch. The process to include the bug fix (or any other activities) in the main development branch is called Replication. However, there are also scenarios where simple merging works just fine.

Work item and state of the work item: work item as the name suggest, is an item that need some work. It is similar to WBS in MS project.   Each work item could have various states depends on the methodology the project team practicing. In MS agile process a work item have Active/Open and Closed / Completed states. In other processes more states are included. However the work item template can be customized to provide more sophisticated state transition.

An example of such customized state transition could be: New-> Active-> Resolved -> Reviewed -> Deployed -> Tested -> Closed. Another example could be <New> -> <In Progress> -> <Completed> -> <Reviewed> -> <Deployed> -> <Tested> ->< Closed>

Build Master: is the person or group who is in charge of making the deployment build and publish the build to the target region. In some project, it is called as configuration manager, in some project it is called as Configuration manager. Some call it Build Master. Basic they are the person who pushes the code from version control to the running environment. In some organization, for security reason, this role is isolated from development team.  In .Net platform, Strong name and assembly signing are provided to ensure this separation.

2.     Iteration Releasing


 At the end of each iteration, a build / deployment process will be carried out to deploy the code base with the newly developed features to a targeted region (IT/QA/UAT/CIT).   The following principles are observed by many development projects:

1.       The pre-condition of any release is that the current state of the code base is stable to be built and published. The project manager, development team, and build master need to be coordinated and communicated well to ensure the success of the release and the integrity of the release.

2.       It is advisable to schedule the deployment / release activities during less active period, e.g. after office hour or in the weekend when no or few check-ins are expected.

3.       When that is not possible or when the build process does not take much time.  The release can be scheduled in normal business hour when the development team members are working on the code branch. In this situation, The build master  should  make announcement before the build process started, ( 10 -15 minutes prior the build starts) to ask all development team members to check in whatever they think need to be checked in and hold off what they think should be held off.

4.       When it is the time to start the build, the first action of the build process is “Get Latest”, and start the build and until the build / release process is completed, no non release related check-ins should be expected.

5.       All deployment activities is to take the code source from version control repository (it does not matter what version control tools the project is using), none is from other source. (Not from server to server, not from database to database).

6.       All different components of the system must be in sync in the version control. An example of violation of this principle could be taking one component from one version and taking another component from another)

7.       Specifically, it is important to treat the database schema in the same way other source code is treated.  The database schema needs to be in the same label and same branch where the other source code lives.

Upon the completion of the release / build / deployment,   a label to be created on where the release is made from (for example, the main development branch) . (Not before the build is made, not many labels) and the label should cover all source code involved in making the build including database schema.

After creating the label, the build master shall send another announces to the team to notify the completion of the release and the team can resume checking activities. This is to ensure the integrity of the build. 

 If it is the main development branch or subsequence minor release is expected on it, a code branch is to be created from the branch (e.g. main development branch) based on the label made.

At this point of time, it is advisable to query the TFS work items based on iteration path and state to generate release note and also update the state of the involved work items from Resolved to Deployed. (This can be done manually or automatically by writing some small program using TFS API).  To further automate the process, customized report could be developed using SQL/ Server Report Service by accessing the data store behind the TFS Server)

3.     Minor Release (Intermediate release)


The process of handling intermediate release is very similar to the way iteration release is handled. The only difference is that intermediate does not need to create branch in most of time.  Other than that, labeling and the respective principles are the same.

4.     Labeling vs. Branching 


The purpose of labeling is to enable us to make and deploy the identical build into different regain at different time.  (An example will be you first made a build and deployed to IT region, when all tests are good, the same codebase is approved to be deployed to QA then UAT then production. As one of the version control best practices, you do not want to version control the generated files like .exe or .dll files, but you also want to be able deliver the same codebase at different time. The solution is labeling.  At any point of time, you can pull the code base out from the version control based on the specific label and make a build to produce the same binary files for deployment. 

The purpose of branching is to enable code change on a released branch for a minor release after the iteration is concluded.  If the project does not foresee such need, branch can be omitted.

5.     Development process in release branch


When you are given a task to work on, you want to know for sure, the work is to be done on which branch (if there are more than one branch). If project manages  development activities using TFS work item tracking, you want to make sure the work item you are given or you are about to create is in the correct iteration path. Then based on the iteration path you work on the code changes in its respective code branch.  Iteration path of the work item is very important. One should be very careful about it before making code changes. Even you do not use TFS or use some other tools or even you want to manage your project using excel file. You want to be very sure that iteration is an attribute for all WBSs.

When a bug fix  in the release branch ( iteration 1)  is completed and tested and accepted by the user or client, a related work item (bug) is to be created in the main development iteration ( iteration 2) and assigned it to himself or herself.  And the original work item is to be closed 

6.     Replication process

 

The project manager or project scheduler will make some adjustment on the resource allocation, it is considered more effective that the related work item to be handled by the same person who provided the fix in the release branch, but it can be different due to all kind of considerations.

With this arrangement, the replication process is been managed in the same way the other development activity. The only difference is the development resource can refer back to the original work items for descriptions, for test cases and for code changes. 

The time frame of the replication process for a specific work item is from the work item be accepted in release branch to the time the subsequence iteration is released.  The project manager / scheduler or individual developer could use some judgment call on case by case basis.

7.     QA process of the replicated big fixes


At the time of the release, the Release Manager or Build Master could update the state of the work items based on iteration path and state from resolved to Deployed. And generate the release note from it.
At the any point of time, QA team can simply query the work items in TFS based on their state, iteration path to manage its testing activities. For example, before release is made, the QA manager can query the work item in a specific iteration to get the scope of the projected release.  He or she also can query the resolved work items in the projected iteration to plan its testing activities.  After the deployment, the work items will be changed to Resolved to Deployed.

8.     Branch retiring


Other than the main development branch, release branches only in active mode for a short period of time.  The maximum of it is the duration of the subsequence iteration or the duration of 2 iterations.
However, up to the time when the subsequence iteration is about to be release, there might be some open bugs filed on the release branch yet to be resolved. In this situation, all open work item shall be migrated to the future iteration and to be handled then. (Simply change the iteration path, this is also can be done programically with TFS API)

When the work item   migration is done, the code branch can be retired. One should publish some kind of notification to inform the team. General recommendation is to keep the retired code branch in version control for tractability. However, they could be purged after certain cold down period defined by the team.

9.     General guidelines related to Release / Branching process


a.       The fewer branches lead to less replication activities. The more branches leads to better capability on provide fixes based on certain release. The project team needs to balance between these 2 conflict goals.   General practices are that for any release that could be deployed to for production operation, you want to create a branch for it. This is to prepare the code base for production support. In another cases, you want to negotiate with your clients as hard as you can for not to provide fixes in release branches. However, there are situations where the exit criteria of the iteration are associated to certain numerical figure of open defects in each category. In these situation , the project manager is motivated to let the team touch on the release branch and get these bugs / defects resolved in order to meet the exit criteria and secure the payment from the client.

b.      For each major release branch the code base based on label. Always label first and branch based on label.

c.       Try not to work on bug fix in the release branch as much as you can. The decision point is : do we need to provide another build for the bug fix on the original code base.  A example could be like this: you completed iteration one and processed to iteration two, then a bug is identified in iteration one deployment.  Your question will be: do you want to see the solution implemented and deployed before iteration two is completed and Do I want to make a build with the bug fix implemented in iteration two and deployed to a region before iteration two is completed.  In most of time, your answer to the second question is No, and the answer you might get from the project management or client for the first question could be yes.   If you get Yes, No answers for these 2 questions you will end up touch the code in release branch, in any other situation you will not.

d.      Pay attention to the priority difference when you work on the work item on release branch and on main development branch. In support branch, you need to get it resolved and resolved fast. You want to leave less fingerprint possible, while in the main development branch, you can take your time to do some design and have larger fingerprints.

e.      In main development branch, it is highly advisable to conduct certain level regression prior to releasing for client facing test or production operation.
f.     As it is replicated individually based on work item, the developer can conduct unit test in reference to the original bug, the QA also can conduct the test in reference to its original test case. The quality of the replication will be improved as comparing to batch replication based on change sets. on the other end, it will take a bit more effort as comparing batch replication based on change set.    

No comments:

Post a Comment