Software Review

Since I am involved in the software review & guideline team at my work, I've spent sometime to study about review process, which I want to share with you in this blog.

The benefits of software review:
• Increase software quality, reduce bugs.
• Opportunities to learn (for both the code authors and the reviewers), as a mean for knowledge transfer to junior developers.
• To foster communication between developers.
• Various study showed that review process save costs (e.g. $21 million reported by HP). It's cheaper to fix the bugs in the earlier phases (design, development) than in the later phases (QA/test phase, shipped products)
• As a part of best practices/standard e.g. PSP, CMMI3.
• Motivate the developers to improve their code quality in order to avoid "bad scores" during review. This ego-effect still works even when the random-review covers only 30% of the total codes

The disadvantages of review:
• Cost time. Solution: limit time (e.g. max 1-2 hours)
• Developers have to wait for the reviewers, might create delay in the pipeline. Solution: the project manager has to include software review process in the plan including the time & resources (the reviewers for review, the developers for rework).
• The code author feels hurt when someone else points their mistakes. Solutions: be sensitive/friendly when discussing the findings, both the reviewers & authors agree on the positive benefits, the reviewers give also positive feedbacks to the authors, focus on the codes not the authors.
• The developers think that they have better things to do. Solution: support from the management (e.g. enforce the review process formally).

---

An example of a review process, which is consisting of 3 steps:
1. "Over the shoulder" short session (30min)
The author guide the reviewer through the code: the entry point, most important classes and the relationships between them. He also explains the flow, sequence & concurrency mechanism and the patterns/algorithms used. This session is similar to a pair programming session. However we need to aware of the disadvantages of this method:
• the author has too much control with the scope & pace of the review.
• The reviewer has barely time to check properly.
• The reviewer tends to condone the mistakes after hearing the authors explanations.
That's why we need to keep this session short and follow this session with a private-review session.

2. Private-review (30-90min)
Without involvement of the author, the reviewers check out the code from SCM (e.g. svn), check some documentations (usecase, specs, design), do fast sanity check, perform some test/validation (e.g. soapui), check against checklists & specifications, and read some parts of the codes.

3. Past review activities:
• The reviewer discussing the findings with the author (30 min)
• consult with the product owner, architect, team-lead, project-manager regarding the risks, bug priorities & reworking impact for the plan
• create bug tickets in the trac/bugzilla
• make an appointment for follow up

---

Some best practices:

Determine the scope / part of the project for review, based on: risk analysis and author's error log, e.g. Based on his personal log, Bob (the author) knew that he often made mistakes with web security, so he advised Alice (the reviewer) to concentrate to the security issues of his web codes.

To improve the process, you need to define metrics in order to measure the effect of the changes. These metrics can be external (e.g. #bugs reported by QA team, #customer tickets) or internal (e.g. #bugs found, time spent, loc, defect density, complexity measure). Based on the #bugs found by 2 reviewers, you can guess the estimated total bugs and review yield.

Checklist is the most efficient tool for reviewer. The checklist should be short (less than one A4) and describe the reasons, risk level (based on damage/impact, popularity, simplicity to perform), references for further information.

Perform self-review using personal checklists (since everybody has a unique tendency for certain mistakes).

Take advantage of automatic tests (e.g. checkstyle, findbugs, pmd in java). Some companies include these tests in their continuous integration test and their metrics then can be showed in a graph to highlight the quality trend (e.g. bug reduction vs sprint cycles).

Review meeting is not the most effective way to review. It costs more man-hours. It's better for the reviewer to be alone concentrating reading the code.

Maintain reviewer's concentration by limiting the time for each review (max 1.5 hour, 400 loc (line of code)/review). Slowing down to read the code carefully (max 500 loc/review).

Having the code authors annotate the code appropriately (e.g. the patterns/algorithms used: visitor pattern, quick sort, etc.).

The code authors provide notes for his/her project: starting point/files to begin, important classes, dependency/class diagram, patterns used, where to find the documentations (use case, specs, design doc, installation guide), test cases (e.g. web service request-response examples). This information is useful not only for the reviewers but also in case the author leaves the company (e.g. a temporary external consultant). You can use a trac/wiki as a collection point for the information.

Verify/follow up if the bugs is really fixed.

Beware not to compare orange with apple: the #bugs found in a code is not only depend on the developer expertise but also:
• how complex the problem is
• how many reviewers, time spent, their expertises
• specification & code maturity (development version, beta version, shipped product)
• programming languages
• tools (IDE, validator, etc)

Review-team lead / process owner responsibilities:
• maintain expert knowledge of the reviewers, arrange trainings if necessary
• establish and enforce review policies
• lead the writing and implementation of the review process and action plans
• define the metrics, make sure that they're collected and used
• monitor the review practices and evaluate their effectiveness

Process assets:
• process description
• guidelines & checklists
• issues tracking system e.g. trac/bugzilla

Where to do the review in a typical software process:

To be continued: http://soa-java.blogspot.nl/2012/09/the-review-process.html


Source: Steve's blogs http://soa-java.blogspot.com/

Any comments are welcome :)

---

Literatures:

11 Best Practices for Peer Code Review
http://support.smartbear.com/resources/cc/11_Best_Practices_for_Peer_Code_Review.pdf

Best Kept Secrets of Peer Code Review by Jason Cohen
Plus: recently new, supported by scientific analysis of literature & field studies, down to earth advices (instead of management jargons high in the clouds). minus: repetitive advertisements for their review-tools.


Peer Reviews in Software: A Practical Guide
by Karl Wiegers (Paperback)



Seven Truths About Peer Reviews by Karl E. Wiegers
http://www.processimpact.com/articles/seven_truths.html

OWASP code review guide
https://www.owasp.org/images/2/2e/OWASP_Code_Review_Guide-V1_1.pdf

Agile estimation & planning

You might encounter several weird buzzwords when you're reading about agile estimation methods. This blog tries to give down to earth discussions to help understanding.

Planning Poker
A method to estimate work amounts, which inherits some similarities to the poker game.
The steps:
1. every team member gives an estimation for each user stories (e.g. 8 man-hours for the "customer registration GUI" story), but the number is hidden until all the members have given their estimations. You can use a real set of poker cards to make the process more fun, each team member chooses a card number then puts it upside down to hide the number.
2. when everybody has given his/her estimate, discuss the numbers (e.g. Jip believes that building the GUI will cost 6 points using Struts, Janneke gives 2 points instead using simple PHP, the team will discuss the pros/cons of each method/estimation)
3. repeat the process until the numbers converge (the team members estimates become closer & closer to each other)

Variations:
• online instead of face2face planning meeting, e.g. using http://planningpoker.com/ , but I think face2face will be better & more fun than an online one, but perhaps not always handy for distributed teams.
• publish the user stories and let people outside the team to estimate. e.g. All the developers in the Jip & Janneke team has no experiences with SOA. Annie, a colleague in another team, has extensive experiences with SOA, so Annie can help to estimate the SOA user story. Beware that as the estimation comes from outside the team, it might not reflect the velocity of the team, but at least it can be a starting point for further discussion/estimation.


Tabletop estimation
Another estimation methods by arranging the user stories on a tabletop.
The steps:
1. write all the user stories in small cards
2. compare and arrange the positions of the cards based on their magnitudes (e.g. the story "buying an ice cream " is simpler than "arrange a marriage" so we put "buying ice cream" on the left)
3. Discuss the orders and change if necessary.
4. assign the numbers for each of the stories
5. discuss the numbers.


Simple Release planning
The numbers of iterations = total story points / velocity
The numbers of remaining iterations = remaining story points / velocity
There are many ways to define the velocity, e.g. the average the N last velocity or pessimistic (the average the N worst velocity).

Slacks/buffers
A slack is extra time that you put in your schedule. It might help you to save your deadline when you're overrun with schedule, otherwise you can use the slack time for:
• refactoring / paying technical debt
• learning a new framework / research time
Buffers basically serve the same idea, you can use features buffers and/or schedule buffers.

Fibonacci numbers
When coosing an estimate number, Mike Cohn proposed to use Fibonacci numbers (i.e. each number is the sum of its previous two) e.g. 0,1,2,3,5,8,13, 21,34... instead of just a simpler integers (e.g. ...,4,5,6,7,8,...). The reason is that the bigger the order of magnitude the bigger also the uncertainty (i.e. the gap between the numbers). Personally I prefer to use just simple integers.


Incorporating risks to your planning
The risks will prolong the remaining number of iterations above:
The numbers of remaining iterations = ((remaining story points / velocity)/ risk multiplier)- total risk exposure.

So there are 2 risk factors that will prolong the project: the total risk exposure (due to the project-specific risks e.g.some developers may sick/leave) & the risk multiplier (due to general risks e.g. how stable the development process in this company):

The delay due due to the project-specific risks is quantified by:
The Total risk exposure = sum_i (probability_risk_i * cost_risk_i). For example we have 2 risks: Bumba the GUI engineer who is often sick (probability 0.4 to cost the project 0.5 iterations more), and the Circus server thats need an update this month (probability 0.9, will cost the project 1 iteration more). So the total risk exposure = 0.4*0.5 + 0.9*1 = 1.1 iteration.

The velocity reduction due to general risk is quantified by a risk multiplier table:

As we see in this table the general risk is expressed in probability, thus the resulting remaining iterations estimation is not a (deterministic) point but a (probability) bar. So in a risky process (e.g. the Circus devteam just moved to the SOA environment for the first time), their velocity will be affected by 2x slower in the 0.5 probability case and 4x slower for 0.9 probability case.

---

How's about your experiences with planning/estimation methods? Which works and which doesn't? Why? Please share in the comment.

Source: Steve's blog http://soa-java.blogspot.com

References:
Agile Estimating and Planning


The Art of Agile Development


Agile Software Requirements

Why planning based on features instead of activities

The conventional way when you make a Gantt charts for planning is by typing the list of activities (which are derived from your statement of work/SOW) in your Work Breakdown Structure (WBS). However, in Scrum/agile, we are recommended to setup our plan based on the software features (e.g. user stories, requirements) instead of a list of task activities.

The reasons, some problems may arise by focusing in the activities instead of features during planning:
So you focus on your development activities (which may have less values to customer) instead of customer values.

Your focus is on finishing activities instead of delivering features which the users need. When the project is behind schedule, people tend to crash by reducing the works to do. If you focus on the activities, you will reduce the activities which are prioritized for the convenience of the development team. In this process you may dropping features and some of the dropped may have greater values than those are delivered.

When you check the completeness of the works, you will check the completeness of the activities instead of the requirements of the product (which are more relevant to the users.)

What are you experiences with this issue? Please write your comments.

---

Source: Steve's blog http://soa-java.blogspot.com

References:
Agile Estimating and Planning

Choosing open source / free Scrum tools

I would like to have a free Scrum tool which have these features for planning & progress reporting:
• virtual task board (to show which user stories in the current sprint / sprint plan, release plan)
• burndown chart
• backlog (prioritized list of user stories)
• velocity chart
• Gantt chart / iteration timeline

I found at least 2 tools that fulfill those criteria:
• Icescrum http://www.icescrum.org/
• Agilo http://www.agile42.com/cms/pages/agilo/
Agilo has a better documentation than Icescrum and also offers integration with trac & svn, but you need to pay for Pro license for using the virtual board (& many other handy functions), about 800 euros for 10 users/year. So finally I decide to use the Icescrum instead. Icescrum is just a web application (a war file) that you can install for example in Tomcat. You can testdrive the icescrum here: http://www.icescrum.org/demo/

Some screenshots from http://www.icescrum.org/:

Burndown chart & activities


Virtual task board: release plan /sprint plan


Which agile/scrum tools/project management tools do you use? What are the advantages/disadvantages of those tools? Please write your comments.

---

Source: Steve's blog http://soa-java.blogspot.com

References:

Comparing Open Source Agile Project Management Tools by Brad Swanson
http://olex.openlogic.com/wazi/2009/comparing-open-source-agile-project-management-tools/

Scrum and XP from the Trenches by Henrik Kniberg
www.crisp.se/henrik.kniberg/ScrumAndXpFromTheTrenches.pdf

Agile Estimating and Planning

Design document: to write or not to write

One of the tenets from agile manifesto says "Working software over comprehensive documentation". Unfortunately many lazy (agile) developers use this manifesto to excuse that "we don't have to write documentations at all". Some developers claim that they are agile when they follow the no-documentation principle without practicing other agile principles (such as early & frequently deliveries.)

I believe that writing simple documents (such as use-case, requirements list and architecture design) is still beneficial yet in the agile context.

There are several benefits of a written design document (whether as up-front design or during the iterations):
• you can distribute it to many people (also across the continent) to get feedbacks, so you don't have to explain again & again orally.
• To get feedbacks in the early phase while you defining architecture is important. Despite using agile process, you can't denied that the cost of changing your initial architecture decisions will get more and more expensive as you proceed in the development phase. The cost of repairing a wrong decision in the early phase (architecture) is expensive. You can get better feedback by writing the requirements & the architecture clearly in documents.
• if you don't write you can easily overlook/forgetting important details (beware: the devil is in the details), other people can review the list in your documents and add/comment.
• You can postpone some risky decisions and delay the design which most likely to change, but at least you need to mention the indecisive issues, the risks and the assumptions in your design documents, it will serve like a checklist that you/others can refer later to discuss.
• If you leave the company/project or if you delegate the project to others, the next person doesn't have to start from scratch. It will be easier also for the next people who have to extend the finished product, so they understand the reasons behind you decisions.

Please read my blog about "Software architecture design document"
http://soa-java.blogspot.com/2011/06/software-architecture-design-document.html

How about your experiences with trade-off between the rigorous (RUP) documentations & agile simplicity in your organization? Please write your comments.

---

Source: Steve's blog http://soa-java.blogspot.com

Reference:
Agile Manifesto