Tracer bullet software development

The principle behind its name:
by tracing your bullet, you can see where it's going so that you can adjust your aim to the target better.

The steps:
• define the highlevel subsystem objects (e.g. UI client, database access layer), by all the developers in the team instead of just an architect
• the developers define the interfaces of these objects & how they communicate (e.g. via webservice)
• implement the interfaces with mock objects, integrate early (proof of concepts how the subsystem communicate)
• implement tests with user's scenarios & canned data
• implement the functional code (start with the hardest problems/new technology first), only accept working code (which not breaking the test)
• refactor & refine

SOA & spring lend themselves to this method. In the wsdl-first SOA development you inherently start with defining the interfaces (via wsdl contracts). In Spring you can start with defining the interfaces and later bring in the implementations using dependency injections.

The benefits:
• teams/developers can work in parallel
• the whole teams/developers understand the architecture
• promote communications between teams/developers
• doesn't waste time with unproven low level designs
• you can give demos to the customer earlier to get earlier feedbacks
• the application management (technisch applicatiebeheer) can test the integration between subsystems earlier thus reducing the risk that the project will be late in the product acceptation/deployment phase or that the developer need to redesign and reimplement parts of the systems.
• the QA team can test the performance & security earlier for earlier feedbacks

Source: Steve's blogs http://soa-java.blogspot.com/

Any comments are welcome :)

---

Source: Steve's blog http://soa-java.blogspot.com

References:
The Pragmatic Programmer


Ship It!

Software (architecture) design document (technische ontwerp)

Source: Steve's blogs http://soa-java.blogspot.com/

Why writing design documents:
• to communicate design decisions and why
• to document the benefits & risks of your design
• to serve as a written contract (between you, your manager, your team, product manager, client / product manager): to limit changes, #works, risks.
• to provide common terminologies
• to facilitate peer review / feedbacks from shareholders, to minimize unexpected risks by addressing them before implementing the code

---

An exhaustive table of contents:

1. doc purpose, terminology, reference, distribution list, version,

2. high level summary: purpose/problem/why, who will use, gap analysis (condition now vs advice/solution)

3. scope: benefits, assumption, risks/issues, relation with other projects/dependency, standards
4. requirements:
o use cases:
 actors
 trigger
 preconsitions
 postconditions
 priority
 level: user-goal/sub-function/summary
 frequency
 type: interactive/batch/interface
 flow (basic flow, alternative flow, error handling flow): what the user does & system response, not how/why
 data dictionary (e.g. what the user enters in the UI):
o field name
o type (input,output)
o required y/n
o format: Numeric, texts, y/n, enums
o validation
 special requests (e.g. browsers & resolutions, availability)
 storyboard: to show user expectation of system behaviour e.g. GUI information presented to the user/entered by users, actions/requests which users can perform, screenshots
o other (non-functional) requirements (e.g. performance, legal, licence, security, tools, standards, compatibility with legacy system, 3rd party, OS/environment)

5. high level design/ system architecture (4N+1+more):
o logical view: list of main elements: roles/responsibilities/interactions, architecture diagram, organization (subsystems, layers), frameworks
o design constraints: application type (e.g. web apps, webservice), architecture style (e.g. layered, domain driven, soa), technologies (e.g. languages/framework, database vendor, OS) , compatibility, dependency, corporate policies, standards
o design trade offs/rationale / use-case view / traceability, for example:





o implementation view: artifacts/executables, module/package structure
o process view (concurrency, synchronization)
o (human) business process: (can refer to use-case)
 forms: sample forms, handling
 procedures: trigger/conditions, handling-steps/order/process-diagram, data needed, business rules, expected results, time limit, error handling
o quality attributes:
 security concerns: authentications/authorization mechanism, encryption, password (min strength, expiration), database/file-system access level (read only, write)
 performance (e.g. response time, load/throughput): goal (e.g. response 5 sec with 100 sessions), degradation mode (e.g. response 5-10sec with 150 sessions), measure, correction action (e.g. if timeout then show a "please try again later" page)
 reliability: transaction/locking, validation, defect rate (e.g. product is accepted when no critical bugs left in the buglist), accuracy, recovery, restart, mtbf (max time before fail), mttr (max time to recover) , error handling, logs, troubleshooting / error code
 usability/user friendliness: resolution, browser, font/color, standardization of GUI components & terms, help/user-manual, max time to complete task, training requirement)
 maintainability/scalability: log, doc, standard, parameterization (e.g. internationalization, changing contents/config)
 availability
 reusability
o crosscutting concerns (e.g. cache, authentication, communication, exception management, log) and how to address this issue (e.g. aspect oriented programming)
o test: risk-level & tests per design aspects/requirements/usecase ,techniques/framework (e.g. selenium GUI test, jmeter stress test)
o deployment view: hardware/networks/software configuration (e.g. database, mds, firewalls, clusters, soa/clouds configuration), compatibility, protocols (e.g. https, soap), deployment-settings, configuration management (e.g. via console/file/centralised-server), installation procedure

6. low level design (e.g. for GUI/presentation layer, business classes, webservices, database layer):
for each subsystems describes:
o role/function
o artifacts (e.g. jar/war/dll), how to be used (e.g. webservice, lib, web application)
o input/output, interface (e.g. webservice or lib)
o constraints: dependency, framework
o class diagram
o sequence diagram, process view (concurrency, synchronization)
o business process diagram
o error handling
o configuration (e.g. hardware/software/version needed, wsdl)
o for UI components: screenshots, screen objects, actions, events, files/classes (templates/jsp/php), resolutions & browsers
o for data objects: how to store/datasource (e.g. database/file), process, data(class) dictionary (type, description, attributes, methods), tabel dictionary (data type, keys/contraints), data model/tabel relationship diagram, accessibility/security
o unit tests

7. planning: development schedule/time, development cost, development organisation (e.g. resources/skills-needed/roles, raportage/meetings), procurement/cost(hardware/software/tools/licence/workplace/training)

---

Tips:
• some items in the content list above are optional and can be removed due to duplications (with similar items within this document or with other documents such as statement of work/SOW, product requirement doc/PRD, plan)
• you can separate the contents to several documents for several target audiences (e.g. use case doc, PRD, software architecture doc, product acceptance plan, test plan, development plan). Personally for small projects (less than 100 thousands euros / 400 man hours) I prefer to write a single document as concise as possible (less than 20 pages) instead of writing 6 separated documents.
• you don't have to be 100% UML compliance, the diagrams are just supplements
• don't try to be perfect in the first iteration, the design document is a living document, keep your document lightweight/easy to update
• provide unique identifier for design elements (e.g. usecase, requirements) so you can refer to
• Suppose as an architect you need to write a design document for your developers, how much details should you put in your design document? The amount of design work and the formality of that work depend on two factors: the expertise of the project’s developers and the difficulty of the project. If the project has expert programmers and is a simple project, the design can be less detailed. But if the project has inexperienced programmers, uses unfamiliar or untested technology or demands high reliability, then a more detailed design approach may be warranted.

Any comments are welcome :)

---

References:
• Software Project Survival Guide by McConnell

• Microsoft Application Architecture Guide

• Applying UML and Patterns by Larman

• http://blog.slickedit.com/2007/05/how-to-write-an-effective-design-document/
• How to Write a Software Design Document by Alissa Crowe-Scott http://www.ehow.com/how_6734245_write-software-design-document.html
• RUP op maat by Dekker

How to write use cases

Source: Steve's blogs http://soa-java.blogspot.com/

How to find use cases:
1. identify the system boundary
2. identify the primary actors
3. identify the goals of the primary actors
4. define use cases to satisfy those goals

Do:
• use UI-free style e.g. the student authenticates himself
• be terse
• uml (use case) diagrams are just supplements/optional
• focus on the users/actors and their goals than tasks
• use a verb as a name

Don't:
• UI specific e.g. the student fills his name in the username-inputbox at the right corner then clicks the OK button next to this inputbox
• describe the internal workings e.g. the system then performs this sql query: select .....

Consider tests:
• boss test: is your boss happy if you do this use case the whole day?
• elementary business process: add business value
• size test: a use case is seldom a single step

Contents:
• actors
• trigger
• preconsitions
• postconditions
• priority
• level: user-goal/sub-function/summary
• frequency
• type: interactive/batch/interface
• flow (basic flow, alternative flow, error handling flow): what the user does & system response, not how/why
• data dictionary (e.g. what the user enters in the UI):
   o field name
   o type (input,output)
   o required y/n
   o format: Numeric, texts, y/n, enums
   o validation
• special requests (e.g. browsers & resolutions, availability)
• storyboard: to show user expectation of system behaviour e.g. GUI information presented to the user/entered by users, actions/requests which users can perform, screenshots

If I forget something please comments :)

Reference:
Applying UML and Patterns by Larman

Using cache in Oracle Service Bus (OSB)

There are at least 2 benefits of caching:
• to reduce the processing/network loads, thus increase the performance & reduce the hardware costs
• to reduce the calls to paid services (e.g. postcode-directory webservice), hence reduce the costs

Simply saying, caching works like a table that you keep for a certain period of time. The table consists of a pair of a key/token and the data. For example, you have to build a proxy for CarPrice Webservice that has 2 input variables: the car type & the year, then you can combine these in a cache token such as cartype_year. The first time you call the OSB proxy with the arguments "BMW-X1" & "2011", the OSB business service will invoke the CarPrice webservice to get the response that will be saved in a cache with a token "BMW-X1_2011". Next time you call the OSB proxy again with the same input arguments, the OSB business service will use the cache token "BMW-X1_2011" to return the same result without invoking the external webservice.

It's easy to setup a cache in the OSB workshop/IDE:
• select your business service
• select the message handling tab
• expand the advanced setting
• check the "result caching" checkbox
• specify the key/token of the data (in xquery expression, you can use xpath to retrieve the values from your SOAP messages)
• specify how long OSB will keep the cache data (you can also query this from your SOAP messages using xquery)



In the OSB console:
• Operations > Configuration > Global Settings > enable result caching


• Project explorer (select your project) > operational settings tab > enable result caching


• You can also change/specify the OSB cache in the console (instead of using the OSB workshop/IDE): Project explorer (select your project) > configuration details tab

P.S. OSB caching makes use of the Oracle Coherence framework. Most of time you don't have to bother with this. When you install weblogic with standard setting, the installer will install & configure the Coherence automatically. However when you use a non standard configuration (e.g. clusters), you may need to configure the coherence appropriately.

The original post: http://soa-java.blogspot.com
Comments are welcome!

Refactoring to Patterns

Please click here to get the presentation that I made years ago about refactoring and design pattern.

The key ideas:

What is Refactoring:
• Improving design by restructuring / clean up code
• Without altering external behavior / adding functionality

When to do refactoring:
• Maintenance / fixing bugs
• To understand the codes before adding functionality (e.g. legacy code, systems with integration problem)

When not to do refactoring:
• when the code is too messy, better to rebuild from scratch
• Close to deadline

Best practices:
• Build regression-tests first
• Small steps
• Reversible (e.g. version control)
• Apply Pareto principle, using profiling to prioritize
• The end result targeting Design patterns

Reasons for Refactoring:
• to increase readibility
• to reduce duplication,
• to increace flexibility,
• to increace modularity,
Which lead to:
• Easier to maintain
• Easier to extend
• Faster software development
• Easier to find & solve bugs


Signs that your code needs refactoring:
• Too large classes
• Duplicated codes
• Too long methods
• Too long parameters
• Comments to explain difficult code
• A class / method that doesn’t do much
• Inappropriate intimacy
• Divergent changes in a class
• Shotgun surgery: changes in many classes


Design patterns:
• proven solutions to commonly occurring problems in software design
• don’t reinvent the wheel
• Provide common terminologies for communication
• Variations on implementations depends on domain, tools (e.g. adapter pattern: multiple inheritance in C++, inheritance & interface/reference in Java)
• Nice way to learn software design: peruse design pattern books if you want to become an architect


Conclusions:
• Refactoring: improving design by restructuring without altering external behavior
• Refactoring can help understanding the codes and bugs fixing
• Some best practices during refactoring are: build test first, small & reversible steps, prioritize (Pareto principle)
• Design Patterns: proven solutions to commonly occurring problems in software design. Design Patterns can provide targets for refactoring.
• In practice: refactoring BB-Volg+ shows improvements of some metrics (e.g. reduce duplication, method length, nested depth, cohesion), & other benefits: easier to extend, reusable.

Furthermore, this presentation also discuss:
• several refactoring techniques e.g. extract class, extract methods, parameter object, explaining object, refactoring conditionals, refactoring database, refactoring css and many other techniques.
• several (enterprise) design patterns e.g. singleton, template/strategy, observer, wrapper, factory, mvc,
• a case study: refactoring in one of my project, which resulting in improvements of some metrics (e.g. reduce duplication, method length, nested depth, cohesion), & other benefits: easier to extend, reusable.

PS. You don't have to implement these design patterns from scratch. For example I use Java frameworks such as Struts (MVC pattern, observer pattern, page controller pattern), Hibernate (an object-relational-mapping that implements active records and many transaction management/concurrency patterns), Velocity (template pattern), and Spring (many patterns... proxy, factory, MVC, template, etc).

Please leave your comments to discuss about refactoring & design patterns.

In future I will write blogs about refactoring and design patterns for SOA, so stay tuned!

---

This presentation is based on my study with these books:

Refactoring
A classic, one of the standard read for software developers.


Patterns of Enterprise Application Architecture
One of the most famous books for J2EE developers


Design Pattern
A seminal literature, one of the standard read for software developer.


Code Complete
Another standard read for software developers.

Learning the basic principles of UI / web design

If you're already savvy with the technical sides of web sites (html, javascript, css, jsp) but you still feel awkward with the design sides, then you are not the only one. I was in this situation when I had to build a web application from scratch and I had only 2 days to learn web design principles. Thanks God finally all the shareholders were happy with the results. The user panels voted my design better than a design from a professional designer.

So I'd like to share with you which books that I read in 2 days to learn web design:

The Non-Designer's Web Book
Discuss the CRAP (Contrast, Repetition, Alignment, Proximity) design principles. These principles apply also for print designs/ desktop publishing, which is also my hobby. Also contains some examples of good & bad designs and practical suggestions (e.g. avoid aliasing in the images, use sans-serif fonts etc).

---

Don't Make Me Think
Discuss usability principles:
• be straightforward as possible (such that the users doesn't have to think)
• users scan the web instead of read
• users don't make optimal choices, they don't read all the information, they just guess and satisfice
• users don't read explanations
• if the users find something works they stick to it (e.g. people used to the menu layout in common Windows application), avoid surprise

Also some design principles:
• clear visual hierarchy/structure
• clearly defined areas (e.g. navigation menu, content etc)
• conventions (e.g. colour scheme)
• make it obvious what's clickable
• minimize noise / minimize items in your web

---

Web Design for Developers
A book from the pragmatic programmer series. For developers who have never built a website. contains choosing colors & fonts, mock design, photoshops, how to apply css.

---

Web Design in a Nutshell
A classic book, which I read years ago. Discusses basic factors which affect your design such as screen resolution, browser, flexible vs fixed-width layout. Also contains references for (x)html, css, image formats, etc.

---

Please leave comments if you know any interesting books to learn web / UI design.

In near future I will write blogs about web design for mobile devices, so stay tuned!

Which free Virtual Machines to use?

VMware Player
• free
• easy to use (don't need to read documentation)
• support both windows & linux as OS hosts
• Even though the name is "player", you can also create new virtual machines with some basic settings (memory size, virtual storage size), to use port forwarding for NAT-virtual networking you need to use a configuration utility which is "hidden" in one the the cab files.

VMware Server
• free
• it seems that VMware stopped maintaining this software so it has compatibility problems with the new OS (e.g. Windows 7, Fedora 14), you can resolve the problems but it costs time.

VMware vSphere ESX
• a newer version of VMware Infrastructure
• hypervisor architecture: faster & you don't have to pay an extra license for the host OS
• bad points: need a dedicated machine
• bad points: expensive license

VMware Infrastructure
• not free (evaluation period for 2 months, so backup your virtual machine images just in case that someday you can't access them anymore due to an expired license)
• old


Oracle Virtual Box:
• good points: free, open source
• documentation: oracle website

Microsoft Virtual PC
• good points: free, easy to use (don't need to read documentation)
• negative points: Windows host only

Xen
• good points: free
• negative points: linux host only

KVM & QEMU
• good points: free
• negative points: linux host only

---

Finally I chose VMware Player due to reasons:
• free
• support both windows & linux as OS hosts
• support windows & linux as OS guests (even with tricks you can run MacOS too, handy for example for iPhone application development)
• in my work we use VMware products, so I need to work with VMware compatible virtual machine images
• APIs (e.g. Perl, COM) are available to control virtual machines in your development/test environments via scripting / (web)applications.
• I have feeling that VMware products are the best (due to their pioneering & dominance in the virtual machine market for years). After installing add-in tools, you can easily copy-paste / click-drag to copy text, files, etc.
• currency/maintained

PS. Although in my work we have a licence for VMware, for example we use also VMware Lab Manager (a configuration manager for VMware ESX virtual machines), I was looking for a free virtual machines for personal use.

Please leave any comments if you know any other good virtual machines or you have different opinions about the virtual machines discussed here.

---

References:
An outdated book but still worth to read for ideas/best practices. It discusses real world scenarios about managing virtual machines in your development/test environments, and the examples of COM/Perl API usages.


Another good book about VMware vSphere admin: