Friday, April 29, 2011

Refactoring to Patterns


Please click here to get the presentation that I made years ago about refactoring and design pattern.

The key ideas:

What is Refactoring:
• Improving design by restructuring / clean up code
• Without altering external behavior / adding functionality

When to do refactoring:
• Maintenance / fixing bugs
• To understand the codes before adding functionality (e.g. legacy code, systems with integration problem)

When not to do refactoring:
• when the code is too messy, better to rebuild from scratch
• Close to deadline

Best practices:
• Build regression-tests first
• Small steps
• Reversible (e.g. version control)
• Apply Pareto principle, using profiling to prioritize
• The end result targeting Design patterns

Reasons for Refactoring:
• to increase readibility
• to reduce duplication,
• to increace flexibility,
• to increace modularity,
Which lead to:
• Easier to maintain
• Easier to extend
• Faster software development
• Easier to find & solve bugs


Signs that your code needs refactoring:
• Too large classes
• Duplicated codes
• Too long methods
• Too long parameters
• Comments to explain difficult code
• A class / method that doesn’t do much
• Inappropriate intimacy
• Divergent changes in a class
• Shotgun surgery: changes in many classes


Design patterns:
• proven solutions to commonly occurring problems in software design
• don’t reinvent the wheel
• Provide common terminologies for communication
• Variations on implementations depends on domain, tools (e.g. adapter pattern: multiple inheritance in C++, inheritance & interface/reference in Java)
• Nice way to learn software design: peruse design pattern books if you want to become an architect


Conclusions:
• Refactoring: improving design by restructuring without altering external behavior
• Refactoring can help understanding the codes and bugs fixing
• Some best practices during refactoring are: build test first, small & reversible steps, prioritize (Pareto principle)
• Design Patterns: proven solutions to commonly occurring problems in software design. Design Patterns can provide targets for refactoring.
• In practice: refactoring BB-Volg+ shows improvements of some metrics (e.g. reduce duplication, method length, nested depth, cohesion), & other benefits: easier to extend, reusable.

Furthermore, this presentation also discuss:
several refactoring techniques e.g. extract class, extract methods, parameter object, explaining object, refactoring conditionals, refactoring database, refactoring css and many other techniques.
several (enterprise) design patterns e.g. singleton, template/strategy, observer, wrapper, factory, mvc,
• a case study: refactoring in one of my project, which resulting in improvements of some metrics (e.g. reduce duplication, method length, nested depth, cohesion), & other benefits: easier to extend, reusable.

PS. You don't have to implement these design patterns from scratch. For example I use Java frameworks such as Struts (MVC pattern, observer pattern, page controller pattern), Hibernate (an object-relational-mapping that implements active records and many transaction management/concurrency patterns), Velocity (template pattern), and Spring (many patterns... proxy, factory, MVC, template, etc).

Please leave your comments to discuss about refactoring & design patterns.

In future I will write blogs about refactoring and design patterns for SOA, so stay tuned!




This presentation is based on my study with these books:

Refactoring
A classic, one of the standard read for software developers.


Patterns of Enterprise Application Architecture
One of the most famous books for J2EE developers


Design Pattern
A seminal literature, one of the standard read for software developer.


Code Complete
Another standard read for software developers.

Learning the basic principles of UI / web design

If you're already savvy with the technical sides of web sites (html, javascript, css, jsp) but you still feel awkward with the design sides, then you are not the only one. I was in this situation when I had to build a web application from scratch and I had only 2 days to learn web design principles. Thanks God finally all the shareholders were happy with the results. The user panels voted my design better than a design from a professional designer.

So I'd like to share with you which books that I read in 2 days to learn web design:

The Non-Designer's Web Book
Discuss the CRAP (Contrast, Repetition, Alignment, Proximity) design principles. These principles apply also for print designs/ desktop publishing, which is also my hobby. Also contains some examples of good & bad designs and practical suggestions (e.g. avoid aliasing in the images, use sans-serif fonts etc).






Don't Make Me Think
Discuss usability principles:
• be straightforward as possible (such that the users doesn't have to think)
• users scan the web instead of read
• users don't make optimal choices, they don't read all the information, they just guess and satisfice
• users don't read explanations
• if the users find something works they stick to it (e.g. people used to the menu layout in common Windows application), avoid surprise

Also some design principles:
• clear visual hierarchy/structure
• clearly defined areas (e.g. navigation menu, content etc)
• conventions (e.g. colour scheme)
• make it obvious what's clickable
• minimize noise / minimize items in your web






Web Design for Developers
A book from the pragmatic programmer series. For developers who have never built a website. contains choosing colors & fonts, mock design, photoshops, how to apply css.






Web Design in a Nutshell
A classic book, which I read years ago. Discusses basic factors which affect your design such as screen resolution, browser, flexible vs fixed-width layout. Also contains references for (x)html, css, image formats, etc.






Please leave comments if you know any interesting books to learn web / UI design.

In near future I will write blogs about web design for mobile devices, so stay tuned!

Which free Virtual Machines to use?

VMware Player
• free
• easy to use (don't need to read documentation)
• support both windows & linux as OS hosts
• Even though the name is "player", you can also create new virtual machines with some basic settings (memory size, virtual storage size), to use port forwarding for NAT-virtual networking you need to use a configuration utility which is "hidden" in one the the cab files.

VMware Server
• free
• it seems that VMware stopped maintaining this software so it has compatibility problems with the new OS (e.g. Windows 7, Fedora 14), you can resolve the problems but it costs time.

VMware vSphere ESX
• a newer version of VMware Infrastructure
• hypervisor architecture: faster & you don't have to pay an extra license for the host OS
• bad points: need a dedicated machine
• bad points: expensive license

VMware Infrastructure
• not free (evaluation period for 2 months, so backup your virtual machine images just in case that someday you can't access them anymore due to an expired license)
• old


Oracle Virtual Box:
• good points: free, open source
• documentation: oracle website

Microsoft Virtual PC
• good points: free, easy to use (don't need to read documentation)
• negative points: Windows host only

Xen
• good points: free
• negative points: linux host only

KVM & QEMU
• good points: free
• negative points: linux host only


Finally I chose VMware Player due to reasons:
• free
• support both windows & linux as OS hosts
• support windows & linux as OS guests (even with tricks you can run MacOS too, handy for example for iPhone application development)
• in my work we use VMware products, so I need to work with VMware compatible virtual machine images
• APIs (e.g. Perl, COM) are available to control virtual machines in your development/test environments via scripting / (web)applications.
• I have feeling that VMware products are the best (due to their pioneering & dominance in the virtual machine market for years). After installing add-in tools, you can easily copy-paste / click-drag to copy text, files, etc.
• currency/maintained

PS. Although in my work we have a licence for VMware, for example we use also VMware Lab Manager (a configuration manager for VMware ESX virtual machines), I was looking for a free virtual machines for personal use.

Please leave any comments if you know any other good virtual machines or you have different opinions about the virtual machines discussed here.


References:
An outdated book but still worth to read for ideas/best practices. It discusses real world scenarios about managing virtual machines in your development/test environments, and the examples of COM/Perl API usages.


Another good book about VMware vSphere admin:

Friday, April 15, 2011

Using unix scripts or tools (e.g. sed , awk, find) in Windows

Suppose you work in Windows platforms but you miss the power of sed, awk, find, grep, or other handy tools in Unix, or you want to reuse your Linux scripts? Just install cygwin, it's a kind of Linux terminal for Windows (a simplified explanation.)

In cygwin, your Windows drives are located in the /cygdrive. For example if you want to delete all ".svn" directories in D:/My Workspace/ in cygwin you can type:
cd /cygdrive/d/My\ Workspace
rm -rf $(find . -type d -name .svn)

Monday, April 11, 2011

Showing the svn / build version number in your (web) application

It is often useful to include the version information in your (web) applications so you can track which code/build that your customers use. This comes handy especially when you need to solve bugs. Always mention which version of code/build where the bug occurs when you write a bug reproduction scenario in your bug tracking tools (e.g. Bugzilla.) As an example, one of the most common way to include the version information is in the "About" dialog.


Showing a version number in a JSP web application
For example you can include this in your jsp:

Version: <%= application.getInitParameter("version-svn ") %>

which reads the version-svn parameter value in the web.xml:

<context-param>
<param-name>version-svn</param-name>
<param-value>170</param-value>
</context-param>


Changing the web.xml parameter value using Ant script
In the Ant build.xml you can use XmlTask to change the version number in the web.xml:

<taskdef classname="com.oopsconsultancy.xmltask.ant.XmlTask" name="xmltask">



<target name="changeversionwebxml" depends="getsvnver">

<xmltask dest="${webinf}/web.xml" source="${webinf}/web.xml" failwithoutmatch="true">

  <replace path="/*[local-name()='web-app']/*[local-name()='context-param'][1]/*[local-name()='param-value'][1]/text()" withtext="${revision.max}">

 </xmltask>

</target>

The xpath predicate "/*[local-name()='name of an element']" is necessary to resolve the namespaces in the web.xml. The [1] is necessary since in my web.xml I use more than one "web-app/context-param/param-value" element, so I need to tell the xmltask that I want to change the first "param-value" that it finds. The ${revision} property is set using another Ant target described below.

Getting the svn revision number using Ant script
We can use SvnAnt to get the highest svn revision number of your source tree starting from the ${basedir}.

<path id="svnant.classpath">

<!-- if you use EclipseIDE, add the svnant jars using eclipse>preference>ant>classpath -->

<fileset>

<include name=" ">

</fileset>

</path>

<typedef classpathref="svnant.classpath" resource="org/tigris/subversion/svnant/svnantlib.xml">
<target name="getsvnver">
<svn>
<wcVersion path="${basedir}"/>
</svn>
<echo message= "Subversion revision.max: ${revision.max}" />
</target>

The ${basedir} points to your working directory which contains your source codes along with .svn files (svn bookkeeping files), one way to define the basedir in the build.xml for example:

<project name="project name" basedir="../" default="your default target">


Using a build number
Actually the svn revision information is more appropriate for developers. For a shipped product, a build number will be more useful. Instead of the svn number, you'd better include extra info such as time, build machine, build owner (developer who run the build task). For example using Hudson (a continuous integration tool) you can access the build information using environmental variables:
<property environment="env"/>
<property name="buildinfo" value="${env.JOB_NAME}-${env.BUILD_NUMBER}"/>

You can substitute the ${revision.max} in the xmltask above with this ${buildinfo}.

Please post your comments / suggestions. Thank you.

ShieldsUP! to test the vulnerability of your firewalls

A handy web application to test the vulnerability of your firewalls, written by another Steve: http://www.grc.com/default.htm

Do you know other cool security tools? Please share in the comments. Thanks, Steve

Which SVN client in Eclipse

Everytime I (re)install Eclipse, I need to remind myself which svn client should I choose. Basically there are 2 popular svn plugins in Eclipse: subclipse and subversive. Some people said that subversive has more features and handier (e.g. for merging). But when I used subversive I noticed that I couldn't use other svn client (e.g. subclipse, tortoise) nor other versions of the subversive. Because of this compatibility issue, I chose to use subclipse.


---
Update Sep 2011:
I used subversive since by default subclipse uses javahl (to connect to svn) which has problem with 64bit OS. I can use also svnkit instead of javahl but I don't have time now for work arounds.

Which svn client do you prefer to use? Please share in the comments.

The motivation for this note
Following a suggestion in the "97 Things Every Software Architect Should Know" that I should note my rationale so I don't have to redo research again every time I have to make similar decisions.

Friday, April 8, 2011

XSLT to transform a flat XML to a hierarchy / tree using grouping

A common problem in the SOA development is that you need to provide a webservice which produce a hierarchy/tree-structure XML such as:


<toys>
<category> Train </category>
<members>
<name>Thomas </name>
<name> Percy</name>
</members>
</toys>
<toys>
<category> Sesame street Monsters </category>
<members>
<name> Elmo </name>
<name> Zoe </name>
</members>
</toys>

Where the data come from a relational database table such as:























Category Name
Train Thomas
Train Percy
Monster Elmo
Monster Zoe



When you build this webservice using Oracle SOA suite, first you make a database adapter (the right side in the picture bellow) then expose this to a web service (the left side in the picture below). You can add a mediator between them which is handy for routing (if you use more than one database adapters / external webservice calls) and/or transformations.

However the output from this database-webservice is apparently just a flat XML such as:

<toy>
<category>Train </category>
<name>Thomas </name>
</toy>
<toy>
<category>Train </category>
<name>Percy </name>
</toy>
<toy>
<category>Monster </category>
<name>Elmo </name>
</toy>
<toy>
<category>Monster </category>
<name>Zoe </name>
</toy>

Thus we need to transform this flat XML to the hierarchical XML as mentioned above. With Oracle SOA Suite, you can define de Schema in the exposed webservice and then apply a transformation in the reply route from the database adapter to the exposed webservice.

A common XSLT solution for this problem is the Muenchian method. However, thanks to XSLT 2.0 now we have a much easier solution using xsl:for-each-group. Here is an example of the XSLT code that I used:



You're welcome to drop comments or discuss more XSL/XML/SOA tricks.

To learn more about grouping or XSLT in general, please read the "Beginning XSLT and Xpath."


Oracle SOA Suite is a great way to build a web service to publish your database. This solution can be built within minutes. Please read "Getting Started With Oracle SOA Suite" to learn more.

Website information using netcraft

With netcraft you can see some info regarding a website such as its IP, webserver, OS, owner, and the registrar.

http://toolbar.netcraft.com/site_report?url=http://www.abnamro.nl
well, this well known bank is rich enough to invest in powerful hardware

http://toolbar.netcraft.com/site_report?url=http://www.berkeley.edu
Unix... what else?

http://toolbar.netcraft.com/site_report?url=http://www.arizona.edu

Another tipi: no windows, apache inside

http://toolbar.netcraft.com/site_report?url=http://www.ascc.edu/

A smaller institution using IIS in Windows, a sensible solution within budget.


The most popular sites
Http://toolbar.netcraft.com/stats/topsites
facebook, google, youtube, amazon, ebay, twitter... well it surprised me that facebook is the number one, more than google


Please drop your comments (e.g. if you know other interesting networking tools).