VMware virtual network configuration: NAT port forwarding

Playing with virtualization is fun. You can entertain yourself as if you have an army of many computers, connected with one or more (virtual) networks.

There are 3 possible configurations of virtual networking in VMware:

1.Host-only: the virtual computers (VM) are connected each other in the virtual networks inside a host (myPC), which is connected to the internet via IP 123.456.789.012. They can access the internet data but it's impossible for a computer in the internet (e.g. Annie) to send request to these VM since the virtual networks are private (e.g. IP in the 198.168.X.X range).

2.Bridge: you're lucky. The network administrator in your company is your best friend, so he gave you several free slots of IP addresses. Thus the VM has its own IP address (e.g. 123.456.789.013). The VM acts just like a real PC, the computers in the internet can reach this VM.

3. NAT: Well, your organization has a limited IP addresses, but with NAT you can still have possibility to let other computers in the internet to access your VM. For example you have a webserver using port 333 in the VM 198.168.11.12, using Network Address Translation (NAT) you can use the port 444 in the myPC host 123.456.789.012 to expose this webserver to the internet.

The vmware network configuration can be set using vmware-config.pl in Linux or vmnetcfg.exe in Windows. Here is a screenshot example over how to configure the NAT / port forwarding from the host 123.456.789.012:444 to the VM webserver in 198.168.11.12:333.



Notes:
How to get vmnetcfg.exe for VMware Player Windows:

1. Download the installer
2. Extract the installer: VMware-player-installer.exe /e tempdir
3. Extract the network.cab in the tempdir, it contains vmnetcfg.exe

If you update the VMware Player vmnetcfg.exe is not extracted by standard installation so the vmnetcfg.exe in your VMware directory  is still the old version that missmatch with the new VMware version. Thus you need to repeat this procedure to replace the old vmnetcfg.exe in your VMware directory.


Source: Steve's blogs http://soa-java.blogspot.com/

Any comments are welcome :)

---

Note: you can achieve the same thing with iptables service in linux, but this vmware-config approach is easier.

References:

Android Simulator : simple up and running...

You don't have a smartphone yet but you're curious about how your website looks like in a smartphone?

Cost: free
Time needed: 5 minutes
Steps:
1. download and install the Android development SDK: http://developer.android.com/sdk/index.html
2. create an Android Virtual Device (AVD) profile: startmenu>manage ACD>create an AVD (e.g. Android 2.3.3 HVGA)
3. run the emulator: e.g. emulator -avd name-of-your-avd

---

Source: Steve's blogs http://soa-java.blogspot.com/

Any comments are welcome :)

Mobile web development

Why mobile webs are different
You need to adapt your web application since the usability of mobile usage is different compared to the desktop:
• smaller screen size (e.g. 320x480)
• vast variations of browsers & platforms (each with different limitations & compatibilities e.g. some doesn't support javascript yet)
• competing attentions / multitasking users (e.g. the user may use the mobile while waiting for a bus or talking to friends), this leads to more distractability and shorter session (3 min in average instead of 10 min average of desktop users).
• different input devices (e.g. multitouch & virtual keyboard via touch screen instead of desktop keyboard & mouse)

Design strategies
• define user's goals and how they can accomplish their goal in your web with minimum efforts (i.e. minimum clicks/inputs)
• prioritize the features of the desktop version of your web, implement only the top 20% in your mobile version

Usability tips
Simplicity
• implement only 20% of the features of your desktop web application
• minimize user efforts & user inputs, try to infer the context from history/cookies, geolocation, IP address
• no more than 3 clicks (or pages) depth
• limit the main navigation to 4 links, limit the total links in a page to 10
• minimize text, use short/simple words
• to the point, no welcome screen
• limit the bandwidth: simple image, don't use text-image. The bigger the bandwidth, the more users have to pay & the slower your service is.

Layout
• avoid horizontal scrolling
• avoid multi columns
• use all area, 100% width (don't use side menu, side advertisements etc)
• use fluid layout instead of fixed layout

General tips
• always provide a link to the desktop version
• the most used features at the top (e.g. login in a bank service, search in a library service)
• provide enough space (min 20px) for clickable elements and links, since finger touches need more space than using a mouse
• use background colours to separate sections
• use a list instead of a table

Beside these, many of the desktop UI usability rules are still apply in the mobile world, such as visual consistency, legible fonts, clear structure, consistent alignments, etc

Device awareness and content adaptation
A common approach to handle the variations of mobile browsers/platforms is
by grouping according to device/browser capabilities (the groups don't need to be mutually exclusive) for example based on screen resolution, portrait/landscape orientation support, javascript/ajax support, geolocation support, markup languages (old wml,xhtml-mp,html5).

Starting with a basic version of your web (e.g. a plain html without css & javascript), create a different version for each group (e.g. different screen resolution) using different css and technologies (javascript, geolocation, etc).

Your web application needs to aware about the capabilities of the client browser/device and then adapts the content according to which group this device falls to. So first we need to know which browser/device the client uses, using http request header information, for example an (rather old) iphone:

User-Agent: Mozilla/5.0 (iPhone; U; CPU iPhone OS 2_2_1 like Mac OS X;
en-us) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1
Mobile/5H11 Safari/525.20
Accept: text/xml,application/xml,application/xhtml+xml,


Then using device libraries (e.g. WURFL) to get the list of device/browser capabilities (screen size, javascript/ajax support, etc).

The weakness of this approach is that you need to maintain several versions of your code (e.g. iphone4.css, iphone3.css, android23.css, vintagemobile.css, ....). The mobile/device list is changing fast, you need to keep up with this (and hopefully also the maintainers of the device library you use). The information in the request header & the device libraries may not be accurate. That why it's recommended to limit the number of groups as minimum as possible.

An example of comparison of a desktop and mobile website
Amazon is one of the most progressive dot.com company, we can learn by observing their design. This is the desktop version:


This is the mobile version (Android simulator, 320x480):

The mobile version has
• much less features than the desktop version
• less text, less images
• list (for links)
• one column
• the most important features in the top of the pages (the shopping cart and the search.)

In the future I will discuss how to implement geolocation mobile web to determine in which building a student is located. Based on this, we provide contextual information to the student for example about the course room/schedule, computer labs availability, the locations of other students from the same course/study-year.

Source: Steve's blogs http://soa-java.blogspot.com/

Any comments are welcome :)

---

References:
Programming the Mobile Web


Beginning Smartphone Web Development


The UI snapshot examples are the courtesy of www.amazon.com

Java Callout in OSB

Sometime it's handier to implement your algorithm in Java than in OSB. This short tutorial shows you how to call a java class from OSB.

The Java code
Suppose you have implemented your algorithm in a Java class as follow:

package tud;

public class JavaHelper {
public static String occupy(String street){
//do occupy
return street+" is occupied.";
}
}

The method has to be static. Package this class to a jar (e.g. using Ant or Maven or "Eclipse export to jar" or "jar -cf jarname packagename". Deploy this jar in weblogic (go to /console then choose deployment>install).

The Java callout action in OSB
Create a Java callout action in message flow in your proxy:

* In the "method" field you specify the package.class.operation name of your Java jar.
* In the "action expression" field you specify the input parameter for your java class, e.g. I take the "Input" element from the body.
* In the "result value" field you specify the variable that will contain the result, e.g. the "occupy" in this case.

Test
using this input "Sesame street" you will get the occupy variable "Sesame street is occupied." due to the occupy java method.



In this article we use a primitive type (in this case string), you can also pass an XMLobject (e.g. using XMLbeans framework to create java class types from Schema) as described in Eric Elzinga's blog: http://www.xenta.nl/blog/2011/08/29/oracle-service-bus-java-callouts-with-xmlobjects/

Source: Steve's blog http://soa-java.blogspot.com

Agile estimation & planning

You might encounter several weird buzzwords when you're reading about agile estimation methods. This blog tries to give down to earth discussions to help understanding.

Planning Poker
A method to estimate work amounts, which inherits some similarities to the poker game.
The steps:
1. every team member gives an estimation for each user stories (e.g. 8 man-hours for the "customer registration GUI" story), but the number is hidden until all the members have given their estimations. You can use a real set of poker cards to make the process more fun, each team member chooses a card number then puts it upside down to hide the number.
2. when everybody has given his/her estimate, discuss the numbers (e.g. Jip believes that building the GUI will cost 6 points using Struts, Janneke gives 2 points instead using simple PHP, the team will discuss the pros/cons of each method/estimation)
3. repeat the process until the numbers converge (the team members estimates become closer & closer to each other)

Variations:
• online instead of face2face planning meeting, e.g. using http://planningpoker.com/ , but I think face2face will be better & more fun than an online one, but perhaps not always handy for distributed teams.
• publish the user stories and let people outside the team to estimate. e.g. All the developers in the Jip & Janneke team has no experiences with SOA. Annie, a colleague in another team, has extensive experiences with SOA, so Annie can help to estimate the SOA user story. Beware that as the estimation comes from outside the team, it might not reflect the velocity of the team, but at least it can be a starting point for further discussion/estimation.


Tabletop estimation
Another estimation methods by arranging the user stories on a tabletop.
The steps:
1. write all the user stories in small cards
2. compare and arrange the positions of the cards based on their magnitudes (e.g. the story "buying an ice cream " is simpler than "arrange a marriage" so we put "buying ice cream" on the left)
3. Discuss the orders and change if necessary.
4. assign the numbers for each of the stories
5. discuss the numbers.


Simple Release planning
The numbers of iterations = total story points / velocity
The numbers of remaining iterations = remaining story points / velocity
There are many ways to define the velocity, e.g. the average the N last velocity or pessimistic (the average the N worst velocity).

Slacks/buffers
A slack is extra time that you put in your schedule. It might help you to save your deadline when you're overrun with schedule, otherwise you can use the slack time for:
• refactoring / paying technical debt
• learning a new framework / research time
Buffers basically serve the same idea, you can use features buffers and/or schedule buffers.

Fibonacci numbers
When coosing an estimate number, Mike Cohn proposed to use Fibonacci numbers (i.e. each number is the sum of its previous two) e.g. 0,1,2,3,5,8,13, 21,34... instead of just a simpler integers (e.g. ...,4,5,6,7,8,...). The reason is that the bigger the order of magnitude the bigger also the uncertainty (i.e. the gap between the numbers). Personally I prefer to use just simple integers.


Incorporating risks to your planning
The risks will prolong the remaining number of iterations above:
The numbers of remaining iterations = ((remaining story points / velocity)/ risk multiplier)- total risk exposure.

So there are 2 risk factors that will prolong the project: the total risk exposure (due to the project-specific risks e.g.some developers may sick/leave) & the risk multiplier (due to general risks e.g. how stable the development process in this company):

The delay due due to the project-specific risks is quantified by:
The Total risk exposure = sum_i (probability_risk_i * cost_risk_i). For example we have 2 risks: Bumba the GUI engineer who is often sick (probability 0.4 to cost the project 0.5 iterations more), and the Circus server thats need an update this month (probability 0.9, will cost the project 1 iteration more). So the total risk exposure = 0.4*0.5 + 0.9*1 = 1.1 iteration.

The velocity reduction due to general risk is quantified by a risk multiplier table:

As we see in this table the general risk is expressed in probability, thus the resulting remaining iterations estimation is not a (deterministic) point but a (probability) bar. So in a risky process (e.g. the Circus devteam just moved to the SOA environment for the first time), their velocity will be affected by 2x slower in the 0.5 probability case and 4x slower for 0.9 probability case.

---

How's about your experiences with planning/estimation methods? Which works and which doesn't? Why? Please share in the comment.

Source: Steve's blog http://soa-java.blogspot.com

References:
Agile Estimating and Planning


The Art of Agile Development


Agile Software Requirements

Performance testing

Don't guess, test it!

Why performance test:
• to know the performance (e.g. can I handle 100 users, how fast I can refresh the GUI)
• to know how well it scales (e.g. how if I got 10x more customers)
• for tuning the parameters, design / architecture decision, queries

Tips:
• instead of postponing the performance test until the development finish you can do the performance test early in the architecture design phase / spike solutions, to choose which framework to use e.g. JSP or GWT/Ajax, hibernate or ibatis. In the early phase of architecture definition, the cost of a wrong decision can be expensive/irreversible. Consider that if you wait until you finish with the product and then you discover that you have made a wrong choice in the early design then your developer need to spend time again to redo the works (e.g. redo the JSP to GWT/ajax).
• With agile/test driven development spirit, even you may incorporate the performance test to your continuous integration / regression test.
• environment specific: the test result in your development PC will be different with the result in the production server, so use test environment as close as possible mimicking the production
• use different hardware to measure the statistics than the hardware which is being tested
• remove data which damage the quality of statistics (e.g. the initial burn-in period)
• use random think time

Steps:
1. preliminary test: explore (e.g. range #users), set test environtment params (e.g. jvm heapsize)
2. baseline test: typical usage for base comparison
3. stress test to the limit: increase the load until breaking, to know the limit, the bottleneck, the behavior in degradation mode
4. endurance test: load test (high load but not yet overwhelm) for long hours, to detect memory leak, resources/connections which are not properly closed, to know the fail behaviour (e.g. error handling)
5. stress test to fail: deprive the resources (e.g. overwhelm the cpu with other heavy tasks, turn off the networks), to know the fail behaviour (e.g. error handling)
6. vary the parameters (e.g. java heapsize, #clusters) / architecture (e.g. load balancer) / sql query / business logic, then repeat the baseline and stress test to limit fase, compare the results if it improves.

monitoring (to know where's the bottleneck):
• application: eclipse tptp profiler
• jvm: java profiler
• database: oracle trace, query optimizer
• os: top, vmstat, perfmon
• networks: packet sniffers (e.g. tcpdump), netstat, netprotocol analyser (e.g. ethereal)

How
1. define the metrics
2. simulate usage. Run a testrun, which run testscripts (which consist of requests/queries to simulate usage profile)
3. define sampling methods (e.g. fixed #cycles or fixed time window)

Performance metric
• use a clear/specific performance criteria, e.g. not "the web should be as fast as possible" but "the user get a confirmation after click the submit button with max response time 10 sec (given 100 simultaneous users activities)"
• for GUI, the metric can be max response time given #users
• for back-end (e.g. web services or database), the metric can be #transaction per sec (TPS)
• you can define also over limit behaviour e.g. 10% degradation response time with 1000 users.

The statistics:
a. Average response time (ART): aritmetic mean for all user
caveat: if the ART is within limit it doesn't mean that all user instances are within limit.
suggestion: draw the ART vs time (test cycles)
b. Average ART (AART): normalize ART with number of requests, so it will be comparable between different testscripts.
c. define the throughput transaction per sec (TPS): e.g. #(group)queries/sec in database or #messages/sec in JMS
d.Quality: standard deviation / average, prefer bellow 0.25.
e.g. compute standard deviation of ART(as a function of cycles) and divide by ART average over #cycles.


Tools, examples:
test: soapui, httpperf,junitperf,grinder,jmeter


Please write your comments about your experiences with performance testing.

---

Source: Steve's blog http://soa-java.blogspot.com

References:

My blog which mentioned about the use of performance test in the countinous integration test:
http://soa-java.blogspot.com/2011/03/soa-continuous-integration-test.html

J2EE Performance Testing by Zadrozny et.al.


Grig Gheorghiu's blog, a good place for information about testing, python and linux tips.
http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html

Performance Engineering
http://performance.punebids.com/

Why planning based on features instead of activities

The conventional way when you make a Gantt charts for planning is by typing the list of activities (which are derived from your statement of work/SOW) in your Work Breakdown Structure (WBS). However, in Scrum/agile, we are recommended to setup our plan based on the software features (e.g. user stories, requirements) instead of a list of task activities.

The reasons, some problems may arise by focusing in the activities instead of features during planning:
So you focus on your development activities (which may have less values to customer) instead of customer values.

Your focus is on finishing activities instead of delivering features which the users need. When the project is behind schedule, people tend to crash by reducing the works to do. If you focus on the activities, you will reduce the activities which are prioritized for the convenience of the development team. In this process you may dropping features and some of the dropped may have greater values than those are delivered.

When you check the completeness of the works, you will check the completeness of the activities instead of the requirements of the product (which are more relevant to the users.)

What are you experiences with this issue? Please write your comments.

---

Source: Steve's blog http://soa-java.blogspot.com

References:
Agile Estimating and Planning