FitProgrammer@Work

foobar2000 - A pretty nice music player for windows

2012-01-04T21:23:00.001-05:00

foobar2000: I discovered today foobar2000 and have used it for almost the whole day today and i must say that its very light and fast (you really can feel it is so). Highly recommended - go check it out.

'via Blog this'

Batch script to change Java Home between different JDKs

2012-01-03T19:11:00.001-05:00

I recently happened to be working on multiple releases with some of them requiring the code to be compiled with JDK 6 whereas the more recent releases requiring code to compile with JDK 7. So i wrote up the following batch script to make the job of switching between the 2 JDKs easier. Before i run my ant build script i run the below script to set to ‘7’ for JDK 7 or ‘6’ for JDK 6.

@ECHO OFF

echo %1%

if "%1"== "" (

    echo "Syntax: javahome 7 or 6"

    Goto :end

if "%1" == "7" (

    echo "Setting JDK 7"

    set JAVA_HOME=C:\Program Files\Java\jdk1.7.0_01

    GOTO :printver

if "%1" == "6" (

    echo "Setting JDK 6"

    set JAVA_HOME=C:\Program Files\Java\jdk1.6.0_23

    GOTO :printver

:printver

echo %JAVA_HOME%

set NEW_PATH=%PATH%

FOR /F "delims=;" %%P IN ("%PATH%") DO (

    @ECHO path = %%~P

set PATH=%JAVA_HOME%\bin;%NEW_PATH%

echo %PATH%

java -version

:end

Some new movie reviews

2011-12-25T00:57:00.000-05:00

My family has been away for last 2 weeks in which i had the free-time to catch up on some of the new movie releases. Here are the ones i watched this holiday season, in order:

Hugo (3D) - Throughout his extraordinary career, Academy Award-wining director Martin Scorsese has brought his unique vision and dazzling gifts to life in a series of unforgettable films. This holiday season the legendary storyteller invites you to join him on a thrilling journey to a magical world with his first-ever 3-D film, based on Brian Selznick's award-winning, imaginative New York Times best-seller, "The Invention of Hugo Cabret." Hugo is the astonishing adventure of a wily and resourceful boy whose quest to unlock a secret left to him by his father will transform Hugo and all those around him, and reveal a safe and loving place he can call home.
Sherlock Holmes 2 - A game of shadows - Sherlock Holmes (Robert Downey Jr.) has always been the smartest man in the room...until now. There is a new criminal mastermind at large-Professor Moriarty (Jared Harris)-and not only is he Holmes' intellectual equal, but his capacity for evil, coupled with a complete lack of conscience, may actually give him an advantage over the renowned detective. When the Crown Prince of Austria is found dead, the evidence, as construed by Inspector Lestrade (Eddie Marsan), points to suicide. But Sherlock Holmes deduces that the prince has been the victim of murder-a murder that is only one piece of a larger and much more portentous puzzle, designed by Professor Moriarty. The cunning Moriarty is always one step ahead of Holmes as he spins a web of death and destruction-all part of a greater plan that, if he succeeds, will change the course of history.
The girl with the dragon tattoo - The Girl with the Dragon Tattoo is the first film in Columbia Pictures' three-picture adaptation of Stieg Larsson's literary blockbuster The Millennium Trilogy. Directed by David Fincher and starring Daniel Craig and Rooney Mara, the film is based on the first novel in the trilogy, which altogether have sold 50 million copies in 46 countries and become a worldwide phenomenon.
Adventures of Tintin - Paramount Pictures and Columbia Pictures Present a 3D Motion Capture Film The Adventures of Tintin directed by Steven Spielberg from a screenplay by Steven Moffat and Edgar Wright & Joe Cornish. Starring Jamie Bell as Tintin, the intrepid young reporter whose relentless pursuit of a good story thrusts him into a world of high adventure, and Daniel Craig as the nefarious Red Rackham.
Mission Impossible 4 - Ghost Protocol - This is not just another mission. The IMF is shut down when it's implicated in a global terrorist bombing plot. Ghost Protocol is initiated and Ethan Hunt and his rogue new team must go undercover to clear their organization's name. No help, no contact, off the grid. You have never seen a mission grittier and more intense than this.

All were nice movies (or else i would not have gone to watch them :)) but i liked the "Adventures of Tintin" the most, followed by MI4 and Sherlock holmes, followed by Hugo and then Girl with the dragon tattoo. The problem with the GWTDT is it is not as well made as the sweedish version of the same movie (which i had watched before the english one).

VMware vSphere Performance Manager APIs

2011-12-19T12:55:00.001-05:00

PerfCounterInfo

Figure 1 – PerfCounterInfo

PerfInterval

Figure 2 - PerfInterval

Performance Provider

1. A "performance provider" (PerfProviderSummary) is any managed object that generates utilization or other performance metrics.

2. Performance providers include managed entities, such as hosts, virtual machines, compute resources, resource pools, datastores, and networks. Performance providers also include physical or virtual devices associated with these objects, such as virtual host-bus adapters and network-interface controllers (NICs)

3. Each performance provider—the instrumented device or entity—has its own set of counters that provides metadata about its available metrics. Each counter has a uniquekey, referred to as the counterId.

Performance Counter

4. Counters are organized by groups of finite system resources, such as memory, CPU, disk, and so on.

5. PerfCounterInfo data object, shown in Figure 1, represents a performance counter.

Name	Type	Description
groupInfo	ElementDescription	The group of the performance counter with its label and summary details.
Key	Int	A system-generated number that uniquely identifies the counter in the context of the system. The performance counter ID.
Level	int	1..4, 1 is default. Higher the setting more data is collected by vCenter. See below for details. Note this is only applicable for vCenter and not ESX host.
nameInfo	ElementDescription	Counter name
perDeviceLevel	Int (since 4.1)	>= level.
rollupType	PerfSummaryType	One of average, latest, max, min, none, summation
statsType	PerfStatsType	One of absolute, delta or rate
unitInfo	ElementDescription	Unit for values of performance counter.

6. The performance counter can be represented by the following dotted string notation:

[group].[counter].[rollupType]

Example, disk.usage.average

7. Here are a list of four levels and what counters are included:

a. Level 1: includes basic metrics: average usage for CPU, memory, disk, and network; system uptime, system heartbeat, and DRS metrics. It does not include statistics for any device.

b. Level 2: includes all counters with rollup types of average, summation, and latest for CPU, memory, disk, and network; system uptime, system Heartbeat, and DRS metrics. It does not include any statistics for device either.

c. Level 3: includes all metrics (including device metrics) for all counter groups except these with rollup types of maximum and minimum rollup types.

d. Level 4: includes all metrics supported by VirtualCenter, including maximum and minimum rollup types.

Performance Metric

8. The cpu.usage.average is a performance counter for average CPU utilization. When the counter is collected on CPU No. 1 of a host, a performance metric is formed. The performance metric is represented by PerfMetricId data object which consists of two parts:

a. counterId: The integer that identifies the performance counter.

b. instanceId: The name of the instance such as “vmnic1” or “vmhba0:0:0”.

Performance Interval

9. The interval has to be longer than the sampling interval, which can be found as refreshRate in the PerfProviderSummary data object returned by queryPerfProviderSummary() method, normally 20 second. For VirtualCenter Server systems, instances of this data object are referred to as “historical intervals” because they control how data collected from the ESX systems will be aggregated and stored in the database.

10. vCenter default setting is level 1 and it retains sampled stats data as follows:

· 5-minute samples for the past day

· 30-minute samples for the past week

· 2-hour samples for the past month

· 1-day samples for the past year

11. PerfInterval is explained below:

Name	Type	Description
Enabled	Boolean	If disabled vCenter will not collect performance data for that interval or higher intervals. For example, disabling the "Past Month" interval disables both "Past Month" and "Past Year" intervals. The system will aggregate and retain performance data using the "Past Day" and "Past Week" intervals only.
Key	Int	Id for interval
Length	Int	Seconds that the stats corresponding to this interval are kept on system
Level	Int	1..4. higher the level more data is collected.
Name	String	Name of historical interval. Example, “Past Day”, “Past Week” etc.
samplingPeriod	Int	Number of seconds that data is sampled for this interval. The real-time samplingPeriod is 20 seconds.

Default properties for the four built-in historical intervals include:

KEY	SAMPLINGPERIOD	LENGTH	NAME	LEVEL	ENABLED
1	300	86400	Past day	1	true
2	1800	604800	Past week	1	true
3	7200	2592000	Past month	1	true
4	66400	31536000	Past year	1	true

12. In general, you should avoid changing the intervals as much as possible except the levels.

Real time vs historical performance statistics:

a. Real time stats collected at 20 secs sampling interval and kept for a length of 1 hour.

b. These real time samples are then processed to generate historical performance stats.

c. ESX only maintains 5 mins interval historical stats for one day length. vCenter does more aggregation and maintains stats for longer durations.

Important references:

1. vSphere SDK 5.0 reference on Performance Manager - http://vijava.sourceforge.net/vSphereAPIDoc/ver5/ReferenceGuide/vim.PerformanceManager.html

2. A decent introduction to performance manager APIs: http://www.doublecloud.org/2010/03/fundamentals-of-vsphere-performance-management/

Installing VMware vCenter Server 4.1

2011-09-23T20:15:00.000-04:00

Installing VMware vCenter Server 4.1:

Very helpful instructions with screenshot to walk you through the entire vCenter server 4.1 installation with a SQL Server 2008 (not the express edition one).

'via Blog this'

VMware vSphere 4 Networking Essentials

2011-06-29T13:30:00.001-04:00

1. vSwitch similarities to a physical L2 switch:

A vSwitch functions at Layer 2,
maintains MAC address tables,
forwards frames to other switch ports based on MAC address,
supports VLAN configuration,
is capable of trunking using IEEE 802.1q VLAN tags, and
is capable of establishing port channels.

2. vSwitches are configured with a specific number of ports: 8, 24, 56, 120, 248, 504, or 1016. VMKernel reserves 8 ports for its own use.

3. Changing the number of ports on a vSwitch requires a reboot of ESX/ESXi host.

4. vSwitch dissimilarities to physical L2 switch:

Does not support dynamic negotiation protocols for establishing 802.1q trunks or port channels like DTP (Dynamic Trunking Protocol) or PAgP (Port Aggregation Protocol).
A vSwitch cannot be connected to another vSwitch thereby eliminating a potential looping configuration. Because there is no possibility of looping, the vSwitches do not run Spanning Tree Protocol (STP).
A vSwitch authoritatively knows the MAC addresses of the virtual machines connected to that vSwitch so there is no need to learn MAC addresses from the network.
Traffic received by a vSwitch on one uplink is never forwarded out another uplink. So a vSwitch cannot be used as a transit path between 2 physical switches for example, because traffic on one uplink won’t be forwarded out another uplink.

5. Following 3 types of ports and port groups can be configured on a vSwitch:

Service console port
VMKernel port
Virtual Machine port group

All of the above types can be represented by one table/type:

std_vswitch_portgroup - model's VMKernel Ports, Service Console Ports and VM Port groups

· network_label
· vlan_id
· type - one of vmportgroup (0), vmkernel(1), serviceconsole(2)
· vmkernel_port_operations (bitmask - values) - only applicable for VMKernel ports

000 - Use this portgroup for iSCSI/NAS traffic

001 - Use this portgroup for VMotion

010 - Use this portgroup for fault tolerance logging

100 - Use this portgroup for management traffic - only applicable for ESXi

6. vSphere client combines the creation of vSwitch with creation of new ports or port groups. vSphere client does not ask about creating a new vSwitch, but rather what type of port or port group to create (connection types – Virtual Machine, VMKernel or Service Console are the options).

7. Unlike ports or port groups, uplinks are not necessarily required in order for a vSwitch to function. VMs connected to a vSwitch without any uplinks can communicate with each other but cannot communicate with VMs on other vSwitches or physical systems. Such a configuration is known as an “Internal-only vSwitch”. Communication between VMs connected to an internal-only vSwitch takes place entirely in software and happens at whatever speed the VMKernel can perform the task.

8. VMs connected to an internal-only vSwitch are not VMotion capable. But if the VM is disconnected from the internal-only vSwitch, VMotion will succeed if all other requirements have been met.

9. A vSwitch can also be bound to multiple physical NICs – this configuration is called NIC team. This takes advantage of load distribution and redundancy.

10. vSwitch associated with a physical NIC provides VMs with the amount of bandwidth the physical NIC is configured to support.

11. A single physical NIC cannot be associated with multiple vSwitches.

12. Max number of physical NICs on a ESX/ESXi host is 32 of which only 4 can be 10 Gbps adapters.

13. Service Console ports:

ESX supports up to 16 service console ports.
At least one service console port is required to exist on any one vSwitch on an ESX host.
We will discover but not perform any create/update/delete operations for this port type.

14. VMKernel ports:

Provide network access for the VMKernel’s TCP/IP stack (which is separate and independent from the Service Console TCP/IP stack).
VMKernel ports are used for VMotion process, iSCSI/NAS access and VMware FT.
With ESXi hosts, VMKernel ports are also used for management.
A VMKernel port comprises of 2 components:

A VMKernel port on a vSwitch
A VMKernel NIC – vmknic

e. The vmknic is configured with the interface IP address in the process of creating the VMKernel connection type in vSphere client. The IP address should be a valid IP for the network to which the physical NIC is connected to. One can optionally provide the default gateway if VMKernel NIC is required to reach remote subnets.

15. VLAN

a. IEEE 802.1Q Tagging – marking traffic as belonging to a particular VLAN. VLAN tag (aka VLAN ID) is value between 1 and 4094 which unique identifies the VLAN across the network.
b. VLANs are handled by configuring different port groups within a vSwitch. A port group can be associated to only 1 VLAN at a time, but multiple port groups can be associated with a single VLAN.
Figure 16 - Configuring VLAN on Port groups of a vSwitch
c. To make VLANs work properly with a port group, the uplinks for the vSwitch must be connected to a physical switch port configured as a trunk port. A trunk port understands how to pass traffic from multiple VLANs simultaneously while also preserving the VLAN IDs on the traffic. So the physical switch passes the VLAN tags up to the ESX server, where the vSwitch tries to direct the traffic to a port group with that VLAN ID configured.
d. The default native VLAN is VLAN ID 1. This is untagged VLAN ID meaning the switch port will strip this native VLAN id from the traffic as they pass. If you want to pass traffic on VLAN 1 to ESX server, then you need to maintain the tagging in the traffic and so you need to configure another VLAN id as native default VLAN. So an untagged VLAN (default native VLAN) can also be any id between 1 and 4094.
e. A good convention to follow while naming port group or port network label is: VLANXXX-Network Description. For example, VLAN11-IPStorage.
f. Although VLAN reduce the cost of constructing multiple logical subnets by separating the network segments logically but still all traffic runs on the same physical network underneath. For bandwidth-intensive network operations this disadvantage of shared physical network might outweigh the scalability and cost savings of a VLAN.

16. NIC Teaming

a. Uplink is a physical adapter bound to the vSwitch and connected to physical switch.
b. NIC teaming involves connecting multiple adapters to single vSwitch. It provides redundancy and load balancing of network communications to Service Console, VMKernel and virtual machines.
Figure 17 - NIC Teaming
c. As seen in figure above, both of the vSwitches have 2 uplinks and each uplink connects to a different physical switch.
d. ESX/ESXi can have a max of 32 uplinks and these uplinks can be spread across multiple vSwitches or all tossed into a NIC team on one vSwitch.
e. Building a functional NIC team requires that all uplinks be connected to physical switches in the same broadcast domain. If VLANs are used then all the switches should be configured for VLAN trunking and the appropriate subset of VLANs must be allowed across the VLAN trunk.
f. The load balancing feature of NIC teaming does not function like the load balancing feature of advanced routing protocol and hence is not a product of identifying the amount of traffic transmitted through a network adapter and shifting traffic to equalize data flow through all available adapters. Rather the load balancing algorithm for NIC teams in a vSwitch is a balance of the number of connections – not amount of traffic. NIC teams on a vSwitch can be configured with one of following 3 policies:

i. vSwitch port based load balancing (default)
This policy setting ensures that network adapter connected to a vSwitch port will consistently use the same physical network adapter. In the event one of the uplinks fails, the traffic from failed uplink will failover to another physical adapter.
This setting is best used when number of virtual network adapters is greater than number of physical network adapters. Link aggregation using 802.3ad teaming is not supported with this and MAC based load balancing policies.
ii. Source MAC based load balancing
It has same limitations as previous method – it’s also static mapping of vNIC MAC to a pNIC. Traffic originating from a vNIC will always go through the same physical NIC in this approach (Same as previous).
iii. IP hash based load balancing (out-IP policy)
It uses source and destination IP addresses to determine the physical network adapter for communication. This allows traffic originating from a single vNIC to go over multiple physical NICs when communicating with different destinations.
This policy setting requires all physical NICs be connected to same physical switch. Also the switch must be configured for link aggregation (which can increase the throughput by combining the bandwidth of multiple physical NICs for use by a single vNIC of a VM). ESX/ESXi supports standard 802.3ad teaming in static (manual) mode and does not support LACP or PAgP.

g. The load balancing feature on vSwitch applies only to the outbound traffic.
h. Failover detection with NIC teaming can be configured to use either a link status method or a beacon probing method.

i. Link status method – failure of an uplink is determined by link status provided by physical network adapter. But this can only identify the link status of pNIC to edge switch and not the link status between the edge switch to an upstream switch.
ii. Beacon probing failover detection – included link status as well as sends Ethernet broadcast frames across all physical network adapters in NIC team which help detect upstream network connection failures as well and will force failover when STP blocks ports, when ports are configured with wrong VLAN or when a switch to switch connection has failed. When a beacon is not returned on a pNIC the vswitch triggers failover.

i. Failback option controls how ESX will handle a failed network adapter when it recovers from failure. Default setting is yes which means the adapter will be returned to active duty immediately upon recovery and it will replace the standby adapter that may have taken its place during the failure. Setting it to No means the recovered adapter will remain inactive until another adapter fails.
j. One can even use the setting “Explicit failover order” in which case traffic will move to next available uplink in the list of active adapters. If no active adapters are available then traffic will move down the list to the standby adapters.

17. vNetwork Distributed Virtual Switches:

a. A dvSwitch spans multiple servers instead of each server having its own set of vSwitches.
b. First you create a dvSwitch and you add hosts to it during or after creation.
c. When an additional ESX host is added to a dvSwitch, all of the dvPortgroups will automatically be propagated to the new host with the correct configuration. This is the distributed nature of the dvSwitch – as configuration changes are made via the vSphere client a vSphere server pushes those changes out to all participating hosts in the dvSwitch.
d. A host cannot be removed from a dvSwitch if it still has VMs connected to a dvPortgroup on that dvSwitch.
e. Adding dvPortgroup:

i. Name of dvPortgroup – unique across member hosts of dvSwitch.
ii. Number of ports: 128 default, max configurable value is 8192.
iii. VLAN Type:

1. None – dvPortgroup will receive only untagged traffic
2. VLAN – dvPortgroup will receive tagged traffic and uplinks must connect to switch ports configured as VLAN trunks
3. VLAN Trunking – dvPortgroup will pass VLAN tags up to guest OS on any connected VMs
4. Private VLAN

18. FAQ:

a. Can we have vNIC created without portgroup association? NO - API requires you to specify an existing portgroup while creating vNIC.
b. You can assign any name to vswitch - for eg. 'ViMaster Switch'
c. Standard vSwitch Port group name is unique within a host
d. Hybrid deployments (vDS + vSS) is a supported deployment scenario. If vCenter fails then you won’t be able to manage your vDS so it is recommended that we use vSS for at least the VMKernel and Service Console connections. One can have the VM port groups on vDS though.
e. We can assign a virtual hard disk carved out of a VMFS datastore over an iSCSI LUN to a VM with no network card (vnics).
f. For software iSCSI initiator we can tell the vmknic interface being used by identifying which iscsi target is in the same subnet as the vmknic interface.
g. For hardware iSCSI initiator no vmkernel port needs to be configured and it will show up as a normal FCHBA (Storage adapters) in vSphere client. The ip address will be configured on the HBA directly.

Book Review: The Kite Runner

2011-06-14T03:00:00.002-04:00

I listened to the audio book “The Kite Runner” read by the author Mr. Khaled Hosseini. This book has a very strong story line to keep the reader’s attention occupied. I liked the story and following are some thoughts on the book:
1. Relation between Amir and Hassan – the loyalty and unconditional devotion which Hassan has for Amir speaks for the mindset that society can impose on an individual of lower rank that such an individual can never come out of. In spite of Amir’s cowardly manners Hassan never blames him and even accepts the false charges of being a thief just so that Amir is not caught red handed was quite touching.
2. The way Hazara community was treated in Afghanistan or the Shudras were treated in India or the Jews were treated by Nazis is a fact that humanity should never forget just so that it never repeats those mistakes in the future. It is works like this book that remind us of our abhorrent mistakes of the past and reinforces into us the realization that we should never let this repeat itself.
3. Reader gets a feel of the Afghan culture – what a pushtun clan can value, respect for elders in the society and the values like this that parents teach their children.
4. How the Afghan people were scared of the Taliban and some mentions of the atrocities that Taliban militia men conducted on people.
5. Amir adopts Hassan’s son Sohrab and brings him to US and the problems that he faces with the consulate even though he was a US Citizen was also quite believable and something any foreign national living in US can relate to.
Overall the story was well told – its not very big book and I could complete listening to it within 4 days. Now I will like to see the movie and will update this post with how I found that later.

Rushil learning swimming

2011-05-14T16:54:00.004-04:00

This is the picture of Rushil learning swimming at DACA Swim Center (www.daca.org). He is 3.5yrs old now. He is enjoying the experience of swimming this summer.

My .vimrc

2011-03-08T05:36:00.003-05:00

One of my favorite .vimrc config file: http://www.scribd.com/doc/50269230/vimrc

Book Review: The Fountainhead

2011-02-25T05:26:00.002-05:00

I recently completed the book The Fountainhead by Ayn Rand. Following is the review of the book:
Fountainhead is the story of an architect, named Howard Roark, who is extremely passionate about the field of architecture but his ideas about it are not in tune with the times in which he lives in. In his time the teachers in his college to the professional architects practicing business were all of the view that the field of building design and architecture has already reached its pinnacle of evolution and needs no further change from what the masters of the past have already established as the best approach to building design possible – namely the gothic architecture. Howard Roark was a modernist and did not endorse to the idea of gothic architecture and wanted to do the building design in a more modern way in tune with the demands of the present times. As a result, he is faced with several challenges while practicing architecture his own way. He wont get any contracts, he does not work if people don’t accept his design exactly as he designed it without any change, and there were people who worked against him to ensure he succumbs to the pain they inflict on him. Ayn Rand has very elegantly tried to convey the message that in this world most of us are mere second-handers who adopt the genius of one man and keeping towing to that line without wanting or accepting any change. When there comes a man who tries to bring in something new something different than the norm then the whole world goes against such a man – since that man represents the one that most of us mere humans are not – such a man is more able than any of the average men around him and so the world does not want to let him be since that man makes others realize that they are inferior than him. Such average mere mortals have low self-respect, no integrity and they endorse but one thing – that good of the community is more important than the good of the individual. An individual may be sacrificed for the good of the community if need be. Such people cannot stand alone and require support of the other like minded men. They only know to feed on the genius of the first handers – men like Howard Roark. Ayn Rand goes on to expound on the virtues of being selfish – to love oneself is not sin – but a primary duty of an individual toward himself. She goes on to mention that where there is some sacrifice being done by someone there is someone getting what is being sacrificed. In the name of religion and politics, in the name of brotherhood we come across several such people who vehemently try to emphasize to others the virtues of being unselfish and to practice sacrifice/renunciation.
Howard Roark says – I shall not live for any other man nor will I expect any man to live for me.
The philosophy behind this story is gripping and forces one to think about whether one has integrity or are we going to end up as one of the mere second handers.

The important characters are:
Howard Roark – the hero or the protagonist.
Dominique Francone – the heroine who loves Roark
Peter Keating – Roark’s college mate who starts with being a go-getter and then realizes that in his passion to succeed he has lost his integrity.
Gayle Wynand – the industrialist who is the alter ego of Howard Roark
Ellsworth Toohey – the villain – he feeds on the souls of the unsure the second hander population which needed some moral support from him and his guidance since they lacked their own decision their own integrity
Henry Cameron – an architect who Roark works for in his first job
Highly recommended. Even if one does not like the philosophy one should still read this book for the literary value it has – the prose of Howard Roark’s statement in the court, Ellsworth Toohey’s lecture to Peter Keating and in general the dialogues exchanged between the characters are very well composed and a pleasure to read.

Book Review: The Read-Aloud Handbook

2011-02-09T05:07:00.001-05:00

The read-aloud handbook by Jim Trelease is a treatise on the importance of reading aloud books to your children (by parents or by teachers in classroom). Mr Trelease brings forth several examples of how some parents were able to inculcate into their kids the habit of reading by beginning to read to them from early childhood. I will try to summarize my own experience of following this idea with my kid Rushil (who is 3.5yrs old).

I don’t remember exactly when I started to read on a regular basis during bed time to Rushil but it was approximately around 1.5yrs of age and only recently around 2.5-3yr age Rushil has started to show very positive interest in the books and just wants me to continue to read to him one book after another. Nowadays I read to him for at least 30mins during bed time but that is the least and mostly I have to read 2 stories (either from one book or 2 books). We visit the library once a week on Saturday’s after him gym class and of late that has become a routine and Rushil has come to expect that on Saturday’s and going to library is a pleasure trip for him as he gets to pick books and also DVDs there. We bring usually 2-5 books at a time with some books containing multiple stories. Apart from bedtime stories I also try to read to him while he is eating but that’s not a routine. Until now I have only been reading to him picture books and have also tried some chapter picture books (especially the Usborne anthologies ones which are picture books divided into chapters with about 70 pages for each story). I am yet to try reading a chapter from a Novel (like Charlotte’s Web by E. B. White). Once we also brought an audio book collection from library of Fairy tales and I was pleasantly surprised that Rushil was listening to the story of the Ugly Duckling quite intently. The audio books can be a good variety to play in the car at times.

Now going back to the book, the read-aloud handbook, it contains a list of books to be read aloud organized by the type of books and the kid’s age range. We can also find good lists on Amazon web site on which books to read to children of certain age. The list comes in handy in deciding which books are good to be read-aloud to children. In the first part of the book the author gives instances of several people who have written to him or spoken to him retelling their own experiences of reading to their students or children and the changes they witnessed in the kids. Also he mentions of research works which corroborate the fact that reading aloud to children is the best thing parents can do for their children’s education – by helping them build a solid foundation and imagination and most importantly inculcating in their kids the value of reading as a pleasure giving activity so that children are encouraged to read more as they grow. The book also recommends that parent’s themselves should lead by example by becoming avid readers. Reading aloud should also be done even after children start reading on their own. The book mentions of using close caption feature of TV while letting kids watch TV so that they also get to see the text together with what they hear – as this also may help in familiarizing the child with the sound and the text of words which will help them eventually in learning to read those text.

If you are a parent then it’s a good book to read to know about the values of reading aloud to your children.

WattOS distro review

2011-02-05T12:04:00.010-05:00

WattOS is a very fast Ubuntu based distro which i have started using now on my not-so-old laptop (Dell Vostro). I tried several re-mastered Ubuntu distros but somehow i was not getting the optimum performance out of my laptop. It was generally running slower than my work laptop which has windows XP and even though my Vostro has modest RAM and processor but even with Ubuntu (Gnome desktop one) i could see the processor being used upto 40% with just some apps in operation (an IDE, a terminal etc). Also the browsing speed was sub-par. With WattOS i see the performance on the same Vostro laptop very good. I hope this is going to remain stable. WattOS only supports 32 bit for now and even though my Vostro has Core 2 Duo (64 bit processor) i was ok with using a 32 bit distro too as even Ubuntu site mentions that 32 bit edition is more stable release. It uses LXDE/OpenBox desktop environment. WattOS uses the latest Ubuntu Linux as a basis using only the core system and then slowly adding base modules to ensure a small memory footprint, and speed.

I tried several distros before and even liked them but had it been a faster machine the distros like pinguy OS are very good and beautiful indeed but for my Vostro they were just too bulky. WattOS does not come with any games at all nor does it have several of the applications that come with the standard Ubuntu distribution but comes with bare minimal of the applications only - for office they have AbiWord, Leafpad for text editior, KeepassX for password management, fotoxx for photo management, Firefox as the default browser (which i replaced with Chromium), and then you have the Synaptic Package manager to add any other that you want. I will be using this system mostly for Java development and programming stuff and not for any document editing at all (as for those i have a windows 7 desktop) so i wont care too much about how complete a desktop it has and i will prefer if it does not have too many extra apps that i will not use like at all. So i am happy with the speed and performance of the machine and feel like my old laptop is rejuvenated. I wish i had found such an OS as wattOS earlier without having to waste my time in trying out others. Now my CPU usage under similar load conditions as perviously described is around 3% and memory usage is 1% (i have 4GB RAM).

Introduction excerpted from the WattOS site:
wattOS is a lightweight Linux operating system remastered from the core Ubuntu Linux build. It is a free operating system that focuses on a small footprint, low power, and a simple quick interface. Bring your old computer back to life again with a fresh install of wattOS!

Book Review: Kidnapped (Graphic Novella)

2011-01-31T03:47:00.001-05:00

I recently got hold of this book – Kidnapped by R. L. Stevenson, from the local library and it proved to be an interesting read and the illustrations in the novel were very good too. Its an abridged edition of course but is able to hold the interest of the reader quite well indeed. Following is some summary of the book in my words – David Balfour goes seeking for his inheritance after his father’s demise – and meets his uncle Ebenezer Balfour who is the evil kind who will not want David to get his share of the estate and conspires to kill David but fails and eventually manages to sell David off for bondage labor to a captain of the Covenant ship. There David happens to meet a guy of Scottish decent who he befriends and who finally helps David to come back to England and get his property back from his uncle. The adventures of David and his friend while they escape from the Covenant till they reach England forms the major part of the story line. Overall a nicely done novella – good for a quick read.

Book Review: The Call of the Wild

2011-01-30T15:34:00.002-05:00

This is my first review of any book on my blog. I have recently started paying attention to my reading habit and have completed the first book – by author Jack London – The Call of the Wild. Though much has already been written about the book I will try to summarize it in my words below. This story has its appeal for me for the fact that I am presently staying in Santa Clara California where Buck used to live in Judge Miller’s house and I have also once visited Alaska where most of the story is based on.
This is a classic American story about a dog (Buck) who is sold by Judge Miller’s (in who’s household Buck used to inhabit as a pet born of a Saint Bernard father and a Scotch Shepherd Mother) gardener, Manuel, to the wrong hands at the time when (strong sled) dogs were in great demand (due to Klondike Gold Rush). And so Buck was relocated from the Santa Clara valley California to the north land (Alaska). The story is about how Buck experiences and sustains through the drastic change in his life being brought about by being moved from a domesticated pet dog to a hard working sled dog in the extreme climate of the north land. Buck survives it all because of his physical strength, ability to adapt to the changing environment, and above all, not loosing his values and remaining true to his self. Buck was passed on from one master to the next but they all liked him except for the trio who were themselves bad masters and they end up loosing Buck to Thornton. Bucks affection for Thornton is very heart warming. Eventually Buck avenges for his master’s death and returns to answer the call of the wild – where he does not kill for pleasure but only for his need to survive.
This is a children’s classic and is a quick read and a nicely told story.

My new 2011 Honda Odyssey EX-L

2010-11-27T09:55:00.002-05:00

Its been quite some time that I have been keeping silent on this blog but today I just could not resist the need to announce it to the who-so-ever-is-interested (may be its just me – and yes I do like to read what I wrote in the past some times) that I bought my second car – which is not a car but a mini-van and the best in its category at the time of the purchase – the new 2011 Honda Odyssey. I went for the EX-L trim which comes with leather seats and many other luxuries which one does not often need but are nice to have. The color is Polished Metal (dark grey) and the interior leather color is beige (light greyish brown). I just drove it a little bit till now (from dealer’s showroom to home) but the feel of driving a brand new Odyssey is just as I read about it described on the several forums (which I visited while researching which of the Honda Odyssey, Toyota Sienna or Acura MDX should be my next car). They said it is a driver’s mini-van (meaning you will have perfect control on the vehicle while steering it – the steering control is almost perfect) and which makes even a big mini-van quite easy to maneuver. Also the 2011 is state-of-the-art with many of the tech features coming standard (like Bluetooth, Auxiliary Jack for MP3 players, USB port to charge iPODs etc). There are some unique convenience features like the cool box refrigerator, a trash holder in the 2nd row which are unique to the Odyssey 2011 (no other mini-van has it right now).

After juggling between Sienna, Odyssey and MDX we decided to go for Odyssey because of its better ratings as the best mini-van and economical mileage, nice technical package that came standard (which definitely increase the ease of use of the vehicle), the 2011 is a design refresh year for Honda Odyssey so many improvements have been made over the previous model, the cost was bearable (compared to MDX), even though it does not have a 4WD I decided to go for it because I already have the Honda Element which is a 4WD vehicle and in the bay area we don’t usually need the 4WD unless you are travelling to the Sierra’s (lake tahoe etc) where there is snow fall. We also considered the equally priced Honda Pilot EX-L but somehow after having one SUV (even though Element is a compact SUV) I did not want to own another similar (only little bigger) vehicle. Mini-vans are more roomy, more comfy and the 2011 model of Ody also looks and feels more like driving a car than a mini-van.

I have a 3.5yr old kid and the purpose of this van was to mainly drop and pickup the kid from school. But we also wanted to buy something more comfortable, not very luxurious but at least one that makes those couple family vacations to LA and the like more pleasurable. I first wanted to buy an MDX but I realized that it was too much of a stretch to go for (in terms of the cost of the car and its maintenance cost – only take premium gas and mileage is 16mpg, parts are expensive etc.). The other option was to get a used MDX but then I somehow prefer to own a less luxurious or less powerful *new* car than a fully loaded luxury car which is used one (I know this is not exactly being practical but Ody seemed to be a nice compromise and here’s why – I could get a fully loaded MDX Touring with RES/NAV used 2005 model with 50K+ miles on it comes for 24K approx. and if I spend 13k more I get a new Honda Odyssey 2011 which is not the fully loaded model but has all that I care for with just 2 miles on it ). I did not need the RES/NAV but what I cared for more was the leather seats, power doors, power tailgate, blue tooth hands free support. For RES and NAV I did not want to shell out another 3-4k more. I can always put a GPS in the car for much less price. Also the mini-van is in general more roomy and hence more comfortable on longer trips than any SUV. Also climbing in and out of the van for kids and older people is generally easier.

Enough with the pros of Ody, now the cons are: it does not come with 4WD, parking is a hassle some times, does not look as cool as an SUV would. But for me the pros outweighed the cons. Between Sienna and Ody, the Ody looked much better than the 2011 Sienna (which to me looked more like an ambulance). Sienna has a 4WD option but the mileage was minus 3 mpg from the 2WD one. Also the reviews that 2011 Ody got were better than 2011 Sienna.

Setting up NFS Server on Linux

2010-04-21T20:52:00.002-04:00

Steps to setup the NFS server on RHEL 5 Linux:
For details you can refer to http://nfs.sourceforge.net/.

1. Check if NFS is not already running by typing the command: rpcinfo -p

# rpcinfo -p

program vers proto port

100000 2 tcp 111 portmapper

100000 2 udp 111 portmapper

100024 1 udp 895 status

100024 1 tcp 898 status

100011 1 udp 725 rquotad

100011 2 udp 725 rquotad

100011 1 tcp 728 rquotad

100011 2 tcp 728 rquotad

100003 2 udp 2049 nfs

100003 3 udp 2049 nfs

100003 4 udp 2049 nfs

100021 1 udp 32774 nlockmgr

100021 3 udp 32774 nlockmgr

100021 4 udp 32774 nlockmgr

100003 2 tcp 2049 nfs

100003 3 tcp 2049 nfs

100003 4 tcp 2049 nfs

100021 1 tcp 54361 nlockmgr

100021 3 tcp 54361 nlockmgr

100021 4 tcp 54361 nlockmgr

100005 1 udp 791 mountd

100005 1 tcp 794 mountd

100005 2 udp 791 mountd

100005 2 tcp 794 mountd

100005 3 udp 791 mountd

100005 3 tcp 794 mountd

If the output looks something similar to above then NFS service is already running and you may need to just restart it in case you are modifying the /etc/exports file. If you don't see the nfs service being printed as output of command rpcinfo -p then follow the next steps.

2. Create a /etc/exports file (if not already present). For details and other examples refer to the man exports documentation.

Format for entries in this file is:

/directory/to/share nfs_client_host(options)

Following entry is only good for testing purposes (as it is insecure).

----- edit start -----

/vm_share *(rw,insecure)

----- edit end ------

3. Now run the below command to export all the directories.

$ /usr/sbin/exportfs -a

4. Restart the nfsd from the /etc/init.d script file. This may vary across distributions of linux. In case of RHEL 5, this file happens to be /etc/init.d/nfs. Pass it the option to start or restart.

5. Again verify that rpcinfo -p now prints nfs service too (as shown above).

Now you can use any NFS client to access the shared NFS directory /vm_share on the linux host.

Singleton Pattern with Lazy Initialization On Demand Holder

2009-12-24T14:29:00.005-05:00

This is one of the better ways to implementing a singleton class (class which lets user create one and only one instance).

It uses Initialization on demand holder pattern.


public class Singleton {
 private static class LazyHolder {
  private static final Singleton instance = new Singleton();
 }
 
 public static Singleton getInstance() {
  return LazyHolder.instance;
 }
         
 private Singleton() {
  //initialize singleton instance
 }
}

Using Apache Thrift for RPC over TCP socket

2009-12-22T04:13:00.001-05:00

Apache thrift is a framework for cross language RPC service/client implementation. Thrift has a small IDL (interface definition language) and comes with an IDL compiler that generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages.

First you need to download thrift and then install Cygwin (i used 1.7 version of Cygwin).

Since the thrift is distributed in source form so we first need to build the thrift.exe IDL compiler. Follow this document to generate the compiler exe on windows (mine is Windows 7 home premium x64).

Following is an example usage. We intend to write a C# service and a Java client that implement the following interface (hello.thrift):

namespace java hello
namespace csharp hello

enum Operation {
  ADD = 1,
  SUBTRACT = 2,
  MULTIPLY = 3,
  DIVIDE = 4
}


struct Work {
  1: i32 num1 = 0,
  2: i32 num2,
  3: Operation op,
  4: string comment,
}


exception InvalidOperation {
  1: i32 what,
  2: string why
}


service HelloService {

   void ping(),

   i32 add(1:i32 num1, 2:i32 num2),

   i32 calculate(1:i32 logid, 2:Work w) throws (1:InvalidOperation ouch),
   
   string greeting(),

}

Compile the above thrift file with the thrift compiler to generate both java and c# code.

thrift.exe -r --gen java hello.thrift

thrift.exe –r –gen csharp hello.thrift

Following is the service (Program.cs):

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Thrift;
using Thrift.Transport;
using Thrift.Protocol;
using Thrift.Server;
using hello;

namespace thriftcsserv
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                HelloServiceHandler handler = new HelloServiceHandler();
                HelloService.Processor processor = new HelloService.Processor(handler);
                TServerTransport transport = new TServerSocket(34568);
                TServer server = new TSimpleServer(processor, transport);

                // Use this for a multithreaded server
                // server = new TThreadPoolServer(processor, serverTransport);

                Console.WriteLine("Starting the server...");
                server.Serve();
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
            }
        }

        public class HelloServiceHandler : HelloService.Iface
        {
            public void ping()
            { }
            public int add(int num1, int num2)
            {
                return (num1 + num2);
            }
            public int calculate(int logid, Work w) 
            {
                int val = 0;
                switch (w.Op)
                {
                    case Operation.ADD:
                        val = w.Num1 + w.Num2;
                        break;
                    case Operation.SUBTRACT:
                        val = w.Num1 - w.Num2;
                        break;
                    case Operation.MULTIPLY:
                        val = w.Num1 * w.Num2;
                        break;
                    case Operation.DIVIDE:
                        if (w.Num2 == 0)
                        {
                            InvalidOperation io = new InvalidOperation();
                            io.What = (int)w.Op.GetTypeCode();
                            io.Why = "Cannot divide by 0";
                            throw io;
                        }
                        val = w.Num1 / w.Num2;
                        break;
                    default:
                        InvalidOperation io1 = new InvalidOperation();
                        io1.What = (int)w.Op.GetTypeCode();
                        io1.Why = "Unknown operation";
                        throw io1;
                }
                return val;
            }
            public string greeting()
            {
                return "Hello world of RPC using thrift from c# service";
            }
        }
    }
}

And the HelloClient.java is below:

package hello.client;

import hello.HelloService;
import hello.InvalidOperation;
import hello.Operation;
import hello.Work;

import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;

public class HelloClient {
  public static void main(String[] args) {
    try {

      TTransport transport = new TSocket("localhost", 34568);
      TProtocol protocol = new TBinaryProtocol(transport);
      HelloService.Client client = new HelloService.Client(protocol);

      transport.open();

      client.ping();
      System.out.println("ping()");
      
      String str = client.greeting();
      System.out.println(str);
      
      int sum = client.add(1, 1);
      System.out.println("1+1=" + sum);

      Work work = new Work();

      work.op = Operation.DIVIDE;
      work.num1 = 1;
      work.num2 = 0;
      try {
        int quotient = client.calculate(1, work);
        System.out.println("Whoa we can divide by 0");
      } catch (InvalidOperation io) {
        System.out.println("Invalid operation: " + io.why);
      }

      work.op = Operation.SUBTRACT;
      work.num1 = 15;
      work.num2 = 10;
      try {
        int diff = client.calculate(1, work);
        System.out.println("15-10=" + diff);
      } catch (InvalidOperation io) {
        System.out.println("Invalid operation: " + io.why);
      }

      transport.close();

    } catch (TException x) {
      x.printStackTrace();
    }
  }
}

Now run the service and then the client. The o/p is as follows:

ping()

Hello world of RPC using thrift from c# service

1+1=2

Invalid operation: Cannot divide by 0

15-10=5

Hope you find this useful. I did not find a direct C# service example but i could easily translate the Java server example to C#.

JSON-RPC between .NET service and Java client

2009-12-10T13:45:00.007-05:00

Recently i experimented with Jayrock .NET library for JSON-RPC and following its tutorial it was very simple to run a .NET JSON-RPC service on my Win7 Home Premium (IIS 7.5). From my earlier experience with JSON-RPC in Java, i wrote a Java client using json-rpc-client library and could get the RPC communication happening without any issue.

The ease with which the code samples worked for the below helloworld-ish sample, motivated me to write about it. JSON-RPC is simpler to debug and implement than SOAP and can be secured by implementing it over https. Most of the time in systems integration all we need is getting the .NET software to communicate with the Java software and in such simpler scenarios where RPC mechanism which is firewall friendly like SOAP is needed then thats where JSON-RPC is a good option.

helloworld.ashx - the JSONRPCHandler


using System;
using System.Web;
using Jayrock.Json;
using Jayrock.JsonRpc;
using Jayrock.JsonRpc.Web;


namespace jayrockserv
{
    public class HelloWorld : JsonRpcHandler
    {
        [JsonRpcMethod("greetings")]
        public string Greetings()
        {
            return "Welcome to Jayrock!";
        }

        [JsonRpcMethod("add")]
        public int sum(int a, int b)
        {
            return (a+b);
        }
    }

}

The Java JSON-RPC client:


Main.java:

import org.apache.commons.httpclient.HttpState;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.codebistro.jsonrpc.Client;
import org.codebistro.jsonrpc.HTTPSession;
import org.codebistro.jsonrpc.TransportRegistry;

public class Main {
 private static final Log log = LogFactory.getLog(Main.class);
 private static final String rootURL= "http://localhost/jayrock/helloworld.ashx";
 /**
  * @param args
  */
 public static void main(String[] args) {
  // Register HTTP
  HTTPSession.register(TransportRegistry.i());
  HTTPSession httpSession= (HTTPSession)TransportRegistry.i().createSession(rootURL);
  HttpState state= new HttpState();
  httpSession.setState(state);
  Client client= new Client(httpSession);
  RpcIF rpc = client.openProxy(null, RpcIF.class);
  String str = rpc.greetings();
  log.debug("Output:" + str);
 }

}


RpcIF.java:
public interface RpcIF {
 String greetings();
}

O/P on client side:


JSON RPC REQUEST ==> {"id":1,"method":"greetings","params":[]}
JSON RPC RESPONSE ==> {"id":1,"result":"Welcome to Jayrock!"}

No screens found error for X server in Ubuntu Jaunty

2009-10-17T13:04:00.003-04:00

I recently encountered an error when all of a sudden after a reboot X server will not start up.

It stated the following:

(EE) intel(0): No kernel modesetting driver detected. (EE) Screen(s) found, but none have a usable configuration.

My configuration at the time of this error was:

9.04 x64, kernel 2.6.28-15-generic, with intel GM965 graphics running xserver-xorg-video-intel:  2:2.9.0~git20091007.03e8e64f-0ubuntu0tormod3~jaunty

First i selected recovery mode to boot into last good configuration.

I struggled with it for a while trying some of the suggestions i could find on google searches. In the process i tried upgrading to the linux kernel 2.6.30-020630 and tried enabling the KMS (Kernel Mode Setting) for Jaunty as described here and that fixed the issue.

Enabling RDP on Windows Vista Home Premium 64 Bit (SP2)

2009-09-27T11:17:00.003-04:00

I found the hack mentioned in this blog http://andrewblock.net/?p=411 work for me pretty well. Am able to RDP from Win XP Pro to my Vitsa Home Premium 64 Bit SP2 with no issue so far.

Update: For enabling RDP on Windows 7 Home Premium, refer to this forum post. It worked for me.

Some useful Eclipse shortcut keys

2009-08-05T15:08:00.000-04:00

Cnrtl + O - show all methods and properties of the current editor's class.
Cntl + Shift + O - fix import statements.
Cntl + Shift + F - format code.
Cntl + 1 - show quick fix recommendations.
Cntl + F6 - toggle between editors
Cntl + E - show all open editors
Alt + left/right arrow - move to last edit location
Cntl + M - toggle maximize and minimize current editor
Cntl + J - incremental search (searches as you type)
Cntl + Shift + J - incremental backwards search.
Cntl + K - find next
Cntl + Shift + K - find previous
Cntl + Shift + L - Shows currently defined shortcut keys (same as Help > Key Assist).
Cntl + L - go to line.
Cntl + Shift + G - generate getters/setters.
Cntl + Alt + G - find text in workspace.

Eclipse 3.5 Galileo and clearcase plug-in

2009-07-23T21:09:00.005-04:00

I installed the Eclipse Galileo recently and found that the Rational Clearcase plugin by IBM does not work anymore in 3.5. Of course the latest plug-in explicitly mentions its just for Eclipse 3.4. But i tried anyways and it did not work. Fortunately, there is a clearcase plugin that works with Eclipse 3.5 at this point (http://eclipse-ccase.sourceforge.net/documents/user_guide.pdf). May be soon IBM clearcase plug-in will also be released that is compatible with Eclipse 3.5.

Updated: IBM Rational clearcase SCM Adapter version is available too now for Eclipse 3.5.

Upgrade to Ubuntu 9.04

2009-04-24T19:06:00.002-04:00

I was pleasantly surprised to see the dist upgrade to Ubuntu 9.04 from the previous 8.10 installation going well. It did not work so well last time during upgrade from 8.04 to 8.10. It crashed the root file system for me then. But the lazy me, hoped that it will work this time and it did and upgrade went fine. Web is already replete with praises for jaunty release.

Installing Subversion 1.5.3 on windows

2009-03-16T12:28:00.001-04:00

Steps are:

Download subversion installer from http://subversion.tigris.org.
Install at location say C:\svn.
At cmd prompt, create a repository:

svnadmin create "C:\svnrepo"

Edit C:\svnrepo\conf\svnserve.conf and uncomment the following lines:

anon-access = read
auth-access = write
password-db = passwd

Edit C:\svnrepo\conf\passwd file:add users in the format username = password one per line and remove the default users.
Set environment variable SVN_EDITOR to your favorite text editor (say Textpad.exe).
Set Subversion as a service with the following command at cmd prompt:
- sc create Subversion binpath= "c:\svn\bin\svnserve.exe --service --root c:\svnrepo --listen-port 9999" displayname= "Subversion Repo" depend= Tcpip start= auto
- Go to service control manager to start the Subversion service or type the following on :
```
net start “Subversion Repo”
```

Import an existing project in subversion repo:

svn import C:\myproject svn://localhost:9999/myproject

After importing data, note that the original tree is not under version control. To start working, you still need to svn checkout a fresh working copy of the tree.

svn checkout svn://localhost:9999/myproject

Install 32 or 64 bit version of tortoise SVN from http://tortoisesvn.net/. After installing you will need to reboot your system. Tortoise SVN is a windows shell extension which makes the use of SVN from windows explorer very easy and intuitive.

There's a standard convention in Subversion to start with the "TTB folders" at the root of any project:

Because Subversion uses regular directory copies for branching and tagging (see Chapter 4, Branching and Merging), the Subversion community recommends that you choose a repository location for each project root -- the "top-most" directory which contains data related to that project -- and then create three subdirectories beneath that root: trunk, meaning the directory under which the main project development occurs; branches, which is a directory in which to create various named branches of the main development line; tags, which is a collection of tree snapshots that are created, and perhaps destroyed, but never changed.

So make it a practice to add the trunk, tags and branches directories to your project and all your project files will go under trunk then.

Introduction to Vim for Programmers

2009-02-16T21:55:00.007-05:00

Vim editor:

It has 3 modes:

Ex mode <----> Command mode <-----> Insert mode.

You can never go from Insert to Ex mode directly or vice versa. You always
first need to go to command mode. Enter command mode by typing Esc and enter Ex mode by typing :.

For cursor movement:


   k
h      l
   j

Cursor movement commands: {motion} commands:

G - Last line in file
gg - First line in file
% - Jump to matching brace (()}{[])
m{1etter} - example ma to mark a line. There can be at
max 26 marks in the file.
`{mark} - example `a to jump to the named mark.
`` - Jump to the last line where you jumped or searched from.
/expr - forward search where expr = is a regex.
?expr - backward search.
n - repeat last search in the same direction.
N - repeat last search in the opposite direction.
w - move cursor forward by one word.
b - move cursor backward by one word.

Some more cursor movement commands:

Cntl + F - move one screen forward.
Cntl + B - move one screen backward.
f{char} - position cursor at the matching {char} to the right.
t{char} - position cursor just before the matching {char} to the right.
F{char} - position cursor at the matching {char} to the left.
T{char} - position cursor just before the matching {char} to the left.

Basic insert commands:

i - enter insert mode.
a - enter insert mode after character under cursor.
I - enter insert mode starting before first non blank character on the line.
A - enter insert mode starting after last non blank character on the line.
o - enter insert mode starting on a new line after current cursor line.
O - enter insert mode starting on a new line before current cursor line.

Change commands:

C - delete everything from cursor to end of line.
c{motion} - delete everything from cursor position as determined by the motion (one of the cursor movement commands).
Example:
c5{h} - deletes 5 chars to the left (as h is for
left movement).
c2w - deletes 2 words and puts in insert mode.
S - Substitute/change current line entirely. Delete all text on current line and put in insert mode.

Patterns for using change commands:

2cw - change 2 words.
cta - delete everything upto next occurrence of letter a.
5cta - delete everything upto 5th occurrence of letter a.
5S - delete current line and subsequent 4 lines.

Miscellaneous commands:

u - undo
Cntl + R - redo
Cntl + N/Cntl + P - autocompletion with text that already exists in the in the document.
. - repeat the last change.
zt - redraw the screen placing current line to top of screen.
zz - redraw the screen placing current line to the center of screen.
zb - redraw the screen placing current line to the bottom of screen.

Delete, cut and paste commands:

D - Delete from current cursor position to the end of line.
d{motion} - delete from current curson position to the position determined by {motion} command. Example,
d5w - delete next 5 words.
dn - delete till next matching search pattern.
d`a - delete till marked line named a.
y{motion} - yank or copy from current cursor position to the position
determined by {motion} command. Example,
y5w - copy next 5 words.
yG - copy till the end of file.
p - put the character in registry/clipboard after current character under cursor.
P - put the character in registry/clipboard before current character under cursor.
dfa - delete until character first character a from current cursor position.
dta - delete upto character first character a from current cursor position.
yy4p - copy current line and paste it 4 times.
dG - delete everything from current line till end of file.
y50G - copy everything from current line till line 50.
d5l - delete 5 characters to the right (l is for right motion).
5dd - delete 5 lines including current one.

Ex mode:
Ex mode commands are of the form:

:[firstline][,lastline]command

Example:

:25,30d - delete lines from 25 to 30.

Line numbers are optional.
If no line number is specified, command applies to current line.
If only one line number is specified, command applies to that line.
To affect a range of lines, we need to specify both firstline and lastline.

Shortcuts for specifying line numbers in Ex mode:

. - current line number. Example- :.,50d - delete from current to line 50.
$ - last line number in file.
% - entire file - same as specifying 1,$.
`a - line number of bookmarked line a.
/expr/ - next line that matches expr.
?expr? - previous line that matches expr.
\/ - next line that matches the most recent expr.
\? - prev line that matches the most recent expr.
\& - next line that matches the most recent substitution.

Examples:
:/Begin/,/End/d - deletes lines from first line that has word Begin till
last line which has word End.

10,40w bkup.dat - saves the lines 10 to 40 in a file bkup.dat.

Essential Ex Commands:

:w {filename} - save the current buffer to filename.
:q - quite the current buffer.
:x - same as :wq - quite and save unsaved data.
:e {filename} - open named file for editing in a new buffer.
:d - delete current line or range of lines
:map {a} {b} - remap the keys used in command mode.
:set {arguments} - configure vim (mostly to be used in .vimrc)
:help - vim help.

Search and Replace:

:s/search/replacement/flags

If search or replacement string contains / character then either we can
escape it with a \ or use a different delimiter (like #) other than / for
eg:

:s#/usr/share/file1#/usr/share/file2/#

Usage examples:

first occurrence single line :s/bash/perl/
all occurrences single line :s/bash/perl/g
first occurrence between line range: :23,100s/bash/perl/
all occurrences between line range: :23,100s/bash/perl/g
first occurrence in whole text: :%s/bash/perl/
all occurrences whole text: :%s/bash/perl/g

Code browsing:
Run ctags as follows in the source directory and it generates an index file
called tags of source files in current directory and its subdirectories.

ctags -R

Now in vim we can use each source file as a web page with each symbol
(variable, function) having hyperlinks, such that, if you do a cntl + ] on
the current symbol then it will take you to the definition of the symbol.

Following are commands for browsing code in vim:
Cntl + ] - jump to the tag under cursor (ie go to definition of function,
class, declaration of variable).
Cntl + T - Return to current tag from most recent jump off point.
:pop - Same as Cntl + T except that we can specify count to go back
multiple levels.
:tn - Jump to next match (when tag produces more than one match in
case of C++ overloaded functions).
:tp - Jump to previous match (in case of multiple matches).
:ts name - Shows a list of matching tags you can select from.
:tags - Show current tag stack with one line for each tag.

Lastly, to build c/c++ code:
:make {args} - invokes the make program. We can change the make program in to say ant by using Ex mode (or in .vimrc) as
:set makeprg=ant

One nice .vimrc can be found at:
http://research.iiit.ac.in/~masatran/config/.vimrc

Use

wget http://research.iiit.ac.in/~masatran/config/.vimrc

to get it.

A short introduction to GNU make

2009-02-15T03:19:00.008-05:00

A make rule looks like the following:

target: prerequisite
commands

Example:

.PHONY: all
all: program1 program2
program1: a.o b.o
gcc -o program1 a.o b.o
program2: c.o d.o
gcc -o program2 c.o d.o

This tells make not to search for a file named all and to assume that this target
is always obsolete.

Defining variables:

Variables in a make file are like in an ant build script but they
can only be set once unlike ant properties and dont change value.

FLAGS = first
all:
@echo FLAGS=$(FLAGS)

FLAGS = second
other:
@echo FLAGS=$(FLAGS)

The FLAGS variable is assigned the last value which is set in the build file
viz. second and so even if we run all target, the value echoed for FLAGS
will be second.
* So, a variable should be set just once in the make file. *

VAR = value
VAR := value

The := form allows variables to reference themselves without recursion.

VAR = value

# Wrong! Causes infinite recursion
VAR = $(VAR) more

# Okay, the := prevents recursion
VAR := $(VAR) more

# A GNU extension that does the same thing
VAR = value
VAR += more

make automatically removes leading and trailing white space from your
variable values when you use the traditional syntax.

TRAD= Hello World
all:
@echo "$(TRAD)" -- removes the spaces in the o/p

If we want to preserve the spaces:

TRAD=$() Hello World $()

we can use the built-in empty variables $() to mark the beginning and end.

GNU make comes with many implicit rules to do almost everything you need.

.c.o:
$(COMPILE.c) $(OUTPUT_OPTION) $<

where,
COMPILE.c = $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c
CC = gcc

This says, “If you see a target with an .o extension, and there is no explicit
rulefor it, look for a file with the same base and the .c extension. If you find
it, run these commands.”

We can checkout such implicit rules and built-in variables by:

make -p

CFLAGS and others are not defined but make allows referencing not defined
variables and will replace them with empty strings.

We can set CFLAGS= -g to enable debugging.

Common variables used in implicit rules:

CC = gcc - C Compiler
CXX = g++ - C++ compiler.
CFLAGS = none - flags passed to C compiler.
CXXFLAGS = none - flags passed to C++ compiler.
CPPFLAGS = none - flags passed to C Pre Processor. Typical flags include -I, -D and -U.

Default suffix rules to create object code:

.c = $(CC) -c $(CPPFLAGS) $(CFLAGS)
.cpp, .cc, .C = $(CXX) -c $(CPPFLAGS) $(CXXFLAGS)
.s = $(AS) $(ASFLAGS) -- for assembley code.

Automatic Variables:
Their values can change in the context they are used. Regular variables can
also contain automatic variables.

Example:

build_C_program=$(CC) -o $@ $^

program1: mod1.o mod2.o
$(build_C_program)
program2: mod3.o mod4.o
$(build_C_program)

Note: implicit rule will be used to create the pre-requisites modx.o:

.c.o:
$(COMPILE.c) $(OUTPUT_OPTION) $<

then modx.o will be fed to the build_C_program to generate the programx
executables.

Useful automatic variables in GNU make:

$@ = target filename
$^ = all pre-requisites with duplicate names removed.
$+ = all pre-requisites with duplicate names.
$< = name of first pre-requisite in the rule.
$? = name of pre-requisite newer than target.
$* = base name of target file (without extension).

Manipulating variables with functions:

CSRCS=foo.c
CXXSRCS=foo.cpp
OBJS=$(CSRCS:.c=.o) $(CXXSRCS:.cpp=.o)

or use:

OBJS:=$(addsuffix .o, $(basename $(SRCS)))

shell function:

CURRENT_TIME=$(shell date +%T)
something:
@echo started $(CURRENT_TIME)

Other functions provided by GNU make are:

$(subst from, to, text)
$(patsubst from-pattern, to-pattern, filenames)
$(strip string) -- removes the whitespace chars.
$(findstring match, string)
$(filter patterns, filenames) -- returns files that match one or more space
separated patterns
$(sort text) - lexical order sort
$(word n, text) - nth word in text
$(error message) - fails the build after printing the message.

User can define custom functions which can be called as:

myfunction = @echo $(1) $(2)
all:
$(call myfunction,hello,world)

Conditionals:
ifeq, ifneq
ifdef, ifndef

ifeq ($(shell uname -o),GNU/Linux)
CPPFLAGS += -DLINUX
endif

Note: The text that appears inside the conditional clause may be any valid make lines, but conditionals may not be used inside the command section of rules or inside variable/function definitions.

To ensure that A_VITAL_VARIABLE is defined or else the build has to fail:

ifndef (A_VITAL_VARIABLE)
$(error A_VITAL_VARIABLE is not set)
endif

Generally, projects will have a Makefile for each project source directory.

A sample makefile:
The below makefile resides in the bin/ directory and compiles C and C++ sources in src/ directory. The executables are created in bin/ directory.

.PHONY: all


VPATH=../src
CFLAGS += -Wall -Wformat-nonliteral -Wformat-y2k 
build_C_program=$(CC) $(CFLAGS) -Wstrict-prototypes -o test $^
build_CXX_program=$(CXX) $(CXXFLAGS) -Weffc++ -o hello $^
CSRCS=test.c
CXXSRCS=hello.cpp
COBJS=$(CSRCS: .c=.o)
CPPOBJS=$(CXXSRCS: .cpp=.o)
CURRENT_TIME=$(shell date +%T)  

ifdef (debug)
        CFLAGS += -g
        CXXFLAGS += -g
else
        CFLAGS += -O2
        CXXFLAGS += -O2
endif


all: empty.so ccode cppcode 
        @echo started $(CURRENT_TIME);

ccode: $(COBJS)
        @echo Building C code; \
        $(build_C_program)

cppcode: $(CPPOBJS)
        @echo Building CPP code;\
        $(build_CXX_program)

empty.so: empty.c
        $(CC) -shared -fpic -o $@ $^

run: all
        LD_LIBRARY_PATH=../src ./test

clean:
        rm -f empty.so $(COBJS) $(CPPOBJS)

cleanall: clean
        rm -f $(basename $(COBJS) $(CPPOBJS))

VNCServer Configuration

2009-02-13T14:30:00.001-05:00

In the ~/.vnc/xstartup we can configure vnc session to start in gnome or kde, as shown below:

#!/bin/sh

# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
/usr/dt/bin/dtwm &
startkde &

#gnome-session &

Then start vnc server as:

$ vncserver

This will tell you the hostname:sessionnum and set the password:

$ vncpasswd

Now you are all set to connect with your favorite vnc client (i use realvnc).

Port Forwarding : Accessing your home computer from work

2009-02-09T04:27:00.004-05:00

In order to access the home ubuntu desktop from work, following is what i needed to do:

Installed open-ssh demon on ubuntu with: sudo apt-get install open-ssh
Tested that i could use a windows box on the home lan to access the sshd using putty.
Once sshd is setup, followed the instructions at http://www.portforward.com/help/pfprogression.htm, which is broadly the following:

First we need to setup static/manual IP on the host computer to which we want to forward the port(s) to. Mine was setup to obtain IP address using DHCP so i first changed that to manual.
Then login to the router (mine is an Air-link AR670W) and enable Port Forwarding for the SSH application port 22 for both TCP/UDP protocols and provided the static IP of my ubuntu desktop.
Saved the configuration and router reboots after that.

Then i noted down the ISP assigned public IP to my router's external interface and used that IP (external IP of my router) from a windows box to putty to. Since the SSH port 22 on router is configured to be forwarded to my ubuntu desktop, so i was allowed my way to the ubuntu desktop.

http://www.portforward.com/routers.htm has a listing of several routers and the documentation on how to configure that router for port forwarding.

Also checkout how to setup a easy to remember domain name for your router so that you dont have to remember your router IP (http://www.techthrob.com/tech/dyndns.php). This way even if your router is assigned a new external IP by your ISP, the ddclient demon will automatically update the mapping of your domain id to the new IP. The dyndns.com site has a script generator which can generate the ddclient.conf file for you. Use the generator and it will work without fail. You can find it here.

To configure ubuntu 8.10 for static IP, just use the System > Preferences > Network Configuration UI and select Manual for Method in IPv4 Settings tab and provide your static IP address (say 192.168.1.100), the netmask and the gateway (router ip - like 192.168.1.1). Also you may provide DNS servers (check your router's status page for knowing what DNS servers to use - it will automatically have got it from your ISP).

Extending Net-SNMP 5.4.2.1 Agent

2008-11-26T14:18:00.001-05:00

This article describes how to extend Net-SNMP agent toolkit to instrument the EtherLike-MIB (RFC 3635) with hardcoded defaults. This article will describe the setup on windows (I used XP SP3) and Linux (I used openSUSE 11.0) required to develop this agent.

The following instructions were carried out for net-snmp source version 5.4.2.1. This article assumes the following default paths:

Net-SNMP un-archived on windows in C:\net-snmp and ~/net-snmp on Linux.
Net-SNMP installed in C:\usr on Windows and /usr/local on Linux.

1.1. Setup the development environment

Download and install MSVC++ Express 2005 from http://www.microsoft.com/express/2005/download/default.aspx. There is some compilation issue of the net-snmp code with MSVC++ 2008 (Orcas) and i did not bother to fix it but used the 2005 instead.
Download and setup platform sdk as described in http://www.microsoft.com/express/2005/platformsdk/default.aspx. Be sure to setup the development environment using the PSDK as described in the link step-wise.
Download net-snmp source from http://www.net-snmp.org/download.html. The net-snmp version 5.4.2.1 can be downloaded from there.

1.2. Build and install net-snmp from source on windows

Un-archive the net-snmp source to say c:\net-snmp.
Follow the README.win32 to build the net-snmp code (Read the section Microsoft Visual C++ - Workspace – Building).
1. We need to first load the winsdk.dsw in the MSVC++ 2005 express. Below is the screenshot of all projects that make up the win32sdk workspace in MSVC++.

Figure 1 - Projects in win32sdk.dsw

Modify the win32\netsnmp\netsnmpconfig.h file to add:

               #define HAVE_WIN32_PLATFORM_SDK 1

Build in order, the following project for (Release | Debug) configuration.

i. libagent

ii. libhelpers

iii. libnetsnmptrapd

iv. libsnmp

v. netsnmpmibsdk

Go to Build > Batch Build… and select the (Debug | Release) configuration of all projects except for the ones you built in step c above.

If everything is okay with setup, then build should go fine too.

Using cmd line on windows, go to c:\net-snmp and run the command: win32\install_netsnmp.bat to install net-snmp you built above to location c:\usr.

Set the c:\usr\bin to PATH environment variable for easy access to the command line snmp client and agent executables.

1.3. Build and install net-snmp from source on Linux

Un-archive the net-snmp source to say ~/net-snmp.

Follow the INSTALL to build the net-snmp code. Run the following commands on shell prompt:
1. cd ~/net-snmp
2. ./configure
3. make
4. make install (run as root)

The make install step will install the net-snmp binaries by default in /usr/local path.

1.4. Configuring the snmpd agent (snmpd.conf)

Configure snmp agent by running snmpconf command (see man page) or copy the C:/net-snmp/Example.conf and modify.
1. snmpconf –g basic_setup

I used the following configuration. Copy-paste the text below into a file named snmpd.conf. The agent is configured for snmp v2c and with rwcommunity string of public.

      com2sec local     localhost       public

      group MyRWGroup    v1         local

      group MyRWGroup    v2c        local

      group MyRWGroup    usm        local

      view all    included  .1                               80

      #                context sec.model sec.level match  read   write  notif

      access MyROGroup ""      any       noauth    exact  all    none   none

      access MyRWGroup ""      any       noauth    exact  all    all    none

      syslocation Right here, right now.

      syscontact Me <me@somewhere.org>

Save the file in path :
1. C:\usr\etc\snmp\snmpd.conf (on Windows)
2. /etc/snmp/snmpd.conf (on Linux)

1.5. Load the MIB module and generate the C code

The EtherLike-MIB has 5 tables (columnar variables).

For mib2c code generation I used linux host. Run the following commands on shell prompt:

      mkdir ethmib_src

      cd ethmib_src 

      export MIBS=ALL

EtherLike-MIB was part of the net-snmp distribution so all we need is to tell mib2c to use it by setting the environment variable MIBS.

      mib2c –c mib2c.create-dataset.conf dot3StatsTable 
    

This will generate dot3StatsTable.c and dot3StatsTable.h files.

Similarly run the similar command for the other tables in the mib.

      mib2c –c mib2c.create-dataset.conf dot3PauseTable 

      mib2c –c mib2c.create-dataset.conf dot3HCStatsTable 

      mib2c –c mib2c.create-dataset.conf dot3CollTable 

      mib2c –c mib2c.create-dataset.conf dot3ControlTable

Use the ethmib_src and use it for windows agent extension too.

1.6. Use the generated source files to instrument the mib on Windows

On Windows, the above generated source files can be copied to c:\net-snmp\agent\mibgroups. You may also create a folder names EtherLike-MIB and sub-folders for each table like dot3StatsTable etc, and keep the respective table *.c and *.h files in the folder with the table name.

Add the generated sources in the MSVC++ netsnmpmibssdk project as shown below:

Figure 2 - EtherLike-MIB files added to the netsnmpmibssdk project.

Add the dot3StatsTable.h and dot3StatsTable.c files to your 'netsnmpmibssdk' project in VC++.

Next edit the '<sourcedir>\win32\mib_module_includes.h' file to add an include to your .h file.

      #include "mibgroup/EtherLike-MIB/dot3StatsTable/dot3StatsTable.h"

      #include "mibgroup/EtherLike-MIB/dot3HCStatsTable/dot3HCStatsTable.h"

      #include "mibgroup/EtherLike-MIB/dot3CollTable/dot3CollTable.h"

      #include "mibgroup/EtherLike-MIB/dot3ControlTable/dot3ControlTable.h"

      #include "mibgroup/EtherLike-MIB/dot3PauseTable/dot3PauseTable.h"

Next edit the '<sourcedir>\win32\mib_module_inits.h' file to add code to call your initialize function.

      if (should_init("dot3StatsTable")) init_dot3StatsTable();

      if (should_init("dot3HCStatsTable")) init_dot3HCStatsTable();

      if (should_init("dot3CollTable")) init_dot3CollTable();

      if (should_init("dot3ControlTable")) init_dot3ControlTable();

      if (should_init("dot3PauseTable")) init_dot3PauseTable();

1.7. Instrument the EtherLike-MIB

This section only instruments the dot3StatsTable generated source to return some default data. Other table instrumentation can be done in the similar manner.

Edit the dot3StatsTable.c file as shown below to return some hardcoded data. The changes to the generated code have been highlighted.

      /*

       * Note: this file originally auto-generated by mib2c using

       *        : mib2c.create-dataset.conf 9375 2004-02-02 19:06:54Z rstory $

       */

      #include <net-snmp/net-snmp-config.h>

      #include <net-snmp/net-snmp-includes.h>

      #include <net-snmp/agent/net-snmp-agent-includes.h>

      #include "dot3StatsTable.h"

      /** Initialize the dot3StatsTable table by defining its contents and how it's structured */

      void

      initialize_table_dot3StatsTable(void)

      {

          static oid dot3StatsTable_oid[] = {1,3,6,1,2,1,10,7,2};

          size_t dot3StatsTable_oid_len = OID_LENGTH(dot3StatsTable_oid);

          netsnmp_table_data_set *table_set;

          // variables declared – wrajnees

          netsnmp_table_row *row;

          static int _max_cols = 21; // there are 18 columns in this table.

          int column = 0;

          int index = 1;

          int val = 20;

          static oid      objid_etherchipset[] = { 0 };     /* ethernetChipset vendor oid */

          // end variables declared.

          /* create the table structure itself */

          table_set = netsnmp_create_table_data_set("dot3StatsTable");

          /* comment this out or delete if you don't support creation of new rows */

          table_set->allow_creation = 1;

          /***************************************************

           * Adding indexes

           */

          DEBUGMSGTL(("initialize_table_dot3StatsTable",

                      "adding indexes to table dot3StatsTable\n"));

          netsnmp_table_set_add_indexes(table_set,

                                 ASN_INTEGER,  /* index: dot3StatsIndex */

                                 0);

          DEBUGMSGTL(("initialize_table_dot3StatsTable",

                      "adding column types to table dot3StatsTable\n"));         

          netsnmp_table_set_multi_add_default_row(table_set,

                                                  /*COLUMN_DOT3STATSINDEX, ASN_INTEGER, 0,

                                                  NULL, 0,*/

                                                  COLUMN_DOT3STATSALIGNMENTERRORS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSFCSERRORS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSSINGLECOLLISIONFRAMES, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSMULTIPLECOLLISIONFRAMES, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSSQETESTERRORS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSDEFERREDTRANSMISSIONS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSLATECOLLISIONS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSEXCESSIVECOLLISIONS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSINTERNALMACTRANSMITERRORS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSCARRIERSENSEERRORS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSFRAMETOOLONGS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSINTERNALMACRECEIVEERRORS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSETHERCHIPSET, ASN_OBJECT_ID, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSSYMBOLERRORS, ASN_COUNTER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSDUPLEXSTATUS, ASN_INTEGER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSRATECONTROLABILITY, ASN_INTEGER, 0,

                                                  NULL, 0,

                                                  COLUMN_DOT3STATSRATECONTROLSTATUS, ASN_INTEGER, 0,

                                                  NULL, 0,

                                    0);

          /* registering the table with the master agent */

          /* note: if you don't need a subhandler to deal with any aspects

             of the request, change dot3StatsTable_handler to "NULL" */

          netsnmp_register_table_data_set(netsnmp_create_handler_registration("dot3StatsTable", NULL,

                                                              dot3StatsTable_oid,

                                                              dot3StatsTable_oid_len,

                                                              HANDLER_CAN_RONLY),

                                  table_set, NULL);

          // Add code - wrajnees

          /*

           * create the a row for the table, and add the data 

           */

          row = netsnmp_create_table_data_row();

          /*

           * set the index to the IETF WG name "snmpv3" 

           */

          netsnmp_table_row_add_index(row, ASN_INTEGER, (u_char*)&index, sizeof(index));

          /*

           * set the column 2 and above

           */

          for (column = 2; column <= _max_cols; column++) {

              // Following columns are not valid.

              switch(column) {

                  case 12: // INVALIDs

                  case 14:

                  case 15:

                      break;

                  case 17: // OID

                      netsnmp_set_row_column(row, column, ASN_OBJECT_ID,

                                     (u_char*)objid_etherchipset, 1*sizeof(oid));

                      break;

                  case 19: // INTEGER

                  case 21:

                      netsnmp_set_row_column(row, column, ASN_INTEGER,

                                     (u_char*)&val, sizeof(val));    

                      break;

                  case 20: // TRUTH_VALUE

                      netsnmp_set_row_column(row, column, ASN_INTEGER,

                                     (u_char*)&val, sizeof(val));    

                      break;

                  default: // COUNTER

                      netsnmp_set_row_column(row, column, ASN_COUNTER,

                                     (u_char*)&val, sizeof(val));    

                      break;

              }

          }

          /*

           * add the row to the table 

           */

          netsnmp_table_dataset_add_row(table_set, row);

          /*

           * Finally, this actually allows the "add_row" token it the

           * * snmpd.conf file to add rows to this table.

           * * Example snmpd.conf line:

           * *   add_row netSnmpIETFWGTable eos "Glenn Waters" "Dale Francisco"

           */

          netsnmp_register_auto_data_table(table_set, NULL);

          // End add code - wrajnees

      }

      /** Initializes the dot3StatsTable module */

      void

      init_dot3StatsTable(void)

      {

        /* here we initialize all the tables we're planning on supporting */

          initialize_table_dot3StatsTable();

      }

Now rebuild the netsnmpmibssdk project and then snmpdsdk project, in order.

Run the following command to re-install the modified agent with the dot3StatsTable changes.

      cd c:\net-snmp 

      win32\install_netsnmp.bat

Start the new agent as follows:

      cd c:\usr\bin 

      snmpd.exe –f –Lo –V

Open a MIB Browser to walk the dot3StatsTable instrumentation:

1.8. Instrumentation on Linux

Copy the generated source to the path ~/net-snmp/agent/mibgroups and run the following commands:

      ./configure –with-mib-modules=”dot3StatsTable dot3PauseTable dot3HCStatsTable dot3CollTable dot3ControlTable” 

      make 

      make install (run as root)

Start the snmpd as:

      /usr/local/sbin/snmpd –f –Lo –V 
    

Run a command line snmpwalk to test it.

      $ snmpwalk -v 2c -mAll -c public localhost dot3StatsTable 

      EtherLike-MIB::dot3StatsAlignmentErrors.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsFCSErrors.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsSingleCollisionFrames.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsMultipleCollisionFrames.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsSQETestErrors.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsDeferredTransmissions.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsLateCollisions.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsExcessiveCollisions.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsInternalMacTransmitErrors.1 = Counter32: 20 

      EtherLike-MIB::dot3StatsCarrierSenseErrors.1 = Counter32: 20

2. Co-existance with MS Windows SNMP Agent

We have 2 approaches to getting the net-snmp agent to co-exist with the Microsoft provided SNMP agent:

As of Net-SNMP 5.4, the Net-SNMP agent is able to load the Windows SNMP service extension DLLs by using the Net-SNMP winExtDLL extension. In this scenario, MS SNMP agent is installed but disabled. This is required because winExtDLL extension and existing windows extensions use Windows SNMP API from snmpapi.dll. The limitations of this approach are:

linkUp/Down generic traps are not received for some unknown reason.

sysUpTime.0 does not report the correct uptime for the agent. This is because the Windows extension checks for the uptime of the SNMP service, which is not running when Net-SNMP is running.

Alternatively, Net-SNMP agent can run as a proxy SNMP agent and will proxy for the MS SNMP agent (running at a non-default port) for those MIBs that MS SNMP agent instruments. For all other MIBs, Net-SNMP agent can service the SNMP requests. This approach does not suffer from the winExtDLL approach’s limitation, but then we need to run two snmp agents (net-snmp and MS snmp agents).

This sums up in short, the development setup required to get started with extending the net-snmp agent.

SCDJWS 1.4 Study Guide | Pothi.com

2008-10-22T14:10:00.001-04:00

SCDJWS 1.4 Study Guide | Pothi.com

Posted using ShareThis

Vim quick reference

2008-08-17T14:15:00.005-04:00

The following content is excerpted from http://linuxconfig.org/Vim_Tutorial. It covers most of the essentials that one needs while editing code. One thing which i know and use and is not covered is :e < filename > to open multiple files without exiting vim and then navigating among the open files using :b < num > , where num is file buffer number for the open files . Other than that, there were quite a few points in the tutorial that were really good and i did not use/know them before. Also see other good links to learn more on vi/vim at http://delicious.com/rwatsh/vi.

Vim Tutorial Summary

VIM novice level Summary

Moving around with cursor:
h key = LEFT, l key = RIGHT, k key = UP, j key = DOWN
Exiting vim editor without saving:
press ESC to get into command mode, enter :q! to exit.
Deleting characters in vim command mode:
delete with x key

Inserting / appending text:
Press i or a in command mode and type
Saving changes and exit:
in command mode :wq or SHIFT+zz

VIM Operators and Motions summary

Deleting words:
delete word with d operator and w or e motion
Deleting to the end of the line:
delete to th end of the line with d operator and $ motion
Using operators, motions and counts:
beginning of th line 0, end of the line $, end of the 2nd word 2e beginning of the 4th word 4w

Deleting multiple words:
to delete 3 words you would use d3w
Deleting lines:
to delete single line dd, delete n lines ndd
Undo changes:
undo changes with u

VIM apprentice user summary

Paste command:
paste your cache memory with p command
Replace characters:
rt replace current character with t

Change characters:
ce to change single word, c$ to change to the end of the line

VIM experienced user summary

Advanced Navigation:
end of the file G, begging of the file gg or 1G, to get on line n use nG
instruct vim display file information CTRL+g
Search text with vim:
search forward /, search backward ?, next search n , previous search N

Vim Substitution :
first occurrence single line :s/bash/perl/
all occurrences single line :s/bash/perl/g
first occurrence between line range: :23,100s/bash/perl/
all occurrences between line range: :23,100s/bash/perl/g
first occurrence in whole text: :%s/bash/perl/
all occurrences whole text: :%s/bash/perl/g

VIM veteran user summary

Execute external commands on shell from vim:
:!ls will execute ls command on your shell
Writing to files advanced::w saves current file without quit, :w bash.sh whites to file bash.sh

Highlight text ans save to different file:highlight text with v operator and save it with :w
Retrieve text from different file::r will retrieve content of file

VIM expert user summary

Using o operator:
:o insert line bellow you cursor, O inserts line above your cursor
Copy and paste:
yank line with y and paste it with p

Customize vim's environment:
edit ~/.vimrc file to customize vim's environment

Free Java Programmer's Test

2008-04-18T04:14:00.004-04:00

I stumbled on the site http://www.betterprogrammer.com today and found that it offers an open book programming test meaning you can use all reference (like one can do at work) but complete 5 programming exercises pertaining to use of core Java language for writing algorithms. It requires lots of Java collections framework usage and writing some recursion algorithms, writing tree traversal code, calculating prime numbers and stuff like that. The time limit is not very strict and it even allows you to take a break and then continue with the test. In the end you are given a free certificate based on percentile (which it states will keep changing - read may improve - with time, as more people take this free test). The questions were good quality and if you have some free time and like to write code then its quite a good avenue to spend some time on.

My certificate is here: http://www.betterprogrammer.com/certificate/BP1Q9GB5

They claim to maintain this certificate for you in their database and that you can retake the tests to improve (not sure if the questions remain the same - if they dont change then thats a bummer) but anyway take it once and you will love it. If you have taken this test already then feel free to leave your comments on how you liked it or discuss the questions you liked/found interesting.

Windows Vista SP1 Released Today

2008-03-19T02:31:00.002-04:00

Get it from http://www.microsoft.com/downloads/details.aspx?FamilyID=b0c7136d-5ebb-413b-89c9-cb3d06d12674&DisplayLang=en.

I had the RC1 installed and so when i tried installing the SP1 release then it gave me an error that SP1 is already installed.

So, you must remove all previous version of SP1 before upgrading to the next.

Control Panel > Programs and Features > Installed Updates (upper left corner
under Tasks) > Highlight "Service Pack KB936330" > Uninstall

It will take awhile, there will be 2-3 reboots. Then you can install the
new version.

Follow the above steps and then go to the given link above (for MS download center for Vista SP1) and download and install the SP1 here. To know the features of SP1 please refer to this article.

Using SAP Memory Analyzer for Java memory leak detection

2008-03-18T07:32:00.003-04:00

[Click to watch video of using SAP Memory Analyzer]

I recently had opportunity to use the SAP Memory Analyzer tool for analyzing an HPROF heap dump generated on OutOfMemory error in code. I found that its an excellent and user friendly tool for job of memory leak analysis with good documentation. Though i generally preferred to use JHAT for heap walking but i realized that this SAP tool is much more intuitive and memory leak detection in code was a breeze. The above video tutorial talks about using the version 1.1.1 (which happens to be the latest at this time). Highly recommended.

Emma vs Cobertura

2008-03-13T04:56:00.003-04:00

The 2 open source Java code coverage tools that are the best among the lot are cobertura and emma. Both have their own pros and cons.

Cobertura - http://cobertura.sourceforge.net/
vs
Emma - http://emma.sourceforge.net/

Points borrowed from: http://raibledesigns.com/rd/entry/emma_vs_cobertura_for_code
Video : http://video.google.com/videoplay?docid=820584080702226910

Emma:
1. Stats on both class and method coverage
2. Partial/fractional line coverages is unique trait - shown in yellow when there are multiple conditions in a conditional block like if (x < 0 and x > 10) and say x > 10 never gets executed this is shown in yellow. This is important feature which lets us determine if the tests cover all conditions of such more than one conditions conditional blocks.
3. Not being actively developed.
4. Stricter code coverage.
5. Integration with Eclipse available - http://www.eclemma.org/
6. Better documentation than cobertura.
7. Instrumentation process is faster than cobertura.
8. Standalone library and does not have any external dependencies.
9. Common public license 1.0 friendlier that GPL.

Cobertura: (since 2002)
1. GPL'd version of JCoverage (which is commercial). Project older than Emma.
2. Prettier reports.
3. Actively developed.
4. Branch/block and line coverages only - no class or method level coverage.
5. How many times a line has been executed - unique about cobertura.
6. <cobertura-check> where one can specify percentage of coverage that's a MUST or else build fails.
7. Data merge feature - good for QA labs... for merging coverage data to prepare historical trend graphs. Emma also supports it now but it seems its better with cobertura. Project long coverage collection possible.
8. Depends on other third party libraries.

Common factors in both of these code coverage tools:
1. bytecode instrumentation.
2. reports are filterable so you can tell what needs to be evaluated for code coverage.
3. offline instrumentation (most recommended approach) - separate instrument/execute/report tasks – this is what we adopted. The other approach is on-the-fly instrumentation.
4. ant integration.
5. testng integration.

The above information can be used in deciding about the right tool for your project. We went for Emma as it seemed to have good enough reports and was fast. I did not get a chance to experiment with Cobertura but will surely try it out soon.

Using TestNG with Emma for automating test code coverage report generation

2008-03-13T04:39:00.007-04:00

TestNG and Emma can be used together to automate the generation of code coverage report after every test run in the ANT builld script.
Following ant build script snippet shows how both tools are used together for this important metrics collection.

To run it:
$ ant emma test

This will generate the code coverage report for the TestNG tests in ./coverage directory (where, basedir = ".").



<property name="coverage.dir" value="${basedir}/coverage" />


<property name="emma.dir" value="${lib}/emma" />


<path id="emma.lib">
<pathelement location="${emma.dir}/emma.jar" />
<pathelement location="${emma.dir}/emma_ant.jar" />
</path>


<taskdef resource="emma_ant.properties" classpathref="emma.lib" />

<property environment="env" />
<path id="classpath">
<fileset dir="${lib}">
<include name="**/*.jar" />
</fileset>
</path>

<target name="emma" description="turns on EMMA's instrumentation/reporting">
<property name="emma.enabled" value="true" />


<property name="emma.filter" value="" />
</target>


<taskdef name="testng" classpathref="classpath" classname="org.testng.TestNGAntTask" />



<target name="test" description="execute testng tests" depends="dist">

<emma enabled="${emma.enabled}" verbosity="verbose">
<instr instrpath="${build}" mode="overwrite" metadatafile="${coverage.dir}/metadata.emma">


<filter value="${emma.filter}" />
</instr>
</emma>

<javac srcdir="${test.src.dir}" destdir="${build}" classpathref="classpath" deprecation="${compile.deprecation}" />

<testng classpathref="test.classpath" outputDir="${testng.report.dir}" sourcedir="${test.src.dir}" haltOnfailure="true">
<xmlfileset dir="${test.src.dir}" includes="testng.xml" />
<jvmarg value="-Demma.coverage.out.file=${coverage.dir}/coverage.emma" />
<jvmarg value="-Demma.coverage.out.merge=false" />
</testng>


<emma enabled="${emma.enabled}">
<report sourcepath="${src}" sort="+block,+name,+method,+class" metrics="method:70,block:80,line:80,class:100">

<fileset dir="${coverage.dir}">
<include name="*.emma" />
</fileset>


<txt outfile="${coverage.dir}/coverage.txt" depth="package" columns="class,method,block,line,name" />
<xml outfile="${coverage.dir}/coverage.xml" depth="package" />
<html outfile="${coverage.dir}/coverage.html" depth="method" columns="name,class,method,block,line" />
</report>
</emma>

</target>

Experience using apache commons EqualsBuilder class

2008-01-09T17:48:00.000-05:00

Following are my learnings on how to use the EqualsBuilder class in apache commons lang library.

C:\Work\EqualsProto\src\equalsproto\Main.java


package equalsproto;

//~--- non-JDK imports --------------------------------------------------------

import java.util.Arrays;
import org.apache.commons.lang.builder.EqualsBuilder;


class A
{
   private String s = "Watsh";
   private int    x = 10;
   private float  y = 20.2f;

   //~--- constructors --------------------------------------------------------

   A() {}
   
   A(int i, int i0, String string)
   {
      this.x = i;
      this.y = i0;
      this.s = string;
   }

   //~--- methods -------------------------------------------------------------

   /**
    * Method description
    *
    *
    * @param obj

    *
    * @return
    */
   @Override
   public boolean equals(Object obj)
   {
      if (obj instanceof A == false) {
         return false;
      }

      if (this == obj) {
         return true;
      }

      A rhs = (A) obj;

      /** Note:

       * Do not use appendSuper when the super class is java.lang.Object as 

       * default implementation of equals in Object class will return true only

       * when two references are pointing to the same object instance and hence

       * the effect is not desirable.
       */
      return new EqualsBuilder().append(s, rhs.s).append(x,
         rhs.x).append(y, rhs.y).isEquals();
   }
}



class B 
{
   private String z = "Rajneesh";
   private A      a;
   private A[]    array;

   //~--- constructors --------------------------------------------------------

   B(String z, A a, A[] array)
   {
      this.z = z;
      this.a = a;
      this.array    = array;
   }
   
    @Override
   /**
    * Learning: 
    * 1. appendSuper() should not be used as it then calls super.equals()

    * for java.lang.Object class which will return true only when both lhs and 

    * rhs references point to the same object instance and hence will return false

    * when the 2 object instances are different but meaningfully equivalent.

    * 
    * 2. To compare arrays, you will either need to use the 

    * EqualsBuilder.reflectionEquals() approach or if you are using the 

    * EqualsBuilder.append() approach then append(array1, array2) calls 

    * array1.equals(array2) which will only do a shallow comparison for the 

    * 2 arrays involved. So in such a case, you must use Arrays.deepEquals() for 

    * all array members of your class and once that equality is met you can use

    * EqualsBuilder.append() for rest of the non-array instances.

    * 
    * NOTE: I have not tested for how this approach works for Collection classes.

    */
   /*public boolean equals(Object obj)
   {
      if (obj instanceof A == false) {
         return false;
      }

      if (this == obj) {
         return true;
      }

      B rhs = (B) obj;

      return new EqualsBuilder().append(z, rhs.z).append(a,

         rhs.a).append(array, rhs.array).isEquals();
   }*/
   public boolean equals(Object obj) {
        return EqualsBuilder.reflectionEquals(this, obj);
   }
}



/** Testing the use of EqualsBuilder.
 *
 * @author wrajnees

 */
public class Main
{

   /**
    * @param args the command line arguments

    */
   public static void main(String[] args)
   {
        A a1 = new A(2, 43, "xxx");
        A a2 = new A(2, 43, "xxx");
        if (a1.equals(a2)) {
            p("a1 equals a2");
        } else {
            p("a1 not equals a2");
        }
        
        A[] array1 = new A[2];
        A[] array2 = new A[2];
        
        array1[0] = a1;
        array1[1] = new A();
        
        array2[0] = a2;
        array2[1] = new A();
        
        // comparing arrays

        if (Arrays.deepEquals(array1, array2)) {
        //if (array1.equals(a2)) { // -- does not work
            p("arrays are equal");
        } else {
            p("arrays arent equal");
        }
       
        // comparing more complex object with containment and array

        B b1 = new B("Test", a1, array1);
        B b2 = new B("Test", a2, array2);
        if (b1.equals(b2)) {
            p("b1 equals b2");
        } else {
            p("b1 not equals b2");
        }        
   }
   
   private static void p(String s) {
       System.out.println(s);
   }
}

Detecting memory leaks in Java SE

2008-01-08T01:23:00.001-05:00

Today i have learned about a nice approach to detecting memory leaks in the Java SE applications using the jmap and jhat (Java Heap Analysis Tool). The process to be followed is described below:

1. Run your application.

2. Run the command jps to know the process id of the J2SE application you ran.
% jps
1234 MyApp
...

3. Perform those actions in your application which you feel will cause the memory leak. You can observe the real time heap usage plot in jconsole. Launch jconsole and select your application in it to connect to.
% jconsole

4.Then run the command jmap to dump the heap.
% jmap -dump:file=myapp.bin 1234

This will produce a heap dump in myapp.bin file with the heap profile.

4. Run the JHAT (Java Heap Analysis Tool) as follows:
% jhat -J-mx512m heap.bin

The above command starts a small Http server at port 7000 by default.

5. Browse to http://localhost:7000 and you will have you the heap browser - a hyperlinked set of pages from where you can trace every object allocated and who all reference the object at the point at which the heap dump was created.

6. The important pages to browse to are:
http://localhost:7000/histo/ - to see the histogram of heap usage.
http://localhost:7000/showInstanceCounts or http://localhost:7000/showInstanceCounts/includePlatform/ to see biggest types with most object allocations (ie instances).

and some advanced features of using SQL to query values of instance members:
http://localhost:7000/oql/

7. So browse to http://localhost:7000/showInstanceCounts/. Investigate "Instances" and not "Classes". Use “Reference Chains from Rootset” (Exclude weak refs!!!) to see who’s holding the instance. This tip i found in one of the links below in the reference section and it really was what was required to find the memory leaking code.

Some good references are:

TestNG versus JUnit4

2007-12-03T13:44:00.000-05:00

Comparing JUnit 4 and TestNG 5.7

Excerpts from http://www.ibm.com/developerworks/java/library/j-cq08296/ by Andy Glover.

JUnit is geared more towards unit testing - testing an object class in isolation.
TestNG provides more features and flexibility to facilitate its use not only for unit but integration, regression, functional, acceptance testings etc.

1. The setup method (annotated with @BeforeClass) needs to static and public with JUnit 4 but thats not required by TestNG. Thus TestNG is more flexible of the two.

2. Dependency testing:
Unlike JUnit, TestNG welcomes test dependencies through the dependsOnMethods attribute of the Test annotation. With this handy feature, you can easily specify dependent methods, which will execute before a desired method. What's more, if the dependent method fails, then all subsequent tests will be skipped, not marked as failed.

In JUnit 4, you can specify test orders using fixtures but if one test A fails then a test B that depends on test A will also be marked as failed.

TestNG's trick of skipping, rather than failing, can really take the pressure off in large test suites. Rather than trying to figure out why 50 percent of the test suite failed, your team can concentrate on why 50 percent of it was skipped! Better yet, TestNG complements its dependency testing setup with a mechanism for rerunning only failed tests.

3. Fail and rerun:
The ability to rerun failed tests is especially handy in large test suites, and it's a feature you'll only find in TestNG. In JUnit 4, if your test suite consists of 1000 tests and 3 of them fail, you'll likely be forced to rerun the entire suite (with fixes). Needless to say, this sort of thing can take hours.

Anytime there is a failure in TestNG, it creates an XML configuration file (testng-failed.xml) that delineates the failed tests. Running a TestNG runner with this file causes TestNG to only run the failed tests. So, in the previous example, you would only have to rerun the three failed tests and not the whole suite.

This feature doesn't seem like such a big deal when you're running smaller test suites, but you quickly come to appreciate it as your test suites grow in size.

4. Parametric testing:
By placing parametric data in TestNG's XML configuration files, you can reuse a single test case with different data sets and even get different results. This technique is perfect for avoiding tests that only assert sunny-day scenarios or don't effectively verify bounds.

JUnit testers often turn to a framework like FIT in this case because it lets you drive tests with tabular data. But TestNG provides a similar feature right out of the box.

This feature not only facilitates reuse of the test case code but also allows non-programmers to specify test data (since test data is in xml file).


public class TestWebServer {
  @Test(parameters = { "number-of-times" })
  public void accessPage(int numberOfTimes) {
    while (numberOfTimes-- > 0) {
     // access the web page
    }
  }
}

5. Advanced Parametric testing:
While pulling data values into an XML file can be quite handy, tests occasionally require complex types, which can't be represented as a String or a primitive value. TestNG handles this scenario with its @DataProvider annotation, which facilitates the mapping of complex parameter types to a test method.

Example:


//This method will provide data to any test method that declares that its Data Provider
//is named "test1"
@DataProvider(name = "test1")
public Object[][] createData1() {
 return new Object[][] {
   { "Cedric", new Integer(36) },
   { "Anne", new Integer(37)}, 
 };
}

//This test method declares that its data should be supplied by the Data Provider
//named "test1"
@Test(dataProvider = "test1")
public void verifyData1(String n1, Integer n2) {
 System.out.println(n1 + " " + n2);
}

6. Groups:

You can define groups at the class level and then add groups at the method level. You can also specify groups and methods to be included and excluded.


@Test(groups = { "checkin-test" })
public class All {

  @Test(groups = { "func-test" )
  public void method1() { ... }

  public void method2() { ... }
}

and then in testng.xml:


<test name="Simple example">
  <groups>
    <run>
      <include name="checkin-test"/>
      <exclude name="broken"/>
    </run>
  </groups>
 
  <classes>
    <class name="example1.Test1">
    <methods>
       <include name="testMethod" />
    </methods>
  </classes>
</test>

TestNG - java testing framework

2007-12-01T02:10:00.000-05:00

Recently i got introduced to TestNG (version 5.7) at work. I was familiar to JUnit from the past and i kind of knew about the existance of TestNG and that it had improvements over JUnit but i never thought that it will gain so much traction that i will be made to use it soon. Here are some of the features:

JDK 5 Annotations (JDK 1.4 is also supported with JavaDoc annotations).
Flexible test configuration - using multiple testng XML configuration files one per test suite.
Support for data-driven testing (with @DataProvider).
Support for parameters - you can pass parameters to test methods from the testng.xml file.
Allows distribution of tests on slave machines - support for parallel execution of tests and methods.
Powerful execution model (no more TestSuite) - test classes are annotated POJOs and don't have to extend any class or implement interface to have test methods.
Supported by a variety of tools and plug-ins (Eclipse, IDEA, Ant, Maven, etc...).
Embeds BeanShell for further flexibility.
Default JDK functions for runtime and logging (no dependencies).
Dependent methods for application server testing. - one can specify the dependsOnMethods attribute to the @Test annotation to specify a list of methods that should execute before a certain test method executes. This is a powerful feature and is required for any kind of dependent testing. If a dependent method fails, then all subsequent tests will be skipped, not marked as failed (unlike JUnit).

A good article stating improvements in TestNG over JUnit 4 is found at http://www.ibm.com/developerworks/java/library/j-cq08296/.

I used TestNG today for the first time and found the framework very easy to use and within a day i had it integrated into our build system and made a presentation to the team about its usage in our project. In this post, i am detailing the steps i performed to start using TestNG:

1. Wrote a class using just the 3 basic TestNG annotations to start with:

@BeforeClass
@Test (groups = {"xyz.groupname"}) - at class level which gets inherited by all public methods in the class.
@AfterClass

See http://testng.org/doc/documentation-main.html#annotations for complete list of supported annotations.

2. Wrote a master testng.xml which included all project suite-files and was referenced from the ant build script. Also wrote a testng-regression.xml which was imported in the master testng.xml. In the testng-regression.xml, defined the test runs in the suite. Each such testng-xxx.xml file is for xxx named test suite. Each suite can have one or more test runs. Each test runs identifies the class(es) or package(s) to lookup for annotated test methods. Each test run also identifies filter criteria based on groups to include and exclude in the test run. See http://testng.org/doc/documentation-main.html#testng-xml for more on testng.xml.

One powerful feature i found was the group names could be specified in dot separated (java package name like) notations and follow a hierarchy akin to the Log4j Logger naming hierarchy. So you can use wildcards in the testng.xml to not only include all classes of a group but also include classes from child groups. For example, i could just say xyz.* to include xyz.abc and xyz.def group classes.

3. Lastly, used the ant build file to call the testng ant task and pass the testng.xml location to it so that testng can execute the tests we wanted. We can have multiple targets defined for different types of tests that we may want to automate. See http://testng.org/doc/ant.html for examples.

In the latest releases of JUnit 4, it too uses JDK5 annotations and thus makes up for some of the shortcomings that led Cedric Beust to develop TestNG framework.

If you have not had a chance to explore TestNG so far, then i hope after reading this post you will have the good sense to do so now :).

Case for Web services with JSON RPC

2007-11-10T04:06:00.000-05:00

I have recently been working on developing JSON RPC based web services (over https) and using Java client. The server side JSONRPC services were developed using the JSON-RPC-Java and later also using the JSON-RPC C libraries.

The only client side JSON RPC stack in Java that is available at the time of this writing is http://code.google.com/p/json-rpc-client/. It supports JSON RPC over http (using apache commons httpclient library). It was easily extensible to support JSON RPC over https. In this post, i am going to put down my experiences of using JSON RPC.

JSON is a fat-free XML. (Read more at http://json.org/xml.html).
JSON RPC is an alternative RPC mechanism over http (or https).
JSON RPC is simpler to learn and implement than SOAP. The stacks are much less lines of code compared to SOAP stacks.
JSON RPC is simple as it does not include an Interface Definition Language like WSDL for SOAP based web services. So there is no contract definition between client and server in a IDL rather contract is defined on paper and then implememted in respective languages of server-side and client-side.
JSON RPC spec is very loosly written and hence leaves alot of room for vendors to come up with their own solutions. Like metaparadig folks have their proprietary way of implementing class hinting (viz the way to identify the class type to the other end so that JSON message can be mapped to a class type and an instance of the class can be created with the passed in values in the JSON stream).
The interoperability between JSON RPC C/C++ service and Java client is limited in following aspects:

No Java collections can be used. This is same for even SOAP web services. The root cause for this limitation is that the pre JDK 1.5 Java had no generics and hence all collection classes (like ArrayList) could have held more than one Object types so it was hard to tell the type of the element held in the collection. This is solved by proprietary class hinting ismplementations when both client and server are in Java but across languages this becomes an issue. So the solution is to use arrays instead.
Enum types are not supported by the metaparadigm JSON-RPC-Java stack at present as its a newer JDK 1.5 feature. So use int instead.

Security: Though several approaches may be possible but the simplest solution is to implement JSON RPC over https with basic authentication for client. You may have a self-signed certificate for the web service to keep the deployments simple. But if you really want the most security possible then go for a trusted CA signed certificate for the web service but then you will require a certificate signing infrastructure in place to be able to create a certificate for each instance of web service installed.
JSON RPC spec does not have anything to say about intermediary message handlers but it is easy to think of creating JSON RPC intermediary nodes although the spec does not have provisions for extensible message control headers like SOAP spec has. So JSON RPC is pretty much limited to being used between two nodes (the client and the server) - the message source and the message destination or end point. Its not really meant for "document" style messaging for which SOAP is used in B2B applications.

So if you want to build a robust, fat-free (read faster) distributed RPC infrastructure then you can base it on JSON RPC.

JSON RPC makes most sense in web applications where the client is in Javascript language as JSON maps directly to Javascript objects and hence you dont need to parse the message and extract the data, its done automatically. But other than AJAXing your web pages, you can also use it for straight forward RPC architectures where SOAP may be an overkill. You will have a working JSON RPC solution much sooner and it is of course much easier to comprehend and implement than SOAP. So when you are using SOAP web services with RPC style then think twice as you have an more able alternative approach in JSON RPC.

Let me know your thoughts by leaving your comments.

Using Basic authentication and HTTPS (w/ self-signed certificates) in Java

2007-11-06T14:06:00.001-05:00

1. Client Authentication is in practice only used for B2B type applications.
2. In some cases we may even be okay with not authenticating the server on the client end during SSL handshake, for sake of:
o simplicity (no certificate signing infrastructure is required) and
o performance (we only use SSL for encryption and not for server authentication).

This approach is of self-signed certificate which the server can sign for itself and client will by-pass server authentication.

3. We first need to configure web server for SSL. Tomcat currently operates only on JKS, PKCS11 or PKCS12 format keystores.
4. We can use the JDK keytool to generate self-signed certificate for the host running tomcat as shown below:


$ keytool -genkey -alias tomcat -keyalg RSA -keystore example.keystore
Enter keystore password:  secret
Re-enter new password: secret
What is your first and last name?
  [Unknown]:  localhost
What is the name of your organizational unit?
  [Unknown]:  
What is the name of your organization?
  [Unknown]:  <My Company Name>
What is the name of your City or Locality?
  [Unknown]:  <City>
What is the name of your State or Province?
  [Unknown]:  <State>
What is the two-letter country code for this unit?
  [Unknown]:  <Country Code>
Is CN=localhost, OU=Unkown, O=<My Company Name>, L=<City>, ST=<State>, C=<Country> correct?
  [no]:  yes

Enter key password for 
        (RETURN if same as keystore password):  <Enter>

The example.keystore is then generated and is in JKS (Java Key Store) format.

5. Copy it to the Tomcat root directory say C:\Program Files\Apache Software Foundation\Tomcat 6.0 path.

6. The final step is to configure your secure socket in the $CATALINA_HOME/conf/server.xml file, where $CATALINA_HOME represents the directory into which you installed Tomcat 6.


<Connector protocol="org.apache.coyote.http11.Http11Protocol"
               port="8443" minSpareThreads="5" maxSpareThreads="75"
               enableLookups="true" disableUploadTimeout="true" 
               acceptCount="100"  maxThreads="200"
               scheme="https" secure="true" SSLEnabled="true"
               keystoreFile="./example.keystore" keystorePass="secret"
           clientAuth="false" sslProtocol="TLS"/>

NOTE: You can refer to the http://tomcat.apache.org/tomcat-6.0-doc/ssl-howto.html for more configuration options.

With the above settings you can verify that browsing to https://localhost:8443 returns the splash page of tomcat home.

7. We will also make sure that tomcat has a role named "manager" and some user associated with the role. We can edit the tomcat_users.xml for that:


<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="manager"/>
  <user username="admin" password="admin" roles="manager"/>
</tomcat-users>

8. Now, we can enforce that a certain URL pattern for our web application always requires https access. To do this, we need to edit the web.xml of the web application:


<security-constraint>
 <display-name>some name for service</display-name>
 <web-resource-collection>
     <web-resource-name>My Service</web-resource-name>
     <description/>
     <url-pattern>/secure/XYZ/*</url-pattern>
     <http-method>GET</http-method>
     <http-method>POST</http-method>
     <http-method>HEAD</http-method>
     <http-method>PUT</http-method>
     <http-method>OPTIONS</http-method>
     <http-method>TRACE</http-method>
     <http-method>DELETE</http-method>
</web-resource-collection>
<auth-constraint>
     <role-name>manager</role-name>
 </auth-constraint>
 <user-data-constraint>
     <description/>
     <transport-guarantee>CONFIDENTIAL</transport-guarantee>
 </user-data-constraint>
</security-constraint>
<login-config>
 <auth-method>BASIC</auth-method>
 <realm-name>MY_SECURE_REALM</realm-name>
</login-config>
<security-role>
 <description>manager api can use this role.</description>
 <role-name>manager</role-name>
</security-role>

With the above configuration, we have Basic authentication and HTTPS enabled for all resources accessed by the URL pattern /secure/XYZ/*.

So even if you try to access the resource at /secure/XYZ/* using http then tomcat will redirect you to the page using https scheme and thus enforce secure use. Since we also use the Basic authentication so browser client will prompt you entering user credentials.

9. If you are using API based http client access from say a J2SE client (using apache commons httpclient 3.x) then you will need to set the credentials for the realm MY_SECURE_REALM (which defines the Authentication Scope on the web server) in the Http header.


HttpState state = new HttpState();
state.setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, 
"MY_SECURE_REALM"), new UsernamePasswordCredentials(user, passwd));

Also you will need to use the org.apache.commons.httpclient.contrib.ssl.EasySSLProtocolSocketFactory to be able to by-pass the agent authentication on client side. Apache commons httpclient comes with this contrib code which is included with the source distro but is not bundled in the jar file. So you will need to pull the source out from contrib/ssl path and use it in your project.

Basically you will need to check if the uri in use has scheme type of https then associate the EasySSLProtocolSocketFactory as the protocol handler for the scheme.


if (uri.getScheme().equals("https")) {
 Protocol easyhttps = new Protocol(uri.getScheme(), new EasySSLProtocolSocketFactory(), uri.getPort());

 Protocol.registerProtocol("https", easyhttps);
}

The way it works is, EasySSLProtocolSocketFactory in turn uses the EasyX509TrustManager (again from contrib/ssl) to just do a agent certificate validity from and to time validation (so that the ceritificate is not expired and is not before the validity start date). As long as the certificate in use by the agent is valid the EasyX509TrustManager will be okay to bypass doing any authentication for the self-signed certificate for the agent.

That completes the simple discourse on how to use Basic authentication with HTTPS (using self-signed certificate for the server end).

Salient points about log4j

2007-10-26T16:45:00.000-04:00

0. Log4j has three main components: loggers, appenders and layouts.

1. Loggers are named entities which follow hierarchical naming.
2. A logger is said to be an ancestor of another logger if its name followed by a dot is a prefix of the descendant logger name. A logger is said to be a parent of a child logger if there are no ancestors between itself and the descendant logger. For example, the logger named "com.foo" is a parent of the logger named "com.foo.Bar".
3. The root logger resides at the top of the logger hierarchy. It is exceptional in two ways:

1. it always exists,
2. it cannot be retrieved by name.
Invoking the class static Logger.getRootLogger method retrieves it.
4. Loggers may be assigned levels. The set of possible levels, that is:

TRACE,
DEBUG,
INFO,
WARN,
ERROR and
FATAL

are defined in the org.apache.log4j.Level class.

5. The inherited level for a given logger C, is equal to the first non-null level in the logger hierarchy, starting at C and proceeding upwards in the hierarchy towards the root logger.

6. Here are the basic Logger class methods:


package org.apache.log4j;

public class Logger {

   // Creation & retrieval methods:
   public static Logger getRootLogger();
   public static Logger getLogger(String name);

   // printing methods:
   public void trace(Object message);
   public void debug(Object message);
   public void info(Object message);
   public void warn(Object message);
   public void error(Object message);
   public void fatal(Object message);

   // generic printing method:
   public void log(Level l, Object message);
}

7. A logging request is said to be enabled if its level is higher than or equal to the level of its logger.

A log request of level p in a logger with (either assigned or inherited, whichever is appropriate) level q, is enabled if p >= q.

This rule is at the heart of log4j. It assumes that levels are ordered. For the standard levels, we have DEBUG < INFO < WARN < ERROR < FATAL.


   // get a logger instance named "com.foo"
   Logger  logger = Logger.getLogger("com.foo");

   // Now set its level. Normally you do not need to set the
   // level of a logger programmatically. This is usually done
   // in configuration files.
   logger.setLevel(Level.INFO);

   Logger barlogger = Logger.getLogger("com.foo.Bar");

   // This request is enabled, because WARN >= INFO.
   logger.warn("Low fuel level.");

   // This request is disabled, because DEBUG < INFO.
   logger.debug("Starting search for nearest gas station.");

   // The logger instance barlogger, named "com.foo.Bar",
   // will inherit its level from the logger named
   // "com.foo" Thus, the following request is enabled
   // because INFO >= INFO.
   barlogger.info("Located nearest gas station.");

   // This request is disabled, because DEBUG < INFO.
   barlogger.debug("Exiting gas station search");

8. In fundamental contradiction to biological parenthood, where parents always preceed their children, log4j loggers can be created and configured in any order. In particular, a "parent" logger will find and link to its descendants even if it is instantiated after them.

9. Log4j makes it easy to name loggers by software component. This can be accomplished by statically instantiating a logger in each class, with the logger name equal to the fully qualified name of the class. This is a useful and straightforward method of defining loggers. As the log output bears the name of the generating logger, this naming strategy makes it easy to identify the origin of a log message. The developer is free to name the loggers as desired.Nevertheless, naming loggers after the class where they are located seems to be the best strategy known so far.

10. Log4j allows logging requests to print to multiple destinations. In log4j speak, an output destination is called an appender.

Currently, appenders exist for the console, files, GUI components, remote socket servers, JMS, NT Event Loggers, and remote UNIX Syslog daemons. It is also possible to log asynchronously.

11. More than one appender can be attached to a logger.

The addAppender method adds an appender to a given logger.

12. Each enabled logging request for a given logger will be forwarded to all the appenders in that logger as well as the appenders higher in the hierarchy.

In other words, appenders are inherited additively from the logger hierarchy. For example, if a console appender is added to the root logger, then all enabled logging requests will at least print on the console. If in addition a file appender is added to a logger, say C, then enabled logging requests for C and C's children will print on a file and on the console.

It is possible to override this default behavior so that appender accumulation is no longer additive by setting the additivity flag to false.

13.The layout is responsible for formatting the logging request according to the user's wishes, whereas an appender takes care of sending the formatted output to its destination.

The PatternLayout, part of the standard log4j distribution, lets the user specify the output format according to conversion patterns similar to the C language printf function.

See the conversion characters to use in the link below:
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html

For example, the PatternLayout with the conversion pattern "%r [%t] %-5p %c - %m%n" will output something akin to:


176 [main] INFO  org.foo.Bar - Located nearest gas station.

The first field is the number of milliseconds elapsed since the start of the program.
The second field is the thread making the log request.
The third field is the level of the log statement.
The fourth field is the name of the logger associated with the log request.
The text after the '-' is the message of the statement.

14. To use log4j by reading the configuration from a properties file:


import com.foo.Bar;

import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;

public class MyApp {

  static Logger logger = Logger.getLogger(MyApp.class.getName());

  public static void main(String[] args) {


    // BasicConfigurator replaced with PropertyConfigurator.
    PropertyConfigurator.configure(args[0]);

    logger.info("Entering application.");
    Bar bar = new Bar();
    bar.doIt();
    logger.info("Exiting application.");
  }
}

And a sample configuration file:


log4j.rootLogger=DEBUG, A1
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout

# Print the date in ISO 8601 format
log4j.appender.A1.layout.ConversionPattern=%d [%t] %-5p %c - %m%n

# Print only messages of level WARN or above in the package com.foo.
log4j.logger.com.foo=WARN

Example output:


2000-09-07 14:07:41,508 [main] INFO  MyApp - Entering application.

This will only print warn, error or fatal but not info, debug or trace messages for all components inheriting from com.foo logger name hierarchy.

Example using multiple appenders:


log4j.rootLogger=debug, stdout, R

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

# Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n

log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=example.log

log4j.appender.R.MaxFileSize=100KB
# Keep one backup file
log4j.appender.R.MaxBackupIndex=1

log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %c - %m%n

Example output:


INFO [main] (MyApp2.java:12) - Entering application.
DEBUG [main] (Bar.java:8) - Doing it again!
INFO [main] (MyApp2.java:15) - Exiting application.

15. Under certain well-defined circumstances however, the static inializer of the Logger class will attempt to automatically configure log4j.

The exact default initialization algorithm is defined as follows:

1. Setting the log4j.defaultInitOverride system property to any other value then "false" will cause log4j to skip the default initialization procedure (this procedure).
2. Set the resource string variable to the value of the log4j.configuration system property. The preferred way to specify the default initialization file is through the log4j.configuration system property. In case the system property log4j.configuration is not defined, then set the string variable resource to its default value "log4j.properties".
3. Attempt to convert the resource variable to a URL.
4. If the resource variable cannot be converted to a URL, for example due to a MalformedURLException, then search for the resource from the classpath by calling org.apache.log4j.helpers.Loader.getResource(resource, Logger.class) which returns a URL. Note that the string "log4j.properties" constitutes a malformed URL. See Loader.getResource(java.lang.String) for the list of searched locations.
5. If no URL could not be found, abort default initialization. Otherwise, configure log4j from the URL. The PropertyConfigurator will be used to parse the URL to configure log4j unless the URL ends with the ".xml" extension, in which case the DOMConfigurator will be used. You can optionaly specify a custom configurator. The value of the log4j.configuratorClass system property is taken as the fully qualified class name of your custom configurator. The custom configurator you specify must implement the Configurator interface.

Under Tomcat 3.x and 4.x, you should place the log4j.properties under the WEB-INF/classes directory of your web-applications. Log4j will find the properties file and initialize itself. This is easy to do and it works.

Generally one would want to have the flexibility to choose between different logging implementations. In that case, one can use apache commons logging which provides similar interface as Log4J's described above (only it calls it Logger as Log and uses a LogFactory.getLog() to get the named Log instance) but comes with adapters for several other logging implementations in Java (Avalon, JDK's Logging etc).

EJB 3.0 with JBoss 4.2.1 GA and Netbeans IDE 5.5.1

2007-09-03T14:13:00.000-04:00

I have started to read O'reilly's EJB 3.0 5th Edition by Bill Burke and Richard Monson-Haefel. The book covers EJB 3.0 and Java Persistence 1.0 in detail. It comes with a JBoss workbook for JBoss 4.0.3 release. I installed the current stable release JBoss 4.2.1 GA for my practice. Unlike the 4.0.x releases, the JBoss 4.2.x release has the EJB 3 enabled by default. I used Netbeans 5.5.1 IDE for development. It supports JBoss 4.x and even 5.x (which is still in beta). It readily recognized my JBoss 4.2.1 installation in the server manager. I created a project as Enterprise Application (with both web and ejb modules). The persistence configuration was simple and i used the default datasource HSQL DB 1.8.

There were some gotchas before i could get chapter 4 "Developing your first bean" examples working:
1. one has to change this "DefaultDS" to "java:/DefaultDS" in the META-INF/persistence.xml of your ejb module.
2. also, in the client application, you will need to reference the "TravelAgentBean/remote" as "<Your EAR application name>/TravelAgentBean/remote". To be sure as to where in the jndi tree has your session bean got registered, you can browse to jmx-console (http://localhost:8080/jmx-console) and look for "service=JNDIView". Click on the link and in the following page, invoke the method list() to see the list of names in the JNDI tree. From there you can know for sure what name to use for lookup at client end for your session facade.
3. lastly, i had mistakenly had my entity bean's id annotated as @GeneratedValue in which case when i used to call the setId() method in my session facade bean, then it used to throw an exception while persisting using EntityManager's persist() that the entity instance is detached one. So, i could either remove the annotation of generated value or don't set the id.
4. at client end, there is no need for PortableRemoteObject.narrow() method anymore. You can simply use Java casting.

For my simple example, i did not have to pass the JNDI bootstrap params as properties instance to InitialContext() constructor.

Eclipse 3.3 (Europa) has a very nice OR mapping tool called Dali, but i am yet to figure out how to make it work for me (basically if i don't have the corresponding table already created in the DB, then Eclipse is unable to map the Entity type's members to columns in the database table and so far i dont know how to turn that error off. With Netbeans i did not get that issue and when i first ran my application the table was automatically created by hibernate as i declared it to do so by setting hibernate property hibernate.hbm2ddl.auto with value "update"). Europa release supports JBoss 4.2.x release. Red Hat is also developing a comprehensive IDE solution for JBoss in partnership with Exadel to facilitate easier Rich web application development.

My SCDJWS Study Notes using Scribd

2007-08-17T21:10:00.000-04:00

My new Honda Element 2007

2007-07-25T23:05:00.001-04:00

I recently bought a Honda Element 2007 compact SUV and after having driven it today for a week, i feel a very satisfied owner of my first 4 wheeler vehicle ever. This model of SUV is unique with capacity to seat only 4 passengers and the doors are wide-opening clamshell type with no pillar between the front and back seats (but one has to first open the front doors to be able to open the back doors). Other auto makers also have SUVs of similar kind like Toyota’s FJ Cruiser and Nissan’s Xterra. The mileage is 21mpg in city and 26mpg on highway (which is decent compared to other compact SUV’s). The engine is 166 hp and its a 4 wheel drive vehicle. It comes fully loaded with power window, mirror, steering and doors. There is a AM/FM/CD player and a skylight glass top. The thing which i loved about my new Element is its very spacious (alot of leg room and room for cargo). With wide opening doors, loading cargo is easy (as there is no pillar between seats). The rear seats can be folded to the sides too thus making extra space for cargo. Also the look is off beat and trendy (in my opinion).

Charting the web with Cewolf/JFreeChart - Producing Time Series plots

2007-06-17T20:38:00.000-04:00

I recently had an opportunity to use the Cewolf 1.0 at work for some Time Series plots (a variant of the XY Chart, shown in the figure above, where the X-axis is for time values). This blog is about Cewolf and how to create time series plots with it.

Cewolf is a JSP tag library which uses JFreeChart for rendering the charts. It comes with a controller servlet which is used for interpreting the parameters passed through the JSP tag and accordingly generating the chart image in-memory (no files created on the file system of the server) and embeds the image as tag in the HTML output to the client response stream.

IMO, Cewolf/JFreeChart is the best free charting package for a web application required to draw charts and being developed in Java EE. It supports several different types of charts and one of them was the Time Series plots. Here is some code which can produce a simple time series plot (using cewolf).

1. To install Cewolf you just need to copy the jars from its lib/ path (which includes the JFreeChart jar too) to WEB-INF/lib of your web application.

2. We need to write a data producer which gets the data set (in {time, value} pairs for the time series plot). A typical time series data producer is given below:


//~--- non-JDK imports --------------------------------------------------------

import de.laures.cewolf.DatasetProduceException;
import de.laures.cewolf.DatasetProducer;

import org.jfree.data.time.Minute;
import org.jfree.data.time.TimeSeries;
import org.jfree.data.time.TimeSeriesCollection;

//~--- JDK imports ------------------------------------------------------------

import java.io.Serializable;

import java.util.Date;
import java.util.Map;

/**
 * A sample data producer for the time series plot.
*/
public class MyDataProducer implements DatasetProducer, Serializable
{
    public MyDataProducer()
    {
    }

    public Object produceDataset(Map map) throws DatasetProduceException
    {
        /*
         *   To this time series collection we can add more than one time series
         *   where each time series will be represented by its own line on a
         *   plot.
         */
        TimeSeriesCollection ts = new TimeSeriesCollection();

        try {
            String[] allSeries = { "series1", "series2" };

            // Loop through all series and add the data to series and series to
            // timeseries collection (ts).
            for (int i = 0; i < allSeries.length; i++) {

                // Get data for series from some kind of datasource
                MyDataSet[] myDataSet = GetDataForSeries(allSeries[i]);
                TimeSeries  mySeries  = new TimeSeries("My Data Series " + i, Minute.class);

                // Add data to series
                for (MyDataSet data : myDataSet) {
                    mySeries.add(new Minute(new Date(data.getTimestamp().getTime())), data.getYValue());
                }

                // Add the series to the collection
                ts.addSeries(mySeries);
            }
        } catch (Exception e) {
            e.printStackTrace();

            throw new DatasetProduceException();
        }

        return ts;
    }

    public boolean hasExpired(Map map, Date date)
    {
        return false;
    }

    public String getProducerId()
    {
        return "My Data Producer";
    }

    private MyDataSet[] GetDataForSeries(String string)
    {
        // Get data from DB or some data source
        // return an array of time/value pairs (for instance, as an array
        // of MyDataSet instances.       
    }
}

MyDataSet class is:


import java.sql.Timestamp;

/** A sample time/value pair data. An array/list of this type will constitute
 * the data set for the plot.
*/
public class MyDataSet
{
    private Timestamp timestamp; // You can use other date/time types in Java SE here.
    private double    yValue;

    public MyDataSet()
    {
    }

    public Timestamp getTimestamp() {
        return timestamp;
    }

    public void setTimestamp(Timestamp timestamp) {
        this.timestamp = timestamp;
    }

    public double getYValue() {
        return yValue;
    }

    public void setYValue(double yValue) {
        this.yValue = yValue;
    }
}

In the JSP page you include the chart now:


<jsp:usebean id="myPlotData" class="com.mycompany.MyDataProducer">

<cewolf:chart
    id="MyChart"
    type="timeseries"
    title="My Plot Title"
    xaxislabel="Time"
    yaxislabel="My Data Value">
    <cewolf:data>
    <cewolf:producer id="myPlotData" usecache="false">
    </cewolf:data>
</cewolf:chart>
<cewolf:img chartid="MyChart" renderer="/cewolf" width="1000" height="400">

Lastly, one needs to configure the CewolfServlet in the web.xml:

<servlet>
   <servlet-name>CewolfServlet</servlet-name>
   <servlet-class>de.laures.cewolf.CewolfRenderer</servlet-class>
</servlet>

<servlet-mapping>
   <servlet-name>CewolfServlet</servlet-name>
 <url-pattern>/cewolf/*</url-pattern>
</servlet-mapping>

The generated charts on tomcat does require one to increase the JVM heap size to at least 256MB from the default 64MB.

The pros of using Cewolf/JFreeChart in a Java EE web application:
1. Compared to the other open source packages, JFreeChart happens to be the best in terms of the look and feel of the plots and the ease of use of the API.
2. Cewolf contributes to the glory by making the JFreeChart available as JSP tag library. And as far as i know, there isn't any other better option for plotting in the open source Java world.

The duo of Cewolf/JFreeChart are lacking in a few important features:
1. AJAX support for real-time plots. So if we want the server to be able to asynchronously (or by the virtue of some background polling from client) refresh the chart in (near) real-time then its not something supported today in Cewolf/JFreeChart. If this feature is required then JViews Charts or Chart Director are the two commercial offerings that i know of that can do AJAX based live charts.
2. Zoom and Pan interactions are not easily supported by the API.

Working with JMaki

2007-06-10T01:55:00.001-04:00

I recently had an opportunity to use some of the JMaki UI components and it took me some googling to figure out how to pass data dynamically (which is what most of the time you will want and unfortunately all examples use some static data in the JSON format) to the UI components. JMaki's integration with Netbeans makes it really simple to have those nice Web UI components working for you in a jiffy (like grid, tree, menu, captcha, autocomplete etc). Though i am a big fan of DWR (having used the reverse ajax in DWR 2.0 for an event browser application to show events in real-time) for Ajax support in my work, i did like the Ajax-enabled UI Components that come with JMaki. Another nice thing about JMaki is, it provides a common data model for multiple implementations of a certain UI component. For example, you have a Yahoo UI Tree and a dojo toolkit tree component. Since JMaki provides the abstraction by keeping the data models same for both these tree components, so we have the option to switch between these implementations with almost no change to code.

Now going back to the point (the reason i am writing this post after all) ... the JMaki components accept data dynamically in JSON format and though one can create the JSON format string to pass as value attributes to the widgets, it becomes cumbersome for widgets like trees or grid to escape the quotes and construct the strings. To make our lives easy in constructing the JSON formatted dynamic data, JMaki comes bundled with org.json.* classes (JSONObject, JSONArray etc) using which one can create the data to pass in an elegant and maintainable way. You will need to convert the JSONObject or JSONArray types to their Object literal form using the following code which Greg Murray (JMaki project manager) released in reply to one post on JMaki users forum:


    /**
     * Converts a JSON Object to an Object Literal
     *
     *
     * @param jo
     * @param buff
     *
     * @return
     *
     * @throws JSONException
     */
    public static String jsonToObjectLiteral(JSONObject jo, StringBuffer buff)
            throws JSONException
    {
        if (buff == null) {
            buff = new StringBuffer("{");
        } else {
            buff.append("{");
        }

        JSONArray names = jo.names();

        for (int l = 0; (names != null) && (l < names.length()); l++) {
            String key   = names.getString(l);
            String value = null;

            if (jo.optJSONObject(key) != null) {
                value = key + ":";
                buff.append(value);
                jsonToObjectLiteral(jo.optJSONObject(key), buff);
            } else if (jo.optJSONArray(key) != null) {
                value = key + ":";
                buff.append(value);
                jsonArrayToString(jo.optJSONArray(key), buff);
            } else if (jo.optLong(key, -1) != -1) {
                value = key + ":" + jo.get(key) + "";
                buff.append(value);
            } else if (jo.optDouble(key, -1) != -1) {
                value = key + ":" + jo.get(key) + "";
                buff.append(value);
            } else if (jo.opt(key) != null) {
                Object obj = jo.opt(key);

                if (obj instanceof Boolean) {
                    value = key + ":" + jo.getBoolean(key) + "";
                } else {
                    value = key + ":" + "'" + jo.get(key) + "'";
                }

                buff.append(value);
            }

            if (l < names.length() - 1) {
                buff.append(",");
            }
        }

        buff.append("}");

        return buff.toString();
    }

    /**
     * Converts a json array to string.
     *
     *
     * @param ja
     * @param buff
     *
     * @return
     *
     * @throws JSONException
     */
    public static String jsonArrayToString(JSONArray ja, StringBuffer buff)
            throws JSONException
    {
        if (buff == null) {
            buff = new StringBuffer("[");
        } else {
            buff.append("[");
        }

        for (int key = 0; (ja != null) && (key < ja.length()); key++) {
            String value = null;

            if (ja.optJSONObject(key) != null) {
                jsonToObjectLiteral(ja.optJSONObject(key), buff);
            } else if (ja.optJSONArray(key) != null) {
                jsonArrayToString(ja.optJSONArray(key), buff);
            } else if (ja.optLong(key, -1) != -1) {
                value = ja.get(key) + "";
                buff.append(value);
            } else if (ja.optDouble(key, -1) != -1) {
                value = ja.get(key) + "";
                buff.append(value);
            } else if (ja.optBoolean(key)) {
                value = ja.getBoolean(key) + "";
                buff.append(value);
            } else if (ja.opt(key) != null) {
                Object obj = ja.opt(key);

                if (obj instanceof Boolean) {
                    value = ja.getBoolean(key) + "";
                } else {
                    value = "'" + ja.get(key) + "'";
                }

                buff.append(value);
            }

            if (key < ja.length() - 1) {
                buff.append(",");
            }
        }

        buff.append("]");

        return buff.toString();
    }

So, after you have your dynamic data put in JSONObject or JSONArray, you can invoke the corresponding conversion method stated above to get the String form of your JSON data ready to be passed to the component.

For instance, in your tree builder code, you will need to do the following (the example is from this post where the solution was posted by Greg Murray):


public static JSONObject buildTreeData(AuthorizedTeams ateams)
throws JSONException {

 JSONObject retValue = new JSONObject();
 JSONObject root = new JSONObject();
 root.put ("title", "Organizations");
 root.put ("expanded", true);
 JSONArray data = new JSONArray();

 Team[] teams = ateams.getTeams();

 for (int i=0; i<teams.length; i++) {
   JSONObject teamObj = new JSONObject();
   teamObj.put("title", teams[i].getTeamName());
   teamObj.put("expanded", true);

   JSONArray children = new JSONArray();

   User[] teamUsers = teams[i].getMembers();
   for (int j=0; j<teamUsers.length; j++) {
     JSONObject childObj = new JSONObject();
     childObj.put("title",teamUsers [j].getUserName());
     children.put(childObj);
   }
   teamObj.put("children", children);
   data.put(teamObj);
 }
 root.put ("children", data);
 retValue.put ("root", root);

 return jsonToObjectLiteral(retValue, new StringBuffer());
}

Here is the JSP snippet:

 <jsp:useBean id="teams"
   class="com.myapp.assignment.AuthorizedTeams"
   scope="request"/>
 <a:widget name="dojo.tree" value="${teams.teamsData}">

JavaServer Faces Part 1 - Introduction

2007-05-08T10:41:00.001-04:00

This is first in the series of blogs on JSF.

JSF = JavaServer Faces.

It’s a web framework. The 3 independent elements that make up a usable JSF component in a page are:

UIComponent class – defines behavior of component. Eg. UISelectOne
Renderer class – provides specific renderings of component. For eg, a UISelectOne can be rendered in HTML as either a group of radio buttons or a select menu.
A JSP tag – which associates a Renderer with a UIComponent and makes them usable in JSP as a single tag, eg <h:selectOneMenu>

JSF UI components are bound to server-side Java beans (which are registered as Managed Beans in faces-config.xml). In the JSP pages, the UI components are bound to Managed Beans using the JSF Expression Language (which in JSF 1.2 is same as JSTL 2.1’s EL and is now called Unified EL). Once bound, updating bean properties or invoking bean methods from a web interface is handled automatically by JSF request processing lifecycle. This ability to automatically synchronize server-side Java Bean properties to a hierarchical set of components that are based on UI presented to the client user is a major advantage of JSF over other web frameworks like Struts.

JSF Request Processing Lifecycle

When a JSP page with JSF components is requested first time, then JSF runtime creates an in-memory components tree on server side.
In between requests, when nothing is happening in application, the component tree is cached on server.
Upon a subsequent request, the component tree is reconstituted, and if form input values are sent in request, they are processed and validations are executed.
Upon successful validation, server-side managed bean properties are updated.
Once all event processing and updates are over, the response is sent to client.

To enable JSF support in a Java EE web application, following needs to be done:

An entry for Faces Servlet in web.xml and mapping of this servlet to *.faces or /faces/* etc. (A request that uses the appropriate faces URL pattern can be considered a faces request and when received by faces controller, it processes the request by preparing an object known as the JSF context, which contains all accessible application data and routes the client to appropriate view page based on the navigation rules as defined in the faces-config.xml.)
A JSF configuration file – faces-config.xml in WEB-INF/ path.
Following jar files in WEB-INF/lib path:
1. JSF jars – jsf-api.jar and jsf-impl.jar
2. Apache commons jars – commons-beanutils.jar, commons-collection.jar, commons-digester.jar, and commons-logging.jar.
3. JSTL jars – standard.jar and jstl.jar

For a JSP page to be JSF enabled,

we need to include at least the following taglibs from Sun’s JSF RI (you may also use Apache MyFaces implementation of JSF spec):

   1: <%@taglib uri=”http://java.sun.com/jsf/core” prefix=”f”%>
   2: <%@taglib uri=”http://java.sun.com/jsf/html” prefix=”h”%>

In the JSP page body, we must add <f:view> tag which becomes the base UI component of component tree in memory on server side when the page is requested for viewing.

If page processes form input, then we can add <h:form> tag.

Example code:

inputname.jsp – shows a form to user to enter name

If outcome is “greeting” then show greeting.jsp to user

The input name between the two pages is stored in memory in PersonBean’s personName field. The personName is registered as managed bean and JSF’s EL is used in the JSP pages to access the PersonBean’s personName field values.

   1: inputname.jsp:   2:     3: <%@ taglib uri="http://java.sun.com/jsf/html" prefix="h" %>   4: <%@ taglib uri="http://java.sun.com/jsf/core" prefix="f" %>   5: <f:loadBundle basename="jsfks.bundle.messages" var="msg"/>   6:     7: <html>   8:  <head>   9:   <title>enter your name page</title>  10:  </head>  11:  <body>  12:    <f:view>  13:      <h1>  14:       <h:outputText value="#{msg.inputname_header}"/>  15:      </h1>  16:      <h:form id="helloForm">  17:       <h:outputText value="#{msg.prompt}"/>  18:       <h:inputText value="#{personBean.personName}" />  19:       <h:commandButton action="greeting" value="#{msg.button_text}" />  20:      </h:form>  21:    </f:view>  22:  </body>  23: </html>

Where, message bundle is defined in a message.properties file (which needs to be put in WEB-INF/classes path in your web applications WAR) as,

   1: inputname_header=JSF KickStart   2: prompt=Tell us your name:   3: greeting_text=Welcome to JSF   4: button_text=Say Hello   5: sign=!

We bind a PersonBean to the inputText filed in helloForm. To do so we also need to register the PersonBean as managed bean in faces-config.xml. We also need to define the navigation rule from :

   1: <?xml version="1.0"?>   2: <!DOCTYPE faces-config PUBLIC   3:   "-//Sun Microsystems, Inc.//DTD JavaServer Faces Config 1.1//EN"   4:   "http://java.sun.com/dtd/web-facesconfig_1_1.dtd">   5:     6: <faces-config>   7:   <navigation-rule>   8:    <from-view-id>/pages/inputname.jsp</from-view-id>   9:     <navigation-case>  10:      <from-outcome>greeting</from-outcome>  11:      <to-view-id>/pages/greeting.jsp</to-view-id>  12:    </navigation-case>  13:   </navigation-rule>  14:    15:   <managed-bean>  16:     <managed-bean-name>personBean</managed-bean-name>  17:     <managed-bean-class>jsfks.PersonBean</managed-bean-class>  18:     <managed-bean-scope>request</managed-bean-scope>  19:   </managed-bean>  20: </faces-config>

And here’s what the greeting.jsp is:

   1: <%@ taglib uri="http://java.sun.com/jsf/html" prefix="h" %>   2: <%@ taglib uri="http://java.sun.com/jsf/core" prefix="f" %>   3: <f:loadBundle basename="jsfks.bundle.messages" var="msg"/>   4:     5: <html>   6:   <head>   7:    <title>greeting page</title>   8:   </head>       9:   <body>  10:      <f:view>  11:          <h3>  12:       <h:outputText value="#{msg.greeting_text}" />,  13:       <h:outputText value="#{personBean.personName}" />  14:          <h:outputText value="#{msg.sign}" />  15:         </h3>  16:      </f:view>  17:  </body>      18: </html>

And the managed bean PersonBean.java:

   1: package jsfks;   2:     3: public class PersonBean {   4:     5:    String personName;   6:        7:    /**   8:    * @return Person Name   9:    */  10:    public String getPersonName() {  11:       return personName;  12:    }  13:    14:    /**  15:    * @param Person Name  16:    */  17:    public void setPersonName(String name) {  18:       personName = name;  19:    }  20: }

This completes the short introduction to JSF 1.1.