Using Quartz Scheduler in a Java EE Web Application

At times, you may have wanted to perform some action periodically in your web application. Quartz is an enterprise grade scheduler which can be used for such a task. Read here for the complete list of features of Quartz. For using the Quartz scheduler library in a Java EE web application, following needs to be done:

• Include the quartz jars (quartz-all.jar and others in the lib path). In my case, some of the commons-xxx.jar files were already included in the project due to the dependency of another library (displaytag) on those jar files. So in my quartz setup i had to disinclude them. In the lib/build path, i only included, jta.jar and also everything which was not already there in the project from lib/optional path too (they are not many anyway).

• We then had to create the tables required by quartz for storing job details and triggers across restart of application. This is an optional feature but an important one (which made us decide to use quartz in the first place over the JDK Timer). We used the docs/dbTables/tables_mysql.sql script to create the tables.

• Then we copied (example_quartz.properties), modified and saved the quartz.properties file in the project and changed the packaging settings to include the properties file in the WEB-INF/classes path in the IDE. In the properties file, we changed the configuration to have quartz point to the data store we created in step 2.
  

# Configuring Main Scheduler Properties
org.quartz.scheduler.instanceName = MyScheduler
org.quartz.scheduler.instanceId = 1
org.quartz.scheduler.rmi.export = false
org.quartz.scheduler.rmi.proxy = false

# Configuring ThreadPool
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 30
org.quartz.threadPool.threadPriority = 5

# Configuring JobStore
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties = false
org.quartz.jobStore.dataSource = quartzDS
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = false

# Configuring datasource
org.quartz.dataSource.quartzDS.driver = com.mysql.jdbc.Driver
org.quartz.dataSource.quartzDS.URL = jdbc:mysql://localhost:3306/mydb
org.quartz.dataSource.quartzDS.user = me
org.quartz.dataSource.quartzDS.password = secret
org.quartz.dataSource.quartzDS.maxConnections = 31

# Rest of config was retained from example_quartz.properties.

• We added following lines to our web.xml:
  

<servlet>

<description>Quartz Initializer Servlet</description>
<servlet-name>QuartzInitializer</servlet-name>
<servlet-class>org.quartz.ee.servlet.QuartzInitializerServlet</servlet-class>
<init-param>

<param-name>shutdown-on-unload</param-name>
<param-value>true</param-value>

</init-param>
<init-param>

<param-name>start-scheduler-on-load</param-name>
<param-value>true</param-value>

</init-param>
<load-on-startup>1</load-on-startup>

</servlet>

This sets up the initializer servlet which can initialize the default scheduler and start the scheduler at application bootstrap time.

• Now in our webservice/jsp/servlet of our web application we do the following:

try {

// A. Get the default scheduler.
Scheduler sched = StdSchedulerFactory.getDefaultScheduler();

// B.Generate a unique name identifier for jobs and
// triggers of your application as required.
// One way is to use hashCode() if its a string param.
String dataToPass = "someParamToPassToJob";
int id = dataToPass.hashCode();

// C. Create/Replace a poll job and add it to the scheduler.
JobDetail job =
new JobDetail("job_"+id, "SomeJobGroup", com.mycompany.MyJob.class);
job.setRequestsRecovery(true);
// Pass data to the poll job.
job.getJobDataMap().put("param", dataToPass);
sched.addJob(job, true);

// D. Create a Trigger with unique name
SimpleTrigger trigger = new SimpleTrigger("trig_"+id, "SomeTriggerGroup");


// E. Check if a trigger is already associated with this job
// This step is optional and depends on your application's requirement.

Trigger[] jobTriggers;
jobTriggers = sched.getTriggersOfJob("job_"+id, "SomeJobGroup");

boolean isTriggerAlreadyAssociated = false;
for (Trigger trig : jobTriggers) {

if (trig.getName().equals("trig_"+id) &&
trig.getGroup().equals("SomeTriggerGroup")) {

// the job already has this trigger associated with it
isTriggerAlreadyAssociated = true;

}

}

// F. Associate this trigger with the job
trigger.setJobName(job.getName());
trigger.setJobGroup(job.getGroup());

// G. Initialize the trigger with duration and resolution to fire
trigger.setStartTime(startTime.getTime());
trigger.setEndTime(endTime.getTime());
trigger.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
trigger.setRepeatInterval(repeatInterval); //in milliseconds

if (isTriggerAlreadyAssociated) {
// Reschedule the job with the existing trigger.
sched.rescheduleJob("trig_"+id, "SomeTriggerGroup", trigger);
} else {
// Schedule the job with the new trigger.
sched.scheduleJob(trigger);
}

} catch (SchedulerException se) {

}

• Of course, the last thing is to write the Job class which does the actual work. The following is code from examples of Quartz.

public class PrintPropsJob implements Job {

public PrintPropsJob() {
}

public void execute(JobExecutionContext context)
throws JobExecutionException {

JobDataMap data = context.getJobDetail().getJobDataMap();
System.out.println("someProp = " + data.getString("someProp"));
System.out.println("someObjectProp = " + data.getObject("someObjectProp"));

}

}

• Some important points to note about jobs and triggers:

1. Jobs have a name and group associated with them, which should uniquely identify them within a single Scheduler.
2. A Job can be associated with multiple triggers.
3. Triggers are the 'mechanism' by which Jobs are scheduled.
4. Many Triggers can point to the same Job, but a single Trigger can only point to one Job.
5. JobDataMap holds state information for Job instances. JobDataMap instances are stored once when the Job is added to a scheduler. They are also re-persisted after every execution of StatefulJob instances.
6. JobDataMap instances can also be stored with a Trigger. This can be useful in the case where you have a Job that is stored in the scheduler for regular/repeated use by multiple Triggers, yet with each independent triggering, you want to supply the Job with different data inputs.
7. The JobExecutionContext passed to a Job at execution time also contains a convenience JobDataMap that is the result of merging the contents of the trigger's JobDataMap (if any) over the Job's JobDataMap (if any).
8. We can have different job types:

• Stateful Jobs - where state passed to job (in JobDataMap) is remembered (like static values) across executions of the job. Also, stateful jobs are not allowed to execute concurrently, which means new triggers that occur before the completion of the execute(xx) method will be delayed.
• Interruptable job - provide a mechanism for having the Job execution interrupted by implementing a callback method interrupt(), which will be called when scheduler's interrupt method is invoked on the Job.

That's all, in short, about how to integrate Quartz scheduler library in a web application.

Is it okay to put business logic in stored procedure?

In my experience, i have come across maintaining some software where the business logic was written in stored procedures (or functions/triggers). Recently i collected some points on why this is not such a good idea. The main reason for the use of stored procedures in one product was to have multiple client types to be able to invoke the same business logic. But then we can achieve the same effect applying MVC (model view controller) pattern and keeping the model/business logic in the application code rather than stored procedure. Here are some other reasons why its not a good idea to write BL in stored procedures:

NOTE: Most of the ideas presented below are excerpted from this article.

1. If there are more than one interfaces and the BL is part in stored procedure and part in presentation tier then it becomes a maintenance headache to keep the different presentation tiers in synch.

2. Stored procedures form an API by themselves. Adding new functionality or new procedures is the "best" way to extend an existing API.This means that when a table changes, or behaviour of a stored procedure changes and it requires a new parameter, a new stored procedure has to be added. When stored proc is changed, the DAL/BL code needs to change too to call the changed/new stored proc whereas, if the SQL is generated on the fly from the DAL/BL code and there is no stored proc then only DAL code will change.
Microsoft also believes stored procedures are over: it's next generation business framework MBF is based on Objectspaces, which generates SQL on the fly.
In Java world, ORM (Object to Relational Mapping) frameworks like Hibernate and TopLink (and now Java Persistence Architecture, JPA) are meant to generate SQL on the fly too.

3.Business logic in stored procedures is more work to test than the corresponding logic in the application. Referential integrity will often force you to setup a lot of other data just to be able to insert the data you need for a test.Stored procedures are inherently procedural in nature, and hence harder to create isolated tests and prone to code duplication. Another consideration, and this matters a great deal in a sizable application, is that any automated test that hits the database is slower than a test that runs inside of the application. Slow tests lead to longer feedback cycles.

4. While stored procedures may run faster, they take longer to build, test, debug and maintain, therefore this extra speed comes at a price.

5. BL in stored procs does not scale - If you have a system with 100's of distributed databases it is far more difficult to keep all those stored procedures and triggers synchronized than it is to keep the application code synchronized.

6. Locked in to the DB for which stored procs are written.

7. Porting the data will be one exercise, but porting the stored procedures and triggers will be something else entirely. Now, if all that logic were held inside the application, how much simpler would it be?

All changes made to the database can be logged without using a single database trigger. How? By adding extra code into the DAO to write all relevant details out to the AUDIT database. This functionality is totally transparent to all the objects in the Business Layer, and they do not need any extra code to make it work.

An Events Browser - Using Displaytag and DWR

Problem:

To view application-specific events in the web browser.

Requirements:
The implementation should satisfy the following requirements:

1. support viewing all the logged events (in a DB) with provision for specifying the number of rows per page to show in a data grid of event records.
2. support pagination.
3. support sorting on columns like event time, id, type etc.
4. support exporting events data to CSV, excel etc formats.
5. change row color to highlight rows based on severity of events.
   
6. enable viewing live events instantaneously as they occur.

Solution:
The first 5 requirements are easily met by Displaytag, an excellent JSP tag library for displaying data grids. We can also enable AJAX support for displaytag data grid pagination and sorting using ajaxtags library.

We did not like the live grid type views (which may be better suited to the search examples) for our requirement as we believe the pagination with Ajax support is more appealing and common experience for users.

The last requirement is met by DWR 2.0 (which as of this writing is still being developed and the current release is 2.0 RC2). DWR 2.0 has a new feature called Reverse AJAX, which enables pushing the data from server to browser client, asynchronously.

So, we have two data grids in our UI;

1. First data grid, displays the existing event records from the DB.
   
2. Second data grid, displays the live events using DWR's Reverse Ajax feature. One can use the recently added proxy for scriptaculous Javascript effects library, to highlight the new rows as they get added to the live events data grid. Adding new rows to the HTML table is facilitated by org.directwebremoting.proxy.dwr.Util class' addRows(String elementId, String[][] data) method. The implementation can choose to route the events to all browsers or selected browser sessions only - an event routing behavior. An example on reverse ajax feature is found here (see stock ticker).

The user can navigate the existing data, paginate, sort, export and at the same time can continue to view the live events being added to the second data grid.

I did further work on improving the way the events displayed in the live events section above. The live events section which uses reverse ajax to asynchronously display events occurring in near real-time, is best kept separated from the archived events browser above. So the live events page can be made to show only the specified size of latest events in a queue fashion (where new incoming events replace some of the existing events (on an oldest removed first basis) when the defined window limit is reached). So if window limit is say 20 events, and we have the latest 19 events displayed so if there are 2 new events to show, we remove the last 1 event from the existing 19 and show the 20 events with the latest 2 being shown at top of the table. The window limit is a client side parameter and can be (re)set by the user. The events will need to be cached locally on the clients to be able to display the lastest on top (as addRows() API of DWR will only append to the table tbody). Also, by caching the events locally, we can easily control the window size. The idea to have a limit to the number of live events is required as we cannot just keep the events table growing forever as the user may just want to see the latest 20 or 50 or 100 or 1000 (some limited set of real time events only as he also has the archived events table to study a past event for analysis). The purpose of the live events table is to alert the operator at the time the event occurs so that he/she can do further analysis by studying the archived events or doing some data collection on the supposedly affected elements.

Started learning Kenpo Karate

I have started attending Kenpo Karate (one of the forms of Karate martial arts) classes today at a local training institute. Today i attended the first class and learned about some basic stances and some ways to block an attack. The best part is the initial warm up exercise that we get to do in a group. It was very tiring for the first day and we did it for 30mins. There will 3 classes per week and 1.5hr per class.

It was in 1993 that i attended Taekwon-do martial art classes for 3 months but had to leave it because of the study pressure. I joined a gym when i got into engineering college in 1994 and for the first year i could go to the gym regularly but slowly with mounting study pressure i became irregular with gym too and though i continued on an irregular basis till i got my first job in 1999, i could not keep up with gyming once i was in Banglore in the winter of 2000. We had a gym in my Bangalore apartment too but i never could kindle the fire to exercise there. But it has been my desire always to keep fit (as also noticed in this blog's name: FitProgrammer@Work) and when it was time to come to the USA i decided i will get trained in Boxing in USA. But my wife was not very happy with that idea :) considering the first impression one has of Boxing is that its associated with gore. I could not find a local boxing institute in the Maynard region (the nearest was 15mi from my residence) but i could find one martial arts training center very close to my workplace (0.2mi from both my residence and workplace - literally on the way between my residence and work place). Martial arts is not going to be as dull as going to a gym where you train yourself alone and so you require to be very self motivated to persist in the regimen. My experience has been that i tend to loose the fizz with time when i went to gym. I persisted better with martial arts. Probably the idea of exercising in a group environment appeals more to my psyche.

So lets see how it goes... i will keep posting on my learnings in Kenpo(aka Kempo) Karate.

Updates:

6 May, 2007: I have learnt all that is required of a non-ranked student of Griffin Kenpo which includes,

1. Basic stances and Half-mooning
   
2. Kicks - Iron broom, dropping knees, sliding knees, front, back, side, hook, crescent, reverse crescent and their combinations.
3. Punches and 8 Blocks
4. Rolling and Falling
5. 3 techniques to free my hand when caught

Though i should have completed this learning much earlier but for some or the other reason i was not regular in my classes but the good thing is that i am continuing to learn Karate and i am enjoying it a lot.

Integrating Java and PHP: the Web Services way

Recently on a project, we were faced with the question:

1. do we put the business logic in the web tier (which was written in PHP 4.7.3) or
2. do we write the business logic in Java (better tools for development, reliable libraries for some of the tasks we wanted to perform as a part of our business logic, better OO language features than PHP, easy to debug in an IDE, code hidden in class files).

The first option was what we usually call the model 1 architecture, where business logic is written in the same script which serves the presentation code.

In the second approach, we were getting to the point of introducing the MVC pattern to our PHP based web tier by introducing a Java layer to play the model and let PHP scripts be controller and view. This had the added advantage that we could leverage the model written as Java web services in other clients (and we did have other clients than web interface too for our product which provided an alternate interface for user/program to interact with our product and hence we required that the business logic we write for our web interface could be leveraged in other client types too, written in other languages, possibly, which in our case was C/C++). Hence, we opted for the 2nd choice.

We had two options for integrating Java and PHP:

1. Do it using PHP/Java bridge
2. Expose the business logic written in Java as web services.
   

The bridge option is not well supported as of this writing in the PHP world. The Java extension for PHP language is labeled as experimental. So we decided to go with the other option.

Our business logic did not necessitate the use of EJBs (did not require container managed transactions or persistence) for the present, so going the Java path to write our business logic, we wanted to use Tomcat as web container (which could easily be integrated with the Apache web server, running the existing PHP code, using mod_jk(2)).

We developed our web services in Java using the latest JAX-WS 2.1 RI library, primarily because JAX-WS spec makes writing web services in Java pretty easy as compared to the predecessor JAX-RPC 1.1 release (aided by annotations, no more of XML hell). We also made a design choice to use the document/literal style of messaging for maximum flexibility. Here's how a typical JAX-WS code looks:

MyWebServiceImpl.java:

@WebMethod
public MyClassA myWebMethod( @WebParam(name = "aParam") int aParam) throws MyAppExceptionA, MyAppExceptionB { //.. implementation code }

We chose to use the nusoap 0.7.2 as the PHP web service stack and here's how a typical WS client was written in PHP using nusoap:
client.php:

<?php
require_once('../lib/nusoap.php');

// create a soapclient from wsdl url
$wsdl = 'http://192.168.1.101:8080/MyWebservices/MyWebServiceImpl?wsdl';
$client = new soapclient($wsdl, true);
$err = $client->getError();
if ($err) {
// Display error
}

// specify the soap body content for doc/literal request
$msg = "<web:mywebmethod xmlns:web="\"http://webservice.mycompany.com/\"">
<aparam>3334</aparam>
</web:mywebmethod>";

$namespace = 'http://webservice.mycompany.com/';
$method = 'myWebMethod'; // web method to invoke

$result = $client->call($method, $msg, $namespace, '', false, null, 'document', 'literal');

// Check for a fault
if ($client->fault) {
// Here is how we can do exception handling.
$myAppExceptionA = $result['detail']['MyAppExceptionA'];
if ($myAppExceptionA) {
print "Exception: ". $myAppExceptionA['message'];
}

$myAppExceptionB = $result['detail']['MyAppExceptionB'];
if ($myAppExceptionB) {
print "Exception: ". $myAppExceptionB['message'];
}
} else {
// Check for errors
$err = $client->getError();
if ($err) {
// Display the error - You will not come here in working condition :)
} else {
// Display the result
print_r($result);
// Grab the values in the $result array
}
}
?>

Nusoap stack on PHP end will convert the result in an associative array (as shown in the case of fault handling in the code above). To know the structure of response message to get more clarity on how to extract the data out of result, you can add the following 3 magic debug statements towards the end of the above PHP client code:

echo "<h2>Request</h2><pre>" . htmlspecialchars($client->request, ENT_QUOTES) . "</pre>";
echo "<h2>Response</h2><pre>" . htmlspecialchars($client->response, ENT_QUOTES) . "</pre>";
echo "<h2>Debug</h2><pre>" . htmlspecialchars($client->debug_str, ENT_QUOTES) . "</pre>";

I also used the SoapUI tool (version 1.6) for unit testing my web services and found it to be very useful, especially when we needed to know the exact soap request body content which we needed to embed on the PHP side while invoking the web method. You can do several things with this tools, like generate request templates which you can fill with values to test individual web methods, save such requests and do regression testing later, save the requests to do load testing, view test reports etc.

That's in short about how we can get PHP web tier to invoke methods on a Java web service based business logic tier. I had initially had some trouble in figuring out the way to pass the right arguments to the soapclient() method of nusoap library and after some web searching we could identify how to get it working.

As an aside, this was my first experience of working with PHP language and in my self learning mode i used WAMP distribution (version WAMP5_1.6.5 which includes PHP 5.1.6, Apache 2.0.59, MySQL 5.0.24a and phpmyadmin 2.8.2.4) to quickly setup an environment on my Windows workstation for experiments. I also found the EasyEclipse for PHP (version 1.2.1.1), a very useful IDE for PHP development. And a book i use as reference (only when i need to know how a certain programming thing is done in PHP) is Programming PHP by Kevin Tatroe, Rasmus Lerdorf, and Peter MacIntyre. Its excellent book and has stood by me till date.

The opinion i have formed about PHP till now is that its ideal for a model 1 architecture web application. Its better in such cases than say Servlets/JSP duo because of its simplicity in terms of programming constructs. It simply does not have those advanced features of JSP tag libraries. So for developers who are not fussy about mixing their HTML code with some PHP scripts and have a previous background in Perl or some other such dynamic languages, will be more at ease with PHP than say Java development. Java is object oriented language (a hybrid one though like C++) but PHP is object based (ie PHP has construct for defining classes as Perl but it does not support inheritance, polymorphism etc). So most of the PHP code is more or less procedural, at least thats what is more commonly done (like you are programming in C and creating libraries of functions). Also PHP eases the learning curve as its a dynamic language and does not have standard data types other than arrays (indexed and associative) and scalars. Of course you can define your ADTs but most of the time you will be good working with arrays alone. The arrays in PHP are like a mix of Java's ArrayList and HashMap ie you can use an array as an indexed or associative one. So, in general, i feel if there is a simple web application where all that has to be done is create a catalog out of a database and then allow CRUD using web forms then PHP (and hence model 1 architecture) is suitable, since it gets the work done quicker and resulting code is maintainable. But if you want to support multiple client types (hence model 1 architecture is not what you want) and probably need some advanced middleware services (like persistence, security, transactions) then you are better off with Java EE and other such frameworks.

Share your thoughts by leaving your comments.