Setting nocache headers in Struts Applications

In older versions of struts, it was a common practise to extend the ActionServlet and set the no-cache response headers there, so that they are applied to all requests.

In Struts 1.2.x, there is a easier way - without writing a single line of Java code.
Just add the "nocache" attribute to the controller element in struts-config.xml file.

<controller
processorClass="MyRequestProcessor"
nocache="true" />

Disabling keep-alive connections in Tomcat

I was using TcpTrace utility to dump my HTTP requests/responses. But the problem was that my Server (Tomcat) was using keep-alive connections and bcoz of this, the output in TcpTrace was all muddled up.
Disabling keep-alive connections in Tomcat 5.5 was quite simple:

Just add the maxKeepAliveRequests="1" attribute to the Connector tag in server.xml

CSS caching in IE

A strange problem regarding CSS caching had frustrated me for a whole 2 hours today.

The problem was reported by one of the developers that even if the CSS on the server was changed, IE was still used the cached version. I checked the application and found that there were no Cache headers that were set. This meant that IE should have made a GET "if-modified-since" request everytime for the CSS.

I even dumped the requests using the TcpTrace sniffer. I could see that IE did send a request for the CSS file and also the server responding with the new CSS file. Still no change on the page ?? This really baffled me ...If I hit F5/refresh, then the new CSS was loaded, but if I revisited the page, then still the old CSS was being used..

Then I learned from some site, that IE behaves a bit oddly here : IE usually caches CSS files by browser session (until you exit the browser). With a new session, the browser does a head request to see if the file has been modified, if so it reloads. So if I logged off the application and logged in again, the CSS would be reloaded. Strangely, this would not always happen...But if I close the browser and open the application again, I always get the new CSS..
It seems that IE is a bit erratic in this respect...so to be absolutely sure that U get the new CSS, you may have to close the browser and open it again :)

Magic of JQuery

We know that in IE, a form gets submitted whenever we press 'Enter' in a text-field. To disable it would require trapping the event in Javascript and disabling the default behaviour.

<input onkeydown="if (event.keyCode == 13){returnfalse;}" name="email">

But to do this in all text fields across hundreds of html pages would be a impossible task. Then suddenly I got the idea of using JQuery's powerful selector concept to apply this to all textfields across all pages. I just needed to add the following lines to the JS file that was present in all pages.

<script src="jquery-1.1.2.js"></script>
<script type="text/javascript">
$(document).ready(function(){
$("input[@type=text]").keypress( function() {
if (event.keyCode == 13){return false;}
} );
});
</script>

Mean, Median, Mode, Midrange and Standard Deviation

A definition of terms to help my frail memory when I need it most :)

The arithmetic mean is the sum of the numbers, divided by the quantity of the numbers.

The geometric mean of two numbers is the square root of their product.The geometric mean of three numbers is the cubic root of their product.

The quantity obtained by adding the largest and smallest values and dividing by 2, statisticians call the midrange.

The median is the central point of a data set. To find the median, you would list all data points in ascending order and simply pick the entry in the middle of that list. If there are an even number of observations, the median is not unique, so one often takes the mean of the two middle values.

The standard deviation is the most common measure of statistical dispersion, measuring how widely spread the values in a data set are. If the data points are close to the mean, then the standard deviation is small. Conversely, if many data points are far from the mean, then the standard deviation is large. If all the data values are equal, then the standard deviation is zero.It is defined as the square root of the variance.

For lists, the mode is the most common (frequent) value. A list can have more than one mode. For histograms, a mode is a relative maximum ("bump"). A data set has no mode when all the numbers appear in the data with the same frequency. A data set has multiple modes when two or more values appear with the same frequency.A distribution with two modes is called bimodal. A distribution with three modes is called trimodal.

DST implementations in Operating Systems and Java

I often wondered how operating systems knew about Day light saving offsets. Recently USA decided to change its DST settings and it immediately crossed my mind as to how Operating Systems and Applications would react to this.

Some basics first - In what format does a OS store date/time information ? All machines keep their clocks in synch with the GMT using Network Time Protocol. Then the machine calculates it local time using the local time zone settings that are stored on the machine.
Local time is always computed as = GMT + timezone offset + DST offset

But where does the OS get information about DST. All DST information is stored in a binary database called ZoneInfo database or TimeZone DB. On Solaris this database can be found at "/usr/share/lib/zoneinfo"

In Java, the JRE picks up DST information from the Timezone database stored at "%JRE_HOME%/lib/zi"

Tip: In Java, U can change the default timezone that the application uses by setting the user.timezone property
e.g. java -Duser.timezone=Area/City
or
java -Duser.timezone=GMT[+5.30]

web.xml SAXParseException in Websphere v6.1

I encountered a wierd problem while re-deploying a war file on Websphere v6.1. I started getting a SAXParseException in web.xml. The only change I had made was to introduce the "error-page" tag.

The war file was working fine in Tomcat, Weblogic and Geronimo. Digging into the logs of Websphere, I found this message:

org.xml.sax.SAXParseException: The content of element type "web-app"
must match "(icon?,display-name?,description?,distributable?,
context-param*,filter*,filter-mapping*,listener*,servlet*,
servlet-mapping*,session-config?,mime-mapping*,welcome-file-list?,
error-page*,taglib*,resource-env-ref*,resource-ref*,security-constraint*,
login-config?,security-role*,env-entry*,ejb-ref*,ejb-local-ref*)". [wsjspc]

I wondered if the order of the tags was important to WAS. So I placed the error-page tag between the welcome-file-list and taglib tags..and the SAXException disappeared :)
But I found it rather strange that the order of the tags should be important..not sure if it is part of the servlet spec.

Cool tools - Squirrel SQL Client, IzPack Java Installer

Came across 2 tools that I liked :)

Squireel SQL client: Use it to connect to any database using JDBC.
http://squirrel-sql.sourceforge.net/

IzPack Java Installer: A simple opensource free installer for Java. But this requires that JDK be present on the installation machine.
http://izpack.org/

Show friendly HTTP error messages in IE

In web.xml, I added the tag to redirect to an error page when ever a 500 internal server error occurs. But in IE, the page was still not getting displayed.

I added a sniffer to check what was going on. I could see that the custom error page was returned by the server in the HTTP response, yet IE was ignoring it. Googling around, I found out that IE has a feature turned on by default - Show friendly HTTP error messages (goto tools-> internet options-> advanced -> uncheck show friendly HTTP error message)

If I unchecked this checkbox, then the custom page was getting displayed correctly. But this was obviously not a solution that was practical. I perused thru more info about this IE feature here. IE uses this feature only if the response content for a 500 status-code response was less than 512 bytes.

So now, I suddenly had 2 solutions to the above problem (without having to change IE settings on the client)
Solution 1: Increase the content of the error page to more than 512 bytes.
Solution 2: Change the HTTP status code sent in the response header of the error page to HTTP 200.
Code snippet (at the top of the error jsp page):
response.setStatus(200);

UnsatisfiedLinkError while loading native libraries

I encountered the following problem when calling native code from a Java application deployed on Websphere.
Whenever I was redeploying the WAR file, I started getting UnsatisfiedLinkError - native library already loaded by a classloader.

I knew that the call to "System.loadLibrary" should occur only once in a classloader context. We typically put this code in a static block of a class. But it seemed that in Websphere, when the web application context (classloader) was reloaded, the native libraries were not unloaded. Hence whenever I reloaded the webapp, the above error manifested itself.

To solve the above problem, there were 2 options:
- Put the call to System.loadLibrary inside a static block of a class and load this class as a shared library in Websphere. This will load the class using the System classloader and not the web-application classloader.
- Restart the application server everytime the war/ear file was updated. This solution may or may not be feasible depending on your infrastructure setup.

Imp Caveat: Any crash in the call to the native library crashes the JVM itself. Hence for critical applications, other design alternatives should be thought of. For e.g. writing a separate application that will have a RMI/SOAP/XML interface that will do all the JNI plumbing. This separate application would run in a separate JVM outside of the JEE container JVM. Your webapp would then call this application.

Sequence numbers from a Oracle RAC cluster

Today, I faced a peculiar problem. Our application was using a Sequence to autogenerate unique numbers required for a particular functionality. When testing the application against a standalone Oracle server, the sequence numbers generated were in order.

But in the production evnironment, where we were using a Oracle RAC cluster, the sequence numbers returned were not in order..This was creating a lot of confusion. The reason was that by default sequences in 10g RAC are created without ordering. This is to reduce performance bottleneck as there would be synchronization requried across all nodes of the cluster. Each node creates a cache of the sequence as specified in the SQL.

If we need to maintain the order then we need to add the ORDER argument to the sequence.
Example of Sequence Syntax in Oracle RAC:

CREATE SEQUENCE "MY_DEV"."CON_SEQ" MINVALUE 1 MAXVALUE 999999999999999999999999999 INCREMENT BY 1 START WITH 111785 CACHE 500 ORDER NOCYCLE ;

NTLM authentication using Commons HttpClient

Recently I wanted to access a site outside the firewall thru Apache commons HttpClient. The proxy server was using NTLM authentication.

Thankfully, HttpClient had support for NTLM authentication. Here's the code snippet for the same:
--------------------------------------------------------------
HostConfiguration hConf= httpClient.getHostConfiguration();
hConf.setProxy("proxy.mycompany.com", 8080);

httpClient.getState().setProxyCredentials(
new AuthScope("proxy.mycompany.com", 8080, "domainName"),
new NTCredentials("username", "password","my_localhost","domainName")
);
--------------------------------------------------------------

Reset method of FormBeans in Struts

We all know that FormBeans in Struts can have either a 'request' scope or a 'session' scope.

In earlier versions of Struts, I recollect that the reset method of the FormBean was called by the Struts framework whenever the scope is request and not called for session scope. This was because, for 'Request' scope, a pool of FormBean objects were created.

But with new versions of JDK (especially since JDK1.4) object creation is no longer a performance penalty. In fact, short-lived objects are quickly garbage collected because they are allocated in the younger generation of the heap.

Looking at the latest Struts source code (v1.9), I saw that if the scope of the FormBean is Request, then a new FormBean object is created for each request.

Also the reset method is called for both request and session scope FormBeans.
But there is an important difference - if the scope is request, then the constructor of the formbean is also called everytime (for each request), where as for session scope the constructor would be called only once.

So, now what to do? The default reset method does nothing. We only need to override it in the following cases:

1. There are checkboxes and form-bean scope is session: In the reset method we need to reset the checkbox fields, because the browser does not submit a checkbox value if it is clear.

2. Nested beans, arrayLists are used and scope is request: In this case, we need to create a arrayList with the correct size.

JQuery Javascript library

Came across this cool cool Javascript library that adds that extra jazz to Javascript programming.

Some concepts I liked in this library:
- Separation of presentation markup with behavior (event handling etc.)
- Cool implementation of Chain of Responsibility pattern.
- Good effects such as slow show etc.

Overall, a good library that can be used in web projects.

URL: http://jquery.com/

Private network addresses

Can U look at an IP addresses and guess whether it belongs to a private network or is on the internet? The answer is yes.
The Internet Assigned Numbers Authority (IANA) has reserved thefollowing three blocks of the IP address space for private networks:
10.0.0.0 - 10.255.255.255
172.16.0.0 - 172.31.255.255
192.168.0.0 - 192.168.255.255

Earlier the IP range was divided as ClassA, ClassB , Class C etc. but now we have CIDR (Class less inter-domain routing) which allows us to slice and dice networks the way we want to.

Ruminating on dates, DST and days-between calculations

In one of my programs, I needed to calculate the difference between 2 dates in days. Ironically I could not find any method for the same in the core Java API of Calendar and Date.

So, I decided to write a utility method for the same. But then it struck me that I need to take care of DST and leap seconds. Googling around led me to a interesting journey of Atomic clocks, Quartz clocks, leap secs, etc.

Some cool links to get started with the concepts:
http://www.xmission.com/~goodhill/dates/deltaDates.html
http://www.exit109.com/%7Eghealton/y2k/yrexamples.html

Finally the algorithm I used to calculate the days-between 2 dates was:
1. Get the UTC time of both dates (imp if dates are from different locales)
2. Set the time on both dates as 12 noon. (good practice as it avoids DST and leap sec issues)
3. Calculate the milli-sec difference between the dates.
4. Divide the difference with 24*60*60 and take a Math.abs

Programming to interfaces

We all know that as a good programming practice we should always program against interfaces and not actual subclasses. That's the reason when libraris are designed, they return an interface to an collection and not the actual implementation class.

Recently the developers in my team were facing a problem regarding ClassCast exceptions using Spring JDBC when they upgraded the commons-collection package. After analysis of the source code, it was revealed that the developer was type-casting the List interface to a solid class, i.e. the LinkedHashMap class.

Spring JDBC decides at runtime what Map implementation should be returned. If the latest version of jakarta-commons is available in the classpath, an implementation of org.apache.commons.collections.map.ListOrderedMap is returned, else java.util.LinkedHashMap is used.

Hence the program would start throwing ClassCastExceptions ...See log below comparing what happens depending on the classpath jars:

-- When the latest version of jakarta-commons is not there in the classpath
16-Feb-2007 14:29:47 DEBUG (CollectionFactory.java:142)
org.springframework.core.CollectionFactory -
Falling back to [java.util.LinkedHashMap] for linked case-insensitive map

-- When the latest version of jakarata commons is present
16-Feb-2007 14:31:36 DEBUG (CollectionFactory.java:138) org.springframework.core.CollectionFactory -
Creating [org.apache.commons.collections.
map.ListOrderedMap/CaseInsensitiveMap]

Moral of the story: Always program to interfaces.

Starting a java daemon on Solaris

We were using Quartz as a scheduler to run our cron-like java programs on Solaris. Quartz gives us much more flexibility than the cron-tab, as we can add listeners and plug-ins.

Normally, when we exit a shell, all the programs that were started in the shell are terminated.
To avoid this, we made use of the 'nohup' command that keeps running the program even if the shell is closed. nohup = no-hang-up.

Also important thing to remember is that if the Std. input is kept open, then the shell does not exit. In the Java program, the main method automatically opens the Std input stream. Hence we need to close it. Here is the complete syntax.

nohup java com.domain.main_class <&- &

The ampersand at the end, starts the program in the background. nohup creates a file called nohup.log that contains the output to the standard stream.

Daylight saving time

Being brought up in India, we never have to worry about Daylight saving chores. But its very interesting and also important for any system designer to know all about DST.
More info at this link : http://webexhibits.org/daylightsaving/

DST also comes into play when we schedule jobs using cron or Quartz.
For e.g. In the US, many states implement DST at 2 a.m.
If we have a job scheduled at 2.15 am, then it won't be fired on the day when the cock is shifted ahead from 2.00 to 3.00 am :)

Spring JDBC quirks

Spring JDBC's JdbcTemplate provides us with utility methods that return scalar values. For e.g. methods such as queryForObject, queryForInt, queryForLong etc. These methods are useful when we need to obtain a single row or single column from the database.
But there is a caveat here. Each of these 'scalar' methods throw an IncorrectResultSizeDataAccessException if the there are no results or more than one row in the resultset.

For e.g. if there is a valid business condition that no rows may be returned for a particular query, then we need to handle this exception and return null. The other option is to use the generic query methods that return a list. For empty resultsets we can return an empty list.

Printing Stack Trace in Java

Quite often, developers use the statement exception.printStackTrace() to print the stack trace on to the console. It is important to remember that printStackTrace method prints the stack trace to the System.Err stream and not the System.Out stream.
Due to this, there are a lot of exception stacks getting printed in the System.Err files on the application server, but we do not see any trace of it in the Log4J files or System.out files !!!!

As a best practice, I strongly recommend not using e.printStackTrace() in production code. If the project is using Log4J, then the method 'Logger.error(message, exception)' would print the stack trace to all the registered appenders.

Alternatively, I have written a utility method that would return the stack trace as String. Please use this method if you want to get a stacktrace as a string for more flexibility in printing.

---------------------------------------------------------
/**
* Gets the exception stack trace as a string.
* @param exception
* @return
*/
public String getStackTraceAsString(Exception exception)
{
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
pw.print(" [ ");
pw.print(exception.getClass().getName());
pw.print(" ] ");
pw.print(exception.getMessage());
exception.printStackTrace(pw);
return sw.toString();
}
---------------------------------------------------

But what if there is already production code that uses e.printStackTrace() and System.out and you want to redirect them to the Log Files. In that case, we can make use of the System.setOut() and System.setErr() methods. Also try the LoggingOutputStream class in the contribs/JimMoore folder of the log4j release.

Redirecting system.out, system.err to a file on Unix/Solaris boxes

It is simple to redirect a process output to a standard file using the '>' operator.
But if you also want the std error to go to a file, then you need to type:

java className>file 2>&1

To understand what the above command means, it is imp to note that the shell reads arguments from left to right. The shell sees >file first and redirects stdout to file. Next 2>&1 sends fd2 (stderr) to the same place fd1 is going - that's to the file.

Epoch - The start of time for computing

Recently working on a XML data feed program, I had to handle empty date XML elements. I wondered as to what date I can put into the database - the start of time?. What is the start of time used by computer systems?

As Java programmers we all know that the Date, Calendar classes in Java use January 1, 1970 as the start of 'time'. This is known as Epoch. Wikipedia defines epoch as follows:
"The epoch serves as a reference point from which time is measured. Days, hours and other time units are counted from the epoch, so that the date and time of events can be specified. Events that took place earlier can be dated by counting negatively from the epoch."

There are different epocs used by different systems:
- Unix and Java use Jan 1, 1970
- MS DOS uses Jan 1, 1980
- Excel and Lotus = Dec 31, 1899

More info can be found here.
As for me, I decided to use the Java Epoc for my program :)

Using ASCII mode for FTP of text files

Last week, I ran across a common problem -- I had transferred a INI file from Windows to Solaris in binary mode and the file did not work on Solaris. This was obviously due to the difference in newline characters.

I then realized that if we transfer in ASCII mode, then the FTP program automatically converts between windows and unix line feed characters :)
More info at these links:
http://www.editpadpro.com/tricklinebreak.html

Solaris tips - find out RAM and HD size

A few quick commands to help developers new on Solaris 10:

To find out the amount of RAM on the system.
> prtconf / grep Mem

To find out the Hard Disk capacity
> iostat -E / grep Size

To find out the file-system size
>df -h

To find out the CPU usage
> mpstat 12 5
> ps -e -o pcpu -o pid -o user -o args
> top

The following links give more info on common commands:
http://www.cyberciti.biz/tips/how-do-i-find-out-linux-cpu-utilization.html
http://www.geocities.com/arndike/Solaris_probe_process.html
http://projects.ericshalov.com/technotes/notes/solaris

Getting the Client IP address on a J2EE server

The HTTPRequest object has 2 methods 'getRemoteAddr()' and 'getRemoteHost()'. These methods would return us the IPAddress/HostName of the last proxy or the client. So if the client is behind a proxy, then we would get the IPAddress of the proxy.
Some proxies (known as transparent proxies) pass the actual IP address of the client in the header HTTP_X_FORWARDED_FOR.
So on the server side we can code something like this:

if (request.getHeader("HTTP_X_FORWARDED_FOR") == null) {
String ipaddress = request.getRemoteAddr();
} else {
String ipaddress = request.getHeader("HTTP_X_FORWARDED_FOR");
}

But if the proxy is an anonymous proxy, then even this won't work. So the only way to get the Client Address correctly is using an Applet to capture the IP address of the client. For this, the client should be trusted and signed.
Another option that I came across in a forum is to create a Socket connection back to the web server from which you came and asking the Socket for the local address:

URL url = getDocumentBase();
String host = url.getHost();
Socket socket = new Socket(host, 80);
InetAddress addr = socket.getLocalAddress();
String hostAddr = addr.getHostAddress();
System.out.println("Addr: " + hostAddr);

Secure FTP options

I was looking for an secure option to FTP files to a remote server.
There are 2 options available: FTPS and SFTP....and I was a bit confused in understanding the differences between the two.

FTPS is FTP over SSL/TLS.
SFTP (Secure FTP) is FTP tunneled thru SSH.

SFTP provides an interactive interface that is similar to that of FTP. In addition, the command line version of SFTP is scriptable in that it allows you to specify a batch file to control the file transfer process.

If U are using VPN, then another option would be to constrain FTP access to circuits created over VPN connections, then as long as the VPN is secure end-to-end, FTP will also be acceptably secure.

File Encryption utility for windows

I was looking for a small light-weight encryption utility for encrpyting my personal information in files. There are a plethora of programs available on the internet, but I found the following ones cool and easy to use.

Kryptel Lite: http://www.kryptel.com/products/krlite/
Iron Key: http://www.kryptel.com/products/ikey/index.php

FTPing in Java

For performing FTP operations programmatically, I use the Jakarta Commons Net Package. This is a super-cool utility package that makes writting FTP programs a breeze.
The jar file can be downloaded from here.

Tip: The jakarta-commons-net package also requires the ORO package which can be downloaded here.

Another important concept is that of Active and Passive FTP. A good explanation of the differences can be found here.
As a general rule, passive FTP is used as it works behind firewalls too.

The windows FTP can also be run unattended using the '-s' option and specifies a script file that contains the FTP commands.

URL encoding and XML/HTML encoding

Recently in one of our applications we were passing XML data as a POST parameter to the server. For e.g. xml={xml string here}

We were using XML encoding to replace special characters in the XML. For e.g. '&' was replaced with '&amp;' '<' was replaced with & lt; etc.

But on the server side whenever we were trying to get the XML string as a request parameter, we were getting a truncated string. After putting in a sniffer, I found out the reason why this was happening - I was getting confused between URL encoding and XML encoding.

In URL encoding as we know, we need to encode special characters such as '&'. So we replace them with '&amp'. But what if '&' was part of the data; i.e. an address contained &. In this case the URL encoding function should just replace '&' with '%26'. Otherwise the server would treat it as a parameter separator and break the string on the first occurance of '&'.

Now our XML string (passed as a POST parameter) contained the & character, because all XML special characters were encoded with a '&' at the beginning. So the trick in this case was to perform URL encoding of the element values after the XML encoding. Confused??

Ok...consider the following XML element:
<name>Naren&Sons<\name>
Now the above element data contains an & and hence would result in a XML parsing error. Hence I XML encode the data. So it would not look like:
<name>Naren&amp;Sons<\name>

But when this string is passed as a HTTP GET/POST param, then it would be truncated.
Hence we have to URL encode it - replace & with %26
<name>Naren%26amp;Sons<\name>

So when the string reaches the server, the request.getParameter would perform the URL decoding and then we passed the XML string to Castor or any other XML parser.

Another good design to avoid all this confusion is to pass the XML in the request body stream and get the XML on the server side using request.getInputStream().

Even ASP.NET has 2 methods: HtmlEncode and UrlEncode.
HtmlEncode converts the angle brackets, quotes, ampersands, etc. to the entity values to prevent the HTML parser from confusing it with markup.
UrlEncode converts spaces to "+" and non-alphanumeric to their hex-encoded values. Again this is to prevent the the URL parser from misinterpreting an embedded ?, & or other values.