Thursday, December 27, 2007

Difference beween WCM and ECM

Doing some research on Content Management systems, I got a bit confused about the difference between Web content management systems and Enterprise Content Management systems.

This article is a good read to understand and appreciate the differences.
The lines between all content technology families are very blurry and hence the auhor suggests that we evaluate a product based on the scenario requirement.

For publishing company websites, intranet sites, a WCM may be a good solution. A ECM solution may be focussed towards a particular industry.

Difference between a portal and a website

A lot of my friends ask me the difference between portals and web-sites. Typically an organization would have both - a company website and a portal. I found a good explanation here.

Portals typically have authentication and provides us information based on who we are. Personalisation and customization are the hallmarks of portals. So we require portals because:
Different roles require different information. Someone from grounds and buildings needs different info than the chair of computer science. (Customization) Different people with the same role work differently. (Personalization) Efficiency - people get directly to the info they need. (Work Flow) Customization insures they don't miss anything.

Wednesday, December 12, 2007

ThreadPool in Java

Recently one of my friends was looking for a good Thread-pool implementation in Java and asked me for help. I had used Doug Lea's ThreadPool in the past and recommended it to him.

But browsing thru the web, I learned that JDK 1.5 has introduced a java.uti.concurrent.* package that contained a good ThreadPool implementation. Using it was also a breeze. Here are a few code snippets.
ExecutorService pool;
pool = Executors. newFixedThreadPool(3);
pool.execute(new Task( Task(“three”,3));
pool.execute(new Task( Task(“two” two”,2)); ,2));
pool.execute(new Task( Task(“five five”,5)); ,5));
pool.execute(new Task( Task(“six”,6);
pool.execute(new Task( Task(“one”,1);
The Task object is a Thread that implements the Runnable interface and has the run() method. The concurrent package also contains a lot of helper classes that can be used - for e.g. using a thread-safe collection, atomic varibles etc.

Friday, December 07, 2007

Websphere SOA Stack

A lot of people get confused between the different product offering by IBM in the Websphere space.The major products in the Websphere family are Websphere Application Server, Websphere ESB, Websphere Process Server and Websphere Message Broker.

Process Server is based on ESB and ESB is based on Application Server. Message Broker is termed as "Advanced ESB" by IBM and should be used when we have extensive heterogeneous infrastructures, including both standard and non-standards-based applications, protocols, and data formats.

So when to use what? Use Websphere ESB whenever most of the services you want to integrate are based on SOAP, XML standards. The Websphere ESB product can also be used as Websphere Application Server as it is built on the same core. Though Websphere Application Server can host simple services, Websphere ESB gives out of the box support for sophisticated mediation flows found in ESBs. We would need Websphere Process Server whenever we need to orchestrate simple services to create business services and host them.

Another technical difference is that Websphere ESB uses the JMS provider in WAS for messaging, whereas Message Broker uses Websphere MQ as the messaging engine. This is due to legacy reasons, I believe.

More information can be found here.The FAQ available at IBMs site also contains some good info.

Thursday, November 29, 2007

Oracle Fast Connection Failover for a RAC cluster

Problem:Whenever one node of a Oracle RAC cluster goes down, some of the connections allocated to that node go stale in the connection pool. This causes errors such as 'Broken pipe' to be raised whenever the stale connections are used.

Solution:Configure a ONS (Oracle Notification Service) remote subscription client on the App Servers. Whenever any RAC node goes down, a 'DOWN' event would be raised and sent to all subscribers. The ONS client on the Appservers would receive this event and refresh all stale connections in the pool. When the node comes up, a 'UP' event is sent to the AppServer ONS client and again the connections in the pool are balanced against both the nodes.

In Oracle 10.2g, to configure ONS, U just have to add ons.jar file to WEB-INF/lib and 2 lines of code to configure the Connection Cache.

3 easy steps to self-sign a Applet Jar file

Found this link that explains how to self-sign an applet in 3 easy steps:

1. keytool -genkey -keystore myKeyStore -alias me

2. keytool -selfcert -keystore myKeyStore -alias me

3. jarsigner -keystore myKeyStore jarfile.jar me

Thats it..simple and sweet :)

Tuesday, November 27, 2007


Came across this beautiful AJAX control that can be used to emulate a HTML editor text box. - worth a look.

Monday, November 26, 2007

Setting focus on the first field

Whenever a HTML page is loaded, we often want to set the focus on the first input field of that page. Came across this Javascript code that can be put in a JS file and reused everywhere. Simple and Sweet solution :)

Tuesday, November 13, 2007

Reflection in Java-XML binding frameworks

Almost all Java-XML binding frameworks use Java reflection to map Java objects to XML documents. I had used Castor and XMLBeans in our projects and were satisfied with the performance of both of them.

Recently a friend of mine told me that he had reservations on using Castor as it uses Reflection and hence cannot be used in high-performance applications. I contemplated on this and felt that some basic amount of reflection code would be present in any XML-binding framework; or how would the API call 'set' methods on Java objects during unmarshalling?

I found this discussion on the net that explains this same fact.

Castor will still use some amount of reflection (mainly to set values onto your domain objects) even if you are using a mapping file (a binding file is used during code generation only).

Having said that, let me add one thought. At the time most of these articles have been written (2001 to 2002), the use of reflection introduced some performance penalties, noticeable to some degree.

Since then, a lot of time and effort has gone into improving the JVMs/JDK/compilers to lessen this impact further and further. In my personal view, with the event of JDK 1.4 (and even more so with Java 5)
this problems have been addressed sufficiently in that I would not call the use of (moderate) reflection an issue any more. If use of reflection would still be an issue, frameworks like Spring and some of their most
powerful features (run-time AOP through Spring interceptors) would not have been adopted by the market as it happened in the last one or two years.

Btw, in case this is not clear enough, using a mapping file instead of Castor's 'introspector' to establish mappings between the Java domain classes and the XML artefacts does not really improve Castor's
performance, as both modes will cause Castor to use reflection to establish what methods to use on your Java objects to set a value, to get a value (from an instance variable) or to add a value to a collection member. Bear in mind that these activities happen at start-up time only, and thus once and once only.

At run-time, when e.g. unmarshalling an XML document to a predefined object hierarchy, Cstor will (and has to) use reflection to ...
a) create object instances (using Class.newInstance())
b) set and get values into/from your instance variables (Method.invoke())

Wednesday, November 07, 2007

SQL to find out indexes on a table

Quick reference to find out indexes on a table from the command prompt.
select * from user_indexes where table_name = 'MY_TABLE'
select * from user_ind_columns where index_name = 'PK_MY_INDEX'
To find out the triggers on a table:
select * from user_triggers where table_name = 'MY_TABLE'

Tuesday, November 06, 2007

Disk usage commands on Solaris

du and df are the most powerful commands on Solaris to find out the disk usage.
Here is a list of the most useful commands at a glance.
- df -h (shows the file system partitions)
- du -ks /opt/* | sort -nr | head -5 (shows the top 5 dirs in /opt. Subdirs are summarized.)
- du -ak /opt/* | sort -nr | head -5 (shows the top 5 dirs/files in /opt including subdirs)

Monday, November 05, 2007

Modal Dialog session expiry issue in IE 6

In our application we were facing a very queer problem. One of our page flows was opening a modal dialog from another modal dialog. While doing this we were facing an intermittant problem of 'session-expiry'. I put a sniffer on the client machine and saw that a new sessionID was generated. This meant that the JSESSIONID cookie was not being propogated to the child windows.
I also realised that this problem was only with 'Modal Dialogs' and hence IE specific.

I tried in vain to understand where we were doing things wrong. Lady luck was shining on me, when I found this link that explains the problem.
Snippet from the microsoft site:

When Internet Explorer opens a window from a modal or modeless HTML dialog box by using the showModalDialog method or by using the showModelessDialog method, Internet Explorer uses Microsoft Component Object Model (COM) to create a new instance of the window. Typically, the window is opened by using the first instance of an existing Internet Explorer process. This process is different from the process that Internet Explorer uses to open a new window by using the method.
When Internet Explorer opens the window in a new process, all the memory cookies are no longer available, including the session ID.

The workaround was to pass the window object of the parent of the dialog box into the dialog box, and then use that object to open the new window.

For e.g. In the parent window:
var args = new Object;
args.window = window;
showModalDialog("modal.asp", args)

And in the child window:'page1.asp')

Thursday, October 25, 2007

Integrating JAMon war with your application on Websphere

We were using the JAMon package for performance monitoring. We needed to integrate the JAMon WAR file contents into our application so that we could get the web-pages used for monitoring the application.

On Websphere 6.1, the class-loader hierarchy and our security policies did not allow for JAMon libs to be added at the system class-loader level.
One option was to include the contents of the JAMon war file into our WAR file.

But I thought a cleaner option would be to make a EAR file with 2 web-modules (one war of my application and other jamon.war)
On WAS 6.1, I changed the class-loader policy of the application to: Single class loader for application.

The application.xml file for the EAR file contained the following items:

<application id="Application_ID">


Wednesday, October 24, 2007

Access predicates and Filter predicates in Explain plan

In Oracle 10g, when we see the explain plan for a SQL, then there are 2 new items that are of interest - the 'Access predicate' and the 'Filter predicate'.
We know that depending on the predicate information ('where clause') the Oracle optimizer chooses the optimal path for executing the query.

So the 'access predicate' information tells us how the Oracle optimiser is accessing the rows of the table - i.e. if the optimizer is using an index, then what predicate made it use the same.

The 'filter predicate' tells us what criteria Oracle is using to filter rows from the returned rowset (the rowset was fetched based on the 'access predicate') .

Tuesday, October 09, 2007

Webpshere v6.1 deployment hack

Whenever I wanted to change a single file on WAS server, I often used to copy that file to the 'installedApps' EAR directory of the application.

This used to work and though not mentioned in any IBM redbook; was considered as a hack by our team. But one day we ran into a problem which took us ages to decipher.

Whenever we changed the web.xml in WEB-INF, the changes were never reflected. For e.g. if we added a new struts-config entry to web.xml or a new filter, the configuration was never picked up.
We never really noticed this and thought the problem was with our struts-config file.
But the core issue was that WAS always picks up the web.xml file of an application from a different directory

I changed the web.xml file here and all issues with struts-config were solved :)

JAMon JDBCMonProxy -getting the orginal connection

While using JAMon to monitor SQL perf stats, we faced a small problem - at one place we needed the underlying OracleConnection object and not the proxy, as we were calling a vendor-specific method on it.
Thankfully the JAMon API MonProxy object has a "getMonitoredObject()" method through which I could get hold of the OracleConnection object and use it.

Here is the sample code snippet: (just in case someone else needs it :)

// we need to do this as the 'setEndToEndMetrics'
// method is only available on an OracleConnection.
if (Proxy.isProxyClass(connection.getClass()))// if a dynamic proxy object
InvocationHandler invocationHandler =
Object monitoredObject =
(MonProxy) invocationHandler).getMonitoredObject();
((OracleConnection) monitoredObject).
setEndToEndMetrics(metrics, (short) 0);

Friday, September 28, 2007

rm craziness

I was trying to delete some files on Solaris using the rm command.
The file names were something like: "-HqFbc3YQ0wQYhH7u16veP7_BBC_6_090540.xml"
These files were generated by a program using random numbers.

Now no matter what I tried, rm always returned with an error for files starting with a - (minus sign)
I found the solution here.

To remove a file whose name begins with a dash ( - ) character, refer to the file with the following syntax:
rm ./-filename
Using the redundant ./ directory information prevents the dash from occurring at the beginning of the filename, and being interpreted as an option of the rm command.

Wednesday, September 12, 2007

What does a Digital certificate contain?

If you view a digital certificate using FireFox browser or decoding it using some tool such as OpenSSL/IKeyMan; then we can see the following parts/data:-
- Issued by {The CA name}
- Issued to/Subject {CN of the subject (typically the DNS of the server)}
- Valid From {date}
- Valid Till {date}
- Public Key {typically the 1024 bit public key of the subject}
- Public Key Algorithm {The encryption algorithm to be used with the key, e.g. RSA}
- Signature Algorithm {The algorithm used to generate the digital signature, e.g. MD5HashWithRSA, SHA1HashWithRSA}
- Signature {The digital signature}

The digital signature is created by hashing the public key of the subject using MD5 or SHA1 and then encrypting the hash using the private key of a CA.

An entity verifying this digital cert would:
- decrypt the hash using the public key of the CA
- create a new hash of the public key of the subject
- compare both the hash keys and verify if the subject can be trusted.

Formats for Digital certificates

I knew that all Digital certs confirm to the X.509 standard. But why were they so many extensions for certificate files? - e.g. DER, CER, .P7B, PFX etc. I found this link that contains good info about X.509 certs.
Basically a X.509 cert can be in many formats (depending on the encoding used to save the cert)

.CER - CER encoded certificate, sometimes sequence of certificates
.DER - DER encoded certificate
.PEM - (Privacy Enhanced Mail) Base64 encoded DER certificate, enclosed between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----"
.P7B - See .p7c
.P7C - PKCS#7 SignedData structure without data, just certificate(s) or CRL(s)
.PFX - See .p12
.P12 - PKCS#12, may contain certificate(s) (public) and private keys (password protected)

A .P7B file could be a container for more than one digital certificate.

Adding trusted root certificates in Websphere 6.1

We had deployed an application to WAS 6.1 ND. This application contained a JAX-WS 2.0 webservices client that used to call a third-party service using SSL. The digital certificate used by the third-party was self-signed and hence we needed to accept it as a trusted party in our Trust Store.

We imported the certs into a trust-store (JKS format) using the keytool command of the JDK and wrote the following code:
("", trustFilename );
("", "changeit") ;
("", trustFilename );
("", "changeit");

But unfortunately this was not working on WAS6.1. (This works fine on Tomcat).
In earlier versions of WAS, the iKeyman tool provided an interface to manipulate digital certs, keys, Trust stores and Key stores. But in WAS 6.1, all these tasks can be done from the web-based admin console.

So to add the certs to the Trust store, I went to "Security (on the left panel) --> SSL certificate and key management > Key stores and certificates > NodeDefaultTrustStore > Signer certificates"
Add the new root digital certs that need to be trusted to this store. The JAX-WS client should now be able to connect to the HTTPS resource. Remove any system properties that have been set before.

A good article (for WAS 6.0) describing how SSL and Digital certs work in Websphere can be found here.

Hack to deploy JAX-WS 2.0 webservice on Websphere 6.1

Our team had developed a webservice using JAX-WS 2.0 and using the Reference implementation of Sun as the JAX-WS provider. The application was developed using Netbeans and the embedded Tomcat server.

But when the application was deployed on WAS 6.1, then it failed and resulted in ClassCast exceptions. The base installation of WAS 6.1 only supports JAX-RPC 1.1.
IBM did provide an option - installing the webservices feature pack for WAS 6.1
The installation was a bit of a pain, especially on a cluster. You essentially had to create new AppServer profiles, create a new cluster and move all applications to the new cluster. This would also required considerable downtime on a live production server.

I looked for an other option. I tried to understand what was happening under the hoods when Netbeans was creating a webservice. It was clear that the webservice created was using a Servlet provided by the RI of JAX-WS 2.0. Even all the annotations and JAXB libraries were included in the lib directory. Hence there was no dependancy on any server-specific library as such.

I realised that the ClassCast exceptions on WAS 6.1 were happening due to the ClassLoader heirarchy - which was set to 'Parent First'. I decided to try to change this configuration to 'Child First' - so that JAX-WS RI libraries are picked up from WEB-INF/lib first.

There were 2 changes that I made using the web admin console. Go to "Enterprise Applications --> {Application name} --> Class Loader" and changed the following items:
Select "Classes loaded with application class loader first" and "Single class loader for application". Save these changes to the master configuration and restart the AppServer.

This hack worked and we could deploy JAX-WS 2.0 webservices on WAS 6.1 :)

Tip: We ran into a few Linkage errors, which we could resolve by removing all duplicate jar files.
A linkage error would typically occur when a class tries to call another class loaded by a different class loader.

Is POX/HTTP same as REST?

There is a lot of confusion in the industry over REST webservices. A lot of applications offer POX/HTTP interfaces and call them REST webservices. But in theory, just having XML tunneled over HTTP does not satisfy all REST principles.

It's important to understand that REST is not a protocol. It is an architecture style that can be used to design systems. It brings in a noun-oriented approach to access resources (instead of a verbs).

The Apache CFX framework has new features (using the magic of annotations) that make it possible to have true REST style webservices.
More information on this can be found here and here.

Recently came across this article that states the REST principles that should be followed during design:

1. Give every resource an ID. Use URIs to identify everything that merits being identifiable, specifically, all of the "high-level" resources that your application provides, whether they represent individual items, collections of items, virtual and physical objects, or computation results. Examples:

2. Link things together. (Hypermedia as the engine of application state)

Example: Element in XML doc: product ref=''

We could use a simple 'id' attribute adhering to some application-specific naming scheme, too but only within the application’s context. The beauty of the link approach using URIs is that the links can point to resources that are provided by a different application, a different server, or even a different company on another continent because the naming scheme is a global standard, all of the resources that make up the Web can be linked to each other.

3. Use standard methods. For clients to be able to interact with your resources, they should implement the default application protocol (HTTP) correctly, i.e. make use of the standard methods GET, PUT, POST, DELETE

e.g. GET - get order details/list all orders

PUT - update order

POST - add order

DELETE - delete order

4. Communicate statelessly - This is for scalability.

Tuesday, September 04, 2007

Message style and encoding in WSDL documents

The WS-I specs state the following 2 types of encoding to be complaint to the Basic profile:
- RPC/literal
- Wrapped document/literal

Found this cool article on developerworks that explain all the styles and encoding in detail.

WSDL cheat sheet

With the plethora of tools available today for webservices development, we sometimes forget the basic fundamental semantics of a WSDL document. Here is a quick reference of what a WSDL contains:
1. Types: These are the data type definitions - expressed as a schema of complex/simple types.
2. Message: A message consists of a logical order of one or more types.
3. Operation: Defines a operation that consists of a input and output message.
4. Porttype: A collection of operations.
5. Binding: Specifies the concrete protocol data format specifications (e.g. document-literal) for a portType.
6. Port: Specifies a network addresss for a binding, thus defining a single communication endpoint.
7. Service: collection of endpoints or ports

Thus a service is defined by one or more endpoints/ports. A port is combination of a binding with a network address. Each binding is a combination of a porttype and concrete data format specifications. Each porttype is a collection of operations that define messages that are exchanged.

More information can be found here.

Monday, September 03, 2007

Xerces and Xalan packages in JDK 1.5

Since JDK 1.5, Sun has decided to change the package names of the Xalan and Xerces to "" and "". Old package names (org.apache.xalan.xslt etc) are now not shipped with JDK1.5

I think this is a welcome move, bcoz this will enable end-users to use new versions of Xerces and Xalan easily without resorting to class-loader manipulation. Earlier this was done using the endorsing mechanism for JAR files.

Friday, August 31, 2007

HTML menu DIV tag over a SELECT element

We are using a JS menu that used layers created by div tags. The problem was that this layer was hiding behind behind 'select' dropdowns in the page.

I googled around for a solution and found out 2 solutions:
- Hide the select boxes when the menu is clicked and unhide them when the menu rolls back.
- Use the zIndex property and create a transparent IFrame under the menu layer.

The following links were of great help to me:
BEA portal menu.
IFrame option.
Hiding option.

Wednesday, August 29, 2007

Heap size and Perm size

I was getting confused if the heap size specified by -Xmx included the Perm size or it did not? The GC document on Sun's site showed the Perm generation to be part of the Heap size, but I think this is wrong. It seems that the Perm size is different from the Heap size. I found this link that confirms this belief.

The "permanent" generation in the Sun JVM is for class objects, method objects, static fields for those classes (just the static fields, not the objects referenced from the static fields: those are in the regular Java object heap because Java code allocates them), and random other bits like any interned Strings etc. So if U are using a lot of reflection in you code, then a lot of method objects and class objects are created at runtime. For e.g. XML binding frameworks such as Castor etc. use reflection by default.

Hence the total heap size used by the JVM is the Permanent Generation size + the -Xms/Xmx size.

The total memory used by the JVM process will be = Memory used by the JVM for internal management + memory allocated in native code + memory allocated for the permanent space.
This is the memory that U see in Task-Manager in Windows or 'top' on Solaris.

What happens when we allocate more Java heap than the max physical memory available?
The OS would start swapping the process to disk (depending on the Virtual memory configuration) if the total memory reserved by all applications and the OS in the system exceeds the physical RAM installed. This would seriously degrade the performance of the system.

*IMP*: If your application is throwing an OutOfMemoryError then it could be because the PermSize is full, rather than the heap size.

Saturday, August 18, 2007

HttpClient in a multi-threaded environment

Our application was using HttpClient library to connect to various third-party systems. I was proposing to develop a smart resuable component that could be used by other modules and also suffice the performance requirements of the application.

I was joyed to see how the design of HttpClient offers excellent support for multithreaded environments.
The HttpClient and ConnectionManager classes can be shared across the entire application. Each respective thread of execution must have a local instance of HttpMethod and can have a local instance of HttpState or/and HostConfiguration to represent a specific host configuration and conversational state.
Hence these must be local variables in all the methods. At the same time the HttpClient instance and connection manager are shared among all threads for maximum efficiency.

If an proxy configuration is required, then this information can be passed in the execute() method of PostMethod using the HostConfiguration class.

Wednesday, August 15, 2007

Finding the version of Solaris

Commands to find out the version of Solaris the box is running on:
uname -X
more /etc/release

Wednesday, August 08, 2007

Viewing large files on Linux/Solaris

Log files of J2EE applications can become very big. In our application, the log files were of the size ~ 700 MB to 1GB.
To search such big files for information the following unix commands can be used:
- less fileName.txt
To search a string in the forward direction
- /searchString
To move to the next search
- n
To search backwards
- ?searchString

Javascript goodies

Today, I spend quite some time refactoring messy Javascript code:

1.) I needed to extract a number appended to the end of the string - use the power of regular expressions to make this one clean and sleek.
var RowIdNumber = rowId.match(/\d+$/)[0];
2.) To obtain a 'anchor' element inside of a table row and invoke it's click (href) -
Enjoy :)

Tuesday, July 24, 2007

Java Applet Caching Issues

I spent the last 3 days in frustration trying to get Java Plug-in cache mechanisms to work.
Since JDK 1.5, the Java Plug-in has advanced caching options using the "cache_archive" and
"cache_version" tags.

One particular vexing issue that I faced was that inspite of adding all the cache directives in the applet's object tag, the jar files were not cached. A quick google showed 2 bugs:
1. Unfortunately, if the servers do not return Last-Modified or Expires in HTTP response headers, it would disable plugin caching for the connection, and plugin would try to redownload the file again without caching. This results in two HTTP GET requests per file.
2. If a running applet makes a call to URLConnection.setDefaultUseCaches(false), then all subsequent loads from this plugin will exhibit the multiple-load-request behaviour.

In our case, the Applet was setting the cache as false in the code. Hence the cache directives were not working as expected. More information on this can be found here and here.

Search functionality in Textpad and EditPlus

A lot of people use text editors such as Textpad and EditPlus. My favourite is Textpad; it can handle large files and also has a very good interface.

Many times we need to search for muliple words in a line. For e.g. in a log file, search for all lines having words 'abc' and 'xyz' on a single line. Using the regular expression search facility available in these editors, this task becomes a piece of cake.

RegEx for the above task: abc.*xyz
We can keep on adding more words - abc.*xyz.*123

A simple feature, but very powerful while analysing logs etc.

Sunday, July 22, 2007

Favicon is a bit messy

Recently when I was analysing the log files on the server, I came across many reqeuests for a resource - favicon.ico
This was strange, since I knew that there was no such file on the server - so why were browsers making a request for this file.

I did a quick google and came to know about this interesting things:
- Favicon stands for "Favourites Icon". Its the small image U see in front of the URL on the addressbar.
- The icon looks good, but the problem is that some browsers keep on making a request to this file everytime the page is loaded !

Here are some links having more info:

Friday, July 20, 2007

Cool collection of JS code

Found this link that contains a cool collection of Javascript code. Worth a perusal.

I liked the fee menu javascript code available here:

Wednesday, July 18, 2007

Interpreting 2 digit years

As a good practice, it is always recommended to use 4 digits to specify the year. But a lot of applications may have a front-end that accepts 2 digit year as the input from the user. Or there could be a intergration with a third-party that requires us to parse 2 digit years.

We all understand the ambiguity in parsing 2 digit years. The logic that you would need to use depends on the business requirement. For e.g. if you are accepting birth-year then it cannot be greater than today and we can go 100 yrs back.

So how do different API's handle it?

The SimpleDateFormat class follows the following logic:
"For parsing with the abbreviated year pattern ("y" or "yy"), SimpleDateFormat must interpret the abbreviated year relative to some century. It does this by adjusting dates to be within 80 years before and 20 years after the time the SimpleDateFormat instance is created."

The Joda Time API takes a different approach:
"What is particularly interesting about this format is the two digit year. Since the interpretation of a two digit year is ambiguous, the appendTwoDigitYear takes an extra parameter that defines the 100 year range of the two digits, by specifying the mid point of the range. In this example the range will be (1956 - 50) = 1906, to (1956 + 49) = 2005. Thus 04 will be 2004 but 07 will be 1907. This kind of conversion is not possible with ordinary format strings, highlighting the power of the Joda time formatting architecture"

The default setting in Windows 2000 m/cs:
"Under Windows 98 or Windows 2000, two-digit years for the Year argument are interpreted based on user-defined computer settings. The default settings are that values from 0 through 29 are interpreted as the years 2000–2029, and values from 30 through 99 are interpreted as the years 1930–1999. For all other Year arguments, use a four-digit year; for example, 1924."

The Internet mail standard uses the following:
"Two digit years are treated as valid in the loose translation and are translated up to a 19xx or 20xx figure. By default, following the specification of RFC2822, if the year is greater than '49', it's treated as being in the 20th century (19xx). If lower, or equal, then the 21st (20xx). That is, 50 becomes 1950 while 49 is 2049."

JS library for dates

I was looking for a reusable JS function that would allow me to format and compare dates just the way it is done in Java using classes such as DateFormat, Date etc.

I found this JS file on the web - simple and easy to use. And quite powerful too:

Saturday, July 14, 2007

Difference between empty string and null in Oracle Database

In Oracle, when we store a empty string i.e. "" through JDBC, then Oracle stores it as a NULL value. This causes a lot of confusion to Java developers. In Java a null and a "" are 2 different entities.

Hence even if you store an "" string when inserting into the database, when you retrieve the results, you would get the string as null.

If we do a getInt(), getFloat() etc. then we would get 0.

Friday, July 13, 2007

Dynamically making a field as readOnly using Javascript

I was looking for a sleek solution that would make all the fields of my form read-only dynamically - or atleast make them appear to be readonly.

I decided to use the powerful JQuery library to make this happen. Here is the code that can be put in a JSP/ASP page and included in any page that needs to have all fields as readonly.
$(":input").focus(function() {this.blur();} );
{alert("Cannot change this field.");
return false;} );
{return false;} );
{return false;} );
{return false;} );
{alert("Cannot change this field.");
return false;} );
//-- end of code

ReadOnly and Disabled fields

In HTML, we have two options if we want non-editable fields. Mark the fields as readonly or make them disabled.

The difference between the two is that a diabled field is not send as a HTTP parameter to the server when the form is submitted, whereas a readonly field is send back to the server.
This may be important when you use frameworks such as Struts where the formbeans may expect nested properties to come back to the server.

Thursday, June 21, 2007

Snoop on Solaris and Ethereal

I am a big fan of Ethereal tool and use it frequently for analysis of network traffic.
The GUI is great and we can decode HTTP traffic from the TCP packet frames.

Recently I had to capture the traffic on a remote production box running on Solaris and did not have the time to install Ethereal there. I knew about the 'snoop' command on Solaris and had used it in the past to capture network calls.
But I did not know that the snoop command can write all captured network packets to a file that is RFC 1761- compliant. What that means is that I can write the packets to a file and then open that file anywhere in Ethereal :)

This was just what I needed for my scenario. Quick listing of snoop commands:

- To capture packets to a file and only those on port 9080
snoop -o fileName.cap port 9080

- To format a captured file in ASCII
snoop -i fileName.cap -x0

Time complexity and Space complexity of algorithms

I had just downloaded the SAP Memory Analyser and was impressed with its performance.
More information on this tool can be found here. Going thru their wiki, I read how they had taken pains to make critical operations have a time complexity of O(1).

Time complexity and Space complexity are terms used when dealing with algorithms. If the input size (problem size) for a algorithm increases, then how does it affect the time taken for the algorithm to complete and how how much more memory does it take?

If the time consumed by the algorithm is independant of the input size then the algorithm is said to be a complexity of O(1); i.e. a constant-time method. If the time taken is linear then it is known as linear-time method - O(n).

More information can be found at the following links:

Tuesday, June 05, 2007

Generating GUIDs on the client side web browser

If your application requires that a GUID be created to identify the client, then we have 2 options:

- If we are sure that the end users use only IE, then we can use the ActiveX function:
return new ActiveXObject("Scriptlet.TypeLib").
For other browsers, we can write a JS function as shown here.
Basically the function uses the random numbers and padding to generate a unique number.
Snippet of the JS function:
function generateGuid()
var result, i, j;
result = '';
for(j=0; j<32; j++)
if( j == 8 || j == 12|| j == 16|| j == 20)
result = result + '-';
i = Math.floor(Math.random()*16).
result = result + i;
return result

Friday, June 01, 2007

4 GB memory limitation for 32 bit machines

What is the maximum heap size that U can allocate to a JVM or a .NET runtime?
The answer is dependant on the operating system and hardware. On a 32-bit machine, the CPU can use only 32 bits to refer to a memory pointer. Hence 2^32 = 4 gb.

Out of this, 2gb is reserved for the kernel and 2 gb for the applications. Hence we can allocate only 2gb memory to an application. More info on this can be found here.

For a JVM, if we need to allocate more than 2 gb of heap, then we need to install the 64 bit version of the JDK. Also on a solaris box, start the java process with the -d64 option.

Sunday, May 20, 2007

servletContext.getRealPath() issue

Our application was using getRealPath() method to obtain the base path of the web application on the server. I noticed that the method was not behaving consistently across JEE containers.
On Windows/Tomcat5.5, getRealPath returned the path with a '/' at the end.
But on Solaris/Websphere, getRealPath returned the path without a '/' at the end.

The best way to handle this is to have a utility method that would handle all cases :-
String realPath = 

if ((realPath != null)
&& !realPath.endsWith("/"))
realPath += "/";
Also check if using getRealPath is really required. The Servlet API specifies a temp directory that can be accessed using the following code:
File dir=(File)getServletContext().getAttribute

Tuesday, May 15, 2007

Silently print PDFs on the browser

If your web application needs to print PDF's silently on the browser, then check out this blog.

Of all options given, I found the iText trick the simplest to use. The concept is quite simple:
- Create a PDF on the fly using iText
- Embed Acrobat Javascript into the PDF that would print it automatically to the default printer.
- Open the PDF document in a iFrame that is hidden (size 0*0)

A live example can be found here.

Thursday, May 03, 2007

Modal Dialog submit issue

In our application we were using a modal window pop-up. The user has to enter some fields and submit the form. The strange thing was that whenever I submitted the form, a new window was getting poped up.
I knew there was nothing wrong with the javascript I googled around a bit and found the soultion here.

All that is needed is to add the following line inside the "head" tag of the modal page :
<base target=_self>

Generating PDFs on the fly

If we need to generate PDFs on the fly in a JEE web application , then we have 2 open-source options: using Apache FOP or iText.

The approach used by both the leading projects is different. Apache FOP uses XML/XSL to create a XSL-FO file using XSLT Transformation. This XSL-FO is then rendered into a PDF using the PDF FO processor. This is a pretty neat solution as there is a clear separation of concerns. But the only caveat is that U need to have a pretty good hold over the XSL language, which in my personal opinion is very arcane and difficult to learn. But if you gain mastery over XSL, then there is pretty much nothing U cannot do with FOP :)

iText is a pure Java API that allows developers to create PDFs on the fly. For e.g. the iText API would have methods such as 'addPara','addBlock','addImage' etc. etc. So U end up embedding the presentation logic in Java code...but it may not be a big deal for anyone except design purists. It is also possible to use absolute coordinates while creating your PDF's, but I would advice against if there is a possibility of the look&feel of reports changing often.

iText is hugely popular and has been ported to .NET (C-Sharp port known as iTextSharp and the J-Sharp port known as iText.NET)
The popular JasperReports open-source reporting tool uses iText in the background to make PDF documents. Also iText boasts of superb speed - (Creating a 1000 page PDF takes only a few seconds). Even on the FOP site, they recommend iText as a post-processor tool for merging, encrypting, changing PDF documents.

If someone is looking only for a PDF manipulation tool then they should have a look at pdftk. It is also based on iText.

Tuesday, May 01, 2007

Setting Cache headers for Gifs, Css and JS files

By default, browser and servers use the "if-modified-since" header to check if the file should be downloaded from the server or given from the cache.
Though this default behaviour is good for performance, we can increase the performance by giving a cache expiry period. This would make the browser cache the content for that period - i.e. the browser won't even make "if-modified-since" conditional fetch requests.

The best place to set these headers in a J2EE application is thru a filter.
Suppose you want to cache GIFs, CSS and JPEG for 2 hours, just add this header in the response:
Cache-Control: max-age=120

Setting nocache headers in Struts Applications

In older versions of struts, it was a common practise to extend the ActionServlet and set the no-cache response headers there, so that they are applied to all requests.

In Struts 1.2.x, there is a easier way - without writing a single line of Java code.
Just add the "nocache" attribute to the controller element in struts-config.xml file.

nocache="true" />

Disabling keep-alive connections in Tomcat

I was using TcpTrace utility to dump my HTTP requests/responses. But the problem was that my Server (Tomcat) was using keep-alive connections and bcoz of this, the output in TcpTrace was all muddled up.
Disabling keep-alive connections in Tomcat 5.5 was quite simple:

Just add the maxKeepAliveRequests="1" attribute to the Connector tag in server.xml

CSS caching in IE

A strange problem regarding CSS caching had frustrated me for a whole 2 hours today.

The problem was reported by one of the developers that even if the CSS on the server was changed, IE was still used the cached version. I checked the application and found that there were no Cache headers that were set. This meant that IE should have made a GET "if-modified-since" request everytime for the CSS.

I even dumped the requests using the TcpTrace sniffer. I could see that IE did send a request for the CSS file and also the server responding with the new CSS file. Still no change on the page ?? This really baffled me ...If I hit F5/refresh, then the new CSS was loaded, but if I revisited the page, then still the old CSS was being used..

Then I learned from some site, that IE behaves a bit oddly here : IE usually caches CSS files by browser session (until you exit the browser). With a new session, the browser does a head request to see if the file has been modified, if so it reloads. So if I logged off the application and logged in again, the CSS would be reloaded. Strangely, this would not always happen...But if I close the browser and open the application again, I always get the new CSS..
It seems that IE is a bit erratic in this to be absolutely sure that U get the new CSS, you may have to close the browser and open it again :)

Monday, April 30, 2007

Magic of JQuery

We know that in IE, a form gets submitted whenever we press 'Enter' in a text-field. To disable it would require trapping the event in Javascript and disabling the default behaviour.

<input onkeydown="if (event.keyCode == 13){returnfalse;}" name="email">

But to do this in all text fields across hundreds of html pages would be a impossible task. Then suddenly I got the idea of using JQuery's powerful selector concept to apply this to all textfields across all pages. I just needed to add the following lines to the JS file that was present in all pages.

<script src="jquery-1.1.2.js"></script>
<script type="text/javascript">
$("input[@type=text]").keypress( function() {
if (event.keyCode == 13){return false;}
} );

Friday, April 27, 2007

Mean, Median, Mode, Midrange and Standard Deviation

A definition of terms to help my frail memory when I need it most :)

The arithmetic mean is the sum of the numbers, divided by the quantity of the numbers.

The geometric mean of two numbers is the square root of their product.The geometric mean of three numbers is the cubic root of their product.

The quantity obtained by adding the largest and smallest values and dividing by 2, statisticians call the midrange.

The median is the central point of a data set. To find the median, you would list all data points in ascending order and simply pick the entry in the middle of that list. If there are an even number of observations, the median is not unique, so one often takes the mean of the two middle values.

The standard deviation is the most common measure of statistical dispersion, measuring how widely spread the values in a data set are. If the data points are close to the mean, then the standard deviation is small. Conversely, if many data points are far from the mean, then the standard deviation is large. If all the data values are equal, then the standard deviation is zero.It is defined as the square root of the variance.

For lists, the mode is the most common (frequent) value. A list can have more than one mode. For histograms, a mode is a relative maximum ("bump"). A data set has no mode when all the numbers appear in the data with the same frequency. A data set has multiple modes when two or more values appear with the same frequency.A distribution with two modes is called bimodal. A distribution with three modes is called trimodal.

Wednesday, April 25, 2007

DST implementations in Operating Systems and Java

I often wondered how operating systems knew about Day light saving offsets. Recently USA decided to change its DST settings and it immediately crossed my mind as to how Operating Systems and Applications would react to this.

Some basics first - In what format does a OS store date/time information ? All machines keep their clocks in synch with the GMT using Network Time Protocol. Then the machine calculates it local time using the local time zone settings that are stored on the machine.
Local time is always computed as = GMT + timezone offset + DST offset

But where does the OS get information about DST. All DST information is stored in a binary database called ZoneInfo database or TimeZone DB. On Solaris this database can be found at "/usr/share/lib/zoneinfo"

In Java, the JRE picks up DST information from the Timezone database stored at "%JRE_HOME%/lib/zi"

Tip: In Java, U can change the default timezone that the application uses by setting the user.timezone property
e.g. java -Duser.timezone=Area/City
java -Duser.timezone=GMT[+5.30]

web.xml SAXParseException in Websphere v6.1

I encountered a wierd problem while re-deploying a war file on Websphere v6.1. I started getting a SAXParseException in web.xml. The only change I had made was to introduce the "error-page" tag.

The war file was working fine in Tomcat, Weblogic and Geronimo. Digging into the logs of Websphere, I found this message:

org.xml.sax.SAXParseException: The content of element type "web-app"
must match "(icon?,display-name?,description?,distributable?,
login-config?,security-role*,env-entry*,ejb-ref*,ejb-local-ref*)". [wsjspc]

I wondered if the order of the tags was important to WAS. So I placed the error-page tag between the welcome-file-list and taglib tags..and the SAXException disappeared :)
But I found it rather strange that the order of the tags should be important..not sure if it is part of the servlet spec.

Tuesday, April 24, 2007

Cool tools - Squirrel SQL Client, IzPack Java Installer

Came across 2 tools that I liked :)

Squireel SQL client: Use it to connect to any database using JDBC.

IzPack Java Installer: A simple opensource free installer for Java. But this requires that JDK be present on the installation machine.

Show friendly HTTP error messages in IE

In web.xml, I added the tag to redirect to an error page when ever a 500 internal server error occurs. But in IE, the page was still not getting displayed.

I added a sniffer to check what was going on. I could see that the custom error page was returned by the server in the HTTP response, yet IE was ignoring it. Googling around, I found out that IE has a feature turned on by default - Show friendly HTTP error messages (goto tools-> internet options-> advanced -> uncheck show friendly HTTP error message)

If I unchecked this checkbox, then the custom page was getting displayed correctly. But this was obviously not a solution that was practical. I perused thru more info about this IE feature here. IE uses this feature only if the response content for a 500 status-code response was less than 512 bytes.

So now, I suddenly had 2 solutions to the above problem (without having to change IE settings on the client)
Solution 1: Increase the content of the error page to more than 512 bytes.
Solution 2: Change the HTTP status code sent in the response header of the error page to HTTP 200.
Code snippet (at the top of the error jsp page):

Monday, April 23, 2007

UnsatisfiedLinkError while loading native libraries

I encountered the following problem when calling native code from a Java application deployed on Websphere.
Whenever I was redeploying the WAR file, I started getting UnsatisfiedLinkError - native library already loaded by a classloader.

I knew that the call to "System.loadLibrary" should occur only once in a classloader context. We typically put this code in a static block of a class. But it seemed that in Websphere, when the web application context (classloader) was reloaded, the native libraries were not unloaded. Hence whenever I reloaded the webapp, the above error manifested itself.

To solve the above problem, there were 2 options:
- Put the call to System.loadLibrary inside a static block of a class and load this class as a shared library in Websphere. This will load the class using the System classloader and not the web-application classloader.
- Restart the application server everytime the war/ear file was updated. This solution may or may not be feasible depending on your infrastructure setup.

Imp Caveat: Any crash in the call to the native library crashes the JVM itself. Hence for critical applications, other design alternatives should be thought of. For e.g. writing a separate application that will have a RMI/SOAP/XML interface that will do all the JNI plumbing. This separate application would run in a separate JVM outside of the JEE container JVM. Your webapp would then call this application.

Friday, April 20, 2007

Sequence numbers from a Oracle RAC cluster

Today, I faced a peculiar problem. Our application was using a Sequence to autogenerate unique numbers required for a particular functionality. When testing the application against a standalone Oracle server, the sequence numbers generated were in order.

But in the production evnironment, where we were using a Oracle RAC cluster, the sequence numbers returned were not in order..This was creating a lot of confusion. The reason was that by default sequences in 10g RAC are created without ordering. This is to reduce performance bottleneck as there would be synchronization requried across all nodes of the cluster. Each node creates a cache of the sequence as specified in the SQL.

If we need to maintain the order then we need to add the ORDER argument to the sequence.
Example of Sequence Syntax in Oracle RAC:


Thursday, April 12, 2007

NTLM authentication using Commons HttpClient

Recently I wanted to access a site outside the firewall thru Apache commons HttpClient. The proxy server was using NTLM authentication.

Thankfully, HttpClient had support for NTLM authentication. Here's the code snippet for the same:
HostConfiguration hConf= httpClient.getHostConfiguration();
hConf.setProxy("", 8080);

new AuthScope("", 8080, "domainName"),

new NTCredentials("username", "password","my_localhost","domainName")


Reset method of FormBeans in Struts

We all know that FormBeans in Struts can have either a 'request' scope or a 'session' scope.

In earlier versions of Struts, I recollect that the reset method of the FormBean was called by the Struts framework whenever the scope is request and not called for session scope. This was because, for 'Request' scope, a pool of FormBean objects were created.

But with new versions of JDK (especially since JDK1.4) object creation is no longer a performance penalty. In fact, short-lived objects are quickly garbage collected because they are allocated in the younger generation of the heap.

Looking at the latest Struts source code (v1.9), I saw that if the scope of the FormBean is Request, then a new FormBean object is created for each request.

Also the reset method is called for both request and session scope FormBeans.
But there is an important difference - if the scope is request, then the constructor of the formbean is also called everytime (for each request), where as for session scope the constructor would be called only once.

So, now what to do? The default reset method does nothing. We only need to override it in the following cases:

1. There are checkboxes and form-bean scope is session: In the reset method we need to reset the checkbox fields, because the browser does not submit a checkbox value if it is clear.

2. Nested beans, arrayLists are used and scope is request: In this case, we need to create a arrayList with the correct size.

Wednesday, April 11, 2007

JQuery Javascript library

Came across this cool cool Javascript library that adds that extra jazz to Javascript programming.

Some concepts I liked in this library:
- Separation of presentation markup with behavior (event handling etc.)
- Cool implementation of Chain of Responsibility pattern.
- Good effects such as slow show etc.

Overall, a good library that can be used in web projects.


Friday, March 09, 2007

Private network addresses

Can U look at an IP addresses and guess whether it belongs to a private network or is on the internet? The answer is yes.
The Internet Assigned Numbers Authority (IANA) has reserved thefollowing three blocks of the IP address space for private networks: - - -

Earlier the IP range was divided as ClassA, ClassB , Class C etc. but now we have CIDR (Class less inter-domain routing) which allows us to slice and dice networks the way we want to.

Tuesday, February 27, 2007

Ruminating on dates, DST and days-between calculations

In one of my programs, I needed to calculate the difference between 2 dates in days. Ironically I could not find any method for the same in the core Java API of Calendar and Date.

So, I decided to write a utility method for the same. But then it struck me that I need to take care of DST and leap seconds. Googling around led me to a interesting journey of Atomic clocks, Quartz clocks, leap secs, etc.

Some cool links to get started with the concepts:

Finally the algorithm I used to calculate the days-between 2 dates was:
1. Get the UTC time of both dates (imp if dates are from different locales)
2. Set the time on both dates as 12 noon. (good practice as it avoids DST and leap sec issues)
3. Calculate the milli-sec difference between the dates.
4. Divide the difference with 24*60*60 and take a Math.abs

Thursday, February 22, 2007

Programming to interfaces

We all know that as a good programming practice we should always program against interfaces and not actual subclasses. That's the reason when libraris are designed, they return an interface to an collection and not the actual implementation class.

Recently the developers in my team were facing a problem regarding ClassCast exceptions using Spring JDBC when they upgraded the commons-collection package. After analysis of the source code, it was revealed that the developer was type-casting the List interface to a solid class, i.e. the LinkedHashMap class.

Spring JDBC decides at runtime what Map implementation should be returned. If the latest version of jakarta-commons is available in the classpath, an implementation of is returned, else java.util.LinkedHashMap is used.

Hence the program would start throwing ClassCastExceptions ...See log below comparing what happens depending on the classpath jars:

-- When the latest version of jakarta-commons is not there in the classpath
16-Feb-2007 14:29:47 DEBUG (
org.springframework.core.CollectionFactory -
Falling back to [java.util.LinkedHashMap] for linked case-insensitive map

-- When the latest version of jakarata commons is present
16-Feb-2007 14:31:36 DEBUG ( org.springframework.core.CollectionFactory -
Creating [org.apache.commons.collections.

Moral of the story: Always program to interfaces.

Starting a java daemon on Solaris

We were using Quartz as a scheduler to run our cron-like java programs on Solaris. Quartz gives us much more flexibility than the cron-tab, as we can add listeners and plug-ins.

Normally, when we exit a shell, all the programs that were started in the shell are terminated.
To avoid this, we made use of the 'nohup' command that keeps running the program even if the shell is closed. nohup = no-hang-up.

Also important thing to remember is that if the Std. input is kept open, then the shell does not exit. In the Java program, the main method automatically opens the Std input stream. Hence we need to close it. Here is the complete syntax.

nohup java com.domain.main_class <&- &

The ampersand at the end, starts the program in the background. nohup creates a file called nohup.log that contains the output to the standard stream.

Thursday, February 15, 2007

Daylight saving time

Being brought up in India, we never have to worry about Daylight saving chores. But its very interesting and also important for any system designer to know all about DST.
More info at this link :

DST also comes into play when we schedule jobs using cron or Quartz.
For e.g. In the US, many states implement DST at 2 a.m.
If we have a job scheduled at 2.15 am, then it won't be fired on the day when the cock is shifted ahead from 2.00 to 3.00 am :)

Monday, February 12, 2007

Spring JDBC quirks

Spring JDBC's JdbcTemplate provides us with utility methods that return scalar values. For e.g. methods such as queryForObject, queryForInt, queryForLong etc. These methods are useful when we need to obtain a single row or single column from the database.
But there is a caveat here. Each of these 'scalar' methods throw an IncorrectResultSizeDataAccessException if the there are no results or more than one row in the resultset.

For e.g. if there is a valid business condition that no rows may be returned for a particular query, then we need to handle this exception and return null. The other option is to use the generic query methods that return a list. For empty resultsets we can return an empty list.

Printing Stack Trace in Java

Quite often, developers use the statement exception.printStackTrace() to print the stack trace on to the console. It is important to remember that printStackTrace method prints the stack trace to the System.Err stream and not the System.Out stream.
Due to this, there are a lot of exception stacks getting printed in the System.Err files on the application server, but we do not see any trace of it in the Log4J files or System.out files !!!!

As a best practice, I strongly recommend not using e.printStackTrace() in production code. If the project is using Log4J, then the method 'Logger.error(message, exception)' would print the stack trace to all the registered appenders.

Alternatively, I have written a utility method that would return the stack trace as String. Please use this method if you want to get a stacktrace as a string for more flexibility in printing.


* Gets the exception stack trace as a string.
* @param exception
* @return
public String getStackTraceAsString(Exception exception)
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
pw.print(" [ ");
pw.print(" ] ");
return sw.toString();

But what if there is already production code that uses e.printStackTrace() and System.out and you want to redirect them to the Log Files. In that case, we can make use of the System.setOut() and System.setErr() methods. Also try the LoggingOutputStream class in the contribs/JimMoore folder of the log4j release.

Monday, February 05, 2007

Redirecting system.out, system.err to a file on Unix/Solaris boxes

It is simple to redirect a process output to a standard file using the '>' operator.
But if you also want the std error to go to a file, then you need to type:

java className>file 2>&1

To understand what the above command means, it is imp to note that the shell reads arguments from left to right. The shell sees >file first and redirects stdout to file. Next 2>&1 sends fd2 (stderr) to the same place fd1 is going - that's to the file.

Epoch - The start of time for computing

Recently working on a XML data feed program, I had to handle empty date XML elements. I wondered as to what date I can put into the database - the start of time?. What is the start of time used by computer systems?

As Java programmers we all know that the Date, Calendar classes in Java use January 1, 1970 as the start of 'time'. This is known as Epoch. Wikipedia defines epoch as follows:
"The epoch serves as a reference point from which time is measured. Days, hours and other time units are counted from the epoch, so that the date and time of events can be specified. Events that took place earlier can be dated by counting negatively from the epoch."

There are different epocs used by different systems:
- Unix and Java use Jan 1, 1970
- MS DOS uses Jan 1, 1980
- Excel and Lotus = Dec 31, 1899

More info can be found here.
As for me, I decided to use the Java Epoc for my program :)

Tuesday, January 30, 2007

Using ASCII mode for FTP of text files

Last week, I ran across a common problem -- I had transferred a INI file from Windows to Solaris in binary mode and the file did not work on Solaris. This was obviously due to the difference in newline characters.

I then realized that if we transfer in ASCII mode, then the FTP program automatically converts between windows and unix line feed characters :)
More info at these links:

Tuesday, January 16, 2007

Solaris tips - find out RAM and HD size

A few quick commands to help developers new on Solaris 10:

To find out the amount of RAM on the system.
> prtconf / grep Mem

To find out the Hard Disk capacity
> iostat -E / grep Size

To find out the file-system size
>df -h

To find out the CPU usage
> mpstat 12 5
> ps -e -o pcpu -o pid -o user -o args
> top

The following links give more info on common commands:

Monday, January 08, 2007

Getting the Client IP address on a J2EE server

The HTTPRequest object has 2 methods 'getRemoteAddr()' and 'getRemoteHost()'. These methods would return us the IPAddress/HostName of the last proxy or the client. So if the client is behind a proxy, then we would get the IPAddress of the proxy.
Some proxies (known as transparent proxies) pass the actual IP address of the client in the header HTTP_X_FORWARDED_FOR.
So on the server side we can code something like this:

if (request.getHeader("HTTP_X_FORWARDED_FOR") == null) {
String ipaddress = request.getRemoteAddr();
} else {
String ipaddress = request.getHeader("HTTP_X_FORWARDED_FOR");

But if the proxy is an anonymous proxy, then even this won't work. So the only way to get the Client Address correctly is using an Applet to capture the IP address of the client. For this, the client should be trusted and signed.
Another option that I came across in a forum is to create a Socket connection back to the web server from which you came and asking the Socket for the local address:

URL url = getDocumentBase();
String host = url.getHost();
Socket socket = new Socket(host, 80);
InetAddress addr = socket.getLocalAddress();
String hostAddr = addr.getHostAddress();
System.out.println("Addr: " + hostAddr);

Friday, January 05, 2007

Secure FTP options

I was looking for an secure option to FTP files to a remote server.
There are 2 options available: FTPS and SFTP....and I was a bit confused in understanding the differences between the two.

SFTP (Secure FTP) is FTP tunneled thru SSH.

SFTP provides an interactive interface that is similar to that of FTP. In addition, the command line version of SFTP is scriptable in that it allows you to specify a batch file to control the file transfer process.

If U are using VPN, then another option would be to constrain FTP access to circuits created over VPN connections, then as long as the VPN is secure end-to-end, FTP will also be acceptably secure.

File Encryption utility for windows

I was looking for a small light-weight encryption utility for encrpyting my personal information in files. There are a plethora of programs available on the internet, but I found the following ones cool and easy to use.

Kryptel Lite:
Iron Key:

Thursday, January 04, 2007

FTPing in Java

For performing FTP operations programmatically, I use the Jakarta Commons Net Package. This is a super-cool utility package that makes writting FTP programs a breeze.
The jar file can be downloaded from here.

Tip: The jakarta-commons-net package also requires the ORO package which can be downloaded here.

Another important concept is that of Active and Passive FTP. A good explanation of the differences can be found here.
As a general rule, passive FTP is used as it works behind firewalls too.

The windows FTP can also be run unattended using the '-s' option and specifies a script file that contains the FTP commands.

Tuesday, January 02, 2007

URL encoding and XML/HTML encoding

Recently in one of our applications we were passing XML data as a POST parameter to the server. For e.g. xml={xml string here}

We were using XML encoding to replace special characters in the XML. For e.g. '&' was replaced with '&amp;' '<' was replaced with & lt; etc.

But on the server side whenever we were trying to get the XML string as a request parameter, we were getting a truncated string. After putting in a sniffer, I found out the reason why this was happening - I was getting confused between URL encoding and XML encoding.

In URL encoding as we know, we need to encode special characters such as '&'. So we replace them with '&amp'. But what if '&' was part of the data; i.e. an address contained &. In this case the URL encoding function should just replace '&' with '%26'. Otherwise the server would treat it as a parameter separator and break the string on the first occurance of '&'.

Now our XML string (passed as a POST parameter) contained the & character, because all XML special characters were encoded with a '&' at the beginning. So the trick in this case was to perform URL encoding of the element values after the XML encoding. Confused??

Ok...consider the following XML element:
Now the above element data contains an & and hence would result in a XML parsing error. Hence I XML encode the data. So it would not look like:

But when this string is passed as a HTTP GET/POST param, then it would be truncated.
Hence we have to URL encode it - replace & with %26

So when the string reaches the server, the request.getParameter would perform the URL decoding and then we passed the XML string to Castor or any other XML parser.

Another good design to avoid all this confusion is to pass the XML in the request body stream and get the XML on the server side using request.getInputStream().

Even ASP.NET has 2 methods: HtmlEncode and UrlEncode.
HtmlEncode converts the angle brackets, quotes, ampersands, etc. to the entity values to prevent the HTML parser from confusing it with markup.
UrlEncode converts spaces to "+" and non-alphanumeric to their hex-encoded values. Again this is to prevent the the URL parser from misinterpreting an embedded ?, & or other values.