rm craziness

I was trying to delete some files on Solaris using the rm command.
The file names were something like: "-HqFbc3YQ0wQYhH7u16veP7_BBC_6_090540.xml"
These files were generated by a program using random numbers.

Now no matter what I tried, rm always returned with an error for files starting with a - (minus sign)
I found the solution here.

Snippet:
To remove a file whose name begins with a dash ( - ) character, refer to the file with the following syntax:
rm ./-filename
Using the redundant ./ directory information prevents the dash from occurring at the beginning of the filename, and being interpreted as an option of the rm command.

What does a Digital certificate contain?

If you view a digital certificate using FireFox browser or decoding it using some tool such as OpenSSL/IKeyMan; then we can see the following parts/data:-
- Issued by {The CA name}
- Issued to/Subject {CN of the subject (typically the DNS of the server)}
- Valid From {date}
- Valid Till {date}
- Public Key {typically the 1024 bit public key of the subject}
- Public Key Algorithm {The encryption algorithm to be used with the key, e.g. RSA}
- Signature Algorithm {The algorithm used to generate the digital signature, e.g. MD5HashWithRSA, SHA1HashWithRSA}
- Signature {The digital signature}

The digital signature is created by hashing the public key of the subject using MD5 or SHA1 and then encrypting the hash using the private key of a CA.

An entity verifying this digital cert would:
- decrypt the hash using the public key of the CA
- create a new hash of the public key of the subject
- compare both the hash keys and verify if the subject can be trusted.

Formats for Digital certificates

I knew that all Digital certs confirm to the X.509 standard. But why were they so many extensions for certificate files? - e.g. DER, CER, .P7B, PFX etc. I found this link that contains good info about X.509 certs.
Basically a X.509 cert can be in many formats (depending on the encoding used to save the cert)

.CER - CER encoded certificate, sometimes sequence of certificates
.DER - DER encoded certificate
.PEM - (Privacy Enhanced Mail) Base64 encoded DER certificate, enclosed between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----"
.P7B - See .p7c
.P7C - PKCS#7 SignedData structure without data, just certificate(s) or CRL(s)
.PFX - See .p12
.P12 - PKCS#12, may contain certificate(s) (public) and private keys (password protected)

A .P7B file could be a container for more than one digital certificate.

Adding trusted root certificates in Websphere 6.1

We had deployed an application to WAS 6.1 ND. This application contained a JAX-WS 2.0 webservices client that used to call a third-party service using SSL. The digital certificate used by the third-party was self-signed and hence we needed to accept it as a trusted party in our Trust Store.

We imported the certs into a trust-store (JKS format) using the keytool command of the JDK and wrote the following code:

System.setProperty
("javax.net.ssl.trustStore", trustFilename );
System.setProperty
("javax.net.ssl.trustStorePassword", "changeit") ;
System.setProperty
("javax.net.ssl.keyStore", trustFilename );
System.setProperty
("javax.net.ssl.keyStorePassword", "changeit");

But unfortunately this was not working on WAS6.1. (This works fine on Tomcat).
In earlier versions of WAS, the iKeyman tool provided an interface to manipulate digital certs, keys, Trust stores and Key stores. But in WAS 6.1, all these tasks can be done from the web-based admin console.

So to add the certs to the Trust store, I went to "Security (on the left panel) --> SSL certificate and key management > Key stores and certificates > NodeDefaultTrustStore > Signer certificates"
Add the new root digital certs that need to be trusted to this store. The JAX-WS client should now be able to connect to the HTTPS resource. Remove any system properties that have been set before.

A good article (for WAS 6.0) describing how SSL and Digital certs work in Websphere can be found here.

Hack to deploy JAX-WS 2.0 webservice on Websphere 6.1

Our team had developed a webservice using JAX-WS 2.0 and using the Reference implementation of Sun as the JAX-WS provider. The application was developed using Netbeans and the embedded Tomcat server.

But when the application was deployed on WAS 6.1, then it failed and resulted in ClassCast exceptions. The base installation of WAS 6.1 only supports JAX-RPC 1.1.
IBM did provide an option - installing the webservices feature pack for WAS 6.1
The installation was a bit of a pain, especially on a cluster. You essentially had to create new AppServer profiles, create a new cluster and move all applications to the new cluster. This would also required considerable downtime on a live production server.

I looked for an other option. I tried to understand what was happening under the hoods when Netbeans was creating a webservice. It was clear that the webservice created was using a Servlet provided by the RI of JAX-WS 2.0. Even all the annotations and JAXB libraries were included in the lib directory. Hence there was no dependancy on any server-specific library as such.

I realised that the ClassCast exceptions on WAS 6.1 were happening due to the ClassLoader heirarchy - which was set to 'Parent First'. I decided to try to change this configuration to 'Child First' - so that JAX-WS RI libraries are picked up from WEB-INF/lib first.

There were 2 changes that I made using the web admin console. Go to "Enterprise Applications --> {Application name} --> Class Loader" and changed the following items:
Select "Classes loaded with application class loader first" and "Single class loader for application". Save these changes to the master configuration and restart the AppServer.

This hack worked and we could deploy JAX-WS 2.0 webservices on WAS 6.1 :)

Tip: We ran into a few Linkage errors, which we could resolve by removing all duplicate jar files.
A linkage error would typically occur when a class tries to call another class loaded by a different class loader.

Is POX/HTTP same as REST?

There is a lot of confusion in the industry over REST webservices. A lot of applications offer POX/HTTP interfaces and call them REST webservices. But in theory, just having XML tunneled over HTTP does not satisfy all REST principles.

It's important to understand that REST is not a protocol. It is an architecture style that can be used to design systems. It brings in a noun-oriented approach to access resources (instead of a verbs).

The Apache CFX framework has new features (using the magic of annotations) that make it possible to have true REST style webservices.
More information on this can be found here and here.

Recently came across this article that states the REST principles that should be followed during design:

1. Give every resource an ID. Use URIs to identify everything that merits being identifiable, specifically, all of the "high-level" resources that your application provides, whether they represent individual items, collections of items, virtual and physical objects, or computation results. Examples:

http://example.com/customers/1234

http://example.com/orders/2007/10/776654

http://example.com/products/4554

2. Link things together. (Hypermedia as the engine of application state)

Example: Element in XML doc: product ref='http://example.com/products/4554'

We could use a simple 'id' attribute adhering to some application-specific naming scheme, too but only within the application’s context. The beauty of the link approach using URIs is that the links can point to resources that are provided by a different application, a different server, or even a different company on another continent because the naming scheme is a global standard, all of the resources that make up the Web can be linked to each other.

3. Use standard methods. For clients to be able to interact with your resources, they should implement the default application protocol (HTTP) correctly, i.e. make use of the standard methods GET, PUT, POST, DELETE

e.g. GET - get order details/list all orders

PUT - update order

POST - add order

DELETE - delete order

4. Communicate statelessly - This is for scalability.

Message style and encoding in WSDL documents

The WS-I specs state the following 2 types of encoding to be complaint to the Basic profile:
- RPC/literal
- Wrapped document/literal

Found this cool article on developerworks that explain all the styles and encoding in detail.

WSDL cheat sheet

With the plethora of tools available today for webservices development, we sometimes forget the basic fundamental semantics of a WSDL document. Here is a quick reference of what a WSDL contains:
1. Types: These are the data type definitions - expressed as a schema of complex/simple types.
2. Message: A message consists of a logical order of one or more types.
3. Operation: Defines a operation that consists of a input and output message.
4. Porttype: A collection of operations.
5. Binding: Specifies the concrete protocol data format specifications (e.g. document-literal) for a portType.
6. Port: Specifies a network addresss for a binding, thus defining a single communication endpoint.
7. Service: collection of endpoints or ports

Thus a service is defined by one or more endpoints/ports. A port is combination of a binding with a network address. Each binding is a combination of a porttype and concrete data format specifications. Each porttype is a collection of operations that define messages that are exchanged.

More information can be found here.

Xerces and Xalan packages in JDK 1.5

Since JDK 1.5, Sun has decided to change the package names of the Xalan and Xerces to "com.sun.org.apache.xalan" and "com.sun.org.apache.xerces". Old package names (org.apache.xalan.xslt etc) are now not shipped with JDK1.5

I think this is a welcome move, bcoz this will enable end-users to use new versions of Xerces and Xalan easily without resorting to class-loader manipulation. Earlier this was done using the endorsing mechanism for JAR files.

HTML menu DIV tag over a SELECT element

We are using a JS menu that used layers created by div tags. The problem was that this layer was hiding behind behind 'select' dropdowns in the page.

I googled around for a solution and found out 2 solutions:
- Hide the select boxes when the menu is clicked and unhide them when the menu rolls back.
- Use the zIndex property and create a transparent IFrame under the menu layer.

The following links were of great help to me:
BEA portal menu.
IFrame option.
Hiding option.

Heap size and Perm size

I was getting confused if the heap size specified by -Xmx included the Perm size or it did not? The GC document on Sun's site showed the Perm generation to be part of the Heap size, but I think this is wrong. It seems that the Perm size is different from the Heap size. I found this link that confirms this belief.

The "permanent" generation in the Sun JVM is for class objects, method objects, static fields for those classes (just the static fields, not the objects referenced from the static fields: those are in the regular Java object heap because Java code allocates them), and random other bits like any interned Strings etc. So if U are using a lot of reflection in you code, then a lot of method objects and class objects are created at runtime. For e.g. XML binding frameworks such as Castor etc. use reflection by default.

Hence the total heap size used by the JVM is the Permanent Generation size + the -Xms/Xmx size.

The total memory used by the JVM process will be = Memory used by the JVM for internal management + memory allocated in native code + memory allocated for the permanent space.
This is the memory that U see in Task-Manager in Windows or 'top' on Solaris.

What happens when we allocate more Java heap than the max physical memory available?
The OS would start swapping the process to disk (depending on the Virtual memory configuration) if the total memory reserved by all applications and the OS in the system exceeds the physical RAM installed. This would seriously degrade the performance of the system.

*IMP*: If your application is throwing an OutOfMemoryError then it could be because the PermSize is full, rather than the heap size.

HttpClient in a multi-threaded environment

Our application was using HttpClient library to connect to various third-party systems. I was proposing to develop a smart resuable component that could be used by other modules and also suffice the performance requirements of the application.

I was joyed to see how the design of HttpClient offers excellent support for multithreaded environments.
The HttpClient and ConnectionManager classes can be shared across the entire application. Each respective thread of execution must have a local instance of HttpMethod and can have a local instance of HttpState or/and HostConfiguration to represent a specific host configuration and conversational state.
Hence these must be local variables in all the methods. At the same time the HttpClient instance and connection manager are shared among all threads for maximum efficiency.

If an proxy configuration is required, then this information can be passed in the execute() method of PostMethod using the HostConfiguration class.

Finding the version of Solaris

Commands to find out the version of Solaris the box is running on:

uname -X
more /etc/release

Viewing large files on Linux/Solaris

Log files of J2EE applications can become very big. In our application, the log files were of the size ~ 700 MB to 1GB.
To search such big files for information the following unix commands can be used:
- less fileName.txt
To search a string in the forward direction
- /searchString
To move to the next search
- n
To search backwards
- ?searchString

Javascript goodies

Today, I spend quite some time refactoring messy Javascript code:

1.) I needed to extract a number appended to the end of the string - use the power of regular expressions to make this one clean and sleek.

var RowIdNumber = rowId.match(/\d+$/)[0];

2.) To obtain a 'anchor' element inside of a table row and invoke it's click (href) -

rows[i].getElementsByTagName("a")[0].click();

Enjoy :)

Java Applet Caching Issues

I spent the last 3 days in frustration trying to get Java Plug-in cache mechanisms to work.
Since JDK 1.5, the Java Plug-in has advanced caching options using the "cache_archive" and
"cache_version" tags.

One particular vexing issue that I faced was that inspite of adding all the cache directives in the applet's object tag, the jar files were not cached. A quick google showed 2 bugs:
1. Unfortunately, if the servers do not return Last-Modified or Expires in HTTP response headers, it would disable plugin caching for the connection, and plugin would try to redownload the file again without caching. This results in two HTTP GET requests per file.
2. If a running applet makes a call to URLConnection.setDefaultUseCaches(false), then all subsequent loads from this plugin will exhibit the multiple-load-request behaviour.

In our case, the Applet was setting the cache as false in the code. Hence the cache directives were not working as expected. More information on this can be found here and here.

Search functionality in Textpad and EditPlus

A lot of people use text editors such as Textpad and EditPlus. My favourite is Textpad; it can handle large files and also has a very good interface.

Many times we need to search for muliple words in a line. For e.g. in a log file, search for all lines having words 'abc' and 'xyz' on a single line. Using the regular expression search facility available in these editors, this task becomes a piece of cake.

RegEx for the above task: abc.*xyz
We can keep on adding more words - abc.*xyz.*123

A simple feature, but very powerful while analysing logs etc.

Favicon is a bit messy

Recently when I was analysing the log files on the server, I came across many reqeuests for a resource - favicon.ico
This was strange, since I knew that there was no such file on the server - so why were browsers making a request for this file.

I did a quick google and came to know about this interesting things:
- Favicon stands for "Favourites Icon". Its the small image U see in front of the URL on the addressbar.
- The icon looks good, but the problem is that some browsers keep on making a request to this file everytime the page is loaded !

Here are some links having more info:
http://www.favicon.co.uk/
http://www.htmlkit.com/services/favicon/

Cool collection of JS code

Found this link that contains a cool collection of Javascript code. Worth a perusal.

I liked the fee menu javascript code available here:

http://javascriptkit.com/script/cutindex23.shtml

Interpreting 2 digit years

As a good practice, it is always recommended to use 4 digits to specify the year. But a lot of applications may have a front-end that accepts 2 digit year as the input from the user. Or there could be a intergration with a third-party that requires us to parse 2 digit years.

We all understand the ambiguity in parsing 2 digit years. The logic that you would need to use depends on the business requirement. For e.g. if you are accepting birth-year then it cannot be greater than today and we can go 100 yrs back.

So how do different API's handle it?

The SimpleDateFormat class follows the following logic:
"For parsing with the abbreviated year pattern ("y" or "yy"), SimpleDateFormat must interpret the abbreviated year relative to some century. It does this by adjusting dates to be within 80 years before and 20 years after the time the SimpleDateFormat instance is created."

The Joda Time API takes a different approach:
"What is particularly interesting about this format is the two digit year. Since the interpretation of a two digit year is ambiguous, the appendTwoDigitYear takes an extra parameter that defines the 100 year range of the two digits, by specifying the mid point of the range. In this example the range will be (1956 - 50) = 1906, to (1956 + 49) = 2005. Thus 04 will be 2004 but 07 will be 1907. This kind of conversion is not possible with ordinary format strings, highlighting the power of the Joda time formatting architecture"

The default setting in Windows 2000 m/cs:
"Under Windows 98 or Windows 2000, two-digit years for the Year argument are interpreted based on user-defined computer settings. The default settings are that values from 0 through 29 are interpreted as the years 2000–2029, and values from 30 through 99 are interpreted as the years 1930–1999. For all other Year arguments, use a four-digit year; for example, 1924."

The Internet mail standard uses the following:
"Two digit years are treated as valid in the loose translation and are translated up to a 19xx or 20xx figure. By default, following the specification of RFC2822, if the year is greater than '49', it's treated as being in the 20th century (19xx). If lower, or equal, then the 21st (20xx). That is, 50 becomes 1950 while 49 is 2049."

JS library for dates

I was looking for a reusable JS function that would allow me to format and compare dates just the way it is done in Java using classes such as DateFormat, Date etc.

I found this JS file on the web - simple and easy to use. And quite powerful too:
http://www.javascripttoolbox.com/lib/date/

Difference between empty string and null in Oracle Database

In Oracle, when we store a empty string i.e. "" through JDBC, then Oracle stores it as a NULL value. This causes a lot of confusion to Java developers. In Java a null and a "" are 2 different entities.

Hence even if you store an "" string when inserting into the database, when you retrieve the results, you would get the string as null.

If we do a getInt(), getFloat() etc. then we would get 0.

Dynamically making a field as readOnly using Javascript

I was looking for a sleek solution that would make all the fields of my form read-only dynamically - or atleast make them appear to be readonly.

I decided to use the powerful JQuery library to make this happen. Here is the code that can be put in a JSP/ASP page and included in any page that needs to have all fields as readonly.
$(document).ready(function(){
$(":input,checkbox,radio").addClass("readonlytextbox");
$(":input").focus(function() {this.blur();} );
$("input[@type=checkbox]").click(function()
{alert("Cannot change this field.");
return false;} );
$("input[@type=checkbox]").keydown(function()
{return false;} );
$("input[@type=radio]").click(function()
{return false;} );
$("input[@type=radio]").keydown(function()
{return false;} );
$("select").focus(function()
{alert("Cannot change this field.");
return false;} );
//-- end of code
});

ReadOnly and Disabled fields

In HTML, we have two options if we want non-editable fields. Mark the fields as readonly or make them disabled.

The difference between the two is that a diabled field is not send as a HTTP parameter to the server when the form is submitted, whereas a readonly field is send back to the server.
This may be important when you use frameworks such as Struts where the formbeans may expect nested properties to come back to the server.

Snoop on Solaris and Ethereal

I am a big fan of Ethereal tool and use it frequently for analysis of network traffic.
The GUI is great and we can decode HTTP traffic from the TCP packet frames.

Recently I had to capture the traffic on a remote production box running on Solaris and did not have the time to install Ethereal there. I knew about the 'snoop' command on Solaris and had used it in the past to capture network calls.
But I did not know that the snoop command can write all captured network packets to a file that is RFC 1761- compliant. What that means is that I can write the packets to a file and then open that file anywhere in Ethereal :)

This was just what I needed for my scenario. Quick listing of snoop commands:

- To capture packets to a file and only those on port 9080
snoop -o fileName.cap port 9080

- To format a captured file in ASCII
snoop -i fileName.cap -x0

Time complexity and Space complexity of algorithms

I had just downloaded the SAP Memory Analyser and was impressed with its performance.
More information on this tool can be found here. Going thru their wiki, I read how they had taken pains to make critical operations have a time complexity of O(1).

Time complexity and Space complexity are terms used when dealing with algorithms. If the input size (problem size) for a algorithm increases, then how does it affect the time taken for the algorithm to complete and how how much more memory does it take?

If the time consumed by the algorithm is independant of the input size then the algorithm is said to be a complexity of O(1); i.e. a constant-time method. If the time taken is linear then it is known as linear-time method - O(n).

More information can be found at the following links:
http://pages.cs.wisc.edu/~hasti/cs367-common/
notes/COMPLEXITY.html
http://www.leda-tutorial.org/en/official/ch02s02s03.html

Generating GUIDs on the client side web browser

If your application requires that a GUID be created to identify the client, then we have 2 options:

- If we are sure that the end users use only IE, then we can use the ActiveX function:

return new ActiveXObject("Scriptlet.TypeLib").
GUID.substr(1,36);

For other browsers, we can write a JS function as shown here.
Basically the function uses the random numbers and padding to generate a unique number.
Snippet of the JS function:

function generateGuid()
{
var result, i, j;
result = '';
for(j=0; j<32; j++)
{
if( j == 8 || j == 12|| j == 16|| j == 20)
result = result + '-';
i = Math.floor(Math.random()*16).
toString(16).toUpperCase();
result = result + i;
}
return result
}

4 GB memory limitation for 32 bit machines

What is the maximum heap size that U can allocate to a JVM or a .NET runtime?
The answer is dependant on the operating system and hardware. On a 32-bit machine, the CPU can use only 32 bits to refer to a memory pointer. Hence 2^32 = 4 gb.

Out of this, 2gb is reserved for the kernel and 2 gb for the applications. Hence we can allocate only 2gb memory to an application. More info on this can be found here.

For a JVM, if we need to allocate more than 2 gb of heap, then we need to install the 64 bit version of the JDK. Also on a solaris box, start the java process with the -d64 option.

servletContext.getRealPath() issue

Our application was using getRealPath() method to obtain the base path of the web application on the server. I noticed that the method was not behaving consistently across JEE containers.
On Windows/Tomcat5.5, getRealPath returned the path with a '/' at the end.
But on Solaris/Websphere, getRealPath returned the path without a '/' at the end.

The best way to handle this is to have a utility method that would handle all cases :-

String realPath = 
getServletContext.getRealPath(path);

if ((realPath != null) 
&& !realPath.endsWith("/"))
     realPath += "/";

Also check if using getRealPath is really required. The Servlet API specifies a temp directory that can be accessed using the following code:

File dir=(File)getServletContext().getAttribute
("javax.servlet.context.tempdir");

Silently print PDFs on the browser

If your web application needs to print PDF's silently on the browser, then check out this blog.

Of all options given, I found the iText trick the simplest to use. The concept is quite simple:
- Create a PDF on the fly using iText
- Embed Acrobat Javascript into the PDF that would print it automatically to the default printer.
- Open the PDF document in a iFrame that is hidden (size 0*0)

A live example can be found here.