Difference between CPU time and elapsed time

In Performance metrics jargon, we often come across two measurements: CPU time and total elapsed time. So whats the difference between the two?

CPU time is the time for which the CPU was busy executing the task. It does not take into account the time spent in waiting for I/O (disk IO or network IO). Since I/O operations, such as reading files from disk, are performed by the OS, these operations may involve noticeable amount of time in waiting for I/O subsystems to complete their operations. This waiting time will be included in the elapsed time, but not CPU time. Hence CPU time is usually less than the elapsed time.

But in certain cases, the CPU time may be more than the elapsed time !
When multiple threads are used on a multi-processor system or a multi-core system, more than one CPU may be used to complete a task. In this case, the CPU time may be more than the elapsed time.

Difference between different editions of VS 2008

I was looking for a official comparision between the different VS2008 editions and thankfully I found it here.

The express editions are different downloads for each language. i.e. for C#, VB.NET
So if you want to open different projects in different languages, then go for standard or professional. Then the express editions do not have visual designers for WPF and WWF.

Some of the differences between the standard edition and professional edition are that you can use the professional edition to build MS Office applications. Also the professional edition has support for Crystal reports, Unit testing etc.

More information can be found at the following links:
http://msdn2.microsoft.com/hi-in/vs2008/products/default(en-us).aspx
http://msdn2.microsoft.com/hi-in/vs2008/products/bb894671(en-us).aspx

ETag response header

HTTP 1.1 introduced a new kind of cache validator i.e. ETag.

ETags are unique identifiers that are generated by the server and changed every time the resource is updated. An ETag is a string that uniquely identifies a specific version of a component. The only format constraints are that the string be quoted.

ETag: "10c24bc-4ab-457e1c1f"

The problem with ETags is that they typically are constructed using attributes that make them unique to a specific server hosting a site. So in a typical clustered environment, if the next request for a cached resource goes to a different server, then the ETag won't match and the resource would be downloaded again.

Most modern Web servers will generate both ETag and Last-Modified validators for static content automatically; you won't have to do anything.

More info about ETag can be found at the following links:
http://developer.yahoo.com/performance/rules.html#etags
http://nerdmonkey.com/blog/archives/000098.html
http://www.web-caching.com/mnot_tutorial/how.html

The advantage of ETag over the 'last-modified-date' tag is that the fact that ETags don't require a date but will take any string. So U can compose ETag with any custom logic and source.

Parallel download in browsers

We all know that browsers make multiple requests to servers for each and every resource that is contained in the page. But what is the max number of persistent connections that the browser can make to download all elements of a page?

Well to my suprise, the W3C specification on HTTP 1.1 actually states the same.
"A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. "

What this essentially means is that broswers can download no more than two components in parallel per hostname. U always discover new things :)

YSlow for Firebug - cool cool extension

I have been a die-hard fan of the 'Firebug' add-on to Firefox. Firebug has proved to be invaluable to me for diagnosing front-end performance issues.
Now Yahoo has extended Firebug with another feature called 'YSlow' that shows good aggregate data and also recommends changes to be made in the page.

More information can be found at http://developer.yahoo.com/yslow/

Difference beween WCM and ECM

Doing some research on Content Management systems, I got a bit confused about the difference between Web content management systems and Enterprise Content Management systems.

This article is a good read to understand and appreciate the differences.
The lines between all content technology families are very blurry and hence the auhor suggests that we evaluate a product based on the scenario requirement.

For publishing company websites, intranet sites, a WCM may be a good solution. A ECM solution may be focussed towards a particular industry.

Difference between a portal and a website

A lot of my friends ask me the difference between portals and web-sites. Typically an organization would have both - a company website and a portal. I found a good explanation here.

Portals typically have authentication and provides us information based on who we are. Personalisation and customization are the hallmarks of portals. So we require portals because:
Different roles require different information. Someone from grounds and buildings needs different info than the chair of computer science. (Customization) Different people with the same role work differently. (Personalization) Efficiency - people get directly to the info they need. (Work Flow) Customization insures they don't miss anything.

ThreadPool in Java

Recently one of my friends was looking for a good Thread-pool implementation in Java and asked me for help. I had used Doug Lea's ThreadPool in the past and recommended it to him.

But browsing thru the web, I learned that JDK 1.5 has introduced a java.uti.concurrent.* package that contained a good ThreadPool implementation. Using it was also a breeze. Here are a few code snippets.

ExecutorService pool;
pool = Executors. newFixedThreadPool(3);
pool.execute(new Task( Task(“three”,3));
pool.execute(new Task( Task(“two” two”,2)); ,2));
pool.execute(new Task( Task(“five five”,5)); ,5));
pool.execute(new Task( Task(“six”,6);
pool.execute(new Task( Task(“one”,1);
pool.shutdown();

The Task object is a Thread that implements the Runnable interface and has the run() method. The concurrent package also contains a lot of helper classes that can be used - for e.g. using a thread-safe collection, atomic varibles etc.

Websphere SOA Stack

A lot of people get confused between the different product offering by IBM in the Websphere space.The major products in the Websphere family are Websphere Application Server, Websphere ESB, Websphere Process Server and Websphere Message Broker.

Process Server is based on ESB and ESB is based on Application Server. Message Broker is termed as "Advanced ESB" by IBM and should be used when we have extensive heterogeneous infrastructures, including both standard and non-standards-based applications, protocols, and data formats.

So when to use what? Use Websphere ESB whenever most of the services you want to integrate are based on SOAP, XML standards. The Websphere ESB product can also be used as Websphere Application Server as it is built on the same core. Though Websphere Application Server can host simple services, Websphere ESB gives out of the box support for sophisticated mediation flows found in ESBs. We would need Websphere Process Server whenever we need to orchestrate simple services to create business services and host them.

Another technical difference is that Websphere ESB uses the JMS provider in WAS for messaging, whereas Message Broker uses Websphere MQ as the messaging engine. This is due to legacy reasons, I believe.

More information can be found here.The FAQ available at IBMs site also contains some good info.

Oracle Fast Connection Failover for a RAC cluster

Problem:Whenever one node of a Oracle RAC cluster goes down, some of the connections allocated to that node go stale in the connection pool. This causes errors such as 'Broken pipe' to be raised whenever the stale connections are used.

Solution:Configure a ONS (Oracle Notification Service) remote subscription client on the App Servers. Whenever any RAC node goes down, a 'DOWN' event would be raised and sent to all subscribers. The ONS client on the Appservers would receive this event and refresh all stale connections in the pool. When the node comes up, a 'UP' event is sent to the AppServer ONS client and again the connections in the pool are balanced against both the nodes.

In Oracle 10.2g, to configure ONS, U just have to add ons.jar file to WEB-INF/lib and 2 lines of code to configure the Connection Cache.

3 easy steps to self-sign a Applet Jar file

Found this link that explains how to self-sign an applet in 3 easy steps:

1. keytool -genkey -keystore myKeyStore -alias me

2. keytool -selfcert -keystore myKeyStore -alias me

3. jarsigner -keystore myKeyStore jarfile.jar me

Thats it..simple and sweet :)

FCKeditor

Came across this beautiful AJAX control that can be used to emulate a HTML editor text box.

http://www.fckeditor.net/ - worth a look.

Setting focus on the first field

Whenever a HTML page is loaded, we often want to set the focus on the first input field of that page. Came across this Javascript code that can be put in a JS file and reused everywhere.
http://www.codeproject.com/jscript/FocusFirstInput.asp Simple and Sweet solution :)

Reflection in Java-XML binding frameworks

Almost all Java-XML binding frameworks use Java reflection to map Java objects to XML documents. I had used Castor and XMLBeans in our projects and were satisfied with the performance of both of them.

Recently a friend of mine told me that he had reservations on using Castor as it uses Reflection and hence cannot be used in high-performance applications. I contemplated on this and felt that some basic amount of reflection code would be present in any XML-binding framework; or how would the API call 'set' methods on Java objects during unmarshalling?

I found this discussion on the net that explains this same fact.

Snippet:
Castor will still use some amount of reflection (mainly to set values onto your domain objects) even if you are using a mapping file (a binding file is used during code generation only).

Having said that, let me add one thought. At the time most of these articles have been written (2001 to 2002), the use of reflection introduced some performance penalties, noticeable to some degree.

Since then, a lot of time and effort has gone into improving the JVMs/JDK/compilers to lessen this impact further and further. In my personal view, with the event of JDK 1.4 (and even more so with Java 5)
this problems have been addressed sufficiently in that I would not call the use of (moderate) reflection an issue any more. If use of reflection would still be an issue, frameworks like Spring and some of their most
powerful features (run-time AOP through Spring interceptors) would not have been adopted by the market as it happened in the last one or two years.

Btw, in case this is not clear enough, using a mapping file instead of Castor's 'introspector' to establish mappings between the Java domain classes and the XML artefacts does not really improve Castor's
performance, as both modes will cause Castor to use reflection to establish what methods to use on your Java objects to set a value, to get a value (from an instance variable) or to add a value to a collection member. Bear in mind that these activities happen at start-up time only, and thus once and once only.

At run-time, when e.g. unmarshalling an XML document to a predefined object hierarchy, Cstor will (and has to) use reflection to ...
a) create object instances (using Class.newInstance())
b) set and get values into/from your instance variables (Method.invoke())

SQL to find out indexes on a table

Quick reference to find out indexes on a table from the command prompt.

select * from user_indexes where table_name = 'MY_TABLE'
select * from  user_ind_columns where index_name = 'PK_MY_INDEX'

To find out the triggers on a table:

select * from user_triggers where table_name = 'MY_TABLE'

Disk usage commands on Solaris

du and df are the most powerful commands on Solaris to find out the disk usage.
Here is a list of the most useful commands at a glance.
- df -h (shows the file system partitions)
- du -ks /opt/* | sort -nr | head -5 (shows the top 5 dirs in /opt. Subdirs are summarized.)
- du -ak /opt/* | sort -nr | head -5 (shows the top 5 dirs/files in /opt including subdirs)

Modal Dialog session expiry issue in IE 6

In our application we were facing a very queer problem. One of our page flows was opening a modal dialog from another modal dialog. While doing this we were facing an intermittant problem of 'session-expiry'. I put a sniffer on the client machine and saw that a new sessionID was generated. This meant that the JSESSIONID cookie was not being propogated to the child windows.
I also realised that this problem was only with 'Modal Dialogs' and hence IE specific.

I tried in vain to understand where we were doing things wrong. Lady luck was shining on me, when I found this link that explains the problem.
Snippet from the microsoft site:

When Internet Explorer opens a window from a modal or modeless HTML dialog box by using the showModalDialog method or by using the showModelessDialog method, Internet Explorer uses Microsoft Component Object Model (COM) to create a new instance of the window. Typically, the window is opened by using the first instance of an existing Internet Explorer process. This process is different from the process that Internet Explorer uses to open a new window by using the window.open method.
When Internet Explorer opens the window in a new process, all the memory cookies are no longer available, including the session ID.

The workaround was to pass the window object of the parent of the dialog box into the dialog box, and then use that object to open the new window.

For e.g. In the parent window:

var args = new Object;
args.window = window;
showModalDialog("modal.asp", args)

And in the child window:

dialogArguments.window.open('page1.asp')

Integrating JAMon war with your application on Websphere

We were using the JAMon package for performance monitoring. We needed to integrate the JAMon WAR file contents into our application so that we could get the web-pages used for monitoring the application.

On Websphere 6.1, the class-loader hierarchy and our security policies did not allow for JAMon libs to be added at the system class-loader level.
One option was to include the contents of the JAMon war file into our WAR file.

But I thought a cleaner option would be to make a EAR file with 2 web-modules (one war of my application and other jamon.war)
On WAS 6.1, I changed the class-loader policy of the application to: Single class loader for application.

The application.xml file for the EAR file contained the following items:

<application id="Application_ID">
    <display-name>MyApplication.ear</display-name>
    <module>
        <web>
            <web-uri>MyApplication.war</web-uri>
            <context-root>MyApplication</context-root>
        </web>
    </module>

    <module>
        <web>
            <web-uri>jamon.war</web-uri>
            <context-root>jamon</context-root>
        </web>
    </module>
</application>

Access predicates and Filter predicates in Explain plan

In Oracle 10g, when we see the explain plan for a SQL, then there are 2 new items that are of interest - the 'Access predicate' and the 'Filter predicate'.
We know that depending on the predicate information ('where clause') the Oracle optimizer chooses the optimal path for executing the query.

So the 'access predicate' information tells us how the Oracle optimiser is accessing the rows of the table - i.e. if the optimizer is using an index, then what predicate made it use the same.

The 'filter predicate' tells us what criteria Oracle is using to filter rows from the returned rowset (the rowset was fetched based on the 'access predicate') .

Webpshere v6.1 deployment hack

Whenever I wanted to change a single file on WAS server, I often used to copy that file to the 'installedApps' EAR directory of the application.

This used to work and though not mentioned in any IBM redbook; was considered as a hack by our team. But one day we ran into a problem which took us ages to decipher.

Whenever we changed the web.xml in WEB-INF, the changes were never reflected. For e.g. if we added a new struts-config entry to web.xml or a new filter, the configuration was never picked up.
We never really noticed this and thought the problem was with our struts-config file.
But the core issue was that WAS always picks up the web.xml file of an application from a different directory

-/opt/IBM/WebSphere/AppServer/profiles/
{profileName}/config/cells/{CellName}/applications/
{Application_EAR}/deployments/{Application_WAR}/WEB-INF

I changed the web.xml file here and all issues with struts-config were solved :)

JAMon JDBCMonProxy -getting the orginal connection

While using JAMon to monitor SQL perf stats, we faced a small problem - at one place we needed the underlying OracleConnection object and not the proxy, as we were calling a vendor-specific method on it.
Thankfully the JAMon API MonProxy object has a "getMonitoredObject()" method through which I could get hold of the OracleConnection object and use it.

Here is the sample code snippet: (just in case someone else needs it :)

// we need to do this as the 'setEndToEndMetrics' 
// method is only available on an OracleConnection.
   if (Proxy.isProxyClass(connection.getClass()))// if a dynamic proxy object
   {
    InvocationHandler invocationHandler = 
                   Proxy.getInvocationHandler(connection);
    Object monitoredObject = 
                   (MonProxy) invocationHandler).getMonitoredObject();
    ((OracleConnection) monitoredObject).
                    setEndToEndMetrics(metrics, (short) 0);
   }

rm craziness

I was trying to delete some files on Solaris using the rm command.
The file names were something like: "-HqFbc3YQ0wQYhH7u16veP7_BBC_6_090540.xml"
These files were generated by a program using random numbers.

Now no matter what I tried, rm always returned with an error for files starting with a - (minus sign)
I found the solution here.

Snippet:
To remove a file whose name begins with a dash ( - ) character, refer to the file with the following syntax:
rm ./-filename
Using the redundant ./ directory information prevents the dash from occurring at the beginning of the filename, and being interpreted as an option of the rm command.

What does a Digital certificate contain?

If you view a digital certificate using FireFox browser or decoding it using some tool such as OpenSSL/IKeyMan; then we can see the following parts/data:-
- Issued by {The CA name}
- Issued to/Subject {CN of the subject (typically the DNS of the server)}
- Valid From {date}
- Valid Till {date}
- Public Key {typically the 1024 bit public key of the subject}
- Public Key Algorithm {The encryption algorithm to be used with the key, e.g. RSA}
- Signature Algorithm {The algorithm used to generate the digital signature, e.g. MD5HashWithRSA, SHA1HashWithRSA}
- Signature {The digital signature}

The digital signature is created by hashing the public key of the subject using MD5 or SHA1 and then encrypting the hash using the private key of a CA.

An entity verifying this digital cert would:
- decrypt the hash using the public key of the CA
- create a new hash of the public key of the subject
- compare both the hash keys and verify if the subject can be trusted.

Formats for Digital certificates

I knew that all Digital certs confirm to the X.509 standard. But why were they so many extensions for certificate files? - e.g. DER, CER, .P7B, PFX etc. I found this link that contains good info about X.509 certs.
Basically a X.509 cert can be in many formats (depending on the encoding used to save the cert)

.CER - CER encoded certificate, sometimes sequence of certificates
.DER - DER encoded certificate
.PEM - (Privacy Enhanced Mail) Base64 encoded DER certificate, enclosed between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----"
.P7B - See .p7c
.P7C - PKCS#7 SignedData structure without data, just certificate(s) or CRL(s)
.PFX - See .p12
.P12 - PKCS#12, may contain certificate(s) (public) and private keys (password protected)

A .P7B file could be a container for more than one digital certificate.

Adding trusted root certificates in Websphere 6.1

We had deployed an application to WAS 6.1 ND. This application contained a JAX-WS 2.0 webservices client that used to call a third-party service using SSL. The digital certificate used by the third-party was self-signed and hence we needed to accept it as a trusted party in our Trust Store.

We imported the certs into a trust-store (JKS format) using the keytool command of the JDK and wrote the following code:

System.setProperty
("javax.net.ssl.trustStore", trustFilename );
System.setProperty
("javax.net.ssl.trustStorePassword", "changeit") ;
System.setProperty
("javax.net.ssl.keyStore", trustFilename );
System.setProperty
("javax.net.ssl.keyStorePassword", "changeit");

But unfortunately this was not working on WAS6.1. (This works fine on Tomcat).
In earlier versions of WAS, the iKeyman tool provided an interface to manipulate digital certs, keys, Trust stores and Key stores. But in WAS 6.1, all these tasks can be done from the web-based admin console.

So to add the certs to the Trust store, I went to "Security (on the left panel) --> SSL certificate and key management > Key stores and certificates > NodeDefaultTrustStore > Signer certificates"
Add the new root digital certs that need to be trusted to this store. The JAX-WS client should now be able to connect to the HTTPS resource. Remove any system properties that have been set before.

A good article (for WAS 6.0) describing how SSL and Digital certs work in Websphere can be found here.

Hack to deploy JAX-WS 2.0 webservice on Websphere 6.1

Our team had developed a webservice using JAX-WS 2.0 and using the Reference implementation of Sun as the JAX-WS provider. The application was developed using Netbeans and the embedded Tomcat server.

But when the application was deployed on WAS 6.1, then it failed and resulted in ClassCast exceptions. The base installation of WAS 6.1 only supports JAX-RPC 1.1.
IBM did provide an option - installing the webservices feature pack for WAS 6.1
The installation was a bit of a pain, especially on a cluster. You essentially had to create new AppServer profiles, create a new cluster and move all applications to the new cluster. This would also required considerable downtime on a live production server.

I looked for an other option. I tried to understand what was happening under the hoods when Netbeans was creating a webservice. It was clear that the webservice created was using a Servlet provided by the RI of JAX-WS 2.0. Even all the annotations and JAXB libraries were included in the lib directory. Hence there was no dependancy on any server-specific library as such.

I realised that the ClassCast exceptions on WAS 6.1 were happening due to the ClassLoader heirarchy - which was set to 'Parent First'. I decided to try to change this configuration to 'Child First' - so that JAX-WS RI libraries are picked up from WEB-INF/lib first.

There were 2 changes that I made using the web admin console. Go to "Enterprise Applications --> {Application name} --> Class Loader" and changed the following items:
Select "Classes loaded with application class loader first" and "Single class loader for application". Save these changes to the master configuration and restart the AppServer.

This hack worked and we could deploy JAX-WS 2.0 webservices on WAS 6.1 :)

Tip: We ran into a few Linkage errors, which we could resolve by removing all duplicate jar files.
A linkage error would typically occur when a class tries to call another class loaded by a different class loader.

Is POX/HTTP same as REST?

There is a lot of confusion in the industry over REST webservices. A lot of applications offer POX/HTTP interfaces and call them REST webservices. But in theory, just having XML tunneled over HTTP does not satisfy all REST principles.

It's important to understand that REST is not a protocol. It is an architecture style that can be used to design systems. It brings in a noun-oriented approach to access resources (instead of a verbs).

The Apache CFX framework has new features (using the magic of annotations) that make it possible to have true REST style webservices.
More information on this can be found here and here.

Recently came across this article that states the REST principles that should be followed during design:

1. Give every resource an ID. Use URIs to identify everything that merits being identifiable, specifically, all of the "high-level" resources that your application provides, whether they represent individual items, collections of items, virtual and physical objects, or computation results. Examples:

http://example.com/customers/1234

http://example.com/orders/2007/10/776654

http://example.com/products/4554

2. Link things together. (Hypermedia as the engine of application state)

Example: Element in XML doc: product ref='http://example.com/products/4554'

We could use a simple 'id' attribute adhering to some application-specific naming scheme, too but only within the application’s context. The beauty of the link approach using URIs is that the links can point to resources that are provided by a different application, a different server, or even a different company on another continent because the naming scheme is a global standard, all of the resources that make up the Web can be linked to each other.

3. Use standard methods. For clients to be able to interact with your resources, they should implement the default application protocol (HTTP) correctly, i.e. make use of the standard methods GET, PUT, POST, DELETE

e.g. GET - get order details/list all orders

PUT - update order

POST - add order

DELETE - delete order

4. Communicate statelessly - This is for scalability.

Message style and encoding in WSDL documents

The WS-I specs state the following 2 types of encoding to be complaint to the Basic profile:
- RPC/literal
- Wrapped document/literal

Found this cool article on developerworks that explain all the styles and encoding in detail.

WSDL cheat sheet

With the plethora of tools available today for webservices development, we sometimes forget the basic fundamental semantics of a WSDL document. Here is a quick reference of what a WSDL contains:
1. Types: These are the data type definitions - expressed as a schema of complex/simple types.
2. Message: A message consists of a logical order of one or more types.
3. Operation: Defines a operation that consists of a input and output message.
4. Porttype: A collection of operations.
5. Binding: Specifies the concrete protocol data format specifications (e.g. document-literal) for a portType.
6. Port: Specifies a network addresss for a binding, thus defining a single communication endpoint.
7. Service: collection of endpoints or ports

Thus a service is defined by one or more endpoints/ports. A port is combination of a binding with a network address. Each binding is a combination of a porttype and concrete data format specifications. Each porttype is a collection of operations that define messages that are exchanged.

More information can be found here.

Xerces and Xalan packages in JDK 1.5

Since JDK 1.5, Sun has decided to change the package names of the Xalan and Xerces to "com.sun.org.apache.xalan" and "com.sun.org.apache.xerces". Old package names (org.apache.xalan.xslt etc) are now not shipped with JDK1.5

I think this is a welcome move, bcoz this will enable end-users to use new versions of Xerces and Xalan easily without resorting to class-loader manipulation. Earlier this was done using the endorsing mechanism for JAR files.