Can U request Google, Yahoo to not index Ur site?

I knew the way web crawlers/bots work to index your website. In fact Google also has a feature of submitting a SiteMap to better index the pages in Ur site. But what if U don't want some pages to be crawled. Well, today I learned that there is a way in which we can request crawlers to ignore certain pages in the site. The trick is to place a 'robots.txt' file in the root directory of the site. This text file contains folders and URLs that need not be crawled.
The protocol, however, is purely advisory. It relies on the cooperation of the web robot, so that marking an area of a site out of bounds with robots.txt does not guarantee privacya

Loosely typed vs Strongly typed webservices

This article gives a good overview of the difference between a loosely coupled and strongly coupled webservice.

Jotting down some points from the article here:

A loosely typed webservice is one where the WSDL (interface definition) does not expose the XML schema of the message to be transferred. One example of a loosely typed description is Web service that encodes the actual content of a message as a single stringThe service interface describes one input parameter, of type xsd:string, and one output parameter.

Another way of indicating that no strict definition of a message format exists in a WSDL file is the use of the element. Its occurrence in a schema simply indicates that any kind of XML can appear in its place at runtime. In that respect, it behaves much like the case where single string parameters are defined. The difference is that here, real XML goes into the message, as opposed to an encoded string.

Pros and Cons:
The service consumer and service provider need to agree on a common format that they both understand, and they both need to develop code that builds and interprets that format properly. There is not much a tool can do to help here, since the exact definition of the message content is not included in the WSDL definition.

On the positive side, if the message structure changes, you do not have to update the WSDL definition. You just have to ensure that all the participants are aware of such a change and can handle the updated format.

Examples where Loosely coupled services make sense:
1. For example, assume you have a set of coarse-grained services that take potentially large XML documents as input. These documents might have different structures, depending on the context in which they are used. And this structure might change often throughout the lifetime of the service. A Web service can be implemented in a way that it can handle all these different types of messages, possibly parsing them and routing them to their eventual destination. Changes to message formats can be made so that they are backward compatible, that is, so that existing code does not have to be updated.

2. Another example is the use of intermediaries. They have a Web service interface and receive messages. But many times, they provide some generic processing for a message before routing it to its final destination. For example, an intermediary that provides logging of messages does not need a strongly typed interface, because it simply logs the content of any message that it receives.

3. Finally, a message format might exist that can not be described in an XML schema, or the resulting schema cannot be handled by the Web service engine of choice.

A strongly typed service contains a complete definition of its input and output messages in XML Schema, a schema that is either included in the WSDL definition or referred to by that WSDL definition.

Strongly typed Web services provide both consumers and providers of services with a complete definition of the data structure that is part of a service. Tools can easily generate code from such a formal contract that makes the use of SOAP and XML transparent to the client and server-side code.

Smart clients....Has the pendulum swung back?

Before the emergence of the 'web' most of the UI development was 'rich-thick-client' based on MS platforms such as VB, MFC, etc. VB development enjoyed a lot of popularity because of the component based model and the ready availability of rich UI widgets and development environments such as Visual Studio.
Such rich clients suffered from 2 major drawbacks: DLL hell and application upgrade/deployment.
Maintaining the correct version of the application across all clients distributed across locations was a big pain.

To address this problem, many thick clients were migrated to a web-based UI with a centralized server and repository. But this 'silver bullet' again had 2 main drawbacks - the user experience was nowhere close to that of a rich client. Even with the emergence of RIA (Rich Internet Applications) and AJAX, those applications requiring tons of data input and fast keyboard based navigation across screens, the web UI offered serious limitations.

And it is here that .NET smart clients come into the picture. A .NET smart client has the following features:
- Uses the 'click once' platform feature to automatically update DLLs on the client machine. Thus deployment is no longer a problem.
- Smart clients build on .NET resolve the DLL hell problem using metadata versioning of assemblies.
- Smart clients can talk with webservices to obtain data and fulfill other business requirements.
- Smart clients can work in offline mode - offering either limited or full functionality.
- Smart clients can easily integrate with other applications on the desktop providing an integrated approach to the user.
- Using WPF, users can be given a rich user interface that they demand.

Deep linking in AJAX applications

AJAX appplications face the challenge of deep-linking because many times the page URL does not change, only the content changes using AJAX.
And without deep linking, spiders and bots may not be able to index your page.

Currently there are 2 strategies for deep-linking AJAX sites:
- Using anchors. Onload, javascript checks location.href for an anchor and then calls the existing ajax methods to add and remove the appropriate content.
- place the stateful URL ("Link to this page") somewhere within the page, at a location and using a style that will become well-known conventions. Both Google Maps and MSN Earth follow this technique.

"Neither Google Maps nor MSN Virtual Earth records the state of a particular map view on the browser’s URL-line. Google Maps hides all the parameters. MSN Virtual Earth presents only a GUID whose purpose I don’t yet understand, but which doesn’t record the map’s state. In both cases, the stateful URL is found elsewhere than on the URL-line. Google Maps presents it directly, with a link icon and a Link to this page label. MSN Virtual Earth presents it indirectly — clicking Permalink leads to a popup window containing the stateful URL."

Technical Architect responsibilities defined

This weekend, I was going thru some of my old books in my library and found one book that was my favourite during my developer days - J2EE Architects handbook. I perused through the first chapter that tried to define the role of a technical architect. The tasks given in the book were something that I was doing day in and out for projects. The simple and lucid language used in that chapter prompted me to blog some snippets from it.

Technical Architect

• Identifies the technologies that would be used for the project.
• Recommends the development methodologies and frameworks for the project.
• Provides the overall design and structure to the application.
• Ensures that the project is adequately defined.
• Ensures that the design is adequately documented.
• Establishes design/coding guidelines and best practices. Drives usage of design patterns.
  
• Mentors developers for difficult tasks.
• Enforces compliance with coding guidelines using code reviews etc.
• Assists the project manager in estimating project costs and efforts.
• Assists management in assessing technical competence of developers.
• Provides technical advice and guidance to the project manager.
• Responsible for ensuring that the data model is adequate.
• Guiding the team is doing POCs and early risk assessments.

Basic tools for Solaris Performance monitoring

One of the development teams I was consulting for was facing some performance problems on the Solaris platform. The Websphere JVM was hanging and the performance of the application dropped.

The three basic tools on the Solaris platform that are invaluable at this time are:
1. vmstat - gives CPU stats, memory utilization
2. iostat - Gives I/O stats
3. netstat - gives network stats.

This link gives good info on understanding the thump rules to apply to analyze the output of these commands.

Another command that is very powerful on solaris is the 'prstat' command. prstat is more versatile than 'top' that is present on most unix systems.

To find out the top 5 processes consuming CPU time:
prstat -s cpu -a -n 5

To find out the top 5 processes consuming most memory:
prstat -s size -n 5

To dump the CPU usage of a process every 15 secs to a file
prstat -p 2443 15 > server.out &

A very nice feature of prstat is that by using the -L switch, prstat will report statistics for each thread of a process.
prstat -L -p 3295

The following links also provide good info about some basic troubleshooting tips:
http://developers.sun.com/solaris/articles/prstat.html
http://www-1.ibm.com/support/docview.wss?rs=180&uid=swg21162381
http://www-1.ibm.com/support/docview.wss?uid=swg21052644

Capabilities of an ESB

Just saw Mark Richards presentation on ESB. The presentation is simple and easy to understand without all the fancy jargon.
The streaming video is available at: http://www.infoq.com/presentations/Enterprise-Service-Bus

Capabilites of an ESB:

Routing: This is the ability to route a request to a particular service provider based on some criteria. Note that there are many different types of routing and not all ESB providers offer them i.e. static, content-based, policy-based. This is a core capability that an ESB has to offer (at least static routing). Without it a product cannot be considered an ESB.

Message transformation: Ability to convert the incoming business request to a format understood by the service provider (can include re-structuring the message as well).

Message enhancement: We don't want to change the client and in order to do that we often need to enhance (removing or adding information, rules based enhancement) the message sent to the service provider.

Protocol transformation: The ability to accept one type of protocol as input and communicate to the service provider through a different protocol (XML/HTTP->RMI, etc).

Message Processing: Ability to perform request management by accepting an input request and ensuring delivery back through message synchronization i.e queue management.

Service orchestration: Low-level coordination of different service implementations. Hopefully it has nothing to do with BPEL. It's usually implemented through aggregate services or interprocess communication.

Transaction management: The ability to provide a single unit of work for a service request for the coordination of multiple services. Usually very limited since it's difficult to propagate the transaction between different services. But at least the ESB should provide a framework for compensating transactions.

Security: Provides the A's of security. There are no "silos" in SOA so the ESB has to be able to provide this.

YWorks - products

Came across this company that calls itself a 'diagramming company'.
A list of their product offerings can be found here.

Two interesting 'free' products that are included are:
- ANT script visualizer
- Java obfuscator

Ready to go templates

Believe it or not! There is a open-source project that aims to provide ready-to-go templates for all phases of a product or an application.
Link: http://readyset.tigris.org/nonav/templates/frameset.html

Difference between CPU time and elapsed time

In Performance metrics jargon, we often come across two measurements: CPU time and total elapsed time. So whats the difference between the two?

CPU time is the time for which the CPU was busy executing the task. It does not take into account the time spent in waiting for I/O (disk IO or network IO). Since I/O operations, such as reading files from disk, are performed by the OS, these operations may involve noticeable amount of time in waiting for I/O subsystems to complete their operations. This waiting time will be included in the elapsed time, but not CPU time. Hence CPU time is usually less than the elapsed time.

But in certain cases, the CPU time may be more than the elapsed time !
When multiple threads are used on a multi-processor system or a multi-core system, more than one CPU may be used to complete a task. In this case, the CPU time may be more than the elapsed time.

Difference between different editions of VS 2008

I was looking for a official comparision between the different VS2008 editions and thankfully I found it here.

The express editions are different downloads for each language. i.e. for C#, VB.NET
So if you want to open different projects in different languages, then go for standard or professional. Then the express editions do not have visual designers for WPF and WWF.

Some of the differences between the standard edition and professional edition are that you can use the professional edition to build MS Office applications. Also the professional edition has support for Crystal reports, Unit testing etc.

More information can be found at the following links:
http://msdn2.microsoft.com/hi-in/vs2008/products/default(en-us).aspx
http://msdn2.microsoft.com/hi-in/vs2008/products/bb894671(en-us).aspx

ETag response header

HTTP 1.1 introduced a new kind of cache validator i.e. ETag.

ETags are unique identifiers that are generated by the server and changed every time the resource is updated. An ETag is a string that uniquely identifies a specific version of a component. The only format constraints are that the string be quoted.

ETag: "10c24bc-4ab-457e1c1f"

The problem with ETags is that they typically are constructed using attributes that make them unique to a specific server hosting a site. So in a typical clustered environment, if the next request for a cached resource goes to a different server, then the ETag won't match and the resource would be downloaded again.

Most modern Web servers will generate both ETag and Last-Modified validators for static content automatically; you won't have to do anything.

More info about ETag can be found at the following links:
http://developer.yahoo.com/performance/rules.html#etags
http://nerdmonkey.com/blog/archives/000098.html
http://www.web-caching.com/mnot_tutorial/how.html

The advantage of ETag over the 'last-modified-date' tag is that the fact that ETags don't require a date but will take any string. So U can compose ETag with any custom logic and source.

Parallel download in browsers

We all know that browsers make multiple requests to servers for each and every resource that is contained in the page. But what is the max number of persistent connections that the browser can make to download all elements of a page?

Well to my suprise, the W3C specification on HTTP 1.1 actually states the same.
"A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. "

What this essentially means is that broswers can download no more than two components in parallel per hostname. U always discover new things :)

YSlow for Firebug - cool cool extension

I have been a die-hard fan of the 'Firebug' add-on to Firefox. Firebug has proved to be invaluable to me for diagnosing front-end performance issues.
Now Yahoo has extended Firebug with another feature called 'YSlow' that shows good aggregate data and also recommends changes to be made in the page.

More information can be found at http://developer.yahoo.com/yslow/

Difference beween WCM and ECM

Doing some research on Content Management systems, I got a bit confused about the difference between Web content management systems and Enterprise Content Management systems.

This article is a good read to understand and appreciate the differences.
The lines between all content technology families are very blurry and hence the auhor suggests that we evaluate a product based on the scenario requirement.

For publishing company websites, intranet sites, a WCM may be a good solution. A ECM solution may be focussed towards a particular industry.

Difference between a portal and a website

A lot of my friends ask me the difference between portals and web-sites. Typically an organization would have both - a company website and a portal. I found a good explanation here.

Portals typically have authentication and provides us information based on who we are. Personalisation and customization are the hallmarks of portals. So we require portals because:
Different roles require different information. Someone from grounds and buildings needs different info than the chair of computer science. (Customization) Different people with the same role work differently. (Personalization) Efficiency - people get directly to the info they need. (Work Flow) Customization insures they don't miss anything.

ThreadPool in Java

Recently one of my friends was looking for a good Thread-pool implementation in Java and asked me for help. I had used Doug Lea's ThreadPool in the past and recommended it to him.

But browsing thru the web, I learned that JDK 1.5 has introduced a java.uti.concurrent.* package that contained a good ThreadPool implementation. Using it was also a breeze. Here are a few code snippets.

ExecutorService pool;
pool = Executors. newFixedThreadPool(3);
pool.execute(new Task( Task(“three”,3));
pool.execute(new Task( Task(“two” two”,2)); ,2));
pool.execute(new Task( Task(“five five”,5)); ,5));
pool.execute(new Task( Task(“six”,6);
pool.execute(new Task( Task(“one”,1);
pool.shutdown();

The Task object is a Thread that implements the Runnable interface and has the run() method. The concurrent package also contains a lot of helper classes that can be used - for e.g. using a thread-safe collection, atomic varibles etc.

Websphere SOA Stack

A lot of people get confused between the different product offering by IBM in the Websphere space.The major products in the Websphere family are Websphere Application Server, Websphere ESB, Websphere Process Server and Websphere Message Broker.

Process Server is based on ESB and ESB is based on Application Server. Message Broker is termed as "Advanced ESB" by IBM and should be used when we have extensive heterogeneous infrastructures, including both standard and non-standards-based applications, protocols, and data formats.

So when to use what? Use Websphere ESB whenever most of the services you want to integrate are based on SOAP, XML standards. The Websphere ESB product can also be used as Websphere Application Server as it is built on the same core. Though Websphere Application Server can host simple services, Websphere ESB gives out of the box support for sophisticated mediation flows found in ESBs. We would need Websphere Process Server whenever we need to orchestrate simple services to create business services and host them.

Another technical difference is that Websphere ESB uses the JMS provider in WAS for messaging, whereas Message Broker uses Websphere MQ as the messaging engine. This is due to legacy reasons, I believe.

More information can be found here.The FAQ available at IBMs site also contains some good info.

Oracle Fast Connection Failover for a RAC cluster

Problem:Whenever one node of a Oracle RAC cluster goes down, some of the connections allocated to that node go stale in the connection pool. This causes errors such as 'Broken pipe' to be raised whenever the stale connections are used.

Solution:Configure a ONS (Oracle Notification Service) remote subscription client on the App Servers. Whenever any RAC node goes down, a 'DOWN' event would be raised and sent to all subscribers. The ONS client on the Appservers would receive this event and refresh all stale connections in the pool. When the node comes up, a 'UP' event is sent to the AppServer ONS client and again the connections in the pool are balanced against both the nodes.

In Oracle 10.2g, to configure ONS, U just have to add ons.jar file to WEB-INF/lib and 2 lines of code to configure the Connection Cache.

3 easy steps to self-sign a Applet Jar file

Found this link that explains how to self-sign an applet in 3 easy steps:

1. keytool -genkey -keystore myKeyStore -alias me

2. keytool -selfcert -keystore myKeyStore -alias me

3. jarsigner -keystore myKeyStore jarfile.jar me

Thats it..simple and sweet :)

FCKeditor

Came across this beautiful AJAX control that can be used to emulate a HTML editor text box.

http://www.fckeditor.net/ - worth a look.

Setting focus on the first field

Whenever a HTML page is loaded, we often want to set the focus on the first input field of that page. Came across this Javascript code that can be put in a JS file and reused everywhere.
http://www.codeproject.com/jscript/FocusFirstInput.asp Simple and Sweet solution :)

Reflection in Java-XML binding frameworks

Almost all Java-XML binding frameworks use Java reflection to map Java objects to XML documents. I had used Castor and XMLBeans in our projects and were satisfied with the performance of both of them.

Recently a friend of mine told me that he had reservations on using Castor as it uses Reflection and hence cannot be used in high-performance applications. I contemplated on this and felt that some basic amount of reflection code would be present in any XML-binding framework; or how would the API call 'set' methods on Java objects during unmarshalling?

I found this discussion on the net that explains this same fact.

Snippet:
Castor will still use some amount of reflection (mainly to set values onto your domain objects) even if you are using a mapping file (a binding file is used during code generation only).

Having said that, let me add one thought. At the time most of these articles have been written (2001 to 2002), the use of reflection introduced some performance penalties, noticeable to some degree.

Since then, a lot of time and effort has gone into improving the JVMs/JDK/compilers to lessen this impact further and further. In my personal view, with the event of JDK 1.4 (and even more so with Java 5)
this problems have been addressed sufficiently in that I would not call the use of (moderate) reflection an issue any more. If use of reflection would still be an issue, frameworks like Spring and some of their most
powerful features (run-time AOP through Spring interceptors) would not have been adopted by the market as it happened in the last one or two years.

Btw, in case this is not clear enough, using a mapping file instead of Castor's 'introspector' to establish mappings between the Java domain classes and the XML artefacts does not really improve Castor's
performance, as both modes will cause Castor to use reflection to establish what methods to use on your Java objects to set a value, to get a value (from an instance variable) or to add a value to a collection member. Bear in mind that these activities happen at start-up time only, and thus once and once only.

At run-time, when e.g. unmarshalling an XML document to a predefined object hierarchy, Cstor will (and has to) use reflection to ...
a) create object instances (using Class.newInstance())
b) set and get values into/from your instance variables (Method.invoke())

SQL to find out indexes on a table

Quick reference to find out indexes on a table from the command prompt.

select * from user_indexes where table_name = 'MY_TABLE'
select * from  user_ind_columns where index_name = 'PK_MY_INDEX'

To find out the triggers on a table:

select * from user_triggers where table_name = 'MY_TABLE'

Disk usage commands on Solaris

du and df are the most powerful commands on Solaris to find out the disk usage.
Here is a list of the most useful commands at a glance.
- df -h (shows the file system partitions)
- du -ks /opt/* | sort -nr | head -5 (shows the top 5 dirs in /opt. Subdirs are summarized.)
- du -ak /opt/* | sort -nr | head -5 (shows the top 5 dirs/files in /opt including subdirs)

Modal Dialog session expiry issue in IE 6

In our application we were facing a very queer problem. One of our page flows was opening a modal dialog from another modal dialog. While doing this we were facing an intermittant problem of 'session-expiry'. I put a sniffer on the client machine and saw that a new sessionID was generated. This meant that the JSESSIONID cookie was not being propogated to the child windows.
I also realised that this problem was only with 'Modal Dialogs' and hence IE specific.

I tried in vain to understand where we were doing things wrong. Lady luck was shining on me, when I found this link that explains the problem.
Snippet from the microsoft site:

When Internet Explorer opens a window from a modal or modeless HTML dialog box by using the showModalDialog method or by using the showModelessDialog method, Internet Explorer uses Microsoft Component Object Model (COM) to create a new instance of the window. Typically, the window is opened by using the first instance of an existing Internet Explorer process. This process is different from the process that Internet Explorer uses to open a new window by using the window.open method.
When Internet Explorer opens the window in a new process, all the memory cookies are no longer available, including the session ID.

The workaround was to pass the window object of the parent of the dialog box into the dialog box, and then use that object to open the new window.

For e.g. In the parent window:

var args = new Object;
args.window = window;
showModalDialog("modal.asp", args)

And in the child window:

dialogArguments.window.open('page1.asp')

Integrating JAMon war with your application on Websphere

We were using the JAMon package for performance monitoring. We needed to integrate the JAMon WAR file contents into our application so that we could get the web-pages used for monitoring the application.

On Websphere 6.1, the class-loader hierarchy and our security policies did not allow for JAMon libs to be added at the system class-loader level.
One option was to include the contents of the JAMon war file into our WAR file.

But I thought a cleaner option would be to make a EAR file with 2 web-modules (one war of my application and other jamon.war)
On WAS 6.1, I changed the class-loader policy of the application to: Single class loader for application.

The application.xml file for the EAR file contained the following items:

<application id="Application_ID">
    <display-name>MyApplication.ear</display-name>
    <module>
        <web>
            <web-uri>MyApplication.war</web-uri>
            <context-root>MyApplication</context-root>
        </web>
    </module>

    <module>
        <web>
            <web-uri>jamon.war</web-uri>
            <context-root>jamon</context-root>
        </web>
    </module>
</application>

Access predicates and Filter predicates in Explain plan

In Oracle 10g, when we see the explain plan for a SQL, then there are 2 new items that are of interest - the 'Access predicate' and the 'Filter predicate'.
We know that depending on the predicate information ('where clause') the Oracle optimizer chooses the optimal path for executing the query.

So the 'access predicate' information tells us how the Oracle optimiser is accessing the rows of the table - i.e. if the optimizer is using an index, then what predicate made it use the same.

The 'filter predicate' tells us what criteria Oracle is using to filter rows from the returned rowset (the rowset was fetched based on the 'access predicate') .

Webpshere v6.1 deployment hack

Whenever I wanted to change a single file on WAS server, I often used to copy that file to the 'installedApps' EAR directory of the application.

This used to work and though not mentioned in any IBM redbook; was considered as a hack by our team. But one day we ran into a problem which took us ages to decipher.

Whenever we changed the web.xml in WEB-INF, the changes were never reflected. For e.g. if we added a new struts-config entry to web.xml or a new filter, the configuration was never picked up.
We never really noticed this and thought the problem was with our struts-config file.
But the core issue was that WAS always picks up the web.xml file of an application from a different directory

-/opt/IBM/WebSphere/AppServer/profiles/
{profileName}/config/cells/{CellName}/applications/
{Application_EAR}/deployments/{Application_WAR}/WEB-INF

I changed the web.xml file here and all issues with struts-config were solved :)

JAMon JDBCMonProxy -getting the orginal connection

While using JAMon to monitor SQL perf stats, we faced a small problem - at one place we needed the underlying OracleConnection object and not the proxy, as we were calling a vendor-specific method on it.
Thankfully the JAMon API MonProxy object has a "getMonitoredObject()" method through which I could get hold of the OracleConnection object and use it.

Here is the sample code snippet: (just in case someone else needs it :)

// we need to do this as the 'setEndToEndMetrics' 
// method is only available on an OracleConnection.
   if (Proxy.isProxyClass(connection.getClass()))// if a dynamic proxy object
   {
    InvocationHandler invocationHandler = 
                   Proxy.getInvocationHandler(connection);
    Object monitoredObject = 
                   (MonProxy) invocationHandler).getMonitoredObject();
    ((OracleConnection) monitoredObject).
                    setEndToEndMetrics(metrics, (short) 0);
   }