Tech Talk

Techniques for handling very large strings in Java

2012-01-03T14:33:00.000+05:30

In my previous blog, I had jotted down the perils of storing large strings in memory. So what are the alternatives? Listing down a few at the top of my head right now.

Stream the string to a file and read chunk-wise from the file when required.
Store an array of strings, instead of storing a large string. A large continuous block of memory may not be available, but there could be small holes in the fragmented heap.
Compress the string using GZIP compression methods. Use the GZIPWriter class to keep appending strings to a byte-buffer.
If the large XML string is to be sent back as a webservice response, utilize the streaming support in SOAP stacks such as Axis 2 and CXF. Evaluate the use of MTOM for large attachments.
If you are operating on a large number of files, first deal with the 'large' files. To understand why, please peruse these links - Link 1 & Link 2

In one of the scenarios, the large XML string had to be fed to the JasperReports engine. Found a few interesting options to deal with this challenge here.

Heap Memory in .NET

2012-01-03T12:32:00.000+05:30

Apropos my previous post, my team was trying to resolve another memory leak problem in one of the .NET applications. It is interesting to note that a .NET program does not have any explict way to specify the heap size. The .NET heap size will keep on growing till it consumes all of the available memory.
A hosted application such as IIS can control the amount of heap allocated to a Application Domain.
The following discussion threads throw more light on this: Link1 Link2

Also found this amazing article by Andrew Hunter (ANTS profiler contributor) explaning the Large Object Heap concept in .NET. Understanding these concepts will make us appreciate how we get an unexpected OutOfMemory error even if our total object size is relatively small.

OutOfMemoryError while using StringBuilder/StringBuffer

2011-12-30T16:42:00.001+05:30

I was helping a friend debug a OutOfMemory exception in a Java web application. The program made heavy use of StringBuilder and was appending a large number of strings. An entire record set (containing thousands of records) was essentially converted into an XML string.
Strangely when the OOM error occured, there was still plenty of heap memory available. Furthur deep-dive debugging and some googling around, taught a few important lessons.

Whenever the internal buffer capacity of StringBuilder/StringBuffer is exceeded, then the next character array size it creates is twice of the original size. A good blog explaining this is here. Hence it is better to initialize the initial capacity of the StringBuilder to a reasonable value beforehand.
StringBuilder needs a continuous block of memory for further allocation. For e.g. you may have 20MB free heap space, but it may be fragmented. Hence even a 5MB StringBuilder allocation may fail and result in a OOM error. Links to forums - Link 1 Link2 Link3
Try to use a 64-bit machine, as there are no practical limitations for the heap memory allocation and also it is much easier to find a continous block of memory due to 64-bit addressing.
Alter the design of the program to not store the string in memory, but in a file. Alternatively stream it directly to the HTTP response.

Taxonomy of Services

2011-12-02T17:01:00.000+05:30

In one of my previous posts, I had blogged about creating a taxonomy of services using functional categorization.
For e.g. Entity Services, Task/Activity Services, Process Services and Infrastructure services.

But services can also be categorized from different perspectives such as layers or intent of use. For e.g.
Categorization based on Service Layer:

1. Business Services: Represent high level business functions that define an enterprise.
2. Application Services: Application specific and usually will be aggregated in a composite service at the business level.
3. Infrastructure Services: Utility functions that deliver cross cutting functions.

Categorization based on scope:
1. Enterprise Services: Multiple LOBs use the service.
2. Domain Services: Applicable only within a LOB.
3. Application Services: Local to the App Level.

Analysis vs Design

2011-11-17T19:01:00.002+05:30

This age-old debate keeps propping up every now and then :)
Found a couple of good articles reflecting on the difference between the two.

http://butunclebob.com/ArticleS.UncleBob.AnalysisVsDesign

http://devhawk.net/2004/03/30/analysis-vs-design-modeling/

Oracle Web Service Manager vs Oracle Enterprise Gateway

2011-11-03T21:45:00.000+05:30

Oracle Web Service Manager is an integral part of the Oracle SOA suite and it allows us to implement security declaratively; without any coding from the developer. Security policies can be enforced at run-time using WSM agents or WSM gateways.
There is another product called "Oracle Enterprise Gateway" that has features that overlap with OWSM - hence this results in lot of confusion.

So, lets understand the concepts one by one. A WSM agent is a component that is installed with the endpoint service. So it provides the 'last-mile' security (last security layer). Now OWSM also a gateway component where security policies can be employed in a central location. A gateway can also perform functions that an agent cannot do, such as message routing, transformations, and failover. OWSM also has an extension for OSB, that allows us to use OWSM policies at the OSB (ESB) layer. These capabilities would suffice the requirements of most intranet SOA infrastructures.

Oracle positions the Oracle Enterprise Gateway as the first line of defence ("perimiter security") when SOA services are exposed to the outside world. This is the equivalent of a DMZ firewall. So it looks like Oracle Enteprise Gateway is a more expansive product that can do everything that OWSM does, plus all the bells and whistles.

Ruminating on the Oracle MDM suite

2011-11-01T14:32:00.001+05:30

Recently during one of our internal brainstorming sessions, there was a lot of confusion over the various components available on the Oracle platform to build a robust MDM solution. Part of the confusion was because Oracle has picked up best of breed components from various acquisitions and integrated them together to form the MDM suite. Its important to understand that when someone talks about Oracle MDM - it is a suite of components and NOT just one product.

Oracle's acquisition of Hyperion has further added to the confusion, as Hyperion has full capabilites to be used as a MDM solution. Oracle promotes Hyperion Data Relationship Manager as a component in its MDM suite - that can be used for managing the relationships between different attributes of master data from disparate sources.

At the fundamental level, to build a MDM end-to-end solution, you need basic components such as a ETL tool, Data Profiler, Data Cleansing Engine that can be used for standarization, de-duplication, validation, etc. Given below are the core components of the Oracle MDM suite, followed by optional components that help in jump-starting your MDM journey.

Oracle Data Integrator Enterprise Edition: ODI can be used for ELT style bulk data movement, or near real-time updates, and data services. ODI can consolidate master data from various sources and also publish master data to downstream applications. (Note: Oracle has acquired Golden Gate product that enables real-time intergration of data across disaparate data-sources. GG can also be used in conjuction with ODI for movement of data)
Oracle Data Quality / Data Profiling: Oracle Data Profiling allows us to profile the master data and investigate the content and the structure of their different data sources. It also gives users the ability to monitor the evolution of data quality over time using Time Series. Oracle Data Quality allows us to standardize, validate, cleanse and enrich master data – for e.g. master list of securities, issuers, official list, etc. Using both these tools will ensure the integrity of data stored in the MDM data store.
Oracle Business Intelligence Suite: OBIEE can be used for analytics and reporting on master data entities.

Besides these core components, Oracle MDM suite also contains pre-packaged MDM solutions such as “Customer Hub”, “Product Hub”, “Site Hub”, “Supplier Hub”, etc. (Some of these are part of Siebel MDM, I believe.)

Ruminating on IRM (Information Rights Management)

2011-10-27T17:54:00.001+05:30

From a security architecture perspective, it is important to consider the need for using IRM technology. Traditionally we have secured access to documents using RBAC patterns for secure access and download.

But how do you control the information once it is downloaded to the users machine? Can the user copy/paste from the document? Can the user print the document? Can the user forward the document to someone or upload it somewhere? Can he run macros on the document? So how can an enterprise have total control on sensitive information?

These questions cannot be answered by classical access control mechanisms, they need a new security framework concept called "Information Rights Management". Many traditional ECM vendors also offer IRM adapters or add-ons to help customers have total centralized control over their digital assets. For e.g. SharePoint 2010 has IRM protectors that can be plugged-in for end-to-end protection of documents on the user's computers.Oracle UCM can be extended with Oracle IRM, etc.

Across all these IRM product architectures, it is necessary to have some form of client application installed on all users machines. Files that get downloaded from the DMS are special encrypted rights-managed files. The file format contains meta-data that defines the access that can be given to the user. The client application would decrypt the file, understand the access constraints and accordingly give rights to the user. On the windows platform, MS has long released the Windows Rights Management Services - a comprehensive API to address IRM challenges on the windows platform.

How does .NET TPL control the number of threads

2011-08-11T18:40:00.001+05:30

I often wondered what heuristics the Task Parallel Library (TPL) in .NET uses to control the number of threads for optimal utilization on multi-core machines.

Found a great discussion thread on StackOverFlow explaining the details.

Techniques for Service Identification in SOA

2011-07-21T15:00:00.000+05:30

Its very important to use proper service identification techniques to identify services in an portfolio. In fact, service identification should be the first step in your service lifecycle management process.

Jotting down some of the techniques that we have been using for identifying services:

Domain decomposition approach - Look at the high level business entities and create entity services for them.
Top down BPM driven approach - Start from the business processes and divide them into sub-processes. Each business process consists of tasks & activities, that would orchestrate between different service components.
Business Goal driven approach - Derive services from business goals. Decompose the business goals into a set of services that would help satisfy the business goal. Provide tracability between business goals and IT services by a Goal/Business Service matix.
Existing systems - Service wrappers are created on existing systems - to surface them for orchestration in a business process or a composite service. This technique may not be appropriate if the existing IT landscape is not aligned with business goals.
UI driven approach - Identify/discover services based on the user interface requirements. UI technologies such as Flash, Silverlight, Ext-JS directly call JSON/REST services on back-end application servers.

Ruminating on SEO

2011-06-28T13:08:00.000+05:30

SEO (Search Engine Optimization) is an integral part of any Internet Marketing Campaign. SEO strives to increase the visibility of a website in search results. For SEO, we have to consider both on-page factors and off-page factors. Given below are some examples of what can be done on the site pages and what needs to be done outside the site pages.

On-Page factors examples:

Meta tags
Headings
Links
Keyword frequency (Internal Keyword linking strategy)
Site Structure (Create and submit site maps)
UI design that is "Search Engine Friendly" (Image Alt tags, Menus, etc.)
Make tagging and bookmarking easy.
Robots.txt
URL Normalization (if different URLs lead to the same content)

Off-Page factors examples:

PageRank analysis (Increase no. of inbound links)
Utilize Social Networks, Forums to form a 'link partnership'. (Social Media Optimization - Social SEO)
Create and submit articles, blogs, RSS feeds that link to the site.
Create Sharable content - Mashup ready.

XSL transformations on the brower

2011-06-27T18:28:00.003+05:30

Recently, I came across a web framework that was quite unconventional - the framework was performing XML transformations using XSLT on the browser. All web requests were directed to a legacy system that returned XML and this XML was directly sent to the brower. The browser had already loaded the necessary XSL, Javascript and does the XML -> HTML transformation.

There are certain pros and cons of this approach.
Advantages:

Clear separation of markup/layout from content.
Heavy XSL processing offloaded to the brower, brower JS code can check for brower compatibilities and spit out right markup code.

Disadvantages:

All browsers do not support XSL in a standard way. Can be quite a pain to make the look-n-feel compatible with all browsers.
Search engines/bots see raw XML, may not be able to interpret and understand.
Disables progressive rendering. User won't see anything at all until entire stylesheet and data is loaded completely.
XSL is pretty tough to master. Resource skill could become a potential bottleneck.
Could run into scalability issues for large XML payloads. When processing XML files, XSLT must load the entire document into memory.

IMHO, although XSLT processing in-browser is fast compared to server side, it is still better if no transformation is required at all. There are tons of server side web-frameworks that can do the job better and faster.

Open source API for read/write to Excel files

2011-06-15T11:50:00.000+05:30

Long back, I had blogged about native APIs in .NET and Java to read/write Excel files.
Recently came across a new native .NET library that can be used to read/write Excel files and supports the binary BIFF format. This is the format used by Excel 2005 files - i.e. xls files. This API is still evolving and looks quite basic at this point of time.
The code is available here: http://code.google.com/p/excellibrary/
There is also a .NET port of the popular Apache POI library available at: http://code.google.com/p/npoi/

The new office (2007, 2010) documents are based on XML standards (i.e. xlsx files). There are a couple of open source projects for creating and modifying 2007 'xlsx' excel files.
1) http://epplus.codeplex.com/
2) http://excelpackage.codeplex.com/

Operational Reports vs MIS Reports

2011-04-13T14:38:00.001+05:30

Once organizations create a data warehouse, a lot of people push all the reporting needs to the DW. But do all reports need to run from a DW?
The answer lies in understanding the difference between operational and informational (MIS) reports. A good article by Bill Inmon on this difference can be found here.

Operational reports are typically detail oriented and shows the latest up-to-date records. Operational reports are used by stakeholders for short term tactical decision making. MIS reports look at summary data over a longer time horizon and are used for strategic decision making.

Examples of operational reporting include bank teller end-of-day window balancing reports, daily account audits and adjustments, daily production records, flight-by-flight traveler logs and transaction logs.

Examples of informational reporting include monthly sales trends, annual revenue, regional sales by product line for the quarter, industry production figures for the year, number of employees by quarter and weekly shipping costs by carrier.

Activity Diagrams vs BPMN Diagrams

2011-03-23T13:06:00.001+05:30

For modeling business processes, there are two standards popular today – UML 2.0 Activity Diagrams and BPMN.

There are semantic differences in notation between the two standards. For e.g. the way OR-splits, AND-splits/joins are shown.

A detailed whitepaper showcasing the differences between the two notations can be found here.
A discussion thread at http://www.bpm-research.com/forum/index.php?showtopic=501 makes an interesting read.

Business Function Models Vs Business Capability Models

2011-01-29T20:42:00.000+05:30

The difference between these models boils down to the difference between a “business function” and a “business capability”. Many organizations use them interchangeably. For e.g. A business capability model may illustrate current-state business functions and also future-state business functions that need to be built to deliver on the business vision.
But there are few folks who would like to draw a clear line of differentiation between the two. Capability can be defined as the ability to perform actions to achieve specific strategic goals/objectives.

The following links provide interesting reading:
Link 1
Link 2

Hence a business capability is much more than a business function – it encompasses other objects such as Actors, Services, Functions, Processes and Infrastructure. Examples of business capability are – the ability to service customers through online channels, capability to survive liquidity crisis, etc.

Business Architecture Models

2011-01-29T20:38:00.002+05:30

While defining the enterprise architecture of an organization, it is essential to understand the various processes, functions and capabilities of the business. Recently came across the FEA Business Reference Model on Wikipedia. The simplicity of the diagram impressed me.

Every organization has different business areas or LOBs. Each business area has a set of business functions that it performs. A ‘Business Function Model’ describes these functional areas and sub-functions in a graphical representation. An example of a BFM can be found here.

Each LOB executes its business functions by following certain business processes. A business process is an orchestration of different business activities and tasks that may be exposed as SOA services.

Various dimensions of Security

2011-01-18T18:48:00.000+05:30

When we design our applications to be secure, we have to consider all aspects of security. I have often seen people associate security with just authentication and authorization, but there are other security principles to be considered as stated below.

Integrity: We have to ensure that all messages/data have not been tampered with. Integrity of messages ensures that the data has not been maliciously modified by 'man-in-the-middle'.
Confidentiality: This security principle ensures that all messages are encrypted and cannot be eavesdropped.
Authentication/Authorization: Ensure that all resource access goes through a proper authentication process.
Non-Repudiation: This ensures that any party involved cannot refute the validity of a message exchange.

Modern toolkits and technologies such as digital certifications satisfy all of the above security principles.

Concurrent Business Engineering

2011-01-17T16:48:00.000+05:30

Some time back, I had blogged about the advantages of having a unified BPM/SOA strategy at the enterprise level. Ran through a Forrestor report that touch a similar concept and calls it as "Concurrent Business Engineering".

Concurrent Business Engineering entails greater colloboration between Business and IT for jointly working on defining new business processes and also defining the technology platform for supporting the processes.
Business services are best designed with a strong understanding of the business process context, hence a top down BPM process-centric view would help in understanding what services need to be surfaced to provide maximum agility to the business process.

This is similar to the idea of having a unified BPM/SOA strategy and utilizes the best of top-down and bottom-up methods for executing the business strategy - as blogged earlier.

Types of Services in SOA

2011-01-14T18:47:00.000+05:30

Found a nice article on MSDN describing the various types of services - a taxonomy for services.
Jotting down the concepts explained in the article.

Entity Services - Expose/Surface business entities in the system.e.g. employee, customer, sales order,etc. They contain CRUD operations and additional domain specific operations - e.g. FindOrderByPrice Entity Services abstract the underlying datasources and persistence mechanisms.

Capability Services - Implement a specific business capability - for e.g. Pricing Service, Credit Card Processing Service, etc. They may use Entity Services for persistence.

Thus Entity Services are "data-centric" and Capability Services are "action-centric".

Process Services - Acts as a facade for a BPM process. Process services would maintain state due to the very nature of a workflow having to maintain state over a long running process. Process Services are typically implemented using BPM tools such as WWF, Biztalk, WPS, etc.

Infrastructure Services (Utility Services) - Common cross cutting functions such as Logging, Auditing, Security, Authorization, etc.

Ruminating on SOA Governance

2011-01-06T15:19:00.001+05:30

SOA Governance has two dimensions. First – the processes and methodologies used. Second – the tools and products used for governance.

Quite often, people assume that the purchase of a SOA Governance tool would suffice for implementing SOA Governance. But the fact is that the tools would only help in automating certain enforcement policies and service lifecycle workflows. What is first needed is a framework of processes, policies and organization structure to be defined. Any governance process needs to embrace the trilogy of “people, process and technology”.

The following diagram illustrates this point and states the various activities that need to be done for implementing SOA Governance.

The Open Group has also published a SOA Governance Framework that can be accessed here.

SOA Registry, Repository and Service Catalog

2010-12-29T13:27:00.000+05:30

While implementing enterprise SOA, it is important to consider deploying a service catalog for services. There is a lot of confusion between the concepts of registry, repository and service catalog.

Traditionally a registry has been a lookup service provided to service consumers. Service providers register their services in the registry and service consumers select an appropriate service for their needs. Standards such as UDDI addressed these needs.The Registry would contain service descriptions, service contracts and service policies that describe a service. Service registries have also been practically used for determining a service end-point address at runtime based on the service unique name.

So what is a repository? As the importance of SOA Governance grew, it became necessary to capture more meta-data about a service. A service repository integrates information about a service from multiple sources and stores it in a centralized database. Service information may include design artifacts, deployment topologies, service code repository, service monitoring stats, etc. Vendors have started positioning their generic asset management products as SOA repositories. For e.g. Rational Asset Manager.

A lot of vendors now sell a combined product that consists of the registry and repository. For e.g. IBM Websphere Registry and Repository.

A service catalog is a concept that can be implemented using SOA registry/repository products.

Entities Vs Value Objects

2010-12-27T14:52:00.000+05:30

In Domain Driven Design, we often separate Entities and Value Objects. Junior architects always get confused between these 2 concepts.
The essential difference is that domain entities have an identity and a lifecycle. So each Entity has a unique identity and with a given domain, no two entities can have the same identity. Value objects need not have an identity. So if we have an "equals()" method that compares the parameter values of each value object, then we can have value objects that are identical. Value objects should ideally also be immutable.
The following links offer interesting stuff on this concept.
1) Lostechies
2) StackOverflow
3) Devlicious

TCO of applications during Portfolio Rationalization

2010-12-17T17:58:00.002+05:30

In my previous blog post , I had narrated the process of portfolio rationalization. During the fact finding process, we need to calculate the TCO of an application. It’s a good idea to have a predefined template for entering all the parameters that add to the total cost of the application, i.e. hardware costs, software license costs, maintenance costs, data center costs, etc.

We should also try to collect the TCO statistics over a time period; i.e. over the last 3-5 years. This data when plotted on a graph would help us in identifying patterns and spotting trends. For e.g. if the TCO of an application is showing steep increase with every passing year, then we need to be wary of the “cost of inactivity”. Cost of inactivity means what will happen if no action is taken?

The TCO of applications should also be compared against the business value that the applications are providing. It may be that 70% of the TCO could be consumed by applications having 30% business value.
Another important dimension to capture would be the usage statistics and performance SLAs over the last few years. If the number of uses are increasing and the SLAs are not been met, then it’s time for some proactive action.

SOA and BPM

2010-12-06T19:09:00.001+05:30

Yesterday, we were having a discussion with one of our customers on the hot topic of SOA and BPM strategy, i.e. can SOA/BPM initiatives be combined?, what are the challenges, pitfalls, best practices, etc. Jotting down some of the key points of the brainstorming session.

To start with, its important to realize that both BPM/SOA has a common goal - greater business agility and to align IT with business. SOA and BPM complement each other and the potential benefits are compounded when you have a unified enterprise wide strategy for them.
BPM drives a process-centric thought process - right from design, implementation, monitoring and continuous optimization. BPM forces a paradigm shift from an application centric view to a process centric view. SOA is an architectural style where as BPM is a management discipline.
A combined BPM/SOA initiative will do the delicate balancing act between incremental and transformational change. Also a combined initiative should enable stakeholders to decide what important processes need that extra agility and prioritize them to be re-engineered as services because funding is always limited.
Top-down BPM appraoch drives the discovery of services since they provide important insights into understanding what parts of the IT portfolio can be exposed as SOA services. Thus BMP can provide a structured approach for identifying reusable business services.
SOA services also enable faster integration in BPM as the need for custom integration touch points reduces and this in turn enables faster deployment of BPM. SOA also enables rapid change of business processes which is not possible if the business process in embedded in a lot of traditiona non-SOA applications. For e.g. when a process needs to change to comply with a new regulation or due to a change in business strategy, then a loosely coupled BPM process orchestrated using SOA services is easier to change. New services can be plugged-in or existing services can be rearranged.

In today's fast changing business dynamics - "as is" and "to be" are simply temporal states of reality. The future state cannot be predicted, we can only stay prepared by keeping our business processes agile.

SONAR tool

2010-12-02T19:19:00.002+05:30

My team has been evaluating the SONAR tool to manage code quality. I was impressed with the features and the user friendliness of the tool. SONAR can be used for both Java and .NET projects. It has a open plug-in architecture that allows any code quality tool to be plugged in.

For example, for static code analysis it combines the power of popular tools such as PMD, checkStyle and FindBugs into an unified user interface that is great to use :)

SONAR also has support for free code coverage tools such as JaCoCo. Code coverage can be measured by unit tests or integration tests. You can even drill down to source code level - something I love to do :)

SONAR also integrates with new tools such as SQALE that have a formal approach for defining code quality in terms of maintainability, Testability, Reliability, Changeability, Efficiency, Security, Portability, Reusability, etc. Overall it is an invaluable tool to access Technical Debt.

Activation bar in UML Sequence Diagrams

2010-10-18T13:31:00.000+05:30

A lot of folks get confused between the "life-line" concept and "activation bar" concept in Sequence Diagrams.

The vertical lines drawn are called the lifeline of the object. When the object is no longer alive, then we can draw an 'X' at the bottom of the line. So why do we need an activation bar? I have seen a lot of architects choosing not to draw the activation bar to keep the diagrams simple.

The activation bar (a.k.a focus of control) represents the time the object is "active", i.e. doing some processing, computing something, waiting for a response from a sub-routine, etc. So it is possible to model multiple interactions in a single diagram - for e.g. a lifeline can have nultiple activation bars.

Good whitepaper on WSRP Portlets

2010-10-15T16:29:00.000+05:30

A friend of mine was confused on the concept of remote portlets and the WSRP protocol. I forwarded him this cool whitepaper that explains the concept of remote portlets in a simple and lucid language.

Some snippets from the whitepaper:

Remote portlets enable dynamic integration of business applications and information sources into portals.
This approach only works if all portlets are physically installed at the employee portal; the process of making new portlets available is tedious and expensive.

Instead of just providing raw data or single business functions that still require special rendering on the portal side, Web Services for Remote Portlets (WSRP) are presentation-oriented, interactive web services.
They are easy to aggregate and can be invoked through a common interface using generic portlet-independent code that is built into the portal. In addition, re-implementation of the presentation layer on each portal is avoided. The use of generic portlet proxies consuming all WSRP services conforming to the common interface eliminates the need to develop service-specific portlets to run on the portal.

The big difference between WSRP services and data-oriented web services is that they are presentation-oriented, interactive services that all have one common interface. This means that they simple plug into portal servers and do not require any service-specific code on the consuming portal.

Another point to consider is the comparison of remote portlets with mashups? Are they the same? Remote portlets and WSRP standards can be used to create mashups, but portlets also bring in the advantage of personalization and customization. Mashup's also need not just refer to aggregation of content. Wikipedia describes mashup as "A mashup is a website or Web 2.0 application that uses content from more than one source to create a completely new service. This is akin to transclusion." For e.g. tons of new applications built on top of Google Maps.

The case for IT Portfolio Rationalization

2010-09-22T18:26:00.000+05:30

Un-rationalized IT portfolio is a critical issue facing many organizations. There are a plethora of reasons why an IT portfolio becomes ‘bloated’ with an ever increasing inventory of applications running of multiple platforms. It could be because of mergers and acquisitions, lack of enterprise architecture standards and IT governance, legacy applications that need to undergo a technology refresh cycle, etc. As a result of this, a majority of the IT budget is spent on “keeping the lights on”, rather than investing on new strategic initiatives. This in turn results in poor business-IT alignment.

Continuous Portfolio Assessment and periodic Rationalization of IT assets is needed to lower the TCO of the IT portfolio and enable business agility. Portfolio assessment and optimization should be an integral part of the organizations EA group.

Any portfolio rationalization exercise would start with an assessment of the current state. Here pre-defined templates to capture the high level IT landscape would be beneficial. More comprehensive templates would be required to capture the details of each application. Information parameters such as application infrastructure, business process supported, TCO, pain areas, etc. would be captured.

The second step would be group applications into clusters. A ‘cluster’ is nothing but a group of applications that are similar in semantics - from a business perspective or a technology architecture perspective. For e.g. applications collaborating or orchestrated to fulfill a business process, a cluster of web applications running of the same technology stack - Websphere App Server on Solaris.

Each application cluster is then evaluated across various dimensions – such as Business value, Business functionality adherence, Current or Future Technology standards adherence, EA standards conformance, SLA conformance, TCO, etc. Based on the rating and the weightage given to each parameter, we would arrive at final scores. Based on the scoring, application clusters would be segmented into 4 categories – retain, enhance, replace, retire/sunset.

The final step is to define roadmaps for the transformation – a set of project streams/initiatives that would be required. A project prioritization framework would help in sequencing the selected initiatives. Pre-defined templates for project prioritization would come in handy here. Here again a metrics driven approach would work; wherein we evaluate the priority of the selected work streams based on parameters such as business impact, risk, technology skills, cost/investment required.

It is also very important to define a governance body to manage the rationalization program and sustain it. The governance body would have participation from both business and IT stakeholders.

The end deliverables of the rationalization exercise would be:
1. Matrix of applications segmented into 4 categories – retain, enhance, replace, retire/sunset
2. Define project streams (rationalization roadmaps) and project sequencing (prioritization) to meet the future state E A vision
3. Define opportunities/initiatives for business process optimization and reduction of TCO

Estimates and Scheduling

2010-08-19T17:50:00.000+05:30

Read an interesting article by Joel on software scheduling. Most organizations either use the use-case point estimation method or the function point estimation method.

But the best estimates come from experienced designers and developers - because you have done something similar in the past, you are more confident of the estimates you give.
But how can you bring in these past experiences of people into your estimation process. We would need to collect data on all past and present projects and extract metrics from them.

In the article, Joel talks about "evidence-based scheduling (EBS)" - i.e. base your future estimates on past experiences and developer productivity. It might be difficult to record the productivity of each and every developer, but we can create clusters for a developer group based on yrs of experience.

The most interesting point regarding EBS was that you do not have a single delivery date - you only have probabilities assigned to a range of dates. I always struggled selling this concept to program managers and customers. The point is that its impossible to predict the exact date of shipment at the start of the project. Based on fine granular tasks and past estimation data, we can use algorithms such as "Monte Carlo Simulation" to arrive at a range of dates each with a probability index assigned to it - similar to the image below.

Use of Generics in .NET webservices

2010-07-05T13:57:00.000+05:30

Recently, one of my team members was trying out an idea of using a generic container "response" object for all return types from the webservice. The advantage of doing so was that we would just have one generic "Response" object instead of many response objects for different operations - hence it would be easier for clients to extract data and consistently get other meta-data information in the Reponse object such as "Success/Error messsage", "Count", etc.
Given below is sample code showing the generic response data contract.
-------------------------------------------


[DataContract]
public class Response
{
[DataMember]
public T Data { get; set; }
[DataMember]
public List DataCollection { get; set; }
[DataMember]
public string ErrorMessage { get; set; }
}

[DataContract]
public class Customer
{
[DataMember]
public string Name { get; set; }
[DataMember]
public int Age { get; set; }
[DataMember]
public decimal ContactNo { get; set; }
}

[DataContract]
public class Vendor
{
[DataMember]
public string VendorName { get; set; }
[DataMember]
public int Rating { get; set; }
}

-------------------------------------------
As you can see, the actual return object (e.g. vendor, customer) is wrapped in a generic response object. Given below is code snippet of the webservice/WCF service.
-------------------------------------------
public class GenericService : IGenericService
{
public Response DoOtherWork()
{
Response res = new Response();
Vendor c = new Vendor();
c.VendorName = "Rocky"; c.Rating = 3;
res.Data = c;
res.ErrorMessage = "Big Errors";
return res;
}

public Response DoWork(int i)
{
Response res = new Response();
List lstcust = new List();
Customer c = new Customer();
c.Name = "Rocky"; c.Age = 26; c.ContactNo = 7688890909;
int j = 0;
Customer c1 = new Customer();
c1.Name = "dfgdfgRocky"; c1.Age = 26; c1.ContactNo = 7688890909;

lstcust.Add(c); lstcust.Add(c1);
res.DataCollection = lstcust;
res.ErrorMessage = "No Errors";
return res;
}
}
-------------------------------------------
It was interesting to see how the .NET framework handled this in the WSDL. What complex types are created in the WSDL? This was important because the WCF services could potentially be used by Java and Ruby clients.
Here we found a hitch - the WSDL schema had 2 complex types such as "ResponseOfCustomer24335423" and "ResponseOfVendor24r58sdf". This WSDL worked fine when we created stubs in Java by importing the WSDL. But only the random numbers appended at the end of the complex schema was a problem from naming standards and clarity point of view. You would not want your callers to keep wondering why such strange names were given.
But here again, we found an easy workaround. While defining the data-contract, we have the option of specifying the name of the complex type it should appear as in WSDL.
So we just added one key value pair to the Data Contract attribute as follows.
-------------------------------------------
[DataContract(Name = "ResponseOf{0}")]
public class Response
{
[DataMember]
public T Data { get; set; }
[DataMember]
public List DataCollection { get; set; }
[DataMember]
public string ErrorMessage { get; set; }
}
-------------------------------------------
This was cool as could have a placeholder "{0}" for the type passed to the generic container class.
Once we did this, the complex types in the WSDL has user friendly names such as "ResponseOfCustomer" and "ResponseOfVendor".

What rules to put in a BRE?

2010-06-23T19:02:00.000+05:30

Most organizations today understand the value of externalizing rules from application code by using a Business Rules Engine. The agility and flexibility derived from BREs allows for quicker roll out of new business rules and better business IT alignment.

But how do we determine what business rules should go into the BRE? Not all business logic needs to go into a BRE. The following methodology can be used to arrive at a good set of externalized business rules.

Prepare a set of variable data/rules for the application. Ask questions such as: What changes? Frequency of change? Who/What triggers the change?

Assign importance of agility to each variable data. What changes are time-critical? Business Impact if a business rule is not changed in time? Business benefit if we can change the variable data quickly.

Check if business users can make changes to the variable data. For e.g. if-else conditions can be handled by a business analyst, but change in complex statistical formulas may not be easy.

If we go through the above thought process, we can arrive at a logical set of variable data and rules that should be externalized by the system.

CLUE database in Insurance

2010-05-18T14:04:00.000+05:30

The CLUE (Comprehensive Loss Underwriting Exchange) is a database of insurance losses and claims that was created and is maintained by ChoicePoint. ChoicePoint was taken over by LexisNexis some time back. LexisNexis C.L.U.E. Property reports contain up to seven years of personal property claims matching the search criteria submitted by the inquiring insurance company. So it's the equivalent of credit reports in the insurance industry. ChoicePoint also maintains a database for personal auto claims history.

When we apply for a homeowner's or auto insurance policy, the insurance company orders and review our CLUE report. A CLUE report is a document containing your personal information, as well as data regarding past property claims that have been paid under your previous insurance policies. CLUE reports includes the type of property (vehicle or home), dates of losses, loss descriptions and amounts paid.

Insurance companies use information on CLUE reports to evaluate homeowner's and auto insurance applicants for acceptability. Information can also be used to determine your policy premiums. CLUE reports are used almost exclusively to underwrite and rate new policies. Most insurers renewing existing policies do not access CLUE reports at renewal, largely because they already have loss histories for these properties in their own database. Only insurance companies that subscribe to CLUE can submit loss data and access CLUE reports.

A software program called Quick Connect has been designed by ChoicePoint for sending and receiving information electronically to and from ChoicePoint.

Conceptual model for SOA

2010-05-13T17:40:00.001+05:30

Just read a very interesting blog post from Nick Malik, where in he provides a Common Conceptual Model for SOA. I was bowled over by the simplicity of the language and diagram. So easy to explain to a fellow architect embarking on a design assignment based on SOA.

The most important things to work on in a SOA project:

Canonical Data Model

Canonical Message Model (Shared Message Contract)

Service Interfaces

Business Event Taxanomy

I have seen that if you get these fundamentals right during the design process, then building scalable SOA based systems is a lot easier.

Why Enterprise Architecture?

2010-05-12T19:03:00.000+05:30

Recently a friend of mine asked me to explain in simple terms the benefit of having an Enterprise Architecture for an organization. The challenge was to translate the message in non-technical non-complex terms without any jargon. He apparently had read an EA white-paper by a big analyst firm and had his head spinning :) . I won’t blame him – this is a problem with ‘complex definitions’.

As architects, the most important challenge that we solve day-in and day-out is in reducing and managing IT complexity. Yet, it’s an irony that many architects/analysts use ‘complex long phrases’ to describe even the simplest of concepts.

Enterprise architecture is required to give a holistic view of the entire IT landscape of the organization. This enables the organization to view its current state and plan for the future state of its IT operations. EA is very useful to plan and prioritize your IT budgets in accordance with your important business priorities. It serves as a powerful communication tool between the business and IT teams and helps align new IT investments appropriately. Organizations have also started using EA as a tool for risk management and SWOT analysis.

Thus having an Enterprise Architecture Framework helps us in visualizing the ‘big picture’ and the relationship between different domains. It also leads to better governance and low maintenance cost enabled by enforcement of technology standards and architecture guidelines.

Quite often, EA is compared to ‘city planning’. In city planning, you look at the big picture and establish zones for specific purposes. It lays down the guidelines and best practices for buildings, roads, water supply, hospitals, etc. The design of each building is analogous to ‘Solution Architecture’. The special skill of designing a kitchen is analogous to ‘technical/application architecture’.

So to summarize, EA has the following advantages:

Helps build a common understanding of the future IT direction of the enterprise.
Provide clear mapping of the business processes and the IT systems supporting it. Provides visibility into how the business processes enable the mission of the enterprise.
Improves interoperability and integration by defining enterprise canonical message models and data models. Also integration standards and guidelines - for e.g. SOA, MOM, etc.
Enables organizational agility - If we need to respond to a business change, what is the impact on the IT systems, etc.
Less cost due to technical homogenity that is easier to support and maintain.
Powerful tool for communicating risk and change to all stakeholders in the enterprise.

There are a number of frameworks for defining EA – popular among them are the Zachman Framework and TOGAF. I had also blogged earlier about tools used for creating an EA.

OpenSource EA tools

2010-05-12T18:14:00.000+05:30

Planning to evaluate a couple of open source Enteprise Architecture tools that are becoming popular. IBM System Architect was the de-facto commercial EA tool in the market and we were actively using it for our customers.

First is "The Essential Project" that was launched last year. It is based on the Protégé ontology tool and allows you to create a meta-model for your EA. It has a EA repository and reporting tools.

Second is a tool called "iteraplan". This tool is more popular in Europe and also has an Enterprise Edition that is free for use, but without the source code.

Creating Dynamic classes in Java

2010-05-11T17:48:00.001+05:30

Earlier I had blogged about the ability to create new classes from scratch in .NET.
I was looking for something similar to Reflection.Emit() in Java. Found a few examples on the web that use the Java compiler to compile source code dynamically and use Reflection to execute the methods.
http://www.javaworld.com/javaworld/jw-06-2006/jw-0612-dynamic.html?page=5
http://www.rgagnon.com/javadetails/java-0039.html

Other option is to use byte-code manipulation libraries such as BCEL, ASM, etc. A good list of open source byte code libraries are given at:
http://www.java-opensource.com/open-source/bytecode-libraries.html

Payment Gateways

2010-05-11T14:20:00.000+05:30

We were exploring the various options for selecting a payment gateway for a new business opportunity. Found this site that has a good explanation of payment gateways and compares the popular payment gateways available in the market.
http://www.bestpaymentgateways.com/
The site has a brief explanation on each payment gateway. The following gateways are explained.

Amazon Payment Gateway
Authorize.net
PayPal PayFlow Pro
VeriSign
Google Checkout
Moneybookers
VerePay
WorldPay

Parsing and reading text from PDF files

2010-05-10T18:00:00.001+05:30

One of my development teams was looking for a PDF parsing library. They essentially wanted to search and extract data from PDF files. At first, I thought that OCR is the only way to achieve this, but there are libraries available to help us :)

PDFBox : This seems to be most popular library for extracting text out of PDF files. This is a Java library, but also has a .NET wrapper around it using iKVM.NET
Simple examples using this library can be found here and here.

iText & iTextSharp : These libraries are very popular for PDF generation and can also be used for extracting text from PDF files. Sample example can be found here.

I have heard that OpenOffice.org also provides you with a Java API that can be used to create and manipulate PDF files, but have not tried it yet.

Datawarehouse vs Datamart

2010-05-06T17:54:00.000+05:30

The debate between creating a datamart or sourcing data from the datawarehouse springs up from time to time across organizations. Found some good links on the web on this great debate:
information-management
exforsys.com
opensourceanalytics.com

Performance issues with an in-memory Datatable select clauses

2010-04-28T21:18:00.000+05:30

Very often, we cache .NET Datatable objects in memory. This could be for lookup tables or for other frequently accessed data. The following article on MSDN points out that using 'select' on a Datatable can be very very slow.

http://msdn.microsoft.com/en-us/library/dd364983.aspx

Snippet from the article:
"Select takes arbitrary criteria and returns an array of DataRows. Essentially, DataTable.Select has to walk the entire table and compare every record to the criteria that you passed in. "

The performance stats for LINQ in the article look very impressive. Looks like Microsoft has made a lot of optimizations on LINQ. It would be great to know (see source code) of what happens behind the scenes and why LINQ is so damn fast!

.NET PDF creation tools

2010-04-28T10:03:00.000+05:30

Some time back, I had blogged about the opensource tools available in Java and .NET for PDF creation. Recently came across another commerical library called "Dynamic PDF" for PDF creation. This library has a Java and .NET version of the API.
The library is easy to use and is very robust. It handles 'over-flowing' tables and text areas in PDF files very gracefully.
Another interesting library that Dynamic PDF sells is the DynamicPDF™ PrintManager for .NET . This library makes it so simple to print PDFs and also has callback handlers for error messages from printers - all pure managed code. Now that's what I like :)

Two opensource .NET PDF generation tools are PDFSharp and iTextSharp.

XSD to XML and vice versa

2010-04-28T00:55:00.000+05:30

I was looking for a quick and free tool that could convert XSD schema files to sample XML files and also generate a XSD file from a sample XML file. Commerical tools such as XML Spy and OxygenXML were powerful tools that provided these features, but I was looking for a free one.
First I checkout the open source Java IDEs.

Netbeans had a beautiful editor to visualize and edit a XSD schema in a graphical tree structure. But unfortunately it did not have the ability to generate sample XML file or vice versa.
Eclipse too had a visual editor and allowed visual editing of schema elements. It could generate a sample XML file from a XSD file, but had no option for the reverse; i.e. generating a schema file from a sample XML.
VS 2008 SP1 had both the options - coversion between XML and XSD. For schema files, right click on a node in XSD Explorer view and select "Generate XML". For XML files, select "Tools -> Generate schema" to create the XSD file. Both these operations are very quick in Visual Studio.
VS 2010 has extensive support for XML tooling. You have 3 different views for schema designing that should suffice for most complex schema definition.

Besides these free tools, there are other command line tools that can be used. For e.g.

XSD.exe tool can be schema to XML and vice versa transformation.

There was another .NET tool that I found on MSDN for generating XML documents from XSD.

Another cool Java desktop tool that supported these features was XMLSpear.

Updatable Views Vs ORM tools (ADO.NET Entity Framework)

2010-04-26T06:27:00.000+05:30

Recently during one of our design brainstorming discussions, our database developer was having a hard time trying to understand the benefit of using a ORM tool such as NHibernate or ADO.NET Entity framework. For our example scenario, he said that he could easily create database views that would serve the same purpose - abstracting the logical schema and helping the developer avoid complex joins for simple business queries.

For the use-case we were working on, his statement made sense. But there are a lot of cases and reasons why database views would not be the best choice. Here is a snippet from a MSDN article.

An obvious question at this point would be why not just use traditional database views for this. While database views can abstract many of the mappings, often that solution won't work for several process and functional reasons: (a) many of the views are simply too complex to be generated and maintained by developers in a cost-effective way, even for some simple conceptual to logical mappings, (b) the classes of views that have the property of being automatically updatable at the store are limited, and (c) databases for core-systems in medium and large companies are used by many central and departmental applications, and having each individual application create several views in the database would pollute the database schema and create significant maintenance workload for the database administrators. In addition, database views are limited to the expressivity of the relational model, and typically lack some of the more real-world concepts of the Entity Data Model, such as inheritance and complex types.

ADO.NET client-views work entirely on the client, so each application developer can create views that adapt the data to a shape that makes sense for each particular application without affecting the actual database or other applications. The class of updatable views supported in the Entity Framework is much broader than those supported by any relational store.

Profilers for .NET

2010-04-26T06:05:00.000+05:30

Way back, I had blogged about the profiling capabilities of VSTS 2008 and Numega DevPartner Studio.

Came across this site that lists down the most of the .NET profiling tools available in the market.
http://sharptoolbox.com/categories/profilers-debuggers

I have had good experience with .NET memory profiler and ANTs profiler.
Both of them allows you to collect stats on elapsed time for the entire call-stack and also allow you to walk the heap and check the effect of GC runs across heap generations.

Referential Integrity across databases in SQL Server

2010-04-21T05:58:00.001+05:30

Having worked on Oracle databases for a long time, I was quite comfortable with the idea of maintaining RI across different schemas in same database. For e.g. Ur database may have a schema containing master tables and your transaction schema references the lookup data in the master tables.
But to my suprise, prior to SQLServer 2005, it was not possible to have separate schemas in SQL Server. This meant that there was no easy way to maintain RI across databases in SQLServer. The only options were using messy triggers or a CHECK constraint with a UDF (user defined function)

Maintaining RI across databases is still not possible in SQLServer, but SQLServer 2005/2008 have added the concept of schemas in databases; i.e. it is possible to have multiple schemas in each database and each schema offers the same logical separation that a separate database would. We can also configure the schema to be located on a separate filegroup or a separate disk, thus maximizing performance. In SQLServer 2005/2008, it is possible to have RI constraints across schemas. I think this is the best design approach to take. If it is not possible to have your master data in the same database, then the second best approach is to use replication to get the master data into your database.

What is Micro-Architecture in software design?

2010-04-06T06:28:00.000+05:30

Recently I came across the term 'Micro-Architecture' on a number of pages on Sun's site. Sun's engineers have an interesting definition for this term. They refer to Micro-Architecture as a composition of a set of patterns that can be used to realize a subsystem.

In the book - "Core J2EE patterns", Micro-Architecture is described as below.

We define micro-architecture as a set of patterns used together to realize parts of a system or subsystem. We view a micro-architecture as a building block for piecing together well-known, cohesive portions of an overall architecture. A micro-architecture is used to solve a coarser-grained, higher-level problem that cannot be solved by a single pattern. Thus, a micro-architecture represents a higher level of abstraction than the individual patterns.

Critical Success Factors for SOA

2010-03-12T05:35:00.000+05:30

A lot of organizations are embarking on the SOA journey and it is important to understand the factors that would make the SOA initiative successful. Based on our experience in SOA consulting, we have learned the following.

1. Adopt an evolutionary approach to SOA: It’s impossible to sustain a big-bang approach to SOA at the enterprise level and even more difficult to make a business case for the large investment. The best approach is to have an enterprise level SOA governance and adopt an incremental and iterative approach. This approach is called the ‘hybrid’ or ‘middle-out’ approach.
After gaining success in early iterations, it becomes easier to make a business case for further investment or create a self funding model. It is a great idea to focus on immediate business pain points during the initial iterations, such as automation, reducing duplicity, etc.

2. Ensure top management support: This is very crucial because SOA strategies would require many departments/LOB’s to come together and function as one unit. And unless you have the top level executive sponsorship to support the SOA initiative, you could get entangled in a mesh of organization politics.
Also it is important to address the cultural issues and involve all layers in the organization during decision making. If we are able to define WIIFM (What’s in it for me?) for each important business stakeholder, then the changes of SOA success are very high.

3. Implement SOA Governance from the start: SOA governance is a very board area and covers many aspects. SOA governance includes 2 dimensions – the methodology/process that is established and the tools and technologies used for governance.
Service Life Cycle Management, Service monitoring and Service SLA management are all important points of SOA governance. In fact, if the organization has a EA practice, then SOA governance should be part of it.

Ruminating on Requirements Elicitation

2010-02-24T12:47:00.000+05:30

Last night, I was ruminating on the lessons learned during my experience in business architecture, business analysis and requirements elicitation. I think at the broad level, there are 2 kinds of requirements.

The first kind is - Where we know, what we don't know: These requirements are easy to capture since we have an idea about the pain areas of the customer, we know what is the vision of the project, what is being planned to be done, etc. Hence we can ask appropriate questions, dig deeper and after a few iterations of interviews can get a pretty good understanding of the requirements.

The second kind of requirements are - Where we don't know, what we don't know !
Digging out these requirements is the most challenging task - because we never ask the right questions about things that don't know, our past memory/experiences make implicit assumptions on certain things. Such kind of requirements result in last minute suprises and everyone wonders why these points were never discussed or brought out !!!

How do we elicit these requirements? I think the following techniques during requirements elicitaiton would help:
1. Question each and every assumption - both implicit and explicit. Assumptions may not hold true in every scenario. Check if the assumption is valid for the current scenario.
2. Ask 'stupid' questions - Yes, this works !!!...I have seen that often, analyst's refrain from asking questions because they are self-conscious, are afraid of being ridiculed for lack of knowledge in a domain area, etc. i.e. they are afraid of sounding 'stupid'. But clarifying even the simplest doubt could open up a plethora of new requirements.

Migrating applications across platforms

2010-02-12T14:50:00.000+05:30

We often require to migrate applications from one platform (OS/hardware) to an other platform - either as part of a portfolio rationalization program or because the current platform is reaching its EOL. For e.g. migrating Java apps from Solaris/SPARC to Linux/IBM Power, etc.
During such migrations, it is essential to have a well documented plan covering all the aspects, so that we do not miss anything. Listed below is a brief set of points that must be included in the brainstorming discussions during the planning phase.

Migration of the Runtime environment: For e.g. miration of the Java runtime from Solaris to Linux. There are many JRE's available on Linux and we need to select the appropriate one.
Migration of the Application Infrastructure: The application may require a webserver, application server, a Message queue, etc. These infrastructure components would need to be ported to the target platform or other alternatives selected.
Migration of application code: Does the code need to be recompiled/linked on the target platform? Was the application using any platform specific API for performance?
Migration of application configuration: What configuration needs to change on the target platform? e.g. file paths, endian-ness, max-memory settings, thread pool settings, etc.
Migration of development environment: Would it be required to migrate the development environment? Will the same IDE work? IDE-Runtime integration issues, etc.
Migration of Build and Deployment process: Need to change build scripts - ANT, Maven, Shell scripts, etc.

Also it makes sense to leverage the expertise of the target platform vendor during the migration exercise. For e.g. IBM, HP, SUN, RedHat, Novell, etc. all have guidelines, recommendations and migration kits for migrating from a target platform to their platform.

What is longitudnal data?

2009-12-15T19:01:00.000+05:30

A dataset is considered to be longitudnal if it tracks the same kind of information at multiple points of time. For e.g. the marks of students over multiple years, patient health records over a period of time, etc.

The most important advantage of longitudnal data is that we can measure change and the effect of various factors over the data-point time values. For e.g. what is the effect a particular drug had on a cancer patient? The effect of different teachers on a student?

So essentially, longitudnal data helps in establishing cause-n-effect relationships. Longitudnal data stores are also being used for predictive modeling and other areas. Longitudnal data stores are very popular in the Life Sciences and Healthcare industry.
I am interesting in learning the best practices for creating and optimizing a data-model for longitudnal data stores.

Difference between biostatistics and bioinformatics

2009-12-15T13:30:00.000+05:30

Working in the healthcare domain, I often come across the terms - biostatistics and bioinformatics and wondered as to what were the differences between the two branches of studies. A quick googling revealed the following:
The term Biostatistics is a combination of the words biology and statistics. So it essentially it is the application of statistics to biology. The science of biostatistics encompasses the design of biological experiments, especially in medicine and agriculture; the collection, summarization, and analysis of data from those experiments; and the interpretation of, and inference from, the results.
Bioinformatics is the application of information technology and computer science to the field of molecular biology. Its primary use has been in genomics and genetics, particularly in those areas of genomics involving large-scale DNA sequencing. Bioinformatics now entails the creation and advancement of databases, algorithms, computational and statistical techniques, and theory to solve formal and practical problems arising from the management and analysis of biological data.
Bioinformatics focuses on applying computationally intensive techniques (e.g., pattern recognition, data mining, machine learning algorithms, and visualization) to understand biological processes.
More information can be found on Wikipedia at:
http://en.wikipedia.org/wiki/Bioinformatics
http://en.wikipedia.org/wiki/Biostatistics

Mapping UI controls to entitlements/operations

2009-11-19T14:09:00.000+05:30

In RBAC, we often need to enable/disable a UI control based on the users role and entitlements. Most programmers write code for mapping a UI control with the operation name, i.e. if 'submitRecord' is not allowed for the user, then disable or hide the button.

Recently I came across a neat way to handle this using attributes in C#.NET. This article describes the use of attributes to specify the 'operationName' and the 'property-value' to be set on the control when we check for entitlements.
Example code snippet:
[YourCompany.Authorization("EditSalary", "ReadOnly", true)]
private System.Windows.Forms.TextBox Salary;

Though the example is that of a .NET program, the same concept can easily be applied in Java 5, which supports annotations. Annotations in Java are roughly equivalent to Attributes in .NET.

XSLT vs XQuery for XML transformations

2009-11-18T19:47:00.000+05:30

During the early years in my career, I had always used XSLT for transformation purposes. Recently I have seen that a lot of applications and products are using XQuery for transformations. For e.g. BEA Aqualogic ESB and Apache Service Mix both use XQuery for message transformation capabilities.

Although XSLT is more powerful than XQuery for transformations, it is much simpler to learn XQuery. I remember the tough learning curve I had to go through when learning XSL, but I could grasp XQuery basics in a couple of hours. Both XSLT and XQuery use XPath for quering XML.

Found this interesting discussion on O'Reilly site that compares the two technologies. David P. in the discussion shares some interesting views on the design philosophy behind the two technologies - XQuery is a MUST UNDERSTAND language where as XSLT is a MAY UNDERSTAND language, i.e. it is more adaptive and template driven. XSLT is untyped; conversions between nodes and strings and numbers are handled pretty much transparently. XQuery is a typed language which uses the types defined by XML Schema. XQuery will complain when it gets input that isn't of the appropriate type. XQuery is better for highly-structured data, XSLT for loosely-structured documents.

We were building a transformation engine to convert between the various ACORD formats. The Acord committee has ensured that any new version of ACORD only 'adds' new elements to the current schema. No elements are deleted or the schema changed for backwards compatibility.
Hence for these transformations, XQuery fitted the bill. We also did not have to create a Canonical Data Model as direct transfortions were possible due to the above mentioned restriction. Hence if the tool supported 15 ACORD versions, only 15 XQuery files were required.

Other links on the same subject:
http://www.xml.com/lpt/a/1549
http://xmlhack.com/read.php?item=1080
http://www.xmlplease.com/xquery-xslt
http://www.oreillynet.com

Use of Constants in code

2009-10-05T17:06:00.000+05:30

Recently there was a big debate during a code reveiw session on the use of constants. The developers had used constants for the following purposes:

1. Each and every message key used in the i18N application was declared as a constant. The application contained around 3000 message keys and hence the same number of constants.

2. Each and every database column name was declared as a constant. There were around 5000 column names and still counting..

Does it make sense to have such a huge number of constants in any application? IMHO, common sense should prevail. Message keys just don't need to be declared as constants. We already have one level of indirection - why add one more?
Reg. database column names, I have mixed opinions. If a column is being used in multiple classes, does it make sense to declare it as a global constant? Maybe declaring a class as 'Table Name' and its members as 'Column Name' would be a good idea? But if you have a large number of tables and columns, it would become very tedious to even create these constants file.

I found a few tools on the net that can automate the creation of these constants files.

- CodeProject

- MobzySystems

- ConstTool

Cool Cool JS toolbox

2009-09-25T13:30:00.001+05:30

Just found this good website having tons of cool JavaScript tools - free for commercial use too.
http://www.dynamicdrive.com/
Check out the dynamic menus - always needed in any web-site development.

Should we call Dispose() on a Dataset?

2009-09-16T16:17:00.000+05:30

During one of my code reviews, I saw that the development team had called 'Dispose()' on all the datasets used in the application. I knew that the Dataset was a disconnected managed object and could not understand what the dispose method would actually be doing? Dispose() is typically called to release unmanaged resources such as file-pointers, streams, socket connections, etc. In most cases, such classes also expose a Close() method that is more appropriate.

Discussions with the developers revealed that FxCop also throws an error when Dispose() is not called on Datasets.
Further investigation revleaded that the Dataset exposes the Dispose() method as a side effect of inheritance. The Dataset class inherits from the MarshalByValueComponent which implements the IDisposable interface because it is a component. The method is not overridden in the System.Data.Dataset class and the default implementation is in the MarshalByValueComponent class. The default implementation just removes the component from the parent container it is in. But in case of Dataset, there are no parent containers and hence the Dispose() method does nothing useful.

Conclusion: It is not necessary to call Dispose() on Datasets. Developers can safely ignore the FxCop warnings too :)

Ruminating over XML Schema concepts

2009-09-16T15:43:00.001+05:30

In a recent discussion with my team members, I sensed a lot of confusion over basic concepts on XML Schemas and namespaces - especially over targetNamespace, DefaultNamespace, elements vs types, etc. Well first, lets get the fundamentals right.

A XML schema defines and declares types (complex and simple) and elements. All elements have a type. If the type has not been specified, it defaults to xsd:anyType. It's easy to understand this by drawing an anology between a class and an object. New elements can be created by referencing existing elements.

In a WSDL message description, a WSDL part may reference an element or a type. If the SOAP binding specifies style="document", then the WSDL part must reference an element. If the SOAP binding specifies style="rpc", then the WSDL part must reference a type.
Now coming to namespaces, users typically get confused over the difference between a targetNamespace and defaultNamespace. The 'targetNamespace' attribute is typically used in schema files to identify and select the namespace into which new elements that are defined are created in. It is the namespace an instance is going to use to access the types it declares. For e.g. An XML document may use this schema as the default schema. Default schema is defined simply by using 'xmlns=' without a prefix.
It is important to remember that XML schema is itself an XML document. Hence a schema can contain a namespace attribute and also a targetNamespace attribute. Typically they are the same.

Jotting down links where more explanation is given:
http://www.coderanch.com/
http://www.oracle.com/technology/

Creating a self-signed certificate

2009-06-26T17:15:00.000+05:30

Jotting down the quick commands in .NET and JDK frameworks that can be used to create a self signed certificate.

In the .NET framework, open the Visual Studio command prompt and type the following:

makecert -r -pe -n "CN=www.yourserver.com" -b 01/01/2000 -e 01/01/2036 -eku 1.3.6.1.5.5.7.3.1 -ss my -sr localMachine -sky exchange -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12

Just replace the CN with the name or IP of your server. The certificate would be created in the default personal store on Windows. Go to MMC and add the 'certificates' snap-in.

In JDK first use the keytool utility to generate a certificate in the keystore.

keytool -genkey -alias myalias -keystore .keystore

You would be prompted to enter the CN and other details. Once the cert is stored in the keystore, it can be exported as a file.

keytool -storepass password -alias myalias -keystore .keystore -export -rfc -file outfilename.cer

.NET Web Methods Tips and Tricks

2009-06-11T02:11:00.000+05:30

It is possible to pass a parameter by reference to a Web Method? I thought this does not make sense, but behind the covers the .NET webservice proxy and runtime make this possible.

A detailed example with sample source code can be found here.

Also given in the above link is an example of supporting polymorphism in Web Methods.

Open Source Application Management Software

2009-06-09T18:23:00.001+05:30

We needed a simple application mangement tool for monitoring our Java applications running on Tomcat servers. My team evaluated the various free/open-source options and gave a demo of the following 2 tools that proved to be interesting.

1. ManageEngine Applications Manager - This is free for use (upto 5 monitors). Very user friendly and professional look and feel.

2. JManage - (open source. Good support for Java console applications)

Passing large .NET datasets across layers

2009-06-09T14:51:00.000+05:30

If your .NET webservice is returning datasets, then you might face performance problems with large payloads consuming the network bandwidth. There are a few strategies around this problem.

1. Convert the dataset into a byte[] using the Binary Formatter. Compress this byte[] using the Deflate classes in .NET. On the client side, the reverse process needs to be done.

Peter Bromberg has written a nice article explaining the details of this strategy.

2. Convert the dataset into a XML string and compress the XML string. This discussion forum thread contains sample source code for the same.

Migration Factory

2009-06-08T16:06:00.000+05:30

Recently, I have been hearing a lot of buzz around the term - "Migration Factory". A lot of IT service companies have jumped on this buzzword and offer conversion services based on the Migration Factory model.

So what exactly is a Migration Factory? A Migration Factory is a methodology that endeavours to adopt a factory model for churning out migrated components. Essentially the migration landscape is segmented into various well-defined tasks and each segmented task is assigned to dedicated teams having specialized skills in that area. An analogy to a vehicle manufacturing factory would give a fair idea of what the methodology tries to achieve. So this essentially is an assembly line methodology with phases such as planning, assessment, conversion, implementation, etc.

The key drivers in the Migration Factory approach are the reuse of patterns, models, tools gained from past experiences. Toolkits for automation of activities such as code converstion, automated regression testing, etc. form a important part of the Factory model.

In my personal experience, I have observed that no migration project is the same. Each project brings its own technology and business challenges, but we can reuse the best practices and other tools developed in previous engagements.

Data compression in .NET

2009-05-16T16:00:00.000+05:30

In .NET, developers often have to choose between 2 options for compression/decompression - GZipStream and DeflateStream. What's interesting to note is that GZip uses the same 'deflate' algorithm as DeflateStream; but in addition also supports CRC checks and has additional headers to store metadata such as version nos, original file name, timestamps, etc.

So GZip is actually a data format and multiple files can be compressed into a single archive. You can open a file written to by the GZipStream using a GZip decompression utility such as WinRAR on Windows, gunzip on Linux, etc.

Sample example code can be found at MSDN. There is also a sample solution that allows working with multiple files using GZip compression. Both these classes only support max 4GB as the stream length.

Interoperability when using datasets in .NET webservices

2009-05-15T18:00:00.001+05:30

Datasets are powerful data containers that are very popular in the .NET world. Quite often, a lot of .NET webservices return datasets in their web methods. This is ok, as long as we are sure that all the clients are also on the .NET platform.

But if we want our services to be interoperable with other platforms (e.g. Java), then passing datasets back to webservice clients would not work. This is because a dataset is a generic container and is populated at runtime with data. Hence at design time, it is not possible to define the dataset datatype with a schema. Also when a dataset is serialized to XML, then the default .NET SOAP serializer adds .NET specific attributes to the XML schema. e.g. isDataset=true, Diffgram, etc.

Typed datasets do have a schema backing them, but here again there is a hack required to make typed dataset accessible to a Java program. The hack is to return a xml string or a XML Node instead of the typed dataset. We would also have to change the autogenerated WSDL and add the typed dataset schema to it. A lot of trouble for using datasets in a heterogenous environment :(

For hassle free interop, the best option is to go for pure schema driven complex types that can be mapped to any language.

More information is available at the following links:

http://msdn.microsoft.com/en-us/magazine/cc188755.aspx

http://msdn.microsoft.com/en-us/library/ms998428.aspx

http://www.cnblogs.com/rickel/articles/157935.html

XML serialization tips in .NET

2009-05-15T15:18:00.000+05:30

Today, my team ran across a strange issue. Some of the properties on a .NET class were not getting serialized to XML when used in webservices. There was no error message or any other warning.

Closer examination revealed that the properties not getting serialized were 'read-only' properties; i.e. they had getters but no setters. The default XML Serializer in .NET calls the property getters/setters and hence if a read-only property is present, then it is not serialized.

Once we added setters to the properties, XML serialization worked fine.

There is another option. We can implement the IXmlSerializable interface and write custom serialization code. But this could be very tedious.

Java to .NET conversion tools

2009-05-13T13:06:00.000+05:30

Today one of our teams was looking for a Java to .NET conversion tool. They had developed resuable components in Java and the client wanted similar components in their .NET environment.

Microsoft used to ship a module called JLCA (Java Language Conversion Assistant) that enables us to covert Java code to .NET code. This tool was shipped as part of VS 2005, but unfortunately is no longer shipped with VS 2008. Please read Microsoft's statement here regarding JLCA.

I had a quick look at the tool by launching it from VS 2005 and selecting a few Java util libraries. The results looked good, but we need to understand that 100% conversion is not possible. The tool puts in a lot of comments in the generated .NET code, so all errors and warning are maked in the generated code. Also a cool HTML report is generated that mentions the errors and warnings. I could find tons of errros/warnings, but most of them fell into expected categories such as no direct mapping between Java and .Net classses, etc.

ArtinSoft offers an enhanced version of the JLCA tool called JLCA Companion. I have not evaluated this, but the documentations states interesting features for customization.

To summarize, I believe such migration tools are useful to jumpstart the process and have a foundation ready to work on.

Async calls in Java and .NET webservices

2009-05-08T13:49:00.000+05:30

Over the past few weeks, the Architecture Support Group that I head at my organization, received quite a few queries on making asynchronous web service calls in a SOA environment. So decided to blog about the various options at hand.

To make asych webservice calls in .NET, the following programming models are available. Please visit the links for furthur information.

1. Using Asychronous Callback delegates.

2. Using Event Based Asych methods

3. Fire and Forget mechanism: Here we can decorate the server side webmethod with the 'OneWay' attribute.

On the Java side, the popular Axis-2 framework supports asych web services calls out of the box, by generating call back handlers in the webservice proxy.

The WS-Addressing specification is also trying to bring in a standard for defining different MEP (Message Exchange Patterns).

Smart Client Applications

2009-04-30T14:10:00.000+05:30

Just read this blog entry about Smart Client technology and an architect's dilemma to choose an appropriate client UI technology. I had blogged about Smart Clients earlier here.

In the blog posts, Philip discusses the scenarios where-in a thin web-based client makes sense. For SmartClient, he says that 'offline' applications are the best fit where it makes sense to use a thick client. I feel there is another strong reason why one would want to select Smart Client UI - a rich user experience. Even with cutting edge Ajax solutions, it is impossible to get the same look and feel of a desktop application over a browser. We also have powerful WinForms/SmartClient third-party controls such as those from Infragistics that enable you to slice-n-dice thousands of records in a grid. Such raw power is not available over a web interface.

Other important point raised by Philip was the security aspect of Smart Client applications. How would we protect the client code and data from being compromised? Possible solutions:

- Encrypt all data
- Obfuscate the dlls

- Resolve data concurrency errors and database replication errors.

Global exception handler in Win Forms and Smart Client

2009-03-15T18:02:00.000+05:30

I always wondered if it is possible to add a global exception handler for a Win Form application. This global exception handler would handle all exceptions that have not been caught in the UI forms. In web forms, we have the onError event handler in Global.asax, but I was not sure if Windows forms had a similar functionality.

I got the answer from Rich Newman's blog here:

http://richnewman.wordpress.com/2007/04/08/top-level-exception-handling-in-windows-forms-applications-part-1/

Another option I had tried out was using the Policy Injection Block to capture all unhandled exceptions. But the caveat here is that all objects need to be proxied, which may be tedious to do.

Priciples of SOA

2009-03-02T10:26:00.001+05:30

Was reading "Thomas Erl's book - Priciples of Service Design" over the weekend.

It's an interesting book with a lucid language and simple to understand examples.

Jotting down some of the design principles for SOA:

Services should be platform neutral and programming language neutral
Services should have a standard interface contract
Services should be loosely coupled
Design for coarse grained services
Service Abstraction - Hide information that is not required by clients. Hide underlying technology details. Promotes loose coupling.
Service Reusability - Position services as enterprise resources with agnostic functional context; i.e. the service can be used in other functional scenarios too. Always design the service in such a way, that it can be used beyond its original context.
Service Autonomy - Services need to have a high degree of control over their underlying runtime execution environment. The higher up a service is in a typical composition hierarchy, the less autonomy it tends to have due to dependencies on other composed services.
Service Statelesness - defer or delegate state to databases, rather than in memory.
Service Discovery - services can be discoved using standards such as UDDI.

.NET FileHelpers Library

2009-01-20T15:12:00.001+05:30

I came across this cool and simple .NET library that I bet would be useful to everyone - FileHelpers Library 2.0

Very often, we have tasks where we need to read comma delimited files or fixed lenth files and load them into the database. Though this talk is very straight forward, the FileHelpers Library actually makes this task a breeze. Also the code looks much more clean and maintanable.

Link: http://filehelpers.sourceforge.net/

API for reading/creating Excel files in Java and .NET

2009-01-20T14:19:00.000+05:30

Recently I was trying to find a good library for reading and writing Excel files.

In .NET, there were quite a few alternatives. Given below are 2 blog-links where the author has given a detailed analysis of the libraries.

http://blog.functionalfun.net/2008/06/reading-and-writing-excel-files-with.html

http://blog.functionalfun.net/2008/08/which-net-excel-io-component-should-i.html

For .NET, you also have the option of using the COM Excel library thats available if you have Excel(MS Office) installed on your box. But using the Excel COM Interop assembly has performance issues.

For Java, my friend pointed out that an excellent free open-source library is available at BIRT exchange. This library is called "e.Spreadsheet Excel API" and a cursory glance at the sample examples impressed me. The library is simple to use and completely free to use in your commercial applications too.

Zachman Framework

2008-12-10T14:40:00.000+05:30

It's been some while since I attended John Zachman's seminar in Mumbai...

Recently someone asked me what the Zachman framework was? I tried to explain to him in simple terms as follows:

The Zachman framework is a classification structure for expressing the basic elements of an enterprise architecture. It a powerful tool for organizing architectural artifacts (design documents, specs, models, etc.) based on context or perspective.

The Zachman Framework is a matrix formed by the intersection of two classifications.

The columns represent the primitive interrogatives - why, how, when, who, where and why. The rows represent six different prespectives, which relate to stakeholder groups (Planner, Owner, Designer, Builder, Implementer and Worker).

The intersecting cells of the Framework correspond to models which, if documented, can provide a holistic view of the enterprise.

To view the model, please view below links:

http://www.enterpriseunifiedprocess.com/images/zachmanFramework.gif

Updatable views using the 'INSTEAD OF' trigger

2008-11-15T03:33:00.000+05:30

All views are not updatable in a database - especially if the underlying query is a complex one. This article gives interesting examples of the challenges faced in updating views.

But latest versions of databases, support a INSTEAD OF trigger, that enables us to make all views updatable. INSTEAD OF triggers fire in place of the triggering action.

The links below show how the 'INSTEAD OF' trigger can be used in different databases:

Oracle: http://www.psoug.org/reference/instead_of_trigger.html

DB: http://www-128.ibm.com/developerworks/db2/library/

techarticle/0210rielau/0210rielau.html

SQLServer: http://msdn.microsoft.com/en-us/magazine/cc164047.aspx

Cold start Vs Warm start of programs

2008-11-11T23:13:00.000+05:30

We often notice that if we start a Windows Forms application the first time after a system reboot, then the start-up time is quite long. But if I close and open the win-form application again, then the application start-up time is reduced drastically. This is true for all programs - Java. .NET, VB... not just WinForms.

For e.g. open a 5 MB word doc right after reboot and then reopen it again. There would be a difference of 5x-10x in startup times. This difference between start-up times is called as Cold start and Warm start.

This happens because of 'disk-cache'. The 'disk-cache' is part of RAM that is dedicated to caching disk pages into memory. So the next time, a program is accessed, the disk cache is hit first and if the required program is found there, then it is loaded into the program space. Hence expensive disk access is eliminated and hence the start-up time is much faster. The pages remain in disk cache as long as possible and are removed based on a FIFO principle.

Operating systems have started using this technique to make programs appear faster.

For example superfetch service on windows vista (xp has prefetch) does this by putting your most frequent used applications into the ram, so when you launch them, they are actually launched not for the first, but second time.

Disabling startup programs

2008-11-04T19:06:00.000+05:30

In Windows XP, if you are getting annoyed by the number of programs that start-up when the OS loads, then there is an easy way to disable the programs we don't need.

Just type "msconfig" in the 'Run' window. A window titled 'System Configuration Utility' would come up. Navigate to the 'StartUp' tab and uncheck those programs that need not be started at boot time.

MVP Vs MVC pattern

2008-10-03T16:07:00.000+05:30

I was recently working on a .NET Smart Client application and came across the 'MVP' (Model View Presenter) design pattern. Having spend years developing applications on the MVC (Model View Controller) pattern (e.g. Struts, Webworks, ROR, etc.), I was intrigued to find out the difference between MVC and MVP (if any)
The core principle behind MVP and MVC remain the same - clean separation of concerns between the presentation tier and the business tier.
Some consider MVP to be a subset of MVC and there are tons of articles on the web giving differences between the two..I found most of the information muggy and more confusing. Finally I came across Phil Haack blog that explains it in simple and lucid words.
Snippets from his blog:

The two patterns are similar in that they both are concerned with separating concerns and they both contain Models and Views. Many consider the MVP pattern to simply be a variant of the MVC pattern. The key difference is suggested by the problem that the MVP pattern sought to solve with the MVC pattern. Who handles the user input?

With MVC, it’s always the controller’s responsibility to handle mouse and keyboard events. With MVP, GUI components themselves initially handle the user’s input, but delegate to the interpretation of that input to the presenter.

In modern GUI systems, GUI components themselves handle user input such as mouse movements and clicks, rather than some central controller. Thus MVP pattern is widely used in WinForms, .NET SmartClient Factory, etc.

In most web architectures, the MVC pattern is used (e.g. Struts, ASP.NET MVC, ROR, Webworks), whereas MVP pattern is mostly used in thick client architectures such as WinForms.

What the hell are all those processes running in the taskbar?

2008-08-04T09:19:00.000+05:30

Ever opened the TaskManager on windows and wondered why there are so processes running?
HowTo-Geek gives valuable information on some of the nagging questions:

1. juschedexe
2. ctfmonexe
3. rundll32
4. svchost

Comparison of iBatis, Hibernate and JPA

2008-07-16T10:05:00.000+05:30

As architects, we often need to decide on what OR mapping tool to use. I came across this article on JavaLobby that compares iBatis, Hibernate and JPA.

Snippets from the article:

iBATIS is best used when you need complete control of the SQL. It is also useful when the SQL queries need to be fine-tuned. iBATIS should not be used when you have full control over both the application and the database design, because in such cases the application could be modified to suit the database, or vice versa. In such situations, you could build a fully object-relational application, and other ORM tools are preferable. As iBATIS is more SQL-centric, it is generally referred to as inverted -- fully ORM tools generate SQL, whereas iBATIS uses SQL directly. iBATIS is also inappropriate for non-relational databases, because such databases do not support transactions and other key features that iBATIS uses.

Hibernate is best used to leverage end-to-end OR mapping. It provides a complete ORM solution, but leaves you no control over queries. Hibernate is an ideal solution for situations where you have complete control over both the application and the database design. In such cases you may modify the application to suit the database, or vice versa. In these cases you could use Hibernate to build a fully object-relational application. Hibernate is the best option for object-oriented programmers who are less familiar with SQL.

JPA should be used when you need a standard Java-based persistence solution. JPA supports inheritance and polymorphism, both features of object-oriented programming. The downside of JPA is that it requires a provider that implements it. These vendor-specific tools also provide certain other features that are not defined as part of the JPA specification. One such feature is support for caching, which is not clearly defined in JPA but is well supported by Hibernate, one of the most popular frameworks that implements JPA. Also, JPA is defined to work with relational databases only. If your persistence solution needs to be extended to other types of data stores, like XML databases, then JPA is not the answer to your persistence problem.

Conclusion:
iBATIS does not provide a complete ORM solution, and does not provide any direct mapping of objects and relational models. However, iBATIS provides you with complete control over queries. Hibernate provides a complete ORM solution, but offers you no control over the queries. Hibernate is very popular and a large and active community provides support for new users. JPA also provides a complete ORM solution, and provides support for object-oriented programming features like inheritance and polymorphism, but its performance depends on the persistence provider.

iRise visualization software

2008-07-06T11:50:00.000+05:30

Recently on request of one of our clients, we were asked to evaluate iRise V (isualization products.
A quick perusal of what the product offered made me smile. There are so many projects that fail because the end-product is not what the business required. Sometimes the business itself is not sure of what it wants or how the end product should look like.
It's estimated that 60% of projects fail because of misunderstanding of requirements.

Using iRise visualization products, business analysis can quickly build UI elements that have the same look-and-feel and navigation as the final end product.
More information at http://www.irise.com

P.S: Just got a mail from Tom Humbarger (iRise Project Manager) that an evaluation copy of iRise Studio PE is now available for free 30-day download from Download.com. Would highly recommend it for a try.

Open Source IDS, Firewall, VPN Gateways

2008-06-23T17:22:00.000+05:30

A few years back, if a SMB(small and medium business) shop wanted to install Firewalls, Network Intrusion Detection systems, VPN Gateways, etc. then it needed to shell out hundreds of dollars for commercial software from giants such as Cisco, Juniper etc.

But in the last few years, we have a slew of options available from the open-source world. I have been closely watching 2 products in this space:

1. Snort

Snort is an open source network intrusion prevention system, capable of performing real-time traffic analysis and packet logging on IP networks. It can perform protocol analysis, content searching/matching and can be used to detect a variety of attacks and probes, such as buffer overflows, stealth port scans, CGI attacks, SMB probes, OS fingerprinting attempts, and much more. Snort has three primary uses. It can be used as a straight packet sniffer like tcpdump, a packet logger (useful for network traffic debugging, etc), or as a full blown network intrusion prevention system.

2. Untangle

Untangle delivers an integrated family of applications that help you simplify and consolidate the network and security products you need, in one place at the network gateway. The most popular applications let businesses block spam, spyware, viruses, and phish, filter out inappropriate web content, control unwanted protocols like instant messaging, and provide remote access and support options to their employees

Today Snort has become the de-factor standard for IDS. Even Untangle uses Snort for its IDS application. I was impressed with the range of applications available on the Untangle Gateway Platform. It includes SPAM blocker, Web Filter, Firewall, IDS based on Snort, VPN Gateway based on OpenVPN, a patented attack blocker, etc. A must try-out product :)

File Watcher programs

2008-06-14T19:25:00.000+05:30

During the early years, we often had to write our own component to monitor file changes or changes to a directory. Recently a friend of mine introduced me to the FileSystemWatcher class in .NET. We needed such a solid component and were contemplating writing it ourselves.
I was impressed with the wide range of features available in the FileSystemWatcher class. Not only will it detect changes in files, but it can monitor folders and sub-folders too.
And it can monitor a wide range of attributes - not just whether a file exists or not. For e.g. it can monitor whether the file size has changed, renamed, deleted etc.

I started wondering why there was no equivalent component in the Java SDK. The answer lies in the fact that such kind of event-raising is not available on Unix platforms. So polling is the only option that works cross-platform. When we poll, we cannot detect directory changes like renaming or moving. Also polling for a directory and all sub-directories has a big performance hit!

There a few programs available in Java that a developer can use instead of reinventing the wheel. Here are a few links that provide File watcher programs in Java:
http://jpoller.sourceforge.net/
Java-File-Watcher
Jahia File watcher

VSTS 2008 has a memory profiler

2008-06-10T11:08:00.001+05:30

Visual Studio Team System 2005 had a profiler that was useful in obtaining response times and diagnosing time taken by each component/method.

Now in v2008, the Profiler comes equipped to do heap analysis. Information on how to enable heap analysis can be found at this blog
At the end of it, we get a report of the top methods allocating most memory, the types occupying most memory, etc.
This report is not as comprehensive as that provided by Numega DevPartner Studio.

Numega DevPartner Studio

2008-06-09T19:14:00.000+05:30

Recently I installed DevPartner Studio and have explored the features provided by the tool. The tool integrates seamlessly into VS.NET and has the following high-level features:

- Static Analysis (Similar to FxCop or Code Analysis in VS Team Studio)

- Error Detection (You actually run the application/program and the tool would give a report of possible errors. This tool is more valuable in case of COM usage through .NET and in C, VC++ projects to detect OutOfBounds errors and dangling pointers, etc)

- Code Coverage (You run the application and at the end of it, a report is generated giving us code paths that were not executed. So helpful in finding dead code.)

- Memory Analysis (Heap analysis, shows object graph till root objects, no of objects created/destroyed etc.)

- Performance Analysis (E.g. U can create snapshots while running the application and the tool would give you the time spend in each method and many other stats. E.g. top 20 methods consuming most time, call graph, etc. Another cool feature is the source code window with time stats for each line on the left side. No more guessing what is taking so much time. )

- In Depth Performance Analyzer (CPU stats, Disk IO, Network IO, etc)

Visual Studio Team System provides us with many similar features. A VS product comparison can be found here.

Interesting features in Web 2.0 feature pack of Websphere v6.1

2008-06-02T15:40:00.000+05:30

IBM has recently released a new feature pack for WAS 6.1 that has a host of cool features to build Web 2.0 applications.
AJAX support is provided by the DOJO Toolkit. There are a host of gadgets available (some added by IBM on top of DOJO) for RIA screens. Some features that were of particular interest to me are:

A javascript SOAP client that will enable web clients to make webservices requests directly.
Web-remoting RPC component that would enable a JS client to call an EJB method or a POJO method directly.
Apache Abdera library for manipulating ATOM/RSS feeds
JSON4J library on the server side to convert between JSON text and Java objects
AJAX messaging to implement Server side push. This is totally cool. I always wanted to experiment with the CometD functionality available in DOJO and here the interation between the DOJO Message Bus (client) and Websphere Service Integration bus was provided out of the box :)

A good example showing Stock Quote Streaming can be found here.

Finding the geographical location of client web requests

2008-05-31T20:36:00.000+05:30

Nowadays, we see many sites that show our geographical location the screen when we hit the page.
So how do the sites find out this information? Well, on the server side, we can obtain the IP address of the client (or the Internet provider proxy) from the request object.

From this IP address, we can query a database(typically csv files) and obtain the location information. The country information is free and available for download here.

For state and city information, U got to stuff out some moolah.. :)
More information can be found out here-
http://www.ip2location.com/faqs-ip-country.aspx

Kernel mode and User mode

2008-05-28T13:58:00.000+05:30

In IIS 6.0, the core HTTP engine (HTTP.SYS) runs in kernel mode and all worker processes run in user mode. So what exactly is the difference between kernel mode and user mode programs?

User mode and kernel mode refers to the privilege level a process has to the system hardware. The closer to the hardware the process becomes, the more sensitive the system is to provoking system failure. In any OS, you want to separate applications from OS services because you want the OS to remain functional if an application crashes.

Typical OS architecture has two rings: one ring running in system mode, and a ring running in user mode. The kernel has full control of the
hardware and provides abstractions for the processes running in user mode. A process running in user mode cannot access the hardware, and must use the abstractions provided by the kernel. It can call certain services of the kernel by making "system calls" or kernel calls. The
kernel only offers the basic services. All others are provided by programs running in user mode.

Kernel mode program also run much faster than User mode programs as they are much closer to the hardware.

Overcoming the 2 connection limit in browsers

2008-05-21T00:21:00.000+05:30

The HTTP 1.1 protocol dictates that browsers cannot make more than 2 connections per domain name. This limitation can cause slow page loads for pages containing a lot of images or other references to external resources.
To overcome this limitation, we can create sub-domains and serve other static content from these sub-domains, so that the browser can create extra 2 connections for each sub-domain.
This trick is explained in the following links:

http://brainspl.at/articles/2006/12/29/speed-up-page-loads
http://www.die.net/musings/page_load_time/

Actually the above technique is only useful if there are a lot of external resources that need to be loaded by the page. HTTP 1.1 also brought in the concept of HTTP pipelining. This means that over those two connections the browser can send the requests back-to-back before even before receiving a single response. This completely eliminates the dead-time between getting back the last packet of the previous request and then sending the next request. Instead, all the requests are queued at the server, which sends out responses as fast as TCP/IP would allow. If HTTP pipelining is enabled, then the page load speed improves dramatically.

But unfortunately, HTTP pipelining is disabled by default in IE/Firefox- bcoz not all proxies and servers support it still. To enable pipelining in Firefox type "about:config" in Firefox's location bar and enable the "network.http.pipelining" preference.

Oracle RAC and Dataguard

2008-03-21T00:32:00.000+05:30

A lot of IT managers get confused between what a Oracle RAC does and what Dataguard does.

Oracle RAC provides us with a cluster of Oracle instances for the same database/datastore. This enables massive scalability and availability. But it does not provide protection against failure of the database or corruption of the data. e.g. natural disaster that resulted in data-loss or data-corruption.

Oracle dataguard enables creation of a standby database at a DR site and thus provides data protection.

ViewState in ASP.NET

2008-03-14T12:26:00.000+05:30

There is so much confusion over what ViewState does and why is it required. Finally the following link helped me in understanding a lot of things:
http://weblogs.asp.net/infinitiesloop/archive/2006/08/03/Truly-Understanding-Viewstate.aspx

Snippets from the above article:
Server Controls utilize ViewState as the backing store for most, if not all their properties. That means when you declare an attribute on a server control, that value is usually ultimately stored as an entry in that control's ViewState StateBag.
ASP.NET calls TrackViewState() on the StateBag during the OnInit phase of the page/control lifecycle. This little trick ASP.NET uses to populate properties allows it to easily detect the difference between a declaratively set value and dynamically set value.
When the StateBag is asked to save and return it's state (StateBag.SaveViewState()), it only does so for the items contained within it that are marked as Dirty. That is why StateBag has the tracking feature. In order for data to be serialized, it must be marked as dirty. In order to be marked as dirty, it's value must be set after TrackViewState() is called.

When the page first begins to load during a postback (even prior to initialization), all the properties are set to their declared natural defaults. Then OnInit occurs. During the OnInit phase, ASP.NET calls TrackViewState() on all the StateBags. Then LoadViewState() is called with the deserialized data that was dirty from the previous request. The StateBag calls Add(key, value) for each of those items. Since the StateBag is tracking at this point, the value is marked dirty, so that it may be persisted once again for the next postback.

ViewState is only one way controls maintain values across postbacks. Regular good old HTML FORMS play a role, too. For example, disable viewstate on a textbox, and it will still maintain its value, because it is POSTING the value with the form. Make that TextBox invisible then do a post, and the value is lost. Thats where ViewState helps, which would allow it to maintain the value even if its invisible.

The option to set control properties in the OnPreInit event in order to avoid those values from being entered into the viewstate is a good idea to avoid storing these values in the ViewState. To avoid Datagrids from storing values in the ViewState, rebind the data to the grid on each page load.

Security Principles

2008-03-06T15:15:00.000+05:30

Browsing thru the OWASP site, I came across the following security principles that are of interest while designing the security architecture. Every security policy has 3 objectives:
a) Confidentiality b)Integrity c)Availability

Examples of security principles:
1: Securing the weakest link (The chain is only as strong as its weakest link) - e.g. Attackers will not target the firewall, but the applications accessible through the firewall.

2: Minimize Attack Surface Area - design the system such that the potential areas for intrusion are reduced.

3. Principle of Least Privilege - only give those permissions to the user that are required.

4. Principle of Defense in Depth - e.g. Do validations at the front-end using Javascript, in the web-tier using validation logic, in the database using constraints and triggers.

5. Fail securely - If an application/program fails, then it should not leave the system in an insecure state.

6. Separation of Duties - e.g.an administrator should be able to turn the system on or off, set passwordpolicy but shouldn’t be able to log on to the storefront as a super privileged user, such as beingable to buy goods on behalf of other users.

7. Don't just rely on security by obscurity. Use security by design. Use elements of both the strategies.

The following links provide interesting material for reading:
http://www.infoworld.com/article/05/08/19/34OPsecadvise_1.html
http://en.wikipedia.org/wiki/Honeypot_%28computing%29

XSS and CSRF attacks

2008-03-05T19:23:00.000+05:30

Of late, I have been doing a lot of study to understand 'Cross-Site Scripting' attacks and 'Cross Sitre Request forging' attacks.
XSS flaws occur whenever an application takes user supplied data and sends it to a web browser without first validating or encoding that content. XSS allows attackers to execute script in the victim's browser which can hijack user sessions, deface web sites, possibly introduce worms, etc. http://www.technicalinfo.net/papers/CSS.html
http://www.shocking.com/~rsnake/xss.html

How to prevent XSS atacks? Tips from the www.owasp.org site
1. Use proper input validation techniques
2. Encoding the output. This includes data read from files and databases.
For input validation, its better to go for a ‘positive’ security policy that specifies what is allowed rather than a ‘Negative’ or attack signature based policies as they are difficult to maintain and are likely to be incomplete.

The following link at MSDN contains some good info about preventing XSS attacks:
http://msdn2.microsoft.com/en-us/library/ms998274.aspx
Microsoft even has a "Anti-Cross Site Scripting Library" available at:
http://msdn2.microsoft.com/en-us/library/aa973813.aspx

The OWASP site defines CSRF as follows:
Cross-Site Request Forgery (CSRF) is an attack that tricks the victim into loading a page that contains a malicious request. It is malicious in the sense that it inherits the identity and privileges of the victim to perform an undesired function on the victim's behalf, like change the victim's e-mail address, home address, or password, or purchase something. CSRF attacks generally target functions that cause a state change on the server but can also be used to access sensitive data.

One technique to prevent event CSRF attacks is to use the 'Token Synchronization' pattern. Sample filters for JEE and .NET are available on the OSWAP site at the following links:
http://www.owasp.org/index.php/CSRF_Guard
http://www.owasp.org/index.php/.Net_CSRF_Guard

Storing sensitive information in properties file

2008-03-05T19:13:00.000+05:30

In JEE and .NET applications we often store data access configuration parameters such as username, password, datasource URL in properties files. But how to protect this sensitive information? The obvious answer is to encrypt the file or the 'string' properties. But then the question is - where do U store the key? If U encrypt the key, then U would require another key to decrypt this key...a classic chicken-and-egg problem.

Browsing thru OWASP site pages, I came across this page that contained some interesting ideas for this problem. Snippet from the site:

"Some environments have protected locations, such as the WebSphere configuration files, or the system registry. You can use a file in a protected location, that uses OS access control to limit access to only the application. You could put the key in the code, but that makes it difficult to change and deploy securely. You can also force the master key to be entered when the system boots up so it's only in memory, but that means you lose automatic reboot. You can even split the key and put parts in more than one of these locations."

The page also contains a sample implementation in Java for encrypting properties file.

Various types of performance testing

2008-03-04T14:38:00.000+05:30

Came across this site (http://www.loadtest.com.au/types_of_tests.htm) that lists down the various types of testing that can be done to performance test a system. Given below are snippets from the above page.

Load Testing: Load Tests are end to end performance tests under anticipated production load. The primary objective of this test is to determine the response times for various time critical transactions and business processes and that they are within documented expectations (or Service Level Agreements - SLAs). The test also measures the capability of the application to function correctly under load, by measuring transaction pass/fail/error rates.
Stress Testing: Stress Tests determine the load under which a system fails, and how it fails. This is in contrast to Load Testing, which attempts to simulate anticipated load. It is important to know in advance if a ‘stress’ situation will result in a catastrophic system failure, or if everything just 'goes really slow'. There are various varieties of Stress Tests, including spike, stepped and gradual ramp-up tests. Catastrophic failures require restarting various infrastructure and contribute to downtime, a stress-full environment for support staff and managers, as well as possible financial losses.
Volume Testing: Volume Tests are often most appropriate to Messaging, Batch and Conversion processing type situations. In a Volume Test, typically the Response times are not measured. Instead, the Throughput is measured. A key to effective volume testing is the identification of the relevant capacity drivers. A capacity driver is something that directly impacts on the total processing capacity. For a messaging system, a capacity driver may well be the size of messages being processed. For batch processing, the type of records in the batch as well as the size of the database that the batch process interfaces with will have an impact on the number of batch records that can be processed per second.
Network Testing: Network sensitivity tests are tests that set up scenarios of varying types of network activity (traffic, error rates...), and then measure the impact of that traffic on various applications that are bandwidth dependant. Very 'chatty' applications can appear to be more prone to response time degradation under certain conditions than other applications that actually use more bandwidth. For example, some applications may degrade to unacceptable levels of response time when a certain pattern of network traffic uses 50% of available bandwidth, while other applications are virtually un-changed in response time even with 85% of available bandwidth consumed elsewhere.This is a particularly important test for deployment of a time critical application over a WAN.
Failover Testing: Failover Tests verify of redundancy mechanisms while under load. For example, such testing determines what will happen if multiple web servers are being used under peak anticipated load, and one of them dies. Does the load balancer react quickly enough? Can the other web servers handle the sudden dumping of extra load? This sort of testing allows technicians to address problems in advance, in the comfort of a testing situation, rather than in the heat of a production outage.
Soak Testing: Soak testing is running a system at high levels of load for prolonged periods of time. A soak test would normally execute several times more transactions in an entire day (or night) than would be expected in a busy day, to identify and performance problems that appear after a large number of transactions have been executed. Also, due to memory leaks and other defects, it is possible that a system may ‘stop’ working after a certain number of transactions have been processed. It is important to identify such situations in a test environment.

Schema validation in webservices

2008-02-29T17:12:00.000+05:30

Recently a friend of mine was working on a .NET SOA project and was facing a peculiar problem.The XML Schema he was using had a lot of restictions such as minOccurs, maxOccurs, int ranges, string regular expression patterns, etc.

But unfortunately in .NET 2.0 webservices, there was no easy way to validate input messages based on this schema. The reason for this behavior has to do with XmlSerializer, the underlying plumbing that takes care of object deserialization. XmlSerializer is very forgiving by design. It happily ignores XML nodes that it didn't expect and will use default CLR values for expected but missing XML nodes. It doesn't perform XML Schema validation and the consequence is that details like structure, ordering, occurrence constraints, or simple type restrictions are not enforced during deserialization.

Unfortunately there is no switch in .NET 2.0 that can turn on Schema validations. The only way to validate schemas is to use SOAP extensions and validate the message in the extension code block before deserialization occurs. The following links show us how to do it:
1. MSDN-1
2. MSDN-2

In the J2EE world, JAX-WS 2.0 has become the default standard for developing webservices. In most toolkits, we essentially have 2 options: Let the SOAP engine do the data-binding and validation or separate the data-binding and schema validation from the SOAP engine. For e.g. In Sun Metro project, validation can be enabled for the SOAP engine using the @SchemaValidation attribute on a webservice.(https://jax-ws.dev.java.net/guide/Schema_Validation.html)
In Axis 2, there is support to enable schema validation using the XMLBeans binding framework.

I found this whitepaper quite valuable in understanding the various options for schema validation in JEE webservices.

Problems with Struts SSLext

2008-02-27T18:36:00.000+05:30

One of the projects I was consulting on, was using the SSL-ext plugin module of Struts to redirect all secure resources to SSL. More information on how to mix HTTP and HTTPS in a web-flow can be found in my blog post here.

The SSL plug-in module used a simple technique - a custom request processor/servlet filter would check the URL of each request and then check in the configuration file if the requested resource was secure and needed to be accessed using SSL. So if we receive a plain HTTP request for a secure resource, then a redirect HTTP 304 response would be sent back to the user. The redirected URL would have the HTTPS scheme.

Now, in the production environment, it was decided to move the SSL encryption/decryption to a Cisco Content Switch (at the hardware level). The Content switch would decrypt the content and then forward the request to the webserver as plain HTTP.
But this caused a problem at the AppServer level as the SSL-ext in Struts always sent redirect responses for all secure resources. This resulted in a recursive infinite loop of request-response traffic between the browser and the servers, due to which the browser showed a 'hanged' screen.
I was given the task to find an innovative solution to this problem. The challenge here was that we wanted to use the SSL-ext plug-in feature functionality so that it is impossible to access secure resources using plain HTTP. All we needed was some way for the content switch to pass information to the AppServer that the request that reached the content switch was HTTPS.

I decided to do this using HTTP headers. We configured the content switch to set the 'Via' HTTP header to "HTTPS" whenever a SSL request reaches the content switch. We changed the source code of SSL-ext to check for this header instead of the URL scheme and port number and send a redirect if necessary. This simple solution worked just fine and earned me kudos from the team :)

P.S: I got the idea of using HTTP header after reading the below URL. A must read :)
http://www.nextthing.org/archives/2005/08/07/fun-with-http-headers

Architecture Principles

2008-02-26T15:02:00.000+05:30

A member of my team recently asked me a simple question - "What are Architecture principles? And how are they different from the guidelines, best practices and NFRs that we already follow?"

Well the line of difference between the two is very blurred and in fact there could be a lot of overlap between them.

Principles are general rules and guidelines(that seldom change) that inform and support the way in which an organization sets about fulfilling its mission.
Architecture principles are principles that relate to architecture work. They embody the spirit and thinking of the enterprise architecture.These principles govern the architecture process, affecting the design, development and maintenance of applications in an enterprise.

The typical format in which a principle is jotted down is -
1. Statement - explaining what the principle is
2. Rationale - explaining why?

The following 2 links contain good reading material.
http://www.opengroup.org/architecture/togaf8-doc/arch/chap29.html
http://www.its.state.ms.us/its/EA.nsf/webpages/principles_home?OpenDocument

Examples of some interesting principles from the above sites:

Principle: Total cost of ownership design
In an atmosphere where complex and ever-changing systems are supporting all aspects of our business every hour of the day, it is easy to lose track of costs and benefits. And yet, these critical measures are fundamental to good decision-making. The Enterprise Architecture can and must assist in accounting for use, for change, for costs, and for effectiveness.
Rationale:Total costs of present and proposed alternatives, including unintended consequences and opportunities missed, must be a part of our decisions as we build the architecture of the future.

Principle: Mainstream technology use
Production IT solutions must use industry-proven, mainstream technologies except in those areas where advanced higher-risk solutions provide a substantial benefit. Mainstream is defined to exclude unproven technologies not yet in general use and older technologies and systems that have outlived their effectiveness.
Rationale:The enterprise may not want to be on the leading edge for its core service systems. Risk will be minimized.

Principle: Interoperability and reusability
Systems will be constructed with methods that substantially improve interoperability and the reusability of components.
Rationale:Enables the development of new inter-agency applications and services

Principle: Open systems
Design choices prioritized toward open systems will provide the best ability to create adaptable, flexible and interoperable designs.
Rationale:An open, vendor-neutral policy provides the flexibility and consistency that allows agencies to respond more quickly to changing business requirements.This policy allows the enterprise to choose from a variety of sources and select the most economical solution without impacting existing applications. It also supports implementation flexibility because technology components can be purchased from many vendors, insulating the enterprise from unexpected changes in vendor strategies and capabilities.

Principle: Scalability
The underlying technology infrastructure and applications must be scalable in size, capacity, and functionality to meet changing business and technical requirements.
Rationale:Reduces total cost of ownership by reducing the amount of application and platform changes needed to respond to increasing or decreasing demand on the system.Encourages reuse.Leverages the continuing decline in hardware costs

Principle: Integrated reliability, availability, maintainability
All systems, subsystems, and components must be designed with the inclusion of reliability and maintainability as an integral part. Systems must contain high-availability features commensurate with business availability needs. An assessment of business recovery requirements is mandatory when acquiring, developing, enhancing, or outsourcing systems. Based on that assessment, appropriate disaster recovery, and business continuity planning, design and testing must take place.
Rationale:Business depends upon the availability of information and services. To assure this, reliability and availability must be designed in from the beginning; they cannot be added afterward. The ability to manage and maintain all service resources must also be included in the design to assure availability.

Principle: Technological diversity is controlled to minimize the non-trivial cost of maintaining expertise in and connectivity between multiple processing environments.
Rationale:There is a real, non-trivial cost of infrastructure required to support alternative technologies for processing environments. There are further infrastructure costs incurred to keep multiple processor constructs interconnected and maintained. Limiting the number of supported components will simplify maintainability and reduce costs.

Tower Servers -> Rack Servers -> Blade Servers

2008-02-25T16:05:00.000+05:30

The early servers were tower servers, so called because they were tall in height and took a lot of space and resulted in 'server-sprawl' in data-centers. The 90's saw the introduction of rack servers that were compact and saved a lot of 'real-estate' in the data-centers. A standard rack is 19 inch wide and 1.75 inch high. This dimension is called 1U. So a server component may occupy 1U, 2U or 4 half-U. The most common computer rack form-factor being 42U high, this configuration allows for 42 servers to be mounted on a single rack. Each server has it own power supply and network and switch configuration.
In the past few years, blade servers are gaining a lot of popularity. The advantage of blade servers is that instead of having a number of separate servers with their own power supplies, many blades are plugged into one chassis, like books in a bookshelf, containing processors, memory, hard drives and other components. The blades share the hardware, power and cooling supplied by the rack-mounted chassis -- saving energy and ultimately, money.
Another advantage is that enterprises can buy what they need today, and plug in another blade when their processing needs increase, thus spreading the cost of capital equipment over time.

In a rack server environment, typically 44% of the electricty consumption is by components such as power supplies and fans. In a blade server, that 44 percent is reduced to 10 percent because of the sharing of these components.This gives the blade server a tremendous advantage when it comes to electricity consumption and heat dissipation.

The only current disadvantage of blade servers is the vendor lock-in that comes in when you buy a blade environment and the system management software. In a rack system, it is possible to mix and match servers inside of a rack and across racks at will.

Are filters invoked when I do a RequestDispatcher.forward() ?

2008-02-22T18:35:00.000+05:30

Before Servlet 2.4 specification, it was not clear whether filters should be invoked for forwarded requests, included requests and requests to the error page defined in

But in Servlet 2.4 specs, there is an extra element inside the filter mapping tag.
<filter-mapping>
<filter-name>DispatcherFilter</FILTER-NAME>
<url-pattern>/products/*</URL-PATTERN>
<dispatcher>FORWARD</dispatcher>
<dispatcher>REQUEST</dispatcher>
</filter-mapping>

Possible values are REQUEST, FORWARD, INCLUDE, and ERROR (Request is the default)