CLUE database in Insurance

The CLUE (Comprehensive Loss Underwriting Exchange) is a database of insurance losses and claims that was created and is maintained by ChoicePoint. ChoicePoint was taken over by LexisNexis some time back. LexisNexis C.L.U.E. Property reports contain up to seven years of personal property claims matching the search criteria submitted by the inquiring insurance company. So it's the equivalent of credit reports in the insurance industry. ChoicePoint also maintains a database for personal auto claims history.

When we apply for a homeowner's or auto insurance policy, the insurance company orders and review our CLUE report. A CLUE report is a document containing your personal information, as well as data regarding past property claims that have been paid under your previous insurance policies. CLUE reports includes the type of property (vehicle or home), dates of losses, loss descriptions and amounts paid.

Insurance companies use information on CLUE reports to evaluate homeowner's and auto insurance applicants for acceptability. Information can also be used to determine your policy premiums. CLUE reports are used almost exclusively to underwrite and rate new policies. Most insurers renewing existing policies do not access CLUE reports at renewal, largely because they already have loss histories for these properties in their own database. Only insurance companies that subscribe to CLUE can submit loss data and access CLUE reports.

A software program called Quick Connect has been designed by ChoicePoint for sending and receiving information electronically to and from ChoicePoint.

Conceptual model for SOA

Just read a very interesting blog post from Nick Malik, where in he provides a Common Conceptual Model for SOA. I was bowled over by the simplicity of the language and diagram. So easy to explain to a fellow architect embarking on a design assignment based on SOA.



The most important things to work on in a SOA project:

• Canonical Data Model


• Canonical Message Model (Shared Message Contract)


• Service Interfaces


• Business Event Taxanomy

I have seen that if you get these fundamentals right during the design process, then building scalable SOA based systems is a lot easier.

Why Enterprise Architecture?

Recently a friend of mine asked me to explain in simple terms the benefit of having an Enterprise Architecture for an organization. The challenge was to translate the message in non-technical non-complex terms without any jargon. He apparently had read an EA white-paper by a big analyst firm and had his head spinning :) . I won’t blame him – this is a problem with ‘complex definitions’.

As architects, the most important challenge that we solve day-in and day-out is in reducing and managing IT complexity. Yet, it’s an irony that many architects/analysts use ‘complex long phrases’ to describe even the simplest of concepts.

Enterprise architecture is required to give a holistic view of the entire IT landscape of the organization. This enables the organization to view its current state and plan for the future state of its IT operations. EA is very useful to plan and prioritize your IT budgets in accordance with your important business priorities. It serves as a powerful communication tool between the business and IT teams and helps align new IT investments appropriately. Organizations have also started using EA as a tool for risk management and SWOT analysis.

Thus having an Enterprise Architecture Framework helps us in visualizing the ‘big picture’ and the relationship between different domains. It also leads to better governance and low maintenance cost enabled by enforcement of technology standards and architecture guidelines.

Quite often, EA is compared to ‘city planning’. In city planning, you look at the big picture and establish zones for specific purposes. It lays down the guidelines and best practices for buildings, roads, water supply, hospitals, etc. The design of each building is analogous to ‘Solution Architecture’. The special skill of designing a kitchen is analogous to ‘technical/application architecture’.

So to summarize, EA has the following advantages:

• Helps build a common understanding of the future IT direction of the enterprise.
• Provide clear mapping of the business processes and the IT systems supporting it. Provides visibility into how the business processes enable the mission of the enterprise.
  
• Improves interoperability and integration by defining enterprise canonical message models and data models. Also integration standards and guidelines - for e.g. SOA, MOM, etc.
• Enables organizational agility - If we need to respond to a business change, what is the impact on the IT systems, etc.
• Less cost due to technical homogenity that is easier to support and maintain.
• Powerful tool for communicating risk and change to all stakeholders in the enterprise.
  

There are a number of frameworks for defining EA – popular among them are the Zachman Framework and TOGAF. I had also blogged earlier about tools used for creating an EA.

OpenSource EA tools

Planning to evaluate a couple of open source Enteprise Architecture tools that are becoming popular. IBM System Architect was the de-facto commercial EA tool in the market and we were actively using it for our customers.

First is "The Essential Project" that was launched last year. It is based on the Protégé ontology tool and allows you to create a meta-model for your EA. It has a EA repository and reporting tools.

Second is a tool called "iteraplan". This tool is more popular in Europe and also has an Enterprise Edition that is free for use, but without the source code.

Creating Dynamic classes in Java

Earlier I had blogged about the ability to create new classes from scratch in .NET.
I was looking for something similar to Reflection.Emit() in Java. Found a few examples on the web that use the Java compiler to compile source code dynamically and use Reflection to execute the methods.
http://www.javaworld.com/javaworld/jw-06-2006/jw-0612-dynamic.html?page=5
http://www.rgagnon.com/javadetails/java-0039.html

Other option is to use byte-code manipulation libraries such as BCEL, ASM, etc. A good list of open source byte code libraries are given at:
http://www.java-opensource.com/open-source/bytecode-libraries.html

Payment Gateways

We were exploring the various options for selecting a payment gateway for a new business opportunity. Found this site that has a good explanation of payment gateways and compares the popular payment gateways available in the market.
http://www.bestpaymentgateways.com/
The site has a brief explanation on each payment gateway. The following gateways are explained.

• Amazon Payment Gateway
• Authorize.net
• PayPal PayFlow Pro
• VeriSign
• Google Checkout
• Moneybookers
• VerePay
• WorldPay

Parsing and reading text from PDF files

One of my development teams was looking for a PDF parsing library. They essentially wanted to search and extract data from PDF files. At first, I thought that OCR is the only way to achieve this, but there are libraries available to help us :)

PDFBox : This seems to be most popular library for extracting text out of PDF files. This is a Java library, but also has a .NET wrapper around it using iKVM.NET
Simple examples using this library can be found here and here.

iText & iTextSharp : These libraries are very popular for PDF generation and can also be used for extracting text from PDF files. Sample example can be found here.

I have heard that OpenOffice.org also provides you with a Java API that can be used to create and manipulate PDF files, but have not tried it yet.

Datawarehouse vs Datamart

The debate between creating a datamart or sourcing data from the datawarehouse springs up from time to time across organizations. Found some good links on the web on this great debate:
information-management
exforsys.com
opensourceanalytics.com

Performance issues with an in-memory Datatable select clauses

Very often, we cache .NET Datatable objects in memory. This could be for lookup tables or for other frequently accessed data. The following article on MSDN points out that using 'select' on a Datatable can be very very slow.

http://msdn.microsoft.com/en-us/library/dd364983.aspx

Snippet from the article:
"Select takes arbitrary criteria and returns an array of DataRows. Essentially, DataTable.Select has to walk the entire table and compare every record to the criteria that you passed in. "

The performance stats for LINQ in the article look very impressive. Looks like Microsoft has made a lot of optimizations on LINQ. It would be great to know (see source code) of what happens behind the scenes and why LINQ is so damn fast!

.NET PDF creation tools

Some time back, I had blogged about the opensource tools available in Java and .NET for PDF creation. Recently came across another commerical library called "Dynamic PDF" for PDF creation. This library has a Java and .NET version of the API.
The library is easy to use and is very robust. It handles 'over-flowing' tables and text areas in PDF files very gracefully.
Another interesting library that Dynamic PDF sells is the DynamicPDF™ PrintManager for .NET . This library makes it so simple to print PDFs and also has callback handlers for error messages from printers - all pure managed code. Now that's what I like :)

Two opensource .NET PDF generation tools are PDFSharp and iTextSharp.

XSD to XML and vice versa

I was looking for a quick and free tool that could convert XSD schema files to sample XML files and also generate a XSD file from a sample XML file. Commerical tools such as XML Spy and OxygenXML were powerful tools that provided these features, but I was looking for a free one.
First I checkout the open source Java IDEs.

• Netbeans had a beautiful editor to visualize and edit a XSD schema in a graphical tree structure. But unfortunately it did not have the ability to generate sample XML file or vice versa.
• Eclipse too had a visual editor and allowed visual editing of schema elements. It could generate a sample XML file from a XSD file, but had no option for the reverse; i.e. generating a schema file from a sample XML.
• VS 2008 SP1 had both the options - coversion between XML and XSD. For schema files, right click on a node in XSD Explorer view and select "Generate XML". For XML files, select "Tools -> Generate schema" to create the XSD file. Both these operations are very quick in Visual Studio.
• VS 2010 has extensive support for XML tooling. You have 3 different views for schema designing that should suffice for most complex schema definition.

Besides these free tools, there are other command line tools that can be used. For e.g.

XSD.exe tool can be schema to XML and vice versa transformation.

There was another .NET tool that I found on MSDN for generating XML documents from XSD.

Another cool Java desktop tool that supported these features was XMLSpear.

Updatable Views Vs ORM tools (ADO.NET Entity Framework)

Recently during one of our design brainstorming discussions, our database developer was having a hard time trying to understand the benefit of using a ORM tool such as NHibernate or ADO.NET Entity framework. For our example scenario, he said that he could easily create database views that would serve the same purpose - abstracting the logical schema and helping the developer avoid complex joins for simple business queries.

For the use-case we were working on, his statement made sense. But there are a lot of cases and reasons why database views would not be the best choice. Here is a snippet from a MSDN article.

An obvious question at this point would be why not just use traditional database views for this. While database views can abstract many of the mappings, often that solution won't work for several process and functional reasons: (a) many of the views are simply too complex to be generated and maintained by developers in a cost-effective way, even for some simple conceptual to logical mappings, (b) the classes of views that have the property of being automatically updatable at the store are limited, and (c) databases for core-systems in medium and large companies are used by many central and departmental applications, and having each individual application create several views in the database would pollute the database schema and create significant maintenance workload for the database administrators. In addition, database views are limited to the expressivity of the relational model, and typically lack some of the more real-world concepts of the Entity Data Model, such as inheritance and complex types.

ADO.NET client-views work entirely on the client, so each application developer can create views that adapt the data to a shape that makes sense for each particular application without affecting the actual database or other applications. The class of updatable views supported in the Entity Framework is much broader than those supported by any relational store.

Profilers for .NET

Way back, I had blogged about the profiling capabilities of VSTS 2008 and Numega DevPartner Studio.

Came across this site that lists down the most of the .NET profiling tools available in the market.
http://sharptoolbox.com/categories/profilers-debuggers

I have had good experience with .NET memory profiler and ANTs profiler.
Both of them allows you to collect stats on elapsed time for the entire call-stack and also allow you to walk the heap and check the effect of GC runs across heap generations.

Referential Integrity across databases in SQL Server

Having worked on Oracle databases for a long time, I was quite comfortable with the idea of maintaining RI across different schemas in same database. For e.g. Ur database may have a schema containing master tables and your transaction schema references the lookup data in the master tables.
But to my suprise, prior to SQLServer 2005, it was not possible to have separate schemas in SQL Server. This meant that there was no easy way to maintain RI across databases in SQLServer. The only options were using messy triggers or a CHECK constraint with a UDF (user defined function)

Maintaining RI across databases is still not possible in SQLServer, but SQLServer 2005/2008 have added the concept of schemas in databases; i.e. it is possible to have multiple schemas in each database and each schema offers the same logical separation that a separate database would. We can also configure the schema to be located on a separate filegroup or a separate disk, thus maximizing performance. In SQLServer 2005/2008, it is possible to have RI constraints across schemas. I think this is the best design approach to take. If it is not possible to have your master data in the same database, then the second best approach is to use replication to get the master data into your database.

What is Micro-Architecture in software design?

Recently I came across the term 'Micro-Architecture' on a number of pages on Sun's site. Sun's engineers have an interesting definition for this term. They refer to Micro-Architecture as a composition of a set of patterns that can be used to realize a subsystem.

In the book - "Core J2EE patterns", Micro-Architecture is described as below.

We define micro-architecture as a set of patterns used together to realize parts of a system or subsystem. We view a micro-architecture as a building block for piecing together well-known, cohesive portions of an overall architecture. A micro-architecture is used to solve a coarser-grained, higher-level problem that cannot be solved by a single pattern. Thus, a micro-architecture represents a higher level of abstraction than the individual patterns.

Critical Success Factors for SOA

A lot of organizations are embarking on the SOA journey and it is important to understand the factors that would make the SOA initiative successful. Based on our experience in SOA consulting, we have learned the following.

1. Adopt an evolutionary approach to SOA: It’s impossible to sustain a big-bang approach to SOA at the enterprise level and even more difficult to make a business case for the large investment. The best approach is to have an enterprise level SOA governance and adopt an incremental and iterative approach. This approach is called the ‘hybrid’ or ‘middle-out’ approach.
After gaining success in early iterations, it becomes easier to make a business case for further investment or create a self funding model. It is a great idea to focus on immediate business pain points during the initial iterations, such as automation, reducing duplicity, etc.

2. Ensure top management support: This is very crucial because SOA strategies would require many departments/LOB’s to come together and function as one unit. And unless you have the top level executive sponsorship to support the SOA initiative, you could get entangled in a mesh of organization politics.
Also it is important to address the cultural issues and involve all layers in the organization during decision making. If we are able to define WIIFM (What’s in it for me?) for each important business stakeholder, then the changes of SOA success are very high.

3. Implement SOA Governance from the start: SOA governance is a very board area and covers many aspects. SOA governance includes 2 dimensions – the methodology/process that is established and the tools and technologies used for governance.
Service Life Cycle Management, Service monitoring and Service SLA management are all important points of SOA governance. In fact, if the organization has a EA practice, then SOA governance should be part of it.

Ruminating on Requirements Elicitation

Last night, I was ruminating on the lessons learned during my experience in business architecture, business analysis and requirements elicitation. I think at the broad level, there are 2 kinds of requirements.

The first kind is - Where we know, what we don't know: These requirements are easy to capture since we have an idea about the pain areas of the customer, we know what is the vision of the project, what is being planned to be done, etc. Hence we can ask appropriate questions, dig deeper and after a few iterations of interviews can get a pretty good understanding of the requirements.

The second kind of requirements are - Where we don't know, what we don't know !
Digging out these requirements is the most challenging task - because we never ask the right questions about things that don't know, our past memory/experiences make implicit assumptions on certain things. Such kind of requirements result in last minute suprises and everyone wonders why these points were never discussed or brought out !!!

How do we elicit these requirements? I think the following techniques during requirements elicitaiton would help:
1. Question each and every assumption - both implicit and explicit. Assumptions may not hold true in every scenario. Check if the assumption is valid for the current scenario.
2. Ask 'stupid' questions - Yes, this works !!!...I have seen that often, analyst's refrain from asking questions because they are self-conscious, are afraid of being ridiculed for lack of knowledge in a domain area, etc. i.e. they are afraid of sounding 'stupid'. But clarifying even the simplest doubt could open up a plethora of new requirements.

Migrating applications across platforms

We often require to migrate applications from one platform (OS/hardware) to an other platform - either as part of a portfolio rationalization program or because the current platform is reaching its EOL. For e.g. migrating Java apps from Solaris/SPARC to Linux/IBM Power, etc.
During such migrations, it is essential to have a well documented plan covering all the aspects, so that we do not miss anything. Listed below is a brief set of points that must be included in the brainstorming discussions during the planning phase.

1. Migration of the Runtime environment: For e.g. miration of the Java runtime from Solaris to Linux. There are many JRE's available on Linux and we need to select the appropriate one.
2. Migration of the Application Infrastructure: The application may require a webserver, application server, a Message queue, etc. These infrastructure components would need to be ported to the target platform or other alternatives selected.
3. Migration of application code: Does the code need to be recompiled/linked on the target platform? Was the application using any platform specific API for performance?
4. Migration of application configuration: What configuration needs to change on the target platform? e.g. file paths, endian-ness, max-memory settings, thread pool settings, etc.
5. Migration of development environment: Would it be required to migrate the development environment? Will the same IDE work? IDE-Runtime integration issues, etc.
6. Migration of Build and Deployment process: Need to change build scripts - ANT, Maven, Shell scripts, etc.

Also it makes sense to leverage the expertise of the target platform vendor during the migration exercise. For e.g. IBM, HP, SUN, RedHat, Novell, etc. all have guidelines, recommendations and migration kits for migrating from a target platform to their platform.

What is longitudnal data?

A dataset is considered to be longitudnal if it tracks the same kind of information at multiple points of time. For e.g. the marks of students over multiple years, patient health records over a period of time, etc.

The most important advantage of longitudnal data is that we can measure change and the effect of various factors over the data-point time values. For e.g. what is the effect a particular drug had on a cancer patient? The effect of different teachers on a student?

So essentially, longitudnal data helps in establishing cause-n-effect relationships. Longitudnal data stores are also being used for predictive modeling and other areas. Longitudnal data stores are very popular in the Life Sciences and Healthcare industry.
I am interesting in learning the best practices for creating and optimizing a data-model for longitudnal data stores.

Difference between biostatistics and bioinformatics

Working in the healthcare domain, I often come across the terms - biostatistics and bioinformatics and wondered as to what were the differences between the two branches of studies. A quick googling revealed the following:
The term Biostatistics is a combination of the words biology and statistics. So it essentially it is the application of statistics to biology. The science of biostatistics encompasses the design of biological experiments, especially in medicine and agriculture; the collection, summarization, and analysis of data from those experiments; and the interpretation of, and inference from, the results.
Bioinformatics is the application of information technology and computer science to the field of molecular biology. Its primary use has been in genomics and genetics, particularly in those areas of genomics involving large-scale DNA sequencing. Bioinformatics now entails the creation and advancement of databases, algorithms, computational and statistical techniques, and theory to solve formal and practical problems arising from the management and analysis of biological data.
Bioinformatics focuses on applying computationally intensive techniques (e.g., pattern recognition, data mining, machine learning algorithms, and visualization) to understand biological processes.
More information can be found on Wikipedia at:
http://en.wikipedia.org/wiki/Bioinformatics
http://en.wikipedia.org/wiki/Biostatistics

Mapping UI controls to entitlements/operations

In RBAC, we often need to enable/disable a UI control based on the users role and entitlements. Most programmers write code for mapping a UI control with the operation name, i.e. if 'submitRecord' is not allowed for the user, then disable or hide the button.

Recently I came across a neat way to handle this using attributes in C#.NET. This article describes the use of attributes to specify the 'operationName' and the 'property-value' to be set on the control when we check for entitlements.
Example code snippet:
[YourCompany.Authorization("EditSalary", "ReadOnly", true)]
private System.Windows.Forms.TextBox Salary;

Though the example is that of a .NET program, the same concept can easily be applied in Java 5, which supports annotations. Annotations in Java are roughly equivalent to Attributes in .NET.

XSLT vs XQuery for XML transformations

During the early years in my career, I had always used XSLT for transformation purposes. Recently I have seen that a lot of applications and products are using XQuery for transformations. For e.g. BEA Aqualogic ESB and Apache Service Mix both use XQuery for message transformation capabilities.

Although XSLT is more powerful than XQuery for transformations, it is much simpler to learn XQuery. I remember the tough learning curve I had to go through when learning XSL, but I could grasp XQuery basics in a couple of hours. Both XSLT and XQuery use XPath for quering XML.

Found this interesting discussion on O'Reilly site that compares the two technologies. David P. in the discussion shares some interesting views on the design philosophy behind the two technologies - XQuery is a MUST UNDERSTAND language where as XSLT is a MAY UNDERSTAND language, i.e. it is more adaptive and template driven. XSLT is untyped; conversions between nodes and strings and numbers are handled pretty much transparently. XQuery is a typed language which uses the types defined by XML Schema. XQuery will complain when it gets input that isn't of the appropriate type. XQuery is better for highly-structured data, XSLT for loosely-structured documents.

We were building a transformation engine to convert between the various ACORD formats. The Acord committee has ensured that any new version of ACORD only 'adds' new elements to the current schema. No elements are deleted or the schema changed for backwards compatibility.
Hence for these transformations, XQuery fitted the bill. We also did not have to create a Canonical Data Model as direct transfortions were possible due to the above mentioned restriction. Hence if the tool supported 15 ACORD versions, only 15 XQuery files were required.

Other links on the same subject:
http://www.xml.com/lpt/a/1549
http://xmlhack.com/read.php?item=1080
http://www.xmlplease.com/xquery-xslt
http://www.oreillynet.com

Use of Constants in code

Recently there was a big debate during a code reveiw session on the use of constants. The developers had used constants for the following purposes:

1. Each and every message key used in the i18N application was declared as a constant. The application contained around 3000 message keys and hence the same number of constants.

2. Each and every database column name was declared as a constant. There were around 5000 column names and still counting..

Does it make sense to have such a huge number of constants in any application? IMHO, common sense should prevail. Message keys just don't need to be declared as constants. We already have one level of indirection - why add one more?
Reg. database column names, I have mixed opinions. If a column is being used in multiple classes, does it make sense to declare it as a global constant? Maybe declaring a class as 'Table Name' and its members as 'Column Name' would be a good idea? But if you have a large number of tables and columns, it would become very tedious to even create these constants file.

I found a few tools on the net that can automate the creation of these constants files.

- CodeProject

- MobzySystems

- ConstTool

Cool Cool JS toolbox

Just found this good website having tons of cool JavaScript tools - free for commercial use too.
http://www.dynamicdrive.com/
Check out the dynamic menus - always needed in any web-site development.

Should we call Dispose() on a Dataset?

During one of my code reviews, I saw that the development team had called 'Dispose()' on all the datasets used in the application. I knew that the Dataset was a disconnected managed object and could not understand what the dispose method would actually be doing? Dispose() is typically called to release unmanaged resources such as file-pointers, streams, socket connections, etc. In most cases, such classes also expose a Close() method that is more appropriate.

Discussions with the developers revealed that FxCop also throws an error when Dispose() is not called on Datasets.
Further investigation revleaded that the Dataset exposes the Dispose() method as a side effect of inheritance. The Dataset class inherits from the MarshalByValueComponent which implements the IDisposable interface because it is a component. The method is not overridden in the System.Data.Dataset class and the default implementation is in the MarshalByValueComponent class. The default implementation just removes the component from the parent container it is in. But in case of Dataset, there are no parent containers and hence the Dispose() method does nothing useful.

Conclusion: It is not necessary to call Dispose() on Datasets. Developers can safely ignore the FxCop warnings too :)

Ruminating over XML Schema concepts

In a recent discussion with my team members, I sensed a lot of confusion over basic concepts on XML Schemas and namespaces - especially over targetNamespace, DefaultNamespace, elements vs types, etc. Well first, lets get the fundamentals right.

A XML schema defines and declares types (complex and simple) and elements. All elements have a type. If the type has not been specified, it defaults to xsd:anyType. It's easy to understand this by drawing an anology between a class and an object. New elements can be created by referencing existing elements.

In a WSDL message description, a WSDL part may reference an element or a type. If the SOAP binding specifies style="document", then the WSDL part must reference an element. If the SOAP binding specifies style="rpc", then the WSDL part must reference a type.
Now coming to namespaces, users typically get confused over the difference between a targetNamespace and defaultNamespace. The 'targetNamespace' attribute is typically used in schema files to identify and select the namespace into which new elements that are defined are created in. It is the namespace an instance is going to use to access the types it declares. For e.g. An XML document may use this schema as the default schema. Default schema is defined simply by using 'xmlns=' without a prefix.
It is important to remember that XML schema is itself an XML document. Hence a schema can contain a namespace attribute and also a targetNamespace attribute. Typically they are the same.

Jotting down links where more explanation is given:
http://www.coderanch.com/
http://www.oracle.com/technology/

Creating a self-signed certificate

Jotting down the quick commands in .NET and JDK frameworks that can be used to create a self signed certificate.

In the .NET framework, open the Visual Studio command prompt and type the following:

makecert -r -pe -n "CN=www.yourserver.com" -b 01/01/2000 -e 01/01/2036 -eku 1.3.6.1.5.5.7.3.1 -ss my -sr localMachine -sky exchange -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12

Just replace the CN with the name or IP of your server. The certificate would be created in the default personal store on Windows. Go to MMC and add the 'certificates' snap-in.

In JDK first use the keytool utility to generate a certificate in the keystore.

keytool -genkey -alias myalias -keystore .keystore

You would be prompted to enter the CN and other details. Once the cert is stored in the keystore, it can be exported as a file.

keytool -storepass password -alias myalias -keystore .keystore -export -rfc -file outfilename.cer

.NET Web Methods Tips and Tricks

It is possible to pass a parameter by reference to a Web Method? I thought this does not make sense, but behind the covers the .NET webservice proxy and runtime make this possible.

A detailed example with sample source code can be found here.

Also given in the above link is an example of supporting polymorphism in Web Methods.

Open Source Application Management Software

We needed a simple application mangement tool for monitoring our Java applications running on Tomcat servers. My team evaluated the various free/open-source options and gave a demo of the following 2 tools that proved to be interesting.

1. ManageEngine Applications Manager - This is free for use (upto 5 monitors). Very user friendly and professional look and feel.

2. JManage - (open source. Good support for Java console applications)

Passing large .NET datasets across layers

If your .NET webservice is returning datasets, then you might face performance problems with large payloads consuming the network bandwidth. There are a few strategies around this problem.

1. Convert the dataset into a byte[] using the Binary Formatter. Compress this byte[] using the Deflate classes in .NET. On the client side, the reverse process needs to be done.

Peter Bromberg has written a nice article explaining the details of this strategy.

2. Convert the dataset into a XML string and compress the XML string. This discussion forum thread contains sample source code for the same.