Some interesting code..to tickle your brain cells

Found this code over the internet :) ...Took me some time to debug. Please copy-paste the below code in eclipse and start analysing - why this is happening?

public class TimePass {
    public static void main(String[] args) {
        if ( false == true ) { //these characters are ignored?: \u000a\u007d\u007b
            System.out.println("false is true!");
        }
    }
}

Hint: linefeed and curly braces :)

Using Netty - Lessons learned

We were building a ISO-8583 equivalent card transaction simulator using the powerful Netty framework.The simplicity of the netty design is a great wow factor. Also most of the complexities of NIO are abstracted by the framework by concepts such as Channel, ChannelBuffer, ChannelEvent, Decoders/Encoders, etc.

While we were using the LengthFieldBasedFrameDecoder class, we faced an intriguing problem. The socket client was sending the length of the record in the first 2 bytes. For e.g. "11abcdefghijk"
But strangely, the length derived by the  BigEndianHeapChannelBuffer was something different.
To get to the bottom of this, we enabled debugging and saw the raw buffer byte array that reached the server.

The buffer array for "11abcdefghijk" was [49, 49, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107]
The first 2 bytes contained the unicode charset number of the character '1'. Hence the lenght was also encoded from a character into bytes using the default charset, which resulted in a very large number.. Obviously the LengthFieldBasedFrameDecoder failed to decode the message.

To get around this problem, we had to send the first 2 bytes without any characted encoding. We achieved this using unicode escape sequences for the lenght field - i.e. \u0000 and \u000b; essentially 0 and 11.
The buffer array for "\u0000\u000babcdefghijk" was [0, 11, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107]

Now the  netty decoder would work. But this approach has a serious flaw...for 2 chars - \u000A and \u000D. These chars correspond to \n (linefeed) and \r (carriage return) and hence would not compile itself as the Unicode escapes are pre-processed before the compiler is run.

Hence the best approach is to write the lenght field to the socket stream without using any charset encoding (write raw bytes) and then write the record as a string.

An alternate workaround is to write your own decoder, which too is very simple. 

Given below is a sample decoder that extracts the first 2 chars of a string for the length and returns the buffer upstream.

-----------------------------------------------------------------

import java.util.logging.Logger;

import org.jboss.netty.buffer.ChannelBuffer;
import org.jboss.netty.channel.Channel;
import org.jboss.netty.channel.ChannelHandlerContext;
import org.jboss.netty.handler.codec.frame.FrameDecoder;
import org.jboss.netty.util.CharsetUtil;

public class NarenLengthFieldDecoder extends FrameDecoder {

    private static final Logger logger = Logger.getLogger(NarenLengthFieldDecoder.class.getName());

    public int lengthFieldLength = 2; // some default value

    public NarenLengthFieldDecoder(int length) {
        this.lengthFieldLength = length;
    }

    @Override
    protected Object decode(ChannelHandlerContext ctx, Channel channel, ChannelBuffer buffer) throws Exception {
        // wait until the length prefix is available.
        if (buffer.readableBytes() < lengthFieldLength) {
            // return null to inform frame decoder that frame is not yet
            // complete and to continue reading data
            return null;
        }

        // read length field...create a byteArray for the length field and copy
        // the first bytes into it..
        byte[] array = new byte[lengthFieldLength];
        buffer.readBytes(array); // Imp: readBytes also forwards the readerIndex

        // Mark the current buffer position before reading the length field
        // because the whole frame might not be in the buffer yet.
        // We will reset the buffer position to the marked position if
        // there's not enough bytes in the buffer.
        buffer.markReaderIndex();

        int dataLength = getLength(array);// length of the record

        // wait until the whole data is available.
        if (buffer.readableBytes() < dataLength) {
            // The whole bytes were not received yet - return null.
            // This method will be invoked again when more packets are received
            // and appended to the buffer.

            // Reset to the marked position to read the length field again next
            // time.
            buffer.resetReaderIndex();
            return null;
        }

        // forward remaining buffer to higher up handlers
         // There's enough bytes in the buffer. Read it.
         ChannelBuffer frame = buffer.readBytes(dataLength);
         // Successfully decoded a frame.  Return the decoded frame.
         return frame;
    }

    /**
     * Returns the first 2 or 3 sequence of bytes...that specify the length of
     * the record
     * 
     * @param array
     *            The byte array containing the length
     * @return An integer representing the length of the record
     */
    public int getLength(byte[] array) {
        String temp = new String(array, CharsetUtil.ISO_8859_1);

        int length = 0;
        try {
            length = Integer.parseInt(temp);
        } catch (Exception ex) {
            logger.info("Could not parse the length field of the record >>>" + temp);
        }
        return length;
    }
}

UIaaS - UI as a Service

Some time back, Salesforce popularized the term - UIaaS (UI as a Service), when they launched VisualForce.

UIaaS essentially means the ability to create new user interfaces using pre-built components. UI components could be pages, controls, static resources, etc. So the concept is to not start from scratch, but use off-the-shelf UI widgets. Typically UI designers are web-based and allow for in-browser UI design.

The underlying technologies for creating UIaaS are –
Server Side: JSF, Portlets, .NET WebParts
Client Side: Dojo controls, JQuery controls, Ext-JS controls, etc.

IMHO, the term is just old wine in new bottle. We used to have concepts of UI Widget Factory that encompasses creating reusable widgets and storing them (with meta-data) in a repository. Application developers would then pick and choose their widgets and design new pages. The reusable widgets could be technical widgets such as a "tab-bar","menu-bar","calendar", etc. or business widgets such as "Healthcare Provider Search", "Google Maps Overlay", etc.

HSQL in-memory database

Recently, we were experimenting with the HSQL in-memory database for a particular usecase. It was interesting to observe the default behaviour of persisting the database during a shutdown - The entire database was saved as a SQL script file ! When the server starts again, it loads the SQL script and fires all the CREATE TABLES and INSERT statements to recreate the database in memory.

Wikipedia gives a good overview of this HSQL feature and compares the default memory tables with cached tables. Snippet:

The default MEMORY type stores all data changes to the disk in the form of a SQL script. During engine start up, these commands are executed and data is reconstructed into the memory. While this behavior is not suitable for very large tables, it provides highly regarded performance benefits and is easy to debug.

Another table type is CACHED, which allows one to store gigabytes of data, at the cost of the slower performance. HSQLDB engine loads them only partially and synchronizes the data to the disk on transaction commits. 

I was a bit concerned about the viability of all-in-memory tables for large datasets, but it looks like HSQL is being actively used in projects where millions of rows are stored in memory. The only limitation is that of the Java Heap size that can be configured to be very large on a 64-bit machine.

It is possible to convert from memory tables to cached tables. You need to shutdown the database first. Then edit the .script file and modify the line "CREATE MEMORY TABLE" to "CREATE CACHED TABLE".

Snippet from the FAQ page:

If only memory tables (CREATE TABLE or CREATE MEMORY TABLE) are used then the database is limited by the memory. A minimum of about 100 bytes plus the actual data size are required for each row. If you use CREATE CACHED TABLE, then the size of the table is not limited by the memory beyond a certain minimum size. The data and indexes of cached tables are saved to disk. With text tables, indexes are memory resident but the data is cached to disk.


The current (2.0) size limit of an HSQLDB database is 16GB (by default) for all CACHED tables and 2GB for each TEXT table. If you use large MEMORY tables, memory is only limited by the allocated JVM memory, which can be several GB on modern machines and 64bit operating systems.

The statements that make up the database are saved in the *.script file (mostly CREATE statements and INSERT statements for memory tables). Only the data of cached tables (CREATE CACHED TABLE) is stored in the *.data file. Also all data manipulation operations are stored in the *.log file (mostly DELETE/INSERT) for crash recovery. When the SHUTDOWN or CHECKPOINT command is issued to a database, then the *.script file is re-created and becomes up-to-date. The .log file is deleted. When the database is restarted, all statements of the *.script file are executed first and new statements are appended to the .log file as the database is used. A popular use of HSQLDB is for OLAP, ETL, and data mining applications where huge Java memory allocations are used to hold millions of rows of data in memory.

One limitation of HSQLDB is that it currently does not support server side cursors. (This allows it to run without any writeable media). This means the result of a query must always fit in memory, otherwise an OutOfMemory error occurs. In the rare situation that a huge resultsets must be processed, then the following workaround can be used: Limit the ResultSet using Statement.setMaxRows(1024), and select multiple 'smaller' blocks. If the table is for example 'CREATE TABLE Test(Id INT IDENTITY PRIMARY KEY, Name VARCHAR)' then the first block can be selected using 'SELECT * FROM Test'. The biggest ID should be recorded and the next block should be selected using 'SELECT * FROM Test WHERE Id>(biggest_id)' until no more records are returned. Don't forget to switch off the limit using setMaxRows(0).

Dynamic reports using Jasper

Today, I spend a good 1 hr brainstorming the design to build dynamic reports in Jasper. A lot of customers demand the need to create dynamic reports at run-time; i.e. choose the number of columns, sorting order, group-by, etc.

Now as we know in Jasper, the first thing that needs to be done is to create the *.jrxml file. Typically for static reports, this is done through the report designer. But for dynamic reports, we need to create or modify a part of this jrxml file at run-time based on the user's input. For doing this, we have the the following options.

1) Template Engine: Here we would have a base jrxml file and then manipulate the base template file using a template engine such as 'Velocity' of 'FreeMarker'. The trick would be to have all possible mark-up in the base template and then remove sections as required. The drawback of this approach is that this will work only for simple dynamic requirements; such as add/remove columns. But if we need to add dynamic groups/sub-totals, then it might become cumbersome.

2) JasperDesign Object: A JasperDesign object is a run-time in-memory representation of the jrxml object. We can have a base jrxml and then load it into a JasperDesign object and then manipulate the object. A good example explaining this is available here. This is a good trade-off as you have some basic 'layout logic' in the jrxml template and are just manipulating the dynamic part of the 'layout-logic' using the JasperDesign API code.

Though it is possible to fully create the 'layout logic' through code, it would result in a maintenance nighmare for any small change in the future !!!! The Jasper library also has an example that creates the full jrxml file from scratch.

3) DynamicJasper Library: One alternative approach is to use a far simpler library such as DynamicJasper to create the report template from 100% pure Java code. This API is a bit more high-level than using the low level JasperDesign API as it makes many default layout assumptions that should suffice for 99% of use-cases.
Also you can use a base template *.jrxml file in which common styles, company logo, water mark, etc can be pre-defined. It also supports "Style library" from jrxml files.

Validations - on client side or server side or both?

A few years back, many developers spend a lot of time in coding validation rules for web forms - both on the client side as well as the server side. This was very tedious and a few lazy developers would just write JavaScript validation and not write server-side validation code; thus exposing a serious security flaw in the application.
Good design warrants us to apply the principle of 'security in depth'.

But today, most of the web-based MVC frameworks have OOTB support for validations - both on the client side and server side; with minimal coding. The basic design concept is to annotate your domain objects with validation constraints and then let the framework create the JS code for client side validation and use the framework interceptors for server side validation.

Struts-2 is a popular java web MVC framework that supports this feature. In fact, there is a JSR specification on the usage of annotations for bean validations called JSR 303. Struts-2 also has a plug-in for OVal that implements JSR-303.

In the .NET world, ASP.NET MVC framework also supports this feature of annotation-based validations.

Object Model or Data Model - What comes first?

Throughout my career, I have asked this question multiple times to myself - what to model first? The database entities or the object model? Start with ER diagrams (conceptual, logical) or UML class diagrams.

Well, the answer is that - it depends. Many a times, it also depends on the culture of the organization. In many organizations, the 'data' teams are more strong and powerful and insist on ER modeling first. It is possible to run both these work-streams in parallel, as there is a lot of conceptual common-ness during the initial domain modeling. What bugs me is the differences in the semantics of UML and ER. Does not make sense to do both at the conceptual level.

Now over the past few years, there have been a plethora of ORM tools that bridge the object-relational impedance mismatch. These ORM tools have a host of features that allow us to just work with the object model and abstract away the creation of the data model.
For e.g. Entity Framework 4.0 has full support for 'code-first' approach that is detailed in this article.
I simply loved the 'convention-over-configuration' approach in EF 4.0. These features enable us to only work with the object model and not worry about the data schema at all. Such features would suffice for 70-80% of business cases, I believe. 

A snippet from the article will give the reader an idea of the abstraction that is provided -

In addition to supporting a designer-based development workflow, EF4 also enables a more code-centric option which we call “code first development”. Code-First Development enables a pretty sweet development workflow. It enables you to: 

• Develop without ever having to open a designer or define an XML mapping file 
• Define your model objects by simply writing “plain old classes” with no base classes required 
• Use a “convention over configuration” approach that enables database persistence without explicitly configuring anything 
• Optionally override the convention-based persistence and use a fluent code API to fully customize the persistence mapping

What exactly is Domain Driven Design (DDD)?

We have been using many of the principles and patterns of DDD over the past many years. During domain modeling we have often used the concepts of boundary contexts, entity objects, value objects, aggregates and repository pattern, etc.

But based on my humble experience, I think DDD is much more than the usage of these patterns. DDD is a "thought-process" - the way you think about the problem domain, the way you interact with the domain experts & business stakeholders and the way you articulate the technology realization of the business need. This in a nutshell is the greatest boon of following DDD. The business and IT speak the same 'ubiquitous' language and this in turn bridges the "Business-IT gap" :)

That's the reason, the famous DDD book by Eric Evas states that DDD tackles Complexity in the Heart of Software. Mapping your software model as close to the real-life domain as possible helps us in managing the complexity of our design solutions.

The Microsoft Spain team has some pretty good documentation on this philosophy of DDD, which is available for download here. Also a neat ASP.NET example of a n-tiered DDD application is available for download.

Byte code instrumentation and the ORM magic

All ORM tools use some kind of byte-code instumentation to do the persistance magic behind the scenes. But as an architect, it is important to understand what Hibernate or any JPA tool does to the entity classes?

Hibernate 'enhances' entity classes at runtime using a byte-code library called Javaassist. For e.g. it adds a '_dirty' flag to each field. It also adds a '_loaded' flag for each field to support lazy loading. A good blog explaining these concepts is here. So Hibernate reads the XML configuration or obtains annotations at runtime using reflection to apply byte-code instumentation.

There are various ways of doing byte-code instumentation using libraries such as CGLib, ASM, Javaassist, etc.
This byte-code enhancement can be done at compile-time or run-time. For Hibernate, besides a few special cases which require compile time 'enhancement' to byte-code; all common scenarios can be satisfied with runtime instrumentation.

The following link gives a good overview of all the enhancement options available in JPA.
http://openjpa.apache.org/builds/1.2.1/apache-openjpa-1.2.1/docs/manual/ref_guide_pc_enhance.html

In the .NET world, NHibernate uses the Linfu or Castle Dynamic Proxies byte-code enhancement providers. 
http://nhforge.org/blogs/nhibernate/archive/2008/11/09/nh2-1-0-bytecode-providers.aspx

Mapping between Entity Objects and DTOs

Very often, we need to map between our Entity objects and DTO's. This mapping code can be quite tedious to write.
There is a lot of hot debate on whether to use DTO's or just pass the entity objects directly to the view or webservices. There are pros and cons of each approach. Some good links on this debate are listed here:

Data Transfer Object - MSDN

http://stackoverflow.com/questions/5216633/jpa-entities-and-vs-dtos

Pros and Cons of Data Transfer Objects 

If you are using popular ORM tools such as Hibernate, iBatis or any other JPA complaint tool, then it may not even be possible to use the Enrity objects directly in your service or presentation tier. This is because these ORM toolkits typically use some kind of byte-code instrumentation to do the persistance magic behind the scenes. A good link explaining this is available here.

To avoid the drudgery of writing the 'Adapter/Mapping' code for each Entity object and DTO object, we can use some cool AutoMapper tools. These AutoMapper tools work on Reflection techniques and automatically map the source and target object properties. Custom mapping is supported using XML configuration or through code.

In the .NET world, there is a popular AutoMapper tool that has become the de-facto standard for a lot of .NET projects. In the Java world, there are 2 popular alternatives - Dozer and ModelMapper.
I found Dozer to be more comprehensive with some pretty good features. The usage is super-simple if you use the Singleton Wrapper and place the custom mapping file in the classpath.

If you are using the Spring Framework, then the 'BeanUtils' class has some simple static methods to copy properties from one object to the other. 

Performance benchmarks

Every development project needs a formal performance engineering process - one that emphasizes on early performance testing and benchmarking.

For performance benchmarks, it is recommended to do a shallow and wide implementation of a few critical use-cases and then run the load tests against the target hardware. These test results would help in some basic capacity planning.

But what if you have to do some initial rough capacity planning to allocate budgets and do not have the time to do a formal benchmarking exercise. It is here that standard performance benchmarks help. These standard performance benchmarks take a sample transactional use-case (e.g. Order Processing System) and run this workload on various platforms to gather statistics. There are 2 standards that are quite popular -

1. TPC (Transaction Processing Performance Council) - (TPC) is a non-profit organization founded to define transaction processing and database benchmarks and to disseminate objective, verifiable TPC performance data to the industry. TPC-C is the benchmark for OLTP workloads. 


2. SPECjEnterprise2010 - SPECjEnterprise2010 is an industry-standard benchmark designed to measure the performance of application servers conforming to the Java EE 5.0 or later specifications.

Interesting results of the performance benchmarks on various hardware can be found here:
http://www.tpc.org/tpcc/results/tpcc_perf_results.asp
http://www.spec.org/jEnterprise2010/results/jEnterprise2010.html

For the past few years, the Java Day Trader application and its .NET equivalent StockTrader application have been used by vendors to compare the performance of Java vs .NET on their respective platforms. Jotting down some links that point to some interesting debatable data :)

http://www.ibm.com/developerworks/opensource/library/os-perfbenchmk/index.html

http://blogs.msdn.com/b/wenlong/archive/2007/08/10/trade-benchmark-net-3-0-vs-ibm-websphere-6-1.aspx

http://msdn.microsoft.com/en-us/netframework/bb499684.aspx

https://cwiki.apache.org/GMOxDOC22/daytrader-a-more-complex-application.html

JavaDB (Derby) in JDK 1.6 and above

JDK 1.6 and above ship with a default pure Java database called as "JavaDB". Is is based on the open source Apache Derby project.

By default, on a Windows platform JavaDB gets installed at "C:\Program Files\Sun\JavaDB".
Set the 'DERBY_HOME' system property to this path. Also put 'DERBY_HOME/bin' in the PATH property.

There is a good tutorial here that should get you up and running with JavaDB in 10-15 mins :)

Derby does not have a default GUI admin tool, but one can use many third-party tools such as SQuirrel and others. I think JavaDB provides a good alternative to MySQL for some scenarios.

What is a framework?

When someone says they have defined a "framework", what does it mean? Is a framework just a library of resuable components? Or is it something more?

There is a good article on CodeProject on the same topic - http://www.codeproject.com/Articles/5381/What-Is-A-Framework

The book "Applying UML and Patterns" by Craig Larman also gives a very good understanding of the concept. Jotting down snippets from both these resources, in my own words.One may consider them the 10 guiding principles while designing a framework.

1. At the risk of oversimplification, a framework can be defined as a cohesive set of classes/interfaces that provide services for the core part of a logical subsystem. 
2. A framework contains both concrete and abstract classes that define interfaces to conform to, and other object interactions.
3. Frameworks usually allow the end-users to define sub-classes of existing framework classes for customization and extension of the framework services.
4. A framework enforces adherence to a consistent design approach.
5. Relies on the "Hollywood Principle" - "Don't call us, we will call you". This pattern is also called as IoC (Inversion of Control). 
6. A framework makes it easier to work with complex technologies.
7. A framework reduces/eliminates repetitive tasks.
8. A framework is often re-usable across multiple scenarios -  regardless of high level design considerations. Frameworks offer a higher degree of reuse - much more than individual classes.
9. A framework forces the team to implement code in a way that promotes consistent coding, fewer bugs, and more flexible applications.
10.  A framework can be used as a software building block in the system architecture definition. 

Eclipse Memory Analyser

Read the following good reviews on Eclipse Memory Analyser. Looks like it can read both SUN JVM HPROF memory dumps as well as IBM JDK dumps.

http://memoryanalyzer.blogspot.in/2010/01/heap-dump-analysis-with-memory-analyzer.html

http://memoryanalyzer.blogspot.in/2010/02/heap-dump-analysis-with-memory-analyzer.html#more

http://www.eclipse.org/mat/

Some other interesting blogs that would help us resolve OOM errors :)

http://www.rallydev.com/engblog/2011/09/20/outofmemoryerror-fun-with-heap-dump-analysis/

http://www.rallydev.com/engblog/2012/03/16/java-memory-problems-why-is-my-heap-exhausted/

There is also a good article that contains sample code to simulate a Java OOM error and uses the Memory Analyser tool to identify the root cause of the error - http://www.javacodegeeks.com/2012/05/gc-overhead-limit-exceeded-java-heap.html

C heap vs Java heap

Found this interesting discussion on StackOverFlow around C Heap and Java Heap.
A good read and its important to understand that the JVM is also ultimately a C program :)
 
http://stackoverflow.com/questions/78352/what-runs-in-a-c-heap-vs-a-java-heap-in-hp-ux-environment-jvms

RAID basics

Found this good blog that explains in simple terms, the various levels of RAID (Redundant Array of Independent Disks).

RAID 10 has become the defacto standard for relational databases, due to the excellent redundancy and performance given by them.  In RAID 10 (also known as 1+0), blocks are mirrored and also striped. 

'volatile' keywork in Java

Found this excellent article on the web explaining the 'volatile' keyword in Java and how it can be used for concurrency. The tutorial also explains the changes to the volatile keyword functioning in Java 5.

Also found it interesting to understand what 'livelock' is? We often encounter dead-lock and thread starvation in parallel programming, but livelock is also possible :)

Difference between Concurrent Collections and Synchronized Collections in JDK

Traditionally, we have also used object locks (semaphores) and synchronized methods to make our collections thread-safe. But having an exclusive lock on an object brings in scalability issues.

Hence the latest versions of JDK have a new package called "java.util.concurrent". This package contains many new collections objects that are thread-safe, but not so because of synchronization :)

More details at this link: http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/package-summary.html

Snippet from the above link:

The "Concurrent" prefix used with some classes in this package is a shorthand indicating several differences from similar "synchronized" classes. For example java.util.Hashtable and Collections.synchronizedMap(new HashMap()) are synchronized. 

But ConcurrentHashMap is "concurrent". A concurrent collection is thread-safe, but not governed by a single exclusion lock. In the particular case of ConcurrentHashMap, it safely permits any number of concurrent reads as well as a tunable number of concurrent writes.

 "Synchronized" classes can be useful when you need to prevent all access to a collection via a single lock, at the expense of poorer scalability. In other cases in which multiple threads are expected to access a common collection, "concurrent" versions are normally preferable. And unsynchronized collections are preferable when either collections are unshared, or are accessible only when holding other locks. 

Most concurrent Collection implementations (including most Queues) also differ from the usual java.util conventions in that their Iterators provide weakly consistent rather than fast-fail traversal. A weakly consistent iterator is thread-safe, but does not necessarily freeze the collection while iterating, so it may (or may not) reflect any updates since the iterator was created. 

Also a good post on Concurrency basics is available at: http://docs.oracle.com/javase/tutorial/essential/concurrency/memconsist.html (All chapters a must read :)

Another good blog that explains how ConcurrentHashMap maintains several  locks instead of one single mutex to deliver better performance.

Google Chart APIs

The last time (around 1 year ago), when I had evaluated Google Charts API, I was a bit disappointed. The Chart API only enabled you to embed an image (chart) that would be created on Google servers.

But looks like Google has completely revamped the concept and branding and have a brand new Chart API.

The new charting API looks cool and very easy to use. There are also samples and libraries that will help you write server side code to pass chart data to the client. I was particularly impressed with the Java DataSource library and Oracle PL/SQL library.

It is important to note that Google right now, does not allow these JS API to be downloaded offline and used in a web app that does not have internet connection (e.g. intranet applications). Please consider this constraint before you decide to use Google Chart Tools.

The older version of the image charts is still available here.There is also an online tool for quicking creating an image of a chart. Could be useful if you quickly want to create some stuff for your PPTs :)

JavaScript coding guidelines

With RIA applications becoming the norm, developers have to deal with a lot of JavaScript code. It is important to have proper coding conventions for JS too. Was glad to see a good document posted by Google on JavaScript coding conventions.

http://google-styleguide.googlecode.com/svn/trunk/javascriptguide.xml

Also, there are tools available to check the code quality of JavaScript code. Here are a few examples available:

http://docs.codehaus.org/display/SONAR/JavaScript+Plugin

http://jslint.com/


 

JAMon lives on !

I have been a big fan of JAMon tool for monitoring Java applications.It is lightweight and has a cool UI that gives you the stats you want. Also it is very clean and simple to use.

The last time I used JAMon was around 5 years ago. I was suprised to see that the project is still alive and kicking and has a few updates that make it even more interesting. I liked the concept of Listeners, which makes it easy to customoize JAMon.

The one thing that was missing in JAMon in the yesteryears was a consolidation app to read stats from multiple nodes in a cluster and present in on a unified dashboard. Luckily there seems to be a project called JARep that just does that. There is no much documentation available on JARep, but there is a good case study on DZone that explains how JARep works.

The case study is available at: http://architects.dzone.com/articles/case-study-performance-tuning--0

Though I have experiemented with a lot of other tools, I found JAMon to be the best and simplest. One can get it up and running in a project with a few minutes.

Passing wilcard * in Runtime.exec() command

One of the projects I was consulting on approached me with a peculiar problem.
The application was executing unix commands from a Java program using RunTime.exec() APIs. But strangely for some reason, the "rm" command was not working. The user account running the Java process had the necessary rights for deletion, so the problem was somewhere else.

A quick googling around answered the problem :)
http://www.coderanch.com/t/423573/java/java/Passing-wilcard-Runtime-exec-command

Its important to remember that when we pass wildcards to the RunTime.exec() API, it will treat it as a string only. It is the Unix shell that understands the "*" syntax.
Hence something like this would work: Runtime.exec(new String[] { "sh", "-c", "rm /tmp/ABC*" });

Cool library for Java NIO

I often felt the need to write a wrapper class libaray around Java NIO packages to reduce the complexity for developers.
I was pleased to find one opensource project that does just that :)

The Netty project provides a very clean and easy API for writing network applications. Very useful if we need to implement a customized protocol for special purpose.

Closed Loop vs Open Loop Models in Card Processing

Found this good article on the internet that describes the differences between closed loop and open loop models.
Excerpts from the article:

"Open-loop payments networks, such as Visa and MasterCard, are multi-party and operate through a system that connects two financial institutions—one that issues the card to the cardholder, known as the issuing financial institution or issuer, and one that has the banking relationship with the merchant, known as the acquiring financial institution or acquirer—and manages information and the flow of value between them.
In a typical closed-loop payments network, the payment services are provided directly to merchants and cardholders by the owner of the network without involving third-party financial institution intermediaries. Closed-loop networks can range in size from networks such as American Express and Discover, which issue cards directly to consumers and serve merchants directly."


The site also has another interesting link on how companies such as American Express make money and the competitive advantage they gain because of the closed loop model.

The Architecture of Open Source Applications

Found this cool book on the web, that explains the history behind many successful open source projects.

The chapters also contain a good amount of technical information - which was a pleasure to read :)

http://www.aosabook.org/en/index.html

Techniques for website design on all devices

Today, there is a growing demand to create websites that can render across multiple devices such as desktop browsers, tables and smart phones. New standards such as HTML5 help in this regard.

But it's important to apply proper design principles when we develop web pages that can reder across a wide array of devices. Found this cool article that describes a few techniques that can be used. The author of this blog 'Ethan Marcotte' has also written a book on this called "Responsive Web Design".

The core philosophy is how can we use CSS3/HTML5 to enable our application to detect the capabilities (size, resolution, JS support, etc.) of the browser and adapt the layout of the page accordingly. 

Lightweight UML sketching tool - UMLet

For years, I have been searching for a lightweight tool to quickly draw UML diagrams - in order to brainstorm an idea with my team. The traditional tools I have been using for UML were Rational Software Architect, IBM System Architect, Visual Studio 2010, Visio, ArgoUML, etc.

All the above tools are good, but are quite heavy to use and require installation. If you need a robust and quick UML sketching tool, then UMLet will blow your mind :)

I was particularly impressed with the simplicity of the tool. Drawing class diagrams, sequence/activity diagrams, package structures, deployment diagrams are a breeze....And it can run standalone as a JAR file or as an eclipse plugin. Loved the 'geeky' properties editor. U need to use it to appreciate it :)
It does not support code generation, as it was not the design intention of the tool.
Highly recommended for all agile architects!

How to ensure that IOCP is used for async operations in .NET?

In my last post, I had blogged about IO Completion Ports and how they work at the OS kernel level to provide for non-blocking IO.

But how can the 'average Joe' developer ensure that IOCP is being used when he uses async operations in .NET?

Well, the good news is that a developer need not worry about the complexities of IOCP as long as he is using the BeginXXX and EndXXX methods of all objects that support async operations. For e.g. SQLCommand has BeginExecuteReader/EndExecuteReader that you can use for asynchronously reading data from a database. FileStream, Socket class all have BeginXXX/EndXXX methods that use IOCP in the background. Under the bonnet,  these methods use IO completion ports which means that the thread handling the request can be returned to the threadpool while the IO operation completes. 

Some versions of Windows OS may not support IOCP on all devices, but the developer need not worry about this. Depending on the target platform, the .NET Framework will decide to use the IOCompletionPorts API or not, maximizing the performance and minimizing the resources.

An important caveat is to avoid using normal async operations for non-blocking IO - such as "ThreadPool.QueueUserWorkItem, Delegate.BeginInvoke", etc. because these do not use IOCP, but just pick up another thread from the managed thread pool. This defeats the very purpose of non-blocking IO, because then the async thread is drawn from the same process-wide CLR thread pool.

Non blocking IO in .NET (Completion Ports)

Non blocking IO is implemented in Windows by a concept called 'IO Completion Ports' (IOCP).
Using IOCP, we can build highly scalable server side applications that can perform asynchronous IO to deliver maximum throughput for large workloads.

Traditionally server side applications were written by assigning one thread to a socket connection. But this approach seriously limited the number of concurrent connections that a server can handle. By using IOCP, we can overcome the "one-thread-per-client" problem, because 'worker' threads are not blocked for IO. Rather there is a separate pool of IO threads called 'Completion Port Threads' that wait on a special kernel level object called 'Completion Port'.

A completion port is a kernel level object that you can bind with a file handle - either a file stream, database connection or a socket stream. Multiple file handles can be bound to a single completion port. The .NET CLR maintains its own completion port and can bind any file handle to it. Each completion port has a queue associate with it. Once a IO operation completes, a message (completion packet) is posted to the queue. IO threads block or 'wait' on this completion port queue, till a message is posted. The waiting IO threads (a.k.a completion port thread) pick up the messages in the queue in FIFO order. Hence any thread may handle any completion message packet. It is important to note that threads are 'woken' in a LIFO order, so chances are that caches are still warm.

The following links throw more light on this:
http://blog.stevensanderson.com/2008/04/05/improve-scalability-in-aspnet-mvc-using-asynchronous-requests/
http://www.codeproject.com/Articles/1052/Developing-a-Truly-Scalable-Winsock-Server-using-I

Why does the .NET Thread Pool have a separate worker thread pool and a Completion Port pool?
I believe that technically there is no fundamental difference in the nature of the threads associated with each pool. Worker threads are meant to do active work, where as Completion Port threads are meant to wait on completion ports. Since IO threads wait on CPs, they may block for longer periods of time. Hence the .NET framework has created separate categories for them. If there was a single pool, then there could be a situation where a high demand on worker threads exhausts all the threads available to dispatch native I/O callbacks,
potentially leading to deadlock.

Looks like in IIS 7, the threading model has undergone drastic changes. More info available here. 

Why no delegates in Java? And will Closures come to Java?

Having worked across Java and .NET platforms, I often compare the features of one over the other. One of the interesting features of the .NET platform is the concept of 'delegates'.

At first, a Java guy may take some time to understand the concept of delegates, but once you are hooked on to it, you tend to use it everywhere...because it is so convienient. The .NET framework uses delegates extensively throughtout its event framework.Java folks have traditionally used the 'Listener' interface pattern for eventing. Even concurrent/parallel libraries in .NET heavily use delegates, whereas Java folks have to still stick with interfaces :(  The closest equivalent to delegates in Java is the anonymous inner class - which IMHO is messy to read and write.

StackOverFlow has a series of interesting discussion threads on this topic:
http://stackoverflow.com/questions/44912/java-delegates
http://stackoverflow.com/questions/2635013/why-not-net-style-delegates-rather-than-closures-in-java
http://stackoverflow.com/questions/1340231/is-there-an-equivilent-of-c-sharp-anonymous-delegates-in-java
http://stackoverflow.com/questions/1973579/why-doesnt-java-have-method-delegates


Another interesting feature that many dynamic languages have is 'closures'. A closure is similar to the concept of delegate, but they are not quite the same. Martin Fowler has a good bliki post explaining the concept of Closures and the difference compared to delegates.

.NET supports both closures and delegates. Found this good article explaning closures in .NET.