I was getting confused if the heap size specified by -Xmx included the Perm size or it did not? The GC document on Sun's site showed the Perm generation to be part of the Heap size, but I think this is wrong. It seems that the Perm size is different from the Heap size. I found this link that confirms this belief.
The "permanent" generation in the Sun JVM is for class objects, method objects, static fields for those classes (just the static fields, not the objects referenced from the static fields: those are in the regular Java object heap because Java code allocates them), and random other bits like any interned Strings etc. So if U are using a lot of reflection in you code, then a lot of method objects and class objects are created at runtime. For e.g. XML binding frameworks such as Castor etc. use reflection by default.
Hence the total heap size used by the JVM is the Permanent Generation size + the -Xms/Xmx size.
The total memory used by the JVM process will be = Memory used by the JVM for internal management + memory allocated in native code + memory allocated for the permanent space.
This is the memory that U see in Task-Manager in Windows or 'top' on Solaris.
What happens when we allocate more Java heap than the max physical memory available?
The OS would start swapping the process to disk (depending on the Virtual memory configuration) if the total memory reserved by all applications and the OS in the system exceeds the physical RAM installed. This would seriously degrade the performance of the system.
*IMP*: If your application is throwing an OutOfMemoryError then it could be because the PermSize is full, rather than the heap size.