Friday, December 30, 2011

OutOfMemoryError while using StringBuilder/StringBuffer

I was helping a friend debug a OutOfMemory exception in a Java web application. The program made heavy use of StringBuilder and was appending a large number of strings. An entire record set (containing thousands of records) was essentially converted into an XML string.
Strangely when the OOM error occured, there was still plenty of heap memory available. Furthur deep-dive debugging and some googling around, taught a few important lessons.
  1. Whenever the internal buffer capacity of StringBuilder/StringBuffer is exceeded, then the next character array size it creates is twice of the original size. A good blog explaining this is here. Hence it is better to initialize the initial capacity of the StringBuilder to a reasonable value beforehand.
  2. StringBuilder needs a continuous block of memory for further allocation. For e.g. you may have 20MB free heap space, but it may be fragmented. Hence even a 5MB StringBuilder allocation may fail and result in a OOM error. Links to forums - Link 1  Link2 Link3
  3. Try to use a 64-bit machine, as there are no practical limitations for the heap memory allocation and also it is much easier to find a continous block of memory due to 64-bit addressing.
  4. Alter the design of the program to not store the string in memory, but in a file. Alternatively stream it directly to the HTTP response.