I added a sniffer to check what was going on. I could see that the custom error page was returned by the server in the HTTP response, yet IE was ignoring it. Googling around, I found out that IE has a feature turned on by default - Show friendly HTTP error messages (goto tools-> internet options-> advanced -> uncheck show friendly HTTP error message)
If I unchecked this checkbox, then the custom page was getting displayed correctly. But this was obviously not a solution that was practical. I perused thru more info about this IE feature here. IE uses this feature only if the response content for a 500 status-code response was less than 512 bytes.
So now, I suddenly had 2 solutions to the above problem (without having to change IE settings on the client)
Solution 1: Increase the content of the error page to more than 512 bytes.
Solution 2: Change the HTTP status code sent in the response header of the error page to HTTP 200.
Code snippet (at the top of the error jsp page):