Thursday, March 13, 2008

Important Tip for Clustered OracleAS

by Eduardo Rodrigues

Hi everybody!

Recently we've upgraded OracleAS environments in one of our biggest clients here in Brazil from version to version To do that, we've applied patchset #6148874 (which can be downloaded from Oracle Metalink) carefully following all the instructions found in its installation guide. As expected, the patchset application went perfectly fine and all of our configured containers and applications were still there, up and running, and everybody was happy... until we noticed that one of our oldest and most annoying problems was still there: using ASControl to manage our clustered production environment sometimes was just impossible due to its unacceptable performance when trying to access anything in cluster topology context.

Unfortunately, the only workaround we knew was to restart ASControl application in the cluster's "master node" (the node elected to run the ascontrol application). Fortunately, today I found the problem and also the solution for this situation.

There's a documented bug on Metalink (bug #6601697) explaining that this problem is caused by the underlying RMI communication between the ASControl and the other components of the cluster. ASControl uses RMI protocol to connect to other nodes, makes some request and then waits for a response. The problem is that there's no timeout for this response waiting. If one or more components are in a heavy load situation, for example, ASControl will keep waiting for their responses, which can take too long, causing the impression that ASControl stopped working.

In fact, this problem is known since version but it seems that patchset does not include the correction as it should. So, for now, the solution is to apply the one-off patch #6124143, also found on Metalink. This patch is originally targeted to OracleAS but, as written in bug #6601697, it's ok to apply it on version though. We just have to follow these simple instructions, found in the bug, before actually applying it:
  1. Edit the patch file etc/config/actions and replace by

  2. Edit etc/config/inventory and replace by

Once it's successfully applied, we must use ASControl to navigate to the OC4J instance where ascontrol application is running (typically the "home" instance). Click on Administration -> Server Properties -> Command Line Options -> Start-parameters: Java Options and add "-Drmi.client.connection.timeout=X", where "X" should be the number of seconds to wait for a RMI response before timeout. The bug suggests 5 seconds.

We've successfully applied this one-off patch in our OracleAS environment and, until now, it seems to have solved the problem.

I feel this might be a problem faced by many other OracleAS 10.1.3 users so I hope this post can help them.

Cheers and... keep reading!


Dan Norris said...

Thanks for the tip, I'm sure this will come in very handy at some of my clients as we upgrade to

Anonymous said...

Hi Sir,I am very new to this,
i need to add some java option in oracleAS, but i am not able to Server properties in that.
I read this but I dont know exactly, how to go that patch file etc\config\actions or inventory.
Could you please help me, how to resolve this.