OBIEE 11.1.1 - Error: "Too many running queries. Server is too busy to process any more queries at this time"

Symptoms:

 

During normal users load, OBIEE fails frequently with the following error:

"OAMP2OPY:GIQKCQ7 Too many running queries. Server is too busy to process any more queries at this time."

Possible Causes / Solution:

1. The Oracle Database instance is configured by default to run only 150 PROCESSES; the maximum load for dedicated connections is determined by the PROCESSES parameter. When the listener believes the current number of connections has reached maximum load, it may set the state of the service handler for an instance to "blocked" and begin refusing incoming client connections with the following error (in biserver1.log set in debug level):

"TNS-12516 TNS:listener could not find instance with matching protocol stack"

Tip: Verify the Oracle database processes parameter setting.  It is also a good idea to make sure the Oracle database sessions and open cursors settings are large enough.

2. Increase the number of concurrent client connections (sessions) to OBI server. Each client request to OBI Server uses one SERVER_THREAD. Each SERVER_THREAD uses 0 or more DB_GATEWAY_THREADs depending upon the number of DB queries executed. Following NQSConfig.INI parameters need to be tuned (refer to OBIEE tuning guide for details on initial values to be set):
[ SERVER ]
MAX_SESSION_LIMIT
SERVER_THREAD_RANGE
DB_GATEWAY_THREAD_RANGE

3. Cleanup Web catalog via instanceconfig.xml using the steps documented in "Validating the Catalog" in System Administrator's Guide for Oracle Business Intelligence Enterprise Edition.

4. In instanceconfig.xml file for each instance, add the following parameters within Cache/Query tags:

<Cache>

       <Query>

           <MaxEntries>10000</MaxEntries>

           <AbsoluteMaxEntries>20000</AbsoluteMaxEntries>

           <CruiseEntries>3000</CruiseEntries>

           <ForceLRU>true</ForceLRU>

       </Query>

</Cache>

5. Make sure you have enough system resources available on BI Servers (sawserver, nqsserver).

 6. Unstable network connection link between OBIEE servers and Database server.