Problemas no startup do Oracle agent 12c: 'Target Interaction Manager failed at Startup java.lang.OutOfMemoryError: Java heap space' - gcagent_errors.log

Na tentativa de iniciar o agent do Oracle Enterprise Manager Coud Control em um servidor me deparei com alguns alertas informando estouro de memória do Java "OutOfMemoryError". Realizei o aumento de memória através do parâmetro agentJavaDefines encontrado dentro do arquivo de configurações $AGENT_HOME/sysman/configemd.properties, contudo o problema persistiu. 

Buscando no MOS encontrei a solução via Doc ID 1902124.1 - "EM12c: emctl start agent Fails With 'Target Interaction Manager failed at Startup java.lang.OutOfMemoryError: Java heap space' reported in gcagent_errors.log"

Abaixo visão do problema e solução.

Tentativa de startup (falha):

srvtst-banco=orcl-> $AGENT_HOME/emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation.  All rights reserved.
Starting agent .............................................. failed.
Fatal agent error: State Manager failed at Startup
Fatal agent error: State Manager failed at Startup
Fatal agent error: State Manager failed at Startup
EMAgent is Thrashing. Exiting watchdog
Consult emctl.log and emagent.nohup in: /u01/app/oracle/agent/agent_inst/sysman/log

Verificando log1:

srvtst-banco=orcl-> vi /u01/app/oracle/agent/agent_inst/sysman/log/emagent.nohup

2016-07-02 14:56:02,661 [1:main] WARN - Missing filename for log handler 'opsscfg'
Agent is going down due to an OutOfMemoryError

Verificando log2:

srvtst-banco=orcl-> vi /u01/app/oracle/agent/agent_inst/sysman/log/gcagent.log

2016-07-02 14:56:16,299 [1:main] FATAL - Fatal error: Target Interaction Manager failed at Startup
java.lang.OutOfMemoryError: Java heap space

Executando procedimento do Doc ID 1902124.1 

1. Parando todos os processos do agent:

$AGENT_HOME/bin/emctl stop agent
ps -ef | grep java | grep 'diretório do agent'
ps -ef | grep perl

Finalize qualquer processo JAVA / PERL ativo no diretório do Oracle agent (AGENT_HOME)

kill -9  numero_do_processo

2. Movendo arquivos antigos para um novo diretório:

srvtst-banco=orcl-> mv $AGENT_HOME/sysman/emd/state/* /orabackup/bkp_cloud/

3. Executando o "clearstate agent"

srvtst-banco=orcl-> $AGENT_HOME/bin/emctl clearstate agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation.  All rights reserved.
EMD clearstate completed successfully

NOTA: O agent mantém internamente informações e status dos componentes que ele monitora, afim de evitar um tráfego de rede desnecessário junto ao OMS, contudo se ocorrer alguma mudança no alvo monitorado o agent inicia o upload das informações. O comando clearstate agent obriga o agente a realizar uma nova leitura de cada componentes e reenvia-las ao OMS.

Segundo a DOC, este comando deve ser executado apenas quando o suporte da Oracle recomenda.

4. Fazendo backup do arquivo de parâmetros e aumentando a memória Java:

srvtst-banco=orcl-> cd $AGENT_HOME/sysman/config
srvtst-banco=orcl-> cp emd.properties emd.properties.bkp

atual:

#
# These are the optional Java flags for the agent
#
agentJavaDefines=-Xmx128M -XX:MaxPermSize=96M

para:

#
# These are the optional Java flags for the agent
#
agentJavaDefines=-Xmx512M -XX:MaxPermSize=96M

* alterado o valor do -Xmx de 128M para 512M

5. Iniciando novamente o agent:

srvtst-banco=orcl-> $AGENT_HOME/bin/emctl start agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation.  All rights reserved.
Starting agent ........................... started.

srvtst-banco=orcl-> $AGENT_HOME/bin/emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 5
Copyright (c) 1996, 2015 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version          : 12.1.0.5.0
OMS Version            : 12.1.0.5.0
Protocol Version       : 12.1.0.1.0
Agent Home             : /u01/app/oracle/agent/agent_inst
Agent Log Directory    : /u01/app/oracle/agent/agent_inst/sysman/log
Agent Binaries         : /u01/app/oracle/agent/core/12.1.0.5.0
Agent Process ID       : 18183
Parent Process ID      : 18003
Agent URL              : https://srvtst:3872/emd/main/
Local Agent URL in NAT : https://srvtst:3872/emd/main/
Repository URL         : https://srvtst:4900/empbs/upload
Started at             : 2016-07-02 15:02:31
Started by user        : oracle
Operating System       : Linux version 2.6.39-400.126.1.el5uek (amd64)
Last Reload            : (none)
Last successful upload                       : 2016-07-02 15:03:01
Last attempted upload                        : 2016-07-02 15:03:01
Total Megabytes of XML files uploaded so far : 0.41
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 48.47%
Collection Status                            : Collections enabled
Heartbeat Status                             : Ok
Last attempted heartbeat to OMS              : 2016-07-02 15:02:52
Last successful heartbeat to OMS             : 2016-07-02 15:02:52
Next scheduled heartbeat to OMS              : 2016-07-02 15:03:53

---------------------------------------------------------------
Agent is Running and Ready
srvtst-banco=orcl->

Espero ter ajudado.!