先说下oem的结构，三个oms服务器(04，05，06)，repository database是个三节点的RAC(A，B，C)，现在04上的oms服务可以起来，oem也能用，但是05，06上的oms服务都起不来，提示信息如下：
./emctl start oms
Oracle Enterprise Manager 11g Release 1 Grid Control
Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
Starting WebTier...
WebTier Successfully Started
Starting Oracle Management Server...
Oracle Management Server Could Not Be Started
Oracle Management Server is Down
Please check /usr/local/oracle/oem/oms11g/em/EMGC_OMS1/sysman/log/emctl.log for error details
log信息如下：
2010-09-26 01:50:18,743 [Thread-1] INFO wls.OMSController run.1094 - Initializing WebLogic Scripting Tool (WLST) ...
2010-09-26 01:50:18,746 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:42,221 [Thread-1] INFO wls.OMSController run.1094 - Welcome to WebLogic Server Administration Scripting Shell
2010-09-26 01:50:42,224 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:42,228 [Thread-1] INFO wls.OMSController run.1094 - Type help() for help on available commands
2010-09-26 01:50:42,232 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:43,615 [Thread-1] INFO wls.OMSController run.1094 - is_admin_host is FALSE
2010-09-26 01:50:43,619 [Thread-1] INFO wls.OMSController run.1094 - admin_start is FALSE
2010-09-26 01:50:48,376 [Thread-1] INFO wls.OMSController run.1094 - Connecting to Node Manager ...
2010-09-26 01:50:52,426 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:52,430 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:52,485 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:52,500 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:52,667 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:53,949 [Thread-1] INFO wls.OMSController run.1094 - Successfully Connected to Node Manager.
2010-09-26 01:50:53,955 [Thread-1] INFO wls.OMSController run.1094 - status of node manager:
2010-09-26 01:50:53,959 [Thread-1] INFO wls.OMSController run.1094 - Currently connected to Node Manager to monitor the domain GCDomain.
2010-09-26 01:50:53,964 [Thread-1] INFO wls.OMSController run.1094 - current status of EMGC_OMS2:
2010-09-26 01:50:53,970 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:53,973 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 01:50:53,978 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 01:50:53,982 [Thread-1] INFO wls.OMSController run.1094 - Starting server EMGC_OMS2
2010-09-26 01:50:53,985 [Thread-1] INFO wls.OMSController run.1094 - Starting server EMGC_OMS2 ...
2010-09-26 02:54:00,375 [Thread-1] INFO wls.OMSController run.1094 - Error Starting server EMGC_OMS2: weblogic.nodemanager.NMException: Exception while starting server 'EMGC_OMS2'
2010-09-26 02:54:00,378 [Thread-1] INFO wls.OMSController run.1094 - new status of EMGC_OMS2:
2010-09-26 02:54:00,384 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:00,386 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:54:00,387 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:00,389 [Thread-1] INFO wls.OMSController run.1094 - wait until EMGC_OMS2 becomes RUNNING
2010-09-26 02:54:00,391 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:00,392 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:54:00,394 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:00,395 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:54:10,416 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:54:10,423 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:10,424 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:54:10,426 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:10,428 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:54:20,433 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:54:20,440 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:20,442 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:54:20,443 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:20,445 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:54:30,450 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:54:30,456 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:30,458 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:54:30,459 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:30,461 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:54:40,466 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:54:40,473 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:40,474 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:54:40,476 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:40,477 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:54:50,483 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:54:50,489 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:50,490 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:54:50,492 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:54:50,494 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:55:00,499 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:55:00,505 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:00,507 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:55:00,508 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:00,510 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:55:10,516 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:55:10,525 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:10,527 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:55:10,529 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:10,531 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:55:20,535 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:55:20,542 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:20,544 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:55:20,546 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:20,548 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:55:30,552 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:55:30,558 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:30,559 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:55:30,561 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:30,562 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:55:40,568 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:55:40,574 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:40,576 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:55:40,577 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:40,579 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:55:50,584 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:55:50,591 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:50,592 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:55:50,593 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:55:50,595 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:56:00,601 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:56:00,607 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:00,608 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:56:00,610 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:00,611 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:56:10,617 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:56:10,625 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:10,627 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:56:10,628 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:10,635 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:56:20,636 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:56:20,646 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:20,649 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:56:20,651 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:20,654 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:56:30,657 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:56:30,663 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:30,664 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:56:30,666 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:30,668 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:56:40,676 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:56:40,682 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:40,684 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:56:40,685 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:40,687 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:56:50,692 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:56:50,699 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:50,700 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:56:50,702 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:56:50,703 [Thread-1] INFO wls.OMSController run.1094 - sleep for 10 seconds...
2010-09-26 02:57:00,709 [Thread-1] INFO wls.OMSController run.1094 - status of EMGC_OMS2:
2010-09-26 02:57:00,715 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:57:00,716 [Thread-1] INFO wls.OMSController run.1094 - FAILED_NOT_RESTARTABLE
2010-09-26 02:57:00,718 [Thread-1] INFO wls.OMSController run.1094 -
2010-09-26 02:57:00,720 [Thread-1] INFO wls.OMSController run.1094 - timed out waiting for EMGC_OMS2 to be RUNNING
2010-09-26 02:57:00,721 [Thread-1] INFO wls.OMSController run.1094 - status of node manager:
2010-09-26 02:57:00,722 [Thread-1] INFO wls.OMSController run.1094 - Currently connected to Node Manager to monitor the domain GCDomain.
2010-09-26 02:57:00,724 [Thread-1] INFO wls.OMSController run.1094 - 1
2010-09-26 02:57:00,725 [Thread-1] INFO wls.OMSController run.1094 - Successfully disconnected from Node Manager.
2010-09-26 02:57:00,727 [Thread-1] INFO wls.OMSController run.1094 - _END_
2010-09-26 02:57:00,727 [Thread-2] INFO wls.OMSController run.1094 - _END_
2010-09-26 02:57:00,979 [main] INFO wls.OMSController processCmd.988 - wlst process is finished
2010-09-26 02:57:00,981 [main] INFO wls.OMSController processCmd.990 - wlst process exited with code 1
2010-09-26 02:57:00,983 [main] INFO wls.OMSController processCmd.992 - wlst outstream joined
2010-09-26 02:57:00,985 [main] INFO wls.OMSController processCmd.994 - wlst errstream joined
2010-09-26 02:57:00,997 [main] DEBUG oms.StatusOMSCmd processStatusOMS.138 - HTTP port in emgc property file: 7201
2010-09-26 02:57:00,999 [main] DEBUG oms.StatusOMSCmd processStatusOMS.139 - HTTPS port in emgc property file: 7301
2010-09-26 02:57:01,001 [main] DEBUG oms.StatusOMSCmd processStatusOMS.155 - Using http port.
2010-09-26 02:57:01,610 [main] DEBUG oms.StatusOMSCmd processStatusOMS.195 - Attempting to connect to http://xxxxxxxx:7201/em/console/logon/logon
2010-09-26 02:57:01,781 [main] INFO oms.StatusOMSCmd processStatusOMS.233 - Failed to connect to em login page
2010-09-26 02:57:01,783 [main] INFO oms.StatusOMSCmd processStatusOMS.234 - Connection refused
2010-09-26 02:57:01,785 [main] INFO wls.OMSController statusOMS.950 - statusOMS finished with result: 8
2010-09-26 02:57:01,788 [main] INFO wls.OMSController startOMS.603 - startOMS finished with result: 1
2010-09-26 02:57:01,793 [main] DEBUG oms.StatusOMSCmd processStatusOMS.138 - HTTP port in emgc property file: 7201
2010-09-26 02:57:01,795 [main] DEBUG oms.StatusOMSCmd processStatusOMS.139 - HTTPS port in emgc property file: 7301
2010-09-26 02:57:01,798 [main] DEBUG oms.StatusOMSCmd processStatusOMS.155 - Using http port.
2010-09-26 02:57:01,801 [main] DEBUG oms.StatusOMSCmd processStatusOMS.195 - Attempting to connect to http://xxxxxxxx:7201/em/console/logon/logon
2010-09-26 02:57:01,819 [main] INFO oms.StatusOMSCmd processStatusOMS.233 - Failed to connect to em login page
2010-09-26 02:57:01,821 [main] INFO oms.StatusOMSCmd processStatusOMS.234 - Connection refused
2010-09-26 02:57:01,824 [main] INFO wls.OMSController statusOMS.950 - statusOMS finished with result: 8
google和metalink找了很多文档，还没解决这个问题，请大家帮忙看下。
