R12 Start Apache Throws an 'Error While Loading Shared Libraries: libdb.so.2' on Linux

After cloning an Oracle EBS Release 12 instance on a Linux server, following error occur while starting up the Apache service.

Apache/HTTP Startup Log:

Executing service control script:
/u01/R12EBS/inst/apps/r12hyupg_hydhost02/admin/scripts/adapcctl.sh start
script returned:                                                          
**************************************************** 
You are running adapcctl.sh version 120.6.12000000.4

Starting OPMN managed Oracle HTTP Server (OHS) instance ...
opmnctl: opmn is already running.                          
opmnctl: starting opmn managed processes...                
================================================================================
opmn id=hydhost02:6210                                                           
    0 of 1 processes started.                                                   

ias-instance id=r12hyupg_hydhost02.hydhost02.example.com
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--------------------------------------------------------------------------------
ias-component/process-type/process-set:                                         
    HTTP_Server/HTTP_Server/HTTP_Server/                                        

Error
--> Process (index=1,uid=453192016,pid=10658)
    failed to start a managed process after the maximum retry limit
    Log:                                                           
    /u01/R12EBS/inst/apps/r12hyupg_hydhost02/logs/ora/10.1.3/opmn/HTTP_Server~1


adapcctl.sh: exiting with status 0

adapcctl.sh: check the logfile /u01/R12EBS/inst/apps/r12hyupg_hydhost02/logs/appl/admin/log/adapcctl.txt for more information ... 

HTTP/Apache log file:

Logfile: /u01/R12EBS/inst/apps/r12hyupg_hydhost02/logs/ora/10.1.3/opmn/HTTP_Server~1

20/04/20 05:52:23 Start process
--------
/u01/R12EBS/inst/apps/r12hyupg_hydhost02/ora/10.1.3/Apache/Apache/bin/apachectl startssl: execing httpd
/u01/R12EBS/apps/tech_st/10.1.3/Apache/Apache/bin/httpd: error while loading shared libraries: libdb.so.2: cannot open shared object file: No such file or directory

Here I noticed that, the error is very clear about the Soft link libdb.so.2 is missing.

Status of the Apache/HTTP_server Services:  http web server unable to start due to missing library.

-bash-4.1$ ./adopmnctl.sh status apps/apps

You are running adopmnctl.sh version 120.4.12000000.3

Checking status of OPMN managed processes...

Processes in Instance: r12hyupg_hydhost02.hydhost02.example.com
-------------------+--------------------+---------+---------
ias-component      | process-type       |     pid | status
-------------------+--------------------+---------+---------
default_group      | oafm               |   10916 | Alive
default_group      | forms              |   10814 | Alive
default_group      | oacore             |   10696 | Alive
HTTP_Server        | HTTP_Server        |     N/A | Down


adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile /u01/R12EBS/inst/apps/r12hyupg_hydhost02/logs/appl/admin/log/adopmnctl.txt for more information ...

-bash-4.1$

Solution for this issue:

1. Shutdown all application services.
2. create soft link for library file  (ln -s /usr/lib/libgdbm.so.2.0.0 /usr/lib/libdb.so.2)
3. Startup all application services.

1) Stopped Application tier services.

2) Create Softlink for library file.

-bash-4.1$ ls -rlt libgdbm.so.2.0.0
-rwxr-xr-x. 1 ovsroot root 24800 Jul 23  2010 libgdbm.so.2.0.0
-bash-4.1$ ls -rlt /usr/lib/libdb.so.2
ls: cannot access /usr/lib/libdb.so.2: No such file or directory
-bash-4.1$

# ln -s /usr/lib/libgdbm.so.2.0.0 /usr/lib/libdb.so.2
#

-bash-4.1$ ls -rlt libgdbm.so.2.0.0
-rwxr-xr-x. 1 ovsroot root 24800 Jul 23  2010 libgdbm.so.2.0.0
-bash-4.1$ ls -rlt /usr/lib/libdb.so.2
lrwxrwxrwx 1 ovsroot root 25 Apr 20 06:08 /usr/lib/libdb.so.2 -> /usr/lib/libgdbm.so.2.0.0
-bash-4.1$

3) Started Application tier services.

-bash-4.1$ cd $ADMIN_SCRIPTS_HOME
-bash-4.1$ ./adopmnctl.sh status apps/apps

You are running adopmnctl.sh version 120.4.12000000.3

Checking status of OPMN managed processes...

Processes in Instance: r12hyupg_hydhost02.hydhost02.example.com
-------------------+--------------------+---------+---------
ias-component      | process-type       |     pid | status
-------------------+--------------------+---------+---------
default_group      | oafm               |   14789 | Alive
default_group      | forms              |   14719 | Alive
default_group      | oacore             |   14634 | Alive
HTTP_Server        | HTTP_Server        |   14566 | Alive

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile /u01/R12EBS/inst/apps/r12hyupg_hydhost02/logs/appl/admin/log/adopmnctl.txt for more information ...

-bash-4.1$
 

Hope this article will help you :-)

Query To Check Concurrent Program Run History in EBS

Following query will helps you to find the history of concurrent programs run in your EBS environment.

You should connect to 'apps' user for executing this query and query will prompt for "NO_OF_DAYS" and "USER_CONCURRENT_PROGRAM_NAME".

NO_OF_DAYS => "Enter Number of Days for History: ";
USER_CONCURRENT_PROGRAM_NAME => "Enter User Concurrent Program Name: ";

-- Query To Check Concurrent Program Run History    

set pages 100 linesize 200
col Parameters for a20 
col "Conc Program Name" for a30 
col "Started at" for a20
col "Completed at" for a20
col "Username" for a10 
SELECT distinct ft.user_concurrent_program_name "Conc Program Name",
fr.REQUEST_ID "Request ID",
to_char(fr.ACTUAL_START_DATE,'dd-MON-yy hh24:mi:ss') "Started at",
to_char(fr.ACTUAL_COMPLETION_DATE,'dd-MON-yy hh24:mi:ss') "Completed at",
decode(fr.PHASE_CODE,'C','Completed','I','Inactive','P','Pending','R','Running','NA') "Phasecode",
decode(fr.STATUS_CODE, 'A','Waiting', 'B','Resuming', 'C','Normal', 'D','Cancelled', 'E','Error', 'F','Scheduled', 'G','Warning', 'H','On Hold', 'I','Normal', 'M',
'No Manager', 'Q','Standby', 'R','Normal', 'S','Suspended', 'T','Terminating', 'U','Disabled', 'W','Paused', 'X','Terminated', 'Z','Waiting') "Status",fr.argument_text "Parameters",
fu.user_name "Username",
round(((nvl(fv.actual_completion_date,sysdate)-fv.actual_start_date)*24*60),2) "ElapsedTime(Mins)"
FROM
apps.fnd_concurrent_requests fr ,
apps.fnd_concurrent_programs fp ,
apps.fnd_concurrent_programs_tl ft,
apps.fnd_user fu, apps.fnd_conc_req_summary_v fv
WHERE 
fr.CONCURRENT_PROGRAM_ID = fp.CONCURRENT_PROGRAM_ID
AND fr.actual_start_date >= (sysdate - &NUMBER_OF_DAYS)
AND   fr.PROGRAM_APPLICATION_ID = fp.APPLICATION_ID
AND ft.concurrent_program_id=fr.concurrent_program_id
AND fr.REQUESTED_BY=fu.user_id
AND fv.request_id=fr.request_id
and ft.user_concurrent_program_name like '&USER_CONCURRENT_PROGRAM_NAME'
order by to_char(fr.ACTUAL_COMPLETION_DATE,'dd-MON-yy hh24:mi:ss') desc; 
 

Hope this query helps you. Thanks :-)