Wednesday, December 29, 2010

Oracle Trainings in support.oracle.com

Hi Viewers,

Oracle is maintaining all the recorded Webcast recorded sessions in the form of Oracle Documents in http://support.oracle.com .

The document id is 740964.1 [Advisor Webcast Archived Recordings ]

You can get all useful and valuable trainings from this document.

Thanks,
Sasikumar.M

"Synchronize WF LOCAL tables" request is failing with "ORA-20002: 4006:" error

Whenever we submit the "Synchronize WF LOCAL tables" request, it is failing with following error message,


FNDWFLSC module: Synchronize WF LOCAL tables
+---------------------------------------------------------------------------+

Current system time is 01-JUN-2010 01:41:09

+---------------------------------------------------------------------------+

**Starts**01-JUN-2010 01:41:09
**Ends**01-JUN-2010 01:41:43
ORA-20002: 4006: While creating necessary indexes, duplicate values were found preventing the creation of a unique index. This is caused by an improper view definition or data corruption that requires the attention of the Product Development Team handling 'PER_ROLE'.
ORA-20002: 4006: While creating necessary indexes, duplicate values were found preventing the creation of a unique index. This is caused by an improper view definition or data corruption that requires the attention of the Product Development Team handlin
+---------------------------------------------------------------------------+
Start of log messages from FND_FILE
+---------------------------------------------------------------------------+
+---------------------------------------------------------------------------+
End of log messages from FND_FILE
+---------------------------------------------------------------------------+

After further troubleshooting, We used the following SQL statement and Identified the duplicate rows in APPS.PER_ROLE_ROLES_V view,

SELECT NAME, ORIG_SYSTEM, ORIG_SYSTEM_ID,PARTITION_ID, COUNT(*) FROM
APPS.PER_ROLE_ROLES_V GROUP BY NAME, ORIG_SYSTEM, ORIG_SYSTEM_ID,PARTITION_ID
HAVING COUNT(*) > 1;


Finally we did the following DML for removing the duplication row,

update per_all_people_f
set effective_end_date = to_date ('23-oct-2009','dd-mon-yyyy')
where person_id= (Person_Id_Return_by_Select_Query)
and effective_start_date= to_date('23-Oct-2009','dd-mon-yyyy')
and effective_end_date= to_date('14-Jan-2010','dd-mon-yyyy');


delete from per_all_people_f
where person_id= (Person_Id_Return_by_Select_Query)
and effective_start_date= to_date('15-Jan-2010','dd-mon-yyyy');

commit;

Reran the "Synchronize WF LOCAL tables" request and it is completing normal.

Thanks,
Sasikumar.M

APEX 4.0.2 installation with E-Business Suite R12

Recently I installed the APEX 4.0.2 in R12 ( 12.0.4 ) instances. Here are the steps which I followed.

1. Download the Apex software from http://download.oracle.com/otn/java/appexpress/apex_4.0.2.zip
2. Unzip the apex_4.0.2.zip file.
3. Take the invalid object counts.
4. Create the new table space APEX402 with 1 GB size

SQL> create tablespace APEX402 datafile '/oracle/VIS/oradata/apex402_01.dbf' size 1G autoextend on;

5. Login as SYSDBA user and execute the following SQL from the unzipped folder

SQL> @apexins.sql APEX402 APEX402 TEMP /i/

6. After 20-30 min, You will get following confirmation message,

Thank you for installing Oracle Application Express.

Oracle Application Express is installed in the APEX_040000 schema.

The structure of the link to the Application Express administration services is as follows:
http://host:port/pls/apex/apex_admin (Oracle HTTP Server with mod_plsql)
http://host:port/apex/apex_admin (Oracle XML DB HTTP listener with the embedded PL/SQL gateway)

The structure of the link to the Application Express development interface is as follows:
http://host:port/pls/apex (Oracle HTTP Server with mod_plsql)
http://host:port/apex (Oracle XML DB HTTP listener with the embedded PL/SQL gateway)

JOB_QUEUE_PROCESSES: 5

Performing Application Express component validation - please wait...

Completing registration process. 13:35:57
Validating installation. 13:35:57
...Database user "SYS", database schema "APEX_040000", user# "587" 13:35:57

7. Once you see this message, the Apex installation to the Oracle EBS database is completed.
8. Unlock the APEX_PUBLIC_USER account from the apex directory.

$ sqlplus "/as sysdba"
SQL> @apxchpwd.sql

9. Copy all the images files under apex/images directory to $COMMON_TOP/imagess (New folder) on the Front End Application server
10.Update the following entires in $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf file

Alias /i/ "/apps/vis/apps/apps_st/comn/images/"
AddType text/xml xbl
AddType text/x-component htc

<Location /pls/apex>
Order deny,allow
PlsqlDocumentPath docs
AllowOverride None
PlsqlDocumentProcedure wwv_flow_file_mgr.process_download
PlsqlDatabaseConnectString testdb.cisco.com:1531:VIS <-- Enter the DB connection Details
PlsqlNLSLanguage AMERICAN_AMERICA.AL32UTF8
PlsqlAuthenticationMode Basic
SetHandler pls_handler
PlsqlDocumentTablename wwv_flow_file_objects$
PlsqlDatabaseUsername APEX_PUBLIC_USER
PlsqlDefaultPage apex
PlsqlDatabasePassword welcome <-- Enter the Admin Password
PlsqlRequestValidationFunction wwv_flow_epg_include_modules.authorize
Allow from all
</Location>

11. Make sure $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf file included in $IAS_ORACLE_HOME/Apache/modplsql/conf/plsql.conf.
12. Ensure $IAS_ORACLE_HOME/Apache/modplsql/conf/plsql.conf is included in $INST_TOP/ora/10.1.3/Apache/Apache/conf/oracle_apache.conf
13. By default, httpd.conf file should included the file $INST_TOP/ora/10.1.3/Apache/Apache/conf/oracle_apache.conf.
14. Bounce Apache services.
15. Access the APEX URL http://host:port/pls/apex/apex_admin and http://host:port/pls/apex

Comments are Welcome!!!

Thanks,
Sasikumar.M

Friday, December 17, 2010

Story of a Failed Grid Control Agent

Unfortunately one of the Grid Control Agent on a Development System was down for a longer time.(2 months)
So wanted to get it up.



oracle@devdb[AGENT]$ emctl start agent
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
Starting agent ............................................... failed.
EMAgent is Thrashing. Exiting watchdog
Consult the log files in: /oracle/product/10.2.0/agent10g/sysman/log
oracle@devdb[AGENT]$


On the first look I thought,  what happened !!! , it runs fine on all the other nodes and only on this it is thrashing.
So looked into the logfiles and found that the agent when starting was trying to discover the targets.


At some point it could not do it and so it failed.
So had a quick look into the mysupport.oracle.com for relevant materials. Read about 20 articles and came across 365252.1


As per the note I did the following.
cd $ORACLE_HOME/sysman/emd   (where $ORACLE_HOME is the Agent's home)
- backed up targets.xml
- Removed all the targets information except the one for agent
- Ran agentca -d to force a new discovery of the targets on the host



oracle@devdb[AGENT]$ agentca -d


Stopping the agent using /oracle/product/10.2.0/agent10g/bin/emctl  stop agent
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
Stopping agent ... stopped.
Running agentca using /oracle/product/10.2.0/agent10g/oui/bin/runConfig.sh ORACLE_HOME=/oracle/product/10.2.0/agent10g ACTION=Configure MODE=Perform RESPONSE_FILE=/oracle/product/10.2.0/agent10g/response_file RERUN=TRUE INV_PTR_LOC=/oracle/product/10.2.0/agent10g/oraInst.loc COMPONENT_XML={oracle.sysman.top.agent.10_2_0_1_0.xml}
Perform - mode is starting for action: Configure
Perform - mode finished for action: Configure
You can see the log file: /oracle/product/10.2.0/agent10g/cfgtoollogs/oui/configActions2010-12-17_08-02-57-AM.log
oracle@devdb[AGENT]$


oracle@devdb[AGENT]$ cat /oracle/product/10.2.0/agent10g/cfgtoollogs/oui/configActions2010-12-17_08-02-57-AM.log
###################################################
The action configuration is performing
------------------------------------------------------
The plug-in Agent Configuration Assistant is running


Performing free port detection on host=testserver.testdomain.com
Performing targets discovery and agent configuration
Starting the agent
AgentPlugIn:agent configuration finished with status = true


The plug-in Agent Configuration Assistant has successfully been performed
------------------------------------------------------
The action configuration has successfully completed
###################################################
oracle@devdb[AGENT]$




Now I started the agent



oracle@devdb[AGENT]$ emctl start agent
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
Starting agent ....... started.
oracle@devdb[AGENT]$ 


Looks good the agent came up and the status was also good.

oracle@devdb[AGENT]$ emctl status agent
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 10.2.0.5.0
OMS Version       : 10.2.0.5.0
Protocol Version  : 10.2.0.5.0
Agent Home        : /oracle/product/10.2.0/agent10g
Agent binaries    : /oracle/product/10.2.0/agent10g
Agent Process ID  : 27849
Parent Process ID : 27836
Agent URL         : https://testserver.testdomain.com:3872/emd/main/
Repository URL    : https://testserver.testdomain.com:4898/em/upload
Started at        : 2010-12-17 08:03:57
Started by user   : oracle
Last Reload       : 2010-12-17 08:03:57
Last successful upload                       : (none)
Last attempted upload                        : (none)
Total Megabytes of XML files uploaded so far :     0.00
Number of XML files pending upload           :     1124
Size of XML files pending upload(MB)         :    59.90
Available disk space on upload filesystem    :    23.16%
Collection Status                            : Disabled by Upload Manager
Last attempted heartbeat to OMS              : 2010-12-17 08:05:08
Last successful heartbeat to OMS             : unknown
---------------------------------------------------------------
Agent is Running and Ready
oracle@devdb[AGENT]$



But the collection status was disabled by upload manager. That means the xml files are not transported.
You can see that the number of XML files are 1124 which is too high for the agent to process them.

Came across 396238.1 , and did the following to recover from the issue.

- Stop the agent
Delete any pending upload files from the agent home
  rm -r /sysman/emd/upload/* 
  rm /sysman/emd/lastupld.xml 
- Start the agent.



oracle@devdb[AGENT]$ emctl status agent
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 10.2.0.5.0
OMS Version       : 10.2.0.5.0
Protocol Version  : 10.2.0.5.0
Agent Home        : /oracle/product/10.2.0/agent10g
Agent binaries    : /oracle/product/10.2.0/agent10g
Agent Process ID  : 5953
Parent Process ID : 5943
Agent URL         : https://testserver.testdomain.com:3872/emd/main/
Repository URL    : https://testserver.testdomain.com:4898/em/upload
Started at        : 2010-12-17 08:26:20
Started by user   : oracle
Last Reload       : 2010-12-17 08:26:20
Last successful upload                       : (none)
Last attempted upload                        : (none)
Total Megabytes of XML files uploaded so far :     0.00
Number of XML files pending upload           :       16
Size of XML files pending upload(MB)         :     8.43
Available disk space on upload filesystem    :    23.21%
Last attempted heartbeat to OMS              : 2010-12-17 08:27:00
Last successful heartbeat to OMS             : unknown
---------------------------------------------------------------
Agent is Running and Ready
oracle@devdb[AGENT]$


Looks good but the agent is not actually uploading the xml files. You will see 16 files are not uploaded
Tried to manually force an upload.



oracle@devdb[AGENT]$ emctl upload
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD upload error: uploadXMLFiles skipped :: OMS version not checked yet..
oracle@devdb[AGENT]$


From the logfile found the following;
2010-12-17 08:28:01,116 Thread-71 ERROR pingManager: Did not receive valid response to ping "ERROR-Agent is blocked. Blocked reason is: Agent is out-of-sync with repository. This most likely means that the agent was reinstalled or recovered. Please contact an EM administrator to unblock the agent by performing an agent resync from the console. Please contact EM adminstrator to unblock the agent"




Then came across  799618.1 , and did the following as per the instructions;
- Login to 10.2.0.5 Grid Console as sysman user
- Navigate to Setup -> Agents -> Click on problematic Agent name
- In Agent home page click on "Resycnhronize" Button on right hand top of the page.
  Choose the 'Unblock Agent' option and click Continue
  Ensure that the Agent is up and running when attempting the Resynchronization.
Once the re-sycnchronization is completed successfully, the Agent should be able to communicate with the OMS




oracle@devdb[AGENT]$ emctl upload
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD upload completed successfully
oracle@devdb[${ORACLE_SID}]$ emctl status agent
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version     : 10.2.0.5.0
OMS Version       : 10.2.0.5.0
Protocol Version  : 10.2.0.5.0
Agent Home        : /oracle/product/10.2.0/agent10g
Agent binaries    : /oracle/product/10.2.0/agent10g
Agent Process ID  : 5953
Parent Process ID : 5943
Agent URL         : https://testserver.testdomain.com:3872/emd/main/
Repository URL    : https://testserver.testdomain.com:4898/em/upload
Started at        : 2010-12-17 08:26:20
Started by user   : oracle
Last Reload       : 2010-12-17 08:26:20
Last successful upload                       : 2010-12-17 08:31:35
Total Megabytes of XML files uploaded so far :    10.42
Number of XML files pending upload           :        0
Size of XML files pending upload(MB)         :     0.00
Available disk space on upload filesystem    :    23.21%
Last successful heartbeat to OMS             : 2010-12-17 08:31:02
---------------------------------------------------------------
Agent is Running and Ready
oracle@devdb[AGENT]$


You will see that all the xml files are uploaded.


All done and the agent was up and running. 
Thanks to documents from mysupport.oracle.com


Tuesday, December 14, 2010

VirtualBox VMs for Developers and DBAs

Good news for those who work on Virtual Box.

Oracle has pre-build VM to support quicker access to its product.

Monday, December 13, 2010

The power of Amazon EC2

Why 'Anonymous' failed to take down Amazon over WikiLeaks - Dec. 9, 2010

Cloud Computing at its best. EC2 expanded and scaled as the traffic increased. Read the above article to know about why it is powerful.

Hosting Oracle Products would be more fun there.

Monday, November 8, 2010

BPELConsole and BPELAdmin Page does not load

All,

When trying to access the BPEL Console or the BPEL Admin page or the EM console after a fresh installation of Oracle Application Server Release 3 and BPEL I was not able to get the initial screen.The browser status always showed as "Loading .....".

After a long time of investigation and going through all the log files.
Found one log file under
$ORACLE_HOME/j2ee/oc4j_bpel/log/oc4j_bpel_default_group_1/oc4j/log.xml which was growing rapidly...on just tailing the log file came across the following error


<MSG_TEXT>IOException in NIOServerSocketDriver:select</MSG_TEXT>


Cause :-

Starting up oc4j on Solaris 10 spawns nioserversocketdriver exceptions,which is a bug.

Fix:-

1. Backup the $ORACLE_HOME/opmn/conf/opmn.xml file.

2. Edit the opmn.xml file to include -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider in the
java-options tag for the start-parameters for "home" and "oc4j_bpel" as mentioned below

<data id="java-options" value="-server -Djava.security.policy=$ORACLE_HOME/j2ee/home/config/java2.policy -Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -Djava.awt.headless=true"/>

3. Restart the Oracle Application Server

$ORACLE_HOME/opmn/bin/opmnctl stopall
$ORACLE_HOME/opmn/bin/opmnctl startall
$ORACLE_HOME/opmn/bin/opmnctl status (to verify)

Now I was able to sucessfully load the BPEL Console ,BPEL Admin and EM page without any issues.

The system classes in the classpath are not compatible with this version of Java when trying to start application using adstrtal.sh

Hello All,

Today when trying to start one of the mid-tier services using adstrtal.sh apps/apps, the script failed with the below message :-

Note :- This was a 11i EBS system.

apphost1:/d51/oracle/viscomn/admin/scripts/vis_apphost1> adstrtal.sh apps/apps

You are running adstrtal.sh version 115.11

The system classes in the classpath are not compatible with this
version of Java.
  Please check that the classpath is correct.
CLASSPATH = /opt/jdk1.3.1_20/j2se/jre/lib/rt.jar:/opt/jdk1.3.1_20/j2se/jre/lib/i18n.jar:/d51/oracle/viscomn/java/appsborg.zip:/d51/oracle/viscomn/java/apps.zip:/d51/oracle/visora/8.0.6/forms60/java:/d51/oracle/viscomn/java:/d51/oracle/visappl/ad/11.5.0/java/adconfig.zip:/opt/jdk1.3.1_20/j2se/bin/../classes:/opt/jdk1.3.1_20/j2se/bin/../lib/classes.zip:/d51/oracle/viscomn/java/loadjava.zip:/d51/oracle/visappl/ad/11.5.0/java/adconfig.zip:/d51/oracle/visappl/fnd/11.5.0/java/3rdparty/stdalone/xmlparserv2.zip:/d51/oracle/viscomn/util/jre/1.1.8/bin/../classes:/d51/oracle/viscomn/util/jre/1.1.8/bin/../lib/classes.zip
Check logfile /d51/oracle/viscomn/admin/log/vis_apphost1/11040716.log for details.


After analysing for minutes found that the tools java home was incorrectly set and pointing to a wrong directory.

1. Took a backup of the Context file and replaced the s_tools_java to point to the correct java location.

<JAVA_HOME oa_var="s_tools_java">/opt/jdk1.3.1_20/j2se</JAVA_HOME>

2. Then ran autoconfig .

3. Now tried to start the application services and it started successfully.

Thursday, October 14, 2010

New System Privilege 'SYSASM' to manage ASM instance in 11g

Hello All,

Today when I was trying to shutdown a 11g ASM instance connected as sysdba it was not allowing me to shutdown and gave 'ORA-01031: insufficient privileges'.

After a bit of research found that in 11g there is a new privilege called 'SYSASM' which is separate for ASM storage administration privilege and for maintaining ASM instance.The basic idea was to separate the storage administrator and the database administrator responsbilities.

Below is what I was trying to do.


[oracle@oelbox1 ~]$ echo $ORACLE_SID
+ASM

[oracle@oelbox1 ~]$ ps -ef | grep -i pmon
oracle 11810 1 0 18:58 ? 00:00:00 asm_pmon_+ASM
oracle 22821 22724 0 22:19 pts/5 00:00:00 grep -i pmon

[oracle@oelbox1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Thu Oct 14 22:19:07 2010
Copyright (c) 1982, 2010, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - Production
With the Automatic Storage Management option

SQL> select instance_name from v$instance;
INSTANCE_NAME
----------------
+ASM

SQL> shutdown immediate
ORA-01031: insufficient privileges
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - Pr oduction
With the Automatic Storage Management option

[oracle@oelbox1 ~]$ sqlplus /nolog

SQL*Plus: Release 11.2.0.2.0 Production on Thu Oct 14 22:19:46 2010
Copyright (c) 1982, 2010, Oracle. All rights reserved.
SQL> connect sys as sysasm
Enter password:
Connected.
SQL>
SQL> shut immediate
ASM diskgroups volume disabled
ASM diskgroups dismounted
ASM instance shutdown
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - Pr oduction With the Automatic Storage Management option

Thursday, August 12, 2010

ORA-20001:FND_CANT_INSERT_USER_ROLE when attaching responsibility to User

Yesterday we had a issue with one user saying after he logs into applications no responsibility displayed.When we did a search for that user on the forms ,surprisingly
there was no responsibilities attached to that user.

We tried to attach responsibility and save it ,but it will not save and came back with the below message

"ORA-20001:FND_CANT_INSERT_USER_ROLE has been detected in FND_USER_RESP_GROUPS_API.INSERT_ASSIGNMENT"

Later we ran concurrent program "Workflow Directory Services User/Role Validation" .Request completed normal and this time the responsibility was visible on the forms when we queried that user.But when the user tried to login no responsibilities still.

We fixed this issue by end-dating the responsibility assigned to the user and saved.Then removed the end date for the responsibility and saved.

Now requested the user to test and he can see the responsibility now and work without any issues.

RMAN-05001: auxiliary filename /u01/oracle/oradata/wipd06.dbf conflicts with a file used by the target database

When performing duplication using RMAN I recieved the following message and duplication came out with errors.
On researching found that the values for parameters 'db_file_name_convert' and 'log_file_name_convert' was not set correctly.

The reasons being last time when duplication was performed the DBA used a different Database as source and the values for 'db_file_name_convert' and 'log_file_name_convert' were still pointing to the previous source database mount points.

So this time without checking or modifying the values the DBA started the duplication and received the above error message.

To fix it

Stop the database,change the values for 'db_file_name_convert' and 'log_file_name_convert' to the correct source Database values and then restart the duplication.

Wednesday, June 30, 2010

Storage Migration on OCR and Voting Disk



Dear Viewer,

Recently I have done with renaming of OCR and Voting Disks during the Storage Migration.

Here are the high level action plan which I performed in our Non-Prod RAC instances as root user,

OCR :
===

ocrcheck
ocrconfig -export /oracle/export/CRS/OCR_Backup -s online <-- Taking OCR Backup -- Monitor the recent log file under $CRS_Home/log//client
ocrconfig -replace ocr /dev/mapper/tst-db-01-02_cluster_ocr_001_1000M
-- Monitor the recent log file under $CRS_Home/log//client
ocrconfig -replace ocrmirror /dev/mapper/tst-db-01-02_cluster_ocrmirr_002_1000M
-- Monitor the recent log file under $CRS_Home/log//client
ocrcheck

VOTING DISK :
============

crsctl stop crs
crsctl query css votedisk
crsctl add css votedisk /dev/mapper/tst-db-01-02_cluster_vote_003_1000M -force
crsctl add css votedisk /dev/mapper/tst-db-01-02_cluster_votemirr_004_1000M -force
crsctl delete css votedisk /dev/mapper/oraMFGSTGclus02_1708_42B2_voting_1000 -force
crsctl delete css votedisk /dev/mapper/oraMFGSTGclus02_1708_42B2_voting_1000 -force
crsctl start crs

-- Monitor /var/log/message file for any errors.

Session Log :
===========

Getting Current Status Information :
==================================

[root@tst-db-01 crs_home]# export PATH=$PATH:/oracle/product/crs_home/bin
[root@tst-db-01 crs_home]# clear
[root@tst-db-01 crs_home]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 1048040
Used space (kbytes) : 3636
Available space (kbytes) : 1044404
ID : 1397237804
Device/File Name : /dev/mapper/MFGSTGclus02_1708_42B6_ocr_1000
Device/File integrity check succeeded
Device/File Name : /dev/mapper/MFGSTGclus02_1708_42B0_ocr_mirror1_1000
Device/File integrity check succeeded

Cluster registry integrity check succeeded

[root@tst-db-01 crs_home]# crsctl query css votedisk
0. 0 /dev/mapper/oraMFGSTGclus02_1708_42B2_voting_1000
1. 0 /dev/mapper/oraMFGSTGclus02_1708_42B3_voting_mirror1_1000

located 2 votedisk(s).
[root@tst-db-01 crs_home]# ps -eaf|grep pmon
root 943 31154 0 07:43 pts/0 00:00:00 grep pmon
[root@tst-db-01 crs_home]#

Voting Disk :
===========

1. Bring down the CRS

[root@tst-db-01 /]# /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.

2. Take the backup

[root@tst-db-01 /]# dd if=/dev/mapper/oraMFGSTGclus02_1708_42B2_voting_1000 of=/oracle/export/CRS/voting.dmp
2096640+0 records in
2096640+0 records out

[root@tst-db-01 /]# ls -l /oracle/export/CRS/voting.dmp
-rw-r--r-- 1 root root 1073479680 Jun 22 08:51 /oracle/export/CRS/voting.dmp

3. Add Voting Disk

[root@tst-db-01 /]# crsctl add css votedisk /dev/mapper/tst-db-01-02_cluster_vote_003_1000M -force
Now formatting voting disk: /dev/mapper/tst-db-01-02_cluster_vote_003_1000M
successful addition of votedisk /dev/mapper/tst-db-01-02_cluster_vote_003_1000M.

4. Add addition Voting Disk

[root@tst-db-01 /]# crsctl add css votedisk /dev/mapper/tst-db-01-02_cluster_votemirr_004_1000M -force
Now formatting voting disk: /dev/mapper/tst-db-01-02_cluster_votemirr_004_1000M
successful addition of votedisk /dev/mapper/tst-db-01-02_cluster_votemirr_004_1000M.

5. Deleting old Voting Disk

[root@tst-db-01 /]# crsctl delete css votedisk /dev/mapper/oraMFGSTGclus02_1708_42B2_voting_1000 -force
successful deletion of votedisk /dev/mapper/oraMFGSTGclus02_1708_42B2_voting_1000.

[root@tst-db-01 /]# crsctl delete css votedisk /dev/mapper/oraMFGSTGclus02_1708_42B3_voting_mirror1_1000 -force
successful deletion of votedisk /dev/mapper/oraMFGSTGclus02_1708_42B3_voting_mirror1_1000.

6. Bring up the CRS

[root@tst-db-01 /]# /etc/init.d/init.crs start
Startup will be queued to init within 30 seconds.
[root@tst-db-01 /]# tail -f /var/log/messages
Jun 22 08:58:47 tst-db-01 prashasi: Oracle Cluster Ready Services starting by user request.
Jun 22 08:59:00 tst-db-01 logger: Cluster Ready Services completed waiting on dependencies.
Jun 22 08:59:03 tst-db-01 last message repeated 2 times
Jun 22 08:59:03 tst-db-01 logger: Running CRSD with TZ =

7. Verify the CRS Services

[root@tst-db-01 /]# crsctl check crs

CSS appears healthy
Cannot communicate with CRS
EVM appears healthy

8. Verify the Voting Disk Changes

[root@tst-db-01 /]# crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@tst-db-01 /]# crsctl query css votedisk
0. 0 /dev/mapper/tst-db-01-02_cluster_vote_003_1000M
1. 0 /dev/mapper/tst-db-01-02_cluster_votemirr_004_1000M

located 2 votedisk(s).


OCR Disk :
========

1. Take the backup

[root@tst-db-01 export]# cd /oracle/export
[root@tst-db-01 export]# cd CRS
[root@tst-db-01 CRS]# ocrconfig -export /oracle/export/CRS/OCR_Backup -s online

2. Add the OCR and OCR Mirror Disk

[root@tst-db-01 CRS]# ls -l /dev/mapper/tst-db-01-02_cluster_ocr_001_1000M
brw-rw---- 1 root disk 253, 385 Jun 21 12:18 /dev/mapper/tst-db-01-02_cluster_ocr_001_1000M
[root@tst-db-01 CRS]# ocrconfig -replace ocr /dev/mapper/tst-db-01-02_cluster_ocr_001_1000M
[root@tst-db-01 CRS]# ocrconfig -replace ocrmirror /dev/mapper/tst-db-01-02_cluster_ocrmirr_002_1000M

3. Verify the OCR Changes

[root@tst-db-01 CRS]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 1048040
Used space (kbytes) : 3636
Available space (kbytes) : 1044404
ID : 1397237804
Device/File Name : /dev/mapper/tst-db-01-02_cluster_ocr_001_1000M
Device/File integrity check succeeded
Device/File Name : /dev/mapper/tst-db-01-02_cluster_ocrmirr_002_1000M
Device/File integrity check succeeded

Cluster registry integrity check succeeded

[root@tst-db-01 CRS]#


Regards,
Sasikumar.M

Tuesday, June 29, 2010

When Opening Database - 'ORA-01113: file 1 needs media recovery '

Hello All,

After restore and recovery is complete ,trying to open database with resetlogs option showed the following error message

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: '/u02/oracle/UPG/apps/apps_st/db/system01.dbf' was restored from an oldbackup

I have verified the time and the scn number until which I ran to restore, everything was correct but still unable to open the database.I tried applying archive logs and then to open failed again.

Research

I have issued the following command to recover 'recover database using backup controlfile' .

After some research and googling I issued the following command to open database

sqlplus > recover database using backup controlfile until cancel;

When prompted issued CANCEL.Now tried to open the database with resetlogs and database opened without any issues.

Session Details

SQL> recover database using backup controlfile until cancel;
ORA-00279: change 370577110 generated at 06/12/2010 16:19:22 needed for thread
1
ORA-00289: suggestion :
/u01/oracle/UPG/db/apps_st/arch/UPG_1_2671_699752349.arc
ORA-00280: change 370577110 for thread 1 is in sequence #2671


Specify log: {=suggested | filename | AUTO | CANCEL}
/u01/oracle/UPG/db/apps_st/arch/UPG_1_2671_699752349.arc
ORA-00279: change 370578253 generated at 06/12/2010 16:27:55 needed for thread
1
ORA-00289: suggestion :
/u01/oracle/UPG/db/apps_st/arch/UPG_1_2672_699752349.arc
ORA-00280: change 370578253 for thread 1 is in sequence #2672
ORA-00278: log file
'/u01/oracle/UPG/db/apps_st/arch/UPG_1_2671_699752349.arc' no longer
needed for this recovery


Specify log: {=suggested | filename | AUTO | CANCEL}
/u01/oracle/UPG/db/apps_st/arch/UPG_1_2672_699752349.arc
ORA-00279: change 370610644 generated at 06/12/2010 18:27:47 needed for thread
1
ORA-00289: suggestion :
/u01/oracle/UPG/db/apps_st/arch/UPG_1_2673_699752349.arc
ORA-00280: change 370610644 for thread 1 is in sequence #2673
ORA-00278: log file
'/u01/oracle/UPG/db/apps_st/arch/UPG_1_2672_699752349.arc' no longer
needed for this recovery


Specify log: {=suggested | filename | AUTO | CANCEL}
CANCEL
Media recovery cancelled.
SQL>
SQL> alter database open resetlogs;

Database altered.

SQL> select * from v$recover_file;

no rows selected

SQL> select distinct status from v$datafile;

STATUS
-------
ONLINE
SYSTEM


- Hope this helps when performing simple recovery to a different server.

RMAN Duplication - UNTIL scn (.......) is ahead of last scn in archived logs (........)

Hello All,


When performing duplication of database with rman connected to the Source received the following error

upg db009 oracle102 > rman target sys/sys@upg auxiliary /

Recovery Manager: Release 10.2.0.4.0 - Production on Tue Jun 29 17:26:47 2010

Copyright (c) 1982, 2007, Oracle. All rights reserved.

connected to target database: UPG (DBID=1899709187)
connected to auxiliary database: STAGE (not mounted)

RMAN>

RMAN> run {
SET UNTIL SCN 379298362;
allocate auxiliary channel t1 type disk;
allocate auxiliary channel t2 type disk;
allocate auxiliary channel t3 type disk;
switch datafile all;
duplicate target database to stage;
}2> 3> 4> 5> 6> 7> 8>

executing command: SET until clause

allocated channel: t1
channel t1: sid=1587 devtype=DISK

allocated channel: t2
channel t2: sid=1586 devtype=DISK

allocated channel: t3
channel t3: sid=1585 devtype=DISK


Starting Duplicate Db at 28-JUN-10
released channel: t1
released channel: t2
released channel: t3
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 06/28/2010 14:58:18
RMAN-06457: UNTIL scn (379298362) is ahead of last scn in archived logs (379296482)

RMAN> exit


Issue and Verification

On Checking the Source DB,found that the last scn was older than what I have specified in run block of RMAN.Hence issued checkpoint connected to the source DB.

SQL> select thread#, status, enabled, checkpoint_time, checkpoint_change# from v$thread;

THREAD# STATUS ENABLED CHECKPOINT_TIME CHECKPOINT_CHANGE#
---------- ------ -------- ------------------ ------------------
1 OPEN PUBLIC 28-JUN-10 379296482



SQL> alter system checkpoint;

System altered.

SQL> select thread#, status, enabled, checkpoint_time, checkpoint_change# from v$thread;

THREAD# STATUS ENABLED CHECKPOINT_TIME CHECKPOINT_CHANGE#
---------- ------ -------- ------------------ ------------------
1 OPEN PUBLIC 28-JUN-10 379299681

SQL>

Now re-started the duplication and it completed successfully.

Monday, June 21, 2010

AD Administration Warning

Hello Again,


When applying any patch or starting adadmin in R12,I happened to notice the following message on the screen ,but adpatch or adadmin continued without errors or failures.

AD Administration warning:
Product Data File ' /u02/oracle/UPG/apps/apps_st/appl/admin/zfaprod.txt ' does not exist for product "zfa".This product is registered in the database but the above file does not exist in APPL_TOP. The product will be ignored without error.

Message also appeared for the following products "zsa" and "jts".


This message was displayed on all the R12 environments we had.Did a bit research and found a note :- 421136.1.

The note states that 'These products were registered in the 11i database, and hence are producing these warning messages. Oracle has this warning for all the products that are licensed in the DB, but are obsolete in R12. Customers will have this warning only for those products which they have licensed in 11i but obsolete in R12.'

Also the note states 'Ignore the warnings as they do not affect the functionality.'
If the patch fails, skip the workers of these products using adctrl and continue on with the patch.

Monday, June 14, 2010

Users Receive Blank Screen when logging into EBS in a SSO Integrated System

Hello All,

Last week was such a hectic one with one kind of a peculiar problem not seen before.
Users created after a particular date,logging into Oracle Apps via third party authentication,integrated with OID/SSO were getting blank pages.

On analysis we found that the user_guid column with the 'FND_USER' table was null for these users.
Verified OID the attribute 'orclguid' was there ,but it was not synching with FND_USER table,inspite the profile option 'Applications SSO Auto Link User" (APPS_SSO_AUTO_LINK_USER)' was set to 'ENABLED'
at site level.

I had a very hard feeling that the linking between EBS and OID was broken.To my surprise that was the case .
I dont know how many of you are aware of an user named 'AppsDN'.This "AppsDN" user is a special administration account which eBiz uses to connect
to OID for management and administrative tasks when integrating with OID/SSO.

- Determine the 'AppsDN' username/password connected to EBS as apps user

SQL> select fnd_preference.get('#INTERNAL', 'LDAP_SYNCH','USERNAME') Apps_Instance_OID_Account from dual;

APPS_INSTANCE_OID_ACCOUNT
--------------------------------------------------------------------------------
orclapplicationcommonname=prod,cn=ebusiness,cn=products,cn=oraclecontext,dc=thirdparty,dc=users

SQL> select fnd_preference.eget('#INTERNAL', 'LDAP_SYNCH','EPWD','LDAP_PWD') Apps_Password from dual;

APPS_PASSWORD
--------------------------------------------------------------------------------
1applin


- Test the username/password.Use the following command on your node where OID is running to confirm if the password works or not

'ldapbind -D 'Apps Instance OID Account' -w 'Apps Password' -h 'OIDserver' -p 'OIDport' '

Eg:-
===
ldapbind -D orclapplicationcommonname=prod,cn=ebusiness,cn=products,cn=oraclecontext,dc=thirdparty,dc=users -w 1applin -h server11 -p 389


- On executing the above ldapbind command you should recieve 'Bind Successful'.In my case it was returning 'Password Policy Error :9000: GSL_PWDEXPIRED_EXCP :Your Password has expired'

- Navigate through the "Entry Management" tree to the appropriate context (the reversed path of the orclApplcationCommonName)
Follow the below steps to set the password in OID

1) Launch oidadmin console
2) Login as user ‘orcladmin’
3) Select and expand ‘Entry Management’
4) Select and expand ‘dc=users’
5) Select and expand ‘dc=thirdparty’ under ‘dc=users’
6) Select and expand ‘cn=OracleContext’ under ‘dc=thirdparty’
7) Select and expand ‘cn=Products’
8) Select and expand ‘cn=EBusiness’
9) Select orclApplicationCommonName=PROD
10) Under the Properties tab update the 'userpassword' attribute with the password from SQL Query in my case '1applin'
11) Click on ‘Apply’

- Then bounce the OID components.

- Now we requested the users to login into Oracle Apps and they no longer get the blank screen and OID was synching up the guid to FND_USER table.

Saturday, June 12, 2010

10gR2 : OPatch failed with Unable to create patchObject



Hi Viewer,

After long time, I am posting an interesting issue which I faced when applying the CPU patch with OPATCH utility.

When I tried to run opatch lsinventory, I got the following error message

[oracle@proddb ContentsXML]$ opatch lsinventory
Invoking OPatch 10.2.0.4.2

Oracle Interim Patch Installer version 10.2.0.4.2
Copyright (c) 2007, Oracle Corporation. All rights reserved.


Oracle Home : /oracle/product/10.2.0.4/PROD
Central Inventory : /oracle/product/oraInventory
from : /etc/oraInst.loc
OPatch version : 10.2.0.4.2
OUI version : 10.2.0.4.0
OUI location : /oracle/product/10.2.0.4/PROD/oui
Log file location : /oracle/product/10.2.0.4/PROD/cfgtoollogs/opatch/opatch2010-06-09_02-10-45AM.log



Inventory load failed... OPatch cannot load inventory for the given Oracle Home.

LsInventorySession failed: Unable to create patchObject

OPatch failed with error code 73
[oracle@proddb ContentsXML]$

So I set the OPATCH_DEBUG=TRUE environment variable for debugging more on this issue, I found the following extra error information,

OracleHomeInventory::load() begins to process raw data from OUI to build up OPatch primitive classes
PatchObject(oracleHomePath,patchID)
PatchObject::PatchObject looks for the actions file /oracle/product/10.2.0.4/PROD/inventory/oneoffs/8576156/etc/config/actions.xml
PatchObject::PatchObject looks for the inventory file /oracle/product/10.2.0.4/PROD/inventory/oneoffs/8576156/etc/config/inventory.xml
parserXMLFile : start
parserXMLFile : start
.
.
JarAction::getCompleteFileListPath() returns :ORACLE_HOME/rdbms/jlib/qsma.jar
PatchObject(oracleHomePath,patchID)
PatchObject::PatchObject looks for the actions file /oracle/product/10.2.0.4/PROD/inventory/oneoffs/8436582/etc/config/actions.xml
PatchObject::PatchObject looks for the inventory file /oracle/product/10.2.0.4/PROD/inventory/oneoffs/8436582/etc/config/inventory.xml
parserXMLFile : start
parserXMLFile : start
.
.
PatchObject(oracleHomePath,patchID)
PatchObject::PatchObject looks for the actions file /oracle/product/10.2.0.4/PROD/inventory/oneoffs/8788977/etc/config/actions.xml
PatchObject::PatchObject looks for the inventory file /oracle/product/10.2.0.4/PROD/inventory/oneoffs/8788977/etc/config/inventory.xml
parserXMLFile : start


Inventory load failed... OPatch cannot load inventory for the given Oracle Home.

Locker::release()
OUISessionManager::unRegister()
Un-Registering the caller : OPatch
LsInventory::getInstance() returns
LsInventorySession failed: Unable to create patchObject
Cleaning up the directory : "/oracle/product/10.2.0.4/PROD/.patch_storage/patch_unzip"...

OPatch failed with error code 73
[oracle@proddb ContentsXML]$

Referred the following support notes for further troubleshooting, but it didn't fix the issue.

SOLARIS: "opatch lsinventory" is failing with error code 73 on Remote Node in 10.2.0.4 RAC environment [ID 793600.1]
Opatch Lsinventory Fails With "Unable to create patchObject" OPatch failed with error code 73 [ID 751152.1]
Opatch error: "Apply Session failed: ApplySession failed to prepare the system. No detail available." [ID 864803.1]

Tried recreating OraInventory using runInstaller utility. But that option also did not help.

After spending a lot of time, I upgraded the OPATCH utility from 10.2.0.4.2 to 10.2.0.4.8. Finally it fixed the issue.

Comments are Welcome!!!

Monday, April 5, 2010

WARNING: Could not find all the required OS utilities in the $PATH. Please review the checks above

When cloning one of the nodes for a R12 application I faced the below error message on invoking adcfgclone.pl


applmgr > /oracle/STAGE/apps/tech_st/10.1.3/perl/bin/perl adcfgclone.pl appsTier

Copyright (c) 2002 Oracle Corporation
Redwood Shores, California, USA
Oracle Applications Rapid Clone
Version 12.0.0
adcfgclone Version 120.20.12000000.12

Enter the APPS password :

WARNING: Could not find all the required OS utilities in the $PATH. Please review the checks above

applmgr >

- Even-though I have set the required executable like ld,ar,make,zip in $PATH.
I was continuously getting this error.

- Verified the $LD_LIBRARY_PATH (all correct)

- Verified $PERL5LIB correctly set.

Issue
======

When I checked the disk space I came to know that the /tmp was full and there was no free space.

On deleting old and unwanted files I reclaimed some space in /tmp.

After clearing files in /tmp on invoking adcfgclone.pl it was successful and clone completed without any issues.

Saturday, March 27, 2010

Oracle Release 12 : Diagnostic Test errors with HTTP 400 - Bad Request

Dear Readers,

One of my application team compliant about the Diagnostic Test for Assets module failed with HTTP 400 - Bad Request.

As per the oracle note, this is the bug in Release 12 and Patch 8283135 is available for fixing this issue.

Current Version of file in the system,

Q$Header: FACheckSetup.java 120.0.12000000.2 2007/08/03 05:24:37 spooyath noship $
N$Header: FAUtility.java 120.0.12000000.2 2007/08/03 05:49:32 spooyath noship $


Patch provides the following version,

$fa/java/diag/FaCheckSetup.java - 120.0.12000000.3
$Header: FAUtility.java 120.0.12000000.3 2009/07/22 10:05:27 glchen noship $;

Finally issue got fixed by applying the patch.

Yours comments are most welcome !!!

Oracle Release 12 : "Diagnostics: Apps Check" Concurrent Request

Readers,

Yesterday, When I ran " Diagnostics: Apps Check" Concurrent request in our Production Release 12 instance, it failed and found the following error in the log file,

+-----------------------------
| Starting concurrent program execution...
+-----------------------------

Arguments
------------
201

------------
+---------------------------------------------------------------------------+
Start of log messages from FND_FILE
+---------------------------------------------------------------------------+
+---------------------------------------------------------------------------+
End of log messages from FND_FILE
+---------------------------------------------------------------------------+
/bin/ksh: line 1: f60desm: not found


Concurrent Manager encountered an error while running SQL*Plus for your concurrent request 18203944.

Review your concurrent request log and/or report output file for more detailed information.


+---------------------------------------------------------------------------+


So, the error is " /bin/ksh: line 1: f60desm: not found " and f60desm is not available in Release 12 instance.

When I investigate further, I found the patch 8310984 which will fix this issue as mentioned in the Oracle Support Note : Diagnostics: Apps Check report with parameter Assets error: bin/ksh: f60desm: not found [ID 863890.1]

Because, our current file version in the system is,

strings -a OMCHECK.sql |grep '$Header'
$Header: OMCHECK.sql 120.2.12000000.9 2008/09/30 06:39:10 vybhatia ship $

Note : Script contains host f60desm help=yes >> &output_file

But, Patch provides the following upper version,

$Header: OMCHECK.sql 120.2.12000000.13 2009/05/23 07:55:26 nshah ship $

Note : Script contains host frmbld help=yes >> &output_file

After applying the patch, " Diagnostics: Apps Check" Concurrent request got completed successfully.

Yours comments are most welome !!!

Thursday, March 25, 2010

Oracle Release 12 : Concurrent Programs are failing with ORA-20100

Dear Readers,

Recently we had ORA-20100: Error: FND_FILE failure. Unable to create file when users were submitting the concurrent request.

The following error message occurred in the logfile,

----------------------------------------------------
Error Is : ORA-20100: Error: FND_FILE failure. Unable to create file, l0862319.tmp in the directory, /apps/orarpt/TEST/utl.
You will find more information in the request log.
-----------------------------------------------------

When we checked the directory /apps/orarpt/TEST/utl at database side, the tmp l0862319.tmp file created at the time of program execution.

So, all directory/file system permission settings are perfect.

Later when we tried to write 100 lines ( from Application node as APPS users ) in tmp on /apps/orarpt/TEST/utl directory using UTL_FILE package, we were able to create it.

set serveroutput on
DECLARE
file_location VARCHAR2(256) := '/apps/orarpt/TEST/utl';
file_name VARCHAR2(256) := 'utlfile.lst';

idx number := 1;
file_id UTL_FILE.file_type;
BEGIN
file_id := UTL_FILE.fopen(file_Location,file_name, 'W');
for idx in 1..100 loop
UTL_FILE.put_line(file_id, idx);
end loop;
UTL_FILE.fclose(file_id);
EXCEPTION
WHEN UTL_FILE.INVALID_PATH

THEN dbms_output.put_line('Invalid path ' || SQLERRM);
WHEN OTHERS
THEN dbms_output.put_line('Others '|| SQLCODE || ' ' || SQLERRM);
END;

But when I tried to write 1000 lines, I was getting the following error,

SQL> set serveroutput on
DECLARE
file_location VARCHAR2(256) := '/apps/orarpt/TEST/utl';
file_name VARCHAR2(256) := 'utlfile1.lst';

idx number := 1;
file_id UTL_FILE.file_type;
BEGIN
file_id := UTL_FILE.fopen(file_Location,file_name, 'W');
for idx in 1..1000 loop
UTL_FILE.put_line(file_id, idx);
end loop;
UTL_FILE.fclose(file_id);
EXCEPTION
WHEN UTL_FILE.INVALID_PATH

THEN dbms_output.put_line('Invalid path ' || SQLERRM);
WHEN OTHERS
THEN dbms_output.put_line('Others '|| SQLCODE || ' ' || SQLERRM);
END; SQL> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20 /
Others -29285 ORA-29285: file write error

PL/SQL procedure successfully completed.
SQL>

After doing lot of investigation, I bounced all DB services including the DB Listener and It fixed the issue.

After Bounce :

SQL> set serveroutput on
DECLARE
file_location VARCHAR2(256) := '/apps/orarpt/TEST/utl';
file_name VARCHAR2(256) := 'utlfile3.lst';

idx number := 1;
file_id UTL_FILE.file_type;
BEGIN
file_id := UTL_FILE.fopen(file_Location,file_name, 'W');
for idx in 1..10000 loop
UTL_FILE.put_line(file_id, idx);
end loop;
UTL_FILE.fclose(file_id);
EXCEPTION
WHEN UTL_FILE.INVALID_PATH

THEN dbms_output.put_line('Invalid path ' || SQLERRM);
WHEN OTHERS
THEN dbms_output.put_line('Others '|| SQLCODE || ' ' || SQLERRM);
END; SQL> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
20 /

PL/SQL procedure successfully completed.

SQL>


And users confirmed that the concurrent programs are completing successfully.

So, Not sure what is the Root cause for this issue, I just simply bounced DB services including listener.

Comments are most welcome.

Thanks for reviewing our blog...

Wednesday, March 24, 2010

Oracle Release 12 : Autconfig failed when executing the adgentns.pl

Issue :

Last week, we had the following autoconfig issue on one of our development instance,

##########################################################################
Generate Tns Names
##########################################################################
ERROR: apps/bin/jre and apps/bin/java not found
Pass option 'java' on command line

Error in getting ClassPath
Logfile: /apps/local/SID/inst/apps/SID_hostname/admin/log/03222220/NetServiceHandler.log
Classpath :

Updating s_tnsmode to 'generateTNS'
UpdateContext exited with status: 1
Unrecognized option: -contextfile
Could not create the Java virtual machine.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
Instantiating Tools tnsnames.ora
Error in instantiating tools tnsnames.ora:
Error in instantiating web tnsnames.ora:

adgentns.pl exiting with status 256
ERRORCODE = 256 ERRORCODE_END
.end std out.
Exception in thread "main" java.lang.NoClassDefFoundError: /apps/local/SID/inst/apps/SID_hostname/appl/admin/SID_hostname/xml
Unrecognized option: -e
Could not create the Java virtual machine.
Unrecognized option: -e
Could not create the Java virtual machine.

.end err out.


Result : FAILED
===========================================================================


Solution :

As per the Oracle Doc : Autoconfig fails when running Adgentns.Pl with ERROR: apps/bin/jre and apps/bin/java not found [ID 731685.1], We performed the following,

Replace current adgentns.pl by adgentns.pl version120.4.12000000.3

1.1 Download patch 7207440 TXK - 12.0.4 Consolidated Patch 1 from Metalink
Do not apply it
1.2 Extract the file adgentns.pl 120.4.12000000.3 from patch 7207440
1.3 Backup your current adgentns.pl
1.4 Replace your current file by this file adgentns.pl 120.4.12000000.3
1.5 Retest autoconfig.

Finally it fixed our issue.

Thanks.

Friday, March 19, 2010

Oracle Release 12 File System Changes : R12 Vs 11i

Dear Readers,

Here are the differences on file systems level between Oracle Release 12i and Oracle Application 11i,

File

Oracle Applications 11i

Release 12

Environment Source file APPSORA.env APPS_.env
Context File (middle tier) $APPL_TOP/admin/$TWO_TASK.xml $INST_TOP/appl/admin/$TWO_TASK_.xml

tnsnames.ora (OH)

$ORACLE_HOME/network/admin/ $INST_TOP/ora/10.1.2/network/admin
listener.ora $ORACLE_HOME/network/admin/ $INST_TOP/ora/10.1.2/network/admin
appsweb.cfg

$OA_HTML/bin

$INST_TOP/ora/10.1.2/forms/server
tnsnames.ora (Apache)

$IAS_ORACLE_HOME/network/admin/

$INST_TOP/ora/10.1.3/network/admin
zone.properties $IAS_ORACLE_HOME/Apache/Jserv/etc

$ORA_CONFIG_HOME/10.1.3/j2ee/oacore/application-deployments/oacore/oa_servlets/orion-web.xml and .../oacore/html/orion-web.xml
(Same file located in both directories)

jsev.properties $IAS_ORACLE_HOME/Apache/Jserv/etc

$INST_TOP/ora/10.1.3/opmn/conf/opmn.xml

httpd.conf $IAS_ORACLE_HOME/Apache/Apache/conf $ORA_CONFIG_HOME/10.1.3/Apache/Apache/conf
apps.conf $IAS_ORACLE_HOME/Apache/Apache/conf $ORA_CONFIG_HOME/10.1.3/Apache/Apache/conf
formservlet.ini

$IAS_ORACLE_HOME/Apache/Jserv/etc

$ORACLE_HOME/forms/server/default.env

topfile.txt

$APPL_TOP/admin

$APPL_CONFIG_HOME/admin

adovars.env

$APPL_TOP/admin

$APPL_CONFIG_HOME/admin

adjborg2.txt

$APPL_TOP/admin

$APPL_CONFIG_HOME/admin

SSL Certificates

$COMMON_TOP/admin/certs

$INST_TOP/certs

AD scripts logs

$COMMON_TOP/admin/log/

$LOG_HOME/appl/admin/log

Concurrent Request logs

$APPLCSF

$APPLCSF

Apache logs

$IAS_ORACLE_HOME/Apache/Apache/logs

$LOG_HOME/ora/10.1.3/Apache

Jserv logs $IAS_ORACLE_HOME/Apache/Jserv/logs $LOG_HOME/ora/10.1.3/j2ee
javacache.log $COMMON_TOP/rgf/ $LOG_HOME/appl/rgf

Environment variables :

Variable

Oracle Applications 11i

Release 12

APPL_TOP $HOME/appl $HOME/apps/apps_st/appl

COMMON_TOP

$HOME/comn $HOME/apps/apps_st/comn
ORACLE_HOME (applmgr) $HOME/ora/8.0.6 $HOME/apps/tech_st/10.1.2
IAS_ORACLE_HOME $HOME/ora/iAS $HOME/apps/tech_st/10.1.3
ORACLE_HOME (oracle) $HOME/db/10.2.0 $HOME/db/tech_st/10.2.0
ORADATA $HOME/data $HOME/db/apps_st/data
JAVA_TOP, OA_JAVA $COMMON_TOP/java $COMMON_TOP/java/classes
OA_HTML $COMMON_TOP/html $COMMON_TOP/webapps/oacore/html
FND_SECURE $FND_TOP/secure/ $INST_TOP/appl/fnd/12.0.0/secure
ADMIN_SCRIPTS_HOME $COMMON_TOP/admin/scripts/ $INST_TOP/admin/scripts
LOG_HOME - $INST_TOP/logs
FORMS_WEB_CONFIG_FILE - $INST_TOP/ora/10.1.2/forms/server/appsweb.cfg
AF_JLIB - $COMMON_TOP/java/lib
JAVA_BASE - $COMMON_TOP/java
INST_TOP - $HOME/inst/apps/

ORA_CONFIG_HOME

-

$INST_TOP/ora

APPLCSF

$COMMON_TOP/admin

$LOG_HOME/appl/conc


Thanks.

Tuesday, March 2, 2010

RPM Dependency Loop

I had to install that dependency package for 11g R1.

When applying the necessary RPM , I came accross this dependency loop situation.
In general it is a loop of the dependency i.e, when you first install a package it will ask for another package and when trying to apply the requested package the dependency checks ask for the parent package.

[root@skrajend rpm]# rpm -ivh elfutils-libelf-devel-0.137-3.el5.i386.rpm
error: Failed dependencies:
        elfutils-libelf-devel-static-i386 = 0.137-3.el5 is needed by elfutils-libelf-devel-0.137-3.el5.i386
[root@skrajend rpm]# rpm -ivh elfutils-libelf-devel-static-0.137-3.el5.i386.rpm
error: Failed dependencies:
        elfutils-libelf-devel-i386 = 0.137-3.el5 is needed by elfutils-libelf-devel-static-0.137-3.el5.i386
[root@skrajend rpm]#


I tried with yum but no success.
[root@skrajend rpm]# yum install elfutils-libelf-devel-0.137-3.el5.i386.rpm
Loaded plugins: security
Setting up Install Process
Examining elfutils-libelf-devel-0.137-3.el5.i386.rpm: elfutils-libelf-devel-0.137-3.el5.i386
Marking elfutils-libelf-devel-0.137-3.el5.i386.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package elfutils-libelf-devel.i386 0:0.137-3.el5 set to be updated
--> Processing Dependency: elfutils-libelf-devel-static-i386 = 0.137-3.el5 for package: elfutils-libelf-devel
--> Finished Dependency Resolution
elfutils-libelf-devel-0.137-3.el5.i386 from /elfutils-libelf-devel-0.137-3.el5.i386 has depsolving problems
  --> Missing Dependency: elfutils-libelf-devel-static-i386 = 0.137-3.el5 is needed by package elfutils-libelf-devel-0.137-3.el5.i386 (/elfutils-libelf-devel-0.137-3.el5.i386)
Error: Missing Dependency: elfutils-libelf-devel-static-i386 = 0.137-3.el5 is needed by package elfutils-libelf-devel-0.137-3.el5.i386 (/elfutils-libelf-devel-0.137-3.el5.i386)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.
[root@skrajend rpm]#


Solution is to use wild cards in the rpm command.

[root@skrajend rpm]# rpm -ivh elfutils-libelf-devel-*
Preparing...                ########################################### [100%]
   1:elfutils-libelf-devel-s########################################### [ 50%]
   2:elfutils-libelf-devel  ########################################### [100%]
[root@skrajend rpm]#

[root@skrajend rpm]# rpm -qa|grep -i elfutils-libelf-devel
elfutils-libelf-devel-static-0.137-3.el5
elfutils-libelf-devel-0.137-3.el5
[root@skrajend rpm]#

Finally got those loop dependency patches installed.

Sunday, February 28, 2010

10g Application Server Installation and Errors

Had to setup 10g Application Server on Oracle Enterprise Linux 5 and came across few errors.


1. Infra-tire Installation failed to startup the service.
Error:
httpd: error while loading shared libraries: libdb.so.2

Solution: 
create a symbolic link "ln -s /usr/lib/libgdbm.so.2.0.0 /usr/lib/libdb.so.2" 


2. Mid-tire Installation failed

Error:
/usr/lib/libXtst.so.6: undefined reference to `__stack_chk_fail@GLIBC_2.4'
/usr/lib/libXtst.so.6: undefined reference to `__fprintf_chk@GLIBC_2.3.4'
/usr/lib/libXtst.so.6: undefined reference to `__sprintf_chk@GLIBC_2.3.4'
collect2: ld returned 1 exit status
make: *** [rwproxy] Error 1


Solution:
A) The following i386 packages are not part of the OS distribution media and must be downloaded separately (from http://oss.oracle.com/projects/compat-oracle/files/Enterprise_Linux for both OEL 5 and RHEL 5) and installed manually:

* openmotif21-2.1.30-11.EL5.i3861
* xorg-x11-libs-compat-6.8.2-1.EL.33.0.1.i386
* binutils-2.17.50.0.6-6.0.1.i3862

B) Link to Motif library for Oracle Application Server 10.1.2 (on OEL 5 and RHEL 5 only)

Perform the following command (as root on your system) to update a required link to a Motif library prior to relinking or patching the 10.1.2 Application Server Oracle Home:

# unlink /usr/lib/libXtst.so.6
# ln -s /usr/X11R6/lib/libXtst.so.6.1 /usr/lib/libXtst.so.6



3. MiddleTire Installation failed to start the Discoverer Service


Errors:
dis51pr: error while loading shared libraries: libpthread.so.0


Solution:
Remove LD_ASSUME_KERNEL parameter from the opmn.xml (in the Discoverer section) file then reloaded it with "opmnctl reload"

ompnctl startproc ias-component=Discoverer

Friday, February 26, 2010

Clearing Cache in Oracle EBS R12

Readers,

Clearing the _pages in R12 creates blank login page issue, as in R12 the jsp files does not get compiled automatically.

For clearing middle tier cache :

+ Navigate to Functional Administrator responsibility > Core Services > Caching Framework > Global Configuration > Clear cache
+ Bounce Apache and OACORE services.

Comments are most welcome...

Thanks.

R12 : Getting Blank Page

Dear Readers,

Recently we had an issue with one of our development instance where the clients were getting blank page when they were accessing the web URL.

After investigating the log files under $LOG_HOME/ora/10.1.3/Apache/*, $LOG_HOME/ora/10.1.3/opmn/* and $LOG_HOME/ora/10.1.3/j2ee/* directory, I came to know that someone removed _pages directory from $COMMON_TOP and the _pages directory was empty.

When I checked the file $INST_TOP/ora/10.1.3/j2ee/oacore/application-deployments/oacore/html/orion-web.xml, I saw the following entries,

<init-param>
<param-name>main_mode</param-name>
<param-value>justrun</param-value>
</init-param>
</servlet>

So, When main_mode = justrun the OC4J container running the OACoreGroup is told that no compilation on the fly is allowed and only (pre)compiled classes are picked up. Since these have been removed the processing of the JSP page is blocked.

Finally as workaround, we regenerated the JSP file using the following commends,

$FND_TOP/patch/115/bin/perl ojspCompile.pl --compile --flush -p 2

and as Long Team Fix, we made the following changes on $INST_TOP/ora/10.1.3/j2ee/oacore/application-deployments/oacore/html/orion-web.xml file on all FE servers,

From :

<init-param>
<param-name>main_mode</param-name>
<param-value>justrun</param-value>
</init-param>
</servlet>

To :

<init-param>
<param-name>main_mode</param-name>
<param-value>recompile</param-value>
</init-param>
</servlet>

and Changed the following in $CONTEXT_FILE,

From : <jsp_debug_parameters oa_var="s_jsp_main_mode">justrun</jsp_debug_parameters>
To : <jsp_debug_parameters oa_var="s_jsp_main_mode">recompile</jsp_debug_parameters>

Reference Oracle Support Note : JSP Pages Hanging in R12 After Removing Cached Class Files in _pages [ID 433386.1]

Comments are most welcome.....

Thanks.