Oracle reading alert log via sql

In many situations, an Oracle developer needs access to the alert log file to explain some problems occurring on the database. Errors that may be logged there are e.g. problems with tablespaces, online / archive log files, deadlocks or Ora-00600 errors. In the event of serious errors, detailed information is logged in trace files. The trace file name is written in the alert file. However, the alert file is located on the Oracle server and the developers usually do not have access to the Oracle server. Well, we are going to see how we can still access the content of this file through SQL.

From my own experience during the installation a new version of our software at the customer. The customer in Germany has a RAC database with two nodes. The database instances were somewhere in France. It was Friday very late and our software aborts when executing an SQL statement. The sessions just disappear without leaving any traces in our software logging. The DBA in France was unfortunately not available at this time (It was quite simply very late on Friday). In the log of our software, I could find nothing relevant. It was clear that the problem has to do with the Oracle Instance. In such cases, it would be very helpful to take look in the alert file for more information.

Then I came up with the following idea (I already knew that something like that is possible, I’ve read in a book): Luckily we had a database user who is allowed to create directories, external table and has select privelige on gv$process and gv$instance.

Why GV$INSTANCE? Because it was a RAC database so several nodes therefore GV$
The Alert file is called (most) alert_SID.log, where SID is instance_name: SELECT instance_name  FROM GV$INSTANCE

Why GV$PROCESS? Because tt was a RAC database so several nodes therefore GV$ With following query you can get the full path on the OS of the alert file.

SELECT DISTINCT INST_ID,SUBSTR(TRACEFILE,1,INSTR(TRACEFILE, '/', -1)) A  FROM GV$PROCESS 

This block conveniently generates the code to create the directories and the tables:

DECLARE
  STMT VARCHAR2(4000) := 'create table 
   alert_log__inst?int? ( msg varchar2(1024) )
organization external (
   type oracle_loader
   default directory dir4alertfile?int?
   access parameters (
      records delimited by newline
   )
   location(''alert_?SID?.log'')
)
reject limit 100000';
BEGIN
  FOR REC IN (SELECT INSTANCE_NAME, P.*
                FROM GV$INSTANCE I
                JOIN (SELECT DISTINCT INST_ID,
                                     SUBSTR(TRACEFILE,
                                            1,
                                            INSTR(TRACEFILE, '/', -1)) A
                       FROM GV$PROCESS) P
                  ON I.INST_ID = P.INST_ID) LOOP
    DBMS_OUTPUT.PUT_LINE('CREATE OR REPLACE directory dir4alertfile' ||
                         REC.INST_ID || ' as ''' || REC.A || '''');
    DBMS_OUTPUT.PUT_LINE(CHR(10) || CHR(10));
    DBMS_OUTPUT.PUT_LINE(REPLACE(REPLACE(STMT, '?int?', REC.INST_ID),
                                 '?SID?',
                                 REC.INSTANCE_NAME));
    DBMS_OUTPUT.PUT_LINE(CHR(10) || CHR(10));
  END LOOP;
END;

generated code:

--first directory (node1)
CREATE OR REPLACE directory dir4alertfile1 as '/u01/app/oracle/diag/rdbms/mydb/INST1/trace/'


--first external table (node1)
create table
   alert_log__inst1 ( msg varchar2(1024) )
organization external (
   type oracle_loader
   default directory dir4alertfile1
   access parameters (
      records delimited by newline
   )
   location('alert_INST1.log')
)
reject limit 100000

--second directory (node2)
CREATE OR REPLACE directory dir4alertfile2 as '/u01/app/oracle/diag/rdbms/mydb/INST2/trace/'


--second external table (node2)
create table
   alert_log__inst2 ( msg varchar2(1024) )
organization external (
   type oracle_loader
   default directory dir4alertfile2
   access parameters (
      records delimited by newline
   )
   location('alert_INST2.log')
)
reject limit 100000

We can use the same to read in the trace files.

Oracle Datapump Issues

ORA-31626: job does not exist

In this passage I would like to talk about my experience with the error „ORA-31626: job does not exist“.

I’ve written a program that exports a few tables using the dbms_datapump package. I implemented these and successfully tested them on the version 11.2.0.4. However, the program has triggered runtime errors at customer as follows:

ERROR in line 1:
ORA-31626: job does not exist
ORA-06512: in „SYS.DBMS_SYS_ERROR“, line 79
ORA-06512: in „SYS.DBMS_DATAPUMP“, line 1137
ORA-06512: in „SYS.DBMS_DATAPUMP“, line 5285
ORA-06512: in line 57

This error surprised me. What is different on customer environment now than on my environment? The program is created by me and the user who runs the program is scripted by me and the customer has the same Oracle version 11.2.0.4! It was not a compile error, it was a runtime error. So my first suspicion was that maybe some ORA-00600 error occurred. In particular, the message „job does not exist“ has confused me. So I requested the customer to check the alert file for error messages. But everything was normal in the alert file!

Well I have no trace………. All I know is that Datapump has to create a master table for all its work when exporting (and importing too). The master table belongs to system user and is created in the default User-tablespace.

I asked the customer if he could create a table with the user and insert some data into it. The answer came quickly. The customer could create the table but could not add any data. He gets the following errors: ora-01950 no privileges on tablespace ‚users‘. For me it was a surprise, but at least the explanation for the problem.

The user is created from a script by myself! The script grants the user „RESOURCE“ role. And I know that in Oracle 11.2.0.4 Quota is hard-coded on tablespaces in this role. It was revealed that the customer revoked this role from my user. After granting quota on Users tablespace to my user the program worked fine.