Showing posts with label Oracle. Show all posts
Showing posts with label Oracle. Show all posts

Wednesday, February 8, 2012

Changing the Oracle Character Set after installation

Sometimes when creating an Oracle Database, we forget to choose the correct character set on the relevant dbca screen. Then, when we then go to import data or create data we discover that the character set is not correct. With Oracle 11g, you can actually change it after the database creation. Here is sql plus commands to make this happen:

conn / as sysdba
-------
SHUTDOWN IMMEDIATE;
STARTUP RESTRICT;
ALTER SYSTEM SET JOB_QUEUE_PROCESSES=0;
ALTER SYSTEM SET AQ_TM_PROCESSES=0;
-------
ALTER DATABASE CHARACTER SET EL8ISO8859P7;
--- EL8ISO8859P7 is greek

-- if the above fails:
ALTER DATABASE CHARACTER SET INTERNAL_USE EL8ISO8859P7;
SHUTDOWN IMMEDIATE;
STARTUP;
 
---if all this is not working run the following command to reload the stylesheet
dbms_metadata_util.load_stylesheets

Wednesday, January 11, 2012

Transferring Data Of a Big 9i Oracle Database to 11gR2

Currently I was involved in upgrading a 40 GB Oracle database installation from 9i to 11g release 2. In place upgrade was out of the question since (1) We moved from Solaris to Linux and (2) Downtime should be kept to a minimum

The 9i setup consisted of an Oracle 9i enterprise server, with multiple database instances. One instance had 40 GB of data. This is the procedure we used, which I believe is the quickest way possible to move this amount of data:

On the 9i database server:
  1. Identify and Clear all tables with temporary data.  This includes temporary report tables, backup tables, log tables etc.  In our case, this saved us a gigabyte. 
  2. Export all database intances using the exp utility on the 9i server. Be careful to not allow connections to the database while exporting since sequences can be out of sync.  Do not export the sys user schema.
  3. Create scripts to create tablespaces, schema users and oracle directories 
Now, on the 11g database server
Note: Steps 1, 2 and 3 below can be scripted using bash and sqlplus.
  1. Create all tablespaces on the target 11g database.  Tablespace reorganization complicates the migration so I would advice against it.  Plus, the imp utility expects to find the same tablespaces when importing data of tables with CLOB or BLOB fields    
  2. Create all schema users on the target 11g database.
  3. Create all directories on the target 11g database, both on the database and also on the operating system level and grant the appropriate rights to the appropriate users.
  4. Run import with ROWS=N CONSTRAINTS=Y INDEXFILE=INDEXES.SQL, to create sql statement script for index creation.  Here's an example below.
    $ export ORACLE_SID=ORADB
    $ imp \'sys/oracle as sysdba\' file=/home/oracle/expdat.dmp log=createIndexes.log FROMUSER=oracleuser TOUSER=oracleuser ROWS=n INDEXES=Y CONSTRAINTS=Y INDEXFILE=INDEXES.SQL
  5. Run imp with options ROWS=n INDEXES=N CONSTRAINTS=N STATISTICS=NONE This will create empty tables in the database w/o constraints and indexes, and also import procedures, triggers, and sequences.
  6. Disable Constraints and Triggers in the database.  (click to get the scripts)
  7. Imports table data by running the imp utility with ANALYZE=n BUFFER=100000000 recordlength=65535 FEEDBACK=10000 IGNORE=Y ROWS=Y INDEXES=N CONSTRAINTS=N STATISTICS=NONE
  8. In sqlplus run INDEXES.SQL (created in step 4 above) to create database indexes
  9. Analyze tables in database.  In sqlplus run:
    exec dbms_stats.gather_schema_stats( 
         ownname          => 'oracleuser', 
         estimate_percent => dbms_stats.auto_sample_size, 
         method_opt       => 'for all columns size repeat', 
         degree           => 34 );
    

Friday, November 25, 2011

SQL Plus script to disable/enable all triggers an Oracle database

Below is an SQL Plus script to disable/enable all triggers for the current user in an Oracle database.  This is useful when importing data in an oracle database and you do not want triggers firing.  Substitute "DISABLE" below with "ENABLE" to enable back triggers after data import.

begin
  for c in (select * from user_triggers a where a.trigger_name not like '%$xd' )
  loop
      execute immediate('alter trigger '||c.trigger_name||' DISABLE');
  end loop;
end;

Thursday, November 17, 2011

Scripts to disable/enable all constraints in an Oracle database

Use the scripts below to enable/disable constraints for an oracle user.  You can create them as procedures in your oracle schema, or simply execute them in sqlplus without the create procedure parts.


create or replace procedure sp_ddl_cons_disable as
begin
for c in (select 'ALTER TABLE '|| a.table_name ||' DISABLE constraint ' ||  a.constraint_name as command
  from user_constraints a
 where a.constraint_type in ('R'))
 loop
       ---dbms_output.put_line(c.command);
       EXECUTE IMMEDIATE (c.command);
 end loop;

end;




create or replace procedure sp_ddl_cons_enable as
begin
for c in (select 'ALTER TABLE '|| a.table_name ||' ENABLE constraint ' ||  a.constraint_name as command
  from user_constraints a
 where a.constraint_type in ('R'))
 loop
       ---dbms_output.put_line(c.command);
       EXECUTE IMMEDIATE (c.command);
 end loop;

end;

Tuesday, November 15, 2011

Sending UTF-8 encoded HTML Email using UTL_SMTP

The PL/SQL procedure below can send html email using the UTL_SMTP Oracle built in package. It handles utf-8 strings in the subject, To/From Name fields and in the message itself.
It was tested on an Oracle 11g R2 database with UTF-8 NLS_CHARACTERSET set and an Exchange smtp server.
CREATE OR REPLACE PROCEDURE send_html_email(p_sender_name VARCHAR2, -- name of person sending email
                                            p_sender_mail VARCHAR2, -- email of person sending email
                                            p_recipient   VARCHAR2, -- recipient of contact email
                                            p_subject     VARCHAR2, -- subject of email
                                            p_message     VARCHAR2 -- message of email
                                            ) is

  conn UTL_SMTP.CONNECTION;

  v_smtp_hostname varchar(200) := 'smtp_server';
  v_smtp_uname    VARCHAR2(200) := 'smtp_server_uname';
  v_smtp_passwd   VARCHAR2(200) := 'smtp_server_password';
  v_sender        VARCHAR2(200) := null;
 
  CHAR_SET constant varchar(200) := 'Content-Type: text/html;charset=UTF-8' ||UTL_TCP.CRLF;
  MIME_VERSION constant varchar(200) := 'MIME-version: 1.0' || UTL_TCP.CRLF;
  CONT_ENCODING constant varchar(200) := 'Content-Transfer-Encoding: quoted-printable ' ||UTL_TCP.CRLF;

BEGIN

  if p_message is null then
    return;
  end if;

  conn := utl_smtp.open_connection(v_smtp_hostname, 25);

  if v_smtp_uname is not null then
    UTL_SMTP.ehlo(conn, v_smtp_hostname);
  
    UTL_SMTP.command(conn, 'AUTH LOGIN');
    UTL_SMTP.command(conn,
                     utl_raw.cast_to_varchar2(utl_encode.base64_encode(utl_raw.cast_to_raw(v_smtp_uname))));
    UTL_SMTP.command(conn,
                     utl_raw.cast_to_varchar2(utl_encode.base64_encode(utl_raw.cast_to_raw(v_smtp_passwd))));
  
  else
    UTL_SMTP.helo(conn, v_smtp_hostname);
  
  end if;

  UTL_SMTP.Helo(conn, v_smtp_hostname);
  UTL_SMTP.Mail(conn, p_sender_mail);
  UTL_SMTP.rcpt(conn, p_recipient);
  UTL_SMTP.OPEN_DATA(conn);

  UTL_SMTP.WRITE_DATA(conn, MIME_VERSION);
  UTL_SMTP.WRITE_DATA(conn, CHAR_SET);
  UTL_SMTP.WRITE_DATA(conn, CONT_ENCODING);

  UTL_SMTP.WRITE_DATA(conn,
                      'Date: ' || TO_CHAR(SYSDATE, 'dd Mon yy hh24:mi:ss') ||
                      ' -0800 (GMT)' || UTL_TCP.CRLF);
  UTL_SMTP.write_raw_data(conn,
                          utl_raw.cast_to_raw('Subject:' || p_subject));
  UTL_SMTP.WRITE_DATA(conn, UTL_TCP.CRLF); -- this crlf is for 'Subject' field

  if p_sender_name is null then
    v_sender := p_sender_mail;
  else
    v_sender := '"' || p_sender_name || '" <' || p_sender_mail || '>';
  end if;
  UTL_SMTP.write_raw_data(conn, utl_raw.cast_to_raw('From:' || v_sender));
  UTL_SMTP.WRITE_DATA(conn, UTL_TCP.CRLF); -- this crlf is for 'From' field
  UTL_SMTP.WRITE_DATA(conn, 'To: ' || p_recipient || UTL_TCP.CRLF);
  UTL_SMTP.WRITE_DATA(conn, UTL_TCP.CRLF);
  UTL_SMTP.WRITE_RAW_DATA(conn,
                          UTL_ENCODE.QUOTED_PRINTABLE_ENCODE(UTL_RAW.CAST_TO_RAW(p_message)));
  UTL_SMTP.WRITE_DATA(conn, UTL_TCP.CRLF);
  UTL_SMTP.CLOSE_DATA(conn);
  UTL_SMTP.QUIT(conn);
END;

Thursday, September 15, 2011

Manual Auditing of Oracle data changes using "Mirror" tables

In one of our projects we had a requirement to log every data change on certain tables, and keep a history of each record. We came across this article by Michael Klaene. We thank him and re publish the code here just in case his page is pulled off the net.
Part 1: Create Mirror Tables
Run the procedure below to create a "Mirror" table for each data table in the database.
create or replace procedure usp_create_audit_tables is
  lv_precision_and_scale VARCHAR2(20);

  --Select tables w/o an audit table
  CURSOR cur_tbl2audit IS
    SELECT table_name
      FROM user_tables 
      WHERE TABLE_NAME NOT LIKE '%$AUD'; -- SKIP AUDIT TABLES, *$AUD 
/**  
  here you can add a WHERE clause to exclude or include certain tables.
  for example: WHERE table_name like 'DATA_PROC%' **/

  --Select table def of unaudited table.
  CURSOR cur_col2audit(p_tbl2audit USER_TABLES.TABLE_NAME%TYPE) IS
    SELECT column_name, data_type, data_length, data_precision, data_scale
      FROM user_tab_columns
     WHERE table_name = p_tbl2audit
          --Add ineligible datatypes here :
       AND data_type NOT IN ('BLOB', 'CLOB', 'RAW')
     ORDER BY column_id;

   v_sql varchar2(4000);

BEGIN
  --Retrieve table names:
  FOR cur_tbl2audit_rec IN cur_tbl2audit LOOP
      v_sql := null;

      v_sql := v_sql || ('CREATE TABLE ' ||
                         SUBSTR(cur_tbl2audit_rec.table_name, 1, 26) ||
                         '$AUD (');

    --Retrieve table columns:
    FOR cur_col2audit_rec IN cur_col2audit(cur_tbl2audit_rec.table_name) LOOP
      IF cur_col2audit_rec.data_type = 'NUMBER' THEN

        --Add precision for NUMBER or provide a default.
        IF cur_col2audit_rec.data_precision IS NULL THEN
          lv_precision_and_scale := '38,0)';
        ELSE
          lv_precision_and_scale := cur_col2audit_rec.data_precision || ',' ||
                                    cur_col2audit_rec.data_scale || ')';
        END IF;

        --RPAD adds spaces for easier reading.
        v_sql := v_sql || (RPAD(cur_col2audit_rec.column_name, 35) ||
                             cur_col2audit_rec.data_type || '(' ||
                             lv_precision_and_scale || ',');

      ELSIF cur_col2audit_rec.data_type IN ('CHAR', 'VARCHAR', 'VARCHAR2') THEN
        v_sql := v_sql || (RPAD(cur_col2audit_rec.column_name, 35) ||
                             cur_col2audit_rec.data_type || '(' ||
                             cur_col2audit_rec.data_length || '),');

      ELSE
        -- no length required.
        v_sql := v_sql || (RPAD(cur_col2audit_rec.column_name,
                                                          35) ||
                                                     cur_col2audit_rec.data_type || ',');
      END IF;
    END LOOP;

    --Add audit fields to table:
    v_sql := v_sql || ('aud_action CHAR(3),aud_timestamp DATE,aud_user VARCHAR2(30),AUD_USER_IPADDRESS VARCHAR2(30) )');
    --v_sql := v_sql || ('-----');
    execute immediate(v_sql);
    --dbms_output.put_line(v_sql);

  END LOOP;

end;
Part 2: Create Triggers to maintain audit data
The second script creates a trigger in each audited table to maintain the "mirror" tables created above.

create or replace procedure usp_create_audit_triggers is
  v_prefix    VARCHAR2(5) := NULL;
  v_condition VARCHAR2(30) := NULL;

  --Select all user tables with a corresponding audit table.
  CURSOR cur_tbl2audit IS
    SELECT table_name
      FROM user_tables a
     WHERE table_name NOT LIKE '%$AUD'
       AND EXISTS
     (SELECT 'x'
              FROM user_tables b
             WHERE b.table_name = SUBSTR(a.table_name, 1, 26) || '$AUD');

  --Select table def of audit table, sans audit columns.
  CURSOR cur_col2audit(p_audittbl USER_TABLES.TABLE_NAME%TYPE) IS
    SELECT column_name
      FROM user_tab_columns
     WHERE table_name = p_audittbl
       AND column_name NOT IN ('AUD_ACTION', 'AUD_TIMESTAMP', 'AUD_USER', 'AUD_USER_IPADDRESS')
       AND data_type NOT IN ('BLOB', 'CLOB', 'RAW')
     ORDER BY column_id;
    
   m_sql varchar2(32000):=null;
     
BEGIN
  FOR cur_tbl2audit_rec IN cur_tbl2audit LOOP
    m_sql := null;
    m_sql :=  'CREATE OR REPLACE TRIGGER AUDTRG$' ||
                         SUBSTR(cur_tbl2audit_rec.table_name, 1, 23) ||
                          CHR(10) ||
                         ' AFTER INSERT OR DELETE OR UPDATE ' || 'ON ' ||
                         cur_tbl2audit_rec.table_name || ' FOR EACH ROW '|| CHR(10);

    v_prefix    := ':new';
    v_condition := 'IF INSERTING OR UPDATING THEN';
    m_sql := m_sql || '-- trigger autogenerated on '|| to_char(sysdate,'dd-mm-yyyy hh24:mi:ss') ||chr(10); 
    m_sql := m_sql || 'DECLARE ' || CHR(10) ||
                         'v_operation VARCHAR2(10) := NULL;'|| CHR(10);
    m_sql := m_sql || 'BEGIN ' || CHR(10);
    IF v_prefix = ':new' THEN
      m_sql := m_sql || '    IF INSERTING THEN ' || CHR(10) ||
                           '       v_operation := ''INS''; ' || CHR(10) ||
                           '    ELSIF UPDATING THEN ' || CHR(10) ||
                           '       v_operation := ''UPD''; ' || CHR(10) ||
                           '    ELSE ' || CHR(10) ||
                           '       v_operation := ''DEL''; ' || CHR(10) ||
                           '    END IF; ' || CHR(10);
    END IF;

    LOOP
      m_sql := m_sql || v_condition || CHR(10);
      m_sql := m_sql || ' INSERT INTO ' ||
                           SUBSTR(cur_tbl2audit_rec.table_name, 1, 26) ||
                           '$AUD (';

      --Loop through 1st to get column names:
      FOR cur_col2audit_rec IN cur_col2audit(cur_tbl2audit_rec.table_name) LOOP
        m_sql := m_sql || cur_col2audit_rec.column_name || ',';
      END LOOP;

      m_sql := m_sql || 'aud_action,aud_timestamp,aud_user,AUD_USER_IPADDRESS) ' ||
                           'VALUES (';

      --Loop a 2nd time for the values:
      FOR cur_col2audit_rec IN cur_col2audit(cur_tbl2audit_rec.table_name) LOOP
        m_sql := m_sql ||v_prefix || '.' ||
                             cur_col2audit_rec.column_name || ',';
      END LOOP;

      m_sql := m_sql ||'v_operation,SYSDATE,SYS_CONTEXT(''USERENV'', ''CLIENT_IDENTIFIER'')';
      m_sql := m_sql ||',SYS_CONTEXT(''USERENV'', ''CLIENT_INFO''));' || CHR(10);

      EXIT WHEN v_prefix = ':old';
      v_prefix    := ':old';
      v_condition := 'ELSE ';
    END LOOP;

    m_sql := m_sql || ' END IF;' || CHR(10) || 'END;' ;
    --v_sql := v_sql || CHR(10) || '/' || CHR(10);
    
    begin
      execute immediate(m_sql);
    exception when others then
     dbms_output.put_line(sqlerrm);
     dbms_output.put_line( length( m_sql));
     dbms_output.put_line( m_sql);
    end;
    
    --exit;
  END LOOP;


END;
Part 3: Procedure to maintain audit tables.
This final procedure can be called to synchronize audit tables with their "parent". It checks for dropped and added fields on every audit table.
create or replace procedure usp_refresh_audit_tables is

  lv_precision_and_scale VARCHAR2(20);

  --Select tables w/o an audit table
  CURSOR cur_tbl2audit IS
    SELECT table_name
      FROM user_tables
     WHERE (table_name like 'DATA_PROC%' or table_name like 'XML%' or
           table_name like 'MANUAL%')
       and table_name not like '%$AUD';
       
  --Select table def of unaudited table.
  CURSOR cur_col2audit(p_tbl2audit USER_TABLES.TABLE_NAME%TYPE) IS
    SELECT column_name, data_type, data_length, data_precision, data_scale
      FROM user_tab_columns
     WHERE table_name = p_tbl2audit
     ORDER BY column_id;

  v_sql            varchar2(4000);
  v_aud_table_name varchar2(30) := null;
  v_table_exists   number(10);
  v_col_exists     number(10);

BEGIN
  --Retrieve table names:
  FOR cur_tbl2audit_rec IN cur_tbl2audit LOOP
    v_sql            := null;
    v_aud_table_name := SUBSTR(cur_tbl2audit_rec.table_name, 1, 26) ||
                        '$AUD';
  
    --Retrieve table columns:
    FOR cur_col2audit_rec IN cur_col2audit(cur_tbl2audit_rec.table_name) LOOP
    
      select count(*)
        into v_table_exists
        from user_tables
       where table_name = v_aud_table_name;
    
      if v_table_exists > 0 then
      
        select count(*)
          into v_col_exists
          from user_tab_columns
         where column_name = cur_col2audit_rec.column_name
           and table_name = v_aud_table_name;
      
        if v_col_exists = 0 then
          v_sql := 'ALTER TABLE ' || v_aud_table_name || ' add ';
        
          IF cur_col2audit_rec.data_type = 'NUMBER' THEN
          
            --Add precision for NUMBER or provide a default.
            IF cur_col2audit_rec.data_precision IS NULL THEN
              lv_precision_and_scale := '38,0)';
            ELSE
              lv_precision_and_scale := cur_col2audit_rec.data_precision || ',' ||
                                        cur_col2audit_rec.data_scale || ')';
            END IF;
          
            --RPAD adds spaces for easier reading.
            v_sql := v_sql || cur_col2audit_rec.column_name || ' ' ||
                     cur_col2audit_rec.data_type || '(' ||
                     lv_precision_and_scale;
          
          ELSIF cur_col2audit_rec.data_type IN
                ('CHAR', 'VARCHAR', 'VARCHAR2') THEN
            v_sql := v_sql || cur_col2audit_rec.column_name || ' ' ||
                     cur_col2audit_rec.data_type || '(' ||
                     cur_col2audit_rec.data_length || ')';
          
          ELSE
            -- no length required.
            v_sql := v_sql || cur_col2audit_rec.column_name || ' ' ||
                     cur_col2audit_rec.data_type;
          end if;
        
          dbms_output.put_line(v_sql);
          execute immediate (v_sql);
        
        END IF;
      end if;
    
    END LOOP;
  
    -- now find colums that were dropped from the main table but still exist 
    -- in audit tables
    dbms_output.put_line('--- drop columns of table:' ||
                         cur_tbl2audit_rec.table_name || chr(13));
    for cdp in (SELECT column_name
                  FROM user_tab_columns
                 WHERE lower(table_name) = lower(v_aud_table_name)
                   and column_name not like 'AUD%'
                minus
                SELECT column_name
                  FROM user_tab_columns
                 WHERE lower(table_name) =
                       lower(cur_tbl2audit_rec.table_name)) loop
    
      v_sql := 'ALTER TABLE ' || v_aud_table_name || ' drop column ' ||
               cdp.column_name;
    
      dbms_output.put_line(v_sql);
      execute immediate (v_sql);
    
    end loop; -- loop of dropped fields
  
  END LOOP;

  usp_create_audit_triggers; -- call procedure to recreate triggers!

end;

Wednesday, July 13, 2011

Script to generate all Primary and Foreign Key Constraints In an Oracle database

STEPS TO EXECUTE:
  1. SAVE THE ATTACHED FILE TO C:\
  2. LOGIN TO SQLPLUS
  3. TYPE: @C:\getConstraints.sql
  4. WHEN THIS FINISHES, ALL DDL WILL BE IN FILE c:\constraintsDDL.sql
---------------- getConstraints.sql -------------------------------
prompt Exporting User Constraints....
set feedback off
set heading off
set termout off
set linesize 3000
set long 90000
set trimspool on
column CODE format a300
set verify off
prompt set define off
spool c:\constraintsDDL.sql
execute DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM, 'PRETTY', true);
execute DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM,'SQLTERMINATOR',true);
select trim(DBMS_METADATA.GET_DDL('CONSTRAINT',t.constraint_name))||';' as CODE
from user_constraints t where t.constraint_type='P';
select trim(DBMS_METADATA.GET_DDL('REF_CONSTRAINT',t.constraint_name))||';' as CODE
from user_constraints t where t.constraint_type='R';
spool off
prompt set define on
set feedback on
set heading on
set termout on
set linesize 100
-----------------------------------------------------

Thursday, May 19, 2011

PL/SQL proc to add a column to a table if it does not exist

create or replace function ddl_column_exists(p_table     in varchar2,
                                             p_fieldname in varchar2) return number is
  
  result number;

begin
  select count(*)
    into result
    from user_tab_columns c
   where lower(c.COLUMN_NAME) = lower(p_fieldname)
     and lower(c.TABLE_NAME) = lower(p_table);

  if result >0 then
    result := 1;
  else
    result := 0;
  end if;

  return(result);
end;
/
CREATE OR REPLACE PROCEDURE "DDL_USP_ADDCOL" (p_tblName varchar2,
                                              p_fldName varchar2,
                                              p_dtype   varchar2) iS
    v_ret number(2) := 0;
    v_sql varchar2(2000);
begin

    select count(*)
      into v_ret
      from user_tab_columns c
     where lower(c.COLUMN_NAME) = lower(p_fldName)
       and lower(c.TABLE_NAME) = lower(p_tblName);

    if v_ret = 0 then

        --print 'adding field'
        v_sql := 'ALTER TABLE ' || p_tblName || ' add ' || p_fldName || ' ' || p_dtype;
        execute immediate v_sql;

    end if;
end;

Thursday, April 14, 2011

Script to export all pl/code in an oracle database

---------------- getcode.sql -------------------------------
prompt Exporting User code....
set feedback off
set heading off
set termout off
set linesize 3000
set trimspool on
set verify off
prompt set define off
spool c:\policecode.sql
select text from (
select a.name, a.type, a.line, decode(line,1,'create or replace ', '' ) || text  as text
  from user_source a
union select a.name, a.type, 9000000000 as line, '/' from user_source a
order by name,type, line);
spool off
prompt set define on
set feedback on
set heading on
set termout on
set linesize 100
-----------------------------------------------------