The first bug on Oracle Database 12c: how to create a PDB by cloning a remote existing pluggable database

1. Articles from Database administrator
workshop
The first bug on Oracle Database 12c: how to create a
PDB by cloning a remote existing pluggable database
In this post I want to describe how to create a PDB by cloning a remote existing
pluggable database: at the end of this process you should have a new PDB (in my
case PDB103) contained in a target CDB (CDB001) cloned from a source PDB
(PDB003) contained in a remote different CDB (CDBTEST).
In my case both source and target databases are located on the same machine.
When I took part at the presentation of the new Oracle Database 12c here in Rome I
took some pictures, but I was not ready with my phone when an interesting slide
was presented.
While testing the scenario of today I suddenly remembered about that missing
pictures: luckily few days later I received from Oracle also the pdf of the event so I'm
now able to show you the following screenshot:
In 2006 I tested the beta release of Oracle Database 11gR1 because at that time my
company needed to test new features for its products.
As you can read from the screenshot many test cases were conducted by Oracle
and by beta testers, but a bug is always behind the corner when you consider a
complex software solution as the Oracle Database and today I'm not able to close
my thread with the usual "That's all" comment.
Anyway let's start describing the content of my current containers. My machine has
two container databases named CDB001 and CBTEST.
view plainprint?
1. [oracle@vsi08devpom ~]$ ps -ef|grep smon
2. oracle 4081 1 0 Jul15 ? 00:01:23 ora_smon_CDB001
3. oracle 16455 1 0 Jul17 ? 00:01:25 ora_smon_CDBTEST
4. oracle 26501 26466 0 14:08 pts/1 00:00:00 grep smon
On the target database CDB001 there is only one pluggable database (PDB101)
view plainprint?
1. [oracle@vsi08devpom ~]$ env|grep SID
2. ORACLE_SID=CDB001
3. [oracle@vsi08devpom ~]$ sqlplus / as sysdba
4.
5. SQL@CDB001> select name, open_mode from V$PDBS;
6.
7. NAME OPEN_MODE
8. ------------------------------ ----------
9. PDB$SEED READ ONLY
10. PDB0101 READ WRITE
On the source database CDBTEST there are four pluggable databases
(PDBTEST1, PDBTEST2, PDBTEST3 and PDB003)
view plainprint?
1. [oracle@vsi08devpom ~]$ env|grep SID
2. ORACLE_SID=CDBTEST
3.
4. SQL@CDBTEST> select NAME, OPEN_MODE FROM V$PDBS;
5.
6. NAME OPEN_MODE
7. ------------------------------ ----------
8. PDB$SEED READ ONLY
9. PDBTEST1 READ WRITE
10. PDBTEST2 READ WRITE
11. PDBTEST3 READ WRITE
12. PDB003 READ WRITE
The PDB003 pluggable database contains few rows into MARCOV.T1 table:
view plainprint?
1. SQL@CDBTEST> alter session set container=PDB003;
2.
3. Session altered.
4.
5. SQL@CDBTEST> select count(*) from marcov.T1;
6.
7. COUNT(*)
8. ----------
9. 100
I'm going to create a database link into the CDB001 container database connecting
to CDBTEST using the ezconnect method.
view plainprint?
1. SQL@CDB001> create database link CDBTEST_AT_VSI08DEVPOM connect to SYSTEM identified by oracle using 'vsi08devpom.myd.it:1521/CDBTEST';
2.
3. Database link created.
I'm able to successfully test the new database link.
view plainprint?
1. SQL@CDB001> select * from cdb_pdbs;
2.
3. PDB_ID PDB_NAME DBID CON_UID GUID STATUS CREATION_SCN CON_ID
4. ---------- ---------- ---------- ---------- -------------------------------- ------------- -----------
- ----------
5. 3 PDB0101 532253118 532253118 E18E282148FD10A5E0430100007FC94D NORMAL 1734497 1
6. 2 PDB$SEED 4063453634 4063453634 E18E1CE36B940C96E0430100007FE168 NORMAL 1720746 1
7.
8. SQL@CDB001> select * from cdb_pdbs@CDBTEST_AT_VSI08DEVPOM;
9.
10. PDB_ID PDB_NAME DBID CON_UID GUID STATUS CREATION_SCN CON_ID
11. ---------- ---------- ---------- ---------- -------------------------------- ------------- -----------
- ----------
12. 2 PDB$SEED 4063610283 4063610283 E1B2A529DB382EACE0430100007F78B8 NORMAL 217 1
13. 3 PDBTEST1 3064465721 3064465721 E1B436871D9E4110E0430100007F9BBC NORMAL 1547881 1
14. 4 PDBTEST2 2395404598 2395404598 E1B43A36FA0B41A9E0430100007F6671 NORMAL 1548944 1
15. 5 PDBTEST3 2434165039 2434165039 E1B43D98C0DC41F6E0430100007F7CE7 NORMAL 1550036 1

2. 16. 6 PDB003 1448206714 1448206714 E2B9BE56B8B936CEE045000000000001 NORMAL 2744910 1
Under the directory of the CDB001 container there are the following files and
directories:
view plainprint?
1. [oracle@vsi08devpom CDB001]$ pwd
2. /opt/app/oracle/oradata/CDB001
3. [oracle@vsi08devpom CDB001]$ ll
4. total 2286044
5. -rw-r----- 1 oracle oinstall 17973248 Aug 5 14:18 control01.ctl
6. drwxr-x--- 2 oracle oinstall 4096 Jul 15 15:52 PDB0101
7. drwxr-x--- 2 oracle oinstall 4096 Jul 15 15:48 pdbseed
8. -rw-r----- 1 oracle oinstall 52429312 Aug 5 14:00 redo01.log
9. -rw-r----- 1 oracle oinstall 52429312 Aug 5 14:18 redo02.log
10. -rw-r----- 1 oracle oinstall 52429312 Aug 5 02:20 redo03.log
11. -rw-r----- 1 oracle oinstall 1226842112 Aug 5 14:15 sysaux01.dbf
12. -rw-r----- 1 oracle oinstall 828383232 Aug 5 14:15 system01.dbf
13. -rw-r----- 1 oracle oinstall 62922752 Aug 5 13:52 temp01.dbf
14. -rw-r----- 1 oracle oinstall 94380032 Aug 5 14:15 undotbs01.dbf
15. -rw-r----- 1 oracle oinstall 5251072 Aug 5 14:06 users01.dbf
As usual to clone the remote pluggable database it must be in READ ONLY mode.
The current open mode of PDB003 pluggable database is READ WRITE.
view plainprint?
1. SQL@CDBTEST> select pdb_name, status from CDB_PDBS;
2.
3. PDB_NAME STATUS
4. ---------- -------------
5. PDB$SEED NORMAL
6. PDBTEST1 NORMAL
7. PDBTEST2 NORMAL
8. PDBTEST3 NORMAL
9. PDB003 NORMAL
10.
11. SQL@CDBTEST> select name, open_mode from V$PDBS;
12.
13. NAME OPEN_MODE
14. ------------------------------ ----------
15. PDB$SEED READ ONLY
16. PDBTEST1 READ WRITE
17. PDBTEST2 READ WRITE
18. PDBTEST3 READ WRITE
19. PDB003 READ WRITE
I need to close and open it in READ ONLY mode.
view plainprint?
1. SQL@CDBTEST> alter pluggable database PDB003 close immediate;
2.
3. Pluggable database altered.
4.
5. SQL@CDBTEST> alter pluggable database PDB003 open read only;
6.
7. Pluggable database altered.
From the target container database I can now issue the following command to clone
the pluggable database PDB003 contained in the remote database container
CDBTEST.
view plainprint?
1. SQL@CDB001> create pluggable database PDB103 from PDB003@CDBTEST_AT_VSI08DEVPOM file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103');
2. create pluggable database PDB103 from PDB003@CDBTEST_AT_VSI08DEVPOM file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103');
3. *
4. ERROR at line 1:
5. ORA-17628: Oracle error 19505 returned by remote Oracle server
6. ORA-19505: failed to identify file ""
On the alert log the following error is logged:
view plainprint?
1. ORA-
17628 signalled during: create pluggable database PDB103 from PDB003@CDBTEST_AT_VSI08DEVPOM file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103')...
Because the database link can connect to either the root of the remote container
database or directly to the remote pluggable database, I decided, after several failed
attempts, to create another database link referencing the remote pluggable
database:
view plainprint?
1. SQL@CDB001> create database link PDB003_AT_CDBTEST connect to SYSTEM identified by oracle using 'vsi08devpom.recupitalia.it:1521/pdb003';
2.
3. Database link created.
4.
5. SQL@CDB001> select * from cdb_pdbs@PDB003_AT_CDBTEST;
6.
7. no rows selected
But even with the different database link the statement continues to fail:
view plainprint?
1. SQL@CDB001> create pluggable database PDB103 from PDB003@PDB003_AT_CDBTEST file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103');
2. create pluggable database PDB103 from PDB003@PDB003_AT_CDBTEST file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103')
3. *
4. ERROR at line 1:
5. ORA-17628: Oracle error 19505 returned by remote Oracle server
6. ORA-19505: failed to identify file ""
Let's try using a different method to connect the two database containers and create
another database link using the following CDBTEST tns entry:
view plainprint?
1. CDBTEST =
2. (DESCRIPTION =
3. (ADDRESS = (PROTOCOL = TCP)(HOST = vsi08devpom.mydomain.it)(PORT = 1521))
4. (CONNECT_DATA =
5. (SERVER = DEDICATED)
6. (SERVICE_NAME = CDBTEST)
7. )
8. )
Creation and testing of the database link using the CDBTEST tns entry.
view plainprint?
1. SQL@CDB001> create database link CDBTEST_TNS_AT_VSI08DEVPOM connect to SYSTEM identified by oracle using 'CDBTEST';
2.
3. Database link created.
4.
5. SQL@CDB001> select * from cdb_pdbs@CDBTEST_TNS_AT_VSI08DEVPOM;
6.
7. PDB_ID PDB_NAME DBID CON_UID GUID STATUS CREATION_SCN CON_ID
8. ---------- ---------- ---------- ---------- -------------------------------- ------------- -----------
- ----------
9. 2 PDB$SEED 4063610283 4063610283 E1B2A529DB382EACE0430100007F78B8 NORMAL 217 1
10. 3 PDBTEST1 3064465721 3064465721 E1B436871D9E4110E0430100007F9BBC NORMAL 1547881 1
11. 4 PDBTEST2 2395404598 2395404598 E1B43A36FA0B41A9E0430100007F6671 NORMAL 1548944 1
12. 5 PDBTEST3 2434165039 2434165039 E1B43D98C0DC41F6E0430100007F7CE7 NORMAL 1550036 1
13. 6 PDB003 1448206714 1448206714 E2B9BE56B8B936CEE045000000000001 NORMAL 2744910 1

3. The statement using the CDBTEST_TNS_AT_VSI08DEVPOM database link still
continues to fail:
view plainprint?
1. SQL@CDB001> create pluggable database PDB103 from PDB003@CDBTEST_TNS_AT_VSI08DEVPOM file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103');
2. create pluggable database PDB103 from PDB003@CDBTEST_TNS_AT_VSI08DEVPOM file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103')
3. *
4. ERROR at line 1:
5. ORA-17628: Oracle error 19505 returned by remote Oracle server
6. ORA-19505: failed to identify file ""
Let's create the latest database link connected to the following tns entry:
view plainprint?
1. PDB003_CDBTEST =
2. (DESCRIPTION =
3. (ADDRESS = (PROTOCOL = TCP)(HOST = vsi08devpom.mydomain.it)(PORT = 1521))
4. (CONNECT_DATA =
5. (SERVER = DEDICATED)
6. (SERVICE_NAME = pdb003)
7. )
8. )
Creation and testing of the database link using the PDB003_CDBTEST tns entry.
view plainprint?
1. SQL@CDB001> create database link PDB003_TNS_AT_VSI08DEVPOM connect to SYSTEM identified by oracle using 'PDB003_CDBTEST';
2.
3. Database link created.
4.
5. SQL@CDB001> select * from cdb_pdbs@PDB003_TNS_AT_VSI08DEVPOM;
6.
7. no rows selected
The statement continues to fail even using the latest available database link:
PDB003_TNS_AT_VSI08DEVPOM
view plainprint?
1. SQL@CDB001> create pluggable database PDB103 from PDB003@PDB003_TNS_AT_VSI08DEVPOM file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103');
2. create pluggable database PDB103 from PDB003@PDB003_TNS_AT_VSI08DEVPOM file_name_convert=('/opt/app/oracle/oradata/CDBTEST/PDB003','/opt/app/oracle/oradata/CDB001/PDB103')
3. *
4. ERROR at line 1:
5. ORA-17628: Oracle error 19505 returned by remote Oracle server
6. ORA-19505: failed to identify file ""
The available database links created and used are the following:
view plainprint?
1. SQL@CDB001> select DB_LINK, HOST, OWNER from dba_db_links;
2.
3. DB_LINK HOST OWNER
4. ------------------------------ --------------------------------------------- -----
5. CDBTEST_AT_VSI08DEVPOM vsi08devpom.mydomain.it:1521/CDBTEST SYS
6. PDB003_AT_CDBTEST vsi08devpom.mydomain.it:1521/pdb003 SYS
7. CDBTEST_TNS_AT_VSI08DEVPOM CDBTEST SYS
8. PDB003_TNS_AT_VSI08DEVPOM PDB003_CDBTEST SYS
And there is still no way to create a pluggable database to a remote target
destination from a local source. You can see on the "SQL Language Reference"
which CREATE PLUGGABLE DATABASE syntax is allowed:
view plainprint?
1. SQL@CDBTEST> create pluggable database PDB103@CDB001_AT_VSI08DEVPOM from PDB003;
2. create pluggable database PDB103@CDB001_AT_VSI08DEVPOM from PDB003;
3. *
4. ERROR at line 1:
5. ORA-00922: missing or invalid option
Even tracing the unix Oracle server process with the strace system util I was not
able to get useful information to know how successfully complete the statement and
why it's failing.
The end of the story: after I have contacted My Oracle Support the Oracle engineer
confirmed the issue, tracked as bug 15931910 and it is still being worked upon by
the development team.
Other platforms could be affected by this bug but I have no reason and time to test
it.
So at the moment there is no way to clone a remote pluggable database using a
database link, but as a workaround you can follow this post to manually create it to
the new target destination