具体任务:现有一个300+ G的dmp文件需要将其导入到Oracle数据库中,并且将其切分为若干的小的dmp文件
通过 docker 去 pull 阿里的镜像
具体流程参考
http://t.csdn.cn/Zbc6P
docker run -d --privileged=true -p 1521:1521 -e ORACLE_ALLOW_REMOTE=true -v [宿主机目录]:[镜像内的目录] --name oracle registry.cn-hangzhou.aliyuncs.com/helowin/oracle_11g
1.登陆并创建用户
sqlplus /nolog --登录
conn /as sysdba
create user oracle_user identified by 【password】; – 创建内部管理员账号密码;
2.授权
grant connect,resource,dba to oracle_user; --将dba权限授权给内部管理员账号和密码;
3.设置
ALTER PROFILE DEFAULT LIMIT PASSWORD_LIFE_TIME UNLIMITED; --设置密码永不过期:
alter system set processes=1000 scope=spfile; --修改数据库最大连接数据;
4.导入数据
注意,这里可以指定表空间导入,可以去找相关帖子
imp [username]/[password]@[ip]:1521/[数据库实例名 helowin] file=‘[导入的文件位置]’ full=y ignore=y;
命令
exp [username]/[password]@[ip]:1521/[数据库实例名] file=[导出的位置] tables=[, …] log=[日志导出位置] grants=y;
解决方案:
连接数据库,获取所有表名,大致计算每个表的大小,将所有表名进行分割后得到导出的exp语句
package org.example;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
/**
* @version 1.0
* @Author kurumi
* @Date 2022/7/26 19:24
* @注释
*/
public class Conn {
static String driverClass="oracle.jdbc.driver.OracleDriver"; //oracle的驱动
static String url="jdbc:oracle:thin:@[ip]:1521:[实例名 这里是helowin]"; //连接oracle路径方式 “”gfs“”是要建立连接的数据库名 1521端口
static String user="[user]"; //user是数据库的用户名
static String password="[password]"; //用户登录密码
public static Connection getconn() { //为了方便下面的讲解,这里专门建立了一个用于数据库连接的一个方法
Connection conn=null;
try {
//首先建立驱动
Class.forName("oracle.jdbc.driver.OracleDriver");
//驱动成功后进行连接
conn=DriverManager.getConnection(url, user, password);
// System.out.println("连接成功");
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return conn; //返回一个连接
}
}
2.连接数据库并处理得到导出语句
oracle11g对应的jdbc版本
cn.easyproject
ojdbc6
11.2.0.4
package org.example;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.ArrayList;
/**
* @version 1.0
* @Author kurumi
* @Date 2022/7/26 19:23
* @注释
*/
public class GetExpInfo {
public static void main(String[] args) {
Connection conn=null;
Statement st=null;
ResultSet rs=null;
ArrayList<String> tableNames = new ArrayList<>();
try {
//1、获取连接对象
conn=Conn.getconn();
//2、创建statement类对象,用来执行SQL语句!!
st=conn.createStatement();
//3、创建sql查询语句
/*select table_name from all_tables where TABLESPACE_NAME ='USERS'
select table_name from all_tables a where a.OWNER = upper('oracle_user') ORDER BY(TABLE_NAME);
*/
String sql="select table_name from all_tables a where a.OWNER = upper('oracle_user') ORDER BY(TABLE_NAME)";
//4、执行sql语句并且换回一个查询的结果集
rs=st.executeQuery(sql);
while(rs.next()) { //循环遍历结果集
String name=rs.getString("table_name");
tableNames.add(name);
}
} catch (Exception e) {
e.printStackTrace();
}
StringBuffer stringBuffer = new StringBuffer();
ArrayList<String> resSQL = new ArrayList<>();
int i = 0;
//每次导出一百个表
String start = tableNames.get(0);
for (String tableName : tableNames) {
if((i+1) % 100 == 0){
String end = tableNames.get(i);
String name = start+"-"+end+".dmp";
stringBuffer.deleteCharAt(stringBuffer.lastIndexOf(","));
resSQL.add("exp [user_name]/[password]@[ip]:1521/helowin " +
"file=/home/oracle/app/oracle/oradata-source/"+name+" tables="+stringBuffer.toString());
stringBuffer.setLength(0);
start = end;
}
stringBuffer.append(tableName+",");
i++;
}
//剩下的表
String end = tableNames.get(i-1);
String name = start+"-"+end+".dmp";
stringBuffer.deleteCharAt(stringBuffer.lastIndexOf(","));
resSQL.add("exp [user_name]/[password]@[ip]:1521/helowin " +
"file=/home/oracle/app/oracle/oradata-source/"+name+" tables="+stringBuffer.toString());
for (String s : resSQL) {
System.out.println(s);
}
}
}
3.服务器写一个脚本自动导入
vi block.sh
#! /bin/bash
exp [user_name]/[password]@[ip]:1521/helowin file=/home/oracle/app/oracle/oradata-source/[your file name1].dmp tables= [table1],[table2],[table3]......
&&
exp [user_name]/[password]@[ip]:1521/helowin file=/home/oracle/app/oracle/oradata-source/[your file name2].dmp tables= [table1],[table2],[table3]......
&&
.......
执行block.sh脚本
EXP-00091: Exporting questionable statistics.
EXP-00091: Exporting questionable statistics.
EXP-00091: Exporting questionable statistics.
1.问题产生的原因
linux下oracle用户环境变量语言集和oracle数据库中的环境变量语言集不同
2.解决方法
只要将oracle环境变量语言集改成和数据库环境变量语言集一样的就可以
参考了
http://t.csdn.cn/2D7jC
1). 查看服务器端字符编码(注意是Oracle安装的服务器,对于docker是容器内的字符编码)
得到的字符集为 ‘AMERICAN_AMERICA.AL32UTF8’
SQL> select userenv(‘language’) from dual;
USERENV(‘LANGUAGE’)
AMERICAN_AMERICA.AL32UTF8
2). 返回命令行设置
[oracle@a5ee52623107 ~]$ export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
3). 导出数据
[oracle@a5ee52623107 ~]$
exp [username]/[password]@[ip]:1521/[数据库实例名] file=[导出的位置] tables=[, …] log=[日志导出位置] grants=y;
错误信息
IMP-00058: ORACLE error 1658 encountered
ORA-01658: unable to create INITIAL extent for segment in tablespace USERS
. . importing table “xxxxxxxxx”
IMP-00058: ORACLE error 1658 encountered
ORA-01658: unable to create INITIAL extent for segment in tablespace USERS
. . importing table xxxxx"
or
. . importing table “xxxxxx” 出错
IMP-00058: ORACLE error 1659 encountered
ORA-01659: unable to allocate MINEXTENTS beyond 4 in tablespace USERS
. . importing table “xxxxxx”
错误原因
表空间大小不足造成的,可以在 navicat 中去实时查看当前要导入的表空间的利用率
另一种查看表空间利用率的方法 执行
SELECT UPPER(F.TABLESPACE_NAME) “【表空间名】”,
D.TOT_GROOTTE_MB “表空间大小(M)”,
D.TOT_GROOTTE_MB - F.TOTAL_BYTES “已使用空间(M)”,
TO_CHAR(ROUND((D.TOT_GROOTTE_MB - F.TOTAL_BYTES) / D.TOT_GROOTTE_MB * 100,2),‘990.99’) || ‘%’ “使用比”,
F.TOTAL_BYTES “空闲空间(M)”,
F.MAX_BYTES “最大块(M)”
FROM (SELECT TABLESPACE_NAME,
ROUND(SUM(BYTES) / (1024 * 1024), 2) TOTAL_BYTES,
ROUND(MAX(BYTES) / (1024 * 1024), 2) MAX_BYTES
FROM SYS.DBA_FREE_SPACE
GROUP BY TABLESPACE_NAME) F,
(SELECT DD.TABLESPACE_NAME,
ROUND(SUM(DD.BYTES) / (1024 * 1024), 2) TOT_GROOTTE_MB
FROM SYS.DBA_DATA_FILES DD
GROUP BY DD.TABLESPACE_NAME) D
WHERE D.TABLESPACE_NAME = F.TABLESPACE_NAME
ORDER BY 1;
解决方法
增加数据文件来达到表空间扩容
注意
1.一般情况下单个数据文件最大32G左右的大小,这里设置了30G
2.就我所执行的任务而言,所需要的表空间大致为dmp文件大小的2倍左右
alter tablespace USERS add datafile ‘[path]/users01.dbf’ size 30720m autoextend on next 10240m MAXSIZE UNLIMITED;
alter tablespace USERS add datafile ‘/home/oracle/app/oracle/oradata/helowin/users02.dbf’ size 30720m autoextend on next 10240m MAXSIZE UNLIMITED;
alter tablespace USERS add datafile ‘/home/oracle/app/oracle/oradata/helowin/users03.dbf’ size 30720m autoextend on next 10240m MAXSIZE UNLIMITED;
alter tablespace USERS add datafile ‘/home/oracle/app/oracle/oradata/helowin/users04.dbf’ size 30720m autoextend on next 10240m MAXSIZE UNLIMITED;
另一种解决思路
可以在导入的时候指定要导入的表空间,如果数据文件很大的话,可以设置bigfile类型的表空间
错误代码
EXP-00028: failed to open /home/oracle/app/oracle/oradata-source/test_dmp.dmp for write
错误原因
用户权限不够
解决方法
su - root
chown -R oracle:oinstall /home/oracle/app/oracle/oradata-source
Oracle服务端字符集编码为AL32UTF8,dmp文件编码字符集为ZHS16GBK,导入过程中由于有中文列发生错误
docker创建一个新的容器后重置数据库的字符集,修改原有数据库的字符集可能会发生一定的风险
docker创建同一个镜像的不同的容器,修改name和端口即可
此外。还需要修改端口,采用默认端口1521可不用修改,这里以创建docker oracle容器时指定端口为1524:1524
vi /home/oracle/app/oracle/product/11.2.0/dbhome_2/network/admin/listener.ora
# listener.ora Network Configuration File: /home/oracle/app/oracle/product/11.2.0/dbhome_2/network/admin/listener.ora
# Generated by Oracle configuration tools.
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1524))
(ADDRESS = (PROTOCOL = TCP)(HOST = )(PORT = 1524))
)
)
ADR_BASE_LISTENER = /home/oracle/app/oracle
# 修改
vi /home/oracle/app/oracle/product/11.2.0/dbhome_2/network/admin/tnsnames.ora
# tnsnames.ora Network Configuration File: /home/oracle/app/oracle/product/11.2.0/dbhome_2/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.
LISTENER_HELOWIN =
(ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1524))
HELOWIN =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1524))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = helowin)
)
)
修改字符集参考:
https://www.cnblogs.com/dll102/p/14822265.html
imp导入发生错误,不晓得是不是创建容器时指定了不同的端口所造成的原因
IMP-00058: ORACLE error 12547 encountered
ORA-12547: TNS:lost contact
IMP-00000: Import terminated unsuccessfully
去掉本机ip,命令改为 imp 用户名/密码@helowin full=y file=dmp文件 ignore=y full=y;
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses ZHS16GBK character set (possible charset conversion)
export client uses ZHS16GBK character set (possible charset conversion)
vi ~/.bash_profile
# bash_profile添加
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
source ~/.bash_profile