在用OCCI 向Oracle中插入数据时,效率不高,使用自动提交数据的情况(默认)下一秒钟只能插入1000条数据左右。因为插入数据库这块是影响系统中效率最明显的地方,因此很有必要提高数据插入这块的效率。在网上找了一些资料后发现,可以重用statement对象,使用批量插入的方法,先把数据保存在内存中,积累到一定数值之后批量插入Oracle,这样平均下来一秒钟可以插入5000多条数据,性能有所提高,不错,这里做一个记录。
#include
#include
#include
#include
#define WIN32COMMON //避免函数重定义错误
#include
#include
#include
CC=g++
OBJS=TestOracle.o
LIB=-L/opt/oracle/oracle11g/product/11.2.0/dbhome_1/lib -L/opt/oracle/oracle11g/product/11.2.0/dbhome_1/rdbms/lib/
INCLUDE=-I/opt/oracle/oracle11g/product/11.2.0/dbhome_1/precomp/public -I/opt/oracle/oracle11g/product/11.2.0/dbhome_1/rdbms/public
Test: $(OBJS)
$(CC) -o Test $(OBJS) $(LIB) -locci -lclntsh
TestOracle.o: TestOracle.cpp
$(CC) -c TestOracle.cpp $(INCLUDE)
clean:
rm -rf *.o & rm Test
每次创建statement对象时,需要在客户端和服务端分配资源,如内存和游标(cursor),用于存储对象及数据。为了不必要的内存重分配,应重用statement对象。statement对象创建后,可以使用setSQL方法进行重用,例如:
Connection* conn = env->createConnection();
Statement* stmt = conn->createStatement();
stmt->setSQL(“INSERT INTO fruit_basket_tab VALUES(‘Apples’, 3)”);
stmt->executeUpdate();
stmt->setSQL(“INSERT INTO fruit_basket_tab VALUES(‘Oranges’, 4)”);
stmt->executeUpdate();
stmt->setSQL(“INSERT INTO fruit_basket_tab VALUES(‘Bananas’, 1)”);
stmt->executeUpdate();'
stmt->setSQL(“SELECT * FROM fruit_basket_tab WHERE quantity > 2”);
ResultSet* rs = stmt->executeQuery();
stmt->setSQL(“INSERT INTO fruit_basket_tab VALUES(:1, :2)”);
stmt->setString( 1, “Apples” );
stmt->setInt( 2, 3 );
stmt->executeUpdate();
stmt->setString( 1, “Oranges” );
stmt->setInt( 2, 4 );
stmt->executeUpdate();
stmt->setString( 1, “Bananas” );
stmt->setInt( 2, 1 );
stmt->executeUpdate();
//prepare the batching process
stmt->setMaxIterations( 3 );
stmt->setMaxParamSize( 1, 8 ); //”Bananas” is longest param
//batch the statements
stmt->setSQL(“INSERT INTO fruit_basket_tab VALUES(:1, :2)”);
stmt->setString( 1, “Apples” );
stmt->setInt( 2, 3 );
stmt->addIteration();
stmt->setString( 1, “Oranges” );
stmt->setInt( 2, 4 );
stmt->addIteration();
stmt->setString( 1, “Bananas” );
stmt->setInt( 2, 1 );
//execute the statements
stmt->executeUpdate();
// insert Bananas
char buf[BUF_SIZE] = "Bananas";
int quantity = 1;
ub2 buflen = strlen( buf ) + 1;
ub2 quantlen = sizeof(int);
stmt->setDataBuffer(1, (dvoid*)buf, OCCI_SQLT_STR, buflen, &buflen);
stmt->setDataBuffer(2, (dvoid*)&quantity, OCCIINT, quantlen,
&quantlen);
stmt->executeUpdate(); // executeArrayUpdate(1) also would work.
// insert Apples
strcpy( buf, “Apples” );
quantity = 3;
buflen = strlen( buf ) + 1;
quantlen = sizeof( int );
stmt->setDataBuffer(1, (dvoid*)buf, OCCI_SQLT_STR, buflen, &buflen);
stmt->setDataBuffer(2, (dvoid*)&quantity, OCCIINT, quantlen,
&quantlen);
stmt->executeUpdate(); // executeArrayUpdate(1) also would work.
//commit the transaction
conn->commit();
setDataBuffer方法可以与迭代执行(iterative executes)和executeArrayUpdate方法结合使用。
char fruit[][BUF_SIZE] = { "Apples","Oranges","Bananas","Grapes" };
int int_arr[]={ 3,4,1,5 };
ub2 fruitlen[4]; // array of size of individual elements
ub2 intsize[4];
for(int i=0 ; i<4 ; i++)
{
intsize[i] = sizeof(int);
fruitlen[i] = strlen( fruit[i] ) + 1 ; // include the null
}
stmt->setDataBuffer(1, (dvoid*)fruit, OCCI_SQLT_STR, BUF_SIZE,
fruitlen);
stmt->setDataBuffer(2, (dvoid*)int_arr, OCCIINT, sizeof(int), intsize);
stmt->executeArrayUpdate(4);
conn->commit();
char fruits[][BUF_SIZE] = {“Apples”, “Oranges”, “Bananas”};
ub2 fruitLen[3];
for( int j=0; j<3; j++ )
{
fruitLen[j] = strlen( fruits[j] ) + 1; //include the null
}
stmt->setMaxIterations(3);
//setDataBuffer only needs to be executed once
//while all the other variables need to be set for each iteration
stmt->setDataBuffer( 1, fruits, OCCI_SQLT_STR, sizeof(fruits[0]),
fruitLen );
stmt->setInt(2, 3); //Apple’s quantity
stmt->addIteration();
stmt->setInt(2, 4); //Orange’s quantity
stmt->addIteration();
stmt->setInt(2, 1); //Banana’s quantity
//execute the iterative update
stmt->executeUpdate(3);
对操作的列使用合适的setXXX和getXXX方法,而非统一作为string处理,可以省去不必要的转换。
在NLS_LANG环境设置中使用合适的字符集,以避免获取字符串时不必要的字符集转换。
自动提交模式
由于所有的SQL DML都是在事务中执行,所以需要确认所有的DML。可以根据具体情况使用“Connection::commit”和“Connection::rollback”方法。“Statement::setAutoCommit”方法可以用来确认其后的每条语句。使用该方法可节省网络传输时间。
//code with AutoCommit
//transaction 1
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Apples”,3));
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Oranges”,4));
stmt->setAutoCommit( TRUE );
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Bananas”,1));
stmt->setAutoCommit( FALSE );
//transaction 2
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Apples”,5));
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Oranges”,6));
stmt->setAutoCommit( TRUE );
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Bananas”,2));
stmt->setAutoCommit( FALSE );
//code without AutoCommit
//transaction 1
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Apples”,3));
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Oranges”,4));
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Bananas”,1));
conn->commit();
//transaction 2
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Apples”,5));
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Oranges”,6));
stmt->executeUpdate(“INSERT INTO fruit_basket_tab VALUES(“Bananas”,2));
conn->commit();
ResultSet* rs = stmt->executeQuery( “SELECT * FROM fruit_basket_tab” );
ResultSet::Status stat = rs->status(); //status is DATA_AVAILABLE
while( rs->next() ) { //process data }
setPrefetchRowCount and setPrefetchMemorySize