PHP使用Sqlite3批量插入调优

工作中用到SQLite，简单好用，但是存在多个进程写同一个db文件时的锁问题，报database is locked错误导致插入失败。

业务中存在批量插入的场景，这里按之前MySQL的优化思路优化了SQLite的插入，下面是一些测试用例。

下面的测试用列是在我虚拟机上执行的，绝对数据没什么意义，可以对比看各个方案的执行效率。

db = new \SQLite3($file_data);
        if (!$this->db) {
            throw new \Exception("SQLite3 construct failed,err:" . $this->db->lastErrorMsg());
        }

        $this->init();
    }

    private function init()
    {
        $table_name = self::TABLE_NAME;

        $sql =<<db->exec($sql);
        if(!$ret) {
            echo("insert failed,err:". $this->db->lastErrorMsg().PHP_EOL);
        }
    }

    public function test($callable)
    {
        $this->db->exec("delete from ".self::TABLE_NAME);
        $tickStart = microtime(true);
        call_user_func_array($callable,[]);
        echo "cost ".(microtime(true)-$tickStart)." ms".PHP_EOL;
    }

    public function forInsert()
    {
        for($i=0;$idb->exec("INSERT INTO ".self::TABLE_NAME."(id,type,action,data,create_time) VALUES(NULL,".$i.",".$i.",".$i.",datetime('now','localtime'))");
            if(!$ret) {
                echo("insert failed,err:" . $this->db->lastErrorMsg().PHP_EOL);
            }
        }
    }

    public function forTrans()
    {
        $this->db->exec("begin;");
        for($i=0;$idb->exec("INSERT INTO ".self::TABLE_NAME."(id,type,action,data,create_time) VALUES(NULL,".$i.",".$i.",".$i.",datetime('now','localtime'))");
            if(!$ret) {
                echo("insert failed,err:" . $this->db->lastErrorMsg().PHP_EOL);
            }
        }
        $this->db->exec("commit;");
    }


    public function forSync()
    {
        $this->db->exec("PRAGMA synchronous = OFF;");
        for($i=0;$idb->exec("INSERT INTO ".self::TABLE_NAME."(id,type,action,data,create_time) VALUES(NULL,".$i.",".$i.",".$i.",datetime('now','localtime'))");
            if(!$ret) {
                echo("insert failed,err:" . $this->db->lastErrorMsg().PHP_EOL);
            }
        }
        $this->db->exec("PRAGMA synchronous = ON;");
    }

    public function forBind()
    {
        $bind_sql = "INSERT INTO ".self::TABLE_NAME."(id,type,action,data,create_time) VALUES(?,?,?,?,?)";
        $rs = $this->db->prepare($bind_sql);
        for($i=0;$ireset();
            $rs->bindValue(1,$i,SQLITE3_INTEGER);
            $rs->bindValue(2,$i,SQLITE3_INTEGER);
            $rs->bindValue(3,$i,SQLITE3_INTEGER);
            $rs->bindValue(4,$i,SQLITE3_TEXT);
            $rs->bindValue(5,$i);
        }
        $rs->execute();
    }
}

$s = new TestSqlite("/tmp/test.db");
$s->test(array($s,"forInsert"));
$s->test(array($s,"forTrans"));
$s->test(array($s,"forSync"));
$s->test(array($s,"forBind"));

cost 1.5027620792389 ms
cost 0.014527082443237 ms
cost 0.036828994750977 ms
cost 0.0043308734893799 ms

从这里可以看出第四种方案是性能最好的。