关于Qt多线程操作数据库

  前几天用Qt写了一个基于线程池的Tcp服务器，发现掉线很频繁，追踪日志发现大多是因为数据库连接名称的冲突导致的，这里记录一下。
  运行环境： ubuntu16 Qt5.9.6

当前线程创建数据库对象和查询对象只能在当前线程中使用，不能跨线程使用

  这里说的是一个线程创建的 QSqlDatabase 对象和 查出来的 QSqlQuery 对象只能在当前线程中使用。一个数据库连接本身比如一个连接的名称是可以在不同线程中使用的。默认连接名称是 “qt_sql_default_connection”

char *QSqlDatabase::defaultConnection = const_cast("qt_sql_default_connection");

多线程连接数据库

1、共用一个连接

  线程1，创建连接，然后不断地去查询：

void Thread1::run()
{
    for (;;)
    {
        QString connectionName = "connection";
        {
            QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
            db.setHostName("127.0.0.1");
            db.setDatabaseName("testdb");
            db.setUserName("root");
            db.setPassword("1234");
            if (!db.open())
            {
                qDebug() << "thread 1 open db failed!";
            }

            while (1)
            {
                QSqlQuery query("SELECT * FROM testtb", db);
                qDebug() << "thread1 error: " << query.lastError();
                while (query.next())
                {
                    qDebug() << "thread1 value:" << query.value(0).toInt();
                }
                QThread::msleep(100);
            }
        }
    }
}

  线程2，获取线程一创建的连接名称，然后不断地查询：

void Thread2::run()
{
    for (;;)
    {
        QString connectionName = "connection";
        {
            QSqlDatabase db = QSqlDatabase::database(connectionName);
            while (1)
            {
                QSqlQuery query("SELECT * FROM testtb", db);
                qDebug() << "thread2 error: " << query.lastError();
                while (query.next())
                {
                    qDebug() << "thread2 value:" << query.value(0).toInt();
                }
                QThread::msleep(100);
            }
        }
        msleep(100);
    }
}

  main函数：

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    Thread1 *th1 = new Thread1;
    th1->start();
    QThread::msleep(100);
    Thread2 *th2 = new Thread2;
    th2->start();
    
	return a.exec();
}

QSqlDatabase::database函数是线程安全的。但是还有一个问题，如果运行上面的代码会出警告：

thread2 error:  QSqlError("2013", "QMYSQL: Unable to execute query", "Lost connection to MySQL server during query")
thread2 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread1 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread2 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread1 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread1 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread2 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread1 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread2 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread2 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread1 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread1 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread2 error:  QSqlError("2006", "QMYSQL: Unable to execute query", "MySQL server has gone away")
thread1 error:  QSqlError("0", "QMYSQL: Unable to execute query", "")

多线程访问一个连接，会存在资源竞争，加锁保护：

...
while (1)
{
    gMutex.lock();
    QSqlQuery query("SELECT * FROM testtb", db);
    qDebug() << "thread1 error: " << query.lastError();
    while (query.next())
    {
        qDebug() << "thread1 value:" << query.value(0).toInt();
    }
    gMutex.unlock();
    QThread::msleep(100);
}
...
while (1)
{
    gMutex.lock();
    QSqlQuery query("SELECT * FROM testtb", db);
    qDebug() << "thread2 error: " << query.lastError();
    while (query.next())
    {
        qDebug() << "thread2 value:" << query.value(0).toInt();
    }
    gMutex.unlock();
    QThread::msleep(100);
}
...

问题解决。

2、多线程动态连接数据库

  动态连接数据库我们采用线程池的方式。为了保证每个线程的连接名称不同，将线程id相关信息设置成连接名称。
  tashk.h:

#ifndef TASK_H
#define TASK_H

#include 
#include 

class Task : public QObject, public QRunnable
{
    Q_OBJECT
public:
    explicit Task(QObject *parent = nullptr);

    void setType(const int &type);

protected:
    void run();

private:
    void thread1Workder();
    void thread2Workder();
    void thread3Workder();
    void thread4Workder();

private:
    int m_type;
};

#endif // TASK_H

  task.cpp:

#include "task.h"
#include 
#include 
#include 
#include 

Task::Task(QObject *parent)
    : QObject(parent),
      m_type(0)
{
}

void Task::setType(const int &type)
{
    m_type = type;
}

void Task::run()
{
    switch (m_type) {
    case 0:
        thread1Workder();
        break;
    case 1:
        thread2Workder();
        break;
    case 2:
        thread3Workder();
        break;
    case 3:
        thread4Workder();
        break;
    default:
        break;
    }
}

void Task::thread1Workder()
{
    QString connectionName = QString::number(*static_cast(QThread::currentThreadId())); 
    QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
    db.setHostName("127.0.0.1");
    db.setDatabaseName("testdb");
    db.setUserName("root");
    db.setPassword("1234");
    if (!db.open())
    {
        qDebug() << "thread 1 open db failed!";
    }
    qDebug() << "thread 1: " << QThread::currentThreadId() << connectionName;  
}

void Task::thread2Workder()
{
    QString connectionName = QString::number(*static_cast(QThread::currentThreadId()));
    QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
    db.setHostName("127.0.0.1");
    db.setDatabaseName("testdb");
    db.setUserName("root");
    db.setPassword("1234");
    if (!db.open())
    {
        qDebug() << "thread 1 open db failed!";
    }
    qDebug() << "thread 2: " << QThread::currentThreadId() << connectionName;
}

void Task::thread3Workder()
{
    QString connectionName = QString::number(*static_cast(QThread::currentThreadId()));
    QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
    db.setHostName("127.0.0.1");
    db.setDatabaseName("testdb");
    db.setUserName("root");
    db.setPassword("1234");
    if (!db.open())
    {
        qDebug() << "thread 1 open db failed!";
    }
    qDebug() << "thread 3: " << QThread::currentThreadId() << connectionName;
}

void Task::thread4Workder()
{
    QString connectionName = QString::number(*static_cast(QThread::currentThreadId()));
    QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
    db.setHostName("127.0.0.1");
    db.setDatabaseName("testdb");
    db.setUserName("root");
    db.setPassword("1234");
    if (!db.open())
    {
        qDebug() << "thread 1 open db failed!";
    }
    qDebug() << "thread 4: " << QThread::currentThreadId() << connectionName;
}

main.cpp:

#include 
#include 
#include 
#include "task.h"

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    int index = 0;
    for (;;)
    {
        Task *pTask = new Task;
        pTask->setAutoDelete(true);
        pTask->setType(index);
        QThreadPool::globalInstance()->start(pTask);

        index ++;
        if (index > 3)
            index =0;
    }
    return a.exec();
}

  运行结果：

QSqlDatabasePrivate::addDatabase: duplicate connection name '469759744', old connection removed.
QSqlDatabasePrivate::addDatabase: duplicate connection name '701179648', old connection removed.

  发现大量重复连接名称的警告。那我们在每一个线程连接完成之后调用QSqlDatabase::removeDatabase()方法，该方法是线程安全的。如下：

void Task::thread1Workder()
{
    QString connectionName = QString::number(*static_cast(QThread::currentThreadId())); 
    QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
    db.setHostName("127.0.0.1");
    db.setDatabaseName("testdb");
    db.setUserName("root");
    db.setPassword("1234");
    if (!db.open())
    {
        qDebug() << "thread 1 open db failed!";
    }
    qDebug() << "thread 1: " << QThread::currentThreadId() << connectionName; 
    QSqlDatabase::removeDatabase(connectionName); 
}
void Task::thread2Workder() { ... }
void Task::thread3Workder() { ... }
void Task::thread4Workder() { ... }

  运行时发现还是有警告，大概意思是连接正在使用，将停止工作。

hread 2:  0x7ff658a5f700 "1487271680"
QSqlDatabasePrivate::removeDatabase: connection '1487271680' is still in use, all queries will cease to work.
thread 1 open db failed!
thread 3:  0x7ff659a61700 "1504057088"

调用close()方法也没什么用，查了一下官方文档：

  大概意思是不应该在有在该连接上进行数据查询时调用该函数，否则会内存泄露。虽然这里并没有查询任务，但我们按它的方式改下试试：

void Task::thread1Workder()
{
    QString connectionName = QString::number(*static_cast(QThread::currentThreadId()));
    {
        QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
        db.setHostName("127.0.0.1");
        db.setDatabaseName("testdb");
        db.setUserName("root");
        db.setPassword("1234");
        if (!db.open())
        {
            qDebug() << "thread 1 open db failed!";
        }
        qDebug() << "thread 1: " << QThread::currentThreadId() << connectionName;
    }
    QSqlDatabase::removeDatabase(connectionName);
}
void Task::thread2Workder() { ... }
void Task::thread3Workder() { ... }
void Task::thread4Workder() { ... }

  运行发现可以了。不再出现警告了。但是如果运行一段时间会发现程序偶尔会挂掉。。。
原因(https://blog.csdn.net/goldenhawking/article/details/10811409)：
  Qt 会动态的加载数据库的plugin, 加载 plugin 的部分，涉及到对本地库文件的管理，这一部分，出现了竞争。于是，很自然的想到在初始连接部分设置 Mutex 保护，从 addDatabase / database到 open 的部分，要保证其原子性，问题再也没有出现。
再稍微改下，经过大量测试发现正常了：

void Task::thread1Workder()
{
    QString connectionName = QString::number(*static_cast(QThread::currentThreadId()));
    {
        mutex.lock();
        QSqlDatabase db = QSqlDatabase::addDatabase("QMYSQL", connectionName);
        db.setHostName("127.0.0.1");
        db.setDatabaseName("testdb");
        db.setUserName("root");
        db.setPassword("1234");
        if (!db.open())
        {
            qDebug() << "thread 1 open db failed!";
        }
        mutex.unlock();
        qDebug() << "thread 1: " << QThread::currentThreadId() << connectionName;
    }
    QSqlDatabase::removeDatabase(connectionName);
}
void Task::thread2Workder() { ... }
void Task::thread3Workder() { ... }
void Task::thread4Workder() { ... }

总结

  1、一个线程创建的数据库对象只能在当前线程使用，但是创建的连接可以多线程使用，唯一需要注意的是，在调用全局方法的时候，要有原子保护。
  2、removedatabase方法是线程安全的，但是在调用的时候确保QSqlDatabase和QSqlQuery等对象已经释放，否则会造成内存泄漏。
  3、多线程访问时不管是同一连接名还是不同连接名称都要进行资源保护。