C# Dapper批量插入大量数据优化

以三万条数据为例:

Dapperr的Excute方法执行插入语句的本质是一条一条的插入,当数据量非常大是会很慢,三万条用了十几分钟

将一组数据一块插入就会相当的快,三万条用了12秒

优化前的批量导入:

//_vfsContext.Conn.Open();
//var tran = _vfsContext.Conn.BeginTransaction(IsolationLevel.Serializable);
//Stopwatch stopwatch = new Stopwatch();
//stopwatch.Start();
//var inserSql = $@"insert into tm_receiveconfirm(ConfirmStamp,OrderNo,InsertTime) 
//                    values(@ConfirmStamp,@OrderNo,@InsertTime)";
//_vfsContext.Conn.Execute(inserSql, tmList, tran);
//tran.Commit();
//stopwatch.Stop();

优化后的批量导入:

                //组织数据
                for (var i = 0; i < OrderNos.Count; i++)
                {
                    tmList.Add(new tm_receiveconfirm()
                    {
                        OrderNo = OrderNos[i],
                        ConfirmStamp = uniqueId,
                        InsertTime = DateTime.Now
                    });
                }

                //临时表插入数据
                //拼接sql
                var strsql = new StringBuilder();
                strsql.Append("insert into tm_receiveconfirm(ConfirmStamp,OrderNo,InsertTime) values ");
                foreach (var item in tmList)
                {
                    strsql.AppendFormat("('{0}','{1}','now()'),", uniqueId, item.OrderNo);
                }
                var inserSql = strsql.ToString();
                var sql=inserSql.Substring(0, inserSql.LastIndexOf(','));
                Stopwatch stopwatch = new Stopwatch();
                stopwatch.Start();
                log.Info($"{sql}");
                _vfsContext.Conn.Execute(sql, null);
                stopwatch.Stop();

                log.Info($"数据插入临时表end:{stopwatch.ElapsedMilliseconds}");

取经地址:https://www.cnblogs.com/wwg1990/p/10362667.html

你可能感兴趣的:(C#)