sql如何避免插入并发_SQL批量插入并发和性能注意事项

sql如何避免插入并发

One of the challenges we face when using SQL bulk insert from files flat can be concurrency and performance challenges, especially if the load involves a multi-step data flow, where we can’t execute a latter step until we finish with an early step. We also see these optimization challenges with constraints as well, as fewer steps to complete a data flow results in saved time, but possibly less accurate data.

在从平面文件使用SQL批量插入时，我们面临的挑战之一可能是并发和性能方面的挑战，特别是如果加载涉及多步骤数据流，那么在完成第一步之前，我们无法执行后面的步骤。 我们还看到这些优化挑战也受到限制，因为完成数据流所需的步骤更少，可以节省时间，但数据准确性可能会降低。

We’ll look at solving concurrency issues, where we prevent a data flow process from accessing data too early in a central repository situation. In addition, we’ve also looked at some techniques to specify options which ensure that our business rules and constraints are followed while data are being inserted. We’ll also look at optimizing our data structures to strengthen performance for when we load data from files using this native tool in T-SQL.

我们将着眼于解决并发问题，在这种情况下，我们将防止数据流过程在中央存储库情况下过早访问数据。 此外，我们还研究了一些指定选项的技术，这些选项可确保在插入数据时遵循我们的业务规则和约束。 当使用T-SQL中的本机工具从文件加载数据时，我们还将研究优化数据结构以增强性能。

SQL大容量插入事务 (SQL Bulk Insert in Transactions)

Identical to other CRUD operations, bulk loads, including SQL Bulk Insert, can be wrapped in a transaction and rolled back entirely if we don’t get our expected results, or if we experience a failure. In the below file and code snippet, we see that the last line of our file doesn’t match the expected bit value of either 0 or 1 and we know based on other tips in this series that this will fail this line. If our software in this example depended on 11 records after the insert and this test failed because there were only 10 records, then we would roll back this insert.

与其他CRUD操作相同，可以将大容量装载（包括SQL大容量插入）包装在事务中，如果我们无法获得预期的结果或遇到失败，则可以完全回滚。 在下面的文件和代码片段中，我们看到文件的最后一行与预期的0或1位值不匹配，并且基于本系列的其他技巧，我们知道该行将失败。 如果此示例中的软件在插入后依赖于11条记录，并且由于只有10条记录而导致测试失败，那么我们将回滚此插入。

Our file’s last line has an incorrect bit value of 3.

我们文件的最后一行的错误位值为3。

IF EXISTS (SELECT OBJECT_ID('etlImport5')) BEGIN DROP TABLE etlImport5 END
CREATE TABLE etlImport5(
	VarcharSpefzTen VARCHAR(10),
	IntSpefz INT,
	BitSpefz BIT,
	VarcharSpefzMax VARCHAR(MAX)
)
 
BEGIN TRAN
 
BULK INSERT etlImport5
FROM 'C:\ETL\Files\Read\Bulk\daily_20180101.txt'
WITH (
	FIELDTERMINATOR = ',',
	ROWTERMINATOR = '\n',
	FIRSTROW = 2,
	MAXERRORS = 10
)
 
---- Query before rollback
SELECT COUNT(*) CountRecs FROM etlImport5
 
ROLLBACK TRAN
 
---- Query after rollback
SELECT * FROM etlImport5

Our query result prior to our rollback.

我们的查询结果在回滚之前。