将PowerShell输出写入SQL Server表的6种方法
PowerShell has become the ultimate choice for many database administrators because of its efficient way of handling and managing automation in a simple, quick way. It’s built on .NET Framework and uses Object Models such as COM, ADSI, ADO, and WMI. PowerShell has replaced the traditional way of scripting that used many legacy scripting practices to monitor SQL instances.
PowerShell已成为许多数据库管理员的最终选择，因为它以简单，快速的方式有效地处理和管理自动化。 它基于.NET Framework构建，并使用对象模型，例如COM，ADSI，ADO和WMI。 PowerShell已取代了传统的脚本编写方式，该方法使用了许多旧式脚本编写实践来监视SQL实例。
I’ve been asked on several occasions about how to store the output of PowerShell WMI data into the SQL table. The question comes up so frequently that I decided to write this article.
我曾多次被问到如何将PowerShell WMI数据的输出存储到SQL表中。 这个问题经常出现，我决定写这篇文章。
When sending data within a system (such as a PowerShell object to a cmdlet), the process is straightforward. However, with non-native data interchange (for instance, WMI to SQL), the process can potentially get complicated. Due to this, many purists suggest sticking to simple interchange formats, such as CSV, JSON or in some cases, XML.
在系统内发送数据（例如将PowerShell对象发送到cmdlet）时，该过程非常简单。 但是，使用非本地数据交换（例如，从WMI到SQL），该过程可能会变得很复杂。 因此，许多纯粹主义者建议坚持使用简单的交换格式，例如CSV，JSON或在某些情况下使用XML。
Let’s get out and see the possible options to transform WMI data to SQL table. In this article, we will:
让我们出发，看看将WMI数据转换为SQL表的可能选项。 在本文中，我们将：
- discuss Invoke-Sqlcmd 讨论Invoke-Sqlcmd
- talk about the .NET class libraries 谈谈.NET类库
- talk about exporting data using various Export* cmdlets 讨论使用各种Export * cmdlet导出数据
- learn how to use Windows Management Instrumentation (WMI) 了解如何使用Windows Management Instrumentation（WMI）
- discuss SQL Constructs to load data from file 讨论从文件加载数据SQL构造
- and more 和更多
This guide details the working example of checking disk space by querying WMI.
本指南详细介绍了通过查询WMI检查磁盘空间的工作示例。
Microsoft brought in WMI in order to simplify the management of the different classes of operating systems. While automation was one of the reasons for bringing WMI, the primary focus was to provide developers with handles that they could use when creating applications for Windows. But of course, WMI has these nifty uses as well.
Microsoft引入了WMI，以简化对不同类别的操作系统的管理。 自动化是带来WMI的原因之一，但主要重点是为开发人员提供在创建Windows应用程序时可以使用的句柄。 但是当然，WMI也具有这些漂亮的用途。
The simplest way to get the disk space information is with the Win32_LogicalDisk class. We filter the content to pick only DriveType=3 which is the type number for local drives.
获取磁盘空间信息的最简单方法是使用Win32_LogicalDisk类。 我们过滤内容以仅选择DriveType = 3，这是本地驱动器的类型号。
Get-WmiObject win32_logicaldisk -ComputerName -Filter "Drivetype=3" |`
select SystemName,DeviceID,VolumeName,@{Label="Total Size";Expression={$_.Size / 1gb -as [int] }},@{Label="Free Size";Expression={$_.freespace / 1gb -as [int] }}|Format-Table -AutoSize
We discuss the transformation of the above data into a SQL Table using some direct as well as indirect methods in this post:
在本文中，我们将讨论使用一些直接方法和间接方法将上述数据转换为SQL表：
- using Invoke-Sqlcmd 使用Invoke-Sqlcmd
- using ADO 使用ADO
- WMI Query WMI查询
- using Export commands such as JSON,XML and CSV 使用导出命令，例如JSON，XML和CSV
先决条件 (Pre-requisites)
Before we proceed, let’s look at what we need before we can proceed:
在继续之前，让我们先看一下我们需要什么：
- SSMS 17.2 SSMS 17.2
- SQL Server PowerShell module SQL Server PowerShell模块
入门 (Getting started)
Let’s now start transforming the data we received, into a SQL table.
现在开始将接收到的数据转换为SQL表。
调用Sqlcmd (Invoke-Sqlcmd)
The Invoke-Sqlcmd is a wrapper class and PowerShell version of SQL Server sqlcmd command with additional capabilities such as data manipulation and data transformations with a focus on the output data.
Invoke-Sqlcmd是SQL Server sqlcmd命令的包装器类和PowerShell版本，具有其他功能，例如数据处理和数据转换（着重于输出数据）。
The process is pretty simple:
这个过程非常简单：
- PoShDisk Table PoShDisk表
CREATE TABLE tbl_PoShDisk ( [SystemName] VARCHAR(40) not null, [DeviceID] VARCHAR(40) not null, [VolumeName] VARCHAR(40) not null, [TotalSize] int not null, [FreeSize] int not null ) - SqlServer Module SqlServer模块
- declare the variable and the connection string 声明变量和连接字符串
- define the function which prepares WMI data for SQL insertion 定义为SQL插入准备WMI数据的函数
- Win32_LogicalDisk class of WMI Win32_LogicalDisk类
- loop through the result-set 遍历结果集
- insert the data into the SQL Table 将数据插入SQL表
Let us prepare the script
让我们准备脚本
#Declare Servername
$sqlServer='hqdbsp18'
#Invoke-sqlcmd Connection string parameters
$params = @{'server'='HQDBT01';'Database'='SQLShackDemo'}
#Fucntion to manipulate the data
Function writeDiskInfo
{
param($server,$devId,$volName,$frSpace,$totSpace)
$totSpace=[math]::Round(($totSpace/1073741824),2)
$frSpace=[Math]::Round(($frSpace/1073741824),2)
$usedSpace = $totSpace - $frspace
$usedSpace=[Math]::Round($usedSpace,2)
# Data preparation for loading data into SQL table
$InsertResults = @"
INSERT INTO [SQLShackDemo].[dbo].[tbl_PosHdisk](SystemName,DeviceID,VolumeName,TotalSize,FreeSize)
VALUES ('$SERVER','$devId','$volName',$totSpace,$usedSpace)
"@
#call the invoke-sqlcmdlet to execute the query
Invoke-sqlcmd @params -Query $InsertResults
}
#Query WMI query to store the result in a varaible
$dp = Get-WmiObject win32_logicaldisk -ComputerName $sqlServer| Where-Object {$_.drivetype -eq 3}
#Loop through array
foreach ($item in $dp)
{
#Call the function to transform the data and prepare the data for insertion
writeDiskInfo $sqlServer $item.DeviceID $item.VolumeName $item.FreeSpace $item.Size
}
#Query the destination table to view the result
Invoke-Sqlcmd @params -Query "SELECT * FROM tbl_PosHdisk" | format-table -AutoSize
Invoke-Sqlcmd @params -Query "SELECT * FROM tbl_PosHdisk" | format-table -AutoSize
The below screen shot shows the steps and the results. The output is queried from the table tbl_PoShDisk
下面的屏幕截图显示了步骤和结果。 从表tbl_PoShDisk查询输出
There we go; we see how the data from the WMI query was transferred to an SQL Table
我们去了； 我们将看到WMI查询中的数据如何传输到SQL表
ADO.NET对象 (ADO.NET objects)
Another method to execute a query in PowerShell is to use ADO.NET libraries, which requires creating a DataSet, then creating a DataAdapter, and finally filling the DataAdapter. For many data retrieval needs from scripts ADO.NET can be a little too heavy. Fortunately, there are several ways to make this task simpler and still retain the benefits of the .NET DataTables. Let’s look at three methods to get SQL Server data from PowerShell. Once you have the data in DataTable, we can transform the data into a number of things including piping the output to one of the built-in cmdlets.
在PowerShell中执行查询的另一种方法是使用ADO.NET库，这需要创建一个DataSet，然后创建一个DataAdapter，最后填充DataAdapter。 对于许多来自脚本的数据检索需求，ADO.NET可能会有点过于繁重。 幸运的是，有多种方法可以简化此任务，并且仍然保留.NET DataTables的优点。 让我们看一下从PowerShell中获取SQL Server数据的三种方法。 将数据存储在DataTable中之后，我们可以将数据转换为许多内容，包括将输出传递给内置cmdlet之一。
ADO.NET is a set of class libraries that are part of the .NET Framework. The ADO.NET classes are generally divided into two types: connected classes and disconnected classes.
ADO.NET是.NET Framework一部分的一组类库。 ADO.NET类通常分为两种类型：连接类和断开连接的类。
Connected class
连接类
- SqlConnection – connects to the SQL Server .NET data provider in order to establish and manage the connection to the target database SqlConnection –连接到SQL Server .NET数据提供程序，以便建立和管理与目标数据库的连接
- SqlCommand – contains the details necessary to issue a T-SQL command against a SQL Server database SqlCommand –包含对SQL Server数据库发出T-SQL命令所必需的详细信息
- SqlDataAdapter – provides a bridge between the connected classes and disconnected classes. This class includes the Fill and Update methods. Use the Fill method to populate a DataSet or DataTable object. Use the Update method to propagate updated data in a DataSet or DataTable object to the database SqlDataAdapter –在连接的类和断开连接的类之间提供桥梁。 此类包括Fill和Update方法。 使用Fill方法填充DataSet或DataTable对象。 使用Update方法将DataSet或DataTable对象中的更新数据传播到数据库
- SqlBulkCopy –SqlBulkCopy – efficiently bulk loads a SQL Server table with data from another source 高效地使用其他来源的数据批量加载SQL Server表
Disconnected classes
断开类
- DataTable – stores the data returned by your query. The data is stored in rows and columns, similar to how data is stored in a database table DataTable –存储查询返回的数据。 数据存储在行和列中，类似于数据存储在数据库表中的方式
Script preparation
脚本准备
#Invoke-sqlcmd Connection string parameters
$params = @{'server'='HQDBT01';'Database'='SQLShackDemo'}
#Server to query WMI class win32_logicalDisks
$server = 'hqdbsp18'
#Prepare Insert Statement
$insert = @'
INSERT INTO [SQLShackDemo].[dbo].[tbl_PosHdisk](SystemName,DeviceID,VolumeName,TotalSize,FreeSize)
VALUES ('{0}','{1}','{2}','{3}','{4}')
'@
Try {
#Define connction string of target database
$connectionString = 'Data Source=HQDBT01;Initial Catalog=SQLShackDemo;Integrated Security=SSPI'
# connection object initialization
$conn = New-Object System.Data.SqlClient.SqlConnection($connectionString)
#Open the Connection
$conn.Open()
# Prepare the SQL
$cmd = $conn.CreateCommand()
#WMI ouput transformation to SQL table
Get-WmiObject win32_logicaldisk -ComputerName $server -Filter "Drivetype=3" |`
select SystemName,DeviceID,VolumeName,@{Label="TotalSize";Expression={$_.Size / 1gb -as [int] }},@{Label="FreeSize";Expression={$_.freespace / 1gb -as [int] }}|`
ForEach-Object{
$cmd.CommandText = $insert -f $_.SystemName, $_.DeviceID, $_.VolumeName, $_.TotalSize, $_.FreeSize
$cmd.ExecuteNonQuery()
}
#Close the connection
$conn.Close()
}
Catch {
Throw $_
}
Invoke-Sqlcmd @params -Query "SELECT * FROM tbl_PosHdisk" | format-table -AutoSize
The output is given below
输出如下
使用SqlBulkCopy (Using SqlBulkCopy)
We write a function to perform the copy operation. The function, Out-DataTable can be found in Appendix (A). This function takes care of converting the output of the WMI query to the data table. The data-table output is then fed to the SqlBulkCopy class in order to write the data to the SQL table. The SqlBulkCopy class loads a SQL Server table with data from another source which in this case is Win32_LogicalDisks.
我们编写一个函数来执行复制操作。 函数Out-DataTable可以在附录（A）中找到 。 此函数负责将WMI查询的输出转换为数据表。 然后，将数据表的输出馈送到SqlBulkCopy类，以便将数据写入SQL表。 SqlBulkCopy类使用来自另一个源（在本例中为Win32_LogicalDisks）的数据加载SQL Server表。
#Invoke-sqlcmd Connection string parameters
$params = @{'server'='HQDBT01';'Database'='SQLShackDemo'}
#function to retrieve disk information
Function Get-DisksSpace ([string]$Servername)
{
Get-WmiObject win32_logicaldisk -ComputerName $Servername -Filter "Drivetype=3" |`
select SystemName,DeviceID,VolumeName,@{Label="Total SIze";Expression={$_.Size / 1gb -as [int] }},@{Label="Free Size";Expression={$_.freespace / 1gb -as [int] }}
}
#Variable to hold output as data-table
$dataTable = Get-DisksSpace hqdbsp18 | Out-DataTable
#Define Connection string
$connectionString = "Data Source=hqdbt01; Integrated Security=True;Initial Catalog=SQLShackDemo;"
#Bulk copy object instantiation
$bulkCopy = new-object ("Data.SqlClient.SqlBulkCopy") $connectionString
#Define the destination table
$bulkCopy.DestinationTableName = "tbl_PosHdisk"
#load the data into the target
$bulkCopy.WriteToServer($dataTable)
#Query the target table to see for output
Invoke-Sqlcmd @params -Query "SELECT * FROM tbl_PosHdisk" | format-table -AutoSize
将PowerShell导出选项与SQL构造一起使用 (Use PowerShell Export options with SQL constructs)
We can also use SQL constructs with the PowerShell Export options. We can:
我们还可以将SQL构造与PowerShell导出选项一起使用。 我们可以：
- export the WMI data to JSON file and query using SQL native JSON constructs 将WMI数据导出到JSON文件并使用SQL本机JSON构造进行查询
- export to XML data and query using Nodes() 导出到XML数据并使用Nodes（）查询
- export the data to CSV format and use SQL native Bulk-Insert command to query the data 将数据导出为CSV格式，并使用SQL本机Bulk-Insert命令查询数据
Let’s look at these processes one by one.
让我们一一看一下这些过程。
使用JSON (Using JSON )
The ConvertTo-Json cmdlet converts an object to a valid JSON string. Using this cmdlet, the output is converted to JSON (JavaScript Object Notation). The JSON file is then queried using JSON construct in SQL Server, called OPEN JSON, to transform the data form the win32_LogicalDisk WMI class, to a relational table.
ConvertTo-Json cmdlet将对象转换为有效的JSON字符串。 使用此cmdlet，输出将转换为JSON（JavaScript对象表示法）。 然后，使用SQL Server中称为OPEN JSON的 JSON构造查询JSON文件，以将数据从win32_LogicalDisk WMI类转换为关系表。
The WMI output is stored in a variable which is then written to a JSON file using Out-File formatting cmdlet.
WMI输出存储在一个变量中，然后使用Out-File格式cmdlet将其写入JSON文件。
$JSON=Get-WmiObject win32_logicaldisk -ComputerName hqdbsp18 -Filter "Drivetype=3" |select SystemName,DeviceID,VolumeName,@{Label="Total SIze";Expression={$_.Size / 1gb -as [int] }},@{Label="Free Size";Expression={$_.freespace / 1gb -as [int] }} |ConvertTo-Json
$json |Out-File \\hqdbt01\f$\PowerSQL\DiskSpace.JSON
The output of JSON file is shown below
JSON文件的输出如下所示
Let’s now feed the JSON into an SQL table
现在让我们将JSON输入到SQL表中
SELECT t.*
FROM
OPENROWSET(BULK N'\\hqdbt01\f$\PowerSQL\DiskSpace.JSON', SINGLE_NCLOB) AS JSON
CROSS APPLY OPENJSON(BulkColumn)
WITH(
Server NVARCHAR(10),
DeviceID NVARCHAR(20),
VolumeName NVARCHAR(20),
[Total SIze] DECIMAL(5,2),
[Free Size] DECIMAL(5,2)
) AS t
The data can be fed to table using the Insert SQL statement
可以使用Insert SQL语句将数据提供给表
使用XML (Using XML)
The XML data comes from the XML file and is stored in a column bulkcolumn; use the XML methods, extract values with xml.value(), project nodes with xml.nodes(), use CROSS APPLY to join its nodes to derive all the values of the nodes.
XML数据来自XML文件，并存储在一列bulkcolumn中。 使用XML方法，使用xml.value（）提取值，使用xml.nodes（）提取项目节点，使用CROSS APPLY联接其节点以导出节点的所有值。
The below query writes WMI output to XML File
下面的查询将WMI输出写入XML文件
Get-WmiObject win32_logicaldisk -ComputerName hqdbsp18 -Filter "Drivetype=3" |`
select SystemName,DeviceID,VolumeName,@{Label="Total SIze";Expression={$_.Size / 1gb -as [int] }},@{Label="Free Size";Expression={$_.freespace / 1gb -as [int] }} |ConvertTo-Xml -as String -NoTypeInformation|`
Set-Content -path \\hqdbt01\f$\PowerSQL\DiskSpace.xml
The below is the generated XML file
下面是生成的XML文件
- load the data into temp table which holds XMLData as its column 将数据加载到将XMLData作为其列的临时表中
- use OPENROWSET clause to load XML data to single column 使用OPENROWSET子句将XML数据加载到单列
- transform each nodes using XML value() and Query() method to derive its values 使用XML value（）和Query（）方法转换每个节点以得出其值
CREATE TABLE tbl_XMLDisk
(
XMLData XML,
)
--truncate table XMLwithOpenXML
INSERT INTO tbl_XMLDisk(XMLData)
SELECT
CONVERT(XML, BulkColumn) AS BulkColumn
FROM
OPENROWSET(BULK 'f:\PowerSQL\DiskSpace.xml', SINGLE_BLOB) AS x;
SELECT XMLData FROM tbl_XMLDisk
SELECT
p.value('(./Property)[1]', 'VARCHAR(20)') AS SystemName,
p.value('(./Property)[2]', 'VARCHAR(30)') AS DeviceID,
p.value('(./Property)[3]', 'VARCHAR(30)') AS VolumeName,
p.value('(./Property)[4]', 'int') AS [Total SIze],
p.value('(./Property)[5]', 'int') AS [Free Size]
FROM tbl_XMLDisk
CROSS APPLY XMLData.nodes('/Objects/Object') t(p)
The XML Query output
XML查询输出
使用批量插入 (Using BULK INSERT)
The first statement we’ll look at is BULK INSERT, which lets you import data from a data file into a table In the following example, I import the data from the CSV file into the SQL table
我们将要看的第一条语句是BULK INSERT ，它使您可以将数据文件中的数据导入表中。在下面的示例中，我将数据文件从CSV文件中导入SQL表中。
Get-WmiObject win32_logicaldisk -ComputerName hqdbsp18 -Filter "Drivetype=3" |`
select SystemName,DeviceID,VolumeName,@{Label="Total SIze";Expression={$_.Size / 1gb -as [int] }},@{Label="Free Size";Expression={$_.freespace / 1gb -as [int] }} |`
ConvertTo-Csv -NoTypeInformation| Set-Content -path \\hqdbt01\f$\PowerSQL\DiskSpace.csv
--Create table.
CREATE TABLE tbl_CSVDisk
(
[SystemName] VARCHAR(40),
[DeviceID] VARCHAR(40),
[VolumeName] VARCHAR(40),
[TotalSize] VARCHAR(40),
[FreeSize] VARCHAR(40)
)
--Load the data into the SQL table starting with 2 row, comma(‘,’) as delimiter and newline as
--rowseparator
BULK
INSERT tbl_CSVDisk
FROM 'F:\PowerSQL\DiskSpace.csv'
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
--Check the content of the table.
SELECT REPLACE(systemName,'"','') systemName,
REPLACE([DeviceID],'"','') [DeviceID],
REPLACE([VolumeName],'"','') [VolumeName],
REPLACE([TotalSize],'"','') [TotalSize],
REPLACE([FreeSize],'"','') [FreeSize]
FROM tbl_CSVDisk
GO
结语 (Wrapping up)
Different methods and various techniques can be used to achieve a specific result. The challenge is to use the right tool, the right way, for the right job: the proverbial “Driving a screw with a hammer” problem. When presented with a new tool set, we shouldn’t try and use it the same way as other tools we’ve had in the past. Instead, we must learn the tool so we can make the best use of it for your process.
可以使用不同的方法和各种技术来获得特定的结果。 挑战在于使用正确的工具，正确的方法来完成正确的工作：众所周知的“用锤子驱动螺丝”问题。 当提供新工具集时，我们不应尝试使用与过去使用过的其他工具相同的方式。 相反，我们必须学习该工具，以便我们可以在您的过程中充分利用它。
In this case, we may have to consider the efficiency of the methods we listed, and make a decision about which method to use.
在这种情况下，我们可能必须考虑所列方法的效率，并决定要使用哪种方法。
附录（A） (Appendix (A))
Function Out-DataTable
{
$dt = new-object Data.datatable
$First = $true
foreach ($item in $input){
$DR = $DT.NewRow()
$Item.PsObject.get_properties() | foreach {
if ($first) {
$Col = new-object Data.DataColumn
$Col.ColumnName = $_.Name.ToString()
$DT.Columns.Add($Col) }
if ($_.value -eq $null) {
$DR.Item($_.Name) = "[empty]"
}
elseif ($_.IsArray) {
$DR.Item($_.Name) =[string]::Join($_.value ,";")
}
else {
$DR.Item($_.Name) = $_.value
}
}
$DT.Rows.Add($DR)
$First = $false
}
return @(,($dt))
}
翻译自: https://www.sqlshack.com/6-methods-write-powershell-output-sql-server-table/