SSIS Catalog1：View Overview
The SSISDB catalog is the central point for working with Integration Services (SSIS) projects that you’ve deployed to the Integration Services server. For example, you set project and package parameters, configure environments to specify runtime values for packages, execute and troubleshoot packages, and manage Integration Services server operations.
The objects that are stored in the SSISDB catalog include projects, packages, parameters, environments, and operational history.
You inspect objects, settings, and operational data that are stored in the SSISDB catalog, by querying the views in the SSISDB database. You manage the objects by calling stored procedures in the SSISDB database or by using the UI of the SSISDB catalog. In many cases, the same task can be performed in the UI or by calling a stored procedure.
一，Catalog objects
1，Catalog的object 有Folder，Project，Package ，Executable 和 Parameter
- 使用 [catalog].[folders] 查看Folder
- 使用 [catalog].[projects] 查看Project
- 使用 [catalog].[packages] 查看Package
- 使用 [catalog].[executables] 和 [catalog].[executable_statistics] 查看Executable
- 使用 [catalog].[object_parameters] 和 [catalog].[execution_parameter_values] 查看 Parameter
一个Executable 是package的一个可执行文件，位于Control flow tab中，是一个Task 或者 Container。
An executable is a task or container that you add to the control flow of a package.
2，[catalog].[object_parameters] ：Displays the parameters for all packages and projects in the Integration Services catalog.
关键的字段：
| Column name |
Data type |
Description |
|---|---|---|
| parameter_id |
bigint |
The unique identifier (ID) of the parameter. |
| project_id |
bigint |
The unique ID of the project. |
| object_type |
smallint |
The type of parameter. The value is 20 for a project parameter and the value is 30 for a package parameter. |
| object_name |
sysname |
The name of the corresponding project or package. |
| parameter_name |
sysname |
The name of the parameter. |
| data_type |
nvarchar(128) |
The data type of the parameter. |
| required |
bit |
When the value is 1, the parameter value is required in order to start the execution. When the value is 0, the parameter value is not required to start the execution. |
| sensitive |
bit |
When the value is 1, the parameter value is sensitive. When the value is 0, the parameter value is not sensitive. |
| description |
nvarchar(1024) |
An optional description of the package. |
| design_default_value |
sql_variant |
The default value for the parameter that was assigned during the design of the project or package. |
| default_value |
sql_variant |
The default value that is currently used on the server. |
| value_type |
char(1) |
Indicates the type of parameter value. This field displays V when parameter_value is a literal value and R when the value is assigned by referencing an environment variable. |
| value_set |
bit |
When the value is 1, the parameter value has been assigned. When the value is 0, the parameter value has not been assigned. |
| referenced_variable_name |
nvarchar(128) |
The name of the environment variable that is assigned to the value of the parameter. The default value is NULL. |
2.1， object_type：The value is 20 for a project parameter and the value is 30 for a package parameter.
2.2，一个parameter 有三种value： design_default_value，default_value， 和 execution_value，
Design Default Value 是指在BIDT中Design package时指定的默认值
Default Value 是指Parameter 部署到IS中的默认值，如果没有修改Parameter的value，那么Default value 就是Design Default Value；当然，可以在SSMS中修改成一个和Data Type 兼容的值，可以和Design Default Value 不同。
Execution Value是指Parameter 是指 package 运行时Parameter 的值，由于在execute package，可以选择设置Parameter的Value 或 选择使用Default Value，所以 Execution Value 就是Package 的一次execution所使用的parameter value。
2.3， value_type
This field displays V when parameter_value is a literal value and R when the value is assigned by referencing an environment variable.
catalog.execution_parameter_values ：Displays the actual parameter values that are used by Integration Services packages during an instance of execution.
| Column name |
Data type |
Description |
|---|---|---|
| execution_parameter_id |
bigint |
Unique identifier (ID) of the execution parameter. |
| execution_id |
bigint |
Unique ID for the instance of execution. |
| object_type |
smallint |
When the value is 20, the parameter is a project parameter. When the value is 30, the parameter is a package parameter. When the value is 50, the parameter is one of the following.
|
| parameter_data_type |
nvarchar(128) |
The data type of the parameter. |
| parameter_name |
sysname |
The name of the parameter. |
| parameter_value |
sql_variant |
The value of the parameter. When sensitive is 0, the plaintext value is shown. When sensitive is 1, the NULL value is displayed. |
| sensitive |
bit |
When the value is 1, the parameter value is sensitive. When the value is 0, the parameter value is not sensitive. |
| required |
bit |
When the value is 1, the parameter value is required in order to start the execution. When the value is 0, the parameter value is not required to start the execution. |
| value_set |
bit |
When the value is 1, the parameter value has been assigned. When the value is 0, the parameter value has not been assigned. |
| runtime_override |
bit |
When the value is 1, the parameter value was changed from the original value before the execution was started. When the value is 0, the parameter value is the original value that was set. |
二，Catalog Configuration
2.1, 使用 [catalog].[catalog_properties] 查看Catalog的配置信息
The following table lists the property names shown in the Catalog Properties dialog box and the corresponding properties in the database view.
| Property Name (Catalog Properties dialog box) |
Property Name (database view) |
|---|---|
| Encryption Algorithm Name |
ENCRYPTION_ALGORITHM |
| Clean Logs Periodically |
OPERATION_CLEANUP_ENABLED |
| Retention Period (days) |
RETENTION_WINDOW |
| Periodically Remove Old Versions |
VERSION_CLEANUP_ENABLED |
| Maximum Number of Versions per Project |
MAX_PROJECT_VERSIONS |
| Server-wide Default Logging Level |
SERVER_LOGGING_LEVEL |
Clean Logs Periodically
The job step for operations cleanup runs when this property is set to True.
Retention Period (days)
Defines the maximum age of allowable operations data (in days). Older data are removed.
The minimum value is one day. The maximum value is limited only by the maximum value of the SQL Server int data.
Periodically Remove Old Versions
The job step for project version cleanup runs when this property is set to True.
Maximum Number of Versions per Project
Defines how many versions of a project are stored in the catalog. Older versions of projects are removed.
2.2, Operations and Project Version Cleanup
Status data for many of the operations in the catalog is stored in internal database tables. For example, the catalog tracks the status of package executions and project deployments. To maintain the size of the operations data, the SSIS Server Maintenance Job in SQL Server Management Studio is used to remove old data. This SQL Server Agent job is created when Integration Services is installed.
You can update or redeploy an Integration Services project by deploying it with the same name to the same folder in the catalog. By default, each time you redeploy a project, the SSISDB catalog retains the previous version of the project. To maintain the size of the operations data, the SSIS Server Maintenance Job is used to remove old versions of projects.
三，Executions
An execution is an instance of a package execution.To cause a running package to pause and create a dump file, call the catalog.create_execution_dump stored procedure. A dump file provides information about the execution of a package that can help you troubleshoot execution issues. For more information about generating and configuring dump files, see Generating Dump Files for Package Execution.
- catalog.executions (SSISDB Database)
- catalog.executables
- catalog.operations (SSISDB Database)
- catalog.event_messages
- catalog.event_message_context
1，Execution
An Execution 是SSIS project 一次执行的实例，每次执行都会生成一个唯一的ExecutionID，并记录Execution 的结果。
Execution Status: The status of the operation. The possible values are created (1), running (2), canceled (3), failed (4), pending (5), ended unexpectedly (6), succeeded (7), stopping (8), and completed (9).
2，Executable
An executable is a task or container that you add to the control flow of a package.
Executable 是run-time的概念，SSIS Project每一次执行，都会生成Executable运行的信息，通过 [catalog].[executables] 查看每个Execution中执行的Executable 列表，通过 [catalog].[executable_statistics] 查看Executable运行的结果。
Executable 的 Execution_result ：0 (Success)，1 (Failure)，2 (Completion)，3 (Cancelled)
3，Operation
Operation是对SSIS执行的操作。
3.1，Operation Type
共有10种 Operation Type
3.2， Object：Operation 影响的对象，The type of object affected by the operation.
The object may be a folder (10), project (20), package (30), environment (40), or instance of execution (50).
3.3，Operation Status：
created (1), running (2), canceled (3), failed (4), pending (5), ended unexpectedly (6), succeeded (7), stopping (8), and completed (9).
3.4 Operation Message
Operation执行期间记录的Log，message 按照内容的不同分为多种MessageType，按照source的不同分为多种 message_source_type。
Each message is logged during an operation in the catalog. The message can be generated by the server, by the package execution process, or by the execution engine.
Operation type list:
| operation_type Value |
operation_type Description |
object_id Description |
object_name Description |
|---|---|---|---|
| 1 |
Integration Services initialization |
NULL |
NULL |
| 2 |
Retention window(SQL Agent job) |
NULL |
NULL |
| 3 |
MaxProjectVersion(SQL Agent job) |
NULL |
NULL |
| 101 |
deploy_project (Stored procedure) |
Project ID |
Project name |
| 106 |
restore_project(Stored procedure) |
Project ID |
Project name |
| 200 |
create_execution and start_execution (Stored procedures) |
Project ID |
NULL |
| 202 |
stop_operation (Stored procedure) |
Project ID |
NULL |
| 300 |
validate_project(Stored procedure) |
Project ID |
Project name |
| 301 |
validate_package(Stored procedure) |
Project ID |
Package name |
| 1000 |
configure_catalog (Stored procedure) |
NULL |
NULL |
Message Type List
| message_type Value |
Description |
|---|---|
| -1 |
Unknown |
| 120 |
Error |
| 110 |
Warning |
| 70 |
Information |
| 10 |
Pre-validate |
| 20 |
Post-validate |
| 30 |
Pre-execute |
| 40 |
Post-execute |
| 60 |
Progress |
| 50 |
StatusChange |
| 100 |
QueryCancel |
| 130 |
TaskFailed |
| 90 |
Diagnostic |
| 200 |
Custom |
| 140 |
DiagnosticEx Whenever an Execute Package task executes a child package, it logs this event. The event message consists of the parameter values passed to child packages. The value of the message column for DiagnosticEx is XML text. |
| 400 |
NonDiagnostic |
| 80 |
VariableValueChanged |
Message Source Type List：
| message_source_type |
Description |
|---|---|
| 10 |
Entry APIs, such as T-SQL and CLR Stored procedures |
| 20 |
External process used to run package (ISServerExec.exe) |
| 30 |
Package-level objects |
| 40 |
Control Flow tasks |
| 50 |
Control Flow containers |
| 60 |
Data Flow task |
4，Event Message
Event Message：在SSIS执行Operation时，记录Event产生的Message。
4.1 Event message
| Operation_id |
bigint |
The type of operation. |
| Package_name |
nvarchar(260) |
The name of the package file. |
| Event_name |
nvarchar(1024) |
The run-time event associated with the message. |
| Message_source_name |
nvarchar(4000) |
The package component that is the source of the message. |
| Message_source_id |
nvarchar(38) |
The unique ID of the source of the message. |
| Subcomponent_name |
nvarchar(4000) |
The data flow component that is the source of the message. When messages are returned by the Integration Services engine, SSIS.Pipeline appears in this column. |
5, Event message context
Event Message context：和Event Message相关的Context
| Column name |
Data type |
Description |
|---|---|---|
| Context_id |
bigint |
Unique ID for the error context. |
| Event_message_id |
bigint |
Unique ID for the message that the context relates to. |
| Context_depth |
int |
As the depth increases, the context is further from the error. When an error occurs, the context depth starts at 1. The value of 0 indicates the state of the package before execution starts. |
| Package_path |
Nvarchar(max) |
The package path for the context source. |
| Context_type |
smallint |
The type of object that is the source for the context. See the Remarks section for a list of context types. |
| Context_source_name |
Nvarchar(4000) |
The name of the object that is the source for the context. |
| Context_source_id |
Nvarchar(38) |
The unique ID of the object that is the source for the context. |
| Property_name |
Nvarchar(4000) |
The name of the property associated with the source of the context. |
| Property_value |
Sql_variant |
The property value associated with the source of the context. |
The following table lists the context types.
| Context type value |
Type Name |
Description |
| 10 |
Task |
State of a task when an error occurred. |
| 20 |
Pipeline |
Error from a pipeline component: source, destination, or transformation component. |
| 30 |
Sequence |
State of a sequence. |
| 40 |
For Loop |
State of a For Loop. |
| 50 |
Foreach Loop |
State of a Foreach Loop |
| 60 |
Package |
State of the package when an error occurred. |
| 70 |
Variable |
Variable value |
| 80 |
Connection manager |
Properties of a connection manager. |
参考文档：
https://msdn.microsoft.com/en-us/library/hh479588(v=sql.110).aspx
https://msdn.microsoft.com/en-us/library/ff878135(v=sql.110).aspx