SSIS概述
What is SSIS?
SQL Server Integration Services is a component of Microsoft SQL Server to perform data migration tasks.
A platform for data integration and workflow applications.
A tool for data extraction, transformation , and loading(ETF) First released with Microsoft SQL Server 2005, SSIS replaced Data Transformation Services, which had been a feature of SQL Server since Version 7.0. Unlike DTS, which was included in all versions, SSIS is only available in the "Standard" and "Enterprise" editions.
SSIS 概述
简单的说，SSIS是一个数据集成和工作流解决方案。它的典型应用就是建立SSIS包，将数据按需求打包转移.
数据的ETL: (Extraction, Transformation, Loading)
SSIS典型应用
1 从各种不同的数据源中取数据
2 存入数据仓库
3 清理和标准化数据
4 自动数据管理和加载
SSIS重要概念
PackagePackages are the root of the hierarchy. Although an SSIS solution can contain a number of packages, they are not held, logically or physical, in any kind of container.
ContainerThe level beneath the package can be either a container or some kind of executable (a Control Flow executable). A container is a logical construct that can hold other containers or components grouped together under the "super-container."
ExecutableA component that implements the necessary SSIS interfaces and so makes itself available for use within SSIS. An example is the Script Component, which supports the running of Visual Basic.NET code within your package against contained Data Flow.
VariablesSSIS provides support for strongly typed partCommon Type System (CTS)–compliant variables as a part of any container or package.
ScopeScope refers to the visibility and usability of a variable to a piece of code, based on where in the code the variable is defined. For example, a variable with global scope is defined at package level and is visible to and usable by any package, container, or component in the hierarchy beneath it. By contrast, a variable defined at Data Flow task level is visible only to that task and its subtasks.
Precedence constraintsControl Flow design surface may have multiple Control Flow items, in this situation we need to decide which item will execute first. Using Precedence Constraints, we can define processing flow for Control Flow items. We can define different flow based on success or failure of any item.
Event HandlersEvent handlers are offered at most levels of the package hierarchy. The types of event handlers differ depending upon the type of object against which you want to handle events.
SSIS架构
- SSIS Service
- SSIS Object Model
- SSIS runtime engine and the runtime executables
-
SSIS dataflow engine and the dataflow components
- Monitors running Integration Services packages and manages the storage of packages
- Integration Services object model
- Includes native and managed application programming interfaces (API) for accessing
- Integration Services tools, command-line utilities, and custom applications
- Runs packages
- Supports logging, debugging, config, connections, & transactions
- SSIS Run-time executables
-
Package, Containers, Tasks and Event Handlers
- Provides In-Memory buffers to move data
- Calls Source Adaptors to files & DBs
- Provides Transformations to modify data
- Destination Adaptors to load data into data stores
- Components
-
Source, Destination Adaptors & transformations
SSIS开发工具:
SQL Server Business Intelligence Development Studio (BIDS)
Control Flow
Container
Containers provide structure in packages and services to tasks in the control flow. Integration Services include the following container types, for grouping tasks and implementing repeating control flows:
- The Foreach Loop container: It enumerates a collection and repeats its control flow for each member of the collection. The Foreach Loop Container is for situations where you have a collection of items and wish to use each item within it as some kind of input into the downstream flow.
- For Loop Container: It's a basic container that provides looping functionality. A For loop contains a counter that usually increments (though it sometimes decrements), at which point a comparison is made with a constant value. If the condition evaluates to True, then the loop execution continues.
-
Sequence Container: One special kind of container both conceptually and physically can hold any other type of container or Control Flow component. It is also called "container container", or super container.
Tasks do the work in packages. Integration Services includes tasks for performing a variety of functions.
- The Data Flow task: It defines and runs data flows that extract data, apply transformations, and load data.
- Data preparation tasks: It copies files and directories, downloads files and data, saves data returned by Web methods, or works with XML documents.
- Workflow tasks: It communicates with other processes to run packages or programs, sends and receives messages between packages, sends e-mail messages, reads Windows Management Instrumentation (WMI) data, or watch for WMI events.
- SQL Server tasks: It accesses, copy, insert, delete, or modify SQL Server objects and data.
- Analysis Services tasks: It creates, modifies, deletes, or processes Analysis Services objects.
- Scripting tasks: It extends package functionality through custom scripts.
-
Maintenance tasks: It performs administrative functions, such as backing up and shrinking SQL Server databases, rebuilding and reorganizing indexes, and running SQL Server Agent jobs.
Precedence constraints connect containers and task in packages into an ordered control flow. You can control the sequence execution for tasks and containers, and specify conditions that determine whether tasks and containers run.
数据流
It's processing responsibilities by employing the pipeline paradigm, carrying data record by record from its source to a destination and modifying it in transit by applying transformations. (There are exceptions to this rule, since some of them, such as Sort or Aggregate require the ability to view the entire data set before handing it over to their downstream counterparts). Items which are used to creating a data flow categorize into three parts.
由以下三个部分组成：
Data Flow Source
Data Flow Transformation
Data Flow Destination
参考资源：
- SQL Server Integration Services official site
- SQL Server Integration Services (SSIS) Resources
- SQL Server Integration Services Discussion Forum
- 5 Tips for a Smooth SSIS Upgrade to SQL Server 2012
- SQLIS.com
- SSIS Expression Examples
- SSIS resources
- SSIS Scripts
- SQL Server 2005 version comparison
- SQL Server Integration Tools
- SSIS Community Tasks and Components
- SSIS package search and documentation tool, SSIS-DTS Package Search
- SSIS Documentation with Object Lineage & Impact Analysis
- SSIS Package Configuration Editor
- SSIS 2005/2008 Documenter
- SSIS 2005/2008 Design Patterns, Tips, and Tricks
- http://msdn.microsoft.com/zh-cn/library/ms137709.aspx
-
下章主要讲述SSIS的实例应用。