ArcGIS Engine,要素类中插入数据效率探究

 

在ArcGIS Engine程序开发过程中是否遇到过要将新获取的数据向已有要素类中插入?对于数据插入的几种方法您是否清楚?各种方法的效率您对比过吗?今天我们讨论的话题为Engine中如何往已有要素类中插入数据,上述问题的答案会为您一一揭晓。

一、往已有要素类中插入数据的几种方式及其优缺点:

1,IFeatureClass.CreateFeature配合IFeature.Store使用。该方法在调用Store时会触发所有与Feature相关的行为,不但会触发IObjectClassEvents,还会触发涉及网络、注记要素,以及参与拓扑的要素等特殊的行为(比如向参加拓扑的要素类中添加要素后会自动创建dirty area)。

该方法在调用CreateFeature方法时,创建要素的OID,执行Store时将要素存入数据库中。

优点:代码简单; 
缺点:效率低,如果仅仅插入几个要素或者进行测试而不考虑性能的话可以考虑。

2,IFeatureClass.CreateFeatureBuffer配合insert cursor使用。不会触发事件,该方法常常用来一次性插入大量要素,比第一种方法效率高很多。此外,IFeatureClass.Insert(bool useBuffering)方法中参数useBuffering建议总将其设为true,这样数据会先缓冲在客户端,然后执行Flush时批量写入,以提升效率。

该方法创建的IFeatureBuffer可以进行多次赋值,在执行InsertFeature时,创建要素的OID,但是此时数据并没有写入到数据库中,只有在执行IFeatureCursor.Flush时数据才会真正写入到数据库中。注意Flush方法是需要手动调用的,如果不执行,在程序释放或者Buffer满了会自动执行,但这样会无法检测错误,比如要素类空间不够了就会报错。

优点:效率高,尤其是插入大量数据时。如果仅就性能而言,除非插入一个要素,其余情况都推荐使用该方法; 
缺点:代码相比第一种,稍显复杂,注意需要释放ICursor和IFeatureBuffer对象。

这里提一下,该方法中涉及到cursor的释放,有两种方法释放游标: 
a, 直接使用ComReleaser.ReleaseCOMObject(cursor); 或者Marshal.FinalReleaseComObject(cursor);

b, 使用using(){ }方式,如下:

 

  using(ComReleaser comReleaser = new ComReleaser())
    {
        // Create a feature buffer.
        IFeatureBuffer featureBuffer = featureClass.CreateFeatureBuffer();
        comReleaser.ManageLifetime(featureBuffer);

        // Create an insert cursor.
        IFeatureCursor insertCursor = featureClass.Insert(true);
        comReleaser.ManageLifetime(insertCursor);

      …
    }

 

执行完using后会自动释放insertCursor和featureBuffer,可以避免程序比较复杂时不确定在哪释放或者忘记释放的情况,是比较优秀的编程方式,推荐使用。

3,使用Load-Only模式插入数据(SDE以及FileGDB都支持)。这种方法是对第二种方法的升级,主要用来在插入数据量特别大的情况下提高性能。Load-Only模式仅能用来插入新的要素,而不能用于编辑已有要素。原理是开启Load-Only模式后,插入数据时会停止更新空间索引和属性索引,解除Load-Only后才重建索引(重建索引是重建该要素类所有要素的索引,如果已有要素类中含有大量要素,而插入数据量不太大时,重建索引可能会影响性能)。开启Load-Only模式时,其它程序不能访问该数据,可以在获取排它锁之后设置Load-Only。如下:

 

    // Cast the feature class to the IFeatureClassLoad interface.
    IFeatureClassLoad featureClassLoad = (IFeatureClassLoad)featureClass;

    // Acquire an exclusive schema lock for the class.
    ISchemaLock schemaLock = (ISchemaLock)featureClass;
    try
    {
        schemaLock.ChangeSchemaLock(esriSchemaLock.esriExclusiveSchemaLock);

        // Enable load-only mode on the feature class.
        featureClassLoad.LoadOnlyMode = true;
        using(ComReleaser comReleaser = new ComReleaser())
        {
            // Create the feature buffer.
            IFeatureBuffer featureBuffer = featureClass.CreateFeatureBuffer();
            comReleaser.ManageLifetime(featureBuffer);

            // Create an insert cursor.
            IFeatureCursor insertCursor = featureClass.Insert(true);
            comReleaser.ManageLifetime(insertCursor);

            …
        }
    }
    catch (Exception)
    {
        // Handle the failure in a way appropriate to the application.
    }
    finally
    {
        // Disable load-only mode on the feature class.
        featureClassLoad.LoadOnlyMode = false;

        // Demote the exclusive schema lock to a shared lock.
        schemaLock.ChangeSchemaLock(esriSchemaLock.esriSharedSchemaLock);
}

 

插入数据常用的就是以上三种方式,下面两种方式也可以插入数据,但是不常用,简要介绍一下。

4,IObjectLoader.LoadObjects方法。该方法只能在Desktop产品下使用,也就是Engine许可无法使用,相比之前的方法,该方法代码简单,对于属性值的复制也更为简单,如下:

 

            IObjectLoader objectLoader = new ObjectLoaderClass();
            IFeatureClass sourceFeatureClass = getFeatureClass("esriDataSourcesGDB.FileGDBWorkspaceFactory", fileGDBPath, sourceFCName);
            IFeatureClass targetFeatureClass = getFeatureClass("esriDataSourcesGDB.SdeWorkspaceFactory", sdePath, targetFCName);

            string sInFieldList = targetFeatureClass.Fields.get_Field(0).Name;
            for(int i=1;i< targetFeatureClass.Fields.FieldCount;i++)
                sInFieldList = sInFieldList + "," + targetFeatureClass.Fields.get_Field(i).Name;

            IQueryFilter queryFilter = new QueryFilterClass();
            queryFilter.WhereClause = queryClause;
            queryFilter.SubFields = sInFieldList;

            IEnumInvalidObject invalidObject;
            objectLoader.LoadObjects(null, sourceFeatureClass as ITable, queryFilter, targetFeatureClass as ITable, targetFeatureClass.Fields, false, 0, false, false, 1000, out invalidObject);

 

 

 

 

 

优点:代码简单,属性值复制也更为简单; 
缺点:只能在Desktop许可下使用。

5,IFeatureClass.CreateFeature配合IFeatureClassWrite.WriteFeature/WriteFeatures使用。IFeatureClassWrite是low-level的接口,不会触发任何事件。只是使用WriteFeature/WriteFeatures替换IFeature.Store将要素写入数据库,用的人很少。

以上就是插入数据的几种方式,您是否都清楚了?接下来,在程序的编写过程中有哪些需要注意的呢?

a, 以下数据类型需要EngineGeoDB许可或者Desktop产品的Standard或者Advanced许可:涉及几何网络、拓扑、Dimension要素类、注记类、ArcSDE数据库的。

b, 以下情况需要开启编辑会话:涉及拓扑、几何网络、terrain、representation以及ArcSDE中的版本数据的。

c, 在给要素赋予Shape时,有必要执行一下ITopologicalOperator.Simplify来验证几何,尤其是往SDE中插入时,有几何错误会导致失败。

d, 插入数据可能导致要素类的Extent发生变化,插入完后建议调用IFeatureClassManage. UpdateExtent更新范围(需要获取要素类的排他锁,也就是不能有别的用户或程序访问该要素类),不然有可能执行缩放到图层或全图时数据显示不全。

e, Geometry的空间参考需要与目标要素类的空间参考相一致,否则会进行动态投影而影响性能。

f, 如果源要素类中有Z值或M值,注意赋Shape时需要使用IZAware或者IMAware进行设置。

二、效率对比

由于经常碰到用户需要将本地数据库的数据往SDE中导入,所以本文以将FileGDB中的点要素类插入到ArcSDE中已有的字段结构相同的要素类中为例进行测试。

代码可参考:ArcObjects帮助文档

测试结果:

1, 将100个点要素从FileGDB中插入SDE要素类中使用五种方法分别用时:2.25秒、0.19秒、0.21秒、0.23秒、2.40秒。

2, 将10000个点要素从FileGDB中插入SDE要素类中使用五种方法分别用时:185秒、14.5秒、11.3秒、13.2秒、204秒。

3, 将100000个点要素从FileGDB中插入SDE要素类中使用五种方法分别用时:1345秒、163秒、127秒、146秒、1409秒。

4, 文中提到使用游标方式插入数据,如果将IFeatureClass.Insert(bool useBuffering)方法中参数useBuffering设为true会提高效率,现在就来测试一下。将100000个点要素插入SDE中,设为false所用时间为:164秒,设为true所用时间为:122秒。

5, 开启编辑会话会不会提高效率呢?将100000个点要素插入SDE中,设为false所用时间为:161秒,设为true所用时间为:117秒。相比不开启编辑会话,几乎没有差别。

从上述测试结果可以看出:

a, 无论是插入少量数据还是插入大量数据,Insert方式都比CreateFeature方式快;

b, 如果仅插入少量数据,使用LoadOnly模式反而会慢(文中插入100个点),如果插入大量要素使用LoadOnly会快(文中插入1万和10万个点),但是如果该要素类中已有大量要素,而仅插入不是太多数据的话重建索引可能会影响性能;

c, IFeatureClassWrite.WriteFeature方法比IFeature.Store还要慢,这大概也是该方法用的很少的原因之一吧;

d, IObjectLoader.LoadObjects方法比Insert稍快,但是比LoadOnly要慢些(文中插入1万和10万个点)。由于测试数据中仅含有两个自定义字段,如果字段个数较多的话,我觉得这种方法会更快;

e, IFeatureClass.Insert(bool useBuffering)方法中参数useBuffering设为true确实可以提高效率;

f, 开启编辑会话与不开启效率几乎一样,当然如果涉及到拓扑,几何网络等复杂情况时必须开启编辑会话。

 

 

 

 

完整代码:

 

using ESRI.ArcGIS.ADF;
using ESRI.ArcGIS.Catalog;
using ESRI.ArcGIS.EditorExt;
using ESRI.ArcGIS.esriSystem;
using ESRI.ArcGIS.Geodatabase;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace insertingFeatures
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }
        string sdePath = @"C:\Users\Xinying\AppData\Roaming\ESRI\Desktop10.4\ArcCatalog\Connection to 192.168.220.132 (3).sde";
        string fileGDBPath = Application.StartupPath + "\\test.gdb";
        string sourceFCName = "testPoint_10w";
        string targetFCName = "testPoint_10w_sde";
        string queryClause = "OBJECTID <101";
        private void createFeatureToolStripMenuItem_Click(object sender, EventArgs e)
        {
            Stopwatch myWatch = Stopwatch.StartNew();

            IFeatureClass sourceFeatureClass = getFeatureClass("esriDataSourcesGDB.FileGDBWorkspaceFactory", fileGDBPath, sourceFCName);
            IFeatureClass targetFeatureClass = getFeatureClass("esriDataSourcesGDB.SdeWorkspaceFactory", sdePath, targetFCName);
          
            createFeature(sourceFeatureClass, targetFeatureClass);

            myWatch.Stop();
            string time = myWatch.Elapsed.TotalSeconds.ToString();
            MessageBox.Show(time + " Seconds");   
           
        }

        public void createFeature(IFeatureClass sourceFeatureClass, IFeatureClass targetFeatureClass)
        {
            IQueryFilter queryFilter = new QueryFilterClass();
            queryFilter.WhereClause = queryClause;
            IFeatureCursor cursor = sourceFeatureClass.Search(queryFilter, true);
            IFeature sourceFeature = cursor.NextFeature();
            while (sourceFeature != null)
            {
                IFeature targetFeature = targetFeatureClass.CreateFeature();

                //如果是线或面要素类需要执行下Simplify,这里用的点要素类,不做验证了
                targetFeature.Shape = sourceFeature.ShapeCopy;

                for (int i = 0; i < sourceFeature.Fields.FieldCount; i++)
                {
                    IField field = sourceFeature.Fields.get_Field(i);
                    if (field.Type != esriFieldType.esriFieldTypeOID && field.Type != esriFieldType.esriFieldTypeGeometry && field.Type != esriFieldType.esriFieldTypeGlobalID && field.Type != esriFieldType.esriFieldTypeGUID)
                    {
                        string fieldName = field.Name;
                        int index = targetFeature.Fields.FindField(fieldName);
                        if (index > -1 && fieldName != "Shape_Length" && fieldName != "Shape_Area")
                            targetFeature.set_Value(index, sourceFeature.get_Value(i));
                    }

                }
                targetFeature.Store();
                sourceFeature = cursor.NextFeature();

            }
            ComReleaser.ReleaseCOMObject(cursor);

            IFeatureClassManage targetFeatureClassManage = targetFeatureClass as IFeatureClassManage;
            targetFeatureClassManage.UpdateExtent();
        }
        private IFeatureClass getFeatureClass(string WorkspaceFactoryProgID, string GDBPath, string featureClassName)
        {
            IWorkspaceName workspaceName = new WorkspaceNameClass
            {
                WorkspaceFactoryProgID = WorkspaceFactoryProgID,
                PathName = GDBPath
            };

            IName workspaceIName = (IName)workspaceName;
            IWorkspace workspace = (IWorkspace)workspaceIName.Open();

            IFeatureClass featureClass = (workspace as IFeatureWorkspace).OpenFeatureClass(featureClassName);

            return featureClass; 
        }       

        private void insertFeatureToolStripMenuItem_Click(object sender, EventArgs e)
        {
            Stopwatch myWatch = Stopwatch.StartNew();

            IFeatureClass sourceFeatureClass = getFeatureClass("esriDataSourcesGDB.FileGDBWorkspaceFactory", fileGDBPath, sourceFCName);
            IFeatureClass targetFeatureClass = getFeatureClass("esriDataSourcesGDB.SdeWorkspaceFactory", sdePath, targetFCName);
           
            InsertFeaturesUsingCursor(sourceFeatureClass, targetFeatureClass);

            myWatch.Stop();
            string time = myWatch.Elapsed.TotalSeconds.ToString();
            MessageBox.Show(time + " Seconds"); 
        }
        public void InsertFeaturesUsingCursor(IFeatureClass sourceFeatureClass, IFeatureClass targetFeatureClass)
        {
            using (ComReleaser comReleaser = new ComReleaser())
            {
                // Create a feature buffer.
                IFeatureBuffer featureBuffer = targetFeatureClass.CreateFeatureBuffer();
                comReleaser.ManageLifetime(featureBuffer);

                // Create an insert cursor.
                IFeatureCursor insertCursor = targetFeatureClass.Insert(true);
                comReleaser.ManageLifetime(insertCursor);

                IQueryFilter queryFilter = new QueryFilterClass();
                queryFilter.WhereClause = queryClause;
                IFeatureCursor cursor = sourceFeatureClass.Search(queryFilter, true);

                IFeature sourceFeature = cursor.NextFeature();

                while (sourceFeature != null)
                {
                    //如果是线或面要素类需要执行下Simplify,这里用的点要素类,不做验证了
                    featureBuffer.Shape = sourceFeature.ShapeCopy;

                    for (int i = 0; i < sourceFeature.Fields.FieldCount; i++)
                    {
                        IField field = sourceFeature.Fields.get_Field(i);
                        if (field.Type != esriFieldType.esriFieldTypeOID && field.Type != esriFieldType.esriFieldTypeGeometry && field.Type != esriFieldType.esriFieldTypeGlobalID && field.Type != esriFieldType.esriFieldTypeGUID)
                        {
                            string fieldName = field.Name;
                            int index = featureBuffer.Fields.FindField(fieldName);
                            if (index > -1 && fieldName != "Shape_Length" && fieldName != "Shape_Area")
                                featureBuffer.set_Value(index, sourceFeature.get_Value(i));
                        }

                    }
                    insertCursor.InsertFeature(featureBuffer);
                    sourceFeature = cursor.NextFeature();
                }

                // Flush the buffer to the geodatabase.
                insertCursor.Flush();
                ComReleaser.ReleaseCOMObject(cursor);
                
            }
            IFeatureClassManage targetFeatureClassManage = targetFeatureClass as IFeatureClassManage;
            targetFeatureClassManage.UpdateExtent();
        }

        private void loadOnlyToolStripMenuItem_Click(object sender, EventArgs e)
        {
            Stopwatch myWatch = Stopwatch.StartNew();

            IFeatureClass sourceFeatureClass = getFeatureClass("esriDataSourcesGDB.FileGDBWorkspaceFactory", fileGDBPath, sourceFCName);
            IFeatureClass targetFeatureClass = getFeatureClass("esriDataSourcesGDB.SdeWorkspaceFactory", sdePath, targetFCName);
          
            LoadOnlyModeInsert(sourceFeatureClass, targetFeatureClass);

            myWatch.Stop();
            string time = myWatch.Elapsed.TotalSeconds.ToString();
            MessageBox.Show(time + " Seconds"); 
        }
        public void LoadOnlyModeInsert(IFeatureClass sourceFeatureClass, IFeatureClass targetFeatureClass)
        {
            // Cast the feature class to the IFeatureClassLoad interface.
            IFeatureClassLoad featureClassLoad = (IFeatureClassLoad)targetFeatureClass;

            // Acquire an exclusive schema lock for the class.
            ISchemaLock schemaLock = (ISchemaLock)targetFeatureClass;
            try
            {
                schemaLock.ChangeSchemaLock(esriSchemaLock.esriExclusiveSchemaLock);

                // Enable load-only mode on the feature class.
                featureClassLoad.LoadOnlyMode = true;
                using (ComReleaser comReleaser = new ComReleaser())
                {
                    // Create the feature buffer.
                    IFeatureBuffer featureBuffer = targetFeatureClass.CreateFeatureBuffer();
                    comReleaser.ManageLifetime(featureBuffer);

                    // Create an insert cursor.
                    IFeatureCursor insertCursor = targetFeatureClass.Insert(true);
                    comReleaser.ManageLifetime(insertCursor);

                    IQueryFilter queryFilter = new QueryFilterClass();
                    queryFilter.WhereClause = queryClause;
                    IFeatureCursor cursor = sourceFeatureClass.Search(queryFilter, true);

                    IFeature sourceFeature = cursor.NextFeature();

                    while (sourceFeature != null)
                    {
                        //如果是线或面要素类需要执行下Simplify,这里用的点要素类,不做验证了
                        featureBuffer.Shape = sourceFeature.ShapeCopy;

                        for (int i = 0; i < sourceFeature.Fields.FieldCount; i++)
                        {
                            IField field = sourceFeature.Fields.get_Field(i);
                            if (field.Type != esriFieldType.esriFieldTypeOID && field.Type != esriFieldType.esriFieldTypeGeometry && field.Type != esriFieldType.esriFieldTypeGlobalID && field.Type != esriFieldType.esriFieldTypeGUID)
                            {
                                string fieldName = field.Name;
                                int index = featureBuffer.Fields.FindField(fieldName);
                                if (index > -1 && fieldName != "Shape_Length" && fieldName != "Shape_Area")
                                    featureBuffer.set_Value(index, sourceFeature.get_Value(i));
                            }

                        }
                        insertCursor.InsertFeature(featureBuffer);
                        sourceFeature = cursor.NextFeature();
                    }

                    // Flush the buffer to the geodatabase.
                    insertCursor.Flush();
                    ComReleaser.ReleaseCOMObject(cursor);
                    IFeatureClassManage targetFeatureClassManage = targetFeatureClass as IFeatureClassManage;
                    targetFeatureClassManage.UpdateExtent();
                }
            }
            catch (Exception)
            {
                // Handle the failure in a way appropriate to the application.
                MessageBox.Show("无法获取该要素类的排它锁,检查ArcMap是否打开了该要素类,建议关闭!");
            }
            finally
            {
                // Disable load-only mode on the feature class.
                featureClassLoad.LoadOnlyMode = false;

                // Demote the exclusive schema lock to a shared lock.
                schemaLock.ChangeSchemaLock(esriSchemaLock.esriSharedSchemaLock);
            }
            
        }

        private void objectLoaderToolStripMenuItem_Click(object sender, EventArgs e)
        {
            Stopwatch myWatch = Stopwatch.StartNew();

            IObjectLoader objectLoader = new ObjectLoaderClass();
            IFeatureClass sourceFeatureClass = getFeatureClass("esriDataSourcesGDB.FileGDBWorkspaceFactory", fileGDBPath, sourceFCName);
            IFeatureClass targetFeatureClass = getFeatureClass("esriDataSourcesGDB.SdeWorkspaceFactory", sdePath, targetFCName);
             
            string sInFieldList = targetFeatureClass.Fields.get_Field(0).Name;
            for (int i = 1; i < targetFeatureClass.Fields.FieldCount; i++)
                sInFieldList = sInFieldList + "," + targetFeatureClass.Fields.get_Field(i).Name;
            
            IQueryFilter queryFilter = new QueryFilterClass();
            queryFilter.WhereClause = queryClause;
            queryFilter.SubFields = sInFieldList;
            
            IEnumInvalidObject invalidObject;
            objectLoader.LoadObjects(null, sourceFeatureClass as ITable, queryFilter, targetFeatureClass as ITable, targetFeatureClass.Fields, false, 0, false, false, 1000, out invalidObject);
            IFeatureClassManage targetFeatureClassManage = targetFeatureClass as IFeatureClassManage;
            targetFeatureClassManage.UpdateExtent();
           
            myWatch.Stop();
            string time = myWatch.Elapsed.TotalSeconds.ToString();
            MessageBox.Show(time + " Seconds"); 
        }

        private void comboBox1_SelectedIndexChanged(object sender, EventArgs e)
        {            
            queryClause = comboBox1.Text;
        }

        private void featureClassWriteToolStripMenuItem_Click(object sender, EventArgs e)
        {
            Stopwatch myWatch = Stopwatch.StartNew();

            IFeatureClass sourceFeatureClass = getFeatureClass("esriDataSourcesGDB.FileGDBWorkspaceFactory", fileGDBPath, sourceFCName);
            IFeatureClass targetFeatureClass = getFeatureClass("esriDataSourcesGDB.SdeWorkspaceFactory", sdePath, targetFCName);
           
            featureClassWrite(sourceFeatureClass, targetFeatureClass);

            myWatch.Stop();
            string time = myWatch.Elapsed.TotalSeconds.ToString();
            MessageBox.Show(time + " Seconds"); 
        }
        public void featureClassWrite(IFeatureClass sourceFeatureClass, IFeatureClass targetFeatureClass)
        {
            IQueryFilter queryFilter = new QueryFilterClass();
            queryFilter.WhereClause = queryClause;
            IFeatureCursor cursor = sourceFeatureClass.Search(queryFilter, true);
            IFeature sourceFeature = cursor.NextFeature();
            IFeatureClassWrite featureClassWrite = targetFeatureClass as IFeatureClassWrite;
            ISet setAdd = new SetClass(); 
            while (sourceFeature != null)
            {
                IFeature targetFeature = targetFeatureClass.CreateFeature();

                //如果是线或面要素类需要执行下Simplify,这里用的点要素类,不做验证了
                targetFeature.Shape = sourceFeature.ShapeCopy;

                for (int i = 0; i < sourceFeature.Fields.FieldCount; i++)
                {
                    IField field = sourceFeature.Fields.get_Field(i);
                    if (field.Type != esriFieldType.esriFieldTypeOID && field.Type != esriFieldType.esriFieldTypeGeometry && field.Type != esriFieldType.esriFieldTypeGlobalID && field.Type != esriFieldType.esriFieldTypeGUID)
                    {
                        string fieldName = field.Name;
                        int index = targetFeature.Fields.FindField(fieldName);
                        if (index > -1 && fieldName != "Shape_Length" && fieldName != "Shape_Area")
                            targetFeature.set_Value(index, sourceFeature.get_Value(i));
                    }

                }
                //setAdd.Add(targetFeature);
                featureClassWrite.WriteFeature(targetFeature);
                sourceFeature = cursor.NextFeature();

            }
            //featureClassWrite.WriteFeatures(setAdd);//与WriteFeature没啥效率上的区别
            ComReleaser.ReleaseCOMObject(cursor);

            IFeatureClassManage targetFeatureClassManage = targetFeatureClass as IFeatureClassManage;
            targetFeatureClassManage.UpdateExtent();
        }
    }
}

 

 

 

 

 

你可能感兴趣的:(ArcEngine)