cGIS Pro differs markedly from existing ArcGIS for Desktop applications in that it is built with a multithreaded architecture designed to leverage modern CPUs/GPUs with multiple execution cores. For the add-in developer extending ArcGIS Pro, this means an altered programming model and the need to familiarize yourself with a few new concepts that may appear puzzling at first. As with anything new, working with these patterns will gradually become easier, and the benefits of multithreading will become increasingly clear.
Challenges for the Multithreading Programmer
The following four key differences distinguish any multithreaded application—including ArcGIS Pro—from a classic single threaded application:
-
To ensure a responsive user experience, the graphical user interface (GUI) thread must be able to take input from the user and produce graphical output smoothly and without interruption. This means that the execution of coded actions must be performed asynchronously on separate worker threads; the GUI thread should never perform work or blocking waits of any kind. This is in contrast to the existing ArcGIS for Desktop applications where most work is performed directly on a single GUI thread.
-
While work is executing on background threads, users must be presented with a logically consistent and informative user interface. Commands, tools, and various other parts of the user interface should be enabled or disabled appropriately based on what operations are executing, and appropriate feedback should be provided. If a long running operation is logically cancellable, an option to cancel should be offered.
-
Conflicting operations should not be executed simultaneously and should always be performed in an appropriate logical sequence. For example, operations on a map cannot be executed while the project that contains the map is still in the process of loading, and a selected set of features cannot be deleted until the selection itself has been fully computed. Most operations initiated through user interaction are logically order dependent and should be executed serially.
-
Care must be taken to ensure that access to volatile state—that is, access to non-constant variables within the program—is properly synchronized when such state is shared between threads. For example, if a collection object is shared between a worker thread and the GUI thread, both threads need to coordinate access to the collection so that one thread isn’t reading items from the collection while the other is simultaneously adding or removing items. This kind of protective coding is common to all kinds of multithreaded programing and is normally accomplished using a lock. In an application where multiple independent parties can extend the application behavior, coordinating operations can become unworkably complex without a common framework to manage how components work together.
A full treatment of multithreaded programming is beyond the scope of this document, but the following information will cover the most common patterns along with how Esri’s APIs and threading model should be used to tackle each of the previously listed challenges.
The ArcGIS Pro internal threading model
Esri engineers have placed a high priority on making ArcGIS Pro as easy to program against as possible in the new multithreaded architecture. To this end, ArcGIS Pro incorporates the latest asynchronous language features from Microsoft along with new application-specific threading infrastructure tailored to reduce coding complexity.
In most cases, add-in developers should only need to contend with two threads: the user interface thread, and a single specialized worker thread provided by the application. Internally, ArcGIS Pro uses a large number of threads for purposes including rasterization, graphics rendering, data loading, and select geoprocessing algorithms that leverage parallelism to speed computation. To keep all of these activities running smoothly and without conflicts requires a considerable amount of coordination and associated complexity; for this reason, these threads are entirely internal and isolated from developers within the implementation of the public SDK. When a method in the public API is called, the internal implementation may—when applicable—split the operation up and delegate fragments to one or more of these specialized internal threads, or queue operations that will ultimately be executed within an external process or web service.
Tasks and the task asynchronous pattern
Methods within ArcGIS Pro SDK fall into three categories:
-
Asynchronous methods that can be called on any thread. Methods of this type are named using an Async suffix and usually return Tasks. In some cases, both a synchronous and an asynchronous version of a method may be provided.
-
Synchronous methods that should be called on the worker thread only. Methods of this type are annotated within the API reference, and a code tip will appear when hovering over the method.
-
Synchronous methods that should be called on the GUI thread only. These types of methods are usually associated with WPF.
If a method on a particular object is called on the wrong thread, the call will generate an ArcGIS.Core.CalledOnWrongThreadException exception. If unsure about a particular case, you can refer to the SDK component help or Microsoft provided help to determine whether a particular method or property has a restriction.
Within the SDK—particularly within the ArcGIS.Core namespace—worker thread bound methods and properties tend to be very fine grained. To reduce the overhead associated with scheduling and thread context switches, these methods are synchronous and must be coded using tasks.
Microsoft’s .NET Task Parallel Library TPL and the associated programming pattern known as the Task Asynchronous Pattern TAP simplify the authoring of asynchronous code within a multithreaded application. The Task class is used to represent an operation executed asynchronously.
In the following example, the PrintReportAsync method is invoked and immediately returns a Task object to the caller. Meanwhile, the printing function continues to run in the background on another thread.
private void Button_Click(object sender, RoutedEventArgs e) { Task t = PrintReportAsync("HP1"); // Wait until the task is done. t.Wait(); MessageBox.Show("Printed report is ready!"); }
The author of the example wants to show a message when the printing is complete and uses the Wait method on the returned Task object to suspend the calling thread until the task is done. This approach has two major problems: First, since the calling thread cannot do anything else while it is waiting, it’s actually less efficient than simply calling a synchronous version of the print function. Second, since the calling thread is a GUI thread in this case, the user interface will freeze. A suspended thread obviously cannot process user input, render graphical elements, or do anything at all for that matter. For these reasons, you should never use the Wait method on a GUI thread.
Luckily, .NET introduced the language features async and await. The async modifier marks the method so that the compiler knows that the method is asynchronous and will be using the await operator. The await operator is most helpful, as this is used to call methods asynchronously and afterward, force the calling thread to automatically return to the next line and continue execution once the asynchronous operation has completed. The calling thread—normally the GUI thread—is not blocked and is free to take other actions while the Task on the worker thread is still running.
Note that the author now accomplishes the original goal with very little change, but doesn’t hang the user interface.
private async void Button_Click(object sender, RoutedEventArgs e) { Task t = PrintReportAsync("HP1"); // Wait (without blocking) until the task is done. await t; // Return here when task is done. MessageBox.Show("Printed report is ready!"); }
Using Run
When an asynchronous function is unavailable, you can easily write your own wrapper functions that internally execute one or more synchronous methods. The following sample uses the static Run method to queue the execution of the function WorkFunc to a random thread in the Task thread pool. Note that the click method immediately returns to the caller, while the WorkFunc continues to execute on the worker thread.
private void Button_Click(object sender, RoutedEventArgs e) { Task t = Task.Run((Action)WorkFunc); } private void WorkFunc() { // Do Work }
Instead of using a separate function, an anonymous function—called a lambda—can be employed. Using lambdas keeps the worker code within the same function and lets you use arguments and local variables within the lambda as if they were part of the containing function.
private void Button_Click(object sender, RoutedEventArgs e) { int steps = GetSteps(); Task t = Task.Run(() => { // I can use the variable “steps” here even though I'm in a // different function running on a different thread! // Do work }); }
Tasks can also be parameterized to return a particular type, as the result of whatever the lambda computes.
Task<double> t = Task.Run<double>(()=> { double result; // Compute floating point result here... return result; });
The await operator can also be used in-line to obtain the result of the asynchronous function, and without having to extract it from the returned Task.
private async void Button_Click(object sender, RoutedEventArgs e) { double computedValue = await Task.Run<double>(()=> { double result = 42.0; // Compute floating point result here... return result; }); // Execution automatically resumes here when the Task above completes! MessageBox.Show(String.Format("Result was {0}", computedValue.ToString())); }
There is a small overhead associated with await, so it’s always more efficient to call multiple synchronous methods within your own lambda than to call many asynchronous functions using await. This is particularly true when coding loops, where the cost of using await through hundreds or thousands of iterations will become substantial.
Using QueuedTask
While Tasks are a regular fixture within any add-in code, Tasks need to be dispatched in ArcGIS Pro differently from traditional TAP. The framework provides a custom Task scheduler that should be used when dispatching Tasks that make calls to synchronous methods within ArcGIS Pro SDK. Rather than calling Task.Run however, add-in developers should call QueuedTask.Run instead.
Task t = QueuedTask.Run(()=> { // Call synchronous SDK methods here });
The QueuedTask class is used instead of the Task class for the following reasons:
Queuing and concurrency control
When Tasks are dispatched using Task.Run, the associated Task will execute on a random thread in the managed thread pool each time it’s called. If a subsequent call to Task.Run is called from anywhere else in the application, the new Task will start running immediately on yet another thread—potentially while the first Task is still running on the first thread. Going back to the list of challenges inherent in multithreaded code, it should be obvious that executing unorganized operations concurrently is likely to lead to crashes and corruption of application state. The queuing behavior of QueuedTask.Run ensures the proper ordering of calls and reduces the risk of conflicts. Remember that the parallelism going on within ArcGIS Pro is accomplished internally; this simplifies the public programming model and greatly reduces the likelihood of conflicts.
Affinity and state
For performance reasons, ArcGIS Pro maintains considerable state on specific threads and in many cases, uses objects that have thread affinity. Thread affinity means that an object is tied to a particular thread and should not be interacted with from any thread but the thread it has affinity with. Affinity constraints are common in operating systems and components, including database connections, windows, controls, input queues, timers, WPF Bitmaps, and COM servers. In WPF for example, calling methods on any object derived from the WPF DependencyObject class will result in an exception if the call is made from a thread the object wasn’t created on.
Threads in the managed thread pool are also incompatible with most COM components, so you should not attempt to use Task.Run with code that might execute COM components directly or indirectly.
Application integration
When Tasks are dispatched using QuededTask.Run, they are automatically integrated with various features within the application as follows:
-
The extended Progress/Cancelation framework where progress, including the programmable progress dialog, is displayed and hidden automatically and where cancellation state is properly communicated between relevant parts of the application.
-
The application busy state system where UI elements such as buttons and tools are automatically enabled and disabled when Tasks are running. Task execution can also be coordinated with critical phases such as view creation and application shutdown.
-
Queued Tasks are enlisted in the framework’s diagnostic facilities, when enabled. This lets developers monitor the sequence of running Tasks, the functions Tasks are executing, and the duration of execution. This kind of information is invaluable in debugging and performance analysis.
Blocking the Gui Thread
Care should be taken when writing the code that will be passed to the QueuedTask.Run within the lambda. Once inside the lambda, any code that triggers a popup or prompt (such as a MessageBox) or other UI window (e.g. a modal dialog) that requires an acknowledgement from the user should be avoided. Displaying any such UI from within the QueuedTask.Run will block the QueuedTask (and all other operations queued on the QueuedTask) from proceeding. The only UI that should be shown from within a QueuedTask.Run is either Progress or Cancelable Progress. Both of these run asynchronously without blocking the QueuedTask. Refer to the Progress and cancelation section below.
Acceptable cases for using Task.Run
There are cases where the use of Task.Run is acceptable—such as when executing independent background operations consisting entirely of managed code—so long as the particular managed components in use do not have thread affinity. The developer takes full responsibility for handling cancellation, displaying progress, enabling/disabling the UI appropriately, coordinating operations, and handling logical conflicts.
Locking Guidelines
Here are a few basic recommendations for using locks safely. This is not a definitive list and there are exceptions to some of these rules in very particular scenarios, but adherence will reduce the risk of problems such as GUI hangs, deadlocks, and crashes.
- Never make cross-thread blocking calls from inside a lock. This includes
Dispatcher.Invokemethods. - Never call any kind of function from inside a lock if you don’t have complete knowledge and control over what that function does. This includes publishing an event or invoking an abstract callback.
- Locks should be placed as close to the resource they protect as possible, not in a method that calls a method, that calls another, that ultimately accesses the resource; it’s too easy to miss a code path where the lock is skipped.
- Operations within a lock should be short in duration and should not depend on another lock. Never enter a lock for an extended period of time as the UI might be waiting on that lock and you’ll hang the GUI as a result. Consider fine grained locking, or chunked operations using block copies to handle longer processing times associated with a protected resource.
- Code should enter and leave a lock within the same function; i.e. never author a method that only enters or only leaves a lock, and relies on subsequent calls to yet another function to leave it.
- Locks need to be placed around both reads and writes on a protected resource.
- Use a lock guard, i.e.
lock(_lock){...}, to ensure that locks are properly exited if an exception occurs while code is executing within a locked region. - Do not enter locks or make cross thread calls from the constructors/destructors of global or static objects.
Also be careful with what you do in ObservableCollection events published from collections with an established binding lock (via BindingOperations.EnableCollectionSynchronization(collection,_lock) for instance). These events will actually be fired from within the lock (refer to recommendation 2).
Progress and cancelation
Asynchronous methods may sometimes accept a Progressor argument, an object that is used by the caller to configure the progress dialog box and cancellation settings, and to coordinate communication between the caller and callee. Asynchronous methods that are not cancelable take a Progressor class, while cancelable methods take a CancelableProgressor class.
Progressor objects follow the pattern established by Microsoft’s CancelationToken and cannot be created directly; instead, the developer must create a ProgressorSource or CancelableProgressorSource.
The “source” objects allow you to configure how the progressor will handle progress without exposing these settings to external code, which might access the progressor. The ProgressorSource object exposes the following constructors:
public ProgressorSource(Action<Progressor> callback)
public ProgressorSource(ProgressDialog progDlg) public ProgressorSource(string message, bool delayedShow = false)
The first override takes a delegate that will be called at regular intervals while the task is running. This option is appropriate when you want to provide specialized feedback during task execution.
The second override takes a separately constructed progress dialog object. If not already shown, the progressor will automatically show this progress dialog box when the task starts executing and automatically hide it when the task completes. If the dialog box is already visible, the progressor will update the contents of the dialog box while running, and it will be the developer’s duty to hide the progress dialog box when appropriate. This option is appropriate when you want to manually control progress dialog box visibility, such as when you need to keep the progress dialog box up across several separate tasks.
The third override will automatically create and show a progress dialog box when the task starts executing and hide it when the task completes. The delayedShow parameter controls whether the progress dialog box should show immediately or delay its appearance to allow quick tasks to complete and avoid appearing at all if unnecessary. If the task is expected to execute quickly, set this parameter to true. If you expect the task to take more than a second or two to complete, set delayedShow to false so that the progress dialog box appears immediately to convey a more responsive feel.
CancelableProgressors require an additional argument that specifies what the cancel message should say. The cancel message will appear as soon as the user clicks the Cancel button on the dialog box.
public CancelableProgressorSource(Action<CancelableProgressor> callback);
public CancelableProgressorSource(ProgressDialog progDlg); public CancelableProgressorSource(string message, string cancelMessage, bool delayedShow = false);
Example Method Implementation Using Cancelation
The specialized CancelableProgressor exposes a CancellationToken property that can be used to communicate cancellation. Within the method’s implementation, code running in loops should check the IsCancellationRequested property and exit the method by throwing an OperationCanceledException (which acknowledges the request for cancellation) as demonstrated below:
public Task<long> CalcFactorialAsync(int x, CancelableProgressor progressor) { return QueuedTask.Run<long>(() => { long result = 1; for (int i = 1; i < x; ++i) { if (progressor.CancellationToken.IsCancellationRequested) throw new OperationCanceledException(); result *= i; } return result; }); }
Using the Integrated Progress Dialog within Asynchronous Methods
If the Progressor has been configured to show progress, the running task can update what information is displayed on the progress dialog box using the progressor (both Progressor and CancelableProgressor support progress dialogs):
public Task<long> CalcFactorialAsync(int x, Progressor progressor) { return QueuedTask.Run<long>(() => { long result = 1; for (int i = 1; i < x; ++i) { progressor.Message = string.Format("Working on step:{0}", i); result *= i; } return result; }, progressor); }
Common complications
Constant state assumptions
Consider the following example authored by an add-in developer. This call is invoked from the GUI thread, and the intent here is to delete the specified layer from the active view’s map.
private Task DeleteSelectedLayerAsync(Layer layer)
{
return QueuedTask.Run(() => { MapView.Active.Map.RemoveLayer(layer); }); }
Though straightforward in appearance, this function will occasionally result in an exception when put into use within the application. The mistake here was to assume that the state of the system remains static across threads. Previously queued operations may be running, and these need to complete before another operation can start executing. During that time, the state of the application may change due to user interaction or the result of operations still running. In this case, the active view may have become a table before the lambda actually started executing, in which case, the map will be null resulting in an exception. The safe approach is to avoid “chaining” calls on member variables or variables passed between threads; use local variables as a snapshot of the application state when the method was called since they won’t change out from under you.
private Task DeleteSelectedLayerAsync(Layer layer)
{
// Take a “snapshot” of the map on the active view. Map m = MapView.Active.Map; return QueuedTask.Run(() => { m.RemoveLayer(layer); }); }
Programmers in a multithreaded environment should also code defensively. Consider a task that alters how a particular layer is symbolized. If such a task ends up queued behind another task that happens to remove this same layer from the map, the second operation is logically invalidated by the first. To handle this properly, the second task should be coded to display a warning, or abort the operation silently when it learns that the layer was deleted.
Thread safe data binding with WPF
By default, WPF data bound collections must be modified on the thread where the bound WPF control was created. This limitation becomes a problem when you want to fill the collection from a worker thread to produce a nice experience. For example, a search result list should be gradually filled as more matches are found, without forcing the user to wait until the whole search is complete.
To get around this limitation, WPF provides a static BindingOperations class that lets you establish an association between a lock and a collection (e.g., ObservableCollection
BindingOperations.EnableCollectionSynchronization(Items, _lockObj);
In the example above, the _lockObj member variable—of type Object—is typically instantiated when the containing class is created and will serve as the coordinating lock. Once EnableCollectionSynchronization is called, WPF will enter the specified lock whenever reading from or writing to the bound collection. As the owner of the collection, you are likewise obligated to enter the lock when reading from or writing to the collection.
ReadOnlyObservableCollection wrappers are commonly used to enforce read-only semantics on observable collection properties. To properly set up the multithreaded synchronization, you’ll need to call EnableCollectionSynchronization on the wrapper instead of the collection itself, since it’s the wrapper that WPF will actually be binding to.
internal class HelloWorld
{
private ObservableCollection<string> _items = new ObservableCollection<string>(); private ReadOnlyObservableCollection<string> _itemsRO; private Object _lockObj = new Object(); internal HelloWorld() { _itemsRO = new ReadOnlyObservableCollection<string>(_items); BindingOperations.EnableCollectionSynchronization(_itemsRO, _lockObj); } // The public property used for binding public ReadOnlyObservableCollection<string> Items { get { return _itemsRO; } } //Within the worker function below, the lock is entered before altering the collection: public void FillCollectionAsync() { QueuedTask.Run(() => { // Reads and Writes should be made from within the lock lock (_lockObj) { _items.Add( GetData() ); } }); }
“Live” objects as properties
Care should be taken when exposing objects—especially collections—as public properties if the collection is likely to change on a separate thread. If someone gets and holds such a property and later starts enumerating through it thread A, an exception may be generated if your own code modifies the collection on thread B since there is no lock to collaborate with. Handing out read-only snapshots of the collection is safer.
Invoking code on the GUI thread
There are occasionally instances where, while your code is running along on a worker thread, you encounter a situation where you need to ask for input from the user before proceeding. You should not try to present a dialog directly from the worker thread as windows have thread affinity. A window or dialog created on the worker thread will not connect to the GUI thread’s input queue and will not honor the z-order and focus policy set by the GUI thread. In general, you can execute code on the GUI thread from a worker thread using the application’s dispatcher object.
This can be done synchronously.
FrameworkApplication.Current.Dispatcher.Invoke(()=>
{
// Do something on the GUI thread System.Windows.MessageBox.Show("Ready!"); });
Or asynchronously:
FrameworkApplication.Current.Dispatcher.BeginInvoke(()=>
{
// Do something on the GUI thread System.Windows.MessageBox.Show("Ready!"); });
Developers should try to collect needed information from the user on the GUI thread before executing work so that you don’t have to use this trick. Blocking calls made between threads risk deadlocks and hold up operations running on the worker thread.
Asynchronous exception handling
Like synchronous functions, asynchronous functions can throw exceptions. This introduces an interesting problem since the caller provides the try/catch on one thread, and the exception is thrown on another. In addition, the calling frame isn’t usually still on the stack when the exception is thrown.
However, .NET allows you to use async/await with try/catch so that if an exception is thrown by the code executing within the task, you’ll be able to catch back where the asynchronous function was called. Note that the asynchronous function must return Task or Task
try
{
var result = await PrintMapAsync(); } catch (Exception e) { // handle exception. }
If an exception is thrown from the worker and you didn’t provide a try/catch around where you awaited the call, the .NET runtime will plug the exception—as an inner exception—into a UnobservedException.
Unobserved exceptions usually show up only when the exception object is collected by .NET’s garbage collection thread, nowhere near where the exception actually occurred. If you get one of these, examine the inner exception to obtain the faulting call stack. In VisualStudio’s watch window, you can use the $exception pseudo variable to examine the current exception object.
Freezable Objects
WPF defines a pattern where certain kinds of objects can be “frozen.” Once an object is frozen, changes cannot be made to the object without generating an exception. Freezing objects can improve performance in some situations, and it also lets you share the object between threads (see thread affinity). For example, if a BitmapImage is created on a worker thread, you cannot later use it on the GUI thread unless you freeze it first.
Consider a common case where databinding is used in conjunction with images that have been generated on a worker thread. The example class VM below, is exposes a property called Img:
public class VM : INotifyPropertyChanged
{
public BitmapImage Img { get { return _image; } } ... }
This property returns an instance of BitmapImage (a Freezable object) which is then databound to a button in XAML:
<Button>
<Image Source="{Binding Img}">Image> Button>
The underlying bitmap is periodically updated on the worker thread as follows; note that the bitmap will be created on the worker thread:
public Task Refresh()
{
return QueuedTask.Run(()=> { var uri = GenerateThumbnail(); Img = new BitmapImage(uri); }); }
In the process of rendering the user interface, WPF will attempt to access the bitmap property from the GUI thread… but this will result in an exception because the Bitmap is still unfrozen and thus anchored to its parent worker thread. This issue can be resolved by simply Freezing the Bitmap after updating it.
var uri = GenerateThumbnail();
Img = new BitmapImage(uri); Img.Freeze();
Note: not all classes that inherit from System.Windows.Freezable can be frozen. Use the CanFreeze property to verify.