Parallel Processing.
SAP SPTA Parallel Processing Framework in ABAP.
With the advent of HANA and In-Memory processing, this topic might look mis-timed. But, there are many organizations, which still have no plan to move to HANA within couples of years. As they say, the show must go on and that motivated us to publish this long pending article here for those ABAPers, who still have to deal millions of rows in batch jobs and who feel the “Nights are getting too shorter to execute those batch jobs in SAP” (inspired from a friends blog).
Why parallel processing required?
Parallel processing is required mainly to improve the performance of any ABAP program. Using parallel processing framework we can significantly improve the processing time of any program, particularly where data volume is very high. The basic concept behind the parallel processing framework is to divide the large volume of data into several small work packets and process different work packets into different tasks. So each work process will be processed at the same time in parallels and it will significantly reduce time. Nowadays every distribution-related projects have a large volume of data so invoking parallel processing framework is very useful to reduce time.
Conventional Parallel Processing
We can use parallel processing framework by calling any RFC enabled function module in NEW TASK. In this way, after determining the number of work packets we can create different tasks for each work packets and process them in parallel.
Also Read: bgRFC in SAP
Why SPTA framework required?
SPTA framework is the most sophisticated and secured framework for parallel processing provided by SAP. If we want to handle multiple records and want to update/check multiple database tables in parallel, in that case, using conventional way to invoke parallel processing is difficult and there can be some ABAP memory issue. But in SPTA framework there are build in security for all the ABAP memory related issues so it is very secure. Also, SPTA framework is very easy to implement and all the parallel processing work is handled by SAP we do not need to bother how to handle it. In this way, it is also a very sophisticated framework.
SPTA Parallel Processing Framework
To invoke SPTA framework we need to call function module SPTA_PARA_PROCESS_START_2. This is a standard SAP provided function module. In this function module, we have to use three subroutines to build our own processing logic.
BEFORE_RFC_CALLBACK_FORM: This routine is called by the function module before calling the RFC function module. Here we have to build different work packets which we want to process in the RFC function module.
IN_RFC_CALLBACK_FORM: This routine is called by the function module after work packets are created. In this routine, we can use our own RFC enabled function module or custom code to process each work packets.
AFTER_RFC_CALLBACK_FORM: This routine is called at the end by the function module. After processing of all the work packets, we have to collect all the processed data.
We have mentioned server group also when calling the function module. The server group can be maintained in the RZ12 transaction. But this is BASIS activity.
In the changing parameter, we have to pass our total internal table which contains all the data. From this internal table, we will create different small work packets (i.e. Internal tables) for parallel processing.
In the call back program name, we have to pass the calling program name.
Hope you are not confusing Parallel Processing with Parallel Cursor Technique. Please read this Graphical and Statistical Analysis of Parallel Cursor Technique in LOOPs.
Now we will discuss the main three subroutines and how to call them in details.
BEFORE_RFC_CALLBACK_FORM: In this routine, we have to create small internal tables which we are referring as work packets for parallel processing in the IN RFC routine. Please refer the below screenshot.
All the parameters which are passed in this subroutine are mandatory. Here first we have to create small work packets. In the above code, it is defined like one work packet will contain 10 records. After creating one work packet I have ENCODE the data for further downstream processing. Also, we have to inform task manager that one RFC can be started by passing ‘X’ in the START_RFC field.
Also Read: What is Passive bgRFC?
IN_RFC_CALLBACK_FORM: In this routine, we have to write own processing logic to process all the data. We can call a RFC enabled function module from this routine or we can write our own logic inside this routine also. For each work packets, different tasks will be created and each task will call this routine for processing of data. Please refer below screenshot.
In the above code, I have first decoded the encoded data which is coming from BEFORE_RFC_CALLBACK_FORM routine for each work packets. Then write your own logic or call RFC enabled function module for processing. In the above example, I just sorted the random data. Then again I have encoded data for the downstream processing in AFTER_RFC_CALLBACK_FORM routine.
AFTER_RFC_CALLBACK_FORM: In this routine after processing of all the data we have to collect data. In this routine basically, we have to prepare final internal table after all the data processing. Please refer the attached screenshot.
In the above example I have decoded the data again and then displayed all the record. Here if any unit fails during processing in IN_RFC_CALLBACK_FORM no data will be available to collect because if any unit fails we must not prepare final table with less number of valid records. We can catch the failed unit by using IF_RFCSUBRC and IF_RFCMSG.
So by using this function module, we can invoke parallel processing framework in a sophisticated and secure manner.
Please download the code used in the above demonstration from Here.
Please note: We can design our own parallel processing technique without using SPTA Framework. The concept remains the same in the custom design too i.e. Records get processed into multiple different tasks and that runs parallel. So the processing time is reduced manifold.
My friend Partha (whom I referred in the first paragraph) has explained the custom Parallel Processing using a Program. Please check here:Parallel Processing Technique in SAP ABAP and it’s Advantages
Also, check SCN Blog on Two Different Types of Parallel Processing Examples
Here Partha has explained the concept and debugging of SPTA framework in a very illustrative way. Please check the Post on Parallel Processing and Debugging.
Reference:
Two Different types of Parallel Processing examples
Parallel Processing ( ABAP Wiki )
SPTA Parallel Processing Framework in ABAP
SAP ABAP 实现多线程