Integrating distributed applications using Application Link Enabling (R/3 with non-R/3 systems) By Shardendu Goswami Author Profile: Shardendu Goswami received a Bachelor's in Technology from the Indian Institute of Technology in Kanpur, India. He has 4.5 years of IT industry experience, including two years of SAP experience as a technical consultant. He has been working as an R/3-ALE consultant for the past year with a "Big Six" firm in Singapore. His current client is in the consumer goods industry. Shardendu would like to thank Chris Spary, an R/3 logistics consultant, and Eng Teck Lee, an R/3 SD consultant, both of whom provided valuable editorial commentary for this article. Shardendu can be reached by email at email@example.com. Introduction This article is aimed at addressing implementation issues involved in integrating SAP R/3 with external non-R/3 applications using Application Link Enabling (ALE). Integration of distributed R/3 applications using ALE is different ball game altogether, which needs to be conceptualized differently. I intend to cover the distributed R/3 scenario separately in future articles. However, wherever necessary I will compare against or include distributed R/3 scenario in order to present a better picture. Though individual data transfer (e.g. master download/upload, one time transactional data transfer) can also be achieved using ALE, I would like to keep it out of the discussion, as it is a special condition of some of the ALE scenarios. The strength of ALE is realized only when we use it to integrate external applications with R/3 in totality. Why distributed application? There can be endless discussions on the rationale behind using distributed applications, which is beyond the scope of this article. However, it is important to list these decisive factors. 1. Though SAP can meet most core business requirements for many industries, some customers still need specialized solutions for some of their operations like Automated Warehouse, Transportation planning system, Point of sales, Process control system, to name but a few. The list becomes endless as we keep on moving across industries. 2. Some of the components of legacy systems are still better than the solution provided by SAP and hence need to be interfaced with SAP for maximum efficiency rather than being thrown away. 3. Sometimes physical separations of sites call for distribution for better efficiency. Distributed database is seldom a viable solution in such cases, for the following reasons: a) The same level of database releases is required at all systems, b) The current network is unable to support the large volume of data exchange, c) Database consistency and validations put additional burden on maintenance, d) Mirrored databases need 2-phase commit (for consistency), which is difficult to achieve in current circumstances. It becomes imperative to have a well-defined, controlled and standardized interfacing solution, which supports all such scenarios. How ALE meets the demand Contrary to the distributed database concept, ALE is based on the distributed application concept, where each system is responsible for its own validation rules and integrity. SAP, being the core business system, takes on the responsibilities of business message definition, communication and distribution. ALE is based on the asynchronous communication mechanism where each transfer is simulated as one logical unit of work (LUW) by means of transactional Remote Function Call (tRFC). In tRFC, a unique transaction id is linked to each communication of message (may consist of several attempts) for which status is maintained at both ends. If a transfer is interrupted, the whole communication process can be repeated again based on the status. At the core of ALE lies Intermediate Document (IDOC) interface. An IDOC is a well-defined structural representation of R/3 system entities. The IDOC interface has two components: Logic (data retrieval for outbound and data posting for inbound) and Structure (IDOC itself). SAP delivers a wide range of standard IDOC interfaces, but, keeping their tradition alive, SAP has also provided the facility to extend these IDOC interfaces or develop custom IDOC interfaces (both structure and logic) from scratch in order to meet even those never-heard-before requirements. But this development requires plenty of effort, both on SAP and at the external system end. All components of ALE are available to custom IDOC interface. There are sufficient tracking mechanisms available to track the status of an IDOC at any point of time. However, SAP loses control over the IDOC status once the tRFC server (tRFC component responsible for transmitting IDocs from SAP to external system) receives the IDOC successfully, for outbound communication. For inbound communication, it takes control only when the tRFC client (tRFC component responsible for transmitting IDocs from external system to SAP) sends the IDOC to the ALE communication layer. In order to track the status all along, a separate status IDOC interface needs to be designed (if not provided by SAP). Both SAP and the external system can use it to exchange the status of data validation as response to any business message received. Within SAP, an express mail can be sent to the responsible person, or a workflow can be initiated to take appropriate action. Data Distribution Model The biggest challenge in designing distributed applications is the data distribution model, which consists of all participating systems and the messages to be exchanged by the system. Unlike the "R/3 to R/3" scenario, the "R/3 to non- R/3" scenario poses difficulty in terms of data mapping, as non-R/3 systems seldom have as rich an organizational representation as R/3 has. Master data, as compared to R/3, are also usually poor, making validation a difficult task. Some examples are partner function, customer hierarchy, unit of measurement, pricing, material master, and bill of material. As far as possible, integration projects should be treated as an opportunity to enhance the functionality of the external system to make room for R/3 data. This approach enhances the maintainability of the integrated system and provides uniform representation of information across the systems. IDOC mapping Once the distribution model is designed and is decided upon by all parties involved, IDOC interfaces need to be developed for non-standard messages and need to be customized for standard messages. Each business message participating in the data distribution model is represented by a message type in SAP R/3. A message type is basically a pointer to the required fields of the IDOC, and this mapping can be achieved by means of several tools provided by SAP. Basically the following points are to be taken into account while mapping any IDOC: 1. 2. 3. 4. SAP to ISO code mapping for unit of measurement (UOM), currencies, country global parameters Conversion rules, filtering and IDOC reduction Change pointers for master data distribution For fields, which are not available in standard IDOC, extend the IDOC and write user-exit to populate corresponding fields. All the above steps are too obvious for any ALE consultant. I am still highlighting them, as careful planning at this stage avoids all those notorious changes towards the end, which may turn out to be disastrous (especially user-exits). As ALE is still evolving, for those who are working on 3.1H or below, it would be a good idea to go through OSS notes on each of the scenarios. Since ALE is more widely embraced by customers now, most of the obvious problems in different scenarios are already solved through OSS notes. It is very frustrating to learn that our scenario is not programmed, once we have configured our IDOC completely. Especially for the complex ones, such as a sales order. Some change pointers for the master data are also enhanced through OSS notes. Workflow with ALE All incoming IDocs can be processed within SAP either through a function module or through a workflow task. Though the workflow task seems to be tempting, it is advisable not to use it too much, for performance reasons. For large volume interfaces, it is better not to use workflow for IDOC processing. However, workflow has proven itself tremendously as far as error handling is concerned. For standard IDocs, SAP has already provided standard task/objects/input methods required for error handling. Just assigning the standard task to a responsible person does the trick. For custom IDocs, objects/tasks/input method needs to be defined. It is fairly simple, and online documentation illustrates the process very well. There are minor bugs in workflow configuration for earlier versions of SAP, which can be corrected by applying OSS notes. For outgoing IDocs also, workflow can be used for exception handling, but it is limited to customizing errors and IDOC syntax errors, as of 3.1H. I hope in future, SAP will enhance it to handle communication errors also. Protection against upgrades One of the major concerns about "R/3 to non-R/3" integration is how to protect our interface from future upgrades of SAP. For "R/3 to non-R/3", this issue is even more crucial as the ALE Converter (to be discussed later) needs to be upgraded along with R/3. An IDOC consists of a number of segments, and each segment has got three structures: release independent ("1"), release dependent ("2"), and documentation ("3"). As long as the ALE connector populates the release dependent segment for each IDOC, then even in future releases, the IDOC will be processed as it is. If the release independent segment name is used, SAP will assume it to be the latest release. For outgoing IDocs, the partner profile can be configured to make it release dependent. IDOC triggering mechanism It is very important to have a proper triggering mechanism for each message being sent from SAP. Most of the famous IDocs have got a standard triggering mechanism, which uses message control to transfer the IDocs based on timing chosen. It is fairly simple and we have done it already for print/fax/EDI. For those who have already worked on Electronic Data Interchange (EDI), the whole IDOC interface remains the same whereas the difference is only in terms of the external port (transactional RFC for ALE, file port for EDI). There are lots of standard IDocs that do not support standard triggering through message control. For those, either SAP has provided a standalone program, or we need to develop the program to retrieve and populate the data. For custom developed IDocs, a program needs to be written for data retrieval and transfer. It is nothing but the logic component of IDOC interface discussed earlier. ALE Converter ALE has three components, which participate in the message exchange process: SAP R/3, the ALE converter, and the External system. The role of the external system is beyond the scope of this article. It only needs to process the output of the ALE converter, which is fairly straightforward and simple. I would like to discuss in detail the ALE converter, which needs to be compatible with SAP R/3 and hence needs to be chosen carefully. There are numbers of SAP R/3 certified ALE converters available in the market. The list can be found on Complementary Software Program (CSP) section of SAP web-site. It is also possible to develop a tRFC based client and server component which can communicate with SAP R/3, and reside anywhere on the network accessible to the external system. The registering feature (tRFC server registers its address on gateway and gateway facilitates program to program communication between SAP and tRFC server) from 3.0C onwards makes it possible for the tRFC server to run as a daemon and wait for a call from SAP. This method allows the tRFC server to work even in Windows NT environment, which doesn't support a remote shell. Without going into technical details, I would like to summarize the minimum requirements of the ALE converter (whether developed or third party) 1. 2. 3. 4. 5. Able to receive and interpret all IDocs participating in message exchange Capable of transaction id management and status tracking Protected against future releases or capable of upgrade with SAP releases Able to map IDOC fields to format required by external system and vice-versa. Independent of R/3 application/gateway server availability Conclusion Though we have discussed the integration of R/3 with non-R/3 applications, it is just a part of the total integration achieved through SAP's business framework. Synchronous implementation of Business Application Programming Interface (BAPI), coupled with the powerful capabilities of ALE distribution, will be talked about even louder in future SAP campaigns. SAP's initiative towards object oriented system design and open frameworks is just what is needed in the current IT world. However, there is still a lot more to come. As time progresses, we will see more front-running applications sitting on top of SAP, though SAP will continue to serve as the Business Infrastructure Backbone.