IDOC Perfromance

sjain · Posted: Tue Feb 07, 2012 10:14 am Post subject: IDOC Perfromance

Hello,

Can somebody guide with how performance can be optimized in case of loading huge data via IDOCs.

sjain · Posted: Tue Feb 07, 2012 12:03 pm Post subject: Re: IDOC Perfromance

IF this is one way of enhancing performance:-
How can i set the number of loaders in case of IDOCS to get the max perfromance?

Werner Daehn · Posted: Wed Feb 08, 2012 7:00 am Post subject: Re: IDOC Perfromance

First step is always to figure out what the bottleneck actually is. You can optimize the loader as much as you like, if it never was the bottleneck but something else, it won't change anything.

For IDOCs in particular, you can set parallel processing in WebAdmin, the RFC Client for IDOCs. Can't remember the details.
And I would check if the the al_engine is busy, e.g. with unnesting the data.

sjain · Posted: Wed Feb 08, 2012 1:24 pm Post subject: Re: IDOC Perfromance

SET RFC client for Idocs is required when we are using IDOC as source not target. Pls correct if my understanding is incorrect.

I will try that way too.

We are pushing data to SAP but SAP is not accepting it in batch size we specify until it finish it process of processing IDOCS.

RFC connections in SAP is might be the reason of bottle neck.

I didnt get any option in IDOC loader to set multiple loaders option as in realtion table loader we genreally get.

Pls suggest

Werner Daehn · Posted: Fri Feb 10, 2012 8:34 am Post subject: Re: IDOC Perfromance

Sorry, I was reading your "loading huge data via IDOCs" as loading a database with IDOCs as source. You are absolutely right, of course.

One IDOC is one change message. You need to balance how many IDOCs you create and how much data each contains. Let's take MATMAS for example, it has the root data structure E1MARA.

I can setup the nesting using the from clauses to have one IDOC with all 10000 materials. Swapping the from clauses, I can have 10000 IDOCs with one material each.
Best will be a mixture of the two, 100 IDOCs with 100 materials each.

In order to go into parallel processing, as there is no number of loader setting on IDOCs, we have to do that manually, either by running the same dataflow multiple times in parallel and using a parameter we read different data. Or we use a Case transform (condition based on a column we created upstream using the mapping mod(gen_row_num(), 3) returning the values 0,1,2 in sequential order) to split the data uniformly into three identical IDOC loaders.

sjain · Posted: Tue Feb 14, 2012 5:13 am Post subject: Re: IDOC Perfromance

Thanks for your suggestion. unfortunaltly I cant apply desgin changes because job is in CIT.
*****
This is regarding the load og Business Partner which mave multiple address, telephone numbers, fax numbers email addresses and web sites.

With the help of basis we have found some settings in SAP, like running the Job in background, switichng off RFC servers, Disable DQM -de duplication.

But THis lead us to create IDOC with status 64, we manually need to make them status 53 via running report.
****
However this gives incredible enhancement in performance now we are able to load around 75 thousands records in 45 seconds.

pratipsen · Forum Member Joined: 29 Jun 2012 Posts: 2

Hi sjain,
Can you help me provide the details - as to how your configured the stuff at the SAP end? I know BODS but am facing similar problem with processing iDocs (00s and 000s) from SAP system which is taking h u g e amount of time.

Your detailed suggestions will be of great help to me. Incase you have a word doc with details (like you mention like running the Job in background, switichng off RFC servers, Disable DQM -de duplication in a cryptic fashion & I do not have much idea on SAP) you can email the same to me.

Thanks & waiting for your support in this regard.

Sen