BOB: Business Objects Board
Not endorsed by or affiliated with SAP

Register | Login 

Follow BOB on Twitter! 
Follow BOB on Twitter! (Opens a new window)  

General Notice: No events within the next 45 days.

[DI] Parallel Execution or Sequential Execution


 
Search this topic... | Search DI: Designer and Job Design... | Search Box
Register or Login to Post    Forum Index -> Data Integrator -> DI: Designer and Job Design  Previous TopicPrint TopicNext Topic
Author Message
raoa_di
Forum Member
Forum Member



Joined: 13 Jan 2005

Posts: 12



PostPosted: Fri Feb 25, 2005 6:05 am 
Post subject: [DI] Parallel Execution or Sequential Execution

Hi,
I want to excute a job where in I am getting all the data from DB2 to oracle. It is a simple one to one mapping, where both source and target structure are same.
There is no dependency between these tables. So I have the option to either run the job parallely or in a sequential way.
Secondly, I have millions and million of records to be loaded. Which option would be better? Should I load the tables, by keeping the dataflows in parallel or should I keep them in a sequential way?
Note: There are 150 tables and there is no dependency between tables.

Please let me know the answere as soon as possible.

Thanks and Regards
Abhinav Rao
Back to top
darius
Senior Member
Senior Member



Joined: 21 Dec 2004

Posts: 44
Location: France


flag
PostPosted: Fri Feb 25, 2005 10:00 am 
Post subject: Re: [DI] Parallel Execution or Sequential Execution

What kind o server do you have ? bi processor ?
I have never played with it, but it seems that you can customize the bulk loader option in dataflows ...
If sources and targets are stricly the same, avoid QUERY ....
Back to top
Werner Daehn
Forum Devotee
Forum Devotee



Joined: 17 Dec 2004

speaker.gif*6
Posts: 10591
Location: Germany


flag
PostPosted: Fri Feb 25, 2005 2:50 pm 
Post subject: Re: [DI] Parallel Execution or Sequential Execution

You can give it a try a load all in parallel. In the \DI\bin\DSConfig.txt is a parameter called MAX_NO_OF_PROCESSES=8 so you will never find more than 7 DF actually executed simultaniously. This is a convinient way to play around.

What will be faster? It depends on soo much. Give it a try, is the best answer to it.
When executing sequentially you tend to have more sequential disk access which is times faster than random access.
When you execute in parallel, all resource tend to be used better, not one process is waiting for something leaving the other resource idle another process could use. Very often you will find that the network will be the bottleneck.

DB2s performance accross one session is not very good, so multiple sessions - either via partitioning or by parallel execution - will be better.

my two cents

Werner

_________________
Werner Daehn
Product Manager Data Services
SAP AG
�The opinions expressed in this post are my own views and not those of SAP.�
Back to top
Display posts from previous:   
Register or Login to Post    Forum Index -> Data Integrator -> DI: Designer and Job Design  Previous TopicPrint TopicNext Topic
Page 1 of 1 All times are GMT - 5 Hours
 
Jump to:  

Index | About | FAQ | RAG | Privacy | Search |  Register |  Login 

Get community updates via Twitter:

Not endorsed by or affiliated with SAP
Powered by phpBB © phpBB Group
Generated in 0.0324 seconds using 17 queries. (SQL 0.0081 Parse 0.0008 Other 0.0235)
CCBot/2.0 (https://commoncrawl.org/faq/)
Hosted by ForumTopics.com | Terms of Service
phpBB Customizations by the phpBBDoctor.com
Shameless plug for MomentsOfLight.com Moments of Light Logo