
General Notice: BOB is going to retire...please see details here. General Notice: No events within the next 45 days. |
Author |
Message |
raoa_di Forum Member


Joined: 13 Jan 2005
              Posts: 12

|
Posted: Fri Feb 25, 2005 6:05 am Post subject: [DI] Parallel Execution or Sequential Execution |
|
|
Hi,
I want to excute a job where in I am getting all the data from DB2 to oracle. It is a simple one to one mapping, where both source and target structure are same.
There is no dependency between these tables. So I have the option to either run the job parallely or in a sequential way.
Secondly, I have millions and million of records to be loaded. Which option would be better? Should I load the tables, by keeping the dataflows in parallel or should I keep them in a sequential way?
Note: There are 150 tables and there is no dependency between tables.
Please let me know the answere as soon as possible.
Thanks and Regards
Abhinav Rao |
|
Back to top |
|
 |
darius Senior Member


Joined: 21 Dec 2004
              Posts: 44 Location: France

|
Posted: Fri Feb 25, 2005 10:00 am Post subject: Re: [DI] Parallel Execution or Sequential Execution |
|
|
What kind o server do you have ? bi processor ?
I have never played with it, but it seems that you can customize the bulk loader option in dataflows ...
If sources and targets are stricly the same, avoid QUERY .... |
|
Back to top |
|
 |
Werner Daehn Forum Devotee


Joined: 17 Dec 2004
             
*6 Posts: 10591 Location: Germany

|
Posted: Fri Feb 25, 2005 2:50 pm Post subject: Re: [DI] Parallel Execution or Sequential Execution |
|
|
You can give it a try a load all in parallel. In the \DI\bin\DSConfig.txt is a parameter called MAX_NO_OF_PROCESSES=8 so you will never find more than 7 DF actually executed simultaniously. This is a convinient way to play around.
What will be faster? It depends on soo much. Give it a try, is the best answer to it.
When executing sequentially you tend to have more sequential disk access which is times faster than random access.
When you execute in parallel, all resource tend to be used better, not one process is waiting for something leaving the other resource idle another process could use. Very often you will find that the network will be the bottleneck.
DB2s performance accross one session is not very good, so multiple sessions - either via partitioning or by parallel execution - will be better.
my two cents
Werner _________________ Werner Daehn
ex Product Manager Data Services at SAP SE
www.rtdi.io |
|
Back to top |
|
 |
|
|