[DI] Parallel Execution or Sequential Execution

[raoa_di]

Hi,
I want to excute a job where in I am getting all the data from DB2 to oracle. It is a simple one to one mapping, where both source and target structure are same.
There is no dependency between these tables. So I have the option to either run the job parallely or in a sequential way.
Secondly, I have millions and million of records to be loaded. Which option would be better? Should I load the tables, by keeping the dataflows in parallel or should I keep them in a sequential way?
Note: There are 150 tables and there is no dependency between tables.

Please let me know the answere as soon as possible.

Thanks and Regards
Abhinav Rao

[darius]

What kind o server do you have ? bi processor ?
I have never played with it, but it seems that you can customize the bulk loader option in dataflows ...
If sources and targets are stricly the same, avoid QUERY ....

[Werner Daehn]

You can give it a try a load all in parallel. In the \DI\bin\DSConfig.txt is a parameter called MAX_NO_OF_PROCESSES=8 so you will never find more than 7 DF actually executed simultaniously. This is a convinient way to play around.

What will be faster? It depends on soo much. Give it a try, is the best answer to it.
When executing sequentially you tend to have more sequential disk access which is times faster than random access.
When you execute in parallel, all resource tend to be used better, not one process is waiting for something leaving the other resource idle another process could use. Very often you will find that the network will be the bottleneck.

DB2s performance accross one session is not very good, so multiple sessions - either via partitioning or by parallel execution - will be better.

my two cents

Werner