Concurrently processing processing files

amhatre · Senior Member Joined: 07 Feb 2012 Posts: 33

Hi DS experts,

I have two different scenario and would like to understand if there are any DS configuration to do simultaneous processing of files and WebService requests, or will the DS handle it automatically? Are there any limitations in DS with concurrent processing?

1. I have a data flow that waits on file arrival. Upon file arrival in a folder it need to parse it and load it to database. There could be up to 20-30 files arrive at same time. Will the DS process them at same time or sequentially? What are my options to load them in parallel? All files to be loaded to same Target table.

2. In 2nd scenario, upon Webservice request want to make entry into database table.

Thanks,

Abhay

dsuser · Forum Member Joined: 12 Mar 2012 Posts: 1

Hi Abhay,

Solution to the 1st scenario:

If all the files are of same structure, you can create a file format as filename_*(Suppose file names will be like filename_1,Filename_2...etc) and load all the data at a time into target table.

Thanks,
DSUSER