BOB: Business Objects Board
Not endorsed by or affiliated with SAP

Register | Login 

Follow BOB on Twitter! 
Follow BOB on Twitter! (Opens a new window)  

General Notice: No events within the next 45 days.

[DI] Table Locks on Flat File Import


 
Search this topic... | Search DI: Designer and Job Design... | Search Box
Register or Login to Post    Forum Index -> Data Integrator -> DI: Designer and Job Design  Previous TopicPrint TopicNext Topic
Author Message
fblau
Principal Member
Principal Member



Joined: 07 Feb 2005

Posts: 220



PostPosted: Sat Feb 19, 2005 2:09 pm 
Post subject: [DI] Table Locks on Flat File Import

I am trying to import a list of large flat files (about 40 files of about 6MB each).

I cannot get anything resembling decent reliable performance no matter what settings I use.

When the process is aborted, the target table retains an exclusive lock too, making it impossible to continue testing until a DBA can go in and clear it.

Does anyone have any advice or suggestions about loading flat files?

Frank
Back to top
Werner Daehn
Forum Devotee
Forum Devotee



Joined: 17 Dec 2004

speaker.gif*6
Posts: 10591
Location: Germany


flag
PostPosted: Mon Feb 21, 2005 6:51 am 
Post subject: Re: [DI] Table Locks on Flat File Import

Isolate the problem. e.g. put a MAP_Transform right in front of the target table and set all to "discard". Thus you will read the entire files but nor load any record. Using this you can tune the reading and transformation part. Later, deal with the loading performance...
_________________
Werner Daehn
Product Manager Data Services
SAP AG
�The opinions expressed in this post are my own views and not those of SAP.�
Back to top
dnewton
Forum Addict
Forum Addict



Joined: 30 Jan 2004
ASUG Icon
Posts: 3458
Location: Massachusetts, USA


flag
PostPosted: Tue Feb 22, 2005 6:31 pm 
Post subject: Re: [DI] Table Locks on Flat File Import

Are the files on a (remote) file server someplace? Would it be better to have a batch job which first copies them locally to the DI server?
Back to top
amirskumar
Forum Member
Forum Member



Joined: 09 Feb 2005

Posts: 18


flag
PostPosted: Fri Mar 18, 2005 7:31 am 
Post subject: Re: [DI] Table Locks on Flat File Import

hi dnewton,

1st i want to know that ur file is txt or xls or anyother flat file.
If it is xls-> surely it will give the errors(if it is veery large file).
b'coz xls always give the problem...


with smiles icon_smile.gif
kumar

_________________
With smile...
Kumar
"Practice makes a man's perfect"
Back to top
TRS-80
Principal Member
Principal Member



Joined: 29 Mar 2005

Posts: 132
Location: Brussels, Belgium


flag
PostPosted: Wed Mar 30, 2005 6:51 am 
Post subject: Re: [DI] Table Locks on Flat File Import

I never had such problem using flat files but you should think about some of the following things :

Tune your commit interval (the number of rows that will be imported before executing a commit).
Be sure to use Bulk Load only if it's applicable (bulk loading can be very dangerous used in unproper way becaus it's performed at one of the database lowest levels).
Do not forget that your network can go wrong : If your files (source) are not stored on the job server (like client side or on a share mapped to your job server) you can have some network slowness because the content of the file will go across netwok from file location to job server then again from job server to database server (etc.)


Hope it can help ...
Back to top
Display posts from previous:   
Register or Login to Post    Forum Index -> Data Integrator -> DI: Designer and Job Design  Previous TopicPrint TopicNext Topic
Page 1 of 1 All times are GMT - 5 Hours
 
Jump to:  

Index | About | FAQ | RAG | Privacy | Search |  Register |  Login 

Get community updates via Twitter:

Not endorsed by or affiliated with SAP
Powered by phpBB © phpBB Group
Generated in 0.0281 seconds using 17 queries. (SQL 0.0032 Parse 0.0009 Other 0.0240)
CCBot/2.0 (https://commoncrawl.org/faq/)
Hosted by ForumTopics.com | Terms of Service
phpBB Customizations by the phpBBDoctor.com
Shameless plug for MomentsOfLight.com Moments of Light Logo