BOB: Business Objects Board
Not endorsed by or affiliated with SAP

Register | Login 

Want to sponsor BOB? 
Want to sponsor BOB? (Opens a new window)  

General Notice: BOB is going to retire...please see details here.
General Notice: No events within the next 45 days.

UK_AddressCleanse and IPC


 
Search this topic... | Search DI: Data Quality... | Search Box
Register or Login to Post    Forum Index -> Data Integrator -> DI: Data Quality  Previous TopicPrint TopicNext Topic
Author Message
toby1kenobe
Forum Member
Forum Member



Joined: 15 Jan 2009

Posts: 3



PostPosted: Wed Apr 01, 2009 9:25 am 
Post subject: UK_AddressCleanse and IPC

Hi am using DS 3.1

When I run a job in AIX which contains one dataflow with a
1 x SOURCETABLE,
1 x UKADDRESSCLEANSE
1 x OUTPUTTABLE

This job only shows 3 processes and takes around 200 seconds to process 2000 rows using a Dataservices jobserver in AIX

I then run the same job but on a Dataservices Windows Server and then
see there are 7 processes, listing in the log what seems to be a reference to IPC? This takes around 48 seconds to prcoess 2000 rows using a Windows Dataservices jobserver


This basically will now make a job that used to run for 4 hours using DQ117 in DI117 to 7 days in Dataservices using AIX! crazy.gif

What is IPC and why is it not showing on AIX and does this effect the
performance?
Back to top
Werner Daehn
Forum Devotee
Forum Devotee



Joined: 17 Dec 2004

speaker.gif*6
Posts: 10593
Location: Germany


flag
PostPosted: Fri Apr 03, 2009 1:31 pm 
Post subject: Re: UK_AddressCleanse and IPC

IPC is the InterProcessCommunication. One of your transforms is set the "execute as seperate process" and hence an IPC reader/loader is added to pass the data from one al_engine to the other. In windows this is important to give the DQ transform a process of its own so it can use 2GB of memory itself. In Unix as DI is a 64bit application there, it does not matter.

You should monitor the CPU usuage, if one CPU is at 100% and the others are not. Assuming that is the case, try setting the dataflow property DegreeOfParallelism to the amount of CPUs you have. Just for a try.

_________________
Werner Daehn
ex Product Manager Data Services at SAP SE

www.rtdi.io
Back to top
Display posts from previous:   
Register or Login to Post    Forum Index -> Data Integrator -> DI: Data Quality  Previous TopicPrint TopicNext Topic
Page 1 of 1 All times are GMT - 5 Hours
 
Jump to:  

Index | About | FAQ | RAG | Privacy | Search |  Register |  Login 

Get community updates via Twitter:

Not endorsed by or affiliated with SAP
Powered by phpBB © phpBB Group
Generated in 0.0268 seconds using 18 queries. (SQL 0.0026 Parse 0.0008 Other 0.0234)
CCBot/2.0 (https://commoncrawl.org/faq/)
Hosted by ForumTopics.com | Terms of Service
phpBB Customizations by the phpBBDoctor.com
Shameless plug for MomentsOfLight.com Moments of Light Logo