BOB: Business Objects Board
Not endorsed by or affiliated with SAP

Register | Login 

Sunset Ride 
Sunset Ride 

General Notice: BOB is going to retire...please see details here. All BOB forums have been converted to "read only" as of September 1st.
General Notice: No events within the next 45 days.

Monitor Statistics


 
Search this topic... | Search DI: Performance and Tuning... | Search Box
Register or Login to Post    Forum Index -> Data Integrator -> DI: Performance and Tuning  Previous TopicPrint TopicNext Topic
Author Message
dwhitten
Forum Member
Forum Member



Joined: 02 Feb 2006

Posts: 9



PostPosted: Thu Feb 02, 2006 4:05 pm 
Post subject: Monitor Statistics

Can anyone explain the elapse time and absolute time statistics found in the monitor log. The documentation has the following explanation:

Elapsed Time Indicates the time (in seconds) since
this object received its first row of data.

Absolute time Indicates the time (in seconds) since
the execution of this data flow (including the current
object) began.

OK, so here a real example. From the trace log...

DF_CONSTITUENT_CLASS_ST
(6.5) 01-31-06 03:55:18 (23007:0001) DATAFLOW: Process to execute data flow <DF_CONSTITUENT_CLASS_ST:WF_CONSTITUENT_CLASS_ST> is started.
(6.5) 01-31-06 03:55:25 (23007:0001) DATAFLOW: Data flow <DF_CONSTITUENT_CLASS_ST> is started.
(6.5) 01-31-06 04:42:27 (23007:0001) DATAFLOW: Data flow <DF_CONSTITUENT_CLASS_ST> is completed successfully.
(6.5) 01-31-06 04:42:27 (23007:0001) DATAFLOW: Process to execute data flow <DF_CONSTITUENT_CLASS_ST> is completed.


about 47 mins right. Now the monitor log...

DF_CONSTITUENT_CLASS_ST
Path name State Row Count Elapsed time (Sec) Absolute time (Sec)
+DF_CONSTITUENT_CLASS_ST/AIView1 STOP 155261 352.820 61778.160
/DF_CONSTITUENT_CLASS_ST/Case STOP 155261 357.700 61778.320
/DF_CONSTITUENT_CLASS_ST/Case_Operation STOP 155261 357.340 61778.500
/DF_CONSTITUENT_CLASS_ST/Split STOP 155193 357.340 61778.500
/DF_CONSTITUENT_CLASS_ST/Load_Constituent_Class STOP 155193 357.350 61778.680
-DF_CONSTITUENT_CLASS_ST/Load_Constituent_Class_CONSTITUENT_CLASS STOP 155193 357.470 61778.910
/DF_CONSTITUENT_CLASS_ST/Invalid_Constituent_Id STOP 4 0.050 61779.030
-DF_CONSTITUENT_CLASS_ST/Invalid_Constituent_Id_WAREHOUSE_ERROR_LOG STOP 4 0.020 61779.150
/DF_CONSTITUENT_CLASS_ST/Invalid_Class_Value_Code STOP 64 0.070 61779.330
-DF_CONSTITUENT_CLASS_ST/Invalid_Class_Value_Code_WAREHOUSE_ERROR_LOG STOP 64 0.030 61779.420
/DF_CONSTITUENT_CLASS_ST/Load_Constituent_Class_Link STOP 155193 357.320 61779.500
-DF_CONSTITUENT_CLASS_ST/Load_Constituent_Class_Link_CONSTITUENT_CLASS_LINK STOP 155193 357.800 61779.710


So, if I understand, all the transforms received their first record with in about 6 mins (i.e. elapse time). However, I am little fuzzy about the absolute time. What I would like to know is what part of the dataflow is consuming the most time. Can this be derived from the monitor information? Thanks in advance for you response.

Dave
Back to top
Werner Daehn
Forum Devotee
Forum Devotee



Joined: 17 Dec 2004

speaker.gif*6
Posts: 10593
Location: Germany


flag
PostPosted: Fri Feb 03, 2006 1:52 am 
Post subject: Re: Monitor Statistics

If I read this right, all transforms 0.020secs to 357.800secs although their living time is 62'000secs.

That would indicate that no heavy processing is done within DI but we a re rather waiting for the source SQL to be executed.

If you look at the monitor file itself (C:\DI\logs\jobserver\monitor*.txt) you can see the time characteristics better. When was the time when the first thread received the first batch of records from the source? If I am not mistaken, it will be a quite some time. And then watch this thread and look at the next set of rows.

_________________
Werner Daehn
ex Product Manager Data Services at SAP SE

www.rtdi.io
Back to top
dwhitten
Forum Member
Forum Member



Joined: 02 Feb 2006

Posts: 9



PostPosted: Mon Feb 06, 2006 10:22 am 
Post subject: Re: Monitor Statistics

This particular dataflow does an auto-correct load. So the time must be being spent at the database (i.e. Oracle) processing the anonymous PL/SQL block. Thanks.
Back to top
Werner Daehn
Forum Devotee
Forum Devotee



Joined: 17 Dec 2004

speaker.gif*6
Posts: 10593
Location: Germany


flag
PostPosted: Mon Feb 06, 2006 10:55 am 
Post subject: Re: Monitor Statistics

True.
Can you check what the target table primary key is, if DI knows about it?

My guess is that either an index is missing or it is the mass of (unnecessary) updates performed by autocorrect load. Check the index and try TC as comparison.

_________________
Werner Daehn
ex Product Manager Data Services at SAP SE

www.rtdi.io
Back to top
Display posts from previous:   
Register or Login to Post    Forum Index -> Data Integrator -> DI: Performance and Tuning  Previous TopicPrint TopicNext Topic
Page 1 of 1 All times are GMT - 5 Hours
 
Jump to:  

Index | About | FAQ | RAG | Privacy | Search |  Register |  Login 

Not endorsed by or affiliated with SAP
Powered by phpBB © phpBB Group
Generated in 0.0277 seconds using 17 queries. (SQL 0.0021 Parse 0.0009 Other 0.0247)
CCBot/2.0 (https://commoncrawl.org/faq/)
Hosted by ForumTopics.com | Terms of Service
phpBB Customizations by the phpBBDoctor.com
Shameless plug for MomentsOfLight.com Moments of Light Logo