BOB: Business Objects Board
Not endorsed by or affiliated with SAP

Register | Login 

Follow BOB on Twitter! 
Follow BOB on Twitter! (Opens a new window)  

General Notice: BOB is going to retire...please see details here.
General Notice: No events within the next 45 days.

Real Time Data Services job running slow


 
Search this topic... | Search DI: Performance and Tuning... | Search Box
Register or Login to Post    Forum Index -> Data Integrator -> DI: Performance and Tuning  Previous TopicPrint TopicNext Topic
Author Message
PeterElliott
Forum Member
Forum Member



Joined: 13 Feb 2014

Posts: 3
Location: Lancashire UK


flag
PostPosted: Mon Aug 24, 2015 9:24 am 
Post subject: Real Time Data Services job running slow

Hi,

I have a real time data services job which is doing the following
a) read and un-nest a MQ message
b) perform basic validation using lookups to a mainframe DB2 table
c) Load the validated data to another Mainframe DB2 table

There are 2 input streams to this job, the first involves users 'publishing' a single page worth of catalogue data which would involve up to 20 messages being sent. The second stream occurs if a user 'publishes' a full catalogue worth of data, usually at the end of a day. This could contain 2000+ messages and multiple catalogues could in theory be published on the same day.

We have been taking timings based on a catalogue which publishes 1000 messages and these are taking 30 mins to process.

We have played around with various setting for DoP, max instances, min instances etc without making a lot of difference to the timings. A saving of 3 mins is the best.

By monitoring the jobs has they run, we believe that the bottleneck is in reading the message queues. Has anybody any suggestions as to how we could speed this process up.

Thanks
Back to top
Display posts from previous:   
Register or Login to Post    Forum Index -> Data Integrator -> DI: Performance and Tuning  Previous TopicPrint TopicNext Topic
Page 1 of 1 All times are GMT - 5 Hours
 
Jump to:  

Index | About | FAQ | RAG | Privacy | Search |  Register |  Login 

Get community updates via Twitter:

Not endorsed by or affiliated with SAP
Powered by phpBB © phpBB Group
Generated in 0.0263 seconds using 17 queries. (SQL 0.0018 Parse 0.0008 Other 0.0237)
CCBot/2.0 (https://commoncrawl.org/faq/)
Hosted by ForumTopics.com | Terms of Service
phpBB Customizations by the phpBBDoctor.com
Shameless plug for MomentsOfLight.com Moments of Light Logo