BOB: Business Objects Board
Not endorsed by or affiliated with SAP

Register | Login 

Follow BOB on Twitter! 
Follow BOB on Twitter! (Opens a new window)  

General Notice: Upcoming Events: PGHBOUG: Oct 19.

Data Cleanse and CPB - Bad Cleanse


 
Search this topic... | Search DI: Information Steward... | Search Box
Register or Login to Post    Forum Index -> Data Integrator -> DI: Information Steward  Previous TopicPrint TopicNext Topic
Author Message
petersjd
Principal Member
Principal Member



Joined: 20 Nov 2003

Posts: 115
Location: Lacey, Washington



PostPosted: Mon Sep 10, 2018 3:27 pm 
Post subject: Data Cleanse and CPB - Bad Cleanse

Not sure if the problem is with my Cleansing Package or my Data Cleanse configuration. Just using base Data Cleanse and a simple Custom Cleansing Package that happens to have multiword Standard Forms and Variations.

Consider the following two CP Entries:

1) Variation (ONE TWO THREE) to Standard Form (One Two)
2) Variation (THREE) to Standard Form (Three Four)

The Standardized results I get when the input is "ONE TWO THREE" is "One Two Three Four". When the input is "THREE", I get the correct results of "Three Four".

BTW, I have removed the auto-generated "Phrase Words" leaving only Standards and Variations. It would seem that the parser is still matching individual words and concatenating multiple Standard Forms based on a single word in the input.

Is there a technique to force the rule to only be applied when the full input string matches the full Variation? Either in the CPB or in the Data Cleanse?

_________________
Jim
Back to top
jlynn73
Forum Associate
Forum Associate



Joined: 27 Oct 2009

Posts: 545
Location: DesMoines Iowa


flag
PostPosted: Tue Sep 11, 2018 7:58 am 
Post subject: Re: Data Cleanse and CPB - Bad Cleanse

Welcome to the lovely world of Data Cleansing.

It looks at each word individually, so even though you define a 3 word entry and a standardized form ... it will never hit that definition. You would have to define classifications for each word and a rule that ties all 3 together. (which is less than ideal)

An easier solution is to use a search and replace on the multi-word fields, stripping the white space out of them and entering them in the Data Dictionary as all one word. Again ... less than ideal, but would work.
Back to top
petersjd
Principal Member
Principal Member



Joined: 20 Nov 2003

Posts: 115
Location: Lacey, Washington



PostPosted: Tue Sep 11, 2018 8:51 am 
Post subject: Re: Data Cleanse and CPB - Bad Cleanse

Well, I examined the rules using Advanced Mode and found that the first auto-generated rule was concatenating the primary attribute to itself 'MyString + MyString'. I said "that can't be right". So I modified the rule to just be 'MyString', and the cleansing is done properly now. But, now, it squaks about AutoGeneratedRule1 not being found, since it is now a user defined rule.

Still seems to work OK, though the behavior in Design mode is different, in that it no longer generates the base Variation (that matches the Std Form), and then once I add the value as a Variation, I can only add more using Advanced mode. I can't seem to find any decent documentation.

_________________
Jim
Back to top
jlynn73
Forum Associate
Forum Associate



Joined: 27 Oct 2009

Posts: 545
Location: DesMoines Iowa


flag
PostPosted: Wed Sep 12, 2018 7:50 am 
Post subject: Re: Data Cleanse and CPB - Bad Cleanse

If it can be done without the use of a Data Cleanse, I would find another way.

My carpal tunnel cant withstand much more rules file manipulation.
Back to top
Display posts from previous:   
Register or Login to Post    Forum Index -> Data Integrator -> DI: Information Steward  Previous TopicPrint TopicNext Topic
Page 1 of 1 All times are GMT - 5 Hours
 
Jump to:  

Index | About | FAQ | RAG | Privacy | Search |  Register |  Login 

Get community updates via Twitter:

Not endorsed by or affiliated with SAP
Powered by phpBB © phpBB Group
Generated in 0.0298 seconds using 17 queries. (SQL 0.0028 Parse 0.0009 Other 0.0261)
CCBot/2.0 (https://commoncrawl.org/faq/)
Hosted by ForumTopics.com | Terms of Service
phpBB Customizations by the phpBBDoctor.com
Shameless plug for MomentsOfLight.com Moments of Light Logo