6.22 Translator issues - is it it even fit for purpose?

jim clendon

The translator interface is simply NASTY.

 

First issue:

We have attempted to use the translator to add a ownership link between some person objects (existing) and some system objects (existing).

 

This appeared intially to work well but upon checking every system that had the link established had all properties, description, and stereotype erased.

 

Is this a known bug?  Is there a way to import asscoaitions / links without corupting records?

 

2nd Issue 

Tried to do an update of a single field across all our systems - 1791 records.  The translator works BUT it erasing every value not in the update, so after our update of 1791 records we had a system name, and 1 property with values.  Every other property (standard and custom), the description, the stereotype were blank.

 

These sort of issues are why we have been extremely reluctant to use the translator and it appears that some work had been done on the translator in 6.22 so I was hoping for some improvement. 

 

We are now going to do a full translator export of systems, append the updated values to the appropiate column and try a FULL import of every value.  In our case we have to provide a list of our system from ProVision to meet a Government imposed regulation deadline so the song 'under presure' is on an endless loop in my head right now.

 

This worked last nite.

 

3rd Issue

We can do a translator export of systems to Excel and immediately try to re-import it ands it fails (shows as tabluar rather than valid,) and give us no opportunity to recognise it, at other time the a file we want to impaort will shgow as 'invalid'.  Why is this interface so flaky.  We are having problems with both the 6.21 client and the 6.22 client. 

 

 

Jim Clendon

I've changed companies now and am working for the New Zealand Internal Affairs Department.

 

This is an update to my translator experiences.  I ended up deploying the 6.2.4 client as this included a new translator feature which would allow new assoications to be created between existing objects, without overwriting the properties of the existing objects.

 

In test with a limited number of objects this appeared to work fine, BUT when I added over a 1000 new associations between existing objects in the notebook all the properties of most of the too object were blanked out.  However I did not discover this until a week or so later.  The 1st 100 or so associations had worked correctly and my quick check using an association grid looked fine.  Luckily I had a translator export for the objects in question (technologies) so was able to re-populate them.

 

This is where it got even more interesting - I now had duplicate objects - the names were exactly the same, but you could tell which object was freshly created as it would have no properties.

 

This stumped me breifly but it turned out that objects that have dimensions in the notebook will appear in your object list with the dimension in square brackets, HOWEVER in 6.2.4 when I use the translator the dimension in square brakets is NOT present in the workbook output.  So in Excel it appears that there are duplicates.  When these were imported we got duplicates in the object inventory.

 

I have yet to fully document this and report it as a fault but just be very cautious using the translator to round trip porpertyy updates, or to add new associations, or where you have used dimensions.  From what i can tell dimensions in square brackets are included when the translator is used in 6.2.1 and 6.2.2.  I havn't used 6.2.3.

 

I am currently waiting for desktop support to install 6.2.4 on my machine and will report the translator dimension issue as a fault.

 

 

Jim Clendon

Issue 3 Update:

On the original post i noted that we were unable to import an excel file via the translator even though we have just exported it via the translator.

 

We got to the bottom of this.  It was a customer property on an object with a data type of date.  The fix was to change the datas type to text and use this to record the date.  Upon changing the customer preperty data type the import was accepted as valid.

 

This took many hours to figure out so we now avoid using any data types of date when defining custom properties.  We also noted that the custom property with a data type of date caused crystal reports to fail. 

Matt_Michael

Hi Jim,

 

I have some notes on what you're reporting.  Typically if you think you've encountered a bug, we would suggest creating a ticket with support.

 

• “On the original post i noted that we were unable to import an excel file via the translator even though we have just exported it via the translator.  We got to the bottom of this.  It was a customer property on an object with a data type of date.  The fix was to change the datas type to text and use this to record the date.  Upon changing the customer preperty data type the import was accepted as valid.”

  • Should be addressed by defect we've logged - Bug - Interface - Translator fails to import Version Date properly. We're hoping to include this in v6.3.0

• “This is where it got even more interesting - I now had duplicate objects - the names were exactly the same, but you could tell which object was freshly created as it would have no properties.  This stumped me breifly but it turned out that objects that have dimensions in the notebook will appear in your object list with the dimension in square brackets, HOWEVER in 6.2.4 when I use the translator the dimension in square brakets is NOT present in the workbook output.  So in Excel it appears that there are duplicates.  When these were imported we got duplicates in the object inventory.”

  • Should be addressed by defect a seperate defect we've logged - Translator import with dimensioned reused objects causes creation of duplicate named objects. Again, we hope to include a fix for this in v6.3.0.

 

 

Thanks,

Matt Michael

Jim Clendon

Thanks Michael.

 

I think when I reported this originally I was pulling my 3rd niter of working till about 1am in the morning trying to help some users 'quickly' make some bulk updates.  I did raise a case and got the response that this worked as expected, was not meant for round trip updates, and I should be suing the CIF web services - which at the time were not documented.

 

I personally I think the translator is an incredibly powerful tool, especially given that bulk corrections of things like mis-entered status values, stereotypes etc have to be fixed record at a time otherwise.  However it is an area where you must check and double check that what you expected to happen actually happened.  It's a bit of a trap as it is deceptively simple to use, yet can completely trash your Metadata.  When I was looking for documentation on using the translator I couldn't find anything that provided me any idea of the risks I was taking, or what it actually did used in the way we assumed it would work (as I said deceptively simple).  It's something accessible in  a way that the CIF web-services are not.

 

However there was an enhancement to the translator in 6.2.4 which claimed you could now create links between objects without overwriting the properties of the existing from and too objects.  This worked a treat with a few objects when I tested it, but  failed under bulk update conditions - which I also raised a case for - not sure of there number - I was with Telecoms at the time.

 

I have recently reported a case 22725 since joining the DIA about the issue with duplicate objects which was an issue I noticed when working at Telecoms.  As I want to use dimensions in my new role I raised the case then tired to replicate it.  It is true that the object name file name is not unique in the translator export name column, but as long as the dimension  fields are populated, you can make changes to the descriptions, run the import and all the updates take with no duplicate objects created.  I've updated the case to that affect. 

 

 

Michael_Langford

This issue with Dimensions still exists in 6.2.4.  But don't dispair, there is a simple workaround. 

 

Go to Tools>Preferences>Dimensions Tab, and select Use full dimension names (not abbreviations).

 

Export a populated data set using Translator and use this different syntax when importing back in.  This will add your dimensioned objects without duplication.  Caution:  You must Import all the tabs that were exported (don't parse the workbook.)

Paul_Gaffney

When updating object properties via Translator, ALL of an existing object's properties will be updated with the values from the input data.  If the input data contains a definition for an object and there is no value for a property the value for the existing object will be cleared by the import.  If the column for a property doesn't exist in the import data it is considered to be empty and the value of the property will be cleared by the import.  This means that you must first export the complete definition of the object with all of the current property values and then update the desired property(ies) prior to importing.

 

When adding links between existing objects, you can prevent existing property values from being overwritten in the same manner - export the full object definition and then add the desired link definitions.  An alternate solution is to import only the link definitions.  For example, if you wish to add Utilization links between existing Technology and System objects then you could import a data set containing only Utilization records.  When importing links into a populated notebook, the 'from' and 'to' objects will be obtained from the notebook if they are not found in the import data.  This requires care when entering the names of the 'from' and 'to' objects since the names must match exactly and are case sensitive but it does ensure that the properties will not be overwritten.

 

Paul Gaffney

Senior Software Engineer/Business Analyst for ProVison

Jim Clendon

Hi Paul

 

Thanks.  We discovered the "features" that wiped out our Metadata.  It would have been nice if the help documentation was as clear as your comments.  However the main issue stopping the translators working was the use of a custom property with a date property type (and had me buring the midnight oil).  I eventually fixed this by changing the field property type to 'short text'.  This was reported as a case and it is apparently fixed in 6.3.0 (although I'll conduct my own tests when it turns up). 

 

The suggestion is your 2nd paragraph to just import the link definition is a good one, it was a feature delivered in 6.2.4.  Previously it was "designed" to overwrite the data in the existing to and from objects, even though you were only adding a link between objects.  I have tested this (it worked for my test examples) and used this.  Unfortunately it has proven unreliable and still can resort to it's old behavior part way through a large import.  I raised another case which was also accepted as a bug and sent to development.  Hopefully 6.3.0 gives us a translator that in more reliable in at least that respect.

 

We still are encountering issues with the translator though - we are getting duplicated objects including objects not even the subject of a translator import - but that's another 3 cases (2 closed as they could not be replicated - 1 open - this time I sent the notebook, gallery, property, and modeling language with the duplicate records still in the notebook).   In one of the closed cases we had over a hundred instances of the same status object after a translator import of links between status and technologies. 

 

At this point in time my role involves building a notebook containing the framework and standards for the NZ all of Government Enterprise Architecture so a reliable translator is the only effective way of importing information from excel spreadsheets and the like - which is where much of the existing architecture assets are held.