nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Using "Enable Incremental Update"

Birty Terrace

Hi All,
 
I am trying to implement "Enable Incremental Updates" for appending only latest data instead of whole data to object store (.data file). But whenever I select "Enable Incremental Update" and "Generate data object", it generates .data file with duplicate data.
 
So my query is, Why is this happening??

Find more posts tagged with

Comments

micajblock

You need to add a filter to the queries so it generates only new data. For example where the transaction date is greater than yesterday.

Birty Terrace

Hello mblock,
Thanks for the reply. But, where can I add filter to the queries??

micajblock

<a data-ipb='nomediaparse' href='http://developer.actuate.com/be/documentation/ihub31-dev/DAG/index.html#page/DAG/do-HowToEnableIncrementalUpdates.html#ww193749'>http://developer.actuate.com/be/documentation/ihub31-dev/DAG/index.html#page/DAG/do-HowToEnableIncrementalUpdates.html#ww193749</a>

Birty Terrace

Hi mblock,
 
I read the document earlier. But, I still have a doubt. Is this talking about "Report parameter" as filter??

micajblock

Yes. So in your query you parameterize the query so it only gets new data. When you run the data object you pass the appropriate value.

Birty Terrace

Hi mblock,
 
I added a parameter whose value is calculated at run time. But, on running a report, I get the exception 17070.
 
I have attached a snap of Report parameter configuration. Can you verify it??
 
For background info, I have a table with "ShippedDate" as one column and from "Select Value Column" in snap, I am filtering all dates greater than yesterday.

micajblock

First of all the value column needs to come from the data set (you can use the script to set the default value). Second is this the parameter on the data object or the report using the data object?

Birty Terrace

Hi mblock,
 
this parameter is a part of .datadesign file

Birty Terrace

Hi mblock,
 
Instead of creating "Report parameter", I created a filter (please find attached snap) in dataset. And this has done half of my job i.e on choosing "Enable Incremental Update" in .datadesign and "Generate data object", it generates .data file with no duplicate data.
 
Now, remaining half of problem is, when I schedule .datadesign file in dashboard to generate .data periodically, it still appends duplicate data. I think filter created above should work here as well but, unfortunately, it is not working..
 
Am I on right track??

micajblock

attached example

Birty Terrace

Hi mblock,
 
Thanks a lot for example. It worked.
In my application, I was using excel as data source and I didn't find any "SQL Select Query" in "Edit Data Set" window for inserting parameter marker. 
 
Thanks for looking into it...

micajblock

OK, so this is a completely different story. What does your Excel file look like. How does one know there is a new row? Can you post the Excel file?

Birty Terrace

Hello mblock,
 
"DeliveryDate" is one of the column by which we can identify newly added rows.
Please find attached excel document.

micajblock

Still not sure about your requirements. Do you get new Excel files every time? In any case attached is an example, that has start date parameter. First time run it with the first date, after that you run it with a start date of the new dates. I used Order Date as the key as it made more sense to me. Just a note that incremental update is really an incremental insert, as we will insert new rows but not update existing rows.

Birty Terrace

Hi mblock,
 
I receive new excel file every week.

micajblock

The use the design I sent.

pam1987

Hi mblock, Hi Birty,
 
How can I implement "Enable Incremental Update" in open source BIRT. As there is no concept of .data and .datadesign as compared to iHub. In BIRT,you can only create "rptdocument" for caching and creating "rptdocument", on daily basis, for lots of records is not feasible. So can you people make any suggestion on this?
 
Any kind of help is greatly appreciated....

micajblock

Correct, there is no Data Objects in OS BIRT. There is also no way to incrementally update an report document. You need to generate the whole document again. A report can have aggregations anywhere on a page so it makes no sense anyway.
 
Is there a specific use case you are thinking of?

pam1987

Hello mblock,
Thanks for verifying my understanding.
 
We are doing a project in which thousands of records (relating to user data) is generated on daily basis. Now, suppose, in a year the records count reaches to millions or billions. So, for having an updated rptdocument, one needs to generate this much of record daily which has fat chances of doing low performance on product side. And in my personal opinion, this is not a good architectural sign.
So, I was googling to find an optimized way to do this. And then I came to this post and thought of poking you guys for making some suggestions

.

micajblock

Why do you need to generate one report with all users? Who will look at that? Create reports with parameters so only the data a specific user needs will be generated.
 
P.S. You can always buy commercial product :-)

pam1987

Hi mblock,
For knowing the maximum capacity of rptdocument, for holding cached data, I need to generate a report with all users. Any suggestion on maximum size of rptdocument, we can generate :rolleyes: ??
Yes, at run time, we can use parameters for filtering data.
And,Yes I can buy commercial product any time

micajblock

I do not think there is a maximum size.

pam1987

Ok..Thnx