Discussions
Categories
Groups
Community Home
Categories
INTERNAL ENABLEMENT
POPULAR
PUBLIC CLOUD
PRIVATE CLOUD
Quick Links
MY LINKS
HELPFUL TIPS
Back to website
Home
Intelligence (Analytics)
Data object for logs
VSafonov
<p>I have a data object that collect log information from some DB tables. I would like to append data to it. Obviously I need to run my select query with some date parameter so it doesn't bring in everything every time I run it and I don't miss any records in case of scheduling issues. I could do it by creating some "control" table in the DB and use it to store the max timestamp for the next run. But I would like to have this functionality on the BIRT side. </p>
<p>Any suggestions?</p>
<p> </p>
<p> </p>
Find more posts tagged with
Comments
kclark
<p>There might be an API for this, but it might be easier to schedule the data object. From what it sounds like, you could schedule the data object to run every n minutes and inside of your query have it be greater than or equal to *your start time*. In the meantime, I'll see if there isn't some documentation I can find that allows you to append a data object directly.</p>
VSafonov
<p>I can not rely 100% on the scheduling. It might fail, it might get delayed by queuing issues. So having a hard coded interval might miss some records. Would be much easier to have some sort of "control" data object that has 1 record with a max(log_timestamp) of sorts to read it in as a parameter. Can't figure out how to create one that is based on a data object - i.e. on the data that already has been processed. this way no matter what the interval is it should pick up everything that is new, then we just refresh the control data and the circle continues. </p>
VSafonov
<p><span style="color:rgb(0,0,0);font-family:helvetica, arial, sans-serif;">Kristopher, </span></p>
<p><span style="color:rgb(0,0,0);font-family:helvetica, arial, sans-serif;">Any luck finding a path to solution?</span></p>