How to remove the duplicates from Data Set / Union / Table?

RaviK
edited February 11, 2022 in Analytics #1
<p>Hi Team,</p>
<p> </p>
<p>Good Day....!!!</p>
<p> </p>
<p>I have a specific case to remove(not hide) the data from the Data set / Union / Table in my iHub report. Can anyone suggest how to implement this?  </p>
<p> </p>
<p>Business case:</p>
<ul><li>I have 3 Data sets, from which i have created 2 joins</li>
<li>in one of the join i have duplicate values (not because of the join condition)</li>
<li>Union created by using 2 joins. Union returns with both joins with duplicate values. Here Union is behaving like Union All. As per the product behaviors Union will return Union ALL.</li>
</ul><p>So is there any way to remove the duplicates from Data Set / Join / Union / Table level?</p>
<p> </p>
<p> </p>
<p>Thanks,</p>
<p>- Ravi </p>

Comments

  • <p>Ravi,</p>
    <p>Is this the same design from this post - <a data-ipb='nomediaparse' href='http://developer.actuate.com/community/forum/index.php?/topic/40086-dynamic-filters-on-data-set/?p=146991?'>http://developer.actuate.com/community/forum/index.php?/topic/40086-dynamic-filters-on-data-set/?p=146991?</a></p&gt;
    <p> </p>
    <p>If yes, I think you have a separate issue with support. If it is the same design, I suspect that the issue is with your design because based on your explanation each of the data sets have different filters so there is no possibility of duplicate rows. </p>
  • <blockquote class="ipsBlockquote" data-author="mblock" data-cid="146992" data-time="1488821464">
    <div>
    <p>Ravi,</p>
    <p>Is this the same design from this post - <a data-ipb='nomediaparse' href='http://developer.actuate.com/community/forum/index.php?/topic/40086-dynamic-filters-on-data-set/?p=146991?'>http://developer.actuate.com/community/forum/index.php?/topic/40086-dynamic-filters-on-data-set/?p=146991?</a></p&gt;
    <p> </p>
    <p>If yes, I think you have a separate issue with support. If it is the same design, I suspect that the issue is with your design because based on your explanation each of the data sets have different filters so there is no possibility of duplicate rows. </p>
    </div>
    </blockquote>
    <p>Dear Mica,</p>
    <p> </p>
    <p>Thank you for your reply. Its a new issue. As mentioned in my previous post, My Data set1 contains a duplicate values. because i am selecting the specific columns from the Fast DB which returns duplicate values in my Data Set. So I would like to remove the duplicate values from the Data Set1. </p>
    <p> </p>
    <p>I am Okay if we remove the duplicates from Data set or Union set or Table level.</p>
    <p> </p>
    <p>FYI, My filters on Data Set will not remove any duplicates.it just get the data from the .data / Fast DB which matches the specific condition.</p>
    <p> </p>
    <p>Thanks,</p>
    <p>- Ravi</p>
  • micajblock
    edited March 6, 2017 #4
    <p>BTW, can you reproduce with a single data set? Otherwise I suspect it might be an issue with your design.</p>
    <p> </p>
    <p>P.S. If you opened a case with support why are you not working with them?</p>
  • <blockquote class="ipsBlockquote" data-author="mblock" data-cid="147002" data-time="1488852152">
    <p>BTW, can you reproduce with a single data set? Otherwise I suspect it might be an issue with your design.<br>
     <br>
    P.S. If you opened a case with support why are you not working with them?</p>
    </blockquote>
    <p>Dear Mica,<br>
     <br>
    I have replicated the issue with Classic Models.data file. Below are the step which i have followed.</p>
    <ul class="bbc"><li>Created 2 data sets with same .data file - Claasic Models.data</li>
    <li>Data Set 1 (Data Set_hold) filtered with Status = Hold. In the preview of the data set, displayed only 4 records with out any duplicates - NO ISSUE IN THIS DATA SET</li>
    <li>Data Set2 (Data Set_Shipped) filtered with Status = Shipped. In the preview of the data set,  displayed  303 records with duplicates.</li>
    <li>Union of both sets displayed 307 records with duplicates.</li>
    </ul><p>Please find the attached report design for your reference.  </p>
    <p> </p>
    <p>Thanks,</p>
    <p>- Ravi</p>
  • <p>The design is empty. In any case how are there duplicates? 4 + 303 = 307.</p>
    <p> </p>
    <p>Please attach a non-empty design.</p>
  • <blockquote class="ipsBlockquote" data-author="mblock" data-cid="147007" data-time="1488895397">
    <div>
    <p>The design is empty. In any case how are there duplicates? 4 + 303 = 307.</p>
    <p> </p>
    <p>Please attach a non-empty design.</p>
    </div>
    </blockquote>
    <p>Hi Mica,</p>
    <p> </p>
    <p>Sorry for the wrong upload in the previous post. Please find the attached updated report.</p>
    <p> </p>
    <p> </p>
    <p>Thanks,</p>
    <p>- Ravi</p>
  • <p>These are NOT duplicate values. Due to the fact that you are filtering on Status which is from the Orders data set you will get one row per order. This is as expected. Now that we have that clear I need more detail on your requirements in order to provide a solution.</p>
  • <blockquote class="ipsBlockquote" data-author="mblock" data-cid="147012" data-time="1488899874">
    <div>
    <p>These are NOT duplicate values. Due to the fact that you are filtering on Status which is from the Orders data set you will get one row per order. This is as expected. Now that we have that clear I need more detail on your requirements in order to provide a solution.</p>
    </div>
    </blockquote>
    <p>Yes Mica, I agreed with you. But the from the result point of view, I have selected limited numbers of columns and the results looks like duplicate data.  As per my business case i need to show distinct set in the results. Just like a attached excel file.</p>
    <p> </p>
    <p>Please check and suggest me how to achieve this.</p>
    <p> </p>
    <p>Thanks,</p>
    <p>- Ravi</p>
  • micajblock
    edited March 7, 2017 #10
    <p>Simply group the report on the key value. Mover everything to the group header and delete the detail row. See attached.</p>
  • <blockquote class="ipsBlockquote" data-author="mblock" data-cid="147015" data-time="1488906782">
    <div>
    <p>Simply group the report on the key value. Mover everything to the group header and delete the detail row. See attached.</p>
    </div>
    </blockquote>
    <p>Yes Mica, I have implemented the same. but i am looking that to be implemented in Data Set level. is there any way to do that? FYI, There is no attachment in the previous post</p>
  • <p>There is now. I do not know of a way to do this on the data set easily. </p>
  • <blockquote class="ipsBlockquote" data-author="mblock" data-cid="147017" data-time="1488907375">
    <div>
    <p>There is now. I do not know of a way to do this on the data set easily. </p>
    </div>
    </blockquote>
    <p>Sorry Mica.. No attachment.</p>
  • <p>in previous post where it was supposed to be</p>