Auto-deduplication is the quickest way to remove all duplicates within your dataset and can be performed both during or after data upload. 


During data upload


1. During Box creation, select Advanced configuration



2. Select your analysis parameters as normal


3. Toggle Automated deduplication on in the final step of the Box creation wizard and click Create




4. Upload data to the Box


5. The AI will automatically identify and delete duplicate files during analysis, retaining the most recent copy of each instance


Alternatively, this option can be turned on from Box Settings if the Box has already been created.



After data upload


1. Select the checkbox next to Select all and click Remove Duplicates to initiate the bulk deduplication process 



2. The most recent version of each duplicate file is kept in Smartbox and the remainder are deleted

  • This process usually takes only a few seconds but is dependent on the size of the dataset and number of duplicates.