Minimum Version Required: 22.01
Pre-Conditions
Table Import requires the TableImport permission to use
The Import Routine must belong to the current system to be editable
Steps
Introduction
This guide provides best practices for optimizing data imports using the Table Import engine. Following these recommendations will help improve import performance and reduce processing time.
Performance Optimization Techniques
Data Volume Management
Import Only Changed Data (DELTA): The single most important optimization is to only import records that have actually changed, rather than reimporting entire datasets. This "delta" approach dramatically reduces processing time.
Key Selection and Mapping
Link on Primary ID: Whenever possible, use the table's main ID field as the linking key rather than composite multi-column links. Single-column keys provide faster lookups and simpler processing.
Minimize Column Mapping: Only include columns that need updating in your import file. If including unnecessary columns can't be avoided, leave these unmapped in the field mapping interface to prevent wasteful processing.
Processing Options
Utilize Bulk Actions: For custom tables where field audits or triggers aren't required, enable the 'Bulk' action options. This allows for much faster data processing by bypassing row-by-row operations.
System Integration Considerations
Optimize Table Triggers: If the target table has custom triggers, review and optimize these trigger scripts to ensure they execute efficiently during large imports. Inefficient triggers can dramatically slow down the import process.
Conclusion
By applying these optimization techniques to your Table Import operations, you'll significantly improve import speeds and reduce system resource consumption. For assistance with complex import scenarios, consult with your data team.