What is data deduplication and how is it performed in Data Cloud?

• A. Data deduplication is the process of removing duplicates by random choice.
• B. Data deduplication relies on full table scan to drop duplicates without matching keys.
• C. Data deduplication creates duplicates to ensure redundancy.
• D. Data deduplication identifies and merges duplicate records using matching keys; use rules with confidence thresholds.

Answer: (D)

Data deduplication in Data Cloud is about identifying duplicate records that refer to the same real-world entity and merging them into a single, authoritative record. This relies on matching keys—such as customer IDs, emails, or other composite identifiers—to recognize when two records represent the same entity. Once potential duplicates are found, rules with a confidence threshold decide whether they should be merged. The threshold helps prevent incorrect merges when similarity is uncertain, and merge rules determine which field values are preserved or combined (for example, keeping the most recent, most complete, or most trusted data). This approach ensures a clean, single source of truth and avoids both arbitrary deletion and the creation of extra duplicates.

The other options miss the essential idea: removing duplicates by random choice can spin out unreliable data; scanning a table without matching keys cannot reliably identify duplicates; and creating duplicates runs contrary to the goal of deduplication, which is to reduce redundancy rather than introduce it.