It has been proposed that representations emerge from a single memory system organized along a continuum of specificity. This continuum is assumed to reflect a scale between the simulation of overlapping and specific features of the traces, which depends on trace distinctiveness. More specifically, higher trace distinctiveness facilitates the simulation of trace-specific features, which increase the discriminability of traces and lead to the emergence of a more specific representation. In two experiments, participants were asked to identify match (low task discrimination demand) or mismatch (high task discrimination demand) associations between actions and characters that were visually either highly or lowly distinctive. The results of Experiment 1 show that in the high-distinctiveness context, performance was better when identifying a mismatch rather than a match, while the opposite was true in the low-distinctiveness context. The results of Experiment 2 show that using a dynamic visual noise to interfere with the participants’ ability to simulate the features of the characters also reduced the benefit of the high-distinctiveness context for the mismatch trials (Experiment 2a and 2b) and increased the benefit of the low-distinctiveness context for the match trials (Experiment 2b). Taken together, these results suggest that the simulation of trace-specific features underlies the emergence of specific representations, which can be beneficial when the discrimination demand of the task is high and detrimental when this demand is low. Memory might therefore be viewed as a scale of simulation between overlapping and specific trace features.