Similar events may reduce the likelihood of the cognitive system to accurately remember a specific memory. Similarity indeed leads to overlap between mnemonic traces, which in turn interferes with the discrimination of the traces. It is therefore important to determine how and when overlapping is detrimental to the discrimination between the traces. According to the Act-In memory model, the specificity of a memory trace is determined by the sensory overlap as well as by the number of participating sensory modalities on which overlap occurs (unimodal vs. multimodal). Increasing overlap should only be critical when the memory traces are the most difficult to discriminate from each other, which is more likely for unimodal than multimodal traces. As such, multimodal events might be more efficient than unimodal events to allow memory specificity. In two experiments, participants had to reproduce visuospatial sequences in a two-by-two matrix. The level of sensory overlap (high vs. low) and the number of components on which overlap occurs in the memory traces (unimodal vs. multimodal-discrimination) were manipulated. The results showed that memory span was lower when the visual overlap was at its highest, but more significantly when trace discrimination was unimodal (Experiment 1 and 2). Moreover, for visually richer stimuli, visual overlap was shown to be detrimental to specific memory only in a condition of visual degradation. Taken together, the results suggest that the sensory overlap is essentially critical to specific memory when it is at its highest, which is the most likely for low richness unimodal stimuli.