Emotional salience and working memory (WM) load are known to affect object processing in the ventral stream. However, the combined effect, which could be either synergistic or antagonistic, remains unclear. Using functional magnetic resonance imaging and a three-factorial design, we investigated the effects of WM load and emotional salience on object processing in the ventral visual stream. Twenty-three female subjects were shown blocks of task-irrelevant and more or less degraded visual stimuli that varied in emotional valence (negative or neutral) and phase coherence rendering them more or less noisy. Superimposed on these pictures, subjects saw colored squares on which they had to perform a demanding WM task (one-back, two-back). This WM task absorbs attentional resources normally available for the perceptual analysis of visual objects and can therefore be interpreted as a manipulation of attention. We hypothesized that attenuated processing resources in the lateral occipital complex (LOC) for these task-irrelevant pictures under high WM load (Rose et al., 2005) could be regained when they were of negative emotional valence. Our results indicate that both emotional salience and WM load critically depend on a minimum level of phase coherence of the stimuli to affect LOC activation. Furthermore, the influences of emotional salience and WM load do not interact with each other in LOC. Rather, emotional salience exerts a general multiplicative gain effect while preserving the difference in activation between low and high WM load. A connectivity analysis suggests that the emotional modulation might originate in the amygdalo-hippocampal junction.