As observers we excel in decoding the emotional signals telling us that a social interaction is turning violent. The neural substrate and its modulation by personality traits remain ill understood. We performed an fMRI experiment in which participants watched videos displaying a violent conflict between two people. Observers’ attention was directed to either the aggressor or the victim. Focusing on the aggressor (vs. focusing on the victim) activated the superior temporal sulcus (STS), extra-striate body area (EBA), occipital poles and centro-medial amygdala (CMA). Stronger instantaneous connectivity occurred between these and the EBA, insula, and the red nucleus. When focusing on the victim, basolateral amygdala (BLA) activation was related to trait empathy and showed increased connectivity with the insula and red nucleus. STS activation was associated with trait aggression and increased connectivity with the hypothalamus. The findings reveal that focusing on the aggressor of a violent conflict triggers more activation in categorical (EBA) and emotion (CMA, STS) areas. This is associated with increased instantaneous connectivity among emotion areas (CMA-insula) and between categorical and emotion (EBA-STS) areas. When the focus is on the victim, personality traits (aggression/empathy) modulate activity in emotion areas (respectively STS and postcentral gyrus/ BLA), along with connectivity in the emotional diencephalon (hypothalamus) and early visual areas (occipital pole).