Historical exposure and sex can have greatly impact on the sensitivity to contaminants of natural population. These factors can increase uncertainty in ecological risk assessments. This study investigated the role of background population in shaping the sensitivity of males and females of the common marine Allorchestes compressa to copper (Cu). Both male and female amphipods were collected from either a Cu polluted site (Geelong) and a clean site (Cliffton Springs). Amphipods were exposed to Cu treatments of 0, 50, 100, and 250 µg/L for 10 days, and followed by a 10-day recovery period. Cu-exposed males from Geelong had a higher tolerance than those from Clifton Springs based on their feeding rates; when exposed to Cu. This can be explained by their higher base level of metallothionein (MT) and glutathione-S-transferase (GST), the key physiological responses for detoxification and defence against damage from Cu toxicity. Males showed higher tolerance to Cu than females and this pattern was similar in both populations which may be associated with a higher level of GST. Amphipods from both Geelong and Clifton Springs could not recover completely from high Cu stress in the short term, and the rate of recovery also depended on gender and prior exposure of each population to contaminants. Our results emphasise the importance of considering site specific prior exposure and sex-dependent sensitivity of invertebrate populations to contaminants in ecotoxicological risk assessments.