So shifting from big front ring to small causes it? or both ways?
The only connection between front and back is your chain, so logically I reckon it's chain tension?
FWIW when setting my rear mech I checked big front small back for outer limit, and small front big back for inner limit