Well you got to think, its the PATH that its taking.
Between you, through rogers, to the Toronto server.. there is maybe.. 4-5 hops.. not a huge amount, a few rogers, then maybe from rogers to whoever the endpoint is, and that's it.
Going to a server in California.. there is MANY more hops inbetween. Jumping from possibly 2-3 or more providers or networks as it goes along its path.
its possible that one of them his having issues. I know there is a major interconnect in the US right now, which a lot of people go through to get to game servers (wow, LoL, etc)
which is causing havoc with their connections.
If you did a traceroute to the end server there.. might show you some HIGH pings in certain areas.. where the problem may emanating from.
Rogers 9865, LG 47LV5400 LED TV, Onkyo HT-R690.