This is quite bullshit cause the ping only goes to the other person and comes back. It doesn't touch the server.
The server is just an opponent seeking service, the actual game after challenging each other is totally peer to peer.
If it was be You > server > person, then you'd probably get 400 ping for everyone because the server is in Japan.