Why would you even make me think about this?? T-T
But anyways, I'm thinking you're pretty spot on. Testament and Anji are probably the most likely to come back, and I'm actually surprised that we haven't seen Testament yet. I think May briefly referred to him in Johnny's episode mode though. I'm not too sure about Justice, I at least think it'll be a while before we see her again. Same with Robo-Ky I do think we have a chance of getting Zappa back eventually though. From what I understood in the story mode, he hasn't completely cured his possession issue, only figured out how to prevent S-Ko from taking control of his body. So they could write him back in as an active fighter.
As for the characters that we haven't seen in any form in Xrd yet, as much as I would love to have A.B.A back, she's possibly the least relevant character that was ever in the main series, and her story in AC was pretty conclusive. That could be easily retconned though, but I'm not expecting her anytime soon. Same deal with "legacy" characters like Kliff and HOS, I would expect them to come back at around the same time as Justice.
And finally, I can't imagine that they would be removing any characters. Perhaps replacing them with different versions of them in the cases of Bedman and Jack-O', but they'll still be there in one way or another. I don't think roster size would be too much of a hurdle for Arcsys to balance. I thought USF4 was pretty decently balanced with a large roster, and Arcsys is one of the most (probably the most) competent and capable developers out there right now.
Anyways, that's just my two cents.