I have run into this many times when working on my vehicles, but not on my bikes yet. When I do encounter it, I stuff a rag into the hole to prevent debris from entering it while I'm working and I take my dremel tool to it with a cut-off wheel. I don't cut all the way through, but very close. Two or three cuts and some tapping with a chisel and they normally come out. Just be cautious so you don?t damage the housing the race is in.
Good luck.