That may not be bad, if they were small, flat, and shiny it could be plating from bearing or other hard parts. I had a similar experence with a new rearend put in a heavy truck. After a breakin oil change I didn't get any more. I swear by magnetic plugs!!
Edit: After reading the top I may be comparing apples to oranges.
Maybe change the oil and check the mag during the next ride--but I'm sure you already knew that.
Edit#2 If it stuck to the magnet its not aluminum
Good luck
Dale