A few things to check: belts, loose core, deflection.
- If you used steel core belts, the steel will eventually break resulting in belts that stretch unevenly.
- A loose core will move under force and may wander in a semi-predictable way.
- If you drive the router too hard, you will see bit deflection.
Check the core first. While not powered, just rotate the router around the X and the Y axis to see how much play you have, and how much force you need to apply to get it to move. Next use a pen mount and draw squares of different sizes and starting at different positions. Verify the size, straightness and squareness of the squares. Repeat the drawing run starting from the same position and verify that the lines match.