I think this is a common issue to almost all printers, since I saw that on almost all prints so far. It seems to be particularly visible on your print though.
In my opinion, this is ultimately a slicer issue. Whenever there are holes, the paths and directions are different, so the nozzle runs and pushes the plastic in a different way, which generates different forces on the gantry, creating these small shifts. These are not real shifts as we would talk about for step losses, my guess being that they are due to some little backlashes in the X and Y axis, which are difficult to avoid.
I made some tests a few month ago on my delta printer to try to solve this issue. On a delta printer (or at least on mine), most of the backlash is due to the ball joints, which have a bit of play. I tried to put some thick molybdenum grease on them to try filling the gaps, which actually reduced backlash quite a lot. It really helped smooth things up, but it was still possible to see some very tiny differences in layer superposition around holes. I think they are unavoidable unless someone comes up with a slicer solution that can take this backlash into account while doing the calculations or unless someone comes up with a mechanical zero backlash solution.
In your case, I think most of the problem is due to play somewhere, so try to find the source.
Also, your first layer seems to be a bit too high according to the picture you posted. You may want to come a little closer for better adhesion.