As a photographer, I know the laws about photos and printed media.
If he took the photo, it is his. If he purchased photographic services and received prints, it is his. It doesn't matter who is in the image as long as he had permission to take the photograph (on private property) at the time the photograph was taken. If you were underage at that time, and you said baby pics so I am guessing that you were, then he had legal rights as your guardian to be able to take those photos. I am sorry, but there is nothing that you can do about him owning baby pictures of you. But look on the bright side, at least he cares about you and wants to cherish those memories.