It's not you, Google Photos is set to become worse at estimating your photo locations
Google Photos is set to become way less adept at approximate geolocation. Google says that Photos is about to stop using your account-wide Location History to determine where specific shots were taken, provided that you opted out of the camera app using your location data due to privacy concerns.
Up until now, if you don't allow the camera app to use your current location, Google Photos utilized two ways to roughly estimate where you took a certain photo. Firstly, the app would would try identifying visible landmarks in your photos, and secondly, it would use your Location History to help narrow the area.
However, with the freshly announced change, Google Photos will solely rely on narrowing your location by trying to identify landmarks in your photos. As you might imagine, this way of location-guessing is not very reliable and will very much depends on what your photos contain. The app will likely heavily rely on the same algorithms that Google Lens uses, which work great when popular landmarks are considered, but might not do just as well with obscure ones.
Once again — this change will only affect those, who have opted out of geolocating the photos they take with their camera. Upon first launch, your camera app should have asked you if you'd allow geolocation data to be attached to images.
Google says that the change has already started rolling to some and will gradually roll out to all Google Photos users in the coming months. The Google Photos app may ask you if you wish to keep or delete existing estimated photo locations.