You're not going crazy, it's simply a Google Maps Timeline bug

The Google Maps Timeline allows you to see the places you visited and the routes you took to get there. It is private and only you can see it. However, there is a problem with the Timeline according to several posts on Google's support page for Maps (via Android Police) from users. For some unknown reason, some of the location data isn't showing up on the Timeline leaving gaps on the graph. Some report that all data on the timeline dating back before early May 2020 has disappeared while some are seeing months to years of data go missing.

Actually, the data might be missing from the Timeline, but it still is connected with your Google account. Google has confirmed this to Android Police; the problem is that for some reason, it is not visible for you to see on your Timeline. A Google spokesman said, "We identified a bug where some users aren’t able to view some of their Location History data in Timeline. No Location History data was actually lost and Location History was and is still being recorded for users who have this feature on. We’re in the process of restoring visibility of Location History data in Timeline for all users, which we expect to be completed in the next couple of days."

One Google Maps user reported that he tapped on one of the dates in the Timeline that was missing data and it still showed the blue route map showing where he was that day and how he got there. Google said in the above statement that it is rolling out a fix that it expects to have disseminated to everyone over the next day or two. If you use your Google Maps Timeline to keep track of the distances you've traveled (you might run a business that charges per mile traveled), keep your eyes peeled for the update.