I would strongly suggest that you not mess around inside the Photos Library package. Small changes, moving files for instance, can cause corruption to the database and make your information unrecoverable. It's best to stay out of there altogether.
You mention using an external drive. For Photos to work with a Library on an external drive, the drive must be formatted in either APFS format or Mac OS Extended (Journaled) format. The drive must be connected directly to the Mac by cable, not networked, clouded, NASed, etc. Additionally, the drive can not have had Time Machine on it since it was formatted. There have been so many problems with using incompatible drives that the newest macOSs won't even allow a Library on a non-Mac formatted drive to open, since there is a chance of damaging the Photos database. See this:
Move your Photos library to save space on your Mac
If this drive is in a an incompatible format, stop running Photos with it immediately! A Photos Library can sit on an incompatible drive, but running it may corrupt the database.
It's a shame you don't tell us what macOS you are using-- things can look different in different OSs.
On a Mac you can have more than one Library. When you plug in an iPhone, it connects to the Library designated as the System Library. You should check to see if the System Library is the one you are expecting it to be.
Open Photos and go to Photos' Settings>General

If "Use as System Photos Library" is grayed out, then this is the System Library. If you can click in this box, then that will change this Library to the System Library.
I'm guessing that you don't use iCloud Photos-- is that right?
You should see this:
Basic Concepts and Terms About Photos - Apple Community