What is the difference between a private investigator and a private detective?
Nothing. A private investigator and a private detective are exactly the same thing. Private detective is simply another term for "Private investigator." A private detective/private investigator, as the term implies, is some who uses their professional investigations skill in the 'private' field, meaning they are not part of a law enforcement organization.
The name of the job can change based on where you live. Some state licensing organizations such as those in New Jersey, Georgia, and Minnesota refer to the term "private detective" or "private detective agency" whereas other states such as Florida and California refer to the term "private investigator" or "private investigation agency." At the end of the day, it's just two different words for the same occupation.
Historically there has been a shift toward the term "private investigator." This is due to the connotation people have between the word detective and the police which was ultimately confusing a lot of people. In order to avoid public confusion between private detectives and police detectives many states decided to use the term "private investigator" instead.