The easiest way to differentiate between these two different types of medical institutions is size and severity of your condition.
Hospitals
Clinics
in Japanese = byouin 病院 or sougou byouin 総合病院
in Japanese = kuriniku クリニック
fewer in numbers across the country
smaller, in various locations such as in front of stations
different medical departments
specialize in one field only e.g. Dermatology, Urology (urinary system), Gastroenterology
latest technology
Where should you go?
If you have an idea of what is wrong, you can select the specialist clinic that is best suited to your needs. However, most people will go to a general clinic for what’s called “Internal Medicine” (naika,内科) which acts kind of like a General Practitioner. To find a clinic near you, simply search クリニック or “clinic” in English, and your area. The majority of clinics will have an Internal Medicine department, unless otherwise stated.
If you have a mild condition, go to a clinic first. They will be able to tell you whether your condition is indeed mild or you need to consult a more specialized doctor at a larger hospital. Refrain from going to a larger hospital if you have a cold or other mild conditions — you won’t necessarily be turned down, but you will be requested to pay an extra fee for their service due to the case not being serious enough.