Yup, you'll want to replace your starter relay. They can get stuck on, or simply fail to activate when you try to start your bike.
Make a note of the specs (voltage, amperage) and get something up to the job.
CPC/Farnell sell some good quality Japanese replacements if you're that way inclined. Else, just a replacement from your LBS or auto-parts store will do.
It'll look something like this (depending on the model):