You cannot "move there" except for your scout, who will need to scout it first, you conquer it then he scouts the next one beside it.
Conquering a province can get you 1 of the following:
- a good deposit so you can produce 5X more goods in the same period and for the same cost. (this is a major advantage)
- get you an expansion to unlock in your build menu
- open a new pvp tower to compete in IF you have at least 1 unit of that age in your ranks when fighting.