As well as the Forts and Holdfasts encountered throughout the map, each town also has a castle.
A visit to the Castle is your opportunity to meet the Lord and Lady who currently own the Town. They may not be in however and the hall may be empty save a couple of guards but if your lucky there will be some notable lurking around.
If the Lord is in residence it is possible to talk to him and introduce yourself. You can ask if the noble has any tasks for you to do. Should you be in the presence of one of the Kings you can offer to become a vassal or join his army as a mercenary.
Possible tasks from Nobles
Deliver message Train troops Hunt bandit Rescue hostage Collect rent