Political situation
Before coming to this country, you have to pay attention to its political situation. Georgia is a multi-party parliamentary representative democratic republic with a parliamentary representative democratic republic. Georgia's President is the ceremonial head of state, while the Prime Minister is the head of government. Executive power is exercised by the Prime Minister and the Government. Both the Government and Georgia's unicameral Parliament have legislative power.
The political situation remains unstable, and these are definitely regions to be wary of, following decades of turmoil that culminated in the 2008 conflict between Georgia and the self-proclaimed independent governments of Abkhazia and South Ossetia. Many countries warn against visiting, however experienced travelers continue to do so.
This guide on obtaining a visa and traveling in Abkhazia is a great and up-to-date resource. The only way to get to South Ossetia is to go through Russia. You will be sent back if you happen to cross the Georgian border during your wanderings around Kazbegi. This description explains what to expect during a visit.