Train travel in the Caucasus is a bargain. For less than $5 you can cross Azerbaijan in reasonable overnight comfort including a reserved sleeping berth and clean sheets. The wide-gauge trains give fairly good leg room and the relatively slow progress means you can get a full night's sleep even on shorter trips such as Baku-Ganja. However, if you're in a hurry it's quicker to move by road. Trains in Georgia are even slower than in Azerbaijan though have a slightly better reputation for cleanliness.
Some carriges have a samovar which, when lit, provides boiling water for passengers to make their own tea. The operation of the samovar as well as the cleanliness and orderliness of the carrage is very much dependent on the personality and integrity of the provodnik (carrage duty officer). These are the ticket collectors/ guards who are at best guardian angels, at worst mini mafiosi to their carrage's passengers. Fortunately the former is much more common.
Somehow, many people seem to have developed the strange misapprehension that train travel is dangerous in the ex-Soviet Union. The only real problem is that carrages can be unbearably hot and, ridiculously, few of the compartments have opening windows. Air-conditioning only becomes effective an hour or two after leaving Baku. Another petty annoyance is that the train lighting is rarely switched on until the train is ready to depart which can make finding your berth rather awkward at night. This does not result in panic or mass thievery, just the need for a torch.