CFLs have a miniaturized power supply in their base. Like any electronics, it can become defective, short out and/or overheat. I am sure that the bases are designed to contain any fire. Otherwise they would not pass UL and CSA certification. Plastic stinks but that can happen with any electronics, even stuff that hasn't failed (especially new devices.) Even if the lamp overheats or some flames escape the lamp, electrical fixtures are designed to prevent the spread of fire. The heat given off by CFLs is much lower than incandescents so fire hazard is probably reduced.
Mercury is contained in the glass bulb. Mercury is only an issue if the glass itself is broken. Most consumer electronics components no longer contain significant amounts of toxic materials (though they are considered to be hazardous waste in some areas.) It is possible that melting or burning plastic and components could release small amounts of potentially harmful gases, I would not be concerned unless smoke is visible because the level will be low and will dissipate quickly. Devices that have failed should be taken out of service immediately and disposed of safely.