I dont see much difference with programmable thermostat, the bill seem to be the same as to keep it same temp setting.
It's pretty difficult to get an accurate before/after comparison since there are so many external influences, but as others have said, all of the studies indicate that you will
save if you set the temperature back a few degrees overnight.
If somebody happens to leave a window or door open for a few hours you can easily burn a few extra litres of oil compensating; sometimes an external event like that is enough to screw up your comparison. You also can't go by your heating oil bill since the price of oil changes from delivery to delivery; at least it always did for me in the Ottawa area. If you have a fixed price for the season then you'd be OK, but you're better off looking at the actual consumption rather than the cost.
I've been tracking my energy usage since we moved into our current house 8 years ago. The best I've been able to do in terms of evaluating my usage is to track the litres consumed against the "heating degree days". HDD is a measure of how much you need to heat your house due to the weather, so tracking your usage relative to the HDD more or less takes the weather out of the equation, so you can meaningfully compare results from a very cold winter to a very mild winter. You can get HDD data for your region from Environment Canada
. Use the "daily data" to see the HDD value for a given day, or simply get the bulk data for the whole year for your closest city.
If you keep your oil bills, you can look at the number of litres between fill ups and the number of days. If you went 15 days between fills and burned 500l in that time, add up the HDD values for those same 15 days. If those 15 days totalled 600 HDD, then you were burning 0.83 l / HDD. Even if you don't have your oil bills filed away, your heating oil company should be able to give you that info - they do the same calculations in order to know when you're going to need a fill-up. They could probably give you the numbers for the last 12 months, so if you start setting back the temperature and tracking the usage you'd be able to see if you're doing any better.
If you track those numbers over time you should see that your litres/HDD consumption stays relatively stable. Having other oil appliances like a water heater will throw the value off a bit since you'll tend to use the same amount of oil for water heating regardless of the weather. You might have to subtract a couple of litres per day for water heating to avoid the skew - your summer oil consumption would give you a rough idea in that case.
In our house with about 6000 sq ft. of living space we were burning around 0.9 litres / HDD. We switched over to natural gas this summer so it will be interesting to see how our gas consumption compares. The NG furnace is a high efficiency model (Carrier Infinity) so we're quite a bit better off right there. The biggest problem will be that I have everything
on NG now rather than an oil/propane split. I'll have to compare my NG usage against the combined oil/propane usage of past years to see if my consumption is actually down.