Understanding PSIP Data as part of Digital OTA ATSC TV

[stampeder]

Some ATSC Tuners have been known to have specific problems with certain DTV stations, such as PC cards that reportedly have trouble with CFTO-DT CTV in Toronto. DHCers indicate that the problem was found to be likely a problem with the station's PSIP (pronounced pee'-sip) data, or the inability of the cards to work with that data. Elsewhere there has been speculation that certain HP ATSC-equipped TVs may be having trouble with PSIP data too.

Okay, if its so important that certain tuners may not work properly on certain stations, what is PSIP data? From www.psip.org:

Program and System Information Protocol (PSIP) is data that is transmitted along with a station's DTV signal that tells DTV receivers important information about the station and what is being broadcast. The most important function of PSIP is to provide a method for DTV receivers to identify a DTV station and to determine how a receiver can tune to it. PSIP identifies both the DTV channel and the associated NTSC (analog) channel. It helps maintain the current channel branding because DTV receivers will electronically associate the two channels making it easy for viewers to tune to the DTV station even if they do not know the channel number.

In addition to identifying the channel number, PSIP tells the receiver whether multiple program channels are being broadcast and, if so, how to find them. It identifies whether the programs are closed captioned, conveys V-chip information, if data is associated with the program, and much more. If broadcasters do not include properly encoded PSIP data in their DTV signals, receivers may not correctly identify and tune to the station. Therefore, it is vital that all broadcasters understand PSIP and include the data in their DTV stations signals. PSIP is a mandatory Advanced Television Systems Committee (ATSC) Standard.

Channel Remapping

For those who don't know about a PSIP capability called channel remapping, it is a feature of the ATSC standard for use during the analogue-to-digital transition that allows broadcast stations to telecast on a newly assigned DTV channel but "trick" the receiver at your home into displaying a channel number that corresponds to the original analogue channel or any other unused channel they please.

For a fictional example:

An analogue station called CKZZ on channel 8 spent 39 years in a market and has branded itself heavily on its channel number. Their DTV assignment is channel 43 so they must move all their programming over to 43.1, but now they face a big advertising job and the loss of almost 4 decades of "Channel 8" market branding. Using PSIP Channel Remapping, CKZZ-DT begins telecasting on 43.1 but sets its PSIP data to display 8.1 on home receivers. Their "Channel 8" identity is thus saved. After the transition period is done (2009 in the U.S., 2011 in Canada) and the analogue Channel 8 transmitter is permanently shut down, CKZZ-DT's owners make the strategic decision to actually broadcast on 8.1 and leave 43.1 behind for Industry Canada and the CRTC to reassign to any other TV station they choose. Effect on the consumer? The home receiver picks up the change and adapts accordingly with no consumer effort needed. Once again, their "Channel 8" identity is thus saved. Everyone is happy.

Originally Posted by Yaamon

stampeder I don't understand what you meant here.

"CKZZ-DT's owners make the strategic decision to broadcast on 8.1 and leave 43.1 behind"

Are you saying that station wil broadcast on is 8.1(Vhf frequency), or set the psip id to 8.1 but broadcast on a UHF frequency ?

This would make more sense to broadcast on uhf and set the psisp to chan id 8-1. If not what will happend to those customers that currently has only a uhf antenna?

DT stations will be allowed to reoccupy/replace their original old VHF analogue channel slots as long as they are in the high band (7 through 13) because VHF-low is being reassigned away from TV. In the example I gave, CKZZ analogue channel 8 goes dark one day, CKZZ-DT digital channel 43.1 (that remaps to 8.1) then shuts down momentarily, then comes back up genuinely on 8.1 and no longer on 43.1, which the authorities are now free to reassign to any other station that wants channel 43.1.

It is much, much more energy-efficient ($$$) to broadcast in the VHF-high band than in the UHF band to cover the same area, so this is a desirable move for some stations to make. Others will decide to just stay where they are with their new assignments.

You can check the FCC database to see which stations in the U.S. near the Canadian border have elected to go back to VHF-high and which have elected to stay in UHF. As always, the situation up here in Canada is murky at best.

This is why I will always keep an excellent VHF-high antenna around... the Channel Master 4228 UHF

Regarding the Real Time Clock features of PSIP, 99gecko writes:

I did some more reading and verified that the only way an ATSC tuner determines the local time is from the PSIP data that is coming from the channel your tuner is currently locked on, i.e. not on the local PBS station - that would require multi-tuner capability anyway. You must however tell your ATSC tuner to observe DST (locally).

from atsc.org:

The ATSC PSIP Standard (A/65A) requires broadcasters to transmit a reference for time of day with the System Time Table (STT). The STT bases its reference for time of day on Global Positioning Satellite (GPS) time, which is measured in terms of seconds and provides a reliable and predictable timebase for specification of future events. The STT even provides daylight savings time information.
Before a receiver can use the STT data to track local time of day, it needs to know the time zone specified as an offset in hours from UTC (Coordinated Universal Time) and whether or not daylight savings time is observed locally. For an over-the air digital television, this information may be entered directlyby the consumer via a unit setup function. For a cable set-top box, this information may be delivered by the cable system operator. System time data is required to be no less accurate than plus or minus one second. By locking to the broadcast station master clock, the DTV receiver will be the most accurate timepiece in the house.

now that is interesting...

The following (edited) is an excerpt taken from a presentation (to broadcast engineers I assume); source:http://www.bitrouter.com/pdf/tutorial-psip.pdf:
It is technical, but what it shows is that the local DST requirement is clearly a matter of proper programming.

User Interface Requirements for Processing System Time Table

• Two values required from the user interface:
– “What time zone are we in?”
– “Is daylight savings time observed at this location?”
• Time zone can be stored internally as number of seconds east or west of Prime Meridian.
– Usually “n” * 3600.
– Use negative number for locations west of Prime Meridian, positive number for east.
• Values should be saved in non-volatile memory.

Relevant Fields from the System Time Table

• “system_time”: Number of seconds since
12:00 AM, Jan 1, 1980. Doesn’t include leap seconds.
• “GPS_UTC_offset”: Number of leap seconds inserted since 12:00 AM, Jan 1, 1980.
• “DS_status” flag:
– Set to one when all time zones within broadcaster’s coverage area have entered daylight savings time.
– Cleared to zero when all time zones within broadcaster’s coverage area have exited daylight savings time.
• “DS_day_of_month”: If non-zero, indicates the day of the current month for which transition into or out of daylight savings time is to occur.
• “Ds_hour”: If non-zero, indicates hour for which transition into or out of daylight savings time is to occur.

Calculating the Current Time
from the System Time Table

• Obtain “system_time” field from STT.
• Subtract “GPS_UTC_offset” field from STT. The result is number of seconds since 12 AM, January 1, 1980. (In Greenwich, England).
• Add time zone adjustment value obtained from User Interface.
• Convert to date and time format. (Use “mktime()” if it is available in your C library).
• If daylight savings time is observed, check if we are in daylight savings time.
• If so, add 3600 to results from step 3, and reconvert to date/time format.

[stampeder]

PSIP Revisited: Getting it Right

September 6, 2006
TVTechnology.com

Nearly two years after the FCC mandated the use of PSIP, reports can still be heard of stations struggling to get it right. Anecdotal evidence indicates that some stations do a much better job than others, and some are apparently still not doing everything required. The FCC incorporated the entire ATSC Program and System Information Protocol for Broadcast and Cable (ATSC document A/65) into its rules in February 2005. Fortunately, ATSC has developed tools that promise to make PSIP implementation easier and more accurate.

http://www.tvtechnology.com/features...Whitaker.shtml

[stampeder]

Judging from the article and from what we DHCers have learned about PSIP over the last few years it is obvious that PSIP has been an awfully difficult system to implement, and so the ATSC authorities have had to sharpen their pencils and come up with a better approach to making it work.

The goal is now to bring in tools based on XML documents that will ease PSIP implementation and administration. I can hear the cheers from irritated OTA station staff already!

[alebowgm]

Are they not going to use PSIP remapping?

[stampeder]

Thanks dsspredator for confirming that the PSIP data problems associated with CFTO and OMNI in Toronto apparently have been fixed. If you haven't done so already, do a rescan and everything should be fine now.

I think pnear deserves a round of applause for his efforts to bring this to the attention of the right people, and also all of you who identified the problem in the first place and shared your knowledge here.

I've closed the 2 original threads about these Toronto PSIP data problems.

I hope this isn't like some 1950s b-grade sci-fi movie that finishes with "The END... or is it?!!!" [queue the weird theramin music]

EDIT: I just put alebowgm's message into this thread because he posted it just as I was locking the OMNI thread.

[pnear]

When I spoke to their engineering lead, the plan was no remapping. But will pose the question to the man responsible for their PSIP data.

[99gecko]

Originally Posted by stampeder

I think pnear deserves a round of applause for his efforts to bring this to the attention of the right people, and also all of you who identified the problem in the first place and shared your knowledge here.

pnear and others,
clap clap clap

[kenmar]

I second that. Thanks Pete. For a while there, I was considering tossing in the towel on OTA (CFTO-DT was unreachable for over 1 month!), but now I have CFTO, OMNI1 and OMNI2 for the first time ... all in HD!

[pnear]

Confirmed - they plan to stick with the channel assignments as they exist today. VCT and RF channel the same.

Pete

[Tom.F.1]

Last night my CFTO which used to be on 9-1, remapped itself back to 40-1. That was on my sanyo tuner.
But on my Evu 9200, it still works on 9-1.

I liked onmi better when it was on 47 & 69. I can't tell which is which with 44 & 66. Aren't all the other toronto channels re-mapped to match their analog numbers?

[Tom.F.1]

Nov 16, yesterday afternoon, i couldn't get CFTO at all. Don't know if they were off air, or just the psip mess up.
After re-scanning a few times i gave up. It wasn't there.
Then i tried again, just before the 6 pm news, and there it was, back on 9-1 just like it was before it was fixed. with station id but still no program info.

[99gecko]

Was going to post this in the offical DST thread, but since the remedy is out of the hands of the consumer and since it only affects OTAer's I posted it here.

The new DST is here and my on-screen guide is still messed up because of faulty PSIP data because of it. I checked the config of my STB (Samsung SIR-T451); it is set to automatically detect DST - probably firm coded for the originally DST dates. I tried turning this on and off, and it doesn't actually do anything. I did an internet search and somebody mentioned that some STB's set their date/time from the local PBS's stations PSIP, regardless of the station you are on. However I think that might have been speculation. Since I don't recall it being a problem before, I think it must because of adaption of the extended DST.

Toronto/Buffalo Channels in DST (compliant):
American: 2, 43
Canadian: 5, 9, 25, 66

Toronto/Buffalo Channels not in DST:
American: 4, 7, 29, 49
Canadian: 57

Anybody else experiencing this?

[99gecko]

I did some more reading and verified that the only way an ATSC tuner determines the local time is from the PSIP data that is coming from the channel your tuner is currently locked on, i.e. not on the local PBS station - that would require multi-tuner capability anyway. You must however tell your ATSC tuner to observe DST (locally).

from atsc.org:

The ATSC PSIP Standard (A/65A) requires broadcasters to transmit a reference for time of day with the System Time Table (STT). The STT bases its reference for time of day on Global Positioning Satellite (GPS) time, which is measured in terms of seconds and provides a reliable and predictable timebase for specification of future events. The STT even provides daylight savings time information.
Before a receiver can use the STT data to track local time of day, it needs to know the time zone specified as an offset in hours from UTC (Coordinated Universal Time) and whether or not daylight savings time is observed locally. For an over-the air digital television, this information may be entered directlyby the consumer via a unit setup function. For a cable set-top box, this information may be delivered by the cable system operator. System time data is required to be no less accurate than plus or minus one second. By locking to the broadcast station master clock, the DTV receiver will be the most accurate timepiece in the house.

now that is interesting...

The following (edited) is an excerpt taken from a presentation (to broadcast engineers I assume); source:http://www.bitrouter.com/pdf/tutorial-psip.pdf:
It is technical, but what it shows is that the local DST requirement is clearly a matter of proper programming.

User Interface Requirements for Processing System Time Table

• Two values required from the user interface:
– “What time zone are we in?”
– “Is daylight savings time observed at this location?”
• Time zone can be stored internally as number of seconds east or west of Prime Meridian.
– Usually “n” * 3600.
– Use negative number for locations west of Prime Meridian, positive number for east.
• Values should be saved in non-volatile memory.

Relevant Fields from the System Time Table

• “system_time”: Number of seconds since
12:00 AM, Jan 1, 1980. Doesn’t include leap seconds.
• “GPS_UTC_offset”: Number of leap seconds inserted since 12:00 AM, Jan 1, 1980.
• “DS_status” flag:
– Set to one when all time zones within broadcaster’s coverage area have entered daylight savings time.
– Cleared to zero when all time zones within broadcaster’s coverage area have exited daylight savings time.
• “DS_day_of_month”: If non-zero, indicates the day of the current month for which transition into or out of daylight savings time is to occur.
• “Ds_hour”: If non-zero, indicates hour for which transition into or out of daylight savings time is to occur.

Calculating the Current Time
from the System Time Table

• Obtain “system_time” field from STT.
• Subtract “GPS_UTC_offset” field from STT. The result is number of seconds since 12 AM, January 1, 1980. (In Greenwich, England).
• Add time zone adjustment value obtained from User Interface.
• Convert to date and time format. (Use “mktime()” if it is available in your C library).
• If daylight savings time is observed, check if we are in daylight savings time.
• If so, add 3600 to results from step 3, and reconvert to date/time format.

[99gecko]

I was multi-tasking when I put in my original post. Explains a few errors including the ommision of channel 23.

Should have read as:

Channels in DST (compliant):
American: 2, 43
Canadian: 5, 9, 25, 66

Channels not in DST:
American: 4, 7, 23, 29, 49
Canadian: 57

[ASA]

The queston now is..... when we get to the "old" dst date in three weeks time, will everything start to work "normally" again?