1. You set the STB to Dolby Digital if using a digital connection like optical or coaxial, or you set it to HDMI for an HDMI connection.
2. For DD2.0 signals, most people use Dolby Pro Logic II Cinema (or movie or similar) IIx if you have a 7.1 system. There are similar DTS options - usually described in the operating manual for the AVR.
3. For DD5.1 signals, you can use the same, or leave it as DD5.1.
4. For BD players, you need to set the BD player to Bitstream or something similar, depending on the BD player, in order to send HD audio to the AVR for the AVR to decode. See the part on HDMI/HD audio near the end of the link below - assuming you use an HDMI connection.
http://www.digitalhome.ca/forum/showthread.php?t=76082
When listening to music, you can use DPL Music, or "5-7 channel Stereo", or simply listen in stereo 2CH. There are no "rules" - experiment with the various settings.
Similar thread:
http://www.digitalhome.ca/forum/showthread.php?t=131844
Then check out the following post, useful for those new to the forum - FAQs, Search Tips, Optimization, etc:
http://www.digitalhome.ca/forum/showthread.php?t=57741