On many AVRs you need to "assign" the various digital inputs. This is usually described in the AVR's operating manual. What this means is that the AVR needs to know which digital input you "assign" to which input you choose - like VCR2. You need to go into the AVR's setup menu to do this.
In some cases some of the digital inputs are "pre-assigned". If you have one of those available, you can try that or simply as a test use one of the known working inputs to make sure that you can hear the sound from the STB - say the CD input which you have working?
You may also need to go into the settings for the STB and tell it to use "Dolby digital" and not HDMI or Other for the audio - press the settings button twice and go to the audio settings. What's the make/model of the STB?
In some cases some of the digital inputs are "pre-assigned". If you have one of those available, you can try that or simply as a test use one of the known working inputs to make sure that you can hear the sound from the STB - say the CD input which you have working?
You may also need to go into the settings for the STB and tell it to use "Dolby digital" and not HDMI or Other for the audio - press the settings button twice and go to the audio settings. What's the make/model of the STB?