If you're happy with the performance of your setup, than going higher isn't necessary. Though in your case, I don't see you having a choice (who wants to argue with their wife!).
In certain circumstances, going higher up can negatively impact reception quality of some channels. This is often caused by signal artifacts from distant transmissions on the same channel. It would be especially noticeable during strong tropo conditions. A narrow beam antenna can usually work around these problems, by putting the distant station in a null of the reception pattern.
However, since you're not go up a significant height, I doubt you'll run into problems.