I find this to be largely true, especially of older people, for foreigners who live there. But not to discourage a visit: you are very welcomed as a tourist in most places in Japan, and any Japanese you speak will be appreciated.