I have had hosts be this direct, and we all went, “okay,” and left. And came back when invited with no hard feelings. Usually D&D games, but also movie nights.