What are you using on the ESP8266 to communicate with the MQTT broker?

If it's code you've written yourself, it should be possible to set the LWT message to be identical to let's say the Off status. This should be done when you initialize the connection to the MQTT broker from your switch.

In ...