Bees (and relatives) do it too. If you need to deal with a wasp nest or something like that, do it at night and their defense will be much less enthusiastic.

When I last dealt with some, knocking down the (small) nests would have a guard harass me until I moved about 10m away from the nest during the day. At night, it would just buzz me a bit before settling back down to rest without me even moving.

Note that I'm not saying it's safe to harass a nest/hive at night, just safer than doing it during the day. The ones I dealt with were small enough that I only ever saw a single guard plus one worker, and even during the day, sometimes I'd just fight the one guard instead of running, since it's hard for a single wasp to sting you if you can track it decently and manage any fear. Trying to deal with a large nest could still be fatal at night.