Smart speakers normally work by constantly listening to a wake-word, that is processed locally, before listening to your more complex command, and send the audio to the cloud for processing. That means most of the time no data is sent to the cloud, as continuously processing audio in the cloud would not be resource-efficient.
However, in isolated cases, the company may want to listen to audio samples to improve their product(s) and it's possible since the hardware is perfectly capable of doing this. Alternatively, hackers could always access your smart speaker. So if you worry about your privacy, while