Yea, it's nonsense to say "stone" means worked by human hands.

Here's what Dictionary.com has to say:

the hard substance, formed of mineral matter, of which rocks consist.

a rock or particular piece or kind of rock, as a boulder or piece of agate.

a piece of rock quarried and worked into a specific size and shape for a particular purpose: building stone.

The etymology of stone indicates same:

stone(n.) "discrete piece of rock," especially not a large one, Old English stan, which was used of common rocks, precious gems, concretions in the body, memorial stones, from Proto-Germanic *stainaz (source also of Old Norse steinn, Danish steen, Old Saxon sten, Old Frisian sten, Dutch steen, Old High German stein, German Stein, Gothic stains).

Anecdotal: I've never once heard anyone, ever, make this distinction. Stone and rock are synonymous.