Action

The other major choice is to start with action. From my rewrite of The Wonderful Wizard of Oz:

Her father suddenly threw aside his newspaper and jumped to his feet. He grabbed the crutch for his broken foot, hobbled quickly to the window, and looked out over their farm and the flat plains of Kansas for only a second. Then he whipped around and screamed, "It's a tornado! It's a gol-darned tornado, coming our way! Quick! To the cellar!"

I like starts with action, but I assume some readers must prefer starts with setting. (Or else why does anyone write them?) So, readers presumably differ on this issue.

Stories differ too – some are more suitable for one type of start versus another. Genre-wise, science fiction and fantasy probably have a higher need for starting with setting, because the world is often different/unexpected and often supposed to be interesting. One theory is that if the setting is the most important start of your story, or the part most likely to interest the reader, you should start with setting.

And there's good authors using each type. So, it's unlikely that one type of start can ever be declared always better than the other – no matter how much I might want to.

But I want you have choices, which means having the skill to do an action start, even if you eventually choose not to. Keep reading.

Interest

A few miles south of Soledad, the Salinas River drops in close to the hillside bank and runs deep and green. The water is warm too, for it has slipped twinkling over the yellow sands in the sunlight before reaching the narrow pool. (Of Mice and Men, Steinbeck)

Boring, right? If someone just wanted to read setting, they could read a travelogue. The main advantage to action is that it's more interesting. The evidence for this is simply that action starts exist -- they are harder to write, so no one would use them except if they thought they were better.

Naturalness

Again, starting with setting/description seems to be more natural, at least for the storyteller. So it's natural for a writer to begin with description.

But don't start with description just because you think you have to; you don't. And don't start with description just because you think you should; there's a good chance action is better. And do not start with description because it's easier on you.

If you decide that starting with description is easier on the reader, that's different. That's when naturalness becomes important. You still have to balance that choice against interestingness, but being kind to your reader is a worthwhile goal.

Context

Merely providing random background information doesn't make a scene more interesting. However, most scenes need certain background information to be interesting. Just as the punch line to many jokes needs particular background information to be funny.

So, if you have a particular scene that you want to be good, it's probably not best to start with it. Or you can, but it won't be as good.