As I understand it, some people have trouble imagining a setting-less sentence:

Her father suddenly threw aside his newspaper and jumped to his feet.

What does the room look like? What does the father look like? So, at least in theory, this event cannot be visualized. But I don't have any trouble imagining that without the accompanying visual fill-ins. There are just, well, vague spots in my mental picture.

But if other people have trouble with that, they will probably want setting first. And of course no reader would know to include his broken foot in imagining that sentence, because the broken foot appears in the next sentence.

So that seems to be an argument for starting with setting. But is setting-first any easier to imagine?

Their house was small

What color is the house? What style is the house? Clearly there is not near enough information to create an accurate visual image of that house. So that sentence of description really has the same problem as a sentence of just action.

He was a huge man with red curly hair

That seems easy to visualize. But you don't know what, for example, his nose looks like. When you visualized that man, did you leave his nose blank? I did, but I can leave out details and be happy. This perhaps has fewer problems than visualizing the father in the sentence given above, but it essentially has the same problems.

And when you imagined that sentence, was he running? Eating dinner? You probably imagined him standing still, but in the scene he might not be standing still.

Even though I didn't like Steinbeck's effort to start his book with setting in Of Mice and Men, I like his later attempts to add setting to a scene. That's a small piece of evidence that setting makes more sense when it's embedded in a story.

The Problem of Attention

Some setting is needed to understand or appreciate the action. I will call that context, for want of a better word. The reader needs to pay attention to and remember that information.

And some setting isn't that relevant. Throwing down the newspaper contributed to the scene – it implied that he was sitting in his chair calmly, then was not. But a magazine would have worked as well. Saying he "threw down what he was reading" is too abstract (and wordy). So the newspaper itself was an irrelevant detail.

Setting can also help with mood. I actually liked this start for its mood:

It was a dark and stormy night; the rain fell in torrents except at occasional intervals, when it was checked by a violent gust of wind which swept up the streets... (Paul Clifford, Bulwer-Lytton)

But none of that was needed for the plot.

And of course some writers present irrelevant detail just because they have been told to. Perhaps so the reader can imagine the scene more vividly.

These different uses create a problem for the reader: Which details need to be attended to and which do not?

I suspect that guess, of what to pay attention to, is easier in the middle of the story than at the start. Which is to say, that's another problem with starting a story with setting: The reader doesn't know what is important context, what is for mood, and what is an irrelevant detail.

And, I don't know, but . . . if a piece of setting is put in for plot-purposes, it will be consistent with the rest of the book. If it's put in for other reasons, I'm not sure if the author cares what it actually says.

A few miles south of Soledad, the Salinas River drops in close to the hillside bank and runs deep and green. The water is warm too, for it has slipped twinkling over the yellow sands in the sunlight before reaching the narrow pool. (Of Mice and Men, Steinbeck)

I don't see how it's possible for a river to do that. As far as I know, a river would be close to a hillside bank only when there were hills on both sides, but the other side of that river is a valley. I also experience a conflict between a river dropping and a river being deep or being a "pool".