I was on a reading tear the last few weeks of the year. My goal was to respond to every requested full and I almost made it (three remain...I ran out of steam on 12/23/16!)
One of the consistent weaknesses I noticed was the story didn't surprise me.
Good writing requires pacing and tension, yes.
But good story telling means your audience gasps.
I want to gasp. I want to put the manuscript down and just enjoy that moment when the author turns the story upside down.
Think of the movie The Sting
(There will be spoilers here just in case you've been on Carkoon for the past 40 years and missed it.)
The movie unfolds as the confidence game being run on Doyle Lonegan is put into play. The audience sees all the important information, but what it MEANS is the surprise.
There are two particular places where the audience gasps with surprise in the movie:
The first is when Hooker (Robert Redford) is leaving the diner via the alley after breakfast and his night with Loretta the waitress. Loretta was gone when he woke up; all her clothes were gone, but his money was still in his wallet. Neither Hooker nor the audience knows what happened to her. But, in the alley, Hooker sees her. She walks toward him, smiling very slightly. He's glad to see her.
Then, a gloved gunman behind Hooker takes aim and shoots. Not Hooker, Loretta. Hooker and the audience are stunned. The gunman runs up, turns Loretta over, revealing a silenced pistol in her right hand. "She was gonna kill ya," the gunman tells Hooker. Turns out Loretta is Loretta Salina, the killer engaged by Doyle Lonegan. There have been references to Salina earlier; but none of them used her first name, and in movie full of gents, a lady assassin was real suprise. Plus, she'd just spent the night with Robert Redford! And now she wants to kill him! Gasp indeed!
The second surprise is of course the end, just after Doyle Lonegan is rushed out of the betting parlor by Sgt. Snyder cause "there are dead guys here." Only of course, neither Gondorff nor Hooker ARE dead. The audience gasps when Hooker opens his eyes, and gasps again when Gondorff comes "back" to life.
All this is a surprise even though we actually SEE Hooker putting blood squibs in his mouth when he dresses that morning.
That's what surprise is: we had the info, but we didn't know what it meant.
One of the best ways to build plot, and surprise, was explained pretty neatly by a terrific author named Jeff Somers: figure out what's supposed to happen. Then do something else.
If you want a great example of that, rent the delightful movie Hopscotch with Walter Matthau and Glenda Jackson. Walter Matthau has worked out a timetable for the trick he needs to pull off. Everything is going well...then bam. His car gets a flat tire. Didn't see that coming! It's a surprise!
Surprise does NOT mean aliens arrive in chapter 14, unless you are writing a novel about aliens arriving. It does not mean Felix Buttonweazer learns to fly when he is defenestrated by his nemesis, unless you are writing magical realism and it's all a dream.
A good surprise makes perfect sense but you just didn't see it coming.
In other words, really really simple stuff...and hard as hell to carry off.