Let me first start off by assuring you that this isn't a question about quitting writing. I know how you feel about the Q word and, therefore, know better than to write in threatening to use it on myself.
But it is a question about changing course. At what point do you ever feel - or would you advise a client to consider - a pivot to be inevitable? Is there a certain number of years or number of attempts that would start to give you doubt? I'm coming off of a particularly brutal 9-month round of querying and rejection, my 5th in ten years.
The feedback, not just for this novel but for every one that came before it, has been consistent. To the point where I've written back to agents asking, very politely, if they really mean it or if their form rejections are always that gentle and encouraging. I've been told they mean it. "Love the writing," they promise. "Beautiful language, great setting, believable characters. But the story doesn't work." The story. never. works.
So often I see people getting rejected for the opposite reason, where they have awesome stories but struggle with the nuts and bolts of telling them clearly, and I feel envious. Because I'm confident that they can learn the nuts and bolts. Things like sentence structure and word choice can always be improved on and there are classes and workshops aplenty to address those kinds of issues. I've taken lots of them myself.
But you can't fix something that isn't there and I'm beginning to wonder if I'm just not a capable storyteller, if I'm simply missing that chip in my brain and I don't know how or where to get it. Last night, for example, I confided my newest idea to my 10 year old daughter and she immediately pointed out a plot hole so big and obvious and embarrassing that I broke down crying as soon as she left the room. I can't help but feel that, at this point in the game, I should know better. I should be better. I've always been a person who writes, it's been part of my identity for 20+ years, but I still don't feel like I am really truly a Writer.
So I have to write, obviously. I can't not do it. But I'm wondering if I need to try something different. Not every writer is a novelist, after all. There are other forms of expression - short fiction and essays and poetry - but even considering them feels, in a way, like giving up and I'm having a very difficult time mustering the energy to convince myself otherwise. A push in either direction would be much appreciated. A slap on the face if I'm being a big baby would probably work too.
I don't think you're being a big baby at all. I think you've identified a shortcoming in your skill set and you don't know how to fix it yourself.
This is why people have coaches.
It seems clear to me that you can either take steps to improve your story telling skills, or you can change what you write. Either of those are honorable options.
The question really is, which one will make you happy?
I believe that you know your purpose and true calling by finding and doing what brings you joy.
And I'm not talking about scotch and cigars and the tender ministrations of a well-oiled cabana boy who thinks crows feet are sexy. (Too much dino porn yesterday, our minds are in the gutter.)
What I mean is what motivates you to leap out bed in the morning clutching your stylus and reaching for your clay tablet (we are old school here, no fancy pen and ink stuff for us.)
If you want to tell stories, but just have a hard time with plot, enroll in a class on plot and story. Often it's just a matter of learning how to outline properly so you can see those plot holes sooner rather than later. Or partner with someone who does know how to plot and work together. Writing teams are all over the place these days.
And honest to godiva, quit beating yourself up about plot holes. It's not like we all haven't done or seen that before. I've seen entire novels without plot at all, let alone a gaping hole or two.
In fact, I was just telling our bright eyed and bushy tailed interns here about a client's novel which I shall Not, Understandably, Mention By name that had gone through my eyeballs, a beta reader, AND a copy edit before one of my interns caught a plot hole.
Verily how we did scamper to get that fixed before sending on submission!
And if the idea of enrolling in a class and working on a new novel just makes you shrink down and want to crawl under your duvet with a pint of Ben & Jerry's Hazed and Confused
well, here have a spoon. I've done that many a time.
Failure is not changing course.
Failure is not pursuing a new art form.
Failure is not trying new things.
Failure is stopping before you find joy.
There are a lot of things in this world that will make you cranky and crazy and scared.
You need joy. Don't let any avenue go unexplored until you find it.