My ex-wife and I were on the founding team of a company (it's still running with a different team). Imagine the high pressure environment of a startup, woven completely into your marriage.
You can't go home and vent to your spouse, because your spouse becomes the person you need to vent about. Or you find yourselves not aligned on a business decision. Or you take different sides in a dispute with someone else on the team.
How do you handle those arguments? How do you handle your home life when your home becomes your office? How do you handle your work life when simple disagreements about UX turn into deeply personal shouting matches?
These are very tricky to navigate, and can drive a wedge in your marriage. Not ALL marriages. But it's a fundamentally differently kind of cofounder relationship than anything else. It is more volatile. And if it blows up? The whole company is at risk.
So, yes, I don't blame investors for steering away. I don't know how they would do diligence on your marriage and communication skills.
If you're a developer married to a designer, I would say, build some fun side projects together. But if one of you has a startup idea, support your spouse by finding a great cofounder to work with. Think of it this way: if you each have your own cofounder buddy, not only will you keep your marriage happy, you're diversifying your portfolio.