As someone who regularly serves as an observer (on behalf of a VC) I can comment that, yes, while I have no offical fiduciary duty to the company (as a director would), I have lots of incentives to be as helpful as possible to maximize the success of the start up. My aim is to be a partner and ally to the management team, to support them, ask questions, and bring issues up that others might have missed. In many cases, I've been asked to be an observer on a board due to my previous industry experience or simply to provide a different perspective than the rest of the board.
As the management teams I've worked with can attest, I am rarely a silent observer, but I aim to refrain from bringing up topics that are a distraction and I am more muted than I would be if a director. Certainly I don't vote on board matters. I also act as a resource outside of formal board meetings.
In the best case I think you can consider observers as another perspective and advisor to help your company without the challenge of adding more board members. If your observer is proving unhelpful or disruptive talk to the other directors on the board about it in private (ideally your chairman) and they can have a conversation with the individual offline.