I took Roger Fisher's (and the Harvard Negotiation Project's) Negotiation class during a winter term at Harvard Law School years ago. It was among the most useful classes I took.
Wikipedia gets us off to a good start: "In negotiation theory, the Best Alternative to a Negotiated Agreement, or BATNA, isthe course of action that will be taken by a party if the current negotiations fail and an agreement cannot be reached. BATNA is the key focus and the driving force behind a successful negotiator. A party should generally not accept a worse resolution than its BATNA."
In many, if not most, situations, the BATNA is a tool for negotiating leverage. You are saying, in effect, "If you don't take my offer of a negotiated agreement, then your best alternative is X" where X is made to look like a worse outcome for the other party than your proposed agreement. That usually means doing research to understand the law, among other things.
Example: Landlord failed to return all of Tenant's deposit in a timely manner.
Option 1: Tenant says, "Please, Landlord, I know you are saving up for that lobotomy, but send me the rest of my deposit." -- Effectiveness Score: 0 of 3
Option 2: Tenant says, "Give me the rest or I'll sue you." -- Effectiveness Score: 1 of 3 (BATNA is only implied, and not concretely)
Option 3: Tenant says, "I ask you to give me the rest of the deposit (or some % of it to reflect litigation) because if I have to bring a suit, you will be on the hook not only for triple the amount you are withholding but also for my attorneys' fees. You'll also waste a lot of your own time. And you'll have to admit, that's a pretty sucky BATNA for you."
The BATNA typically incorporates an analysis of the legal situation, costs, time, inconvenience, reputational harm, and more.