The first thing to know when you think you may buy from a scalper, is to have cash. That may sound obvious, but many people don't carry cash anymore, so you may need to make sure you stop by a bank ahead of time so you don't find a great deal and then have to pass on it because you only carry plastic.
When you do walk up to a scalper, it is kind of like buying a used car. Don't sound too interested in what they have. They will ask what you need, and I usually say something like "your 2 cheapest." Inevitably whatever their cheapest is is more than I want to pay. I typically balk at the offer and start to walk away. In most cases, the scalper will walk after you making secondary offers or pulling even cheaper tickets (that he forgot he had) out of his boxers. If I do have an amount I want to pay, I might say something like "I've got $20 bucks, do you have any for $20 bucks?" At this point I have the $20 in my hand, so they know I am serious. Sometimes they will come down and give you a deal, but other times they legitimately don't have anything they can accept $20 for.
Keep in mind that $20 is a number I float out in Cincinnati. Perhaps in a place like Philly, New York, etc. you might just get laughed at. However, there is nothing wrong with getting laughed at in my opinion. The worst they can do is say "no". The key when buying tickets from a scalper is knowing when you have the advantage.
One instance where the buyer definitely has the advantage is when the team stinks. For many years this was the Reds, and most nights they would do well to have a half full stadium. When that was the case, it is a nice bargaining chip to know that you can walk-up to the gate and buy a $5 ticket and sit wherever you want. It doesn't hurt to remind the scalper of this, mainly so they know that you know. Ultimately you have the power in this case because you truly can walk away and easily get cheap tickets.
Another way to get cheap tickets from a scalper, or at least a situation where the buyer holds the edge is when the game has already started. Although I don't like it, many of you probably arrive to a game after the first pitch has been thrown. Most of the time, scalpers or random individual sellers who have extra tickets, don't want to 'eat' the tickets. This simply means they would rather get something for them than have them go to waste. Obviously this concern increases as time passes by. If you are 2 hours early, they won't have nearly the sense of urgency to cut you a deal as they will once the game has started. If you are running late and don't have tickets, you may be able to work a deal to get tickets way under face value.
Moral of the story when it comes to buying tickets from scalpers; don't be scared. Know what you want to pay and be prepared to walk away. That rhymed!
Another great resource for information on how to get cheap and sometimes free tickets to a game is my buddy Andrew Van Cleve. Check out his website here. This guy is a true sports road warrior.