Should I respond to an "ethical hacker" who's requesting a bounty?

A true "ethical hacker" would tell you what issue (s)he found in your system, not ask money for that; (s)he could offer to fix it as a contractor, but that would be after telling you what the actual problem is; and in any case, it's a completely different thing from just trying to scare you into paying.

This is plain and simple blackmail.

(Also, it's a very real possibility that there is no real vulnerability and someone is just trying to scam you into paying money for nothing).

While this might be blackmail, there are many possibilities for genuine good intents, too. Therefore, here's some more comprehensive thoughts on how one might handle unsolicited vulnerability reports. In short: you have every reason to be cautious, but you do not have to be rude.

Who may find vulnerabilities and why?

Ethical hackers perform their analysis based on a contract typically with predefined targets and limitations. These might be ordered assignments or more loosely defined bug bounty programs, either directly or through a platform like HackerOne. In any case, an ethical hacker (or a white hat hacker) always has an explicit permission.

From the details in this question alone it is hard to tell whether the message you got is a clear scam or someone with good intentions but lack of understanding – or willingness to adhere to ethical standards. The latter grey hats might even violate laws, but they do not have malicious intentions. The penetration testing industry is also extremely trendy, so there are all kinds of self-appointed penetration testers, ethical hackers, security researchers etc. with varying skills (or complete lack of them). In this case they may benefit from some gentle guidance, whereas false accusations might lead them to wrong direction.

I have found several vulnerabilities by accident, without an intention to poke the system in any way. These cases are usually rather harsh, and I do hesitate whether not to report it at all, report it anonymously, or report it with my name, which would give me the possibility to help them with further questions. The reality is that because I did not have a permission, the receiver may interpret or handle my report with unexpected ways, possibly causing me legal charges or other problems. So far, they have been sympathetic towards me.

Do you benefit from these findings?

You are asked to pay for the findings, but without knowing the details you cannot be sure whether they are worth paying at all. Vulnerabilities comes in all shapes and sizes. Some of them are critical, and some are minor. Some may also seem problematic from outside, but are completely irrelevant to you, or within your accepted risk. One simply cannot sell vulnerabilities in pieces, bundles, kilograms, or liters.

Two examples of completely worthless reports I have got recently, both with genuine intent.

1. A message suggested a reward for finding a web page protected by HTTP basic authentication, which indeed is not a secure authentication method. However, as it was only an extra layer of security before an actual login page, and not protecting any critical system anyway, it was not really a vulnerability at all. Therefore, the finding had zero value for the company.

2. A report of a missing SPF record. The explanation was correct and all, but the record was not missing! Instead of querying from DNS, the "bug bounty hunter" had used a web-based SPF lookup tool but used http://example.com instead of example.com. Due to this syntax error it did not show the record.

Therefore, in order to judge the value, some details of the vulnerability must be disclosed. If someone who has found the vulnerability thinks giving out these details may result in losing the reward, the vulnerability may actually be worthless: known, easy to spot with automated tools, within accepted risk, too minor, or otherwise irrelevant. On the other hand, if the vulnerability is severe, it is often also so complex that giving some proof of concept will not completely help fixing it. The additional work required to describe and address the vulnerability is valuable and will be paid.

It's not unusual for someone who discovers a security vulnerability to be paid a bounty for their discovery. A lot of prominent open source projects and web sites have policies of paying a bounty for responsible disclosure of a vulnerability. I don't know how common it is for companies to pay a bounty without having some sort of bounty program set up in advance though.

I received a bounty for reporting a security bug in a very prominent open source web application. Here's how it worked in my case:

• I reported the vulnerability to the development team via their preferred reporting method, including the fact that if the bug was eligible for a bounty I would be interested (they had a public bug bounty program).
• I kept knowledge of the vulnerability confidential while the team identified and patched the issue.
• When a patch was released the notified me that my report was indeed eligible for a bounty and how much they'd be prepared to pay.
• At this point I was free to discuss my vulnerability publicly (although chose not to do so).

The key points here are that:

1. The report was made without holding the details for ransom until I was paid.
2. Details of the vulnerability were not made public until the vendor was able to make a fix.
3. If the issue I reported was not in fact a security bug, I wouldn't be paid.
4. The vendor decided how much the vulnerability was worth. They did have a public table of "Vulnerabilities of type X will be paid up to $Y" on their web site.

While I only have direct experience with this one vendor, I believe this process is pretty typical for most.

In your situation I would:

1. Insist that the vulnerability be disclosed responsibly. i.e. To you, directly, and without any form of public, or semi-public disclosure by your "hacker". You want to be aware of this before everyone else, that's one of the things you're paying for. If your hacker posts about this publicly, or talks to his mates about it, then there's no deal.
2. Insist that details of the vulnerability are to be verified by a security expert. Given you say you're a one man show without a lot of expertise in security, that probably means hiring someone on contract to assist you.
3. If your expert agrees that it's a security problem, they'll be able to give you an idea of its severity and YOU can decide what it's worth.

How much should you pay? That's up to you. In my case, the vendor rated the bug as "critical" then it was patched. It could have led to serious compromise, but would have been difficult to do. I was paid a little under $5k for my efforts, which was near the top end of the range quoted on their web site.

Also, if they're just telling you about a known security vulnerability in a bit of third party software that's probably not worth much. e.g. if you were running an old version of WordPress and the bug was a known WordPress vulnerability.

Is this black mail?

If they insist that you don't get details until a bounty is paid. Yes. That's not how these programs usually work, a proper ethical hacker knows that.

Is this his way of saying you'd better pay me or I'm going to wreak havoc?

A proper ethical hacker isn't trying to wreak havoc. Nor will they be selling the vulnerability to someone else if you don't pay. But that assumes you're dealing with a legit ethical hacker, not some troublemaker who's trying to rip you off or cause trouble.

Or is this a typical and legitimate method for people to make a living without any nefarious intentions?

After I earned my bounty, I did the maths, and figured I could potentially earn a living collecting bounties. It is possible. Whether that's what your guy is up to, who knows. Trying to collect bounties from companies that don't have formal bounty programs is a pretty risky way to go about it though, which counts against your guy IMHO.

Yes, that is blackmail.

The responsible thing to do is to inform you privately. Perhaps with a disclosure policy of eventually going public if no response after some time.

A more polite way of doing business would be a hint that you would get more reports if you offered a reward via a bug bounty or similar. But still forward the issue details regardless.

Considering hiring a security person (not this "hacker") to evaluate your systems. Whatever form that takes, a one-off engagement to do a security assessment, a bounty, or a migration to a hosted platform to outsource operations to someone else.