July 18, 2020
Virtually every technology, from pencils to dynamite, is a dual use solution; it can be used for good or evil. Every company researching, building, or selling dual use solutions has an obligation to consider the consequences of their actions.
We occasionally get questions as to what kinds of questions are permissible for the hCaptcha service, and why particular questions are being asked.
Human beings are pattern-seekers, and sometimes an innocuous question can go from entirely unsurprising to suspicious in an instant, due to changes in the world rather than the motivations of anyone asking the question.
As services become more popular, this sort of occurrence inevitably becomes more frequent. hCaptcha is used by millions of people each day in virtually every country in the world, so we are publishing this case study on how we address these issues in order to be as transparent as possible.
“Umbrella” is a term in ImageNet, the standard benchmark dataset used by computer vision researchers. Reliably identifying people holding umbrellas is also critical for building safe next-generation advanced driver assistance (ADAS) systems.
Umbrellas. (CC0 image courtesy of Piqsels)
This means there are excellent use cases for real-world umbrella annotations. There is room for disagreement on the impact of self-driving cars, but ADAS systems are already saving thousands of lives each year around the world.
However, it is clear that building this kind of dataset is now perceived as a dual-use technology in a way few would have expected a year ago.
We are thus no longer accepting requests for questions related to umbrellas at this time. The potential for confusion when an end-user sees a question about umbrellas in the current moment is too high, and ultimately our goal is to make the world a more pleasant place.
We have a strict AI Ethics policy at hCaptcha, and part of that includes a Know Your Customer (KYC) process.
We always attempt to gain a good understanding of each new customer and their intended use case, both to confirm it is a good fit for our products and services and to ensure that it follows the policies we have adopted as a company.
We also conduct ongoing reviews as necessary. For example, in our recent review of the dates and sources of requests for umbrella-related labeling requests we were quickly able to confirm that no government entity or known state supplier of surveillance software has made requests regarding umbrellas using our services in the past 12 months.
For each new labeling customer, we go through a checklist during the sales process. This includes initial KYC diligence prior to onboarding, as well as verification of all requests made to our analysts, and real-time spot checks and periodic reviews of requests made using our self-service platform.
This review is composed of several sets of criteria:
Ethical concerns criteria: Objective
Ethical concerns criteria: Subjective
If we cannot satisfy ourselves on these points, we will decline the request and may terminate further access to our platform, as outlined in our Terms.
We hope you appreciated this look into how we handle operational questions with an AI Ethics component.
Very few companies offer transparency into their decision-making processes in this area, and we hope others will follow this example!
