Recent revelations about the reliability of Chainalysis' blockchain analysis tool have stirred concerns across Twitter. Elizabeth Bisbee, the head of investigations at Chainalysis Government Solutions, testified that she was “unaware” of any scientific evidence supporting the accuracy of the company's Reactor software.
As L0la L33tz pointed out in her now-removed CoinDesk article, "the fact that Chainalysis’ blockchain demystification tools have become so widespread is a serious threat to the crypto ecosystem." If these tools influence critical decisions in people's lives, their reliability is not just technical but a matter of basic justice.
Bisbee’s statements during her testimony only intensified these concerns. Instead of providing scientific backing or hard data, she mentioned that Chainalysis relies on "customer feedback" to judge the software's accuracy. This raises the question: should a tool that could potentially land individuals in legal jeopardy be evaluated based on customer satisfaction?
This testimony poses significant challenges not only for Chainalysis but for the entire industry that relies on such analysis tools. The argument made by lawyer Tor Ekeland, that the Chainalysis Reactor operates as “a black box algorithm” that “relies on junk science,” puts further pressure on companies to be transparent about their methodologies and the reliability of their findings.
Equally concerning is CoinDesk's decision to withdraw L0la’s report on these revelations.
While it is not clear why CoinDesk decided to remove the article, it does bring to the forefront questions about editorial independence and potential conflicts of interest.
L0la added that for a media outlet that holds itself as a beacon of accurate news and analysis, such a move inevitably leads to speculation about external pressures or self-censorship.
Below is the concluding segment of L0la’s article that was removed from CoinDesk and republished by Bitcoin Magazine:
For Chainalysis' part, the company noted in a July 18th court declaration there is no doubt about the Reactor software's findings. Although not "peer reviewed" in an academic sense, the company's clustering heuristic, the algorithm used to find relationships between blockchain addresses, comes to "deterministic" conclusions that can be independently verified and reproduced.
The same statement noted the company is unaware of margins of error rates for the Reactor software, and has not gathered and recorded false positive and negative rates for its software overall.
Unfortunately for Bisbee and her corporate overlords, we still live in a democracy in which criminal convictions prerequisite the existence of scientific evidence. Maybe Bisbee would be better suited pursuing an art history degree.