Deciphering Crypto Twitter (2024)

3.1. Data Collection

We manually curated a list of keywords related to cryptocurrency and security, ranging as broadly as “crypto” or “blockchain” to specific technologies such as “zk-rollup” or “layer-2.”We also included security-related terms such as “breach” or “hack” to get tweets related to cryptocurrency hacks.In total, we collected our dataset using a list of 199 keywords.The full list of keywords is available in our anonymized repository.The data collection ran over 14 days, from November 09, 2022, to November 23, 2022.The final dataset consists of 39,828,572 unique tweets.

We used Twitter API v1(TwitterAPI, 2021) to scan and scrape tweets that contain the selected keywords in real-time, maintaining our databases using Elasticsearch(Elasticsearch, 2018).If a new tweet contains one or more keywords, it is added to the Elasticsearch index.The volume of tweets during our collection period can be seen in Figure1.The complete index contains 130 features about each tweet.We considered a subset of these features, such as the text or reply/quote structure, in our analysis.Some of the features used in our work are shown in Table1.

3.2. Data Sanitization

We found that some of the tweets in the dataset do not discuss any cryptocurrency-related topics, such as discussing just about “security” unrelated to the domain we are exploring.However, in the later stages of our analysis, the cluster-based and graph-based approaches can identify such tweets and group them accordingly.

In the context of Crypto Twitter, a spam tweet refers to an unsolicited and often irrelevant message, typically intended to promote certain cryptocurrencies, ICOs, or other crypto-related schemes.These tweets are usually characterized by their repetitive nature, excessive hashtags, and links to external sites.They might also include misleading information, false promises of high returns, or attempts to impersonate well-known figures in the crypto community.The primary goal of spam tweets in the crypto context is often to manipulate market sentiment, spread misinformation, or engage in fraudulent activities such as phishing or scams.Unsurprisingly, we found many such spam tweets present in our initial dataset of 40 million tweets.

However, we also note that some of these tweets could have been made by genuine users who share a lot of enthusiasm about the topic or are after an “Airdrop” of a certain crypto token(Allen etal., 2023).This introduces some difficulties regarding our approach to definitively identifying spam tweets.With this in mind, we only removed tweets verified to be linked with a scam attempt.For example, we found that millions of tweets in our dataset contain the same set of sentences “Uniswap is being exploited by this dude. Why is nobody talking about this?…”.These tweets appear to have been made by a coordinated bot attempt to push a phishing campaign(Sinclair, 2022), and were thus removed from our dataset.

After manually filtering the tweets by matching commonly found spam patterns that we were able to verify, we ended with 20,911,310 tweets in the final dataset for the analysis.

In addition to filtering for spam tweets, we replace user mentions with user and any external links to http, which allows us to better capture the style of tweets found in the dataset instead of specific mentions of individual users or links.

3.3. Sentence Embedding

We used a publicly available pre-trained Sentence BERT (SBERT) (Reimers and Gurevych, 2019) model from HuggingFace(HuggingFace, 2022a) to encode the collected tweets into a latent space.SBERT is a BERT-based model that has been shown to outperform BERT in both SentEval and Semantic Text Similarity (STS) tasks (Reimers and Gurevych, 2019).External URLs and user mentions were replaced with the keywords “http,” and “user,” respectively, to avoid overfitting on terms that appear in usernames and URLs.

3.4. Sentiment Analysis

We used a publicly available RoBERTa(Liu etal., 2019) model fine-tuned on the TweetEval sentiment analysis task benchmark(Barbieri etal., 2020) to generate the sentiment scores used in our analysis.The publicly available weights from HuggingFace(HuggingFace, 2022b) were used without any additional fine-tuning.Unlike Sentence-BERT which is fine-tuned using a contrastive objective to predict the semantic similarity of sentences, this RoBERTa model was fine-tuned with a classification layer for predicting sentiment labels corresponding to the input text.The sentences are fed to the model to yield three types of sentiments [positive, neutral, and negative] and the confidence score associated with each label.

3.5. Tweet Clustering

In order to cluster the tweets by their semantic similarity, we used the SBERT embeddings(Reimers and Gurevych, 2019).We reduced the dimensionality of the embeddings by training and applying a linear autoencoder, then performed K-means clustering to group the tweets by their similarity in the latent space.We then used a term-frequency inverse-document-frequency (TF-IDF) matrix to extract the topics for each group.Even after dividing the tweets into different clusters, we faced a very large number of tweets to examine in each cluster.Therefore, to aid in this process, we implemented a dashboard (see Figure2)to search each cluster of tweets effectively.Given a certain query string, this dashboard embeds the query into a SBERT embedding and searches for tweets with similar embeddings (nearest-neighbors by cosine similarity).Using the keywords observed in our TF-IDF analysis, we were able to take a deeper dive into the collected tweets by manually building queries using keywords contained in our clusters.

In order to get a cluster-specific set of topics, we excluded the top 10 terms found in the overall dataset from the top terms of each cluster.

We extracted the sentiment score of each tweet’s raw text using the RoBERTa model fine-tuned on the TweetEval sentiment benchmark.To understand the sentiment change over time, we analyzed the keywords associated with each type of sentiment.We then grouped the tweets in each cluster by their tagged sentiment type and conducted TF-IDF analysis on each sentiment group separately.

3.6. Network-Based Analysis

In addition to clustering tweets using a semantics lens, we examined the dataset at the thread and user levels by constructing graphs of tweets.For this purpose, the reply_to and quoted_status fields of the tweets were utilized that contain the user ID and the tweet ID if it exists.We chose these fields because we wanted to model the user-to-user and tweet-to-tweet interactions to understand the conversation dynamics better.To achieve this, we constructed different graphical views of user interaction and tweets using these fields and analyzed their reply and quote interactions.We thus constructed and analyzed four types of graphs: tweet-quote, tweet-reply, user-quote, and user-reply.

We considered two types of components to analyze, namely weakly connected components (WCC) and strongly connected components (SCC).A weakly connected component is a subgraph where each node is reachable to every other node disregarding the direction of the edges.A strongly connected component is a subgraph where each node is reachable to every other node while considering the direction of the edges.Our analysis involving graphs was implemented using Python networkx library(Hagberg etal., 2008) and Gephi visualization(Bastian etal., 2009).

4.1. Topic Clustering

The keywords associated with positive and negative sentiment in each cluster can be seen in Table3 and Table4, respectively.Similar to how the clusters’ topics were found, we excluded the top 10 terms of each sentiment at the dataset level to reduce the number of overlapping terms across clusters.

In order to capture the keywords of each cluster, we fitted a TF-IDF matrix over the entire document (every tweet) and per cluster, then selected the top 10 keywords for the overall document and each cluster.This process removes the most frequent keywords at the document level from each cluster, allowing for more granular insight into each cluster.The document-level keywords are marked as overall at the top row of each table.However, some overlap between the keywords of different clusters can still be observed. These can be interpreted as the overall trend captured in our dataset that our keyword filtering process could not remove.

The overall sentiment change over time in each cluster can be seen in Figure3.

We found keywords such as “project,” “blockchain,” and “token” associated with positive sentiment, while keywords such as “hack,” “roll,” “bridge” are associated with negative sentiment.Many of the positive sentiments were originated from tweets expressing their gratitude towards developers.We found many tweets similar to “Great project” or “Thank you for this project” in positive sentiments.Looking into the negative sentiments, we found tweets expressing their distrust in “bridge hacks” and the security features of many bridge protocols.We also found many specific mentions to the “Ronin Bridge”, a project related to Axie Infinity, which was hacked in July(Kaaru, 2022) with a loss of over 600 million dollars.

We found many overlapping terms for positive and negative sentiment.Terms such as “security” and “roll ” appear in both sentiments, suggesting that multiple sub-clusters in this cluster express their trust and distrust of centralized exchanges regarding these topics.We also find that “binance” appears as one of the top positive terms, while “ftx” appears among the negative terms.Looking deeper, we find that many of the tweets with a positive sentiment that mentions Binance are thanking the CEO of Binance.In contrast, FTX is mostly mentioned in a negative context, especially after the official bankruptcy announcement.

As pointed out in our earlier analysis, cluster 4 comprises mostly short-form tweets.Many positive and negative keywords are either adjectives or verbs with strong sentiment attached, such as “nice,” “love,” “want” for positive sentiment, “malicious,” “hack,” and “bad” for negative sentiment.We also note that the term “security” appears in both positive and negative sentiment, suggesting that tweets in this cluster also discuss the security aspect of various projects in both positive and negative contexts.

In our earlier analysis of the topics, we found that this cluster is the least coherent in terms of the topics and contains many advertisem*nts for different blockchain assets/protocols.As a result of this, we did not find any specific mentions of projects or exchanges except for Binance, which was a very engaging topic at the time of our data collection.Our sentiment analysis supports the finding that most positive sentiments are associated with advertisem*nts, with keywords such as “just,” “join,” and “best” appearing in the top positive terms.

We found that cluster 6 has the highest number of specific names mentioned in the blockchain space associated with positive sentiments, such as “binance,” “kinderinu,” and “sushi”.Sushi is likely in reference to the Sushi Swap decentralized exchange, while Kinderinu (kinderinu.io) is a type of “meme” token similar to Dogecoin (dogecoin.com), which gained popularity among some investors.Looking deeper, we found that many tweets that mention the “kinderinu” project have the same content, similar to the Uniswap liquidity spam tweets mentioned in the Methodology section.Some of the tweets in this cluster were made under a coordinated attempt, although we could not find a source to verify.

4.2. Tweet Graph

The tweet graph is a directed graph in which each node is a tweet.A directed edge from tweet A to tweet B indicates that tweet A is a reply or quote to tweet B.We filtered for tweets that contained valid text to gather a subset of the dataset, which excluded retweets and tweets that refer to another tweet with a quote/reply.Consequently, each node in our graph is associated with its respective text content.Additionally, we computed RoBERTa sentiment scores for these texts. Given that each tweet (node) can quote or reply to at most one other tweet, resulting in at most one outgoing edge per node, our tweet graph tends to be sparse.Therefore, our analysis primarily concentrated on WCCs, as these represent more significant clusters or groups within the reply and quote dynamics of the graph.

Tweet-Reply Graph:The largest WCC (WCC-1) in this graph mostly comprises tweets with the same two hashtags about a token called “Leonicorn.”At first glance, this appears to be a group of spam tweets.However, many of the user accounts behind these tweets appear legitimate, with up to hundreds of followers and previous activity on Twitter.We conclude that this may be an example of a coordinated attempt by users to bring certain hashtags to the trending list of hashtags on Twitter.While the accounts appear to be regular Twitter users, this activity reveals another example of manipulation on Crypto Twitter.

The third, fourth, and tenth largest WCCs (WCC-3, WCC-4, WCC-10) reveal another interesting insight into spam activity on Crypto Twitter.There is a reply chain composed almost entirely of spam bots with regular users scattered throughout.The bot accounts appear to reply to seemingly randomly chosen users who discuss cryptocurrency-related topics.We also found instances where bots advertising different scams reply to each other in a chain that starts with a tweet from a genuine user.Many of these bots advertise an “exploit” the user can benefit from, such as Maximal Extractable Value (MEV) scripts, which exploit the ability to influence the order of transactions in a block to gain advantages, or an upcoming “strategy” such as a rug pull on an asset, where the developers of a cryptocurrency project suddenly withdraw all their funds from the liquidity pool or project wallet, abandoning the project and leaving investors with worthless tokens or assets.We found several instances where regular users attempt to respond to the bot tweets, only to be swarmed by further tweets advertising a different “exploit.”

The sixth largest WCC (WCC-6) is centered around a tweet announcing a giveaway event from Binance’s official account.While this tweet is legitimate, most replies are tweets from low-follower users that link to an external website in a likely scam attempt, which reveals another pattern of spam bots on Crypto Twitter.Unlike other spam tweets observed in the dataset, these tweets only mention the official Binance account.Instead of mentioning multiple users with a description of the “exploit,” these tweets use a few words such as “Interesting” or “Important.”We conclude that these tweets attempt to lead possible victims into visiting the link out of curiosity and then fall for the scam.Furthermore, we found that many of these links lead to deleted videos on YouTube, which was a popular format for sharing the “tutorials” on these “exploit” scams.

The second and fifth largest WCCs (WCC-2, WCC-5) appear to stem from Binance’s founder and former CEO Changpeng Zhao.WCC-2 is centered around a tweet from Zhao, which says “I think I am a poor speaker. I stutter all the time.”This tweet is met by a majority of positive support from the community, with the users cheering him on and expressing their faith in him.WCC-5 is centered around Zhao’s tweet “Rebuilding starts, to the moon.” announcing a rebuilding of the crypto community after the FTX incident.This tweet is met with mixed sentiment, with many users expressing distrust in the centralized exchange model.

Interestingly, when Binance’s official account tweets, those are met with much more negative responses.The seventh largest WCC (WCC-7) contains three tweets from Binance’s official account, which appear to be from a thread the account posted on its position in light of the FTX instance.Even though the tweets attempt to explain Binance’s decision not to support FTX, many tweets express the users’ discontent with Binance’s actions and distrust of the platform.Figure5 shows a visualization of WCC-7, where the tweets from Binance’s official account are the red ($\bullet$), blue ($\bullet$), and yellow ($\bullet$) larger nodes in the top left, center right, and center bottom of the diagram, respectively.

A tweet from Etherum’s founder, Vitalik Buterin, is found to be the central node of the ninth largest WCC (WCC-9).Interstingly, while Vitalik’s tweet mentions his doubts about the future of ZK-rollup, an overwhelming majority of the replies to this tweet are positive.We found that many of these replies are not related to the original tweet by Vitalik but are either promoting a likely scam or proposing ideas to him, many of which are classified as having a positive sentiment.

4.3. User Graph

We constructed two distinct user graphs: one based on quotes and the other on replies.These are also directed graphs, with nodes representing individual users and edges representing interaction between them.In the user-quote graph, a directed edge from user A to user B signifies that user B has quoted a tweet from user A.In the user-reply graph, a directed edge from user A to user B indicates that user B replied to a tweet from user A.Consequently, we filtered and created edges from only those instances that provide both.Unlike the tweet graph, some nodes in these graphs feature both inbound and outbound edges, along with self-loops.Thus, we turned our attention to the SCCs to analyze significant groups within both the reply and quote graphs.This analysis reveals diverse discussions, spam bot networks, and engagement levels in the conversations.

Table5 presents statistics for the generated graphs.Like the tweet graph, the user-reply graph exhibits higher density than the user-quote graph, suggesting a preference among users for replies as their interaction medium over quotes.We examined the five largest SCCs for both of the graphs.

User-Quote Graph:In the context of SCC-1, the largest component, discussions center around the event regarding the collapse of FTX(Reiff, 2023).This subnetwork is dominated by legitimate users, as evidenced by the high mean and median number of friends and followers (seeTable6).

SCC-2, the second-largest component, is smaller, with 25 nodes and 105 nodes reachable from it.Intriguingly, nearly all nodes, including reachable ones, are identified as spam bot users responsible for disseminating over 16,000 spam tweets, primarily glorifying AdsCoin (https://adsexchange.io).Initial manual examination of tweets confirmed their spam bot nature, and subsequent analyses discussed in later sections further support this claim.

The third-largest component (SCC-3), encompassing 21 nodes with 28 reachable nodes, unexpectedly diverges from crypto-related discussions.The dialogue here revolves around the “Save Kashmir movement” in India.However, since our dataset is filtered with keywords such as attacks, security, etc., relevant to crypto and non-crypto contexts, these data inadvertently entered the dataset, albeit unrelated to the crypto domain.

On a smaller scale, SCC-4, the fourth-largest component, comprising only 11 nodes, has a relatively larger number of reachable nodes (358).In this subnetwork, individuals associated with Cosmos (cosmos.network), a cryptocurrency powering a network of blockchains that can communicate with each other, engage in diverse topics.Unlike a singular discussion focus, Cosmos serves as the unifying element bringing nodes together in this subnetwork.

SCC-5, the smallest component with seven and four reachable nodes, lacks a specific central theme.Instead, it presents a microcosm of a social network where a small group of connected individuals interact on a range of random topics by quoting each other’s tweets.

User-Reply Graph:The largest Strongly Connected Component (SCC-1) within the user-reply graph, comprising a substantial number of nodes (231,841), is centered around Binance (www.binance.com), the world’s largest cryptocurrency exchange by trading volume.The majority of accounts within this component are associated with Binance, engaging in discussions on various topics, such as the decision not to pursue the potential acquisition of FTX and attending the Fintech summit in Indonesia.Although this subgraph is not entirely free from bots, the predominant user base consists of legitimate participants discussing diverse aspects primarily related to Binance.

Interestingly, the second and fourth largest SCCs (SCC-2 and SCC-4) focused on a power outage in Nigeria in November 2022.Tweets with the highest engagement in SCC-2 originate from the official account of Ikeja Electric (www.ikejaelectric.com), Nigeria’s largest Electricity Distribution Company, providing updates and services during the outage.Similarly, in SCC-4, discussions revolve around another company, Abuja Electricity Distribution Company (www.abujaelectricity.com).Like the discussions involving SCC-2, these are also focused on updates and services during that power outage.Although these discussions are not directly related to crypto, Ikeja Electric does mention different tokens, such as energy tokens, which are relevant to their business.Notably, the mean followers count for nodes within SCC-2 is significantly lower than that of nodes reachable from SCC-2, deviating from the expected trend (see Table7).Further investigation revealed that Elon Musk, who has a very large number of followers, is reachable from SCC-2.A more in-depth examination reveals the involvement of a spam bot connecting users discussing the power outage and those related to crypto.Elon Musk replied to a tweet from Chainlink, and a user connected with Chainlink interacted with the spam bot.This way, Elon Musk entered this network, and his substantial follower base caused the anomaly in the mean followers count.Other than this deviation, the subnetwork appears normal, although the context in focus is not very related to crypto.However, SCC-4 doesn’t show such deviation and follows the normal trend.

The third largest SCC (SCC-3) is found to be centered around a Binance bot that posted numerous tweets about the CR7 NFT Giveaway in a short timeframe.This is the same bot network as one of the identified botnets in the tweet-reply graph.

The fifth-largest SCC (SCC-5) also constitutes a spam bot network, where users predominantly post hashtags and random spam replies.Users here were active for a very short period and posted the tweets and replies in just one day.Manual verification of users’ tweet and reply activities in this subnetwork confirmed their bot nature.It is worth noting that Twitter has suspended most of these users due to their spamming activities.

The temporal dynamics of user interactions within the SCCs shed light on the evolving nature of discussions over time.For each of the top 5 SCCs, we observed how the number of data points involved with the SCC nodes changes. To be more specific, for each of the top 5 SCCs, at each timestamp $T$, we counted the number of samples having timestamps less than or equal to $T$ that are connected to/from any of the SCC nodes.

Figure6 and Figure7 illustrate tweet counts over distinct timestamps, thus offering a comprehensive view of the evolution of conversations.While most SCCs of the user-quote graph exhibit a non-linear pattern that mirrors the dynamic nature of real-life discussions with peaks and lulls, SCC-2 (the red (—) curve) deviates by showcasing a distinctive linear trend.This synchronous and unusual linearity aligns with the identified characteristics of SCC-2 as a network primarily composed of spam bot users.SCC-2 and SCC-5 of the user-reply graph also show distinctive patterns for spam bot networks.The temporal analysis captures the rhythm of discussions and provides a visual cue to discern patterns that distinguish bot-driven activity from organic, user-driven discourse.

Examining the number of friends and followers within the SCCs and nodes reachable from them accentuates the distinctions (see Tables6 and7).The friends and followers counts within most SCCs of the user-quote graph significantly exceed those of reachable nodes.Notably, SCC-2 follows a distinct trend where the counts within the SCC and the reachable nodes are small and comparable, with the reachable nodes even exhibiting a higher median friends count than the SCCs.This divergence provides additional evidence supporting the characterization of SCC-2 as a bot network.These particular statistics in the user-reply graph also follow a similar pattern.However, SCC-3 with 0 as all the counts indicate an anomaly supporting the previous claim that this subnetwork consists of bots.The anomalous small counts in SCC-5 and insignificant difference in the counts between the nodes within SCC and the nodes reachable from SCC indicate that this subnetwork is also a spam botnet, reinforcing the previously discussed insight about SCC-5.These friends and followers count within the SCCs and those reachable from them also reveal a concentration of interaction.In general, the counts within an SCC surpass those of reachable nodes, indicating that the primary discourse occurs among influential users.However, the reachable nodes are not negligible users; they serve as observers, sharing tweets through quote tweeting.More followers than friends signifies the natural trend for influential users.