Capturing Authentic Engagement: A Bot-Blocking Solution
OVERVIEW
Inauthentic, bot-like activity on social media can skew metrics making it tricky to get an accurate representation of engagement. This is an issue for Public Affairs Officers (PAO) of the United States Army's Aviation & Missile Center (AVMC), my client for this project.
​
To solve this problem, I designed and developed a web-based application that automates the process of identifying and removing bot activity, significantly reducing the time and effort expended by the PAO. The resulting application is highly effective in capturing high-fidelity data, providing a more authentic picture of social media engagement.
ROLE​
Lead UX Designer & Researcher
TIMELINE​
Fall 2022
CLIENT​
US Army, Aviation & Missile Center
CLASS
Hacking for Defence
The Problem
​
AVMC PAO are often tasked with measuring results from various campaigns on social
 media sites such as Facebook, LinkedIn, and Twitter. They capture metrics that 
demonstrate impressions such as reach, engagements, likes, and comments. Unfortunately, many of the metrics gathered by PAO are skewed by inauthentic, bot-like activity on the sites (screenshots of the problem below).
​
​
AVMC wants a better way to discern and capture real versus artificial engagement. As AVMC PAO generates and disseminates activity reports across the organization, the data shared is neither high fidelity nor particularly useful because of artificial users. This interference skews results, levels of effort, and applicability from other units in AVMC. When briefed, senior leadership has responded ambiguously to the inconsistent metrics in monthly reports. Their inability to create rules for the activity results in even greater confusion regarding engagement results. This led me to the following problem statement:
The Challenge:
How might we develop a more efficient system for detecting and eliminating bot-like or inauthentic activity on social media platforms to capture high-fidelity data that accurately gauges authentic social media engagement?
Research
I used three different research methods to learn more about the problem.
​
​
Competitive Analysis​
I did research on the current tools available on the market that attempt to solve the problem of bot activity. I also looked into Sprinklr, the content management tool that my client is currently using, to compare the other tools to.
This allowed me to better grasp what technology is currently available that solves the problem and what is lacking and needed to solve the problem better.
Data Analysis​
The client provided Excel Sheets containing information they track on their social media sites, including examples of bot activity.
I also did my own investigation and searched through their social medial accounts for further bot activity and noted down any findings. This was helpful to see the bot problem in context.
​
Interviews
​
I interviewed 13 people, including members in AVMC and in other organisations who were experiencing similar problems with their social media accounts.
I asked them about their current experiences with bot activity, the issues they run into, how they currently handle those issues, and a better understanding of their day to day responsibilities and tools used. These interviews provided me a better overall picture and understanding of the bot problem.
​
Insights
​
I constructed an affinity diagram to build a collective understanding of PAO shared opinions, thoughts and issues where several insights and common themes emerged.
​
Key Takeaways
​
After affinity diagramming, I was able to discover and validate the following key takeaways.
There is currently no established protocol or software to effectively detect and eliminate bots.​
1.
​Identifying and removing bots manually is a time-intensive process.
2.
Facebook and Instagram tend to experience greater bot activity than other social media platforms, such as Twitter and LinkedIn.
3.
​Users have gained a familiarity with certain characteristics that are indicative of bot behavior, enabling them to better recognize bot activity.
4.
​Bot-generated engagement can skew the accuracy of engagement metrics and misrepresent user behavior
5.
​It is commonplace for users to take measures such as removing, blocking, hiding, or reporting bot activity as a means of maintaining a genuine online experience.
6.
Information Architecture ​
Before wireframing, I mapped out the information I wanted to include on each screen. This process helped me gain a better understanding of the types of screens needed and allowed me to plan out their organization and structure within the app more effectively.
Wireframing
​
I began wireframing the solution to enable quick iteration and refinement as I tested it with users. Additionally, wireframing facilitated clear communication of my ideas to both the client and target users, without overwhelming them with complex details. For the initial design approach, I opted to focus on the laptop experience since all users I interviewed utilized their laptops to analyze their social media accounts and create reports.
​
Homepage
An analytics dashboard illustrating the bot activity on the social media sites. For example, how many bots have been detecting this month.
Bot History
Table populated with bots detecting on social media sites. Ability to block, remove, or report activity.
Bot Settings & Preferences
Some users noted that they would like the platform to handle all bot activity whereas some users wanted to still be in control of how the bots are handled. Here is where they can select their preference.
Connect Social Media
Ability for users to connect multiple social media accounts.
Solution
​
Once the wireframes were validated and iterated upon, I proceeded to create mid-fidelity prototypes. An interactive Figma prototype was developed and handed off to the client. Below is a walkthrough of the High-Fidelity solution:
​
Next Steps
​
​
​
I have provided the client documentation suggesting how to build out the ML algorithm. Once the ML algorithm has successfully been implemented, the client may move forward with implementing the front-end and back-end of the application.