US Army Data-Driven Bot Detection
Inauthentic, bot-like activity on social media can skew metrics making it tricky to get an accurate representation of engagement. This is an issue for Public Affairs Officers (PAO) of the United States Army's Aviation & Missile Center (AVMC), my client for this project.
To solve this problem, I designed and developed a web-based application that automates the process of identifying and removing bot activity, significantly reducing the time and effort expended by the PAO. The resulting application is highly effective in capturing high-fidelity data, providing a more authentic picture of social media engagement.
ROLE
Lead UX Designer & Researcher
TIMELINE
Fall 2022
CLIENT
US Army, Aviation & Missile Center
CLASS
Hacking for Defence
THE PROBLEM
Metrics skewed by bot activity hinder authentic engagement measurement and confuse decision-making.
AVMC PAO are often tasked with measuring results from various campaigns on social media sites such as Facebook, LinkedIn, and Twitter. They capture metrics that demonstrate impressions such as reach, engagements, likes, and comments. Unfortunately, many of the metrics gathered by PAO are skewed by inauthentic, bot-like activity on the sites (screenshots of the problem below).
AVMC wants a better way to discern and capture real versus artificial engagement. As AVMC PAO generates and disseminates activity reports across the organization, the data shared is neither high fidelity nor particularly useful because of artificial users. This interference skews results, levels of effort, and applicability from other units in AVMC. When briefed, senior leadership has responded ambiguously to the inconsistent metrics in monthly reports. Their inability to create rules for the activity results in even greater confusion regarding engagement results.
QUALITATIVE & QUANTITATIVE RESEARCH
To understand the problem thoroughly, I employed three research methods: interviews, data analysis and a competitive analysis
Interviews
I interviewed 13 people, including members in AVMC and in other organisations who were experiencing similar problems with their social media accounts.
I asked them about their current experiences with bot activity, the issues they run into, how they currently handle those issues, and a better understanding of their day to day responsibilities and tools used. These interviews provided me a better overall picture and understanding of the bot problem.
Data Analysis
The client provided Excel Sheets containing information they track on their social media sites, including examples of bot activity.
I also did my own investigation and searched through their social media accounts for further bot activity and noted down any findings. This was helpful to see the bot problem in context.
Competitive Analysis
I did research on the current tools available on the market that attempt to solve the problem of bot activity. I also looked into Sprinklr, the content management tool that my client is currently using, to compare the other tools to.
This allowed me to better grasp what technology is currently available that solves the problem and what is lacking and needed to solve the problem better.
INSIGHTS
I used an affinity diagram to identify shared opinions, thoughts, and issues among PAOs, revealing key insights and common themes
Insight #1
There is currently no established protocol or software to effectively detect and eliminate bots.
Insight #2
Facebook and Instagram tend to experience greater bot activity than other social media platforms, such as Twitter and LinkedIn.
Insight #3
Bot-generated engagement can skew the accuracy of engagement metrics and misrepresent user behavior
Insight #4
Identifying and removing bots manually is a time-intensive process.
Insight #5
Users have gained a familiarity with certain characteristics that are indicative of bot behavior, enabling them to better recognize bot activity.
Insight #6
It is commonplace for users to take measures such as removing, blocking, hiding, or reporting bot activity as a means of maintaining a genuine online experience.
IDEATION
I chose to create an Information Architecture to define the navigation, content hierarchy, and overall flow of the user interface
IDEATION
Progressing from ideation to wireframes, I facilitated meaningful discussions with the client, effectively translating ideas into tangible design concepts
Homepage
An analytics dashboard illustrating the bot activity on the social media sites.
Bot History
Table populated with bots detecting on social media sites. Ability to block, remove, or report activity.
Bot Settings & Preferences
Some users noted that they would like the platform to handle all bot activity whereas some users wanted to still be in control of how the bots are handled. Here is where they can select their preference.
Connect Social Media
Ability for users to connect multiple social media accounts.
TESTING
Speed-dated mid-fidelity prototypes with target users to rapidly gather feedback and iterate over concept
PROTOTYPING
Bringing the validated high-fidelity designs to life with HTML, CSS, and JS.
WIP! Follow my progress here: https://annarippert.github.io/bot-free/index.html
REFLECTION
What I learned
👩🏼💻
Being able to collect and analyze data is key to creating better designs and user experiences. Users do not always know what they want, so it is important to be able to analyze all the information you can gather to help determine what the user really needs.
Keeping the client engaged throughout the process helps us make better decisions. I made sure to stay in touch with the client every week, sharing updates and asking for their feedback on the project's progress.
✍🏻