Capturing Authentic Engagement: A Bot-Blocking Solution
OVERVIEW
Inauthentic, bot-like activity on social media can skew metrics making it tricky to get an accurate representation of engagement. This is an issue for Public Affairs Officers (PAO) of the United States Army's Aviation & Missile Center (AVMC), my client for this project.
To solve this problem, I designed and developed a web-based application that automates the process of identifying and removing bot activity, significantly reducing the time and effort expended by the PAO. The resulting application is highly effective in capturing high-fidelity data, providing a more authentic picture of social media engagement.
ROLE
Lead UX Designer & Researcher
TIMELINE
Fall 2022
CLIENT
US Army, Aviation & Missile Center
CLASS
Hacking for Defence
The Problem
AVMC PAO are often tasked with measuring results from various campaigns on social media sites such as Facebook, LinkedIn, and Twitter. They capture metrics that demonstrate impressions such as reach, engagements, likes, and comments. Unfortunately, many of the metrics gathered by PAO are skewed by inauthentic, bot-like activity on the sites (screenshots of the problem below).
AVMC wants a better way to discern and capture real versus artificial engagement. As AVMC PAO generates and disseminates activity reports across the organization, the data shared is neither high fidelity nor particularly useful because of artificial users. This interference skews results, levels of effort, and applicability from other units in AVMC. When briefed, senior leadership has responded ambiguously to the inconsistent metrics in monthly reports. Their inability to create rules for the activity results in even greater confusion regarding engagement results.
The Challenge:
How might we develop a more efficient system for detecting and eliminating bot-like or inauthentic activity on social media platforms to capture high-fidelity data that accurately gauges authentic social media engagement?
Research
I used three different research methods to learn more about the problem.
Competitive Analysis
I did research on the current tools available on the market that attempt to solve the problem of bot activity. I also looked into Sprinklr, the content management tool that my client is currently using, to compare the other tools to.
This allowed me to better grasp what technology is currently available that solves the problem and what is lacking and needed to solve the problem better.
Data Analysis
The client provided Excel Sheets containing information they track on their social media sites, including examples of bot activity.
I also did my own investigation and searched through their social media accounts for further bot activity and noted down any findings. This was helpful to see the bot problem in context.
Interviews
I interviewed 13 people, including members in AVMC and in other organisations who were experiencing similar problems with their social media accounts.
I asked them about their current experiences with bot activity, the issues they run into, how they currently handle those issues, and a better understanding of their day to day responsibilities and tools used. These interviews provided me a better overall picture and understanding of the bot problem.
Insights
I constructed an affinity diagram to build a collective understanding of PAO shared opinions, thoughts and issues where several insights and common themes emerged.
Insight #1
There is currently no established protocol or software to effectively detect and eliminate bots.
Insight #4
Identifying and removing bots manually is a time-intensive process.
Insight #2
Facebook and Instagram tend to experience greater bot activity than other social media platforms, such as Twitter and LinkedIn.
Insight #5
Users have gained a familiarity with certain characteristics that are indicative of bot behavior, enabling them to better recognize bot activity.
Insight #3
Bot-generated engagement can skew the accuracy of engagement metrics and misrepresent user behavior
Insight #6
It is commonplace for users to take measures such as removing, blocking, hiding, or reporting bot activity as a means of maintaining a genuine online experience.
Information Architecture
Before wireframing, I mapped out the information I wanted to include on each screen. This process helped me gain a better understanding of the types of screens needed and allowed me to plan out their organization and structure within the app more effectively.
Wireframing
I began wireframing the solution to enable quick iteration and refinement as I tested it with users. Additionally, wireframing facilitated clear communication of my ideas to both the client and target users, without overwhelming them with complex details. For the initial design approach, I opted to focus on the laptop experience since all users I interviewed utilized their laptops to analyze their social media accounts and create reports.
Homepage
An analytics dashboard illustrating the bot activity on the social media sites. For example, how many bots have been detecting this month.
Bot History
Table populated with bots detecting on social media sites. Ability to block, remove, or report activity.
Bot Settings & Preferences
Some users noted that they would like the platform to handle all bot activity whereas some users wanted to still be in control of how the bots are handled. Here is where they can select their preference.
Connect Social Media
Ability for users to connect multiple social media accounts.
Mid-fidelity wireframes
Once the low-fidelity wireframes were established, I began refining the interfaces to help present the interface and solution to the client.
High-fidelity wireframes
After presenting the mid-fidelity prototype to the client, we worked together to finalise key design decisions. Below is a sample of the changes made and the reasons for those changes on the high-fidelity version.
Implementation
Coming soon!
What I learned
👩🏼💻
Being able to collect and analyze data is key to creating better designs and user experiences. Users do not always know what they want, so it is important to be able to analyze all the information you can gather to help determine what the user really needs.
Keeping the client engaged throughout the process helps us make better decisions. I made sure to stay in touch with the client every week, sharing updates and asking for their feedback on the project's progress.
✍🏻