The Fifth Layer: How Transparency Reports Help Provide Actionable Insights for Online Communities

Written by

In the fifth installment of our Five Layers of Community Protection blog series, we dive deeper into how producing meaningful Transparency Reports both internally and externally can help your organization measure and guide community management decisions to stay ahead of evolving trends and legislation.

Layer Five: Transparency Reports

The Five Layers of Community Protection are a holistic approach to health and safety in digital communities. In prior blogs, we learned about the first four layers: 

By applying all five layers to their digital communities, platforms maintain their commitment to safety by design and create an inclusive community where all users are free to interact without fear of discrimination or harassment.  

Layer 5 is focused on better understanding and measuring the health of your community over time. Producing Transparency Reports will build trust within your organization and with your community. 

Why It’s Important  

The world is not static, and neither is your online community. Monitoring and maintaining your community help to keep it healthy and safe as it continues to grow and evolve. Data can be one of your most valuable assets in evaluating the wellness of your community. It can inform improvements to your customer experience and help guide your community’s success.  

An active and engaged community builds the foundation and potential for a successful business. Releasing data that encapsulates the areas of focus and outcomes of your Trust & Safety operations demonstrates your organization’s commitment to your community’s continued health, wellness, and safety which, as we know, directly corelate to player engagement and retention.  

Data Drives Decisions  

Data and insights are critical components for making better community management decisions. At Two Hat, we know we are adding true value for our customers when we provide them with strategic insights into their community, which becomes a leading indicator of their company’s future growth.  

Through Two Hat’s reports, our customers have: 

  • Spotted & addressed inconsistencies in moderator decisions. This resulted in meaningful improvements to their community operations. 
  • Observed an increase in subversive chat, which led them to discover bot activity that was disrupting their community. 
  • Delivered new insights during staff meetings, highlighting all the negative content they have protected their community members from. 

Accessibility is Key 

Releasing Transparency Reports frequently and making them accessible to everyone on your platform is key to building trust within your community. Inclusive reports are formatted so that they are accessible to all members and written using plain language to ensure comprehension. By making these reports inclusive and easily accessible, your organization is letting ALL users know that they’re an integral part of your community and that you are taking them into consideration in all your initiatives.  

Increasing Transparency  

Transparency reports demonstrate to users that your organization is taking strong, proactive action against inappropriate and harmful content. By doing so, you’ll foster a community of trust and openness, which keeps users engaged and promotes a powerful brand image that values its community members.  

Major technology and communications companies, such as Google, Reddit and Microsoft, regularly publish transparency reports, which disclose the amount of data that government agencies request over time and how the data is used. This ensures that platform users maintain visibility and oversight into whom their data is being shared with and when.  

Additionally, transparency reports help organizations stay compliant with existing user privacy rules and regulations, such as the General Data Protection Regulation (GDPR), the Children’s Online Privacy Protection Act (COPPA) and Communications Decency Act (CDA). These reports also help anticipate evolving legislation by disclosing information to and building trust with your users before your company is required to do so. 

A Holistic Approach

To learn how you can holistically build a healthy and thriving community, we encourage you to read the rest of our Five Layers of Community Protection blog series. You can also request a live demo of our Community Sift platform to learn how you can you protect your online community and improve your users’ online experiences. 

Layer Four: How User Reporting Can Help Peers Prevent Harm

Written by

In the fourth installment of our Five Layers of Community Protection blog series, we dive deeper into how Automated User Reporting leads to organizations building healthier and safer communities by giving users a way to report when they’re uncomfortable and helping moderators prioritize actionable items.

Layer Four: User Reporting

We previously learned about Layer 3 and how to leverage User Reputation to more elegantly assess context and deploy the appropriate filter settings.

Layer 4 is focused on creating access for community members to report content they find harmful and/or which should have been blocked. This is both highly empowering to your community, and provides you with valuable feedback on the grey area, the content that requires human expertise, empathy, and insights in order to be fully assessed.

Act Quickly

Communicating your organization’s zero-tolerance policy regarding harmful content, enforcing it, and doing so quickly, shows your community you value them and their reporting. This builds credibility and trust. Following up quickly also discourages violators from acting again, as well as their copycats. Prompt resolution also helps protect a brand’s reputation and increases user engagement and retention.

The Long Backlog

Many community managers and moderators, especially those in large organizations, face a backlog of automated user reports. Small organizations may have only one person handling hundreds of reports. This backlog can hinder a team’s ability to address the most heinous reports in a timely manner. Leveraging AI software, in addition to human review and insight, can help moderators triage priority reports and close out any false or non-urgent ones. Moderators can then act quickly on the actionable and most severe reports that rise through the noise.

Making it Easy & Accessible for Members to Report Harmful Content

The content that doesn’t quite meet your cut-off threshold for proactive filtering is the harder-to-moderate grey area. This is the area that allows for healthy conflict and debate as well as for the development of resiliency, but it also brings the opportunity for critical input from your community so you better understand their needs. It’s critical to make it easy and straightforward for your community to report content that has made them feel unsafe. That means adding an intuitive reporting flow to your product that allows your community members to choose from a list of reporting reasons and provide actionable proof for the report they are sending you.

Get to the Critical Reports First

A critical insight we consistently see across verticals like social apps, gaming, and others is that 30% or less of all user reports are actionable. It’s absolutely essential to have a mechanism to sort, triage, and prioritize reports. Clearly, not all reports have the same level of importance or urgency.

Closing The Feedback Loop

Following up on reports lets concerned users know that their reports led to meaningful action and encourages them to continue to report behavior that violates the platform’s community guidelines. This helps to build trust by assuring users that you are doing your due diligence to keep your community a safe and inclusive place for all.

It’s important to thank community members for submitting reports and helping you maintain a healthy and safe community. A simple “thank you” can go a long way in building relationships with the members of your community as it shows that you take violations of community guidelines seriously.

Improving Scalability + Sustainability

Successfully applying Layer 2: Classify & Filter means that the majority of harmful content is filtered out in accordance with a platform’s community guidelines. When the worst of the worst content is filtered out, there is less negative behavior for community members to report on. This impacts Layer Four directly as it leads up to a 88% reduction* of the number of user-generated reports. This thereby increases the scalability and sustainability of the volume of content moderators need to monitor and helps to decrease the likelihood of burnout.

Empowering Users

Optimizing User Reporting operations empowers community members to take an active role in their safety and the community’s health and helps to build trust with the platform. Leveraging AI helps community managers reduce their workload and prioritize high-risk and sensitive reports. In turn, responding quickly to urgent reports and closing the feedback loop builds trust and credibility.

To find out how else you can better build healthy and thriving communities, read the rest of our Five Layers of Community Protection blog series. You can also request a live demo of our Community Sift platform now to learn how Automated User Reporting can help you protect your online community, address user reports faster and improve your users’ online experiences.

Learn about Layer 5 here.

*Two Hat Analysis 2020

Five Wellness Tips for Community Managers

Written by

Building healthy and safe digital spaces begins with healthy community managers and moderators. We need to help community managers be mindful and take care of their mental health as they are often exposed to some of the worst of the internet – on a daily basis.

Occupational burnout is an all-too-common result that we, as an industry, must highlight and focus on changing. Identifying job stress and giving employees flexibility to prioritize their wellbeing improves our communities.

We suggest that companies encourage community managers to follow these 5 tips to prioritize their wellness and resilience:

1 – Create a wellness plan

Community managers are often repeatedly exposed to the worst online behaviors and are left feeling emotionally drained at the end of the workday. A wellness plan helps them manage their stress and mentally recharge. This actionable set of activities helps community managers to take wellness breaks throughout the day and to create a buffer between work and their personal lives. Whether it’s taking a walk outside, listening to music, meditating, talking to family or friends, a wellness plan can help community managers decompress before transitioning to the next moment of their day.

2 – Leverage AI Plus

Community managers monitor for hate speech, graphic images, and other types of high-risk content. Prolonged exposure to traumatic content can severely impact an individual’s mental health and wellbeing. Content moderators can develop symptoms of P.T.S.D., including insomnia, nightmares, anxiety, and auditory hallucinations as a result of consistent exposure to traumatic content.

By proactively leveraging technology to filter content, reducing the exposure to human moderators, our partners have reduced the workload of their community managers by as much as 88%*. This gives community managers more time to focus on other aspects of their job and protects their wellbeing by minimizing the amount of time they’re exposed to high-risk content.

3 – Be mindful of the types of content you’re moderating for

Rotating the types of content for which each team member is monitoring can help alleviate the negative impact that constant exposure to a singular focus area may cause. Threats of harm and self-harm, racism, sexism, predatory behavior, and child grooming are just a few of the types of content community managers monitor for and are exposed to daily.

4 – Focus on the positive

Most chat, images, and videos in online communities are aligned with the intended experiences of those products. In our experience, about 85% of user-generated content across different verticals is what we classify as low-risk, very positive types of behavior. Think community members discussing matters pertinent to the community, their hobbies and passions, or sharing pictures of their pets. Focusing on the positive side of your community will help you keep this reality in mind, and also remember why you do what you do everyday.

One of the ways in which you can focus on the positive aspects of your community is spending time in your product and seeing how community members are engaging, their creativity and passion. Make a point to do that at least once a week with the intent of focusing on the positive side of the community. Similarly, if you leverage a classification and filtering system like Community Sift, you should dedicate time to looking at chat logs that are positive. After either of these activities, you should write down and reflect on 3 to 5 things that were meaningful to you.

5 – Remember you’re making an impact

Monitoring an endless stream of high-risk content can make community managers feel like their work isn’t making an impact. That couldn’t be further from the truth. Their work is directly contributing to the health and safety of social and online play communities. When community managers identify a self-harm threat or protect children from predators, they are immediately making an impact in the life of that individual. In addition to monitoring content, community managers help to ensure that users have a positive and happy experience when engaging with their platform.

According to a 2020 survey conducted by the Anti-Defamation League, 81 percent of U.S. adults aged 18-45 who played online multiplayer games experienced some form of harassment. Approximately 22% of those community members went onto quit an online platform because of harassment they experienced. Harassment is an issue actively driving community members away from engaging with their favorite platforms. By helping to create a safe and healthy space, community managers are creating an environment where individuals can make friends, feel like they belong to a community, and have overall positive social interactions without the fear of harassment – while also helping drive the success of the community and overall acquisition and retention metrics. A true win-win.

Help protect the well-being of your community managers. Request a demo today  to see how Two Hat’s content moderation platform can reduce your community manager’s workload and exposure to harmful content.

Source:
* Two Hat Customer analysis, 2020

Digital Safety: Combining the Best of AI Plus Human Insight

Written by

As AI and machine learning technologies continue to advance, there is increasingly more hype – and debate – about what it can or cannot do effectively. On June 29, the World Economic Forum released a pivotal report on Digital Safety. Some of the challenges identified in the report are:

  • The pandemic created challenges for countering vaccine misinformation.
  • January 6 (storming the US Capital) has necessitated a deeper look into the relationship between social platforms and extremist activity.
  • Child exploitation and abuse material (CSEAM) has continued to spread online.

Internationally, the G7 has committed to grow the international safety tech sector. We at Two Hat have made several trips to the UK pre-pandemic to provide feedback on the new online harms bill. With attention on solving online harms on the rise, we are excited to see new startups enter the field. Over 500 new jobs were created in the last year and the industry needs to continue attracting the best technology talent to solve this problem.

AI is a Valuable Component

For many, AI is a critical part of the solution. As the largest digital safety provider, we alone handle 100B human interactions a month. To put that in scale, that is 6.57 times the volume of Twitter. If a human could review 500 items an hour you would need 1.15 million humans to review all that data. To ask humans to do that would never scale. Worse, human eyes would gloss over and miss things. Alternatively if they only saw the worst we would be subjecting humans to days filled with looking at beheadings, rape, child abuse, harassment and many other harms leading to PTSD.

AI Plus Humans is the Solution

One of our mantras at Two Hat is, “Let humans do what humans do well and let computers do what they do well.” Computers are great at scale. Teach it a clear signal like “hello” is good and “hateful badwords” are bad and it will scale that to the billions. Humans, however, understand why those “hateful badwords” are bad. They bring empathy, loosely connected context, and they can make exceptions. Humans fit things into a bigger picture while machines (as magical as they may seem) are just following rules. We need both. Thus a human feedback loop is essential. Humans provide the creativity, teach the nuances, are the ethics committee, and stay on top of emerging trends in language and culture. According to Personabots CEO, Lauren Kunze, internet trolls have tried and failed to corrupt Mitsuku, an award-winning chatbot persona, on several occasions due to human supervisors being required to approve any knowledge retained globally by the AI.

We also need multiple forms of AI. “If all you have is a hammer everything looks like a nail” – modern proverb. A common mistake we see is people relying too much on one form of AI and forgetting the other.

Let’s consider some definitions of several parts of AI:

  • Artificial Intelligence refers to any specialized task done by a machine. This includes machine learning and expert systems.
  • Expert System refers to systems that use databases of expert knowledge to offer advice or make decisions.
  • Machine Learning refers to a machine that was coded to learn a task ‘on its own’ from the data it was given, but the decisions it makes are not coded.
  • Deep Learning refers to a specific form of machine learning, which is very trendy at the moment. This type of machine learning is based on ‘deep’ artificial neural networks.

To avoid the “everything looks like a nail” we use Expert Systems, Machine Learning, and Deep Learning within our stack. The trick is to use the right tool at the right level and to have a good human feedback loop.

And, given that we only view AI as a contributor to the solution vs The Solution – it allows us to see the screws and other “non-nails” and use humans and other systems and methods more effectively than the AI hammer to solve those issues.

Don’t Leave It To Chance

There was a great article by Neal Lathia where he reminds us that we shouldn’t be afraid to launch a product without machine learning. In our case, if you know a particular offensive phrase is not acceptable in your community, you don’t need to train a giant neural network to find it. An expert system will do. The problem with a neural network in this case is that you’re leaving it to chance. You’re feeding examples of it into a black box , it begins to see it everywhere, perhaps where you don’t want it. If you give it more examples that are mislabelled or even just too many counterexamples, it may ignore it completely.

At this point we learn something from antivirus companies that impacted how we’ve modelled our company.

  1. Process 100 billion messages a month
  2. be aware of new patterns that are harming one community
  3. have humans write manual signatures that are well vetted and accurate
  4. roll that out proactively to the other communities.

Determined Users will Subvert Your Filter

“The moment you fix a new problem, the solution is obsolete.” Many think the problem is “find badword”, not realizing the moment they find “badword” then users change their behaviour and no longer use it. Now they use “ba.dword” and “b4dw0rd”. When you solve that, they move on to “pɹoʍpɐq” and “baᕍw⬡rd” and somehow hide “badword” inside “goodword” or in a phrase. After 9 years we have so many tests for these types of subversions that would make you want to give these guys an honorary phD in creative hacking.

However if you rely on logical rules alone to find “badword” in all its many subversive forms you run the risk of missing similar words. For instance, if you take the phrase “bad word” and feed it into a pre-trained machine learning model to find words that are similar, you get words like “terrible”, “horrible, and “lousy”. In the antivirus analogy, humans use their imagination to create a manual signature. They might find “badword” is trending but did they consider “terrible”, “horrible”, “lousy”. Maybe – maybe not; it depends on their imagination. This is not a good strategy if missing “lousyword” means someone may commit suicide. Obviously we are not really talking about “lousyword”, but things that really matter.

The Holistic 5 Layer Approach to Community Protection:

How do you get all your tools to work together? Self-driving cars have a piece of the answer. In that context, if the AI gets it wrong someone gets run over. To resolve that, manufacturers mount as many cameras and sensors as they can. They train multiple AI systems and blend them together. If one system fails another takes over. My new van can read the lines on the side of the road and “assist” me by turning the corner on the highway. One day I was coming home from skiing with my kids in the back and it flashed at me telling me humans were required.

To scale to billions of messages we need that multi-layered approach. If one layer is defeated there is another behind it to back us up. If the AI is not confident, it should call in humans and it should learn from them. That is why Community Sift has 5 Layers of Community Protection. Each layer combines AI plus human insight, using the best of both.

  • Community Guidelines: Tell your community what you expect. In this way, you are creating safety via defining the acceptable context for your community. This is incredibly effective, as it solves the problem before it’s even begun. You are creating a place of community so set the tone at the door. This can be as simple as a short page of icons of what the community is about as you sign up. You can learn more about designing for online safety here. Additionally, our Director of Trust & Safety, Carlos Figueiredo, consults clients on setting the right community tone from the ground up and creating community guidelines as the foundational element to community health and safety operations.
  • Classify and Filter: The moment you state in your Community Guidelines that you will not tolerate harassment, abuse, child exploitation and hate, someone will test you to see if you really care. The classify and filter line of defense backs up your promise that you actually care about these things by finding and removing the obviously bad and damaging. Think of this like anti-virus technology but for words and images. This should focus on what are “deal-breakers” to your company; things once seen that cannot be unseen. Things that will violate the trust your community has in you. Just like with anti-virus technology, you use a system that works across the industry so that new trends and signatures can keep you safe in real-time.
  • User Reputation: Some online harm occurs in the borderline content over several interactions. You don’t want to over-filter for this because it restricts communication and frustrates normally positive members of your community. In this layer we address those types of harm by building a reputation on each user and on each context. There is a difference between a normally positive community member exceptionally sharing something offensive and a bad actor or a bot willfully trying to disrupt normal interactions. For example, it may be okay that someone says “do you want to buy” once. It is not okay if they say it 20 times in a row. In a more advanced sense, everything about buying is marked as borderline spam. For new and long standing users that may be allowed. But for people or bots that misuse that privilege, it is taken away automatically and automatically re-added when they go back to normal. The same principle works for sexual harassment, hate-speech, grooming of children, and filter manipulations. All those categories are full of borderline words and counter statements that need context. If context is King then reputation is Queen. Working in concert with the other two layers, user reputation is used to discourage bad actors while only reinforcing the guidelines for the occasional misstep.
  • User Report Automation: Even with the best technology in the above three layers, some things will get through. We need another layer of protection. Anytime you allow users to add content, allow other users to report that content. Feedback from your community is essential to keep the other three layers fresh and relevant. As society is continuously establishing new norms, your community is doing the same and telling you through user reports. Those same reports can also tell you a crisis is emerging. Our custom AI learns to take the same actions your moderators take consistently, reducing manual review by up to 70%, so your human moderators can focus on the things that matter.
  • Transparency Reports: In addition to legislation being introduced worldwide requiring transparency from social networks on safety measures, data insights from the other four layers drive actions to improve your communities. Are the interactions growing over time? Are you filtering too heavily and restricting the flow of communication? Is bullying on the rise in a particular language? How long does it take you to respond to suicide or a public threat? How long are high reputation members contributing to the community? These data insights demonstrate the return on investment of community management because a community that plays well together stays together. A community that stays together longer builds the foundation and potential for a successful business.

To Achieve Digital Safety, Use A Multi-Layered Approach

Digital safety is a complex problem which is getting increasing attention from international governments and not-for-profit organizations like the World Economic Forum. AI is a critical part of the solution, but AI alone is not enough. To scale to billions of messages we need that multi-layered approach that blends multiple types of AI systems together with human creativity and agility to respond to emerging trends. At the end of the day, digital safety is not just classifying and filtering bad words and phrases. Digital safety is about appropriately handling what really matters.

 

Industry Spotlight: Jennifer Cadieux, Ludia

Written by

This week I caught up with Jennifer Cadieux, Ludia Inc.’s Lead Moderation and Social Care Specialist. Two Hat just co-created a case study (How Ludia Leverages Insights From Player Chat to Improve Their Games) with the Ludia team, so this was a great opportunity to catch up with Jennifer and hear more about her work.

Carlos Figueiredo: Can you tell us about how you started your career in the games industry, specifically in the area of social care and moderation?

Jennifer Cadieux: I started as a Functionality Quality Assurance tester at a third-party provider before switching over to their Player Support section. When I first moved over I handled things like customer support tickets and some light community management duties which is where I learned about the social care aspect.

I was lucky enough a few months in to get assigned to a moderation job and fell in love with it. No two days were ever the same and the players kept my days interesting. When I was hired on to Ludia it was only natural to continue in this in a hybrid environment.

CF: Many companies look at content moderation as only something that addresses negative behavior, but we know that it’s much more than that. How do you use Two Hat’s Community Sift to get valuable insights from player chat and make better games and experiences for your audience?

JC: Chat offers a wealth of information that players may not necessarily voice anywhere else.

If we see a trend of a specific word getting moderated over and over that has no negative context then we can use the tool to dig into that and see how and why it’s trending.

Everything from player nicknames for creatures to acronyms they use helps us clarify the players’ experience and relay suggestions and frustrations to our production teams.

CF: What is your favorite feature in Two Hat’s Community Sift and why?

JC: I really like the flexibility to adjust policies per queue; different games have different age restrictions so it’s extremely important to us to be able to tailor these per title and queue.

CF: What’s the best thing for you about the Ludia community? What makes it unique?

JC: Their nicknames for our various creatures crack me and my team up, like The Rat, Golden Chicken, and Dentist.

The community of players as a whole is very witty and clearly have a passion for their chosen titles. As we handle multiple titles we get to see lots of different players but one thing that runs through them all is their clear love of mobile games.

CF: What is something about content moderation in video games that you wish everybody knew about?

JC: We care about our players and their experience. The moderators don’t want to ban you, they want you playing and having fun. 

About Jennifer Cadieux

Jennifer Cadieux is based out of Montreal and created the Moderation and Social Care role at Ludia in 2018. She is an avid learner and can be found bingeing documentaries in her off time. Using knowledge gained from a lifetime of learning about other cultures she is driven to create safe places for players to interact with and learn from each other.

 

Learn more about the wonderful work of the Ludia team by downloading the case study.

Industry Spotlight: Zach Boasberg, Rovio

Written by

This week I caught up with Zach Boasberg, a Player Experience Specialist for Rovio Entertainment Corporation. We recently published a Rovio case study (“Promoting a Welcoming Player Environment“) that we collaborated on with Zach earlier this year. I wanted to get his thoughts on the content moderation and the current state of the industry.

Zach has an incredible knowledge of and passion for fostering awesome player experiences, so I was very excited to hear his insights.

Carlos Figueiredo: Can you describe a day in the life of a Player Experience Specialist at Rovio? What drives your purpose in this area of work?

Zach Boasberg: As part of the Player Experience team, my focus is on bridging the gap between the players and the developers making the games. On a daily basis, I’m using the different player touchpoints at my disposal (support tickets, player communities, reviews) to get an understanding of how our players feel, what they like and dislike, and then bringing that feedback to the developers in a constructive manner.

I care about our players having the best experiences possible in our games, not just because I play games myself and know how they feel, but because ultimately happy players means better business. So I’m always on the lookout for where the pain points are, and making sure that anyone that chooses to play one of our games has a lot of fun!

CF: Many companies still look at designing a positive community and having content moderation as a necessary evil and a cost center. We know that it’s quite the contrary: content moderation is a fundamental piece of a sound strategy to reach new users, retain them, and increase revenue. How does this play a factor at Rovio?

ZB: Social elements are a feature of many of our games, as it’s a great way to keep players engaged. Being able to chat with other people and make friends, or collaborate with a team, is a great driver for retention.

But players aren’t going to want to come back to your game if they have bad experiences while interacting with these social features. This is why we see content moderation as an important part of the puzzle.

By doing what we can to ensure that the interactions players have in our games are consistently positive, friendly, and safe, we’re optimizing the return we get on the social features we’ve invested in.

Ultimately, players want these social interactions, and I don’t see that demand slowing down. So we definitely see it as worthwhile to make sure our social features are as polished as any other feature we put in the game, and that includes having robust content moderation.

CF: We just collaborated with you and your team on a case study. One of the highlights for me is how you have operationalized mechanisms for a positive, welcoming, and safe community across all the games you produce. Why does that matter to you and the team?

ZB: When we began strategizing on how we wanted to approach content moderation, the first key consideration that popped up is our brands. We’ve of course got a very strong brand in our Angry Birds franchise, but we also have our Rovio brand as a game company.

Many of our players know us as a company, and we have a commitment to meet expectations of quality when it comes to Rovio games.

Therefore, by ensuring first that we have a solid baseline of settings for our content moderation, we’re making it so that our players can know they’ll have a consistently safe and fun experience when engaging in social features in any of our games. And in this way, we’re also protecting the integrity of our brands.

However, the other side of it is that our games are quite diverse, and the different communities of each game might have different needs or expectations. So while we’ve decided at a high level what sort of interactions are never OK in our games, we’ve also left room to tailor our content moderation to different genres.

A clear example is that while players in one of our competitive player-versus-player games might expect to be able to trash talk their opponents a bit, that sort of behavior could come across more strongly as bullying in a casual puzzle game.

We try to remain aware of these genre and community specific considerations, and balance having spaces where players can talk they way they’d like, without allowing disruptive players the room to cause harm.

CF: What’s your favorite thing about the community of Rovio players? What makes it unique?

ZB: We have an amazing community of players, and I think part of what makes them so great is their passion! Our fans, especially our Angry Birds fans, love the games as much as we do. They’re also not shy about making their voices heard!

Any time we’re able to hear directly from our players is great, because it gives us a lot of insight, and helps us make even better games.

CF: Last question: How has 2020 fundamentally changed the way you and your team work to ensure players have a positive and welcoming experience? What is the biggest lesson you learned so far this year?

ZB: I think the biggest point we’ve reaffirmed this year has been that content moderation is not something that can be set up once and then forgotten about. Having an automated filter is an important tool as a first line of defense, but if you never go back to check what’s being filtered or not filtered, you miss out on optimizing the overall experience.

On top of that, user generated content is always evolving and changing. This means it’s necessary to stay on top of what’s going on in your communities. This all adds up to taking a proactive approach and treating content moderation as a persistent function, rather than a one-off task.

CF: Thank you for your time today, Zach!

ZB: My pleasure, Carlos!

===

About Zach Boasberg:

Zach is originally from near Boston in the United States, but has been living in Helsinki, Finland since 2015, and has been a Player Experience Specialist at Rovio since 2017. A passionate gamer himself, Zach enjoys being able to interact directly with players and help craft great gaming experiences for them.



How We Maintain a Chat Filter in a Changing World

Written by

Have you ever wondered how Two Hat maintains a dynamic and up-to-date chat filter in an ever-evolving world, in 20 languages?

In light of recent global events, including the COVID-19 pandemic and the #BlackLivesMatter movement, we wanted to provide insight into our internal process.

We spoke to Portuguese Language Specialist Richard Amante, a journalist with 15 years of international experience, about how he and the team stay on top of the latest news.

As Richard explains, the Language & Culture team researches trending topics and new linguistic expressions and adds them to the dictionary in multiple languages. It doesn’t end there.

After the team adds new expressions to the chat filter, they review and adjust how those expressions are used across different communities and in different contexts. That includes finding and fixing false positives (low-risk content incorrectly flagged as high-risk) and false negatives (you guessed it — high-risk content incorrectly flagged as low-risk), detecting new and unexpected linguistic patterns, and upgrading and downgrading the riskiness of a phrase based on cultural shifts.

Especially today, in a world where so many of us are stuck at home, the lines separating our “real life” from our “online life” are blurry, if they exist at all. Whether we’re forming a clan in a multiplayer game or commenting on a friend’s profile pic, we’re having conversations online about everything, from the latest news to pop culture.

You might be wondering — why does this matter? It’s simple. Our clients want to know what their players and users are talking about; and more importantly, they want to keep them safe from abuse, hate speech, and harassment — while still promoting healthy conversations.

Every online community has different standards of what a healthy conversation looks like. An edtech platform designed for 7-12-year-olds will likely have a different standard from a social network catering to millennials. Here is the great power of Two Hat’s Community Sift: While the Language & Culture team provides a baseline by adding new words and phrases to the dictionary, clients can augment those words and phrases in real-time, as needed.

In this way, we work closely with our clients to maintain a living, breathing global chat filter in an ever-changing world.

Want to learn how Two Hat can help you maintain a safe and healthy online community? Request a demo today.

 

Tech Perspectives: Surpassing 100 billion online interactions in a month

Written by

In 2020, social platforms that wish to expand their product and scale their efforts are faced with a critical decision — how will they automate the crucial task of content moderation? As platforms grow from hundreds to thousands to millions of users, that means more usernames, more live chat, and more comments, all of which require some form of moderation. From app store requirements to legal compliance with global legislation, ensuring that all user-generated content is aligned with community guidelines is nothing short of an existential matter.

When it comes to making a technical choice for a content moderation platform, what I hear in consultations and demos can be distilled down to this: engineers want a solution that’s simple to integrate and maintain, and that can scale as their product scales. They are also looking for a solution that’s battle-tested and allows for easy troubleshooting — and that won’t keep them up at night with downtime issues!

“Processing 100 billion online interactions in one month is technically hard to achieve. That is not simply just taking a message and passing it on to users but doing deep textual analysis for over 3 million patterns of harmful things people can say online. It includes building user reputation and knowing if the word on the line above mixed with this line is also bad. Just trying to maintain user reputation for that many people is a very large technical challenge. And to do it all on 20 milliseconds per message is incredible”.  Chris Priebe, Two Hat’s CEO and Founder

Surpassing 100 Billion Online Interactions in a Month
I caught up with Laurence Brockman, Two Hat’s Vice President of Core Services, and Manisha Eleperuma, our Manager of Development Operations, just as we surpassed the mark of 100 billion pieces of human interactions processed in one month.

I asked them about what developers value in a content moderation platform, the benefits of an API-based service, and the technical challenges and joys of safeguarding hundreds of millions of users globally.

Carlos Figueiredo: Laurence, 100 billion online interactions processed in one month. Wow! Can you tell us about what that means to you and the team, and the journey to getting to that landmark?

“At the core, it’s meant we were able to keep people safe online and let our customers focus on their products and communities. We were there for each of our customers when they needed us most”.

Laurence Brockman: The hardest part for our team was the pace of getting to 100 billion. We tripled the volume in three months! When trying to scale & process that much data in such a short period, you can’t cut any corners.  And you know what? I’m pleased to say that it’s been business as usual – even with this immense spike in volume. We took preventative measures along the way, we focused on key areas to ensure we could scale. Don’t get me wrong, there were few late nights and a week of crazy refactoring a system but our team and our solution delivered. I’m very proud of the team and how they dug in, identified any potential problem areas and jumped right in. At 100 billion, minor problems can become major problems and our priority is to ensure our system is ready to handle those volumes. 

“What I find crazy is our system is now processing over 3 billion events every day! That’s six times the volume of Twitter”.

CF: Manisha, what are the biggest challenges and joys of running a service that safeguards hundreds of millions of users globally?

Manisha Eleperuma: I would start off with the joys. I personally feel really proud to be a part of making the internet a safer place. The positive effect that we can have on an individual’s life is immense. We could be stopping a kid from harming themself, we could be saving them from a predator, we could be stopping a friendly conversation turning into a cold battle of hate speech. This is possible because of the safety net that our services provide to online communities. Also, it is very exciting to have some of the technology giants and leaders in the entertainment industry using our services to safeguard their communities. 

It is not always easy to provide such top-notch service, and it definitely has its own challenges. We as an Engineering group are maintaining a massive complex system and keeping it up and running with almost zero downtime. We are equipped with monitoring tools to check the system’s health and engineers have to be vigilant for alerts triggered by these tools and promptly act upon any anomalies in the system even during non-business hours. A few months ago, when the pandemic situation was starting to affect the world, the team could foresee an increase in transactions that could potentially start hitting our system. 

“This allowed the team to get ahead of the curve and pre-scale some of the infrastructure components to be ready for the new wave so that when traffic increases, it hits smoothly without bringing down the systems”. 

Another strenuous exercise that the team often goes through is to maintain the language quality of the system. Incorporating language-specific characteristics into the algorithms is challenging, but exciting to deal with. 

CF: Manisha, what are the benefits of using an API-based service? What do developers value the most in a content moderation platform?

ME: In our context, when Two Hat’s Community Sift is performing as a classification tool for a customer, all transactions happen via customer APIs. In every customer API, based on their requirements, it has the capability to access different components of our platform side without much hassle. For example, certain customers rely on getting the player/user context, their reputation, etc. The APIs that they are using to communicate with our services are easily configurable to fetch all that information from the internal context system, without extra implementation from the customer’s end.

“This API approach has accelerated the integration process as well. We recently had a customer who was integrated with our APIs and went live successfully within a 24 hour period”.

Customers expect reliability and usability in moderation platforms. When a moderator goes through content in a Community Sift queue, we have equipped the moderator with all the necessary data, including player/user information with the context of the conversation, history and the reputation of the player which eases decision-making. This is how we support their human moderation efforts. Further, we are happy to say that Two Hat has expanded the paradigm to another level of automated moderation, using AI models that make decisions on behalf of human moderators after it has learned from their consistent decisions, which lowers the moderation costs for customers. 

CF: Laurence, many of our clients prefer to use our services via a server to server communication, instead of self-hosting a moderation solution. Why is that? What are the benefits of using a service like ours?

LB: Just as any SaaS company will tell you, our systems are able to scale to meet the demand without our customers’ engineers having to worry about it. It also means that as we release new features and functions, our customers don’t have to worry about expensive upgrades or deployments. While all this growth was going on, we also delivered more than 40 new subversion detection capabilities into our core text-classification product.

Would you like to see our content moderation platform in action? Request a demo today.

How to Support Content Moderator Wellness in 2020

Written by

We started 2020 having no doubt about the importance of content moderation for social platforms. As Talerton Gillespie, author of Custodians of the Internet, writes, “Content moderation is constitutional to the functioning of platforms, essential to what platforms are.” However, I believe that we have yet to fully appreciate the work of moderators, support them, and safeguard their wellbeing.

Take a step back to May 2019, when I spoke at and attended the Content Moderation in 2019 workshop hosted by IAPP. Gillespie was the keynote speaker, and he made a compelling case for redefining the moderator role: not as the custodian responsible for keeping the Internet clean, but rather one of guardianship.

In that same workshop, I witnessed the inception of an idea to create a professional association that would defend the interest of moderators and Trust & Safety professionals. Fast forward a year later, and enter The Trust & Safety Professional Association. The soon-to-launch organization will be focused on advancing the trust and safety profession through a shared community of practice. It’s encouraging to see this happening! Content moderators need support, especially now, during the COVID-19 pandemic, that we are spending more time online and moderation duties get more challenging.

“Moderation is the most essential, yet underrated, aspect of social-digital experiences. You couldn’t have an event in Times Square without police officers, Comic-Con couldn’t exist without event staff, and you wouldn’t send your kid to camp without camp counselors. Why should digital experiences be anything else?”

Izzy Neis, Head of Digital at ModSquad (Why Community Moderation Matters)

A critical piece of support needed is the protection of moderator wellbeing. Here at Two Hat, a mission-driven company founded on the ideals of Trust and Safety, we believe in giving moderators the tools and the best practices to ensure that:

  1. The impact of high-risk content on their wellbeing is minimized;
  2. They can focus their time on purposeful moderation, intentionally using their human expertise instead of doing manual work that machines can take care of;
  3. Wellness & resilience is a priority

Reduce Exposure to High-Risk Content

Protecting users from damaging user-generated content like gory images, dangerous/hateful speech, and chat that encourages suicide is a fundamental responsibility for online platforms. Equally important is the need to protect the moderators who ensure your online community remains a productive and thriving space.

Filters that identify and action on abusive behaviors are table stakes. There’s no need to expose moderators to high-risk content that AI can identify and proactively block without the need for human review. Furthermore, by proactively filtering you can immediately reduce the amount of user-generated reports, drastically reducing the workload of moderators.

In a large mobile game, user-generated reports decreased by 88% with the addition of a chat filter
From the e-book Content Moderation in Challenging Times

Purposeful Moderation

The work of a content moderator can feel like an endless battle against an ever-growing pile of content. I know this because I was a moderator once and I felt that challenge every day.

This is why it’s critical for a moderator to feel like they are doing something meaningful and purposeful. Knowing that the work you do every day has a tangible and positive impact on the community and in the world will help you connect and reconnect with why you’re doing that job in the first place. It’s very challenging to moderate and feel that connection without having the right tools, processes, and procedures at your disposal. One of the most important processes is to prioritize staff wellness and resilience.

Wellness & Resilience

May is Mental Health Awareness Month. This is an opportunity to raise awareness of what content moderators contend with from an emotional point of view. As Pascal Debroek, Head of Customer Service at Hatch Entertainment Ltd. said earlier this year:

“You have to understand that these can also be emotionally draining jobs. Most of these are pure customer-facing, and in a lot of the cases deal with either sensitive topics, aggravated end-users dealing with a situation that did not meet their expectations or even outright insults and threats.

Let’s not forget, most contacts with players stem from an emotional state; happy, sad, angry, you’ll encounter them all in these roles. And because it pays off to be empathetic in such roles, it also means your employees are more susceptible to the emotions that surround them.”

Today, I want to share a critical step you can take with your team to level up everyone’s wellness and resilience: build team and individual wellness plans. A wellness plan is an actionable set of activities that help you manage stress and recharge yourself.

For example, my own personal wellness plan might look something like this:

  1. Play drums for at least 15 minutes a day
  2. Take a short walk when it’s sunny outside
  3. Play video games with my family and friends
  4. Meditate (here’s a link to a 1-month free experience in the meditation app I use)
  5. Phone a friend or family member who knows my area of work and is OK with discussing challenging topics with me. (I had to use this one last year when I returned from a child protection conference and heard lots of stories that were heart-crushing. Hearing about such realities already pays a toll. Imagine those true heroes who review child abuse imagery and help protect kids globally.)
  6. Listen to some of my favorite songs

I hope this inspires you to build your own plan. You can go as specific as you want and as prescriptive as you need. Perhaps you want a set of actions you can take every day. Or maybe you prefer to have a pool of actions you pull from depending on the day. Also, as a moderation team, you can build team activities that help you cope with everyday stress. Playing games together might be a great way to do that. In an age of social distancing, you can play online games and also get creative with games you can play via video conferencing!

Back in 2018, we collaborated with therapist and wellness trainer Carol Brusca on a “Stress, Wellness, and Resilience” training session for Two Hat clients. She also shared her top 10 wellness tips for online moderators and community professionals. We just republished that piece adapting a few points to the new social distancing reality we are living.

***

If you saw value in these tips and would like to know more about how technology can protect your moderation team’s wellbeing, we can help.

Request a demo today to see how Two Hat’s content moderation platform can reduce your workload and your exposure to harmful content.



Content Moderators: 10 Tips to Manage Stress During COVID-19

Written by

Back in 2018, we collaborated with therapist and wellness trainer Carol Brusca on a “Stress, Wellness, and Resilience” training session for Two Hat clients. She also shared her top 10 wellness tips for online moderators and community professionals.

Content moderation and player support are tough jobs on the best of days. Today, due to the COVID-19 pandemic and resulting lockdowns, most moderators and player support professionals are now working from home. Online platforms are experiencing exponential growth, which means that moderators are busier — and under more stress — than ever.””

“It’s really important to explain to moderators and staff is that we acknowledge that this is a difficult time, and this is why we [as leaders] are playing our part in terms of doing moderation… Moderators are doing a difficult job at the best of times and right now they’re working a lot of hours and it’s extremely important that we communicate with members of staff about how they’re feeling.” Vernon Jones, Head of Safety at MovieStarPlanet

Content Moderation in Challenging Times webinar

Below, we’ve updated Carol’s original tips for managing stress to reflect today’s new reality.

===

As a community manager or content moderator, you experience the dark side of the internet every day. Whether you are reviewing chat, social media, forum comments, or images, high-risk content can be draining — and you may not even realize the damage it’s doing.

Studies show that community teams on the front lines of chat, image, or video moderation are especially vulnerable to stress-related symptoms including depression, insomnia, vicarious trauma (also known as “compassion fatigue”), and even PTSD. Now, more than ever, it’s critical that you have the right tools and techniques at your disposal to support your mental health.

1. Talk to someone.
Having and using social supports is the number one indicator of resilience. Asking for help from someone who cares about you is a wonderful way to get through a difficult time.

Does your company’s health plan provide access to a mental health professional? Take advantage of it. There’s no shame in talking to a therapist. Sometimes, talking to a stranger can be even more effective than confiding in a loved one.

If you can’t see a therapist in person right now, there are virtual options available.

2. Learn to say no.
If we do not set boundaries with others we can find ourselves feeling stressed out and overwhelmed. If you notice this might be a habit for you, try saying “no” once a day and see if you begin to feel better.

Of course, saying “no” at work isn’t always an option. But if you’re spending too much time reviewing high-risk content, talk to your manager. Ask if you can vary your tasks; instead of spending all of your workday reviewing user reports, break up the day with 15-minute gameplay breaks. Check out our blog post and case study about different moderation techniques you can use to avoid chat moderation burnout.

Setting boundaries is essential when you work from home. You cannot be “on” 24/7. Again, work with your manager to set fair and reasonable expectations.

3. Go easy on yourself.
We are quick to criticize ourselves and what we have done wrong, but not as likely to give ourselves credit for what went right, or all the things we did well.

Remember that you work hard to ensure that your online community is healthy, happy, and safe. Pat yourself on the back for a job well done, and treat yourself to some self-care.

The last few months have been at times scary, confusing, and deeply uncomfortable. David Kessler, who co-wrote On Grief and Grieving: Finding the Meaning of Grief through the Five Stages of Loss with Elisabeth Kübler-Ross, says that the discomfort we are collectively feeling is actually grief. He says that it’s important that we acknowledge our grief and name it:

“We’re feeling a number of different griefs. We feel the world has changed, and it has. We know this is temporary, but it doesn’t feel that way, and we realize things will be different… The loss of normalcy; the fear of economic toll; the loss of connection. This is hitting us and we’re grieving. Collectively. We are not used to this kind of collective grief in the air.”

4. Remember, this too will pass.
There are very few situations or events in our lives that are forever. Try repeating this mantra during a stressful time: this struggle will pass. It will make getting through that time a little easier.

(Maybe just repeat it silently in your head. Your co-workers — aka, spouse and pets — will thank you.)

This is especially hard right now, when we’re stuck at home, trying to find a balance between our work life and home life. While we cannot know when there will be a “return to normal”, it’s still important to acknowledge that the added daily stress isn’t permanent.

David Kessler offers these words of wisdom: “This is a temporary state. It helps to say it… This is survivable. We will survive.”

5. Get plenty of sleep.
We need sleep to replenish and rejuvenate. Often when we are feeling stressed, we struggle with sleeping well. If this happens to you, make sure your bedroom is dark and cool; try some gentle music to help you get to sleep, or use an app that plays soothing sounds on a loop. If staying asleep is the problem, try having a notepad and pen by your bed to write down your worries as they come up.

Pro tip: Save the marathon 3:00 am Animal Crossing sessions for the weekend.

6. Have a hobby.
Having a hobby is a great distraction from the stressors of everyday life. If you can do something outside, all the better. For many people being in nature automatically decreases stress. Remember to wear a mask and practice good social distancing!

Or, stick to video games. Playing Tetris has been proven to help people who experience trauma.

7. Drink tea.
A large dose of caffeine causes a short-term spike in blood pressure. It may also cause your hypothalamic-pituitary-adrenal axis to go into overdrive. Instead of coffee or energy drinks, try green tea.

We know that the smell of a freshly-brewed pot of coffee is like catnip to most moderators… but hear us out. Green tea has less than half the caffeine of coffee and contains healthy antioxidants, as well as theanine, an amino acid that has a calming effect on the nervous system.

8. Laugh it off.
Laughter releases endorphins that improve mood and decrease levels of the stress-causing hormones cortisol and adrenaline. It literally tricks your nervous system into making you happy. Try a comedy movie marathon or a laughter yoga class (this is a real thing; hopefully there’s a virtual version now!).

And hey, a 10-minute meme break never hurt anyone.

9. Exercise.
Getting plenty of exercise will decrease stress hormones and increase endorphins, leaving you feeling more energized and happier.

Ever had a 30-second, impromptu dance party at your desk? (Zoom dance party, anyone?)

No, really!

Often referred to as the “stress hormone,” cortisol is released in our body when we’re under pressure. Excess cortisol can cause you to feel stress, anxiety, and tension. Exercise brings your cortisol levels back down to normal, allowing you to relax and think straight again.

So crank up a classic, stand up… and get down.

10. Try the “3 Good Things” exercise.
Each night, write down three good things that happened during the day. This practice makes you shift your perspective to more positive things in your life — which in turn can shift your mood from stressed to happy, even if the three good things are tacos for lunch, tacos for 2 pm snack, and tacos for 4 pm snack (thank you, SkipTheDishes!). Good things don’t have to be earth-shattering.

Gratitude comes in all sizes, especially now.

So, whether you’re sipping a mug of green tea, talking to a professional, or shaking your groove thing in the name of science and wellness, never forget that a little self-care can go a long way.

===

For tips on reducing your moderation workload during the pandemic, download our e-book Content Moderation in Challenging Times: Techniques to Moderate Chat & Manage Increased Volumes.