Skip to main content

  • Core Products

  • Add-Ons

  • Extensions Marketplace

ActiveFence Video Content Moderation

ActiveFence is the leader in providing Trust & Safety solutions to protect online platforms and their users from malicious behavior and content. By integrating ActiveFence capabilities in your app, design the exact content moderation processes you want with the ActiveFence AI-driven automated detection for text, audio, image and video content across multiple abuse areas and languages. Quickly moderate, enforce policies, manage user flags, send notifications, and more.

The following figure shows the workflow for ActiveFence to moderate content sent by a specific user to a channel:

rtc-channel

This page shows you how to integrate and use the ActiveFence content detection and moderation extension in your app.

Prerequisites​

The development environment requirements are as follows:

  • Android Studio 4.1 or higher.
  • Android SDK API Level 24 or higher.
  • A mobile device that runs Android 4.1 or higher.

Project Setup​

In order to configure ActiveFence:

  1. Setup your ActiveFence account

    1. Either:

      ActiveFence sends you an email with activation details.

    2. Open the ActiveFence email and follow the instructions to activate your ActiveFence account.

      During this process, you open ActiveOS in your browser.

  2. Configure your ActiveFence API Key

    You use ActiveFence API keys for authentication when you send images to the ActiveFence content moderation API.

    1. In ActiveOS, click the Settings icon on the top-right.

      Account Settings opens.

    2. Click Data Management > ActiveFence API Keys, then click Add Key.

    3. In Add API Key, add your Key Name and Description, then click Generate Key.

    4. Click Your API Key and save it to a secure place.

      This key appears only once.

    5. Email Agora this key information. Agora uses this key to share data using ActiveFence API.

  3. Connect your app to your ActiveFence account

    In order for ActiveFence to monitor data streamed to a channel in your app, ActiveFence needs to bind your App ID to your ActiveFence account. To do this:

    1. Send Agora an email at support@agora.io requesting ActiveFence activation with your:

      • Company name
      • Company email
      • Agora APP-ID,
      • ActiveFence API Key

    2. Agora activates communication between your app and ActiveFence and supplies any extra credentials to you.

    3. ActiveFence communicates any extra credentials to you.

  4. Configure Action webhooks

    1. In ActiveOS, click Data Management > Webhook Key Management, then click Add Key.

    2. In Add API Key, fill in the form and click Generate Key.

      You use this key to authenticate calls to your webhook. If do not use authentication, you do not need this key.

    3. In ActiveOS, click DATA MANAGEMENT > Action Webhooks, then click Add Webhook.

    4. In Add Action Webhook, update the following fields and click Save.

      • Action - The name for this webhook.
      • Related Entity - Set to content.
      • Endpoint URL - The FQDM for your webhook. For example: https://my.super.duper/webhook.
      • Request Method - Set to POST.
      • API Key - For authenticated requests to your webhook. Choose the key you configured earlier. If you do not require authentication, set to No auth.

    5. In Body, configure the format of the JSON object and data sent to your webhook. For example:


      _10
      {
      _10
      "contentUrl":"$contentUrl",
      _10
      "status":"$status",
      _10
      "userId":"$userId",
      _10
      "addedAt":"$addedAt",
      _10
      "contentId":"$contentId",
      _10
      "reason":"$violationsCategories",
      _10
      "metaData": "$metaData"
      _10
      }
      _10
      }

  5. Configure the trigger for your webhook

    ActiveFence monitors the data sent in a channel. When content matches conditions in a workflow, ActiveFence sends a PUSH request to your webhook with information about the channel, the data sent and the user who sent it.

    1. Create an Action

      1. In ActiveOS, click the Settings icon on the top-right.
      2. Click Moderation Capabilities > Action Management, then click Add Action
      3. In Choose Action Type, click Enforce, then click Next
      4. Fill Action Name, then Choose Webhook.

      Whenever a workflow or a moderator triggers the Action, the webhook is activated.

    2. Configure automated workflows

      Rule-based workflows in ActiveOS are triggered by any data point, and start any Action you created. To add a new workflow.

      1. In ActiveOS, click the hamburger menu at the top left, click Workflows.

      2. Click Create New and follow the UI to set the:

        • Initiator - the subset of data applicable for this workflow. For example, Abusive or Harmful. In the parameter picker on the right side menu, choose a Violation Type.
        • Condition - further rules routing data according to different criteria. For example, Risk Score equals 90.
        • Action - the result of the workflow for any item that matches its conditions. For example, suspent. This is a system action or a webhook.

        Workflows are only active after you enable them.

  6. Invite users and start moderating with ActiveFence

    1. In ActiveOS, click the Settings icon on the top-right.

    2. Click Users > Account Users.

    3. Click Add user and invite team members who should have access to your account.

    To start moderating on ActiveOS and unlock more value for moderation teams, email support@activefence.com or send an intercom messenger in ActiveOS.

Integrate the extension​

Now that ActiveFence is activated, to check content sent in the channels in your app:

  1. Setup your webhook

    In order to react to calls made by ActiveFence for moderated content, your need to create a server that received and interprets the data sent to you in the webhook you defined in ActiveFence . For example:

    1. In your browser, open agora-activefence-kicker and use the one-click deployment for the server
    2. Add the deployment link + /kick/ to the webhook URL for the desired event in ActiveOS .

  1. Initialise ContentInspectConfig

    Set up the inspect config inside ContentInspectModule. If you use a string UID to join the channel and want to receive it in the webhook callback, pass the string UID to YourExtraInfo inside ContentInspectConfig.

    For example, the following config monitors the images in a video feed, taking one frame every 5 seconds:


    _7
    val contentInspectModule = ContentInspectModule()
    _7
    contentInspectModule.type = ContentInspectConfig.CONTENT_INSPECT_TYPE_IMAGE_MODERATION
    _7
    contentInspectModule.interval = 5
    _7
    val contentInspectConfig = ContentInspectConfig()
    _7
    contentInspectConfig.modules[0] = contentInspectModule
    _7
    contentInspectConfig.moduleCount = 1
    _7
    contentInspectConfig.extraInfo = "YourExtraInfo"

  2. Apply ContentInspectConfig to your Agora Engine instance.

    Enable content inspect after joining a channel and after the local video is published and played.


    _1
    val returnVal = agoraEngine?.enableContentInspect(true, contentInspectConfig)

  3. Catch banning events

    To know when your local user is banned from the scene, use the connectionStateChangedTo delegate method:


    _6
    override fun onConnectionStateChanged(state: Int, reason: Int) {
    _6
    super.onConnectionStateChanged(state, reason)
    _6
    if (state == Constants.CONNECTION_STATE_FAILED && reason == Constants.CONNECTION_CHANGED_BANNED_BY_SERVER ) {
    _6
    showMessage("Stream banned by the server")
    _6
    }
    _6
    }

    This only happen if you have applied the Kicking Server steps above, or created your own kicking server that communicates with ActiveFence.

Reference​