BytePlus Effects

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. This guide is provided by Voicemod. Agora is planning a documentation upgrade program for all extensions on the marketplace. Please stay tuned.

The BytePlus Effects extension encapsulates the core APIs of the BytePlus Effects SDK. By calling the setExtensionProperty or setExtensionPropertyWithVendor method of the Agora Video SDK v4.x and passing in the corresponding key and value, you can quickly integrate BytePlus's capabilities.

Taking setExtensionProperty as an example, the key is named after the BytePlus API, and the value wraps either certain or all of the parameters of that API in JSON. When you call setExtensionProperty and pass in the pair of key and value, it is equivalent to calling the corresponding BytePlus API. The case is the same for setExtensionPropertyWithVendor.

Currently, the extension encapsulates part of the APIs of the BytePlus Effects SDK. For details, see the key-value overview.

The development environment has to meet the following requirements:

• Android Studio 4.1 or later.
• A physical device (not an emulator) running Android 5.0 or later.

The BytePlus Effects extension works together with the Video SDK v4.x. Refer to the following doc to integrate the SDK and start a basic video call:

• SDK quickstart

1. Download the Android package of BytePlus Effects.

2. Unzip the package, and save all .aar files to the /app/libs path of your project folder.

3. Contact Agora to get the resource package of the BytePlus extension. Save the files you need to the /app/src/main/assets path of the project folder. For details, see Resource package structure.

4. In the app/build.gradle file, add the following line in dependencies:

   
   

   _1

   implementation fileTree(dir: "libs", include: ["*.jar", "*.aar"])

   
   

5. Import the required classes:

   
   

   _7

   import io.agora.rtc2.Constants;

   _7

   import io.agora.rtc2.IMediaExtensionObserver;

   _7

   import io.agora.rtc2.IRtcEngineEventHandler;

   _7

   import io.agora.rtc2.RtcEngine;

   _7

   import io.agora.rtc2.RtcEngineConfig;

   _7

   import io.agora.rtc2.video.VideoCanvas;

   _7

   import io.agora.rte.extension.bytedance.ExtensionManager;

   
   

This section describes the call sequence of using the extension. For a detailed parameter description, see the API reference.

When initializing RtcEngine, call enableExtension to enable the extension:

_5

private void enableExtension(boolean enabled) {

_5

// Initialize ExtensionManager before calling enableExtension

_5

ExtensionManager.getInstance(mRtcEngine).initialize(this);

_5

mRtcEngine.enableExtension("ByteDance", "Effect", enabled);

_5

}

To initialize the extension, call setExtensionProperty, and pass in the corresponding keys and values:

1. Check the license Set the key as bef_effect_ai_check_license and the value as the path of the license file LicenseBag.bundle.
2. Initialize the effects Set the key as bef_effect_ai_init and the value as the path of the model file ModelResource.bundle and the device name.

Take Android as an example:

_39

private void initExtension() {

_39

String[] resources = new String[] {"byte_dance/LicenseBag.bundle", "byte_dance/ModelResource.bundle"};

_39

loadBundle(resources, () -> {

_39

File destFile = getExternalFilesDir(null);

_39

_39

// Gets the path of the license file

_39

File licensePath = new File(

_39

destFile,

_39

"byte_dance/LicenseBag.bundle/" + io.agora.rte.extension.bytedance.example.Constants.mLicenseName);

_39

_39

// Check the license

_39

try {

_39

JSONObject jsonObject = new JSONObject();

_39

jsonObject.put("licensePath", licensePath.getPath());

_39

setExtensionProperty("bef_effect_ai_check_license", jsonObject.toString());

_39

} catch (JSONException e) {

_39

Log.e(TAG, e.toString());

_39

}

_39

_39

// Gets the path of the model file

_39

File strModelDir = new File(destFile, "byte_dance/ModelResource.bundle");

_39

_39

// Initializes the effects

_39

try {

_39

JSONObject jsonObject = new JSONObject();

_39

jsonObject.put("strModelDir", strModelDir.getPath());

_39

jsonObject.put("deviceName", "");

_39

setExtensionProperty("bef_effect_ai_init", jsonObject.toString());

_39

} catch (JSONException e) {

_39

Log.e(TAG, e.toString());

_39

}

_39

_39

});

_39

}

_39

_39

// Only updates the key and value when calling setExtensionProperty

_39

private void setExtensionProperty(String key, String property) {

_39

mRtcEngine.setExtensionProperty("ByteDance", "Effect", key, property);

_39

}

Call setExtensionProperty, and pass in the corresponding keys and values. You can implement the following functions:

• Set the orientation of the mobile phone
• Set superimposed effects (face beautification, shape beautification, and beauty makeup) and their intensity
• Set stickers
• Set filters and their intensity

If you want to use superimposed special effects in an environment that does not use stickers, initialize the composer first. If you want to remove an effect, remove the composer node. See the key-value overview for details.

The complete sample code and project is provided on GitHub:

Platform	Language	Sample project
Android	Java	ByteDance/android

​

This section lists the APIs related to using extensions with the Agora SDK.

• addExtension in the RtcEngineConfig class
• enableExtension in the RtcEngine class
• setExtensionProperty in the RtcEngine class
• onChannelMediaRelayEvent in the IRtcEngineEventHandler class

​

This section might not provide the latest information. You can find a more detailed explanation from Effects Interface - C, where you use the name of the key to search the corresponding API.

To implement the BytePlus Effects extension in your app, pass in the corresponding key-value pair when calling setExtensionProperty or setExtensionPropertyWithVendor:

• key: bef_effect_ai_check_license
• value: licensePath. String. The path of the license file.

See Special effects handle license.

• key: bef_effect_ai_init
• value: The value includes:
  • strModelDir: String. The path of the resource file.
  • deviceName: String. The device name. Generally you can enter an empty string ("").

See Initialize special effects handle.

• key: bef_effect_ai_set_composer
• value: strPath. String. The path to composer resource files.

See Initialize composer.

• key: bef_effect_ai_composer_remove_nodes
• value: The value includes:
  • nodePaths: String. The array containing effect resource paths.
  • nodeNum: Integer. The length of the array containing effect resource paths.

See Remove composer.

• key: bef_effect_ai_set_orientation
• value: orientation. The parameter type is bef_ai_rotate_type. You can set it to 0 (no rotation), 1（rotate 90 degrees clockwise), 2（rotate 180 degrees clockwise), or 3（rotate 270 degrees clockwise.

See Set the phone angle.

• key: bef_effect_ai_composer_set_nodes
• value: nodePaths. String. The array containing effect resource paths.

See Set superimposed effects.

• key: bef_effect_ai_composer_update_node
• value: The value includes:
  • nodePath: String. The path of the effect resource.
  • nodeTag: String. The key of the effect. See Functions of Resource Keys.
  • value: Float. The intensity of a single node for the combined effects.

See Set the intensity of superimposed effects.

• key: bef_effect_ai_set_effect
• value: strPath. String. The path of the effect resource.

See Set stickers.

• key: bef_effect_ai_set_color_filter_v2
• value: strPath. String. The path of the effect resource.

See Set filters.

• key: bef_effect_ai_set_intensity
• value: fIntensity. Float. The value ranges between [0.0, 1.0], where 0 means disabling the filter.

See Set filter intensity.