AI Description

Every component on the platform has an optional description field. This description is important for reusing that component again because in some instances, only the title and description fields are available for reference. Descriptions are also helpful for collaborating with different user types. A technical user generally creates the components where a less technical marketing user would use the attributes and audiences to create a campaign. It would be difficult for the less technical user to understand a complex attribute written in SQL.

Another issue is that descriptions are hard to write. Often times they are left blank or they improperly describe the component. This is where AI generated descriptions come in.

The Problem

The Solution

Use generative AI to translate the raw SQL of the component into a natural language description.

The AI generated description needs to be seamless with the platform and the descriptions need to be accurate. The AI feature would live inside the existing description text field. The feature should be discoverable but optional. The user can edit the AI generated text or regenerate the description again. The AI interface can be deployed to anywhere on the platform that has the description field and has access to the raw SQL of the component. The AI needs to look and feel like AI.

User Feedback

Our data scientist worked on testing the AI model to output accurate descriptions. We conducted a test with 10 participants all familiar with the platform and Audiences. Each participant would run the AI Explainer on 10 different Audiences and report on the results. We tested on accuracy, completeness, and if it was business friendly. The results concluded that responses scored highly on accuracy but would sometime miss on completeness. Failing to describe all aspects of the Audience. Some results also came back as not business friendly, describing some lines in SQL terms.

Participants described the interface as intuitive and seamless with the existing flow. Users that previously had to write the descriptions thought that this feature was magical and that it would save them a lot of time.

Next Steps

The interface of AI descriptions gathered good feedback so it was deployed on the Audiences component in beta to start. With a disclaimer that completeness would not be guaranteed.
Our data scientist would continue to test the AI model to output accurate and complete descriptions.
Deploy AI anywhere that has a description field once the AI model was at an acceptable standard.
Continue to work on interface features, including updated designs for insert and rephrase, and what happens when a description is out of date.