Salesforce Data Cloud Now Supports Audio And Video: Here’s Why It Matters

Did you know people stick around 2 minutes longer on sites with videos⁽ⁱ⁾?

Also, 86% of video marketers say video has boosted their web traffic⁽ⁱⁱ⁾.

On the audio side, ads generate over 10,000 attentive seconds per 1,000 impressions⁽ⁱⁱⁱ⁾.

Together, audio-visual content is a powerful driver of engagement. However, without proper data processing, these valuable insights can go untapped, leading to missed opportunities and conversions.

That’s changing now.

Salesforce now enables the processing of audio and video content in Salesforce Data Cloud, alongside structured data. This new capability allows you to analyze unstructured data from customer service calls, webinars, and interactions, unlocking deeper insights into customer behaviors, preferences, and sentiments.

For instance, audio analysis can detect emotions, track trends, and highlight keywords reflecting customer needs, while video data reveals behaviors like in-store product interactions or key moments in webinars that captivate audiences.

In this blog post, we’ll explore why processing audio-visual data matters and how Salesforce’s new capabilities can help you gain a deeper understanding of your customers, enhance service, and make more informed decisions.

The Importance of Processing Audio & Video Data

The rise of multimedia content consumption through social media, user-generated content, and online meetings—like customer service calls, webinars, and interviews—provides valuable insights into your customers’ behaviors, preferences, and sentiments.

By tapping into audio and video data from these interactions, you can uncover actionable insights that help deliver personalized experiences, streamline operations, and boost efficiency.

This data, whether from customer calls, training sessions, product demos, or webinars, offers a clearer understanding of your customers’ needs and expectations, enabling you to respond effectively.

With advanced technologies such as digital asset management (DAM) systems, AI, and machine learning, you can analyze this data to uncover patterns and trends that drive business strategies. As a result, you can improve customer engagement, increase retention, and make smarter, data-driven decisions that propel growth.

However, unlike structured data, unstructured audio and video data do not adhere to predefined models, making them difficult to process and analyze using traditional techniques.

Audio data requires complex natural language processing (NLP) models, while video relies on resource-intensive computer vision algorithms. Managing this data also involves substantial storage, efficient data handling, and ensuring compliance with privacy laws such as GDPR.

Additionally, you should have a robust infrastructure to handle high processing demands, especially for generating real-time insights.

That’s where Salesforce comes in!

Salesforce: Leading the Charge in Audio & Video Data Innovation

Salesforce Data Cloud now offers transcription capabilities for both audio and video, enabling you to index customer interactions, perform audio analysis, and conduct similarity searches.

Seamlessly integrating with the Einstein 1 Platform, this feature transforms audio and video data into Salesforce metadata. As a result, analysts can visualize insights with Tableau, developers can automate workflows using Salesforce Flow, and users can enhance Einstein Copilot’s responses, without needing to fine-tune LLMs. This paves the way for future innovations like advanced interaction analysis, sentiment detection, and actionable insights.

Additionally, Salesforce’s Unstructured Data Processing Hub, powered by OpenAI’s Whisper model, transcribes audio conversations and extracts audio content from video files. These transcriptions are then converted into embeddings within Salesforce’s Vector Database.

Supported File Types:

• Audio: MP3, OGG, FLAC, WAV, M4A, MPGA
• Video: MP4, MPEG, WEBM

Language Support: Currently, transcriptions support English and select Western European languages.

With these new capabilities, you can:

• Understand customer sentiment and preferences from multimedia interactions, allowing for tailored products, services, and communications.
• Analyze audio and video data to pinpoint customer pain points, offering faster issue resolution and proactive support.
• Fine-tune campaigns by analyzing the tone and mood of customer interactions, improving targeting and engagement.
• Quickly index and search multimedia libraries to enhance user experiences in fields like entertainment, eLearning, and corporate training.
• Transcribe calls and meetings, turning one-off conversations into accessible knowledge that benefits teams across the organization.
• Automate follow-ups, generate personalized recommendations, and identify trends across interactions, saving time and boosting sales and support efficiency.

Industry Applications for Audio/Video Processing:

• Retail: Video analytics track customer movements and interactions within stores. This data helps optimize store layouts, improve product placement, and enhance the overall shopping experience while boosting sales and inventory management.
• Public Safety: Law enforcement and security agencies use video analytics for real-time surveillance, threat detection, and incident analysis. Audio processing assists in monitoring emergency communications, contributing to faster response times and improved community safety.
• Telemedicine: Speech recognition aids in diagnosing conditions, while video analysis supports medical imaging and remote patient monitoring.
• Manufacturing: In manufacturing environments, video analytics monitor production lines, detect safety hazards, and optimize workflow. This leads to reduced downtime, improved product quality, and enhanced operational efficiency.
• Media & Entertainment: You can improve content recommendations by analyzing viewer engagement and automating moderation to ensure compliance.
• Financial Services: Audio and video recordings of transactions or meetings. Analysis of this data helps detect fraudulent activities, ensure regulatory compliance, and streamline operational efficiency.

Salesforce Data Cloud empowers you to index and leverage audio and video content from past conversations and recordings, enriching customer interactions and internal knowledge bases for more insightful, data-driven decisions.

Conclusion

Unlocking crucial insights from customer calls and service conversations can be a challenge when the data is trapped in unstructured audio or video files.

Sales managers and support teams often have to manually sift through endless recordings to understand customer sentiment, resolve issues, or spot upsell opportunities, leading to inefficiencies.

Now more than ever, extracting actionable insights from multimedia content is essential for staying ahead in a competitive market.

That’s where Salesforce steps in!

With Salesforce Data Cloud, you can efficiently process audio and video data to deliver personalized customer experiences, streamline operations, and turn multimedia content into a powerful growth asset.

This can be achieved in three simple steps:

• You can seamlessly ingest audio and video data directly from cloud storage providers.
• Once ingested, these files are transcribed and segmented, then embedded and indexed for vector and keyword-based searches. You can even enable hybrid search indexing for more robust search capabilities.
• Data Cloud supports code-based and no-code activation methods, making it easier to leverage Retrieval-Augmented Generation (RAG) in tools like Prompt Builder and Flow Builder.

This feature, introduced in August 2024, is available through Feature Manager. Simply enable Audio and Video File Transcription in the Search Index, set up a search index configuration for audio and video files, and let Salesforce do the rest.

Statistics References:

^{(i) & (ii)} Idomoo
⁽ⁱⁱⁱ⁾ SiriusXM Media