Azure AI Video Indexer

Azure Artificial Intelligence Video Indexer

While machine learning has come in a large way from the initial days of sci-fi robotic visualizations to life-changing experiences across different verticals

One of them is media services. Azure has done a large change in incorporating machine languages to media services by providing multiple capabilities like pattern recognition and video indexing.

One of the upcoming features is going to be OCR recognition in videos. The problem is acute because OCR is not an exact size and evidences from audio and video streams works together to validate the word being captured in OCR. Multiple different fonts, accents complicate the OCR detection. The noise which seeps in a video owing to font size, lighting, shadow effects, filters makes the job difficult.

A sample code showing the various API’s

var apiUrl = "https://videobreakdown.azure-api.net/Breakdowns/Api/Partner/Breakdowns";
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", key);
 
var content = new MultipartFormDataContent();
content.Add(new StreamContent(File.Open(videoPath, FileMode.Open)), "Video", "Video");
 
Console.WriteLine("Uploading...");
var result = client.PostAsync(apiUrl + "?name=some_name&description=some_description&privacy=private&partition=some_partition", content).Result;
var json = result.Content.ReadAsStringAsync().Result;
 
Console.WriteLine();
Console.WriteLine("Uploaded:");
Console.WriteLine(json);
 
var id = JsonConvert.DeserializeObject<string>(json);
 
while (true)
{
    Thread.Sleep(10000);
 
    result = client.GetAsync(string.Format(apiUrl + "/{0}/State", id)).Result;
    json = result.Content.ReadAsStringAsync().Result;
 
    Console.WriteLine();
    Console.WriteLine("State:");
    Console.WriteLine(json);
 
    dynamic state = JsonConvert.DeserializeObject(json);
    if (state.state != "Uploaded" && state.state != "Processing")
    {
        break;
    }
}
 
result = client.GetAsync(string.Format(apiUrl + "/{0}", id)).Result;
json = result.Content.ReadAsStringAsync().Result;
Console.WriteLine();
Console.WriteLine("Full JSON:");
Console.WriteLine(json);
 
result = client.GetAsync(string.Format(apiUrl + "/Search?id={0}", id)).Result;
json = result.Content.ReadAsStringAsync().Result;
Console.WriteLine();
Console.WriteLine("Search:");
Console.WriteLine(json);
 
result = client.GetAsync(string.Format(apiUrl + "/{0}/InsightsWidgetUrl", id)).Result;
json = result.Content.ReadAsStringAsync().Result;
Console.WriteLine();
Console.WriteLine("Insights Widget url:");
Console.WriteLine(json);
 
result = client.GetAsync(string.Format(apiUrl + "/{0}/PlayerWidgetUrl", id)).Result;
json = result.Content.ReadAsStringAsync().Result;
Console.WriteLine();
Console.WriteLine("Player token:");
Console.WriteLine(json);