Covid-19 fast-forwarding studios to leverage AI agility to overcome remote production challenges

To continue operating during the Covid-19 pandemic, studios are producing more live content and working in smaller and more flexible locations to minimise costs. APB+ managing editor Raymond Tan talks to several broadcast solution providers to learn how they are utilising innovations and turning to IP and cloud- and software-based tools to mitigate the impact of smaller production sets and enable more remote workflows.

Even before the onset of Covid-19, studios, production houses and content producers were already handling higher volumes of content than ever before. They also had to access and send content quickly and flexibly across an increased number of delivery channels to meet the growing demands of their audiences.

The Covid-19 pandemic is fast-forwarding this shift with studios facing the reality of either having to shut down the film and TV production or working remotely from smaller locations. 

With budgets slashed, studios are operating with lesser manpower, using simpler logistics and portable equipment and many are turning to virtual streaming and live IP video casting as the next viable option.

“Broadcasters have to be able to handle very short timescales to market – whereas in the past, the time between conception and launch would have been years, now it can be weeks.

“As audience demands change, services must be agile and highly adaptable, and be able to spin up and down very quickly,” says Greg de Bressac, Grass Valley’s vice-president of Sales, Asia.

Straker Coniglio, chief product officer at Vizrt/NewTek, adds: “Over time, broadcasters have implemented automation as a software interface to coordinate their production.

“Today, we are seeing the benefits of this, with the ability to control the production remotely.”

Observing the same trend, Marion Dinayuga, head of Broadcast, Panasonic Systems Solutions Asia Pacific (PSSAP), explains that broadcasters are today looking at new business models and solutions to help reduce the cost of operations. 

He maintains that broadcasters are now facing immense competition and must make the production of content cost-effective in an uncertain environment.

Dinayuga says: “With less revenue to run the business, they have to maintain their operational costs and are unlikely to have a budget to purchase new equipment for production.”

In a similar vein, Takuma Wada, head of Content Creation Solutions Marketing, Professional Solutions, Sony Hong Kong, believes that there is a growing interest in broadcast and production settings now to understand the benefits of automation, driven by the need for content owners to remain competitive and relevant.

He says: “They have to provide dramatically increased volume of content to more platforms with much lesser resources for an audience who is now consuming content anytime and from anywhere.”

Despite traditional drivers like sports and live entertainment being postponed or cancelled, Wada says high value and immersive content as well as timely information from the newsrooms would continue to thrive.

“As such, we believe that media and technology investments might shift towards realising digital automated business models and workflows that can be incorporated seamlessly into existing infrastructures.”

However, transitioning workflows to remote environments does create challenges that are easier said than done for broadcasters and media owners.

“Factors that hinder the adoption of automation in studios would probably be the lack of direction and/or strategy to fully utilise the potential, the costs of deployment as well as the lack of talent and skills for such setups,” explains Wada.

With the growing number of programmes, platforms and services being made available, complexity has also become a critical issue, with studios having to curate and extract from massive quantities of data and content with more demanding workflows.

The use of AI and software-based tools becomes an expedient way to cope with this complexity. Many studios are turning to new remote and automated AI and machine-learning technologies to reduce the complexity and cost of production, streamline and automate time-consuming processes, and to make quicker decisions.

“The advent of IP-based studios is also paving the way for a new era of flexibility in production,” says Grass Valley’s de Bressac.

“There is no longer a need for studios to invest in expensive workstations and related equipment; the cloud has enabled workflow approaches to become more innovative,” he says, adding that the improved flexibility and efficiency from standards-based IP can open up new, smarter approaches to content creation.

He explains that “this is because IP workflows can handle much higher bandwidth than legacy SDI environments, supporting more live streams and removing potential bottlenecks”.

Among the other benefits that IP brings are format/resolution-agnostic workflows and resource sharing.

Vizrt/NewTek’s Coniglio says that working from home has become the “new normal” where public networks and computer systems are the only way to connect to the office.

“In the US, broadcasters can physically relocate big expensive hardware into people’s homes. This isn’t really practical in Asia-Pacific where people don’t have huge garages or basements,” he says, adding that this is where software-defined workflows can help.

“The use of data to transfer tasks and story content makes the production much simpler without constant movement of people. Mediacorp, for example, is running its production workflows on Vizrt’s software tools.

“Bringing software into live production adds an intelligence that is evolving over time as the software feature sets grow,” says Coniglio.

He explains that with traditional video formats, the signals are typically too large to transfer over a public network; however, a combination of software interfaces and NDI technology can solve these tasks intuitively for broadcasters by enabling them to connect with the software and access the video content for immediate response – especially when one looks at control room functions such as monitoring live sources, preview and programme.

“The beauty of pure software-based solutions is that there is no need to relocate hardware, and we have customers remotely running national nightly newscasts from a laptop from their bedroom.

“Even when we go back to our physical premises, these solutions will continue to thrive as REMI (Remote Integration Model, or at-home) workflows become more common for sports and other types of programming.”

Addressing remote challenges

Sony has introduced Ci Media Cloud, a media-centric cloud environment with innovative applications that empower media companies with remote teams to leverage the cloud to streamline production, editorial and delivery workflows.

Ci Media Cloud Integrates AI capabilities to support object recognition, speech-to-text and facial detection that improve discoverability, searchability and provide enhanced transcription.

On the hardware-end, Sony has introduced a compact and lightweight AI-based Edge Analytics Appliance REA-C1000, which is ideal for small studio applications and uses AI-based video analytics technology to analyse the input received from connecting cameras, automatically extract the object in focus and combine it with other images in real time on a GPU.

The technology uses motion/face detection and colour/shape recognition, enabling the REA-C1000 to function as the ‘brain’ to any connecting camera and AV set-up. Together with Sony’s NRC remote cameras, the solution can be used in hard-to-reach or unobtrusive places, with smooth, silent panning that will not disturb the subject

As many broadcasters know, Panasonic has a wide variety of remote solutions such as the PTZ remote cameras. The new PTZ AW-UE150 has a large sensor PTZ which recently adopted the FreeD protocol. This protocol enables the camera to communicate with AR and VR software to produce higher-quality content using graphics.

The company has also built an auto-tracking system that is integrated with machine-learning algorithms like facial recognition and deep learning to track the movements of both the TV host and the guests who are present within the studio. Benefits from this integrated solution include labour-saving, operational costs and less training time required.

Dinayuga says Mediacorp was Panasonic’s first customer in the region to use this solution. “They are currently testing the camera for outdoor productions and yet to be deployed on a full scale.

“We also have customers who are using this solution to create a live shopping show on an online shopping platform.”

Meanwhile, according to de Bressac, Grass Valley is now spearheading the next evolution with a raft of solutions and technologies announced during the company’s Spring Launch – some of which are already being used by GV’s customers – to leverage true distributed production.

He says eSports and interactive entertainment company Blizzard was the launch customer for the new cloud-based SaaS production solution GV AMPP (Agile Media Processing Platform) that allows users to spin resources up or down on an as-needed basis.

“Our technology innovations underpin flexible workflows and allow creative teams to be based in any location. The first application available for the platform, AMPP Master Control, has been on-air with Blizzard since the opening of the Overwatch League 2020 season in early February.”

In response to the Covid-19 crisis and building of this solution set, de Bressac says AMPP has been an essential system enabling the eSports leader to keep its Overwatch League and Call of Duty League matches live since March 28, enabling distributed remote production for operators and staff working from their homes.

Coniglio of Vizrt/NewTek shares: “We use AI where it has a purpose that will benefit the customer. Some technologies we have are fully AI-based. Others are using intelligent technologies that can auto-detect input interaction and make decision based upon that.

“In live production, both technologies can be employed depending on the type of production.

“For example, our AI keying technology is enabling customers to get high-quality keying in scenarios where the lighting conditions are less than ideal. Our MAM system, Viz One, is able to automate metadata creation and get more value out of archival content which is a key source of content right now.

“We continue to look for ways to leverage this technology to help our customers tell better stories.”

Coping with the pandemic

With the uncertainty on how long the pandemic will last, the companies interviewed maintain that they will remain focused on helping customers leverage innovative technologies to continue broadcasting during the crisis.

Covid-19 is creating both opportunities and challenges in the media landscape, says Wada. “Sony is ready to work alongside our customers to learn how we can help their content get the power it deserves.

“We believe that media and technology investments might shift towards realising digital automated business models and workflows that can be incorporated seamlessly into existing infrastructures.”

Panasonic’s Dinayuga adds: “The pandemic is definitely ‘slowing’ customers from adopting our existing and latest solutions due to cost savings and a decrease in demand.

“However, we believe our customers will be looking at new business models and cost-effective solutions to help reduce their cost of operations in the long run.”

On its end, Grass Valley has already put business continuity plans in place to mitigate the impact of any service and delivery disruptions that result from the Covid-19 outbreak. While a small number of components are typically sourced from impacted areas, the broadcast solution provider is making accommodations within the supply chain to source these materials locally.

“In these challenging times, the reality of remote/at-home and distributed production models is significant. Our remote production workflows deliver greater freedom for our customers to cover live events regardless of their location,” says de Bressac.

“Solutions like our DirectIP camera functionality can be effectively leveraged from within quarantined areas to help reduce the level of risk to operators. In fact, the majority of the production team – from camera shaders to the director and replay operators – can avoid the need for travel to the event venue.”

Research from Upwork, says de Bressac, shows 63% of companies use ‘remote workers’ and these figures are only set to increase as organisations across all sectors drive to reduce their environmental footprint.

The pandemic has also brought forward valuable lessons. “Working smarter is something all broadcasters are looking to do more. Workflows that free staff to focus on higher impact creative tasks are compelling, allowing broadcasters and production companies to leverage the best talent – wherever they are in the world.

“Collaborative workflows that allow production staff to work from any location, even if it’s their home, is going to become part and parcel of the way our industry operates,” de Bressac predicts.

Even before the pandemic, Coniglio says Vizrt/NewTek has been working on a ‘PCR in the cloud’ with full graphics, automation and switching. “The current crisis situation accelerated our development in this area and we are using this in our live productions of VizrTV where we see a lot of promise for something that we can soon bring to our customers,” he says.

“With this set-up, we have a single operator executing a remote production where all the talents are in different locations. In a recent production, we even used our ‘virtual presenter’ technology to have a ‘face-to-face’ interview between two people that appeared to be sitting next to each other on one set even though they were both physically sitting in different countries.

“I think innovations like this will fundamentally change the way production is done forever.”