Robots on the Field of Play: Robotic Cameras

Posted on Jul 15, 2023 by FEED Staff

AI, enhanced image quality, player tracking and connectivity are helping to radically increase the adoption of robotic cameras in live sports broadcast production

Robotic video cameras are everywhere. We are surrounded by automated remote eyes observing us in law enforcement, government agencies, transport, education, security, corporate, houses of worship, healthcare, courtrooms and retail. A camera robot can take pictures or visually record information without being held and manipulated by a human.

At the Tier 1 top-end of broadcast sports production, traditional system cameras will likely prevail, where the skill and judgement of an experienced operator remains paramount to nail the action and tell the story and emotion of the game.

However, cameras are getting smaller and robotic control is becoming more sophisticated, allowing producers and OB companies to enhance or augment live sports.

Robotic cameras were thrust into the spotlight during the global pandemic, as social distancing rules and travel restrictions saw producers scramble to keep the show on the road despite the hastily implemented requirement to reduce the number of crew on-site. Where applicable and practical, the use of cameras without operators will likely be retained in productions for compelling reasons of sustainability and work-life balance.

Production budgets are under continuous pressure, audience fragmentation is an ongoing reality, and broadcasters are battling to deliver more content to more outlets as they endeavour to retain their audience. Where can robotic cameras play a role to enhance coverage and reduce or control costs? What are robotic cameras anyway?

AI, enhanced image quality, player tracking and connectivity are helping to radically increase the adoption of robotic cameras in live sports broadcast production

However, cameras are getting smaller and robotic control is becoming more sophisticated, allowing producers and OB companies to enhance or augment live sports.

SCIENCE of automation

Basic surveillance-style cameras have fixed lenses and fixed camera heads. True pan-tilt-zoom (PTZ) cameras can often pan by as much as 340° horizontally and tilt by as much as 120° vertically. This allows a single camera and remote operator to cover a huge area of the local environment. PTZ cameras can be permanently mounted at venues using wall mounts or used for portable applications by mounting on tripods.

Auto-tracking, basing on video analysis technology, is an advanced function of PTZ cameras that enables a single one to cover a large area and automatically detect, follow and record video of moving objects such as humans or vehicles.

It makes PTZ cameras versatile so they can be used in many different situations. The aspects to consider when choosing an auto-tracking PTZ camera include application, size of the space the camera needs to cover, including how much zoom is required, camera control, and powering camera and video signal transmission.

How much pre-programming do you need for your application, and how dynamic do you need your robotic camera to be?

Auto-tracking PTZ cameras create a dynamic live stream without the added cost and complexity of video crews and equipment. They can be used with SDI video production switchers, HDMI extension systems, NDI or IP video workflows and USB, and controlled using an IR remote (for short distances), joystick controller and software. These devices are now possible to control with software such as OBS, vMix, Wirecast, Tricaster, Mimo Live and Livestream Studio to name a few.

Typically, a robotic camera installation will comprise a number of remote PTZ cameras, remote controllers for easy PTZ control of multiple remote cameras, option boards to connect cameras to remote control panels, and video edge analysis. They also feature AI-led video analytics technology, chromakey-less CG overlay for supporting graphics, and PTZ auto-tracking and focus area cropping.

A single Ethernet cable can now be used for low-latency IP video, camera control, and to power a PTZ camera using a PoE (Power over Ethernet) network switch. PoE is an access layer technology that combines data signals and electrical power into a single Ethernet cable connection to enable remote-powered devices.

SCIENCE of automation

How much pre-programming do you need for your application, and how dynamic do you need your robotic camera to be?

Robotic-cameras2-Pic2-Kean-University-eSports

LIVE ACTION WITH BOTS

Two years ago, Big Voice Media Group was contracted by Swimming Australia to provide the host production for the 2021 Australian Swimming Championships and Australian Age Swimming Championships streamed live to Amazon Prime Video. Held at Gold Coast Aquatic Centre in Queensland, the entire event was held outside – forcing Big Voice Media Group to confront challenging weather conditions throughout.

“Swimming is one of those interesting sports where rain doesn’t come into it – the athletes are going to get wet regardless,” says Big Voice Media Group co-founder Kristian Shepherd. “As a production, we needed to have cameras that could work no matter the conditions – rain or no rain.”

The event ran for 13 days – both day and night sessions – and there was constant rain across the opening three days of the Australian Age event. It would only have been suspended if there was severe lighting or dangerous weather in the vicinity of the pool. With that in mind, Shepherd and company enlisted Birddog A200 weatherproof PTZ and P200 PTZ cameras, which allowed the production to go on without any stoppages.

“With the image quality plus the rain wiper, anti-fog, and water resistance, the A200s ticked all the boxes we were looking for when approaching this type of outdoor production,” says Shepherd.

“This was our first interaction with Amazon Prime Video. So there was an expectation that the product that was delivered would reach a certain level of quality – across sound, graphics and picture.

“To meet that standard, we scaled up quite a bit from our normal set-up. For this production, we ran 12 cameras total, six of which were Birddog PTZ cameras – three P200s and three A200s. The A200s were probably the central part we wanted for this event because
it was outdoors.

“They still swim in the rain, everything still runs. The cameras still have to run in that whole set-up. Being able to clean the lenses on the A200s mid-show without having to send someone around to go and do that, having anti-fog to stop the lenses from fogging up, and the fact that they’re waterproof is just great.”

Robotic PTZ cameras are an ideal fit for esports events of all kinds and now a staple in its production ecosystem. For a recent project at Kean University in New Jersey, AV integrator Horizon AVL turned to Marshall Electronics and its CV380 POV cameras to enhance the learning, broadcast and competition experience in the university’s new esports centre.

In its 6v6 setup, Horizon AVL installed 12 Marshall CV380 cameras, one at every player station, to capture each POV position of the player. The captured footage is then streamed onto the team’s Twitch channel, as well as the space’s LED wall, where six players from one team are shown simultaneously as a group.

The CV380 delivers 4K (4096×2160), UHD (3840×2160) and HD (1920×1080) video while maintaining a discrete presence, often used for unique point-of-view shots. For Kean University’s esports centre, the team at Horizon AVL paired the Marshall CV380 cameras with an 8-megapixel 3.8-16mm varifocal lens to get each camera angle optimally dialled in for esports streaming.

In January, meanwhile, Watford FC Women unveiled a partnership for the 2022/23 campaign with Pixellot, provider of AI-automated sports video and analytics solutions, to deliver video streaming, data and analysis to take training to the next level.

As part of the collaboration, Watford FC Women are using the Pixellot Air integrated mobile solution for video capture, review and analysis. Pixellot Air is a portable device with a dual camera array and direct cloud upload. Alongside being easy to set up on any pitch, it’s capable of capturing 12 hours’ play.

LIVE ACTION WITH BOTS

“With the image quality plus the rain wiper, anti-fog, and water resistance, the A200s ticked all the boxes we were looking for when approaching this type of outdoor production,” says Shepherd.

After training sessions or competitive games, Pixellot Air delivers automatically produced video for coding, tagging and annotating on a dedicated coaching platform. Watford FC Women’s coaches can now study video footage, game highlights,
in-depth analysis and video breakdowns.

During the pandemic, with borders closed and international visitors banned, production company Amis Productions relied on Shotover Systems to get close to the action at the 2021 America’s Cup won by Emirates Team New Zealand. In addition to two helicopters equipped with Shotover M1s, the camera chase boat was a recycled Emirates Team NZ catamaran converted to keep up with the AC75 boats nearing 100kph and also kitted out with M1s, which are lightweight, six-axis gyro-stabilised gimbal platforms.

Also during the pandemic, for the first time live at the 2021 Super Bowl, CBS Sports deployed the Trolley Cam, a Shotover G1 gimbal paired with a Defy Dactylcam Pro, that could speed from one end of the Raymond James Stadium to the other, ziplining along a wire and positioned to provide the viewing angle of a fan in the eighth row of the stands in Tampa, Florida. The rig could travel up to 65mph and provided a look at the players from a vantage point not used before in the history of the Super Bowl.

In total, CBS had more than 120 cameras placed throughout the stadium, including 12 4K and 8K cameras to capture close-up shots during the game. The 4K cameras were controlled robotically from the stadium concourse levels, while two Sony 8K cameras were fixed on robotic gimbals from the lower field.

Since matchday five of the 2020-21 season, Sportcast, the TV production subsidiary of the Deutsche Fussball Liga (DFL), has been using a fully automated AI remote camera system called Robotic Camera. It focuses on one particular Bundesliga player, head to toe, following him around throughout a match everywhere he turns using tracking data – all fully automated.

The camera makes it possible to create individual and – especially important for international markets – localised content for media partners without requiring any additional staff. Broadcasters can share this content with their viewers in advance, as well as through digital channels.

The Robotic Camera can be installed in any position in a stadium. The system uses a robot-controlled camera mount on a tripod and a Grass Valley LDX C86N 4K with a Fujinon HA18x5.5. It is located at the centre of the grandstand.

Controlled autonomously by the Polymotion Player software by Nikon/MRMC (Mark Roberts Motion Control) at the stadium using tracking data, the camera always follows the selected player; tracking data is supplied by ChyronHego. Nikon/MRMC can take over control of the player software at the stadium from its London office at any time.

NBC Sports surrounded the 2023 Players Championship with comprehensive live studio coverage across NBC, Peacock and the Golf Channel from The Players Stadium Course at TPC Sawgrass, Florida.

NBC Sports used nearly 65 cameras around TPC Sawgrass to provide its more than 20 hours of live tournament coverage. Production elements and enhancements that viewers saw throughout the week’s coverage included a 25-foot-high Jitacam overlooking the 18th tee along with tracing technology, pushing along with tee shots as they were struck to provide an enhanced perspective on the flight of the ball.

By taking away the footprint of a large crane on the floor, the Jitacam offers enhanced creative expression possibilities by placing the jib above the subject with the freedom of a 360° arm rotation. NBC Sports coverage deployed the Jitacam with a Shotover G1 head combined with AR and ball-tracing technology to cover the final tee shot.

A FlyCam ran a span of 225 yards from the 16th green to the buildout behind the 17th hole to showcase the spectacle of the island green, with a Bunker Cam inside the bunker at the notorious par three. An Atlas Cam depth-of-field camera was deployed to focus on specific cameras walking from shot to shot, plus both a live drone and fixed-wing airplane provided aerial coverage from above TPC Sawgrass golf course.

NBC Sports surrounded the 2023 Players Championship with comprehensive live studio coverage across NBC, Peacock and the Golf Channel from The Players Stadium Course at TPC Sawgrass, Florida.

Machine PERSPECTIVES

According to Brian Allen, head of product design and development at CP Cases, robotics are transforming the outside broadcasting of sports events. “Between augmented reality (AR) and tech becoming smaller and easier to transport, we have seen leaps and bounds in outside broadcasting over the last decade,” he says.

“However, with the increasing demand for innovative OB that involves the viewers in the action, more pressure is placed on the abilities of robotics. Equipment needs to be safely stored, transported and ready to operate on demand – without falling victim to unpredictable environmental hazards.

“A recent use case is for a global winter event where CP Cases developed padded weather covers for robotic camera systems. With cameras and covers present throughout the event, the importance of both robotics and appropriate weather protection are abundantly clear. It saves both time and money, as people are not required to stand around in extreme weather. Keeping robotics cool is also key, with protective covers and cooling cases required when travelling.

“Robotics are compact and easy to transport, which makes them ideal for OB,” he continues. “The design of these robotic cameras makes them perfect for live sports broadcasting as they are compact enough to prevent obscuring advertising and spectator or photographer sightlines.

“Robotics aren’t at the point where they can fully replace humans. Instead of removing humans entirely from OB, their roles are simply changing. For example, instead of manually running a camera, teams are manning joysticks on virtual cameras.

“Robotics can’t predict the unknown, and with live sports broadcasting having so many nuances, humans still need to be present. However, OB has become much more seamless thanks to evolutions in robotic technology. While it’s unlikely producers and directors will be completely replaced by robotics, their jobs should become easier, with much more flexibility on location.”

Machine PERSPECTIVES

Spiideo CEO Patrik Olsson recently appeared with Wilfred Frost on Sky News’ Ian King Live. The interview discussed Spiideo’s AI cameras, automatic production and video analysis platform. Olsson and Frost covered the future of sports and how technology is driving the sporting world into new territory – from rights holders to leagues to teams.

“Essentially, what we do is automate the process of capturing and producing sports on video, removing the need for a camera operator and big sports production studios. We can still produce content that is rich and has audio commentary, overlays, sponsor graphics, scoreboard etc, but without any human interaction.

“We reduce the cost of production from thousands of dollars per game to just a few dollars per game. That opens up entirely new markets that have not been available before. We can now enable millions of games to be available online to viewers who were not able to watch these games in the past,” says Olsson.

“Our partnerships are mostly with rights holders. There is a movement across the world now where the media rights for mid-level sports are being acquired, and that is because we have automated production so we can lower the cost of the production. Then – all of a sudden – you can do pay-per-view, sponsored concerts, ads and things like that to monetise those rights.

“A good example of this is one of our customers in Minnesota who does 20,000 high school ice hockey games per season, and they do that with a team of five people. So every weekend, hundreds of games are running, all automatically produced, and made available to the entire ice hockey community in Minnesota,” he concludes.