A couple of years ago, AJ Hackett Bungy New Zealand had a couple of problems.

“A customer would whip out the latest iPhone, film their jump, then say, ‘What I’m shooting is better than what you’ve got because your cameras are five years old,” recalls Henry van Asch, who co-founded the firm 35 years ago.

Punters also wanted video and photos by the time their feet were back on the ground – and in a format they could immediately share on their social media channels.

Another headache: Backpackers had all but disappeared from NZ with pandemic border closures, drying up the tourism workforce.

That was keenly felt because “our media production process involved video and DSLR cameras operated by up to 35 photographers and a director, similar to a live broadcast”, said AJBNZ technology head Dan Waugh.

Waugh was charged with creating a new system that could address all of those problems at once.

Working with development partner DWS, he and his team created a system that uses a dozen iPhone 15 Pro cameras, which work with two custom-developed apps: EpicShot Experience Capture and EpicShot Guide App.

“[With] our old set-up, to put a site in was around $400,000. This one is around $30,000 per activity to do.” (An “activity” is a single AJ Hackett attraction, like the SkyWalk at Auckland’s Sky Tower).

It also has lower running costs because, Waugh said, the system doesn’t require a technical director cueing up feeds and wrangling cameras during a jump, or an editor to put the footage together. The system now automatically produces a clip ready for a customer to post to their Instagram or TikTok account.

This week in Auckland, AJHBNZ’s two Sky Tower activities –-SkyJump and SkyWalk (which have 21 iPhones capturing the action between them) – will begin using EpicShot before the system is rolled out to the Auckland Bridge Bungy and Kawarau Bridge in Queenstown in the next few months. Its companion software, the EpicShot Guide App is also being launched this week at the Sky Tower for AJHBNZ’s guided operation, SkyWalk, before being introduced at the company’s Auckland Bridge Climb activity.

And Waugh is also heading a new business unit, also called EpicShot, which will look to sell the system to other adventure tourism operators. Costs will depend on the set-up.

It’s not an Apple show-pony project. The tech giant only learned about Waugh’s efforts after someone at Spark mentioned that AJBNZ was buying a couple hundred iPhone 15 Pros.

Why did AJHBNZ choose the iPhone 15 Pro over an Android phone?

“A lot of it was the camera, and testing the camera,” Waugh said.

“But also the ecosystem of what’s behind it, because we used Apple’s SDK [software development kit]; we used their Core ML [the initials stand for machine-learning – an application of AI].” Apple’s augmented reality framework, ARKit, was also used to help develop the new system.

“We thought, ‘We can get all of the things we need within this package, the software and the hardware side of it’. You pay a bit more for it. We could have got a cheaper Android device, but it worked for us in terms of having the whole ecosystem.”

Machine-learning is used to trigger recording, working in concert with each iPhone’s lidar (laser detection and ranging) sensor, which is used to identify and track jumpers. Using a combination of lenses and camera modes on iPhone 15 Pro – including the 0.5x (13mm) Ultra-Wide, versatile Main (24mm), and 5x (120mm) Telephoto cameras, as well as the slow-mo and burst modes – the company has dramatically improved the quality and consistency of its photos and videos, Waugh said.

“We have what we call ‘digital twins’ so each iPhone has a ‘twin’ in the cloud,” Waugh explained. “When a customer walks out, it detects there’s a person in the frame – and that’s using Apple’s Core LM to do that. And then that tells the other cameras to either start recording or take a photo.”

Before the coding came the toy version. Literally.

“We got a Lego set up with little figurines because we had to determine if we could detect the person with an iPhone,” Waugh said.

“We prototyped that and proved we could. Then we built a wooden setup of a jump deck, and we set up all the iPhones, and we brought people forward and tested all of that. Once we proved the system would work, that’s when we went into a full build-out of it.”

Van Asch told the Herald that photographers and technical directors whose roles have been superseded by the system are being deployed elsewhere in the business in what remains a tight labour market for his sector. “The internationals are starting to trickle back, but it’s still tricky he said. (He also bemoaned that many Kiwis still don’t see tourism as a proper career.)

Some are stepping out from behind the camera to be iPhone-toting guides, who can use the EpicShot Guide App to create custom content at key points around an activity like the SkyWalk.

At EpicShot’s Auckland Sky Tower launch, the co-founder said it was a long way from 1989, when he and AJ Hacket would “[stick] a film canister on a bus, then send it into town to get developed”.

His sales pitch to other adventure tourism outfits: “Social media is the new TripAdvisor, and this is the tool to help you stay on top of it.”

Chris Keall is an Auckland-based member of the Herald’s business team. He joined the Herald in 2018 and is the technology editor and a senior business writer.