8 min read / February 1, 2022 / Komatsu Staff Writer
Could one person control all the earthmoving on a job site from a remote location? While the idea may seem far-fetched, it’s closer to reality than you may think.
“The market sees this possibility evolving and coming to fruition at some point down the road,” said Komatsu’s Jason Anetsberger, director of customer solutions. “Mining, has had autonomous trucks hauling billions of tons of materials for quite some time, which gives us an indication of what’s possible.”
Anetsberger added that remote operation, or teleoperation as it’s often called, and autonomy on construction sites are “assets that customers continue to express interest in, especially considering the shortage of workers in the construction industry. Having one or a few people controlling large numbers of machines from an off-site location has the potential to lower costs and increase safety.”
To a degree, the construction industry has already been using remote operation. Several manufacturers offer remote-controlled equipment that allows operators to run machines from outside the cab. These have generally been used in highly sensitive and/or dangerous areas — such as cleaning up land mines or moving contaminated soils — but require operators to be on the site, albeit at a safe distance.
“That concept is now coming to the traditional construction site,” stated Bryce Satterly, Komatsu Smart Construction solutions manager. “The evolution is to move the operator fully off-site and at a greater distance and, instead of being able to run one machine with a remote, they could control several. With teleoperation, they could dig a utility trench with an excavator, switch to a wheel loader to place bedding rock, then run a roller to compact it – all from a remote workstation and out of the elements.”
Remote operation, or teleoperation, can be done at a distance from the machine, either on the job site or from a remote off-site location.
There are already tools that can help prepare operators for remote operation. Sophisticated training simulators use virtual reality to bring the look and feel of a job site task to operators and potential operators.
“Using simulations to deliver training is one of the most effective training methods,” said Jim Colvin, president and CEO of Serious Labs Inc., which offers virtual reality training solutions. “Simulation provides a virtual environment that mirrors actual work conditions, including workspaces, background noise and effects.”
Colvin noted that simulations are continuing to gain popularity in construction training. “There are ways of training nowadays using VR (virtual reality) and AR (augmented reality) that are more engaging, immersive, interesting and effective than traditional training methods,” said Colvin. “Plus, they can be used… without really putting them in harm’s way.”
In addition to safety, the benefits of simulators include faster training, reduced costs associated with fuel and the wear and tear on machines, and the ability to assess a potential operator’s skills before they get in a machine.
Attendees of MINExpo International® 2021 were given a glimpse into the potential future of teleoperation. From the show floor in Las Vegas, a mining excavator—located at Komatsu’s Arizona Proving Grounds more than 400 miles away—was operated remotely. There were no operators in the cab of the excavator or in the cab of the fully autonomous truck being loaded, but multiple cameras and sensors were around the machine and bucket, and there was a 360-degree monitor with machine display to support visibility and situational awareness for the remote operator.
Attendees of MINExpo International® 2021 in Las Vegas could see an operator loading material into an autonomous haul truck via remote operation of a mining shovel at Komatsu’s Arizona Proving Grounds over 400 miles away.
“Scaling that down to the construction site is being researched and developed,” said Satterly. “Mining is a little easier to run remotely and autonomously because typically the trucks are running on a fixed route and the loading machine sits in one position for long periods of time. The difference is that construction sites tend to be more fluid and congested, with a lot of movement and machines are often in multiple areas throughout the day.”
“Most machines today have one or multiple cameras that help the operator see behind and around them,” Anetsberger added. “In the cab they can see the entire site and know where the machine they are operating is in relation to others. Remote operation may involve more cameras to ensure they have that view.” He concluded, “There are logistics to work out, but eventually the industry will get there.”
While the construction industry takes initial steps toward fully automating and teleoperating machines, projects can be managed remotely. In the past, project managers often had to drive from one location to the next, but now they can check on multiple sites from their office, vehicle or job site trailer.
Cell phones and computers ushered in the ability to communicate faster between the office and field personnel, and smartphones are making it even easier with applications designed for timecard management, productivity tracking, job site management and planning and more.
“There are a multitude of apps available that let project managers view the job site in real or near-real time right now,” said Satterly. “With digital plans and satellites, they can view an intelligent machine’s monitor and see exactly where it is on a site and its relation to the final elevations. It lets project managers see productivity remotely and make adjustments faster. If there is an issue such as a change in plans, they can upload it to the machine remotely. That can save time and money, and increase productivity.”
Remote control machines have been available for several years for use in highly sensitive and dangerous areas. “The evolution is to move the operator fully off-site and at a greater distance, and instead of being able to run one machine with a remote, they can control several,” stated Bryce Satterly, Komatsu Smart Construction solutions manager. “With teleoperation they could dig a utility trench with an excavator, switch to a wheel loader to place bedding rock, then run a roller to compact it — all from a remote workstation and out of the elements.”
Hunzinger Construction Company, based in Brookfield, Wis., used drone flyovers to map the site of a large construction project and tracked progress on an app. Data from the flights was used to adjust original cut/fill plans.
“We compared what we saw as existing grade to planned subgrades and found that there was a severe overage of fill not accounted for that would have to be hauled off-site at great expense,” said Andy Rodenkirch, senior project manager, Hunzinger Construction Company.
“Using information from the drone flyovers, we were able to have the design team adjust the grading plan to balance out the site,” Rodenkirch continued. “Completing this adjustment saved time and labor by eliminating rework, ultimately keeping us on schedule and budget. The app was a great tool to collect information we need throughout a project.”
Another Brookfield-based contractor on the same project used a remote application to view and update digital 3D plans. CornerStone One LLC was responsible for digging building foundations and installing site utilities.
“I can log in from anywhere I have internet access and see where we are at the moment in relation to target elevations, and updates can be done in minutes as opposed to hours or days –a huge savings in downtime,” said Scott Christman, CornerStone One’s earthwork division manager. “It increases our flexibility and agility because we’re able to switch tasks or start on a change quickly, without waiting for a third-party surveyor to come lay it out.”
Pat Dorsey, a fleet manager with Taylor’s Excavators Inc. in Stanwood, Wash., recently began using an all-in-one fleet management tool that allows him to track machine hours, location, service and repair history, order parts, see fuel consumption and more.
“Without it, there would need to be five of me,” stated Dorsey. “Instead of running between job sites and checking each machine individually or going off weekly foremen reports, I can log into the app through my desktop computer or tablet and immediately see if there is an issue and address it faster. Instead of having to have a bunch of manuals sitting around, I can look them up and order parts directly from the app.”
Remote fleet management has removed a lot of guesswork, according to Brice Butler, equipment manager for Salt Lake City-based Morgan Asphalt Inc.
“In the past, there was a lot of reliance on guys in the field reporting their hours and production,” recalled Butler. “Understandably, they get busy and sometimes missed reporting that information. Being able to remotely get that information from the machine allows me to be more proactive in scheduling service and addressing potential problems, and it lets those guys in the field concentrate on the work. It’s better for all of us.”
According to Anetsberger and Satterly, virtually managing operations is a gateway to machine teleoperation and ultimately the autonomy of construction equipment.
“Virtual, or remote, machine management is a step in the right direction toward one day having fully-remote or autonomous job sites,” said Satterly. “Understanding how to use that information and apply it to equipment operation and overall job site management is invaluable. It could be a key factor in full autonomy and remote operation.”
Learn more about how Komatsu is helping to advance automation and remote management through its Smart Construction suite of solutions.