Neutral wearables testing allows for a better understanding of products, people, the market, and how consumers will perceive and use them. In 2017, with the understanding that exoskeletons had reached a level of market acceptance and offered capabilities that had barely been tapped, the Army sought out help in understanding the engineering nuances and human-machine relationships of exoskeletons. Previous Army experience with exoskeletons had left the R&D community with an understanding of exoskeleton knowledge and data gaps and what they needed to know in order to hone in on technologies that were ready (or close to ready) to meet Soldier challenges. Rather than accept vendor claims, the Army sought to arm their program with engineering knowledge, strong consumer participation, and continued quality ergonomic assessments. With a history in performing “neutral testing” on a wide range of products in the commercial, medical or Defense realms, U.S. Army Combat Capabilities Development Command – Soldier Systems (CCDC-SC) hired Boston Engineering (BE) to perform independent engineering analysis of the exoskeletons selected by Army stakeholders for consideration.

In May 2020, the Exoskeleton Report team had a one-hour interview with the BE Defense and Security Team – Jose Vazquez, Jim Lestienne and Cory Hofmann who were eager to do a deep dive into their contributions to the world of able-bodied exoskeletons:

What is Boston Engineering?

Boston Engineering is a small company (roughly 70 people). The company was formed in 1995, by two former Foster-Miller engineers – Bob Trieber and Mark Smithers. Foster-Miller (also Massachusetts-based) was well recognized in the Commonwealth for its textile and materials research and Navy Small Business Innovative Research (SBIR) activities. With extensive experience in engineering, problem-solving, and product development, the founders identified a gap in technology commercialization/getting a good concept to market. They founded Boston Engineering to focus on engineering services and commercialization, among other.

While for purposes of this article our interest is in exoskeletons and the Defense work, their client base is mostly commercial, industrial, and medical (they have touched items you probably use). Boston Engineering (BE) helps their clients develop products from scratch or improve the product, overcome policy hurdles (such as FDA) or commercialize for market. They practice “creative business” where they can work with a client with anything from a casual brainstorm or a napkin sketch, one of a kind or large quantity of a product. Whatever it takes.

When and how did you get involved with Exoskeletons?

ExR notes that Jose was a former Department of Homeland Security Office of Science and Technology First Responder Director. He had partnered with the Army in the early 2000’s to assess the Lockheed Martin HULC and other products for Responders. As Mark accepted a contract for exo work with CCDC-SC, Jose was joining the company and supported the exoskeleton independent engineering analysis project (among other):

BE: “With the field of exoskeletons taking off, we had considered the possibility of developing exoskeletons by teaming with a North American company, but the company’s limited understanding of how to do business with DoD and the market opportunities made them very uncomfortable. It just didn’t pan out. We decided the market was moving along fast enough that it was not worth the serious cash needed to make exoskeletons from scratch without a robotic partner. Following the BE model of product improvement and commercialization, we stuck to our niche in doing what we do best: providing independent analysis and assistance to developers in addressing potential user technical concerns regarding a product.”

You don’t build exoskeletons? What services do you render that help the exoskeleton world?

“We felt we can have a broader and bigger impact on the industry by providing independent engineering analysis and support services to not only government but industry and academia entering the exo world. It was a conscious business decision not to develop and market an exoskeleton but to focus on support services, independent analysis and assessment, commercialization, standards and test methods that can help advance the exoskeleton industry.”

Talk about your relationship with the Army/Defense and other potential customers?

“We are working with the U.S. Army CCDC-SC, in Natick, Massachusetts as the independent analyst/3rd party support services component of their exoskeleton program. They selected the systems and we then look at the products to see if they can be used “as is”, adapted or could be suited for military use. The Army sets the expectations on which areas they consider the primary targets for exoskeleton (s) use. We then provide a variety of technical services related to analysis, assessment, and evaluating the products in laboratory and Soldier settings. CCDC-SC’s program is part of the Army Futures Command (AFC), working hand-in-hand with Program Executive Offices for the Army and other sister laboratories to support a broader, larger Soldier-centric ecosystem to get innovative technologies to the users. The Army team works as a lead partner with other Defense agencies with similar interests, NATO and other government agencies (OGA). Whereas we are mostly working with CCDC-SC we are developing relationships with OGA (Airforce, Defense Logistics Agency (DLA), DHS etc.). We like to think that we are “a trusted advisor” by offering analysis, advice and guidance on the technical aspect of products, and being technology agnostic. Our job is not to favor a product over another but provide objective information and data they need regarding the technology and its current and potential use. We help them find the right tool for the task.”

Without disclosing confidential or sensitive information, what have you learned a) about the exoskeleton market, b) the technologies you work, or have worked with?

BE: “How long do we have? That’s a loaded question! Let’s break it down:

The market: This market moves fast in some ways and not in others. Some technologies evolve at a pace that is hard to keep up with, while others get stuck in a rut and oversell their product (s) or try to sell their product before it is ready. It makes it difficult to weed information. It is not a cohesive community and earlier marketing strategies did the community a disfavor. Oversell and under deliver. Or sell before ready because investors demanded sales. Those who have taken their time, are breaking through the glass ceiling. Some companies excel at setting realistic expectations about their product (s) particularly when they take it from one use case to another (unproven and assumed) one. Other technologies oversell their readiness, maybe because they have a type of investor who was sales focused and not innovation-R&D strong. And there are pieces of the puzzle missing such as more FDA, Standards, Test Methods, certifications, and so on, that are needed and underestimated. There are pockets of in-fighting and others of collaboration. Big personalities. We see the market normalizing, but it isn’t there quite yet. Evolving but not there. Unfortunately, the market has not normalized and consumers don’t know what to expect, how to use, or implement products. Some markets such as warehouse or construction are natural environments for exos, and others, such as everyday use by a homeowner power washing their home not so much. There is a desire to see a broader market. As we said… not there yet.

The Technologies: Technologies are at different readiness levels. Different designs and concepts with different levels of challenge and performance. All concepts bring something valuable to the table, but it’s an art to place the right tech with the right tasks and the optimal user or client. Some companies do this brilliantly, and their technology is up to the challenge. Others can’t see the problem with their product and try to force it. Designs limit technology performance. For example: robotic systems, power and batteries, cables hanging, etc. are a challenge. For military uses, robustness of systems isn’t there yet – the tech may be good but wasn’t built for a beating… However, its safe to say, it isn’t always the technology. It’s the people behind the product who become over-zealous and sometimes don’t or won’t listen or the product isn’t suited for the task. It’s not always the technology.”

The industry:  “Is at a fascinating crossroad. As engineers, we are in awe of the significant impact on the human body that is achieved with “relatively simple” mechanisms. With something as a spring or linear actuator, exos have a significant impact on a wearer. However, developers oftentimes don’t have a good understanding on the needs of the users they are targeting. Industry “fiefdoms” have created some level of friction is a challenge to how the community can help build a better exoskeleton industry base and market. There is a thin line between the industry of exoskeletons and the market (product/use and consumers). It’s not as “all over the map” as it was, but the industry still has some challenges to overcome. The proof will be in products being adopted and used.”

How do you perceive the value of 3rd party independent engineering analysis?

“Well used it can help advance the exoskeleton market by easing the understanding of products, performance, consumer perception and use, trends, design, cost, manufacturing, etc. It applies some scientific and engineering rigor to investigate the claims of exoskeleton manufacturers. We do that for a potential consumer or we can do it for an actual vendor. Not all companies can perform their own testing or independent analysis. We can assist or help them in advancing technologies to the point it is best suited for their potential applications. Recognizing the strengths and potentials of a technology from a user and engineering standpoint is extremely relevant and can have an impact on future sales.  An exo design might be great for a use/population but might not be for another. Therefore, in addition to this independent analysis, we think that we can provide value by helping advance designs that optimize for the desired end-state, which in turn helps “the bottom line”. We think it impacts on all of the critical facets of getting a product to market. It can play a role in helping create engineering design standards to appropriately evaluate a broad range of products while helping the vendor at hand improve their product. Independent analysis and looking at the human (and all its paraphernalia) and machine as a symbiotic relationship, we can advance the state of a product and its commercialization process, price point, market share, manufacturing considerations. It’s all related. It brings it together.”

Were companies willing to accept independent engineering analysis?

“The Army made this a requirement of their contract. Some saw it as a pain, others welcomed it. The smaller companies have been very collaborative, and we’ve already seen product improvements as a result of the process. Because we take each product independently and don’t compare them against each other, we can study each one on their merit and within their context. We established strict communication policies to keep information protected and maintain open comms with the vendors. We established the collaboration parameters in the early phases of the project and with each vendor, to put them at ease and guide them through how this would work. We try to be very explicit about our role and set communication guidelines. Only the Army has access to all the vendor’s data at any given time. Each vendor receives its data/information based on what is negotiated upfront and what the Army will allow. The end-product belongs to the Army, and while this may make some companies nervous, they feel it’s in their best interest. There are no surprises. We prefer to be independent/neutral.

There have been some that elected not to participate, for whatever the reason. For the most part, the companies have been more than open to the collaboration and welcoming of our feedback. It hasn’t been all rosy, however. We have had some disagreements and had to work through it. It was expected.”

Does BE believe the Army found value in this process? What is BE’s perception of the Army’s experience? What about BE’s experience with the process and the Army?

“Yes. They welcomed the independent analysis and supporting them bridge the gap between the technical and the research realms, between the engineering, mechanical, electrical, and biomechanical/physiological world. We are helping understand products within the physical demands of the tasks that these Soldiers are performing and how different products approach challenges differently. By understanding the activities, the tasks, the demands on the Soldiers, the mechanics, the movements we can help them select and match the better product for those tasks and use cases. That has been a significant benefit as to this program. Another way they have seen value and benefit is our multi-faceted approach to task evaluation and even a demonstration of these technologies. We look at it as engineering analysts and not developers. We are testing, evaluating in the lab, and the field (pre-Covid). We look at the human factors, system integration, safety and reliability, power (where applicable), noise, robustness, durability, etc. We are looking and observing how exoskeletons are worn and used, user and operational issues, things that might get snagged, anything that might cause harm to a Soldier. This multi-faceted approach to evaluation and presentation of the technology knit with the analysis makes for a much better understanding of the product and its potential use. We bring it together for them. BE’s independent analysis work includes an even wider range – almost like a consumer report: performance under given conditions; validate claims; literature and patent searches; component analysis; failure mode; design for manufacturing (it works but can you make it in quantities?); hazards, etc.. We get to see a lot of different aspects of what it takes to get an exoskeleton to work, work well and get to the user.”

What have you learned, and what would you change to improve the ability to get exos to the consumer?

“In terms of improving the exo industry’s ability to get these products into the hands of the consumer, too many factors. Among them, as an industry, it needs to be better at providing evidence and not overinflating the potential expectations to potential customers, end-users, and investors. It is important to communicate and set clear expectations of what these exoskeletons can do, but perhaps more importantly, what they CAN’T do. A lot of people have these visions of science-fiction/Hollywood, and that sets false expectations that the data can’t support or these technologies aren’t ready to support. We have seen products designed originally for one use that assumed that the safety factors in that environment were transferable to a military environment. When did the analysis break down, the safety margins varied tremendously and could have been dangerous. As a result of the analysis, we identified the issue and were able to recommend the right use of the product (s) during field trial. Thus the industry needs to not only capture data to better understand how these designs (full system, components, materials) are behaving and how these designs are influencing the user. We need to do a better job of communicating the data objectively to avoid potential problems. We think that is going to be an importing pairing: not just capturing the right evidence but also communicating that evidence as broadly as possible. The industry should find a better way to improve consumer trust without overselling or underselling what these technologies can or potentially do.

There won’t be one “magic” fit solution. It will be key to carefully assess what measured and what is a leap of logic based on that measurement and what is a little more reasonable. You have to be very careful with what evidence you are capturing, what the data is telling you, and make sure that there is always this back and forth communication between the full team (engineers, researchers, marketers, decision-makers, etc.). We’d also focus on safety and reliability analysis, to name a few.”

The Soldier testing includes the “softer” “non-quantifiable” data collection such as perception, comfort. Why is it important? Is it irrelevant?

“It’s all-important to the Army because what the Soldier has to say and how they are going to use technology matters to whether or not the Army will buy it and field it. BE treats them as extremely important. We couple traditional engineering measures with a significant amount of usability and human factors/consumer feedback type of analysis (thanks Cory…). These include things like: comfort over time, donning and doffing, limitations to the don and doff (and is that dependent on the environment? Or is there a design challenge?). The softer (consumer feedback or “Soldier Touchpoints (STP) in Army jargon) and quantifiable data (testing) is important for acceptance and adoption. If an exoskeleton provides a substantial benefit and reduces stress to one area of the body but is difficult to put on or is uncomfortable won’t be accepted or used. The converse is also true if we can quantify with objective data that a design has a small benefit but is extremely comfortable, very quick to don and doff, should it be bought? We are obligated to put a substantial amount of effort to understanding both aspects, the subjective and the more objective quantifiable data. You really can’t have one without the other. CCDC-SC does the bulk of the consumer analysis and biomech team and then we take their data and marry with the rest of the analysis. We bring it together.”

Powered, robotic, soft exos. You have seen and tested in-lab and on Soldiers a variety of products. Where do you think the areas of greatest impact reside?

“Design and integration once you figured out it works. It depends on the product as you highlight. For example, in terms of the powered systems, power supply and management is huge. Is it providing enough power? How many batteries? Overheating? Every time you have these electro-mechanical systems [there is a concern] how much weight is added by the batteries or the system itself? Power and water… can it be submerged? Can it be optimized to provide power for long enough durations? Heat – the minute you add any extra layer on a person, it will create heat. As applied to all the designs in general, material analysis for comfort and breathability is important. This is true even for any system. There is a high likelihood of seeing overheating, discomfort, chafing, or blistering depending on where the suit is fixed to the body or the design, and we have witnessed this in the field. How comfortable can we make this system to interact not just with the end-user but with the kit the end-user might be wearing? Soldiers have their clothing and equipment systems. Exo systems must be compatible and integrated with whatever a user already wears. System integration tends to be overlooked. We don’t. Women and men Soldiers. Different sizing and designs.

What about the ability to turn-off on or off easily? Or disengage a product to improve your own doff time, improve the mechanism by which you can remove the exoskeleton from your body or someone injured? Simply put, get it out of the way when it needs to get out of the way – turn on or off when you need it, take it off when you need to. Does it restrict normal movements? The bigger areas for improvement are those that can have the most impact on the usability, comfort, and the effectiveness of different exoskeleton and exosuit designs.

As a follow-up, in general, what should be the primary application, logistics, patrols, load-bearing? What is the greatest potential benefit of having an exoskeleton?

That is something the Army and DoD are working on, precisely by buying and testing systems and getting user feedback. The Army has extensive injury data and operational data that has sent them in directions for first use. They have made it clear that their priority is Movement and Maneuver followed by Combat Support / Sustainment. In layman’s terms: getting Soldiers from point A to point B faster and fresher/less degraded which in turn improves their ability to complete a mission/fight. Combat support or sustainment is more of the “civilian” type of tasks that are repetitive and assume lifting, carrying, building, moving stuff around. There is no “silver bullet”. It is all missions based and Military Occupational Specialty (MOS) tasks relevant.  BE helps understand what the use cases and physical demands are on Soldiers for each type of task and activity and tries to match the technology.  The greatest benefits for the Army so far are in moving people from point A to B and minimizing physiological and cognitive degradation because of the length of the march and the load they carry and also in protecting the body from wear and tear due to repetitive motion, particularly with a load. We are seeing the evidence in the field and in industry.”

As a relatively Small Business in Massachusetts, what does having this type of research partnership with the Army and academics such as the University of Massachusetts Lowell mean to your company?

“What it does is it allows us to connect to the ecosystem of the state and the region and be able to collaborate with them. By working with CCDC SC we have improved partnerships with the NERVE Center at UMass Lowell [The New England Robotics Validation and Experimentation (NERVE) Center at the University of Massachusetts Lowell] and Northeastern University. It has allowed us to work with small companies here in Massachusetts as well as other places, such as New Stone Soup VT, LLC, a Woman-Owned Small Business (WOSB) that has served a connector between the Government – Academia – and Industry. It’s CEO and Founder has been the glue that has brought all of this together [Rita Vazquez-Torres is the most frequent contributor to the Exoskeleton Report and helped organize this interview] because she brings years of government experience and a strong network. It has increased outreach capacity and understand the Army and its needs.

What are the top three challenges BE considers essential in the future and broader adoption of exoskeletons?

“Education! Education is not just marketing but a lot about aligning expectations with reality. Setting realistic expectations of the performance, what it is going to do for you. It also means addressing stigmas, stereotypes, and fictitious expectations. Education in the context of use (and this is where BE adds value) by matching the right technology to the use case and tasks of soldiers. Consumer awareness. Staying on top of the rapid evolution of the technology, the industry, and new players that are coming in and getting to market and putting your product in that context. Keeping track of data and publications. Educating the government – FDA, Health and Human Services, NIST, etc. the agencies involved in consumer product safety. The insurance industry. We can’t understate how important this is.

PRICE! The price point can be a significant limitation to widespread adoption. Depending on what your product can do, will a client be willing to pay for the benefit? Cost and benefits matter.

Standards and Test Methods = improved safety and reliability. People want to know that exos are safe and they meet NIOSH or OSHA, FDA, or workers’ compensation requirements. Standardization can help normalize the market and price-point.

What recommendations does BE have for Exoskeleton newcomers wanting DoD or Government business?

“Understand that working with the Department of Defense is not the same as working with DHS or with a private company. While DoD embraces innovation, it also expects responsible “selling” of your capability and what it can do for a Soldier. You simply can’t build a product in a garage or a lab or a university setting while testing it on a limited civilian population and then put it on Soldiers and expect it to work. It is also a misconception that if you take a technology out to one of the schoolhouses (such as Ft. Benning)  for a “technology day” – that it automatically buys you an “in” that = sales. Government agencies all have different requirements, operational environments, people. Rules are different for testing and buying for different agencies. Testing and/or selling to DoD or a Federal agency is differs from State or local government. Understand that when it comes to exoskeletons, all roads lead to the same core of people. And they talk. Frequently. Good and bad. They share their knowledge and learning curves. No one is immune in this small world. In the U.S. and throughout the NATO community. They will talk about your product and business practices.

Partner – If you don’t understand how to do business with the government – find a partner that does. Teaming with companies like BE, who have built relationships with DoD, can help shorten the delta. Working with the right level of the defense department, working with their R&D community, can be useful for developers and innovators. Become connected with connectors. Network with the networkers: Exoskeleton Report does a great job in spreading the gospel, as does ASTM F48 (addressing Standards and Test Methods), associations. Working collaboratively rather than individually and sharing information with the community will go a long way with DoD.

Listen. Carefully. Shut the voices in your brain. Developers become so enamored with their product that they do not listen to what the “customer” is saying. We have seen developers with high potential to take the market by storm flatline and not get the next contract (s).  Those who listen have a better chance of success.

Gather your facts and data. Check your data. Be honest about your product. And don’t badmouth the competition.”

This concludes our exclusive interview with Boston Engineering. We hope that you have gotten to see first-hand how the exoskeleton industry is viewed from the eyes of a 3rd party independent exoskeleton evaluator with a focus on able-bodied and DoD applications. Special thanks for taking the time to answer questions go to Jose Vazquez, Director Defense and Security (DAS) at Boston Engineering Corporation, James Lestienne, Business Unit Director at Boston Engineering, and Cory Hofmann, Technical Lead, Exoskeletons at Boston Engineering.