Article

Three Core Benefits of Simulation for Software-Hardware Project Development

Simulation Demo

At Fresh, we design and build systems involving custom combinations of hardware and software.  To meet technical requirements and client needs, using simulation for software-hardware project development early and consistently is essential. It allows us to jumpstart software development, decrease hardware integration time, and increase system reliability, the three topics discussed in this post.

Benefit #1: Simulation Jumpstarts Software Development

Whether operating in the custom robotics or machine vision fields—or something else entirely—overall project architecture, designed to meet the system requirements, is usually determined early in a project. However, the selection of specific hardware often comes later. Hardware choices can vary based on project needs, project budget, and parts availability. Using a simple form of simulation for software-hardware project development which fits the requirements of system data flow helps prove out the project concept. It also adds value by enabling software development to begin before hardware decisions are final. 

For example, consider a project to generate a cross-profile of a trench, recording depth and temperature. In this example, the project architecture specifies that a profile depth scanner and a thermal camera will be used and the output from these sensors will be combined into a depth-v-temperature profile along the length of the trench.  

Even without the final hardware selection, drivers are designed which generate simulated data mimicking the output of each sensor and publish the data into the system. The software component that combines the output of the sensors into the cross-profile of the trench is also designed. Finally, a UI application is designed to display the cross-trench profile data. 

This simulated system can be quickly developed. Often a first quick version can be developed within one or two sprint cycles. This first version is used as a proof of concept for the entire system. Reviewing this proof of concept with the client helps give a concrete realization of the original requirements, sparks client discussion about revisions to the system or the requirements, and solidifies the direction of future project development.

All software components are developed and system startup processes are confirmed. A full software system is operational and data flow through the system is tested, even before the final hardware selection decision is made. Once the specific hardware components are chosen, the simulated output of the drivers is adjusted to match the hardware sensor output and the upstream software components are quickly modified as needed.

Simulation for software-hardware project development creates a visualization of complex systems for engineering teams.

Benefit #2: Simulation Decreases Hardware Integration Time

Integrating with hardware is usually a difficult phase in system software development. When working with custom hardware designs, the bringup and integration process can be as much about debugging the software which communicates with the hardware as it is about confirming the functionality of the hardware itself. Simulating hardware interfaces in some detail can lead to easier integration with the actual hardware when it arrives and a decrease in the problem space during system bringup.

For example, consider a project which includes a custom hardware PCA board involving multiple sensor components and a small microprocessor. The microprocessor handles the initialization and interaction with the sensors, and it also provides a simple communications interface for external software such as a CAN-style interface or a custom command-response design. 

A software simulation of the custom board is easily developed which implements the interface and also simulates the effect of reading data from the board and writing specific commands (for instance, to read the value of a sensor or write a configuration to a component). 

With the simulation in place, the driver software and the upstream system software are developed and tested. When the hardware arrives for integration, the simulation is replaced by the hardware and the software integration often happens much more quickly.

There are various use cases for which simulation for software-hardware project development makes sense.

Benefit #3: Simulation Increases System Reliability

System reliability is an important aspect of the specialized design projects implemented by Fresh. Our customers often request solutions with requirements to operate in harsh environments for long periods of time with minimal operator supervision. In cases like these, testing for longevity and system durability before delivery to the client is crucial to meeting the client requirements.

For example, consider a project to use a commercially available manipulator to change and operate custom tools during a manufacturing process. A more complex simulation environment that can provide both a visualization of the operations and a simulation of basic physics interactions might be used, such as Gazebo for ROS 2-based designs (autonomous vehicles or manipulator arms) or Unity for larger systems (perhaps a crane or ROV). Simulated versions of the manipulator and tools are generated using schematics from the manufacturer support sites or CAD designs. These are assembled in the simulation environment to produce a fully functional “twin” corresponding to the real-world operational environment.

The upstream software which controls the tool-changing system can now be run for long periods of time and perform a large number of iterations of the tool-changing process without requiring constant hardware availability and operator supervision. Performing simulated cycles of operation can find issues related to the upstream software which might affect reliability, such as memory usage or system performance issues while under maximum load. These issues are discovered and fixed quickly using the simulation environment and then the full system testing cycle is performed repeatedly. 

New software system features are developed and tested by the engineer at their desk, using the simulation environment, before being tested against a hardware system. The simulation environment also allows for the introduction of specific hardware-related errors in order to test error handling in the software system. 

Working in this environment greatly decreases the find-to-fix cycle for software issues and increases the number of bugs fixed. The addition of continuous integration and testing against a simulated system upon merge/checkin of software code provides for a continually high level of software functionality and reliability. The simulation might not provide extremely complex and detailed physical simulation, but it still allows engineers to verify software functionality and develop additional features and testing scripts to increase reliability. 

Summary

Fresh uses simulation for software-hardware project development early and consistently through projects, and by applying these methods in our project development, product quality has increased, as has client satisfaction.

Please contact us if you are facing challenges like the ones mentioned above or any others in the engineering space. We would love to connect.

Understanding-Robot-Factors
Jeff Soesbe Pic

Jeff Soesbe

Sr. Robotics Engineer

Jeff has over 25 years of experience making systems more valuable for customers.

He started his career as a senior engineer at Apple, working on telecommunications, serial cards, and protocols. Jeff later served at HP developing internet printing solutions and security software, and most recently at Schilling Robotics developing ROV simulation solutions, hardware drivers, and interface applications. He earned an MS in computer science from Stanford University.

In his spare time, Jeff likes writing science fiction and fantasy, running math contests for grade school students, going to theater performances and concerts, and cycling.

37342

John Houston

Robotics Manager

John brings years of experience in engineering, data science, and development to leading Fresh’s robotics team. Throughout his career, he has served as a builder and strategic consultant, bringing his expertise in the industry to dozens of client projects.

Since starting at Fresh, John has designed and built multiple localization and navigation systems, safety systems, computer vision algorithms, robotic chassis, simulations, testers, and software architecture packages. He also brings deep expertise as a team leader, helping guide the launch of United Rentals’ autonomous vehicle program and providing practice area leadership, including onboarding, employee development, supporting sales, and contributing to a broad array of thought leadership.

John holds a BS in Computer Science from Gonzaga University, with a focus in data science. 

Outside work, he enjoys automotive fabrication, building cars and motorcycles, and spending time outdoors.

 

harshil

Harshil Shah

Robotics Engineer

Harshil is a graduate of the University of Washington with a Master’s in Mechanical Engineering and specializes in controls and robotics.

His past work includes sensor and navigation integration, odometry, algorithm-based state estimation, designing R/C aircraft, and developing a waterjet propulsion system. Harshil’s technical expertise includes Python, C++, ROS, MATLAB, Simulink, and instrumentation.

Outside of work, he enjoys playing guitar, joining soccer and cricket games, and competing in strategy-based board games.

Steve Yin Picture

Steve Yin

Principal Software Engineer

After finishing his Ph.D. study in ECE from University of Illinois at Urbana-Champaign, Steve started his career in advanced electrical and biomedical engineering R&D work for clinical applications. He later became a tech entrepreneur in new product development and commercialization.

Steve’s specialties include R&D in signal and imaging processing, computer vision, and AI/ML, along with data analytics, program management, and continuous improvement. His most recent work is in the healthcare and education industries.

When he is not working, he loves traveling, reading, and outdoor activities like jogging and hiking.

Raghav Duddala (2) (1)

Raghav Duddala

Robotics Engineer

Raghav is a Robotics Engineer at Fresh with extensive experience in Python, Linux, ROS, and MATLAB.

Prior to joining Fresh, Raghav most recently worked implementing, training, and optimizing deep neural network architectures for object detection and built simulation frameworks for solving R-based robotic manipulation. He earned his Master’s in  Mechanical Engineering from the University of Washington.

In his free time, Raghav enjoys playing games of cricket and Catan, hiking, and integrating filmmaking with AI.

mikey_headshot_b_edited (1)

Mikey Weller

Sr. Software Engineer

MIkey is a software engineer with over 15 years of experience developing networked embedded systems, and a particular focus on distributed systems and back-end orchestration.

Prior to joining Fresh, Mikey ran a bespoke mechatronics consultancy where he developed a SXSW installation for Capital One and an interactive art installation for the Market Street Prototyping Festival in San Francisco. Mikey has also been a part of several cloud startups, where he developed platforms to support activities such as orchestrating fleets of robots.

In his free time Mikey enjoys backpacking, sailing, and playing defense on the Fresh soccer team.

Avinash-Singh.jpg

Avinash Singh

Robotics Engineer

Avinash (Avi) is a Robotics Engineer who received an ME with a focus in robotics and controls from University of Washington and has accumulated over four years’ experience in robotics and electro-mechanical/mechatronics.

Avi interned at a drone startup doing CAD, prototyping, and testing of a morphing drone, and has worked as a Mechanical Engineer for an engineering and construction company. While earning his ME, he built an insect-scale winged robot and developed a controller in MATLAB/Simulink. His skills include LabVIEW, Python, SolidWorks, and experience with fabrication and rapid prototyping using laser cutters, 3D printers, and machining.

Outside of work, Avi is a hobbyist photographer and foodie who loves soccer, hiking, and travel.