How We Used AT1032S to Hunt Down a HardFault in a Factory Network

We were working on a CANopen-to-Modbus converter for a factory project when the firmware decided to develop a personality.

Every now and then, the converter would crash with a HardFault. Not every hour. Not every test cycle. Not even every shift. More like once every one or two days, just often enough to be a serious problem and just rarely enough to be deeply annoying.

And of course, the bug only showed up in the real installation.

In a factory, that is a terrible place to play "let's try stuff and see what happens." The environment is mission-critical, the system has to keep doing its job, and nobody is excited about an engineer showing up with a cable and a chaotic debugging plan.

So we needed a different approach: bring the factory home.

Learn more about AT1032S

The Bug That Only Showed Up Every Other Day

On paper, the converter was straightforward enough. It listened to one side of the installation over CANopen, translated the data, and exposed what was needed over Modbus. In the lab, it behaved. In the field, it occasionally face-planted.

That is the kind of bug engineers love to describe with technical precision:

"It works fine until it absolutely doesn't."

The real challenge was not understanding that the crash mattered. The real challenge was reproducing it on demand.

Why Debugging in a Factory Is a Different Beast

When an issue appears inside a production environment, there are two immediate constraints:

1. You cannot freely poke at a mission-critical installation.
2. Waiting one or two days for the bug to appear again is not a serious debugging strategy.

Even when you can capture traces from the field, they do not magically turn into a reproducible test setup. Real systems have timing quirks, message ordering differences, startup races, and small delays that never look important until they are exactly the reason your firmware falls over.

So the mission became clear: we had to recreate the same traffic, the same timing, and the same operating conditions in the lab until our converter had nowhere left to hide.

Rebuilding the Real Network in the Lab

This is where the AT1032S became incredibly useful.

Instead of using it only as a generic test sequencer, we used it as a network troublemaker with excellent manners. It let us reproduce the CANopen and Modbus traffic seen in the field, control the setup around the device, and iterate quickly without needing the full factory installation on the bench.

That mattered because this was not just about replaying messages. We needed to recreate behavior:

• CANopen traffic arriving with realistic timing
• Modbus exchanges happening in the same sequence as in the field
• The converter booting and running under the same conditions as the deployed unit
• Small timing adjustments that could make the difference between "everything is fine" and "welcome to HardFault city"

Using AT1032S to Replay the CANopen and Modbus Traffic

NodeJS script controlling the AT1032S to reproduce the factory network in the lab.

Once we had the field captures, we used the AT1032S to emulate both sides of the conversation around the converter.

On one side, it reproduced the CANopen traffic pattern the converter had to process. On the other, it replayed the Modbus activity the system expected to see. Because the platform gives us control over communication, sequencing, and the broader test flow, we could keep refining the scenario instead of running loose one-off experiments.

This is where things started getting interesting.

At first, the converter behaved perfectly in the lab too. Which is engineer-speak for "we are still missing the one detail that matters."

So we kept adjusting.

Tuning the Timing Until the Setup Matched the Field

The breakthrough did not come from one giant revelation. It came from timing.

We changed delays. We shifted message spacing. We adjusted the order of events. We kept pushing the lab setup closer and closer to the real installation until the converter was running in conditions that were effectively the same as the field.

And then, finally, it happened.

The HardFault showed up in the lab.

Now it is a reproducible engineering problem, which is a much nicer category of problem.

Catching the HardFault in the Act

Once the failure became repeatable, everything changed.

We could instrument the firmware properly. We could step through the sequence. We could observe what happened before the crash instead of guessing from field reports and partial traces. And most importantly, we could confirm whether each fix actually solved the issue instead of waiting two days and hoping for the best.

That is the real value of recreating a factory network in the lab: speed.

Without a reproducible setup, debugging depends on luck. With a reproducible setup, debugging becomes engineering again.

What This Saved Us on the Project

Using the AT1032S this way saved us from the worst debugging loop of all:

• wait for the bug
• drive to site
• collect partial clues
• try a theory
• wait another day

Instead, we built a repeatable setup that let us compress that loop dramatically. We could replay the traffic, tweak conditions, reproduce the failure, validate the fix, and move forward with much more confidence.

For industrial products that sit between multiple protocols, that kind of setup is not just convenient. It is often the difference between a vague field issue and a bug you can actually eliminate.

Conclusion

In this project, the AT1032S was not just a test box on the bench. It became our stand-in for a real factory network.

By reproducing the CANopen and Modbus traffic and tuning the timing until the converter saw the same conditions as in the field, we turned a sporadic HardFault into a repeatable failure, and then into a fix.

Sometimes the hardest part of debugging is not understanding the crash. It is convincing the crash to happen in front of you.

That day, the AT1032S was very persuasive.

Learn more about AT1032S Contact us with your questions