7 Benefits of Automated Firmware Testing for EVSE Teams

Common Challenges of Firmware Testing

Whether you are managing a small or large-scale manufacturing line, firmware issues can lead to costly delays, compliance risks, and client dissatisfaction. So, for both EV charger manufacturers and operators, and regardless of the number of chargers, it is important to ensure that the system works steadily and consistently, especially after the updates. We at TimeTick know what problems may arise and have solutions:

• Compatibility of chargers with new firmware. Manufacturers and operators have to be sure firmware runs smoothly on a new charger; it should send the necessary data as expected, accurately respond to system or network commands, and process them correctly.
  How TimeTick helps: Check the accuracy of responses in a perfectly set up virtual environment.

• Correct firmware updates. The new firmware versions and configurations might cause issues if not tested beforehand, and some of them might not even be noticeable right after they’ve been rolled out.
  How TimeTick helps: Make sure an update meant to fix one issue won’t introduce new problems elsewhere in the system, as unstable updates can negatively impact overall performance.

• New industry standards. Staying compliant means continuously updating and validating firmware against the latest protocols, such as OCPP certification requirements.
  How TimeTick helps: Get a platform that’s regularly updated without your manual intervention or the need to track changes yourself.

• Anomalies outside basic prerequisites. Beyond certification, you may want to test your firmware or hardware under extreme conditions that are challenging or too costly to create.
  How TimeTick helps: Set a custom chain of events promptly, verifying behavior in edge cases that standard certification tests don’t cover.

• Inefficiencies and expenses. Manual testing is more time-consuming and pocket-unfriendly than automated one.
  How TimeTick helps: With a full-fledged testing platform, you save time and costs at every stage, from setting up the test environment to analyzing logs.

Aiming to help you overcome all these challenges, TimeTick provides a simulated environment that allows you to assess hardware and firmware simultaneously, making sure every element functions as expected and according to the latest protocols. Let’s explore this process and the platform’s features in more detail.

How TimeTick Works

When you test your EV charger’s hardware or firmware through TimeTick, you use a simulation that mimics real-world conditions. You can evaluate whether firmware is integrated seamlessly with the backend and if there is a risk for an update to cause service disruptions or violate compliance.

In general, you go through the following steps:

1. Choose one or more pre-installed tests to check the charger’s basic functionality and its interaction with the system.
2. TimeTick will run these tests automatically one by one, checking the functionality of your firmware. It will promptly evaluate the accuracy of commands and compare expected and actual outcomes. It can identify incorrect responses, slow response times, unusual patterns, repeated messages, and many other inefficiencies.
3. The system instantly identifies errors and provides a list of them, along with recommendations for eliminating them.

You can also test the firmware faster across various scenarios, ensuring the accuracy of its performance in any situation.

7 Key Benefits of Testing Through TimeTick

1. Getting a fully compliant testing system

The reliability of the testing environment will directly affect the reliability of the firmware itself. So, when running the tests using a correctly optimized cloud, you can identify all the gaps between the expected and received responses to a variety of commands.

TimeTick offers not just realistic but also fully compliant testing conditions. The platform is fully aligned with regulatory and industry standards for EV chargers, such as OCPP 2.0.1, and is regularly updated. It means a significantly higher possibility to get OCPP certified on the first try and get ready for market deployment.

2. Saving time

TimeTick can save a significant amount of time that differs from one process to another. For example:

• OCPP protocol development can be reduced from two months to three weeks.
• Regression testing — from two weeks to two to three days, and does not require HWQA involvement anymore. At the same time, testing efficiency improves due to advanced error detection.
• Dynamic testing — from two days to create scripts to several minutes to make adjustments to the existing predefined processes.
• No need to create demo environments for a few days but simply use the given one, setting it up within 1-2 hours.

In addition, some processes only become possible with TimeTick. For instance, it was impossible for a client to run stress tests because of an insufficient number of available chargers — now, it takes 4-5 hours to set up and test them. Complex test scenarios were also almost impossible to execute due to the excessive amount of time needed to review logs; with TimeTick, it takes only 3-4 hours from the setup to getting detailed and analyzed results. This approach reduces manual effort and allows the team to focus on more important tasks rather than repetitive, tedious work.

3. Saving resources

Saved time directly and indirectly helps save other resources. By automating QA workflows for standard and complex testing, our clients save $10k to $30k monthly:

• Development team: 100 to 480+ hours
• QA team: 80 to 100+ hours
• HWQA: 80+ hours
• Product team: 10 to 30+ hours

It becomes possible to reallocate resources to higher-priority tasks, create demo environments faster, and reduce the number of cases where customer support is needed.

All this results in the economy of remuneration and team maintenance, and at the same time, you ensure the scalability of the testing process. For example, you save resources on setting up the lab environment and make more scenarios feasible: it is possible to test 300+ rules on 20+ chargers, which would be impossible to perform manually in a lab.

4. Eliminating the human factor

A human factor can influence any step of the testing process, from inaccurate setup to missing the fact that incorrect data has arrived to difficulty finding each and every error in an abundance of correct responses to commands.

Also, the platform reduces the risk of the human factor during the development stage since there is no need for extensive scripting. The initial features already guarantee detailed testing of device functionality and quality before deployment.

5. Locating errors immediately

Along with running multiple tests simultaneously, TimeTick identifies and highlights errors automatically, so testers don’t have to spend much time analyzing logs. In addition, you get improved bug detection; TimeTick’s client found six major and 10+ minor previously undetectable bugs that would have taken 3-5 days of HWQA time to uncover using real chargers. Error detection runs in real-time, instantly identifying problems across devices. All this is still possible when you monitor the entire system, including both hardware and software components.

6. Configuring custom test cases

While the most common reason for using TimeTick is preparation for the OCPP certification, you can also run your own, customized tests. The process of setting up scenarios is intuitive, and all steps are even if it’s not a standard logic chain.

7. Identifying data anomalies

While checking compliance with certification requirements is the priority, TimeTick can pinpoint anomalies that may not be initially harmful but might cause problems in the future. For example, it can detect errors in the logic.

Example: data processing delay. Sometimes, an EV charger may function in a way that doesn’t violate any critical rules and requirements, although it is not working according to a protocol. For instance, it sends all the necessary data, but it takes a little longer than expected, with a delay of one or two seconds. This may not be a problem, but it may transform into one later or when combined with some other factors. Early awareness about such a delay allows for timely investigation and helps prevent issues during the upcoming updates.

To find some specific anomalies, your team would need to create complex scripts — TimeTick serves as such a script. It automates the comparison of the charger’s functions with a protocol, identifies anomalies and errors in logic, and creates clear reports.