WebRTC Load Testing with Mersoft test™

Mersoft has been developing solutions to test network technology and web applications for over 15 years.  Load testing any web application requires the ability to start up large quantities of virtual clients, run a series of scripted test scenarios, and collect the results.  The challenge is compounded in WebRTC load testing with a live streaming media application where the quality of all the streams between the end points must be measured as well as client behaviors, server components, load balancers, STUN and TURN servers, etc.

Most testing apps (and there are plenty) will start up the prescribed number of load-generator clients and test app functions and the stream in the same way – from a user’s perspective.  There are a few downsides to this approach:

  • Full Windows+browser virtual clients are resource-intensive and take time to set up
  • The complex environment is error-prone because the odds of changes to the client are high.  They come from operating system patches, browser updates, and the web application being tested.  It gets worse if the application exists in a complex enterprise environment where different groups support the authentication, middleware or other components used by the app.  Teams spend time investigating failures and updating scripts instead of testing the product.
  • Many users will access the app server via a native iOS or Android app, or a set-top box.  Emulating a browser on Windows doesn’t serve any purpose in these circumstances.

When Mersoft decided to put its testing chops to work for WebRTC load testing, we took a different approach.  First, we created a proprietary ‘headless webRTC’ client that runs extremely lean and fast.  Plus, it can gather much more information about the WebRTC media.  Here’s a screen shot of recent test session we did with our own Mersoft move™ and stream™ products.

WebRTC Load Test or WebRTC Performance Monitoring

In this case, it was a small test with only 20 clients and running 3 minutes each.   Scale capacity is technically limitless, but regularly can handle thousands. It takes only a few seconds for each virtual client to get started and only require a small footprint for the OS and other overhead to execute the functions.  The load tests run for as long as needed and, since they are easy to repeat, we can set it up to run in cycles while we intentionally break things to ensure each part reacts properly.   When issues do arise, each client has detailed PCAP files, log files and a video recording.

WebRTC Load Testing and WebRTC Performance Monitoring

Since Mersoft test is in the cloud, it replicates actual user experiences by accessing the apps from the public internet across whatever geography is needed.  This ensures that the firewalls, load balancers and network connections are verified, not just the application and it’s servers.  Mersoft test can also run in a customer’s data center, or even a combination.

Testing with a browser is supported too.

I just said how full browser clients are fat and slow, and error-prone, and offer less visibility to performance, but sometimes you still need them.  For complete WebRTC load testing, it’s valuable to test user interface functions along with WebRTC functions.  So, that’s why Mersoft test™ can create Microsoft Windows virtual machines and can run Chrome, Firefox or Internet Explorer/Edge.  Use full Selenium scripting capability alongside the WebRTC scripting for a comprehensive testing approach. Soon, we will provide virtual Android and iOS devices as well.

In practice, we find that using a full browser client is helpful for other types of testing as well. This includes performance monitoring in production or test environment verification as part of a continuous integration process.   So, it helps to think about the type of testing scenario and then decide the best approach from there.

Check out Mersoft test and schedule a demo or webinar at https://mersoft.com

WebRTC Load Testing and WebRTC Performance Monitoring

WebRTC Load Testing with Mersoft test™2018-01-09T16:47:45-06:00

WebRTC Testing: Proactive Monitoring in Production with Mersoft test™

Here’s the setup: A product or technology exec gets everyone energized about WebRTC.  It may be a contact center video chat experience added to a web commerce site, a media sharing mobile app, or other use case.  The executive is smart and driven and manages to push through a business case to get a new product launched. Great!

There was functional testing, and application load testing but WebRTC testing of media experience wasn’t covered. Why? Well, the QA team knows web app testing inside and out, but didn’t know how to really test WebRTC. Besides, the team thought, this is a new thing and we really don’t have a good way to predict customer adoption.  So, let’s assume moderate volume and see how it goes.

To some, this may seem ad hoc, but in reality it’s pragmatic when the team uses agile development, a DevOps model and scalable cloud hardware. They are watching the analytics. The team can react to scaling issues if the product takes off and quickly add features based on real life use instead of trying to guess at the beginning (which really doesn’t work).

So, now the product is in and it’s getting some great traction. The trouble is, the WebRTC video experience is bad.  There is excessive jitter, or the video start-up time is slow because it’s not efficiently connecting to the right TURN server.  Maybe someone connected a camera with an unexpected format or a user upgraded to a new browser.  Whatever the reason, the customers are not happy.  The app has analytics to track user flow, the servers are watched for CPU and memory. Even the log files are scanned but it’s all after-the-fact. Problems with the WebRTC experience are detected indirectly and slowly, or maybe not until the customer complains. Customers, of course, are a bad service assurance tool.

The piece missing is effective WebRTC testing.  WebRTC testing is unique and separate from web application testing, application performance management or network monitoring.  That’s the hole Mersoft test™ is designed to fill.

Here are three ways it can be used:

  1. Proactive monitoring: Mersoft test™ can be configured to periodically create simulated clients that generate traffic and report issues.  If the product is geo-diverse, the clients can be created regionally or even globally to ensure that traffic is passed through specific load balancers or data centers.  Armed with PCAP files, log files and the ability to play back recorded video, the support team can find a root cause fast.  Problem solving accelerates when they can easily compare a WebRTC testing alert from Mersoft test™ with application performance monitoring data from other tools and quickly find a firewall problem, load balancer issue or server capacity issue.
  2. Troubleshooting or Ad Hoc WebRTC Testing: The same scripts in the proactive monitoring or different scripts can be used to verify an application after a production event. It’s easy to see if a problem has been resolved or even to generate more troubleshooting details by running tests manually.
  3. Continuous Integration: Include Mersoft test™ in the continuous integration process to execute WebRTC testing along with any other verifications before a new software build is deployed to production (or any other environment for that matter).

We recommend creating scripts for load testing or environment testing prior to production launch. It’s helpful to do it this way so the tests behave as intended before they hit production.  A blog on that topic is here. By using the configurable parameters in Mersoft test™, it’s easy to set up new environments to use with the same scripts. Consistent, efficient and proactive. Want to learn more? Let’s talk! Visit Mersoft.Com, email [email protected] or call +1-913-871-6200.

WebRTC Testing: Proactive Monitoring in Production with Mersoft test™2018-01-09T16:47:47-06:00
Go to Top