About jweinerJeff is the VP of Product, Marketing and Sales at Mersoft.
Mersoft has been developing solutions to test network technology and web applications for over 15 years. Load testing any web application requires the ability to start up large quantities of virtual clients, run a series of scripted test scenarios, and collect the results. The challenge is compounded in WebRTC load testing with a live streaming media application where the quality of all the streams between the end points must be measured as well as client behaviors, server components, load balancers, STUN and TURN servers, etc.
Most testing apps (and there are plenty) will start up the prescribed number of load-generator clients and test app functions and the stream in the same way – from a user’s perspective. There are a few downsides to this approach:
- Full Windows+browser virtual clients are resource-intensive and take time to set up
- The complex environment is error-prone because the odds of changes to the client are high. They come from operating system patches, browser updates, and the web application being tested. It gets worse if the application exists in a complex enterprise environment where different groups support the authentication, middleware or other components used by the app. Teams spend time investigating failures and updating scripts instead of testing the product.
- Many users will access the app server via a native iOS or Android app, or a set-top box. Emulating a browser on Windows doesn’t serve any purpose in these circumstances.
When Mersoft decided to put its testing chops to work for WebRTC load testing, we took a different approach. First, we created a proprietary ‘headless webRTC’ client that runs extremely lean and fast. Plus, it can gather much more information about the WebRTC media. Here’s a screen shot of recent test session we did with our own Mersoft move™ and stream™ products.
In this case, it was a small test with only 20 clients and running 3 minutes each. Scale capacity is technically limitless, but regularly can handle thousands. It takes only a few seconds for each virtual client to get started and only require a small footprint for the OS and other overhead to execute the functions. The load tests run for as long as needed and, since they are easy to repeat, we can set it up to run in cycles while we intentionally break things to ensure each part reacts properly. When issues do arise, each client has detailed PCAP files, log files and a video recording.
Since Mersoft test™ is in the cloud, it replicates actual user experiences by accessing the apps from the public internet across whatever geography is needed. This ensures that the firewalls, load balancers and network connections are verified, not just the application and it’s servers. Mersoft test™ can also run in a customer’s data center, or even a combination.
Testing with a browser is supported too.
I just said how full browser clients are fat and slow, and error-prone, and offer less visibility to performance, but sometimes you still need them. For complete WebRTC load testing, it’s valuable to test user interface functions along with WebRTC functions. So, that’s why Mersoft test™ can create Microsoft Windows virtual machines and can run Chrome, Firefox or Internet Explorer/Edge. Use full Selenium scripting capability alongside the WebRTC scripting for a comprehensive testing approach. Soon, we will provide virtual Android and iOS devices as well.
In practice, we find that using a full browser client is helpful for other types of testing as well. This includes performance monitoring in production or test environment verification as part of a continuous integration process. So, it helps to think about the type of testing scenario and then decide the best approach from there.
For those with experience in load and performance testing in network services, web applications, e-commerce, traditional VoIP, or other areas may be veterans in their own domains but may be somewhat new to WebRTC. In these domains, best practices and tools of the trade are established. With WebRTC load and performance testing, there are some unique challenges and complexities to overcome and ensure the application is ready to handle anything that customers throw at it.
At Mersoft, we’ve been building and running testing, diagnostic, performance and analytics tools for over 15 years. So since we started working with WebRTC several years ago, the experience showed we needed some unique tools to make sure our customers’ services were solid. They include applications that leverage real time communications for certain functions, integrations of multiple vendor unified communications (UC) products, live streaming social media and other platforms. Mersoft has several of our own products that use WebRTC so we needed tool for our internal teams too.
When it came time to identify the WebRTC load and performance testing features most important to include in Mersoft test™, we listened carefully to our customers, drew from own experiences building WebRTC products as well as our years of testing and analytics work in other domains. Here’s what we found matters most:
- Test the entire platform: signaling (the network), application functions, media streams. Most teams test only one of these.
- Understand the topology. Check the call setup. Are the SDPs being exchanged successfully? Are there firewall issues?
- Keep track of all the ICE candidates and their various media streams in every direction. When some fail and others succeed, intermittent problems occur, and are tough to find. Cover paths through TURN servers and peer to peer. See how many TURN servers are used, if it uses a TURN-first policy and that affects startup or latency.
- Understand the nature of each client. If it’s a browser, consider brand, version, security, local settings. If it’s a native app (consider for example, if it’s node.js, is it “stomping” on its variables when under load?).
- Test both by using the client itself and by running “headless” for faster speed and richer analytics
- Gain insights. Don’t just get reams of log files. Instead, generate useful information that helps engineers find problems faster
- Draw conclusions about the performance of a group of tests rather than just looking at each test separately
- Include WebRTC testing as part of a continuous integration process. That way, the testing team can spend its time investigating problems and making the testing even better, rather than running tests
At Mersoft, we are passionate about helping our customers develop scalable and robust WebRTC products, whether we are working with global video surveillance companies, startup live streaming social media companies, enterprises managing complex unified communications platforms or others.
- An intuitive dashboard delivering insights as events happen.
- A notification engine configured for SMS, email, Slack and HipChat.
- Microservices to post events to Splunk or AppDynamics.
Want to talk to one of our engineers or see a demo? Email: email@example.com or call 913-871-6200.
New Case Study Published:
Executive Summary – WebRTC Analytics Video Chat Case Study
A major U.S. communications provider worked with Mersoft on load testing and production monitoring. The focus was their video chat services that use WebRTC. The services include 2-way video, audio, text messaging and screen sharing. It was a fairly new product, but gaining in popularity. They wanted to make sure it was ready for large-scale use.
Mersoft and the client collaborated to set up Mersoft test™ for production monitoring and several load tests to identify improvement areas. They quickly found and resolved several issues from node.js issues with the client, web server capacity issues, SDP (session description protocol) exchange issues, ICE candidate issues, and problems when a selective forwarding unit was utilized versus a peer-to-peer connection. They resolved each problem quickly, improving the customer experience.
The engineers used Mersoft test™ to emulate the client experience so as new versions are released, the scripts can be easily rerun. The client now has the capability to manage the unique complexities of a large-scale WebRTC product and can save both time and labor in addition to getting insights unavailable from other products. In addition, all the graphs in this case study were taken directly fromMersoft test™. More graphs are available to help teams visualize the data effectively.
Download it here.
KANSAS CITY – APRIL 3, 2017 – FOR IMMEDIATE RELEASE. Today, Mersoft announces the latest release of its popular and innovative Mersoft stream™. The engineering team of this Kansas City-based technology company has disrupted the market yet again with new features.
A new SFU or “Selective Forwarding Unit”: It creates a 1-to-many relationship for media streams that range from a few connections to thousands and does so without transcoding. As a result, Mersoft stream™ outperforms other technologies with its much lower latency and a very small processing footprint.
HLS integration with CDN integration: Customers can now create even more versatile products by selecting either a WebRTC or HLS stream for use in their client application. This option is especially useful when leveraging a content delivery network (CDN). Now, customers can leverage Mersoft to build apps that can broadcast not just to hundreds or thousands of clients, but now millions. With it’s stateless architecture, Mersoft servers can be horizontally scaled enabling both load distribution and geographic redundancy across the globe.
Mersoft stream™ still runs both as a SaaS solution managed by the excellent pro services team at Mersoft, or hosted in a client’s datacenter. With flexible pricing options and a talented system integration team, it’s never been easier for companies to integrate new live media.
Mersoft CEO Ron Sloop commented, “The Mersoft product team continues to impress. After months of work, this is one of the biggest releases of new features in quite some time. We’re very proud of the team and already getting significant attention especially from companies needing to scale with large volume.“
FOR IMMEDIATE RELEASE October 10, 2016 – Overland Park, KS – Today, Mersoft announces the production release of Mersoft stream™ with audio. Mersoft stream’s unique video protocol adapter technology already offered industry-leading live streaming video for converting RTSP to WebRTC with extremely low-latency. Now, it can process live streaming video and audio.
Together, live streaming video and audio add a new level of versatility for smart home, enterprise security implementations as well as connecting people in a variety of ways. Here are just a few of the many use cases for the new Mersoft stream™ with audio:
– Doorbell-camera integration: See and hear a visitor in real time and communicate with them without lag. If the alert comes when the homeowner is away, the live RTSP streaming to WebRTC connection will sound as if they are right inside.
– Remotely-monitored building security: smaller businesses or ones with multiple locations can have the benefit of 2-way live streaming video and audio communication without onsite staff. People trespassing after business hours? A verbal warning from security through a 2-way audio connection can disperse the group and police can be called if necessary.
– ANY live streaming platform: Platform-as-a-Service (PaaS) providers in live streaming, video conferencing or security can benefit from this new release of Mersoftstream™.
The biggest problem accessing live video over a network connection today is the significant delay in both starting the video and in maintaining an experience that is near-real-time. Instead, many applications will require up to 60 seconds to establish a stream only to deliver a poor experience with a slow frame rate, frequent delays and poor resolution. These problems make live video feed over networks nearly useless. The only alternative being expensive hard-wired or closed-circuit (CCTV) equipment.
Mersoft stream™ is ideal for use with Wi-Fi cameras commonly used in homes and businesses. It extracts the H.264 media frames directly from a RTSP stream and repackages them in WebRTC without transcoding. It buffers both the inbound and outbound streams to ensure the most consistent experience possible. Now, audio is supported with both OPUS and PCM.
In addition, class-leading stats for both audio and video are available to ensure your application is running at its best. Get the Mersoft connect™ SDK and quickly integrate Mersoft into your web, iOS, Android and even TV set-top box applications.
Demos are available by contacting Mersoft: firstname.lastname@example.org or +1-913-871-6210
For more information, visit: https://mersoft.com
Mersoft is a Kansas City based software engineering firm delivering technology products and solutions since 1996. They create enterprise-grade technology for service providers needing messaging, video, presence, and real-time connectivity.
Customers describe Mersoft as the “Navy SEALs” of software technology because they take on tough challenges and consistently deliver. The company culture revolves around excelling in research and development, its engineers are focused on three factors: technology, solutions and services.
MEDIA INQUIRIES: +1-913-871-6200 or email@example.com
August 16, 2016 – Overland Park, KS,
Mersoft announces its newest launch of Mersoft move™ for live streaming security cameras directly to a TV set-top box. Customers can view live cameras in seconds from the biggest and best screen in the house. Seeing who is at the front door when the doorbell rings or checking on the baby at nap time has never been easier. Mersoft move™ and Mersoft stream™ work together to provide the best live camera streaming available. The Mersoft stream™ protocol adapter converts RTSP to WebRTC with extremely low-latency. Clients can include web browsers, mobile (iOS and Android), TV set-top boxes and more. When a customer opens the app, the live stream will begin in seconds.
Jeff Weiner, VP of Product, Marketing and Sales at Mersoft said, “Many companies advertise live streaming but customers complain that it takes a long time to start a video and it’s often delayed. Mersoft solved this problem and delivers a truly real time video experience.”
Why it’s so fast:
Within milliseconds, Mersoft move™ sets up the WebRTC signaling to connect a camera to the client app. Immediately, Mersoft stream™ begins extracting H.264 media frames from the RTSP stream and repackages them in WebRTC without transcoding. It has buffers for both inbound and outbound streams to ensure a great user experience. Jeff Smith, Mersoft CTO said, “Our proprietary technology can extract the frames while retaining timestamps and order incredibly fast. We’ve also made it very efficient with CPU and memory.”
Truly live streaming has become a critical aspect of any security solution. In a recent study, 65% of respondents consider video monitoring among the most desired smart home feature and more than a third (37%) plan to buy a connected camera in the next 12 months.
Contact us today and let us help you integrate Mersoft into your smart home solution to get truly live RTSP to WebRTC streaming video.
For more information, visit: https://mersoft.com
Mersoft is a Kansas City based software engineering firm delivering technology products and solutions since 1996. They provide the world’s best companies in telecom, enterprise security, and smart cities with enhanced product offerings for messaging, video, presence, and real-time connectivity. Company culture revolves around excelling in research, innovation and delivering quality in every product.
MEDIA INQUIRIES: +1-913-871-6200 or firstname.lastname@example.org
Here’s the setup: A product or technology exec gets everyone energized about WebRTC. It may be a contact center video chat experience added to a web commerce site, a media sharing mobile app, or other use case. The executive is smart and driven and manages to push through a business case to get a new product launched. Great!
There was functional testing, and application load testing but WebRTC testing of media experience wasn’t covered. Why? Well, the QA team knows web app testing inside and out, but didn’t know how to really test WebRTC. Besides, the team thought, this is a new thing and we really don’t have a good way to predict customer adoption. So, let’s assume moderate volume and see how it goes.
To some, this may seem ad hoc, but in reality it’s pragmatic when the team uses agile development, a DevOps model and scalable cloud hardware. They are watching the analytics. The team can react to scaling issues if the product takes off and quickly add features based on real life use instead of trying to guess at the beginning (which really doesn’t work).
So, now the product is in and it’s getting some great traction. The trouble is, the WebRTC video experience is bad. There is excessive jitter, or the video start-up time is slow because it’s not efficiently connecting to the right TURN server. Maybe someone connected a camera with an unexpected format or a user upgraded to a new browser. Whatever the reason, the customers are not happy. The app has analytics to track user flow, the servers are watched for CPU and memory. Even the log files are scanned but it’s all after-the-fact. Problems with the WebRTC experience are detected indirectly and slowly, or maybe not until the customer complains. Customers, of course, are a bad service assurance tool.
The piece missing is effective WebRTC testing. WebRTC testing is unique and separate from web application testing, application performance management or network monitoring. That’s the hole Mersoft test™ is designed to fill.
Here are three ways it can be used:
- Proactive monitoring: Mersoft test™ can be configured to periodically create simulated clients that generate traffic and report issues. If the product is geo-diverse, the clients can be created regionally or even globally to ensure that traffic is passed through specific load balancers or data centers. Armed with PCAP files, log files and the ability to play back recorded video, the support team can find a root cause fast. Problem solving accelerates when they can easily compare a WebRTC testing alert from Mersoft test™ with application performance monitoring data from other tools and quickly find a firewall problem, load balancer issue or server capacity issue.
- Troubleshooting or Ad Hoc WebRTC Testing: The same scripts in the proactive monitoring or different scripts can be used to verify an application after a production event. It’s easy to see if a problem has been resolved or even to generate more troubleshooting details by running tests manually.
- Continuous Integration: Include Mersoft test™ in the continuous integration process to execute WebRTC testing along with any other verifications before a new software build is deployed to production (or any other environment for that matter).
We recommend creating scripts for load testing or environment testing prior to production launch. It’s helpful to do it this way so the tests behave as intended before they hit production. A blog on that topic is here. By using the configurable parameters in Mersoft test™, it’s easy to set up new environments to use with the same scripts. Consistent, efficient and proactive. Want to learn more? Let’s talk! Visit Mersoft.Com, email email@example.com or call +1-913-871-6200.