What is Conversation Studio?

Conversation Studio is an Omilia Development Framework which allows building brand customized language applications for customer care, leading to highly engaging user experiences and human-like, natural conversational interactions.

From a Quality Assurance standpoint, it’s a cloud-based web application, built using modern UI libraries and web frameworks implemented following one of the most challenging and complex business logic a tester could ever imagine.

Automated Testing to the rescue!

Conversation Studio is under rapid development with new features added in every release. In order to guarantee the superior / top quality of new features, every release is integrated after heavy use of automation testing. After careful consideration and numerous debates, we decided that we need:

  • an automation framework to support our manual testing activities.
  • an easy to learn and use automation framework for UI, as well as end-to-end tests. This way, not only will we shorten the regression testing time, but we will also improve and better support all development testing activities.

After carefully examining and creatively testing most modern testing frameworks, we decided to move forward with Cypress.io.

Why use Cypress.io to test Conversation Studio?

We are resourced with top class Full Stack, Frontend and Backend Engineers with deep knowledge of Javascript and Python, people contributing a great QA mentality and willing to help our QA Automation engineers to test every aspect and level of Conversation Studio.

  • Cypress tests are written in JavaScript.
  • Cypress supports Storybook, an open source tool for developing UI components. Shifting left at its best.
  • Cypress makes the use of Test-Driven Development and Acceptance Test-Driven Development possible for end-to-end testing.

We strongly believe that Automation Testing Frameworks must pay, not cost. Cypress.io is an All-in-one testing framework, assertion library, with mocking and stubbing which relies on many best-of-breed open source testing libraries that provide stability and familiarity to the platform from the get-go.

  • Easy npm installation.
  • Mocha testing framework for tests structuring.
  • Chai assertions library.

Running, debugging test cases and analyzing test results has never been easier via Test Runner.

  • Intuitive UI interface.
  • Time Travel and Chrome Dev Tools support for test case creation and test results analysis.
  • Open Selector Playground with anti flaky locator strategy.
  • Test Execution Performance Metrics.
  • Fast test execution times.
  • Great documentation. We are spending our breaks reading about automation testing.

Creating end-to-end tests is a demanding task. Bringing the software that needs to be tested in the actual state needed so to test the specific scenarios or edge cases, might be one of the most difficult tasks for an automation engineer to conduct.

  • With Sinon spies, stubs and clocks is super easy to verify and control the backend responses.
  • Stubbing network traffic is a real option as well.

Cypress is a cool open source project with a scalable architecture.

  • BDD with Cucumber is something that should definitely impress your business users.
  • End-to-end tests with Visual Testing nowadays is something really common. Both open source and proprietary libraries are available to try and test.
  • Hooks for CI test reporting in slack is something we can’t live without.
  • Jira integration minimizes Jira housekeeping.
  • Sinon library for stubs and spies.
  • Lolex to fake native timers.

Cypress.io disadvantages

Nothing is perfect, not even Conversation Studio. Cypress.io has some cons and we were bound to find out before anyone else. We already informed Cypress.io engineering team on them and they committed to fixing them for us.

  • Cross-browser support (only Chrome family)
  • It works only for apps that have a DOM (not React native etc)
  • It cannot support multi-tabs because it works in the browser
  • It cannot support also multiple open browsers at the same time
  • Each test is limited to only visiting a single superdomain.

Conclusion

Quality Assurance is everybody’s responsibility. Developers and Testers are both QA Engineers, testing in different levels, but we are all working toward the same goal. The selection of an automation framework is a difficult decision based on multiple criteria like people, skills, testability and must be treated as a Software Development project.

As a rule of thumb, make sure that your automation framework supports the same programming language your development team is using for your codebase; that it supports different testing levels like unit, UI, end-to-end and visual testing; and finally that it has great documentation paired with an active community.

————————–

John Papantoniou has been with Omilia long enough to build a strong QA department and set up procedures, standards and specifications for both own stack of technology as well as deployments acro