Can We Have it All! {Selenium for Web, Mobile & Everything a Product Needs}

The aim of this talk is to understand the most common but challenging pain points of Quality Assurance phase and learn how to be ready to tackle them.

Do you have your product on all major platforms (Desktop Web, Mobile Web, Mobile Native apps etc.) and automating tests for all of them is a big pain? From providing feedback to getting the fixes for issues caught to giving go ahead for release is another ball game? Looking for answers to any such questions or improvisation in your existing system? Come let’s talk…

Pain Points & how to approach them:

1. How to choose the right set of tech stack to support QA of Omnichannel supporting product.
2. How to approach building scalable solutions. I will be taking you through the real time problems from products like E-commerce/B2B huge impacting applications, and understand the dynamics of these problems with respective precautions & solutions.
3. How much time does it take to give test feedback & qualify a release to be production ready while having so many Automated Tests?
4. Are the existing reports detailed and clear enough for dev team to work upon & give fixes themselves without chasing them by explaining the issue verbally again?
5. Monitoring for Release Status in place? When in panic mode, how quick you or anyone in your organization can know which release tags to revert to in case of a bug in production and even a downtime of a few seconds is intolerable because it affects your customer badly (assume a bug in the payment gateway & you are the customer ;-) )

I will take you through the live demo of RSA, I’ve developed for similar needs, and discuss on how you can use it to meet your end-goals.

View the slides

Reliability Patterns for Large-scale Selenium Tests

When you run hundreds of thousands of automated tests a day for active monitoring and uptime measurement, flakiness is not an option! In this talk, we will study challenges of continuously maintaining a near-zero failure rate threshold in a massive execution cycle of UI (web and mobile) tests each day, geo-distributed over multiple data centers worldwide.

You will be introduced to various availability, fault-tolerance and resiliency patterns for Selenium-based infrastructures and tests. We will learn how to apply these patterns using open source libraries and cloud computing solutions. We will also cover practical approaches to test reliability measurements, service performance metrics and root-cause analysis in distributed large-scale testing.

Distributed Testing and Test Reporting

As we are moving into the agile world, continuous integration has a major role to play. So how do we cater for a complete test on every sprint or every release? We can use Selenium for Test Automation. When we use a continuous integration approach it would be helpful to use Selenium Grid. It allows you to run your tests on different machines against different browsers in parallel. Essentially, Selenium-Grid supports distributed test execution. This helps you to run your automated tests on various different machines, operating systems and browsers at the same time. This saves time and would help to run your testing in a nightly build.

Extent Reports will go hand in hand with Selenium Grid as it will help you retrieve all test results including Test Evidences into a comprehensive report. This talk is a quick guide on how to use Selenium. With details on how to create HTML reports (with latest plug-in) which will give an understanding of test execution results for both technical and non-technical people. The highlight of the talk will be on Selenium Grid which permits users to run Selenium test cases on various operating systems and browsers from a specific hub. We will cover quick demonstrations on main browsers used in the industry such as Firefox, Chrome and Internet explorer. There will also be a full demo of how to use with JUnit, TestNG and more.

The State of Browser Automation with JavaScript

This talk covers various techniques on browser automation. We’ll focus on JavaScript, and explore a number of strategies for controlling modern web applications. Automating and testing of playing audio, video, and even animation are some examples of things we’ll look at. With a large number of end-to-end testing frameworks out there, all using JavaScript, this talk will also evaluate the pros of cons of such frameworks.

Finally, we’ll explore some more creative uses of browser automation, going further than just browser based testing. For example, we’ll see how scraping our favourite web services can be accomplished through WebDriver!

Let's Talk Security - Beyond Scanning

In this talk, I’ll speak about Web Security and how to do Penetration testing in an efficient manner. We’ll examine:

1. Cross-site Scripting (XSS)
2. Cross-site Request Forgery (CSRF)
3. Content Security Policy (CSP)
4. Other interesting headers

I’ll also cover some of the interesting vulnerabilities which I have reported to cloud based services and tools which are being used by million of users. We’ll see demos showing vulnerability discovery and exploitation with an attacker perspective. At the end of talk, one should be able to make web applications free from most of the vulnerabilities out there, find vulnerabilities in real web applications, and make the web more secure.

Sharing the pain in Automating with Protractor & WebDriver

I want to take this opportunity to share with my experience - the good and the bad, in the journey of building a Test Automation framework for an AngularJS based application.

We will learn, by a case study, what thought process we applied on the given context (product, team, skills, capabilities, long term vision) to come up with an appropriate Test Automation Strategy. This Test Automation strategy covered all aspects of Test Automation - Unit, Integration, UI - i.e. End-2-End tests (E2E).

Next, I will share how we went about narrowing-down, and eventually selecting a specific Tech Stack + Tools (Javascript / Jasmine / Protractor / Selenium-WebDriver) to accomplish the Test Automation for the product.

Lastly, I will share the challenges that came up in the implementation of the Test Automation, and how we overcame them. This will also include how we managed to get the tests running in Jenkins - a Continuous Integration tool. This discussion is applicable to all team members who are working on Test Automation!

P.S. All code shown will be shared via a sample protractor-based automation framework available on github for anyone to use as a starting point for setting up protractor-based Test Automation framework.

Building a Test Engineering culture

Test Engineering culture emphasises on quality ownership by all involved in a project and redefines the role of testers as engineers as quality enablers instead of gate keepers. With this talk I will be sharing my experience to build a test engineering culture across the organization, how does it vary with the size of the team and the company and the role of test engineers in building this culture by enabling others with tools and processes for quality improvements. Many companies mainly startups tend to invest in testing only when their product is about to go live or sometimes even later. I will be sharing how these teams can make small investment in testing by establishing a test engineering culture can derive larger benefits when product goes live. Lastly how does the test engineering culture redefines the career path for test and QA engineers.

The Screenplay Pattern - a SOLID alternative to Page Objects

The Screenplay Pattern, first created by Antony Marcano, is an alternative model to PageObjects. Today, it is growing in popularity with increasing tool support in popular testing frameworks.

PageObjects provide an easy-to-follow, simple structure that avoids early maintenance issues. They were introduced to help test-developers avoid mistaking flaky tests for problems with Selenium. But, PageObjects break some key OO design rules, making maintenance more difficult over time. They are a useful first step, but why do we stop there?

In this session you’ll learn about the SOLID design principles that PageObjects disregard. You’ll see why this leads to problems. You’ll see how and why PageObjects benefit from refactoring to SOLID design principles. Finally, you’ll meet the Screenplay Pattern – an alternative model based on SOLID principles that saves you the trouble.

Continuously Testing Online, Interactive Curriculum at Code.org

Brian Jordan will give a tour of Code.org’s continuous, automated cross-browser and visual testing suite—the challenges of building for the classroom, the technologies used to test Code Studio and the Hour of Code, and the team-wide lessons learned (and bugs found) along the way.

Code.org is a non-profit dedicated to expanding access to computer science, and increasing participation by women and underrepresented students of color. Their vision is that every student in every school should have the opportunity to learn computer science. Code Studio—Code.org’s free, online computer science curriculum platform—has over 200,000 teachers have signed up to teach intro courses, and over 5,500,000 enrolled students. Students play through curricula of increasingly challenging puzzles, featuring kid-favorite characters from Minecraft, Angry Birds and Star Wars.

Turning iOS End-to-End Testing up to 11

Automated End-to-End testing of iOS Apps is slow. Diagnosing test failures can be a nightmare to debug. But it doesn’t have to be this way!

In this talk you’ll discover how Facebook London’s End-to-End Automation team has vastly improved the developer experience of iOS Engineers by investing in tooling and infrastructure. You’ll see first-hand how to leverage Facebook’s Open Source projects to improve your test automation. Using ‘fbsimctl’, we’ll show you how to collect diagnostic logs, record videos of test runs and fetch all of this valuable data over HTTP. You’ll see how you can iterate on writing tests faster with WebDriverAgent’s Inspector. Finally, we’ll show you how to get the most out of your Macs by running many tests in parallel, decreasing test run times and increasing test throughput.

Big Data Makes the Flake Go Away

Flakey. Non-deterministic. Unreliable. These are all terms commonly associated with end-to-end testing. At WalmartLabs, we knew that building a Continuous Delivery pipeline depends on a test automation system that’s totally reliable. Any perceived flakyness in the system would erode developer confidence in our tools.

But when non-deterministic behavior emerges, how do we crack open the black box and identify the source?

In this talk, we show how we instrumented our test framework with our own internal analytics beacons, to allow us to “zoom out” and spot trends in test performance and reliability that are not always visible at the individual test suite level. Using open source data visualization tools allows us to slice and dice the data to identify root causes. We’ll share how these data and trends provided both immediate insights, as well as raised new mysterious questions that we hope to work with the community to answer.

Selenium tests, the Object Oriented way

When you are writing Selenium tests to check for elements on your page, by taking the classic approach (checking for each element’s properties at a time), you might get to a large number of assert steps. This increases the lines of code your tests have, makes the tests difficult to maintain and tricky to read.

Wouldn’t it be nice if the actual checking part of the test would be small, perhaps one line of code? With the approach I am going to present, you can do just that. Hence your tests will be small and clean. All you will need to do is model the pages/modules/items by using an Object Oriented approach.

Is it or is it not really visible… Selenium’s flawed vision

When WebDriver was born it always had the belief that it would emulate users. Users, unfortunately for WebDriver, have the ability to see when an element is visible.

When commands like isDisplayed() were originally created a lot of the magic of modern CSS did not exist. Animations were done with JavaScript and people tried not to reflow or repaint the canvas as it was slow. This simplicity meant that `isDisplayed() algorithm in Selenium worked almost all of the time.

Browsers started getting more sophisticated and as such are able to conjure up new ways to display elements, ways that seem just like magic to the end user of the application.

This talk will explain the cases where Selenium can’t handle isDisplayed() properly and probably never will.

StarDriver Enterprise: Appium to the Future

Last year, Hugs outlined a vision of WebDriver’s future—one protocol for automating everything, which he dubbed “StarDriver” (or “*Driver”). Appium had already taken the first steps of that vision, by extending the JSON Wire Protocol to mobile applications. Appium’s success speaks to the power of the StarDriver vision. What’s next? Like 80s sci-fi movie sequels, more Appium, of course!

In this talk I’ll outline how Appium has added four new drivers in the last year, from new iOS and Android drivers to Windows Desktop app automation. I’ll highlight some important features and getting started notes. Finally, I’ll discuss Appium’s vision: making the best automation technology available through one protocol for every tester (from individual hackers to Java-slinging enterprise SETs), and how we plan to expand into new areas, such as IoT. At this point, you might be wondering: is Appium really StarDriver? To find out, stay tuned for SeConf 2016!

Moving from Manual to Automated Testing at a Small Company

Moving from manual to automated testing at a small company takes curiosity, research, careful planning and the ability to evolve as you learn. This talk will focus on how to get started, cultural hurdles, technology options, implementation and evolution. Additionally, virtualization and test case management will be covered. The goal for this topic is to show how to go from manual test case execution to an automated framework that can be built upon.

Refactoring for Testability: Breaking the Cycle of Doom

There are a lot of examples out there for how to introduce good testing and architectural patterns when you are starting a new project. In reality, even on newer projects, you are more often working in a less than ideal state. Schedules are tight, customers want more features, but at some point your productivity will fall apart if you do not make good architecture and testing a priority. How do you get out of this state? This talk will present common issues and agile, realistic solutions within the context of real world examples.

Test trend analysis: Towards robust, reliable and timely tests

In this talk you will get ideas about how you can instrument test result information to provide actionable data, paving the way for more robust, reliable and timely test results.

By capturing this information over time, and when combined with visualization tools, we can answer different questions than with existing solutions (Allure / CI tool build history). Some examples of these are:

• Which tests are consistently flaky
• What are the common causes of failure across tests
• Which tests consistently take a long time to run

Using this information we can move away from the ‘re-run’ culture and better support continuous integration goals of having quick, reliable, deterministic tests

Zen and the Art of Open Source Maintenance

Selenium is a large, popular, and active Open Source project. It’s also over 10 years old, yet still evolving. The release of Selenium 3 provides the perfect point to think about the lessons we’ve learned since the start of the project. What defines “good software”, and what defines “Quality”? And how do we apply those lessons as we move towards Selenium 4 and the W3C spec?

Robots: The Nuclear Option

An exploration of various use cases that have come up over the years from customers where using a real physical robot is the only viable way to automate a test.

Automating the Responsive Website Testing

Responsive web design (RWD) is an approach to web design aimed at crafting sites to provide an optimal viewing and interaction experience, like easy reading and navigation with a minimum of resizing, panning, and scrolling across a wide range of devices (from desktop computer monitors to mobile phones).

Responsive web design is becoming more important as the amount of mobile traffic now accounts for more than half of total internet traffic. This trend is so prevalent that Google has begun to boost the ratings of sites that are mobile friendly if the search was made from a mobile device. This has the net effect of penalizing sites that are not mobile friendly.

The responsive web design responds to the needs of the users and the devices they’re using. The layout changes based on the size and capabilities of the device and provides the enhanced user-experience by re-structuring the contents as per the end-user devices. With plethora devices releasing every day, this has gained significance in the web designing and along with it came the testing challenges. In this talk, we are going to discuss the challenges in testing RWD websites and how to overcome those by using the tools available online.

I have chosen Galen tool to demonstrate the automation of Responsive web design testing. Galen is an open source framework built for responsive websites. It provides the feasibility to test the various pages on screen sizes and browsers. The test and spec files can be written in plain English which makes it easier for the business people to understand and contribute. This can also be used as a guide for the UI developers where they build the layout based on the spec files agreed by the business. Come and experience the talk to be enlightened about this new found tool!

View the slides

Advanced Appium

Over the last few years, Appium has become the choice automation tool for mobile application UI testing. Most people are familiar with the basics of Appium, but did you know that you Appium can identify elements using image recognition? Did you know you it’s also possible to automate Windows phone and Desktop apps with Appium? Have you ever seen Appium run the same test on multiple operating systems, or seen an Appium test run using several devices at once?

The talk will cover advanced Appium topics such as these along with best practices to ensure you get the most out of Appium.

View the slides

How to Get Automation Included in Your Definition of Done

While most teams appreciate the benefits of UI automation, it is commonly viewed as too time-consuming to be considered as part of an agile sprint; resulting in automation being done in isolation and typically months after the story has been closed. This can lead to several problems including automation team members being disengaged and missing key aspects of the requirements, as well as teams going through a period where new features are being introduced but no regression testing is occurring.

This talk provides agile-friendly approaches to automation using Selenium which will allow teams to close their sprints with automation in place. These techniques allow scrum teams to work smarter, not harder, and find bugs quicker with a more narrowed scope of the root cause, essentially leading to quicker resolution times. We’ll also walk through an example Story and demonstrate how to apply these techniques to ensure in-sprint automation is achievable.

View the slides

Think inside Container : Building Reliable and Scalable infrastructure for Selenium Tests

Testers are responsible for the reliability of their tests in the same way developers are responsible for their software but this is usually difficult, time consuming and needs collaboration with developers. Docker capabilities can help us in ensure and enhance reliability, speed and scalability of our tests.

This talk will demonstrate how to leverage the container technology to solve the challenges of scaling a reliable testing infrastructure and continuous delivery for selenium and other testing tools, with key focus on:

1. Basics of the containers technology and specifically its application on the test automation.
2. How Docker can reduce the time of test execution, ease the setup of clean test environments and drastically reduce the differences between the development, acceptance and production environments leading to the higher quality of the released software.
3. Examples to containerize entire testing stack together consisting of major automation tools like selenium.
4. Integrating and managing testing container with other application containers to achieve easily manageable continuous delivery pipeline.
5. Best practices and patterns for docker success.

View the slides

Advanced test automation techniques for responsive apps and sites

Responsive web design has become the preferred approach for building sites and apps that provide an optimal viewing and interaction experience on any phone, tablet, desktop or wearable device. However, automatically testing these responsive sites and apps can be quite a challenge, due to the need to cover all supported layouts, their respective navigation, and visible content.

In this session we will implement a complete Selenium-based automated test for a popular responsive website from scratch. You will learn how to effectively design responsive page objects, implement generic tests that work for all the layouts of your app, control the browser’s viewport size in order to accurately target layout transition points, incorporate layout-specific assertions in your tests, and visually validate the correctness of your app’s layout​. We will also share tips and best practices for test planning and execution.

View the slides

A Programmer's Guide to Humans

What makes software development complex isn’t the code, it’s the humans. The most effective way to improve our capabilities in software development is to better understand ourselves. In this talk, I’ll introduce a conceptual model for human interaction, identity, culture, communication, relationships, and learning based on the foundational model of Idea Flow. If you were to write a simulator to describe the interaction of humans, this talk would describe the architecture.

Learn how to understand the humans on your team and fix the bugs in communication, by thinking about your teammates like code! I’m not a scientist or a psychologist. These ideas are based on a combination of personal experience, reading lots of cognitive science books, and a couple years of running experiments on developers. As I struggled through the challenges of getting a software concept from my head to another developer’s head (interpersonal Idea Flow), I learned a whole lot about human interaction.

As software developers, we have to work together, think together, and solve problems together to do our jobs. Code? We get it. Humans? WTF?!

Fortunately, humans are predictably irrational, predictably emotional, and predictably judgmental creatures. Of course those pesky humans will always do a few unexpected things, but once we know the algorithm for peace and harmony among humans, we can start debugging the communication problems on our team.

View the slides

Making Cross-Browser Tests Beautiful

How many times have your selenium test suites run beautifully on one browser, only to fail when run in any other browser? This is an extremely common problem faced when incorporating cross-browser tests into your test runs. Not all browser drivers are created equally, but that doesn’t mean you can’t create a robust suite with cross-browser tests.

During this talk, Meaghan will present strategies for making cross-browser tests invincible. We will discuss key topics such as using as choosing the best locators for all browsers, explicit vs. implicit waits and how to leverage cloud based testing tools.

Participants can expect to come away from the session understanding the importance of cross-browser testing, why browsers behave differently, and a toolkit of how to successfully develop and maintain cross-browser tests in the future.

View the slides

Measuring Web Performance Using Selenium

You are the Selenium master and automate tests in your sleep. You make sure the quality of your companies web page is 100%. But then the team wants you to measure the performance of your site. How do you do that? Is that even possible with Selenium?

In this talk we will look at how we use Selenium in Browsertime & sitespeed.io to measure the speed of your web application and how you can implement it yourself in your own Selenium tests. We will also briefly look at strategies to make sure you can catch regressions in your sites performance.

View the slides

Statistical Element Locator

In this talk we will delve into one the biggest challenges that Test Automation developers face, finding elements i.e. a robust test means finding the same element with high fidelity while the Application Under Test keeps changing.

We will categorise the methods, and show where developer fail, where machines (Record/Playback) fail, and suggest a new way for locating elements, and analyze the skill-set required to overcome those difficulties.

View the slides