expoQA

• International Software Testing & Quality Engineering conference
• 4 tracks of professional presentations and an additional master class track
• Great debate
• http://www.expoqa.com

Julian Harty. Does software testing need to be this way?

• Relation of testing with medical industry
  • Smart testing = quick checks, evidence based, discourage harmful or excessive tests
• Too much data?
• Minimize use of new tools, check beforehand they are useful
• Machines vs tester: The Glass Cage book - Nicholas Carr http://www.nicholascarr.com/?page_id=18
  • A pilot loses hability to manually fly a plane on average 7 weeks, how long takes to a tester using automation to lose skills?

Julian Harty. Does software testing need to be this way?

• Monkey testing. “Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.” http://principlesofchaos.org/
  • Chaos monkey, failure as a service https://github.com/netflix/chaosmonkey
  • Failure injection testing https://www.infoq.com/news/2015/06/netflix-failure-service
• Data mining: 90% of bugs caused by top 10 coding errors
  • Introducing faults to find defects
• Guidance
  • Testylytics project
  • Testing heuristics https://github.com/julianharty/testing-heuristics

Julian Harty. Does software testing need to be this way?

• Feedback systems. Using analytics
  • 99% of users don’t do reviews but 2 star ratings are still a problem
• Guiding testing using data and analytics
  • Testing based on information (locales, user flows, activities…)
  • Testing based on insights (crossing tresholds, anomalies…)
  • The mobile analytics playbook https://saas.hpe.com/en-us/asset/mobile-testing/mobile-analytics-playbook-practical-guide-better-mobile-testing
• People can add discernment
  • Automation complacency & automation bias, humans are biaised to trust automation (Madrid-Barcelona flight through Morocco)
  • Where humans can focus. Dirtycow.ninja https://dirtycow.ninja/
  • Smart objects

Toni Robres. One testing tool to rule them all

One tool (yes, there is a typo)

Toni Robres. One testing tool to rule them all

• Confrontation within dev/test teams because of tools
• Characteristics of the perfect testing tool:
  • Reduce, reuse, recycle
• Unit testing:
  • NOSE. Extended framework for python unit testing
• Api REST testing:
  • Requests. HTTP for humans
• Web testing:
  • Selenium
  • Page object pattern

Toni Robres. One testing tool to rule them all

• Performance testing;
  • MultiMechanize. Runs concurrent Python scripts to generate load against service
• Test definition:
  • BDD -> Behave
• In Summary:
  • Easy to integrate, re usability, collaborative, great community…

Patxi Gortazar. Experiences using Docker in a complex CI environment

• ElasTest open source project http://elastest.io/ “An elastic platform to ease end-to-end testing”
• Run browsers within a container, debug images available for chrome and firefox allow recording to review issues
• Testing with multiple containers:
  • Docker compose, run or spotify-gc (to clean containers and images)
• Challenges:
  • Gathering data
  • integrating data with test execution
  • reproducing real world problems

Mike Jarred.The continuous evolution of testing in the FCA

• Negative stakeholders perceptions are damaging
  • Erodes trust and respect in stakeholders creating additional work (reviews, estimates…)
  • Credibility is key in the relationship with stakeholders

stakeholder languages

Mike Jarred.The continuous evolution of testing in the FCA

• Value for money:
  • Economy, efficiency and effectiveness:
    • Good value for money is the optimal use of resources to achieve the intended outcomes.
    • ‘Optimal’ means ‘the most desirable possible given expressed or implied restrictions or constraints’. Value for money is not about achieving the lowest initial price

Mike Jarred.The continuous evolution of testing in the FCA

• From test governance to assurance:
  • The funnel of fun
  • Relationship with suppliers (penalty driven is not ideal)
  • Test services need to be relevant, targeted, optimal and commensurate with risk appetite
  

Colm Fox. How to measure quality in today’s dynamic projects

• Technical debt is similar to financial debt, requires the payment of interest
  • Types are unintentional (Jr devs) and intentional (deadlines)
  

• Business vs technical vison on technical debt
  • Is for the testers to enable the business to get the full picture about technical debt
  • Technical debt is an indicator and can be either positive or negative

Colm Fox. How to measure quality in today’s dynamic projects

• Software metrics help predict defects in the code, and can be used to measure and increase code quality
  • Cyclonatic complexity
  • Halstead complexity metrics
  • Complexity
  • Hotspots
  • Modularity

Complexity

Eduardo Riol. Testing Tools in the Age of DevOps and Agile

• Story of waterfall testing, v model… early days of testing when testers where considered a stopping point
• Legacy testing software was not built for agile (hard to use, lack of integrations, not cross platform…)
• We need CI integrations, agile workflows, fostering exploratory testing, assistance on test automation

Testing in agile

Eduardo Riol. Testing Tools in the Age of DevOps and Agile

• Tools:
  • Tarantula: improves usability and managing and reporting options
  • Zephyr: bad support for bdd,tdd and ET
  • QTest: testing suite targeted to agile, supports transition from waterfall

THE GREAT DEBATE : Industry vs Experts

• A debate mediated by David Evans, where selected people from the industry and professional sectors, along with the audience, form opposing sides to give their opinions on a series of statements that affect today’s Software Testing and Quality Assurance

Debate Statement 1

THE GREAT DEBATE : Industry vs Experts

Debate Statement 2

• The transfer problem: students do well in previously practiced exams but struggle in similar questions

THE GREAT DEBATE : Industry vs Experts

Debate Statement 3

THE GREAT DEBATE : Industry vs Experts

Debate Statement 4

THE GREAT DEBATE : Industry vs Experts

Debate Statement 5

• Testing does not change the product but we get an output of valuable information

Isabel Evans. Leading, following, managing? Helping your group thrive

• We are animals, often in teams
  • Orcas vs wolfs (big and small organisations)
  • Chimpanzees opportunistic hunters
  • Parasites and simbionts (some parasites are quite attractive but still make their team work harder to cover for them)
• Leadership quadrants:
  • Muddlers are not good managers since they work best alone

Quadrants

Isabel Evans. Leading, following, managing? Helping your group thrive

• Follower types:
  • Effective star followership and leadership:
    • management
    • commitment
    • competence
    • courage

Follower types

Isabel Evans. Leading, following, managing? Helping your group thrive

• Leadership styles
  • Controlling leader (high authority and low autonomy)
  • Guiding leader (medium authority and autonomy)
  • Collaborating leader (low authority and high autonomy)
• Changing styles through coaching:
  • Listen to the alienated and the engage
  • Direct guide and support the sheep’s…
• In order to thrive:
  • Remember we are animals
  • We interact in different ways
  • We are animals of habit and territory
  • We can reason and learn, do not tolerate parasites

Maxi Mannise. Five reasons not to test software!

• Get the project at any cost (wolksvagen) reducing budget for testing which leads to reason 2

Reason 1

Maxi Mannise. Five reasons not to test software!

• Users will test, expectations need to be managed, hurries lead to reason 3

Reason 2

Maxi Mannise. Five reasons not to test software!

• Superheroes, you can’t always have an A team that will solve all the issues. Respect is key. Deliver with bugs so we can go to reason 4

Reason 3

Maxi Mannise. Five reasons not to test software!

• Charge for maintenance later, but still is too expensive to test…

Reason 4

Maxi Mannise. Five reasons not to test software!

• Save the costs, the first bug in prod proves the value of good testing

Reason 5

Leo Diaz. Configuration as Code: The missing link between Development and QA?

• Configuration as code
• CasC bitbucket pipelines using yaml
• Delegate the config to the code tools used by the dev team
• Advantages:
  • Change control
  • traceability
  • standarise
  • fosters collaboration
  • branches are easier to manage

Configuration as code

Cristiano Cunha. Climbing the testing pyramid

• Continuous delivery requires: Everything is in source control, retrocompatibility, independence of applications, build deployed automatically, fast and reliable automated tests, easy ways to understand what went wrong (reports, logs, monitorization), reliable and stable environments:

  – Closest to live as possible
  – Closed, nobody can do manual changes or access these environments
  – Stable, possible to revert changes in a heartbeat

• Shift left of testers

Testing pyramid

Fran O’Hara. The seven deadly sins of SCRUM

• Culture people and mindset: the biggest sin is not having enough focus on those
  • Align all stakeholders (incrementally), team culture, test competency and roles.
• The sin of push vs pull
  • Only the dev team estimate and decide on scope to pull into the sprint. Push leads to technical debt and less transparency
• Planning sins
  • Daily scrum is an opportunity for the team to plan their work not a status reporting session
  • Not involving testers in sprint planing
  • Release planning vs empirical forecasting
  • No big picture

Fran O’Hara. The seven deadly sins of SCRUM

• Product backlog refinement sins
  • Not having valuable items ready for the next sprint
  • INVEST criteria not met (untestable, stories too large…)
  • DEEP (detailed estimated emergent prioritised)
• Definition of done
  • Not having it (or not following it)
  • DOD needs to evolve , retrospectives should review it
• Sprint backlog sins
  • Not having explicit, estimated test tasks
• Scrum master, all of our sins…
  • Not focused on the Agile&Scrum values
  • Manager versus servant leader
  • Part-time
  • Not harnessing the power of retrospectives with the team
  • Team focus only and not an organisational change agent

Ard Kramer. The “magic” of test automation

• Productivity paradox https://en.wikipedia.org/wiki/Productivity_paradox
• Does automation add value?
• Does test automation save money?
• Does test automation recognize risks?
• Does test automation mitigate risks?
• Can test automation replace testers?
• Is test automation agile?

That’s all for now, questions?

Thanks