niedziela, 15 czerwca 2014

Test Metrics, CzechTest 2014 Presentation, 27 June

Testing is all about measuring: measuring how good an IT product is, triangulating where buggy areas are, estimating residual risk and helping others estimate how much is left to do. To some extent, testing measures itself, too: how much testing work has already been done, how much still remains. And, needless to say, whether you want it or not, testing measures project status and quality, and - indirectly - process quality. It is a lot to cover in 40 minutes.

  • Software measurement as part of general discipline of scientific measurement
  • Measurement for decision making
  • Testing as part of software measurement
  • Using testing to measure various quality attributes
    • Measuring functionality
    • Measuring non-functional attributes
    • Test evaluation: interpreting measurement results
  • Scales of measure in testing
    • Nominal scale
    • Ordinal scale
    • Interval scale
    • Ratio scale
  • Statistics and probability in testing
  • Accuracy and precision in testing
  • How testing contributes to project status measurement
  • How testing can contribute to process measurement
  • How to measure test effectiveness and efficiency
  • Measuring test process

Exploratory Testing and SBTM, CzechTest 2014 Tutorial 25 June

Exploratory testing is a very effective and efficient approach to testing software, and other systems, too. Sometimes - very rarely - it should fully replace other approaches. Always, it effectively supplements other approaches.
To an alarming extent, the exploratory approach has become more an ideology than a test approach. This sad fact has nothing to do with the approach itself, but it is a purely social phenomenon. fanned by hype and false propaganda. This has some benefits, since it has helped to diffuse good exploratory ways, but it has serious drawbacks as well, since it may lead to worse testing by - wrongly - discarding most non-exploratory methods.
In my tutorial, I will concentrate on how to use the best parts of exploratory approach, but at the same time to understand its costs and limitations.
  • An experiment: let's test it!
  • The story: four testing schools, context-driven school, exploratory, rapid testing
  • The anarchistic revolution: have agile, lean, exploratory and XP anything in common?
  • Is exploratory testing really context-driven? Only to some extent
  • We all do some exploratory testing even without knowing it
  • What does exploratory really mean?
    1. Context-driven adaptation of test approach to real needs (HTSM)
    2. Using non-standard requirements sources and inspirations
    3. Creatively designing test cases while executing test cases
    4. Simultaneous learning, test execution and test design
    5. No advance test case specification
  • Isn't exploratory testing really exploratory requirements elicitation?
  • How can exploratory principles be used in non-exploratory context? Examples
    1. Context-driven adaptation of test approach to real needs (HTSM)
    2. Using non-standard requirements sources and inspirations
    3. Creatively designing test cases while executing test cases
    4. Simultaneous learning, test execution and test design
    5. No advance test case specification
  • Comparison between exploratory and non-exploratory: relative benefits and costs
  • Finding three right balances: how much QA, how much testing, how much exploration
  • Freestyle versus managed exploratory testing
  • SBTM: how is it done
    • Session planning
    • Session execution
    • Session reporting
    • The full picture: session of sessions

Agile Testing CzechTest 2014 Tutorial 25 June

Agile testing is in principle no different from testing in any other model or framework, but in practice it often differs dramatically, because agile creates a very different environment for quality assurance, including testing.

This tutorial is about how to implement, deploy and use best testing practices - which are the same regardless of development model - in agile framework. Agile is not a miracle medicine, nor a sliver bullet, and things can go seriously wrong in agile projects as easily as they can in sequential projects, unless you know how to fill agile framework with good contents, including good testing.

During those 3 hours 30 minutes we have at our disposal, I cannot teach both agile and testlore; frankly, this time limit in definitely not sufficient for any of them. Instead, my intention is to concentrate on the opportunities agile creates that allow really good testing to be achieved, and how to avoid certain traps which lurk for testers in agile development.

We will talk less about general risks of testing wrongly and badly, because they are very much the same in iterative  and in sequential frameworks.

  • The benefits and the costs of iterative development, and how they influence testing
  • Toppling some popular myths about agile testing to create the room for real stuff
  • Test analysis in agile - who does it, and when?
  • The role of testers in agile scrum, and the co-operation of three amigos
  • Planning testing in agile projects:
    • risk poker and test effort estimation;
    • negotiating DoD
  • Test design in agile: the analysis of test quadrants
    • who does what part of test design?
    • what's the role of exploratory testing in agile?
    • the greatest agile achievement - acceptance criteria as acceptance test cases (what does it achieve?)
  • Regression testing in agile
  • Test automation in agile
  • ATDD: a powerful concoction of (1) complementing requirements with test cases; (2) test-driven design; (3) keyword-driven test automation
  • Test organization and test levels in agile:
    • test environment responsibility
    • test automation responsibility
    • integration acceptance testing
    • incident reports and bug fixing within sprints and outside sprints
  • Definition of Done as test completion criteria
  • Agile test process improvement during sprint retrospectives

środa, 11 czerwca 2014

Requirements on a shoestring

How to cope in industry projects with poor requirements engineering awareness?
In quite many projects and companies, there is too little or even ostensibly no requirements engineering performed. That is, of course, bad, but usually not as bad as it looks. If there was really no requirements engineering at all, no knowledge about the intended IT product and project scope, then nothing could be done, no projects goals achieved.

This seeming incongruity is explained by the fact that requirements engineering is often hidden and performed under other role names, and other project activities.

The long-term solution is of course to improve the project process and put requirements engineering into it properly. However, this may require quite a revolution, mental and organisational. A short-term improvement may be more pragmatically achieved by improving real requirements engineering, however it is done, under the guise of whatever names apply.
This tutorial teaches and gives practical exercise in how to do this.

1. Requirements engineering as part of practical project management

Since project managers are responsible for project resources and deadlines, they must of course be responsible for its scope as well. Therefore, they must often take care of all requirements engineering activities. How can this be improved without changing processes too much? There are many good RE practices hidden in PMI and PRINCE 2, for example.

2. Requirements engineering in agile

Agile framework is very requirements-centred, and creates huge potential for extremely well-focused and disciplined development, but this fact is sometimes hidden behind the façade of little documentation, networking, self-organizing and funny terminology.

Agile can be made sharper still by more consciously following the best practices of “traditional” RE, and there is plenty of room in agile for great RE practices.

3. Requirements engineering in design and programming

Many great RE practices are already implemented in methods better known as programming practices such as DDD, BDD, FDD and TDD, and their full potential can be enhanced if we exploit their RE-related powers fully. But even without these methods, practical RE issues have always been, still are, and will always be solved by programmers, especially by great programmers, who seem to perform their RE-chores on the fly, on their way to great code. We’ll study how it works, and how it can work better still.
Then, of course, RE is to some extent done together with design modelling, and we’ll have a look at this option, too.

4. Requirements engineering in business analysis

Business analysis is an important part of RE, but I have seen many projects, where it seemed to replace complete RE altogether. In these cases, there was a huge yawning gap between business analysis and programming. We’ll try to bridge this gap.

5. Requirements engineering in testing

Testing and requirements engineering aresister (or brother) activities in many respects, but in this part of the tutorial we’ll focus on how test analysis and design are in essence very much requirements elicitation and analysis by other means. Seen this way, both RE and testing can be much improved, and projects can achieve much better results if they are aware of this. We’ll try to discover how.

6. Exploratory testing as ersatz requirements engineering

In many-a-project exploratory approach to testing has managed to find and help remove bugs that other test approaches have failed to identify. Apart from its other good practices, such as continuing test design during test execution, and still more apart from often hysterical and ideological (not to say “religious”) aspects of the popularity of exploratory testing, its real power is in being a very effective complement to, or even replacement for requirements elicitation and analysis, which could not have achieved the same.

Great results can be achieved if we learn how to put these two unwilling allies to co-operation.

7. Requirements management is in everything

Now it seems that the author of this tutorial has caught excessive proselyting fever, but it is actually true: even in most trivial, every-day misunderstandings and failures, the principles well known from RE are at work. We’ll study a number of communication, organizational, group dynamics and other psychological issues for the point of view of RE and discover, how it may help improve not only our IT projects, but virtually all projects, including daily lives.

wtorek, 10 czerwca 2014

Sepcial Invitation to IEEE RE'14 in Karlskrona

Early-Bird Invitation
I would like to invite you to the 22nd IEEE Requirements Engineering Conference. This year, it will take place in Karlskrona, Sweden: a good, green, sunny place to visit 25-29 August! Meet us at You will find more data on both RE’14 and Karlskrona at

Until 20th June, a number of interesting 10% Early Bird rebates are available for conference tutorials: see here for details.

I welcome you as well to participate in my own tutorial “Requirements on a shoestring: how to cope in industry projects with poor requirements engineering awareness”. Meet me at: