00:04 hi everyone uh and welcome to today's webinar um today's webinar features
00:10 myself Phil Lou and uh but this is organized by test ray
00:17 maybe you could um start off the webinar and and welcome
00:22 our guests yeah thank you Phil um welcome
00:29 everybody uh my name is KARK I head the product marketing for Goldfinger group
00:35 and especially for testr and today's webinar uh we're going to talk about unlocking test organ orchestration in
00:42 jira for faster execution um with me is pH L he's the uh CEO and founder of
00:51 xosoft and uh before we get going further uh you know a quick reminder on
00:59 the on the webinar uh we are just going to mute all of you guys and uh you know
01:06 and we we request everyone to ask questions and feel free to uh you know
01:12 drop in the control panel on the left hand side and uh you know whenever possible Phil will you know address the
01:19 questions and we also have a session at the end of uh the the slides so that he
01:25 will talk about and address all your questions um about moving on to the Phil
01:31 L he is the uh distinguished speaker and he has uh you know as as a respected
01:38 leader with a deep passion for software quality and user experience his
01:45 Specialties include software quality process evaluation measurement and
01:50 Improvement um apart from his score expertise he is passionate about his
01:57 family cycling and travel um I'm like you I'm also eager to hear more from
02:04 Phil on his insights philu the platform is yours hi
02:09 everyone welcome to the webinar uh thanks for the introduction uh cardig I was um it's great to be here and good
02:18 evening good afternoon good morning to whoever is here in in the room today
02:23 um as mentioned I'm the CEO and founder of xosoft and we are a software testing
02:29 ser services company and uh testr asked
02:35 me to um provide some good uh knowledge and content on optimizing jira for test
02:45 management uh with a term that we've coined test orchestration which is
02:50 really bringing together the pieces uh of an orchestra right you have the
02:55 string section and and so on at the right time to play the right tune
03:01 and in order to make um test management as smooth as possible so let me move on
03:09 here and I'll get into some of the challenges that we all know today and as we know test data
03:17 management is a big issue TDM especially when you have very complex environments
03:22 in large companies um we just got done working in a bank and you can imagine
03:28 how difficult environment management and the test data management is in a bank
03:34 with all of the different uh security issues and different databases and different levels of permissions and so
03:41 on so um depending on the organization you may have different types of problems
03:48 or challenges uh in your organization depending on your size as well as
03:54 maturity here's just a little tidbit of information from software testing genius
03:60 that 50% of organizations uh deal with slow or
04:07 inefficient test execution resulting in Mis deadlines and basically bad
04:12 quality and uh with testers getting blam for a lot of the bad quality and I'm
04:18 sure this is not a new a new thing if you're in the QA
04:24 field in terms of uh having poor quality and possibly getting getting taking the
04:30 wrap for it and so we know that a lot of these things come from uh some key issues you
04:38 know like number three there uh test prioritization and coverage really trying to understand what is the best
04:46 area to put your effort and win right so depending on how much time you have I
04:54 mean there's all kinds of algorithms and people that'll claim that uh their testing method um works the best um and
05:04 then of course there's always resource availability issues whether it be environments being available or people
05:11 that are knowledgeable to configure the environments as well as uh dependencies
05:18 between different modules and different third-party uh plugins that you may have
05:23 to use or thirdparty data sources to configure in your testing and all of
05:29 these things really create a lot of complexity that didn't exist say five or
05:35 10 years ago when things were not all spread all over the cloud right so
05:40 there's a lot of challenges some of them new and some of them old some of the problems that we have
05:47 that we come across and it's really all the way from the beginning where it comes from user stories that are
05:54 unclear um maybe they're lacking information and and most of all they're not even testable
06:00 people don't know how to write acceptance criteria and if you can't write good acceptance criteria then you
06:07 lead that leads to bad uh poor test cases and sometimes
06:14 uh QA or testers may not be good at writing tests whether they be incomplete
06:21 not cover all the options or be inefficient repetitive
06:27 and and then I think one of the most most uh critical ones is test cases not
06:33 being organized and not reusable so we end up with a lot of duplicate test
06:38 cases or duplicate test scenarios that have quite a bit of overlap and so that
06:44 wastes a lot of our time and then um results results is a
06:51 big one right so management wants more than just pass or fail or did not
06:58 execute and they want to go deeper and we need to really understand what
07:04 management wants and how we can provide information for them to make decisions
07:09 so if any of y all want to um possibly uh chat and give us some of the
07:17 problems that you have that are are major problems uh as we go on that would be great
07:24 um I'm sure that you have other problems as well uh next slide is really kind of uh
07:33 summarizes some of what I said but also add some good information here in terms of uh skills right so testers these days
07:42 are not just manual testers button pushers but I think one of the most important skills that people need is
07:48 domain knowledge for testing the software that they're working in so if you're testing accounting software then
07:56 you really have to know a lot about accounting in order to do to test accounting software you just can't be a
08:01 button Pusher uh because if you if you do that then you can't write really good
08:06 test cases and so not only that you really have to understand the user
08:13 stories very deeply in order to give feedback that the user stories may be incomplete right and testers also have
08:20 to have knowledge about working in different test environments configuring databases all kinds of uh getting access
08:29 different data from different places and plugins and so on and all of these things are very very difficult to get
08:37 coordinated at the right time which is why we use the analogy of an
08:43 orchestra in terms of being able to call in the right pieces at the right time
08:49 and then be able to um execute testing in an effective Manner and once you if
08:56 you can execute the testing of course then of like I said there's the analysis
09:02 and the reporting so if you can't really show any value with your reporting and
09:08 your analysis then you might as well just stay
09:14 home so in summary you know what we call test orchestration or the term that
09:19 we've used test orchestration is for the process of managing the quality
09:25 activities in the de Dev life cycle which to me means getting the right people in the right place at the right
09:31 time and having the the right tools to get stuff done right um you don't you
09:38 don't show up uh on a job site as a carpenter and not have a good Hammer so
09:43 it's important for us as testers or QA to really understand tools very deeply
09:51 and know how to use them correctly and in the right instances and configuring them correctly using the right options
09:58 and so on and and so today uh the title of the webinar is unlock test
10:04 orchestration in Jura for faster test execution I'm going to be showing you
10:10 some of the stuff that we use jira for with our clients and this is the reason that testr wanted to us to come in and
10:17 talk today was that not only do we use test ray we use a lot of different plugins um to help us optimize jira for
10:26 our clients and so let's talk talk a little bit more about first about what test
10:33 orchestration involves so test orchestration involves
10:39 like I said the reporting and Analysis but also involves the in integration
10:44 with other parts of the organization um whether it be pulling in data from different sources working with
10:51 devops or managing all the environments of course there's the test execution
10:57 part but a lot of folks don't really really think too much about the test planning and the test automation now we
11:05 have a lot of teams maybe working on test Automation and we have people writing
11:11 test cases and planning out test runs and so on but sometimes test automation
11:17 is separate from the manual testing group so there's some overlap as well as
11:23 some parts that are end up not being covered um because they're not working
11:28 together as a team I don't know if any of you have heard had this happen before but I've seen it many times where you
11:35 have test automation testers that are writing code that is nothing close to
11:42 what the the manual folks are doing and maybe it shouldn't be but they're so far
11:48 apart and they don't really understand what each other is
11:55 doing like I said if you want to add uh some some questions in the chat uh we'll
12:03 take them right away and try to integrate them into the presentation today so here's some best practices for
12:11 test out orchestration first of all we we need clear objectives right so and these
12:17 objectives need to be a little bit more defined than just uh we want higher
12:23 quality right so um I've seen some of the uh executive management come down
12:30 and say oh we want 100% automation well and it's hard to convince them that 100%
12:37 automation is not the right thing to do but as testers we know that it's not so the question is how can you convince
12:45 management where to automate and where not to automate and automate selectively
12:51 not first the second thing that the other things on the on the top right there are
12:56 getting the right tools standard standardizing the test environment and implementing continuous integration
13:03 those are key elements of test orchestration one area that's overlooked
13:09 and I just mentioned that in terms of test automation along with manual
13:14 testers uh having them work close together to understand what each each other is doing and using that close
13:21 collaboration and working together to iterate and improve and not only working together with in our own QA group but
13:29 working with the developers very very closely to make sure that the requirements understood clearly and then
13:37 being able to execute in parallel mode so that we're not executing serially and
13:43 being able to Monitor and analyze test results so you know a lot of our clients
13:50 they they purchase Zer jira and they think that it's really going to solve a
13:56 lot of their problems and it will but it's it's not going to solve a lot of problems out of the box and there are
14:02 quite a few limitations on it one of the good things about jir is that it's very
14:07 very configurable but that's also one of the bad things about jir is so
14:12 configurable that the learning curve if you want to do more than just use a standard configuration the learning
14:19 curve is is quite uh it's quite steep it takes a lot of time to really optimize it very well and it can be quite complex
14:28 and if if you want to get some of the good reporting and dashboards or integration with other tools and
14:34 customize workflows and and and so on and so on you really have to have a
14:40 pretty much have a jir expert come in and help you it's it's pretty difficult andless you really dig deep into jir to
14:49 really configure it correctly especially for test
14:54 management because jir is not really built for testing it's built as an issue
14:59 management system so here is some of the things that we can do with jira but it's
15:05 not natively made for this and that's why there are so many
15:13 plugins um that jir works with for test management um test R is one of them and
15:21 that's what we're I'm here working with test R today for the webinar and we've also worked with them on several
15:28 projects in implementing cesr but we've also worked with many organizations here
15:33 on this in this little chart here in different types of jro plugins that do
15:40 test management and the one that you choose has to do with you know the
15:45 features that you want the pricing model you want and so on uh test ray is just one of them there are many others um and
15:54 some of are mentioned here there's probably dozens of test management
15:60 plugins that integrate with jira so um if you need some help with that uh
16:06 that's one of things that we can do with you uh if you if you need us
16:12 to so benefits of test oration with jir I think one of the I mentioned earlier
16:18 this kind of model that I have in terms of optimizing Effectiveness is doing the
16:23 right thing at the right time with the right people so if you can do all these things then you can get that improved
16:31 efficiency better test coverage improved collaboration and reduced cost so I mean
16:38 when you think about it the right people means many different things it could mean uh people with the right skills
16:45 right but it also means really thinking about the makeup of your
16:51 team uh at a more emotional level so thinking about introverts and extroverts
16:58 you know you can't have all extroverts working together because it's just not going to work um people are going to be
17:05 talking over each other and not listening and um so that is something
17:11 that is part of the equation of right people not just oh we need somebody that knows uh you know selenium or something
17:18 like that now I'm going to dig into um how we do it right so we've
17:24 worked with uh many different companies as I mentioned before and I'm just going
17:29 to show you some slides or some screenshots of how we worked with these
17:35 companies and we're able to use um a test management tool and in
17:42 this case it's test ray but like I said we've used many other tools or um
17:48 plugins for jira and we're able to we're able to get a
17:53 50% um reduction in in in test time for regression so that's significant when
18:01 you when a company takes their regression test time down by half that's
18:06 an incredible incredible gain does anybody have any questions
18:13 please feel free to ask if you if you need to okay so here's what how we do
18:21 it all right so first thing really the first concept is to tie together
18:27 everything so that it's inte cre together and you can see in this slide
18:32 here at the top this is a a screenshot where um we're testing uh a
18:40 bank I think I mentioned a bank earlier in terms of um you can see at the top is
18:46 the description of the requirement and then below uh is the actual requirements
18:52 along with the development tasks associated with that requirement and then you have below that test case
18:59 and so the whole point is being able to see everything on one screen that is
19:05 related together so let me go on and show you a little bit more detail here
19:11 um as you can see at the top there's requirements which have the uh creating
19:16 a child for a requirement linking a child to a requirement and also having a tree for the requirement in this case we
19:24 have the requirement of a credit card validation and underneath that requ requirement we have two development
19:32 tasks uh Bank info validation and card info
19:37 validation now you can also see that for this particular requirement with the two
19:43 development tasks we have these four test cases related to that and so um the
19:52 four test cases uh F FRS 14 through 17 and then so you can see see how all this
19:59 is working together right and then on the right let me show the next [Music]
20:05 screen on the right oops let me go back sorry on the right you can see the
20:11 defects that are associated with these test cases and the status so you can see that
20:19 to-do means they're they're they're working on it fixing it uh done and in
20:25 progress you can see what's going on with those defects that are associated with these test cases that are
20:31 associated with these development tasks that are associated with that particular requirement of purchasing a
20:39 ticket now you can see here that this particular test case when you when you
20:46 uh drill down into the test case you can see the test case the test steps
20:53 associated with that test case you can also see the person that uh did the test
20:60 case and you can see the requirement that that's associated with and the development tasks that are are in
21:08 progress for that particular test case the next thing is like I mentioned have
21:14 tying everything together right so we have a defect but we want to see everything that's related to that defect
21:20 right so we have of course we have reproduction steps we have the test cases that are
21:26 related to that defect that generate that defect and we have the people that
21:31 are associated with that defect so in this case we have the person that reported the defect as well as the asse
21:39 the one that's supposed to be working on fixing the defect only got seven minutes here I'm G to try to get going okay so
21:47 the next part of we talked about optimizing and being able to execute
21:52 tests faster and in parallel rather than serial you just need to have your test
21:60 plan and so having test plans enables you to have different rounds of
22:06 environments right and different builds and enables you to track all of your
22:12 results in one place so how many people here have difficulty tracking builds and
22:18 environments of course because you don't want to test all builds with all environments right and so most people do
22:25 it a spreadsheet and this can be inefficient we think it's better it's
22:31 easier to see all in one place with uh with a test management plugin as part of
22:37 jira this kind of shows something related it digs down into this particular round of testing with Firefox
22:45 uh just to show you the more details on this particular round with Firefox where we executed these test cases here
22:54 associated with that requirement and lastly you want to have
22:60 a trace what people like to see what management likes to see and what um
23:05 business analyst like to see is what they call traceability Matrix right you
23:10 want to see all the way from the requirement to the test case to the functionality that's being tested as
23:17 well as the test results so colors representing you know blocked or not
23:23 started yet passed in progress these are the kind of things that management wants
23:29 to see all in one place for any particular requirement and then one of the most valuable things about a test
23:35 management plugin is like I was saying being able to integrate manual testing and automated testing to see where the
23:43 overlap is where the overlap should be and where it should not be and so you
23:48 can see here that we've have this able we're able to separate out the automated
23:53 test cases but we're able to see everything in the same place and here we
23:59 have manual and automated test cases we can see we can actually even kick off these test cases automatically with
24:06 Jenkins through integrating the test management plug-in with our
24:12 cicd so like I was saying before um we want to show value as QA we want to show
24:19 value we can we need to show more than just pass or fail and so we need to get
24:25 down and really understand uh where the defects are what requirements are
24:31 generating the defects maybe we need to go back to the drawing board and work with the business analysts and getting
24:38 better requirements maybe we need to um work on
24:43 if we see that certain manual test cases are taking too long maybe we need to think about prioritizing those for
24:50 Automation and of course management audio wants to know when are we done testing right as well as what's my
24:57 coverage that's one of the typical questions and we need to be able to answer that and sometimes that could be
25:02 answered with past fail but most of the times it can't so here's an example here
25:08 of reporting on a requirement so you can see that we able to report on this these
25:15 requirements here and it shows what's passed what's failed what's
25:20 blocked and once after you dig down and you click on one of those uh color those
25:26 graphs you can see the particular ones the particular um requirements and test
25:33 cases that were failed or blocked and so on based on that requirement this just
25:38 gives you a little bit more detail in terms of the requirements and all of their statuses and what platforms or
25:46 test Cycles they were run on um so this gives you an idea of the level of detail
25:52 this particular one is for automated you can see there's a run ID there which
25:58 tells me that it's an automated automated run that was kicked off by
26:03 Jenkins and lets me know what the status of it is the other thing we want to know
26:08 is how our people are doing and this you know we don't want to encourage of course you know number of tests uh run
26:16 or number of defects found but it does give us a good idea of who's who's doing some work right um and we can also break
26:24 this down by requirement because we know that some requirements or or test cases
26:30 are harder to run or take more time than others um so you can you can see the
26:35 level of detail that you can get to with a test management plug-in that enables you to really dig down and get
26:42 information um which you wouldn't get with just a standard uh J configuration
26:48 out of the box here's another example of results that you can
26:55 get per feature in uh in jira and you can do this by breaking down and de uh
27:02 configuring your components correctly in jir let's see well we're coming down to
27:07 the end here right on time okay so I just wanted to kind of summarize in
27:12 terms of you know if you're going to be if you're a software tester or or QA
27:18 person and you really need [Music] to uh there's just never enough time
27:23 right and so what we need to do is organize our testing the best way
27:29 possible I hope that you what you've seen today um kind of demonstrates how you
27:35 can do this not only with jir but with a good test management plugin and having a
27:41 good configuration for your test management plug-in that integrates throughout tying
27:48 all of your uh requirements user stories directly with test cases and acceptance
27:55 criteria and then when those test cases organized into different runs with
28:01 different Builds on different platforms so that you can have your test runs
28:06 organize and then being able to see the test runs along with the test cases that
28:12 are executed for those test runs and then being able to map those test cases
28:18 to defects as well as defects into back into the requirements so you can see you
28:24 can get a good idea of root cause analysis on your defects and be able to
28:30 go back and improve on an iterative basis and just a three-point summary here you know it all starts with good
28:37 requirements testable requirements um whether you want to write uh uh you know
28:44 agile user stories or if you want to write you know they have we used to have these long requirement specs um and then
28:52 we have to have good connection and integration between all of our test artifacts as as well as our development
28:59 and starting from user stories to task and development test cases based on that
29:05 development automative versus manual and being able to understand how they work together the defects that are generated
29:12 from the test cases and then tying all that back into requirements and then
29:18 lastly of course I hope that you've gotten a good idea of how to generate
29:23 good reports other than past fail to really understand deeply um how you can approve your not only
29:31 testing but your whole development process so I think that's taken us a whole uh half hour here I know I've
29:38 talked really fast [Music] um but uh that's the end of today and
29:44 and thank you very much does anybody have any questions that they'd like to um ask uh feel free or you see my
29:53 contact information there you see the the folks at Tess Ray that sponsored the
29:59 webinar uh you will get slides as well as the recording from
30:04 today I hope that you've enjoyed it um I don't see any questions coming in
30:12 so I guess we'll end the webinar and have a good have a good rest of your day and a good weekend so thank you very
30:18 much everyone I see is there a question there yeah I just got one question from one of
30:25 the resent named Daniel um as question was what are the future Trends in test
30:30 orchestration and how it will evolve over the years yeah that's a good uh that's a
30:37 good one and so everybody's talking about about AI right and um we at xosoft
30:45 we've been working on uh using Ai and our services that we
30:51 provide to our clients and I would say that probably the the biggest area area
30:58 that AI is it's just really another I would call it another layer or Nevel
31:03 another type of Automation and so what we see um see it doing for us is helping us
31:12 to write test cases and better requirements and but you know based on our research
31:19 and some of the testing that we've done it can do that right but it's still
31:25 going to require human intervention to to to look at these and sometimes throw
31:31 away some some uh test cases um I think that uh it will improve right um so
31:42 that's one area the other area that we see that is really uh taking off is the
31:47 analysis of data right so analyzing test results data as well as generating test
31:54 data is also an area that uh that AI can help us a lot in but you
32:02 know when you think about it people are afraid that this is going to take away testers jobs and I think it will take
32:09 away jobs at lower level manual testers they're they're always threatened in
32:15 some way but the way that manual testers
32:20 well lower level manual testers that just press buttons they're going to go away okay but senior level testers
32:29 whether they are able to really look at the output of AI and be able to discern what is good and what is not and throw
32:37 away of course what they need to as well as adapt and modify what they need to
32:42 and then be able to judge whether something as good at is as is out of AI whether it be data that's generated for
32:48 test data or it is um test cases that are generated by AI um so those types of
32:57 people people that are senior enough to really understand that they're going to be very very
33:03 valuable and then um the other you know if you're a manual tester and you're not
33:09 into automation your experience in the domain that you're testing is going to be more
33:15 and more valuable because being able to ascertain whether or not some stuff being generated by AI is reasonable uh
33:24 is going to be incredibly valuable so I think that's where I see that test
33:31 orchestration is going in terms of the people and the technology I think I've answered the
33:38 question I hope that one person esmail asked for the page to be bigger I'm sorry about that but uh I did the best I
33:46 can all right all right U thank you so much Phil uh I really enjoy the session
33:52 I'm sure our audience would have also enjoyed and if you have any questions please feel free to drop in the line and
33:60 we'll be more than happy to answer that thank you once again for joining thank you everyone
34:10 bye