00:00 foreign
00:07 [Music] and
00:12 today's webinar is called do developers oh there's bad
00:18 quality do developers write bad code on purpose and the answer is of course not
00:24 and so why do we have so many bugs and we work with a lot of clients these
00:30 days on their entire software quality process and what we find is that
00:36 a lot of the problems start in the beginning that requirements and so
00:43 um one of our our largest focuses now these days in working with companies on
00:48 their software quality processes is making their requirements better and helping them in that aspect so that
00:55 their entire software development um improves and not only in speed but also
01:01 mostly in quality as well so we're fortunate enough to have
01:06 critical logic with us today critical logic we ran into a year or so ago that
01:14 have a great tool that helps companies to put together the requirements in a
01:19 visual way and so I thought I would have a webinar with in
01:24 uh cooperation with critical logic so they could show you some of their product features that go
01:31 with some of the problems that we run into with requirements and helping our clients to solve those problems
01:38 so let me all right so my name is Phil Liu and I'm with
01:45 exposoft I'm the CEO and founder of XPS soft we've been in business since 2006.
01:50 and we work with companies and providing services in software query Consulting and working
01:58 with companies in their processes but also Hands-On testing work we have
02:03 offices in San Francisco and in Beijing and that's our
02:10 Shameless plug about xposoft as I mentioned I've got Bob Johnston
02:16 here from critical logic I'll let Bob talk about critical logic as soon as we introduce Bob
02:22 um but first a few little housekeeping rules all the attendees have been muted but you feel free to ask questions in
02:30 the chat panel we'll be answering them throughout the webinar in
02:36 um as we go along or if we don't get to them we'll answer them at the end but in
02:42 any case everyone will receive the recording for the webinar and if you want to uh do some tweeting
02:49 feel free to do that there's the hashtags there and so let's uh let's move on
02:54 again my name is Phil Liu um been involved in software quality for
02:60 a long long time and actually did my PhD in in software quality measurement so
03:06 I've been around and in the software development for forever I don't want to talk about how old I am though but I do
03:13 on the side I do like to travel I think I've been to over 70 countries and
03:20 cycling is my passion which is why I got my cycling Expo cycling gear on just in
03:25 case somebody wants to go out for a ride so Mom why don't you introduce yourself
03:32 and and talk a little bit about critical logic and uh how you got here today yeah
03:38 thanks Phil I I I'm gonna start calling you Dr Lou I didn't realize that was a
03:43 appropriate um yes I'm I'm Bob Johnston I'm the CEO and founder of uh critical logic uh we
03:50 also were founded in 2006 so we have a very parallel uh history that to xposoft
03:58 um and uh our company we primarily do uh Consulting work very similar to what
04:03 Phil does we focus a little more on business analysis um but we always try to bring quality
04:10 improvement to our um to our engagements uh but we also were kind of a dual company because we
04:16 also have a software product a technology we'll be looking at a little bit here today it helps us a lot with
04:23 our work in terms of creating um unambiguous clear requirements and
04:28 that leads to much better development work and especially in our case much
04:33 better testing work trying to improve the whole quality of software through improved requirements and integration
04:40 and quality from requirements for design through development and right into testing so uh integrated software
04:48 testing or process SLA that's kind of my message here I am in Salt Lake City Utah
04:53 in the United States and although my library so also in San Francisco our
04:59 company is a California Corporation but we are scattered around we are very much a virtual company do a lot of work in
05:06 Pacific Northwest but all over the United States and and we do have some
05:11 Global customers as well um being here in Utah I'm very much avid
05:17 skier and hiker we spent a lot of time in the mountains my wife and I family um hiking and skiing doesn't matter what
05:23 the weather or season is that's very very much a passion for me and I'm also
05:29 very hooked on physics I'm uh anything to do with physics whether it's astrophysics in the new James Webb
05:36 Telescope and all that stuff I'm a sucker for those headlines and I also love uh you know particle physics which
05:44 I also is a passion of mine so I'm not a doctor but I am kind of a amateur
05:51 scientist uh so so Phil that's me okay I'm curious you know you say SLA is that
05:57 service level agreement uh software uh no okay maybe that's the wrong
06:04 software development the whole software development life cycle okay sdlc I love the D out there
06:12 oh okay because it's serious because you know we have a lot of clients that um
06:19 have software that's developed by others right and so they need modifications to the software or getting software
06:26 developed and so there is an SLA and it's interesting to be able to tie an
06:31 SLA to software quality so that's another issue that we're dealing with with some of our clients well that's an
06:37 interesting maybe we'll talk about this a little bit later but our primary focus of Consulting work is implementing
06:44 commercial systems so our our focus is on developing interfaces and data
06:50 Transformations uh for businesses that have bought other commercial software uh
06:56 so in that sense we're testing it and we're an environment where those off-the-shelf software plus a bunch of
07:02 customization so yeah so it's SLA is an important thing because we try to hold
07:07 software vendors speech to the buyer yes right one more thing Bob I noticed you
07:12 have a nice uh critical logic uh jacket on in your profile picture so when I say
07:18 I'd like to trade give you an XPO stuff Jersey and you can give me a jacket is
07:24 that all right it's a deal okay it's a deal all right
07:30 um well let's dig right into it here you know in terms of um could developers writing bad code and of
07:37 course they don't and so some of the things we'll cover today um is you know if they don't write bad code
07:44 on purpose what happens why do we have defects if if developers don't write bad code on
07:50 purpose and a lot of it uh we'll dig in some of the into some of the root causes of that which as you can probably guess
07:57 have to do with requirements and then talk about making requirements uh testable
08:03 and then dig into how iqm studio uh enables people to work with requirements
08:10 and see them in a visual way and then transform requirements down the street Downstream into tests as
08:18 well as automation code yep so uh let's dig into it here
08:26 um I think yeah as you know there's lots of studies that show that you know
08:32 requirements are a huge source of of defects and how expensive it is to fix
08:37 defects later on in the development cycle versus fixing them in the requirements but the thing is that you know a lot of
08:45 uh development shops they still don't pay a lot of attention to requirements but the root cause of defects is often
08:53 not encoding and um there's been lots of studies that show this uh unfortunately
08:59 and there's still just not enough attention paid to requirements and you
09:05 know requirements can be really complex Maybe Bob do you have any it says common wisdom Bob you look like you have a lot
09:11 of wisdom in that area maybe yeah a lot a little bit every year so I
09:16 have a lot of Reason yeah so um the uh yeah we talk about requirements a lot
09:23 and you know in today's world requirements is almost a dirty word uh
09:28 it doesn't really have much meaning especially if you're in an agile environment requirements are kind of shunned or or they're just not defined
09:35 very well but if you look at my little drawing here that really requirements is
09:40 you know the needs what are the businesses what's the intended functionality or behavior of the
09:46 application uh and in some ways you can think of it it's the instructions to the developer team right what should the
09:53 developer team do to build this software and there's a lot a lot of parts here it's very complex but you see there's
09:60 user stories you cases all the things we're familiar with that we think of British requirements but there's also a
10:06 lot of other stuff that goes into uh defining what to actually build standards you exercise looking at the
10:13 old system much of requirements even today we call ourselves modern developers but still today we depend so
10:22 much of wet wear you know knowledge of individuals SME knowledge and and so on
10:27 and there's lots about every project has a whole wide category and this information that we call requirements is
10:34 not very well organized it's not very well integrated there's all kinds of uh
10:39 challenges to take what's in that requirements cloud and and get the
10:45 development work done and what's really causes a lot of problems is that there's
10:51 an interpretation going on in that analysis of what's in that the
10:56 requirements cloud and that re interpretation is done by developers
11:02 independently of testers in most cases so the in the the judgments the analysis
11:07 the decisions the conclusions reached by those two groups is often not
11:13 synchronized very well so they have the different view of what we call the requirements because there's not really
11:20 good strong process to integrate and create a single view of what the requirements really are so the
11:26 development and tests can both be on the same page of getting your work done right so as complex as my message here
11:34 and much of the problem uh is not what the developers or the testers do but
11:40 what they analyze and what they have available to them in the compact the complex file requirements so yeah so
11:48 that's the point here it's complex and people read documents in different ways or read documents and see different
11:55 conclusions yeah I think I think what we've run into like you said is complex so you have
12:01 this complex thing uh going in on the left hand side of the
12:06 cloud and on the right hand side what we found is that a lot of people get into implementation details
12:14 um before the actual requirement is really well known and so talking about
12:19 buttons and so forth that's actually the requirement but it really is not the the
12:25 actual requirement in terms of from a business or end user point of view what they really want to accomplish yeah we
12:32 we call that Phil not practice we call that solution independent requirements
12:37 right that the requirement is what it should do functionally how it does it uh
12:43 if you jump to how how it's supposed to work right away you've already started making some some decisions that may not
12:50 be accurate uh when when you finally look at the actual requirement in completion right okay
12:59 um so let's look at some of the reasons for these complex requirements or poor requirements uh you know a lot of times
13:05 people assume things right and so one developer may assume that a requirement
13:10 means a certain thing but the business analysts or the stakeholder meant something different and so the
13:17 requirements versus design issue is what we just talked about um
13:22 ambiguity is another big issue and there are actually ways that you can uh people
13:29 think well how do I solve ambiguity but there's a lot of tools and ways that you can solve ambiguity
13:36 um and you know here's some examples on the right that from the complexity
13:42 ambiguity of poor requirements right so we've run in these are just some
13:47 examples I talked about on the right on the fourth bullet point you can see that that's already going
13:53 down into the implementation details right and uh so those kinds of things
13:59 when you dig down into implementation versus looking at the requirement and you may see words like expected
14:06 confirmed correct but those things are not really well defined so there's a lot
14:11 of ways that you can even just scan your requirement just for certain words and if you clarify those words that will
14:18 actually you know be part of uh part of your 80 20 approach in cleaning up your requirements
14:24 um right maybe you have some additional knowledge that you can share on poor
14:29 requirements and maybe you know expound on some of these yeah unfortunately I
14:35 actually have some hope that AI is going to help with this pretty soon is looking for ambiguity but in because much of our
14:42 uh requirements that we deal with are written documents right human language so
14:48 um we look for instance another example I would bring here is if you see a user
14:55 story that says if the user does this then this should happen
15:00 uh was always missing uh when you see a written uh narrative that says if then
15:07 what's usually missing and ambiguous is the else part right if then this should
15:14 happen well what if that doesn't happen what else what else should happen right uh uh if that if if I enter a valid user
15:21 ID I should get logged in okay well what happens if I don't either use a valid user ID so then what happens
15:29 right so oftentimes that structure we look for in languages if statements but
15:36 we look for the then and the else and if we don't see an else or are very clear then we will raise that question or
15:43 we'll we'll just tag it as saying hey there's a little ambiguity here let's see if we can't clarify this so uh these
15:51 examples we have on the screen are also kind of Target words we find when we read things so yeah
15:57 well that's interesting when you mentioned the if then else it really makes a lot more sense because as a
16:02 coder you know you think of the select statement and you think of an if then else and there could be
16:08 you know six seven levels of if then else right suggested yeah it can get
16:15 complicated uh and Beyond human ability to really see it I mean when you read a
16:20 sentence like that your brain doesn't think well wait what happened what they don't you don't think in terms of if
16:26 then else or a select statement or you know uh ambiguity at all
16:32 um but uh that's why there's a skill and we need some tools to help us figure out
16:37 if we're looking at ambiguous requirements or not wow
16:44 okay well let's talk a little bit about the other side in terms of ways to make uh requirements good testable
16:52 um we've talked about the ambiguity and incompleteness of requirements making requirements understood and that really
16:58 leads to testable requirements right and so testable requirements we want to make
17:03 sure that as Bob just talked about it in terms of the if then else having all of
17:09 the possible inputs and outputs uh predictable and being able to to know
17:14 what those are one of the things I'd like to mention here is the stakeholder issue that you
17:21 know what we find with a lot of our clients that we work with is that multiple stakeholders are involved and
17:27 they have such different viewpoints on what should happen and what should not happen and then
17:33 when you get to the point of doing acceptance testing and then the stakeholder says oh that's not what what
17:39 the way it should work so how can we can we get around that and avoid that maybe maybe Bob you've had some experience
17:47 with that as well uh yeah very much I mean this is the what we call that as consensus right if
17:54 you don't have a consensus or a False Consensus and that's really what you're
17:59 talking about everybody thinks they know the stakeholders the business the developers the testers they'll go yeah
18:06 I'm good I I know what I'm supposed to do but do they have a consensus of what
18:11 they're supposed to do are they in agreement do they do they see apples or do they see oranges and this leads to
18:18 all kinds of problems right so uh one of the things unless you have really clear
18:23 requirements and they are you know agreed upon that they are valid and that
18:31 all the different stakeholders can read and see the same thing and raise their hands and say yeah that's what we're
18:37 going to do or wait no that isn't what we want to do if you don't have that discussion and that
18:44 um again consensus of what we should be doing and a mechanism to see yes that's
18:50 what we want to do and point at something then then you're going to have problems right you're going to have
18:55 technical debt you're going to have a conflict of an inconsistent rules to how
19:02 the software should behave so yeah that these things here validated by stakeholders is really important
19:09 um to get uh not just clear requirements but agreed upon requirements yeah yeah I
19:16 think uh I think the big problem here is something that came up on my desk today is uh in terms of reading
19:23 uh one of my uh project managers was saying you know I sent the client this
19:28 emails and blah blah blah and I said well what if they don't read that and so a lot of people just don't read right so
19:36 um that's why one of the things that I thought about when I saw the critical logic tool was that the visualness of
19:43 requirements and therefore enabling people to really understand it at a glance without reading through some long
19:50 user story and say oh yeah this is okay and this going like this but being able to look at it quickly and be able to see
19:57 if there's a problem I think that's one of the one of the powers of um you know that's one of the things about humans is
20:04 being able to see things visually and conceptually and I think that's really important yeah visual is good yeah
20:12 um okay so you know when we get to having good requirements part of the uh
20:18 one of the critical elements of a good user story is acceptance criteria and uh
20:24 again we work with our clients on on acceptance criteria and a lot of them
20:29 you know I wouldn't say bad acceptance maybe bad is the wrong word but incomplete acceptance criteria or
20:35 unclear acceptance criteria you know if you look at this example here you know
20:41 user can create a child requirement from a current requirement successfully I
20:46 mean obviously that could mean different things to different people right so what does success mean and what does what
20:54 does Child mean right so a lot of these things you know from one person's point of view maybe a coder they may think one
21:02 thing and then somebody else may think another thing and so I think you know we won't I won't read through these exact
21:09 um ways of correcting that statement but I think it's pretty obvious that when you
21:15 have a requirement and you have a big broad statement like that under the uh ungood or the non-good acceptance
21:22 criteria now we want to be able to clarify those things um and as Bob said the if then else
21:29 thinking about that is also very very important when determining a a good acceptance criteria
21:36 um right yep let's see let's go on here uh so you
21:43 know we have as I mentioned before one of the things about the critical Logics tool iqm Studio that I
21:50 that I really liked was the visualness of it and being able to visualize
21:55 um some cause and effect basically input and output uh you know this happens and
22:01 then you know it goes into this magic cloud and outcomes in your software this is how it comes the result whether it be
22:06 a report or a button that you push or a choice that you make in a
22:12 um choosing between different options and being able to easily visualize that
22:18 makes things easier for for different stakeholders with different points of view to to um
22:25 to agree and get consensus yeah yeah
22:31 that's right um bill that so I think you know what what we're starting to talk about here
22:36 now I think we've defined the problem pretty well right it's not easy to distill and synthesize all these
22:44 various sources of requirements into a common understanding of what should
22:49 really happen so we many many years ago started realizing that was something
22:55 that that required some technology to do so we'll talk more about the technology
23:01 here but the basic idea is to take that cloud of complex requirements and be
23:07 able to pull pieces of that into a what we call a cause effect diagram as Bill
23:13 said a big diagram that we'll look at some real examples but what that diagram represents is The Logical rules of the
23:21 various inputs and outputs that that are encountered in the requirements right so
23:27 if I do these things this should happen if I don't do them this something else should happen this you can't get to this
23:35 state of the system unless these things are true and so on so once we build this
23:40 this visual diagram you see a little picture of here we then can have
23:46 something that is unambiguous and if you are a tester or developer or a stakeholder you can look at that and
23:52 there's no question about where that line goes what's required to get to a certain standard condition uh what the
23:58 various combinations of inputs are that is all the diagram is capable of
24:04 defining all of those relationships and again we'll look at that a little bit more but the idea is we're trying to
24:10 synthesize the important parts of the requirement of plowed into a common view
24:16 of the functional rules of the application okay so um and then that common view can be
24:23 agreed upon reach a consensus of the stakeholders and provided to the testers
24:29 and if you have an enlightened Development Group they will also look at that and say ah yeah that's what I'm
24:35 supposed to be and those are the tests I'm going to have to uh pass so I better
24:40 pay attention yeah so so yeah this is kind of an introduction to the idea of synthesizing Clarity out of the cloud of
24:49 importance right well I'm looking forward to seeing more details of that Center diagram
24:55 there yeah well we'll get there a couple more slides all right oh there's a little it's a little bit bigger so we
25:01 can see a little bit more now okay yeah yeah so yeah one of the powerful things about uh that we've looked at with your
25:08 with your tool is the the ability to take those requirements and then you know if if you can generate
25:15 uh clear and unambiguous requirements that are also complete then it also
25:21 generally you know it naturally leads to test cases which also have all the past defined so that's one of the great
25:28 things that we thought of when we saw your tool Bob in terms of generating test cases with acceptance criteria and
25:36 also um integrating with different tools to generate uh test automation scripts so I
25:42 think it's one of the great things about iqm studio so I'm really looking forward to seeing CNN in action as well yeah so
25:50 so that's that's that's where I feel and you know I won't spend much time on this slide but the one important thing I want
25:55 to point out here is that the process of creating these diagrams uh requires
26:02 certain um the formality that I understand what
26:08 the requirements are right so I mentioned before the if then else statement when we build one of these
26:13 diagrams the logic diagram you see there that forces us to ask and answer
26:19 questions back to the requirement Source right so we've raising ambiguity if we
26:24 we can't complete the diagram because it doesn't tell us in a user story what's supposed to happen then we'll say hey
26:30 what's supposed to happen here so the mirror Act of creating this logic diagram creates this feedback loop that
26:37 improves the requirements naturally because we're going to say if you build a model we have to have enough information to build the model that
26:44 improves the requirements uh kind of as as a side benefit of building the model
26:50 but an important one because if the requirements are incomplete or ambiguous we won't be able to develop the model
26:56 and we won't be able to test right and we can prove it hey if you don't have the right requirements we can't test it
27:03 and then that's the whole point here is the the creation of the diagram works
27:08 both directions Upstream improving requirements Downstream to create very
27:13 useful artifacts of tests and scripts so but actually I guess you could say kind
27:19 of forces you to write good requirements maybe that would be a good way of putting it yeah we we talk about
27:25 validating the requirements so if we take the requirements and they don't make sense to us or they're missing
27:31 pieces we will say hey we can't validate these are correct we have some questions
27:37 and the questions get answered and then the requirements get validated
27:43 yeah that's uh probably validate as a kind word just forcing people to write
27:49 yes we try to not piss people off yeah okay
27:54 so yeah one of the one of the final things before you dig into the actual demonstration is that reporting
28:01 is critical right whether you're doing automation or test results or anything and so I think it's important to realize
28:08 and think about you know what are we going to do with all these requirements what are we going to do with our tests
28:13 and it's it's always a constant problem for QA to share the value of what they're providing so when we get to the
28:21 hello Bob I want to make sure that we dig into uh the results that come out of
28:27 what we're doing so that we can show what value we're adding whether it be tracking defects or uh the completeness
28:36 of requirements or and things like that so I think one of the critical things is
28:42 um is the uh coverage right so that's one of the one of the powerful things about the tool is that ensuring that
28:50 coverage is is complete based on the requirements right so if the
28:55 requirements are complete and the tests are complete then the coverage is going to be good so let's um all right let's
29:01 dig right into that there so I see you have quite a few uh we put in quite a
29:06 few screenshots from from the tool but I think these are probably a little too
29:12 hard to see for folks why don't we just um dig right I'll show you these in real
29:18 time when we get there but yeah there's a variety of reports interested about it as you were saying yeah let's dig right
29:24 in here and oh yeah one of the last things that I just which I just talked to an amazing was the generation of of
29:32 executable code you've got just about uh everything in
29:38 the world on the right hand side there that's fantastic python we do a lot of
29:45 work in Python and Java as well as selenium um this is fantastic most most of our
29:51 clients work with one of those uh platforms on the right hand side so that's fantastic yeah and our our whole
29:58 idea is in our framework is totally configurable to generate executable code of any kind
30:05 um so you know our our framework is very very flexible in terms of the
30:11 syntax of the code the structure of the code and so on so once we generate that code uh one of these tools the one
30:18 you're using or maybe more than one we'll be able to generate the correct syntax so you can run those tests with
30:25 that tool yeah great okay well I'll let you uh take it away
30:31 here bobman you can okay you can I think you need to make me percent or something
30:36 now and I think I'll I will uh
30:42 stop my share okay and you are you are a co-host so
30:48 you can uh share away I'm sure your screen yeah okay
30:55 you are sharing okay get rid of that Zoom meeting
30:60 um go away yes
31:06 okay so um I'll start out I want to start out by
31:11 I'm going to do an example and we'll want to work through a model and the whole processor in the end I'm going to
31:18 go pretty quickly there's a lot of details here that I'll kind of just Breeze through because there's a lot of
31:24 functionality but I I kind of want to set this up um this way I want to set up a oops an
31:31 example I can click the right button here
31:36 there is a I'm going to use this little example here of a testing let's say I want to
31:44 test this sign on this is a a company called cool they sell outdoor clothing
31:49 and Equipment they're here in Salt Lake City I'm a big fan of theirs I'm on their side a lot but in a really simple
31:56 case here I'm going to create a model of the rules for signing in so on this
32:02 screen here just signed into my account there's a sign in screen it has things like use your email and password and so
32:10 on but it has rules for instance if I click sign in without filling any data in right it's going to give me the error
32:17 message says You must enter a valid email address but if I do enter an email address there's another rule
32:23 um you know that says if I don't enter my password it's going to give me another
32:30 error that says hey you got it in your password okay this is pretty simple but the complexity of the rules here is with
32:36 a little surprisingly more than you'd think right because there's you have to do things if I ordered the data has to
32:42 be valid you have to have to put something in the valid not blank and blah blah blah so we're going to analyze
32:48 that a little bit here so let me uh and let me start also that the requirements
32:54 for that sign in logic are something like this I made this up but you can see
32:59 here I have a requirement list of requirement number one and sign in user
33:04 ID is required password is required valid credentials that I am also of
33:11 um uh requirement that the cool logo is displayed and there's a correct
33:17 instructions in the right language on the screen okay and this would be much longer for a real example but the idea
33:24 here is I have some requirements I want to test I want to understand I want to
33:29 trace back to these requirements okay so that's the setup I have a little simple application and a set of requirements
33:35 I'm going to try to test so I will take that set of requirements as an analyst here and I'm going to come to my tool this is
33:43 called iqm studio and we're going to be focusing on modeling and automating but
33:49 as Phil was talking to a minute ago we also want to be able to synchronize things we produce like test cases up to
33:57 a test management system so if you have a test manager system like Azure devops or jira or something like that we want
34:04 to make sure that our test cases that we are generating are also known and managed by another tool we're creating
34:11 integration to that we also have our tracing Camp building where we Trace everything we're generating in our model
34:18 back to that set of requirements I showed you so if you have any requirements in it and as your devops or
34:25 some other tool that has requirements or maybe you just have a spreadsheet requirements we want to track what we do
34:32 back to some external requirements okay so high level let's look at model okay
34:38 so this is what they UI looks like okay we have a drawing template or an area
34:45 where we can draw in this workspace and we have details about the drawing over here what we're looking at is something
34:52 we call a clause effect model and the lines between what we call nodes here
34:57 are logical connections between the nodes okay so we'll get don't want to
35:03 get two into the weeds here about this but let me just walk you through this little example so here this example is
35:11 going to test the sign in screen for the Cool website right so how what are the
35:17 rules to sign into the Cool website well first thing we have to do is log into the Cool website and get to the account
35:24 screen where I can sign in right and once I get to the sign in screen I want
35:29 to be able to click sign in but first I have to enter the user ID and the password or some combination of valid
35:36 and invalid user IDs and passwords so what this is describing is when I when I
35:43 set out to test whether the user ID is correct and valid and certified there's
35:50 really three choices if I have a blank user ID I can leave it blank completely I can
35:56 type in a valid ID or an invalid ID and we're saying okay the rule is the
36:01 requirement so I have to be able to handle those three different situations so here we're saying okay but but this
36:07 is a a part or important part of the logic here only one of those things can
36:13 be true at a time you see the blank valid or inbound right and the same structure here for password you see the
36:21 blank valid or invalid for the usual ID entered right
36:26 so let me just jump ahead a little bit for us to get to the account screen my
36:33 account screen and verify my personal profile information there's several conditions have to be true I have to
36:40 have a valid user ID and you can see this line right here this connecting valid to account screen I also have to
36:48 have a valid password so this connector here is connecting to that and I have to
36:54 click the sign in button click actually click it if I click it and it's valid
37:00 user ID and password I get to the account screen and then I can go on to verify my account data okay now
37:08 if we look at some of the other conditions possible what if I leave the user ID blank well it turns out if the
37:16 user ID is blank and I click sign in I should get an error message that tells
37:21 me the user ID is required you have to enter it right and so on if I if I enter
37:27 the uh an invalid user ID and then I leave the password blank okay
37:34 then I'm going to get a different error message if I enter a valid user ID and
37:40 then I don't even care about the password but if I enter uh invalid
37:45 password then it's going to want a valid password here and that's going to take me up here so it turns out this through
37:52 uh you know there there's a uh some training needed to be able to uh
37:57 actually build this but it's actually pretty straightforward okay when you look at it now this is the the the cost
38:04 effect model view and I put this little note down here this is a case where when
38:11 I read my requirements it wasn't really clear what to happen what's going to happen if you enter the password but you
38:18 left the user ID blank so I said hey it's unclear here to me what's going on uh I made an assumption that if you
38:26 don't enter um a user ID but you're doing a password that something supposed to happen right
38:32 but certain messages supposed to appear but this little note is going to be pushed back from me to the requirements
38:38 okay now let me just stop there and say I've built this little drawing
38:44 now here's what the tool actually does when I click on generate button here
38:51 what's going to happen is going to analyze all the various paths through
38:56 this this logic diagram with various combinations of valid in value and blank inputs and all the things you can do and
39:04 it's going to tell me there are 18 different paths from this simple model but I can test all 18 paths and five
39:10 test cases this is what it's saying right here okay and I can immediately go look at those test cases so here I'm
39:17 going off to my report Center and our simple basic uh test case view is right
39:24 here I'm going to just open this up and this is showing me the test cases that I need to test all
39:32 of those rules that's right so test number one is that I go through valid
39:37 user ID valid password click sign in I should get to this guy and end screen and then I should verify my data test
39:45 two is enter invalid user ID test three is entering valid ID but in the password
39:51 and so on so this mathematically defines all the different
39:57 combinations of input that need to be tested to give us full coverage of all those business rules for that screen
40:04 okay and there's some other views here I won't spend a lot of time but like
40:09 um you know some people like to use a bdg format right this is the same test cases
40:15 but here they are in bgd format given this that when that happens then this
40:21 happened then verify these are given when then structures that represent the same test cases that we generated
40:28 um we're gonna push that button okay so and so on one of the other uh there's
40:34 other report formats or so you may find useful but one of the important things here is the requirement traceability
40:40 Matrix so I'm going to look here and this actually creates a spreadsheet I'm
40:46 gonna go ahead and create that spreadsheet and that spreadsheet then shows me all the requirements that I
40:54 have that I'm fracking okay and all of the the ones that have test cases that
41:01 are testing that okay oh I'm sorry I needed to regenerate well
41:09 generate reports uh okay very well
41:16 uh bye
41:23 spend some time on this critical
41:28 that that way to go
41:35 it's a demo right uh shoot
41:42 I must have I think my Excel crashed on me there we
41:47 go all right okay so again it's going to tell me uh
41:54 what test case uh this is test by test four this is test five this requirement
41:60 is tested by test two and test three and these two requirements are not tested at all okay so this report is telling me
42:07 the model is including functionality for these three requirements and exactly
42:12 where those requirements are tested and there's also other requirements that are not included in the model that are not
42:19 tested okay so this information is retraceability Matrix is very important input information okay now
42:27 um let me uh there's a lot of detail here let me just go one step into this
42:35 let's look at this verify this node here verify profile details or
42:41 um so this when I edit this so there's some information here about what about
42:49 what happens when this node appears in the test case okay and there are some
42:55 requirements that I can say hey when this node is in a test it's testing
42:60 requirement number one so I can assign employment one to that node and wherever that node shows up in the test I know
43:07 that node and that test is executing against that report and I also the other part I'll just jump
43:14 ahead a little bit here the other part that's in this individual known it's the
43:20 automate this is telling me connect a little bit of automation to this node so
43:26 when this node appears in the test case I know how to generate the automation code for that particular step in that
43:32 test okay so let me jump ahead on that too uh I know I'm going faster and
43:38 there's a lot to see but let me jump over to the automated side just to watching our time here
43:45 um so in studio I'm going to navigate over to the automate side and and then
43:52 the automated side we have a little different view of what we have stored in
43:57 this case the model I'm working on is called sign in to and what we see here here are the same five test cases that
44:05 were generated by the model if I open one of these test cases now we're looking at what we call our keyword view
44:12 of the same test so the same test you can read it here multiple app intervalid
44:19 user ID in about password click sign in verify the profile screen is displayed
44:25 and verify the profile screen has the correct data on it okay those are the basic steps of this test case that was
44:33 created by the model and we've also have behind the scenes a little bit old
44:38 Studio what to do on each one of those steps for automation so a little different view here I can
44:45 click this narrative button and generate uh
44:52 this view this is what we call our narrative view which is an exact mapping
44:58 of what the Audi manuscript is going to do step by step by step okay so it's
45:03 going to launch it's going to click it's going to uh enter data into this field it's going to evacuate it into that
45:10 field it's going to verify these text boxes have this data so this is a human
45:16 readable version of the automated script okay if you will so even as a business
45:21 analyst somebody who developed the model somebody who doesn't know anything about code I have a vision of exactly what
45:28 that automation script is going to do step by step so now here's where the kind of magic
45:34 comes in I'm going to generate just by clicking that button there and you can't see it because it's way at the bottom of
45:40 the screen it right then generated a automation script okay and I'm going to
45:46 now bring up all the automation tool I'm using for this demo we use a tool called
45:52 repeats okay that is from a company called uh
45:57 inflectro and this is their tool they use some there are tools based on selenium and
46:03 JavaScript so it's very familiar to a lot of people but this particular Tool
46:09 uh so what I'm going to do is right here right now this this is a view of their
46:16 uh their structure and this is the test case I just generated so let me open
46:22 that up this is the actual JavaScript that I just generated and it shows right
46:28 here this was created by iqm Studio on this date at this time so this I just
46:34 generated in one second into the correct directory structure so that this
46:39 automation tool can actually see and open that script okay so this JavaScript
46:45 right here is generated and commented annotated you can see the step nodes all
46:51 this stuff in here that's generated by our program so that you can read this and it tells you exactly what this
46:58 script is going to do but the real thing is I can actually run this script right
47:03 now when my hands are in the air this script is going to be run by this automation tool okay so you're going to
47:11 share it start the web page navigate off to the account screen
47:18 oh sorry I should I put my hands in there too I put my hands in
47:23 uh yeah you should put your hands in there I put them in the air too soon because I need this changes I'm using uh
47:31 selenium Edge is my browser I need to tell it that so now when I click play
47:39 um it's going to uh interface with using the web drivers from selenium by
47:46 starting up an edge browser clicking through the account screen going ahead and entering the user ID
47:52 that I specified and the password I specify AS Val and clicking the sign in button and bringing up my account
47:59 personal information and verifying the data on that screen okay so that automation script just happened that
48:05 could have been selenium or redirects or any tools you want and you can see here this is the actual
48:11 logging that murderx creates the step-by-step um uh instructions and at the end it
48:18 tells us all steps passed right so it did all those things so what I just did
48:24 right there is go from a model to generated test to automated scripts to a
48:30 script tool that can run that script for me and I did all that in 15 minutes okay so that there was no coding involved no
48:39 test creation all that happened by the tools generating those things okay so
48:45 now when you look at results and result logging that we rely on this tool this
48:51 tool has complete result logging and with interface and keep track through its interface to its automation uh it's
48:58 a test management tool we can create code in these scripts that will log
49:03 errors into jira or wherever you want them right so when these steps run they
49:09 can I imagine log the results including bugs and screenshots and anything else
49:14 you want to put in these scripts uh as if you had written these scripts by hand or some engineer wrote them okay okay so
49:21 I'm that's really the whole end-to-end kind of a rush job but we went from a
49:27 set of requirements um to to a model uh to test generating
49:33 transcription generated to running those scripts okay and that all happened
49:41 basically orchestrated by iqm Studio okay
49:47 okay so Phil that's the demo well that was uh that was fast uh let me
49:54 go back to um back to this here
50:03 all right because everybody can you see my screen now let's see yep alrighty
50:13 okay so if anybody has questions or feel if you want to go back we can look at any detail there that I kind of
50:20 restaurant sure I appreciate the demo it was great there's a you know it was really fast so there are
50:27 a couple pieces um that I think are important that you didn't really talk about much and and that is really the the tying in with the
50:35 test management yes of cases and then the results coming back from the automation going back into
50:42 test management and uh you had mentioned jira but it also as you said works with
50:49 Azure devops as well which is quite important because a lot of our customers there they're working with Azure devops
50:55 or jira right so both of those platforms are really really important
50:60 um so it's important to tie that all together here's just kind of a summary of today I hope that um those that have
51:07 attended or watched the video will get this out of it in terms of the importance of of great requirements and
51:14 complete requirements it's really the start of uh quality software quality
51:19 software is really not all about uh coding but really starting with good requirements for coding right so that
51:26 people know what they're developing and um we work with Bob and iqm Studio as
51:32 well as in flexra so um we're happy to work with our clients on with these
51:38 tools um to help them to generate do all this test case generation and automated code
51:44 generation and I think we've gotten a great demo and summary our overview of
51:50 your tool Bob I really appreciate you being here and sure well my pleasure
51:55 yeah thanks for having me I I actually am looking for Bill to us working
52:01 together coming up on live different stuff I think that you know we're very much have the same value to to customers
52:09 uh I I I I really you know we've been doing this for a long time uh
52:15 and the industry has been kind of stuck right this same this webinar we just
52:21 gave Phil we could have given 10 15 years ago and it really hasn't changed that much and I'm really optimistic
52:28 right now that things are starting to change I think this message of big requirements and bring technology into
52:36 tracking and managing testing uh and and really getting serious about test
52:41 automation I I think we're actually as an industry starting to mature and start
52:47 to do that uh in a serious way great does any if if anybody has
52:52 questions you know please feel to type them into the into the panel um
52:58 but uh this will officially and end the webinar and if you have questions you
53:03 can just write in there's some contact information on here um uh there's lots of resources that you
53:10 can see in those links there in terms of our white papers on uh agile test automation requirements and so on so
53:17 um please write in and let us know how you thought about the the webinar uh the
53:22 webinar recording will be shared with everyone that uh attended as well as those that have registered and
53:29 um I think that'll be it for today so thanks very much Bob some more together
53:36 yes well enjoy the rest of your day and I am going to enjoy the rest of my evening okay all right thanks everyone
53:42 thank you all for attending very much take care bye [Music]
53:53 [Music]