Standard Jet DBnb` Ugr@?~1y0̝cßFN!j7՜a('`{6R߱eCh>3y[5$|*j|] f_Љ$g'DeFx -bT4.0 dv @ @ @     r r r r rrrrrrrrrrrrrrrrrrr r!r"r# r$ r%r& r' r( r)r r*r,r-r.r/r0r1r2r3r4r+r5r7r8r9r:r;r<r=r>r?r6 YS  Y   Y Y  Y Y  Y  Y  Y  r Y s Y E Y a Y d Y 2Y  Y   Y  jY ConnectDatabaseDateCreateDateUpdate FlagsForeignNameIdLvLvExtraLvModule LvPropName OwnerParentIdRmtInfoLongRmtInfoShortTypeniYYIdParentIdName        elBb@&O$!f2b2 Line34 TdHJI Ȁf2ab2 Line35}&-FQQL`GroupHeader0< %txLd.m0<`<b chei j*tblTests.Test Project*tblTests.Test Project@Description of the test project.Times New Roman*tblTests_Test_ProjectU)JGK~`aGroupHeader2(VnBᤏm <`DbcJei tblTests.CycletblTests.Cycle:The cycle or backbone number.Times New RomantblTests_CycleڛTLfK3m <`P b8cJei k(tblTests.Group/Suite(tblTests.Group/Suite>Touchpoints or major test area.Times New Roman(tblTests_Group_Suite{.|gMT-{m ;<`bcJi k Start Startm/d/yyyy(Start date for test.Times New Roman ۉtLznhm ;<`bcJi kEndEndm/d/yyyy$End date for test.Times New RomanUJ)e~GK `J DetailOT|CUVm`4bce&tblHardware.HW Name&tblHardware.HW Name@Short name of the hardware item.&tblHardware_HW_NameZ >rGsЖm;`%b ce9k0tblTest Involve.Quantity0tblTest Involve.Quantity@The number of test items needed.0tblTest_Involve_Quantity9e/Ӱ@ p]m` b` cexkHW DescHW DescXDescription of the hardware item, if needed.HW_Desc\Dj2m;`*btck(tblHardware.Quantity(tblHardware.QuantityXNumber of these items available in this lot.(tblHardware_QuantityiRm;HsUm;`/bHckDate AvailDate Availm/d/yyyyߜThe date on which this lot of these items becomes available for configuration.Date_Avail"Fd MFV0`GroupFooter1? G}&C`PageFooter'H)R1=m ;<`<abc,i j Text31 =Now()Long DateTimes New Roman9[Hi"fFէm ;<`abc,i jk Text32H="Page " & [Page] & " of " & [Pages]Times New Roman 4L׮1`ReOYCSY Y Y  Y 2ACMFInheritableObjectIdSID YObjectIdRoot Entryy PropData Forms @lyReports"@l)[e  2K !"#$%&'()*+,-./0173456LMdir 0yBlob TypeInfo3&PropDataCopyBlobCopyDirDatajTypeInfoCopyModules@l@lScripts@l@lVBA@lU Cmdbars@l@lDataAccessPages @l@lDatabases @l)[e0 @l)[eBlob PropData1PropData20 ;Ǿ)[eBlob YSY  Y Y Y  Y  Y Y  Y AttributeExpressionFlagLvExtra Name1 Name2ObjectId Ordernzf edY"ObjectIdAttributeTotal_Sum<7X$KD1Iu&d`<bGcd Label36Grand Total7M˘) Root Entry7 PropData Forms @l7Reports"@l)[e : =; !"#$%&'()*+,-./0173456<>?2ABCDEFGHIJNOdir 07Blob @TypeInfo3&PropDataCopyBlobCopyDirDatajTypeInfoCopyPropData1PropData20 ;Ǿ)[eBlob s(  0CB0 -YSY Y Y  Y  Y  Y  Y  Y ccolumn grbiticolumnszColumnszObject$szReferencedColumn$szReferencedObjectszRelationship   YYYszObject$szReferencedObjectszRelationship 0CB0ͬ%0Y?N\ :?FormHeaderDetailmIDd ID_LabelID LabeloProject_NameProject Named Project_Name_LabelProject Name LabelmDate_NotedDate Notedd Date_Noted_LabelDate Noted LabelmImpact_DateImpact Dated Impact_Date_LabelImpaVBA1ݲPROJECTwm PROJECTF_VBA_PROJECT  !"#$%&'()*+,-./0456789:;<=>?@ABCHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghi0000 0* pHd3Case Study IVR Network Test Change Management (new)l@  =    n=( %<9stdole>stdole f%\*\G{00020430-;C 0046}#2.0#0#C:\WINDOWS\SYSTEM\StdOle2.Tlb#OLE Automation/gADODB> ADOBC CD1C10-8CAA006D2(EA4C1CPROGRAM FILES\COMMONL9\msado21.tNMicrosoft Acct Date Labelo Change_TypeChange Typed Change_Type_LabelChange Type Labelm Change_DescriptionChange Descriptiond Change_Description_LabelChange Description LabelmImpact_DescriptionImpact Descriptiond Impact_Description_LabelImpact Description LabelmRecovery_PlanRecovery Pland Recovery_Plan_LabelRecovery Plan LabelmText18d Label19mText20d Label21mText22d Label23mText24d Label25ijMSysDbID="{56898E9D-2DC0-4C3F-B649-8EB24A54D010}" Name="Case Study IVR Network Test Change Management (new)" HelpContextID="0" VersionCompatible32="393222000" CMG="4240A6E369E769E769E769E7" DPB="090BEDEEEEEEEEEE" GC="D0D234B735B73548" [Host Extender Info] &H00000001={3832D640-CF90-11CF-8E43-00A0C911005A};VBE;&H0000mText26d Label27mText28d Label29FormFooterCross-References Schedule Impact!Resource Impact"Other Costs/Savings#Test Impact$StatusChangesYv1b N  : k & W  C t/  l Y 2 o\I"ʾChangesChangesChanges p0@@( @@@  MWJbUQkMWJbUQkOJmJJMMQkkfJUQkOJmJLJkQkSdi`k `dOo^Qk iQ^JmYdbkWYfkiQfdimk kMiYfmk kvkiQ^ mJL^Qk+kh+SMWJbUQk+kh+iMWJbUQkMWJbUQk`kvkJMMQkkdL[QMmk`kvkJMQk`kvkdL[QMmk`kvkhoQiYQk`kvkiQ^JmYdbkWYfkJMMQkk^Jvdom`kvkOLko``JivYbSdokQiOQSYbQOoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaLoNo`Io@Ko@oaQ  @ @ @         >23789 : ; < @ A B C D E F  456EGEH   r Costs/Savings#Test Impact$StatusChanges 0CB0ͬ;cj֟vF)7{ReportHeaderdLa0=Babc+e ghij3kg &;@b DID>j{N Ut1+IM8{Aa3@ChangesDC&yt1+IM8{IDgYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date Noted{nU5uB&f Et1+IM8{Impact Date̍ 'L}L;6t1+IM8{Change TypexWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description5N7:K& t1+IM8{Recovery Plan)>*@%0t1+IM8{Cross-References :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/Savings@ dLpÄtt1+IM8{Test Impactoc@qѻmt1+IM8{StatusChanges,Changes Affecting Test Ariald2e12f5g237hghMS Sans Serifi1ghj1ghk1l23:m4n2o2p5r35zijMS Sans Serif`FormHeaderu#K׊` DetailA4fM7_Zm7U`taxbdceIDIDi;O: aVd235U9`<axbcID LabelID:ID_Labelxx]GJ?J@JAJBJ2J3J4>J5>J6>J7JJJJ JJ JJJJ!J"J#\WINDOWS\SYSTEM\StdOle2.Tlb#OLE Automation*\G{00000201-0000-0010-8000-00AA006D2EA4}#2.1#0#C:\PROGRAM FILES\COMMON FILES\SYSTEM\ADO\msado21.tlb#Microsoft ActiveX Data Objects 2.1 Libraryn= Ux 4%AccessVBAWin16~Win32MacVBA6#Project1 stdole`3Case Study IVR Network Test Change Management (new)EADODBs< 0=Babc+e ghij3kg &;@b DID>j{N Ut1+IM8{Aa3@ChangesDC&yt1+IM8{IDgYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date Noted{nU5uB&f Et1+IM8{Impact Date̍ 'L}L;6t1+IM8{Change TypexWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description5N7:K& t1+IM8{Recovery Plan)>*@%0t1+IM8{Cross-References :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/Savings@ dLpÄtt1+IM8{Test Impactoc@qѻmt1+IM8{StatusChanges,Changes Affecting Test Ariald2e12f5g237hghMS Sans Serifi1ghj1ghk1l23:m4n2o2p5r35zijMS Sans Serif`FormHeaderu#K׊` DetailA4fM7_Zm7U`taxbdceIDIDi;O: aVd235U9`<axbcID LabelID:ID_Labelxx]Gj{N Ut1+IM8{Aa3@ChangesDC&yt1+IM8{IDgYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date Noted{nU5uB&f Et1+IM8{Impact Date̍ 'L}L;6t1+IM8{Change TypexWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description5N7:K& t1+IM8{Recovery Plan)>*@%0t1+IM8{Cross-References :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/Savings@ dLpÄtt1+IM8{Test Impactoc@qѻmt1+IM8{StatusChanges,Changes Affecting Test Ariald2e12f5g237hghMS Sans Serifi1ghj1ghk1l23:m4n2o2p5r35zijMS Sans Serif`FormHeaderu#K׊` DetailA4fM7_Zm7U`taxbdceIDIDi;O: aVd235U9`<axbcID LabelID:ID_Labelxx]G?@ABCHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghi0000 0* pHd3Case Study IVR Network Test Change Management (new)l@  =    n=( %<9stdole>stdole f%\*\G{00020430-;C 0046}#2.0#0#C:\WINDOWS\SYSTEM\StdOle2.Tlb#OLE Automation/gADODB> ADOBC CD1C10-8CAA006D2(EA4C1CPROGRAM FILES\COMMONL9\msado21.tNMicrosoft Acct Date Labelo Change_TypeChange Typed Change_Type_LabelChange Type Labelm Change_DescriptionChange Descriptiond Change_Description_LabelChange Description LabelmImpact_DescriptionImpact Descriptiond Impact_Description_LabelImpact Description LabelmRecovery_PlanRecovery Pland Recovery_Plan_LabelRecovery Plan LabelmText18d Label19mText20d Label21mText22d Label23mText24d Label25ijMSysDbID="{56898E9D-2DC0-4C3F-B649-8EB24A54D010}" Name="Case Study IVR Network Test Change Management (new)" HelpContextID="0" VersionCompatible32="393222000" CMG="4240A6E369E769E769E769E7" DPB="090BEDEEEEEEEEEE" GC="D0D234B735B73548" [Host Extender Info] &H00000001={3832D640-CF90-11CF-8E43-00A0C911005A};VBE;&H0000mText26d Label27mText28d Label29FormFooterCross-References Schedule Impact!Resource Impact"Other Costs/Savings#Test Impact$StatusChanges   BGƎ\t1+IM8{Impact Description :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/SavingsAJQ5-CG/Change TypeX !ORlK5-CG/Schedule Impact\+'^G"5-CG/Resource Impact(jWد-KVenF`5-CG/Other Costs/SavingsChangesChanges Arial82JhHP OfficeJet LX@f,,@MSUD$HP OfficeJet LXdT!winspoolHP OfficeJet LX\\PORSCHE\OfficeJetd278d eTimes New Romane24fg37jghm5C Arialn= Arialo3A Arialp4Change TypeImpact Date`ReportHeader|k(B~ cd2`<a<bcdLabel41Changes@ mHG^f2a<b2 Line44o5nAnZ(qC`PageHeader!^4@X_4@MSysAccessObjects$DDDDDDDDDDB s10@Aa3@Changes$M>^<<<0000000. @ B110@210@~sq_rChanges$4MR2KeepLocal Tzzz:::::::8 @ 3@w3@~sq_fChanges$T 4MR2KeepLocal TF::::::8 ` w10@w10@Changes$(@L<<<0000000. @ {u10@{u10@Changes$<@L<<<0000000. @ w24@w24@UserDefined$@LDDD88888886 @ *.r10@*4@SummaryInfo$ @LDDD88888886 @ J710@To@AccessLayout$4MR2KeepLocal T@Kzz:::::::8 @/r10@/r10@SysRel$.........., /r10@/r10@Scripts$0000000000. /r10@/r10@Reports$0000000000. /r10@/r10@Modules$0000000000. /r10@/r10@Forms$,,,,,,,,,,* /r10@/r10@DataAccessPages$@@@@@@@@@@> yL-r10@yL-r10@MSysRelationships%DDDDDDDDDDB yL-r10@yL-r10@MSysQueries%88888888886 yL-r10@yL-r10@MSysACEs%22222222220 yL-r10@yL-r10@MSysObjects%88888888886 yL-r10@yL-r10@MSysDb$\@L:::......., @yL-r10@yL-r10@Relationships%<<<<<<<<<<: yL-r10@yL-r10@Databases%44444444442 yL-r10@yL-r10@Tables%.........., z\> o0cefnK< ON:Project NameProject NameValue ListL"Godzilla Phase I";"Godzilla Phase II"RName of project on which change occurred.Project_Namem;` bHckDate NotedDate Notedm/d/yyyyRDate on which the change was first noted.Date_NotedcJLhm`bP ck$Change Description$Change Description>>>>>>>>>> X_4@X_4@MSysRelationships%DDDDDDDDDDB X_4@X_4@MSysQueries%88888888886 X_4@X_4@MSysACEs%22222222220 X_4@X_4@MSysObjects%88888888886 X_4@X_4@MSysDb$.........., X_4@X_4@Relationships%<<<<<<<<<<: X_4@X_4@Databases%44444444442 X_4@X_4@Tables%.........., ; v1b N  : k        osts_Savingsq}2Cxr!W@m 0`-atbck 4Max Of Other Costs/Savings6=Max([Other Costs/Savings])*$#,##0.00;($#,##0.00)4Max_Of_Other_Costs_SavingsT'u%DpPd^v`PageFooterAׄOLm ;<`<abc,i j Text42 =Now()Long DateTimes New RomanL)]Jzͱ:m ;<`abc,i jk Text43H="Page " & [Page] & " of " & [Pages]Times New Roman$@*Ϗ O.WG+-\`hReportFooter<I3 m 5<`4&bc,i >Schedule Impact Grand Total Sum.=Sum([Schedule Impact])Times New Roman>Schedule_Impact_Grand_Total_SumM-j]C胨4d`<bGcd Label24Grand Total9 @Y& HLm 5<`)bc,i k>Resource Impact Grand Total Sum.=Sum([Resource Impact])Times New Roman>Resource_Impact_Grand_Total_Sum*`5JLY@?d`<bGcd Label30Grand TotalvEm(/Ny'm 05<`-bc,i kFOther Costs/Savings Grand Total Sum6=Sum([Other Costs/Savings])*$#,##0.00;($#,##0.00)Times New RomanFOther_Costs_Savings_Grand_Total_Sum<7X$KD1Iu&d`<bGcd Label36Grand Total7M˘) ted_LabelDate Noted LabeldChange_Description_LabelChange Description Labeld Impact_Description_LabelImpact Description Labeld Schedul @ @MB JA !LVAL f fn  L Nr|Introduce version control and configuration management into the entire project.Get a template for the letter and get an extension.Future performance testing--and the test tools--will need to avoid using self-generation of load.We need better test environIntroduce version control and configuration management into the entire project.Get a template for the letter and get an extension.Future performance testing--and the test tools--will need to avoid using self-generation of load.We need better test environment support.The key player's manager is now on the hook to provide this.Need to talk to project management about this.We just have to get the plan solid, but I'm not sure how Jenna and I can make this happen.None possible, other than trying to get the other pieces in ASAP and finish up in less than three weeks. Unlikely.We need to sit down immediately to put a plan in place for this. I have escalated this to project management.I'm trying to convince Lou to translate it.I have escalated this to Jack today, and have left messages for Milos to resolve it.Try to catch up in the (already-overloaded) BB[2].Try to compress three weeks of testing into one week!None, but checking to see if this changes.Use the CDR server as the cooperating host and use the baby-alpha system as the IVR on EPCOT until Production Two shows up.None needed if the impact is small.We will slip BB[0] to 7/20, then: * move the 2.6/2.11 touchpoint (the PBX/CT Connect basic functions) into BB[0]; * move the 2.4/2.5 touchpoint (login & updated user cache) into the third integration test cycle, BB[2]; * collapse BB[1] into a zero-duration milestone as a reminder of what happened. This causes us to get right back onto schedule with BB[2]. Slick, if it works.None possible. Can't test w/o the SUT!None possible, unless SWint pulls a telephony rabbit out of their hat in the next day or two. We've already lost at least a day.LVAL ~x 6 P Ȱ:1. Bob received a list of tasks necessary to getting Vindaloo up and running on Friday. He left and took t1. Bob received a list of tasks necessary to getting Vindaloo up and running on Friday. He left and took the weekend off, in spite of the criticality. 2. Lisa Burns, the other person, is out for a week or so. 3. Bob can't set up a second server to help us run and debug the network load scripts. 4. Bob, according to Rotem, knows "more about Unix that us (Test), but just barely." This means that his learning curve is on our critical path. 5. Bob has lots of other work, so, when we need urgent support, we have to queue up for it. 6. When we ask Bob for help, someone else--Jenna, Phil, etc.--often ends up on the critical path for permission purposes. This can add days to the response time on a trivial, fifteen-minute task. I would guess that, calendar-wise, this has impacted the network load tests by a week. The lack of urgenHitesh is trying to understand this right now.It will take Lisa about ten days to get this problem fixed.The system may not show up on 7/13 now.This would have caused a one-month slip to ripple through the whole schedule. However, Jack and I have come up with a recovery plan, that, if it works, saves our bacon.Production One will ship Monday or Tuesday. Assuming it ships 7/7, we can expect arrival at 905 on or around 7/13.Day-for-day slip of Stress/Capacity Demo and System TestThe stress and capacity demo, and thus Platform System Test, are slipping day-for-day until this arrives.Stress/Capacity Demonstration starts three weeks late, delaying System Test as well. This also creates pSimonlems in combination with change ID 100--no remote debugging of the scripts--because now Rotem must debug these on Beta. So, we can't send the Burn-in tests to Lily until, at the earliest, 6/29, which delays his shipment of the first two production boxes to us for System Test.At least a week added to the stress/capacity test execution time.`LVAL ж ^ dI may lose one or more test engineer in December.The Vindaloo finally hit the plate on 7Test environment support was an issue again, this time in terms of diagnosing problems with a CD-ROM on JalfreeziI may lose one or more test engineer in December.The Vindaloo finally hit the plate on 7/15, but we are still working on getting it wired and powered up.I'm not really clear on what's happening here, but Production One is still not shipped. Lily tells me she needs it as a reference box for Production Two, but theoretically Beta would do that.Due to a staffing issue in 2.4, the first integration test cycle, BB[0] could not start on 6/22. It slipped out to 7/20 or so.The plan for stress and capacity testing now has changed. HWint will ship us Production One (code name Vindaloo) for the Stress and Capacity test, due to the differences between Beta and the Production systems. (RAID versus HD boot, no terminal server hardware on Beta, etc.) This is the right thing to do, but did introduce another hold up.The latest hold-up on sending Beta has to do with the need to test out the 16 MB telephony cards. I'm still trying to figure out when Beta will actually be in a crate, on a truck, headed for our test lab.Did I say 6/22 arrival for Beta in the last entry? I should know better than to have such optimism. Now, with the Customs angle allegedly under control, Lily has put Beta back together, and, guess what? It doesn't boot. Neither do the two production systems. We have no ETA. This is very scary.The original plan was to deliver Beta for the Stress/Capacity Demonstration around 6/1. Now, the system will not arrive until 6/22 at the earliest. This arose from two primary causes: 1. TestLab slipped environmental testing out a week to complete work on another client's hardware. 2. Client changed its customs broker during this period, and we had to get the new process in place before we could send the system.Ht  P d ^ JVOyz*0O@`@Integration@Dependency@ None needed.@ll``L@*7xu@dIntegration@DeliverableAn increase in project risk.9.9.xNone possible.b@RbbL@*?oN1j@`@Integration@Dependency@@9/3: ended up not happening.xll``L@*0n@@@Allb@Deliverable@None possible.B@Qz^^RR<0*7m2TvX@2TvX@>@Platform (2.2):@Dependency~@:All@2 @QrffRF*;lCpH@@All@Test AdditionUnclear at this time.F@D@yyVV<0*7k2@@@Platform (2.2)@Late Deliverable@:2.2.xNone possible.~rrRF*j|8@@Platform (2.2)@Late DeliverableN@None possible.See subsequent changes.~~rrRF*0iV?8@@Integration@Late DeliverableR@9.9.x@xllL@*h:-P@@&@Platform (2.2)@Late Deliverable@2.2.xNone possible.~rrRF*gm@@Platform (2.2)@Late Deliverablep@!None possible--no SUT, no test.~~rrRF*f@@ @Platform (2.2)X@Late Deliverable@N@~~rrRF*e*0@`@5@Platform (2.2)J@Late DeliverableN/A@2.2.xNone possible.rRF*dS<@@@@Platform (2.2)@8DependencyN/A@2.2.X@wfRF* LVAL  6   " 0827: Waiting for 0831 delivery, allegedly. 9/3: Nope. 9/18: See change record 126.0827: Nope, no translation. They will run this, not us.9/3: Beta was never removed, but it was taken down for this training, which didn't impact us.9/18: Nope. PSimonably won't make it. 1/15: The whole risk-based backbone test effort was cancelled. See a previous record.9/3: Still open as to whether we should test this. 9/18: Still open. 10/2: Definitively deleted.0827: This issue remains open. 0921: A snapshot: Per Rotem, "System administration support on the IVR boxes is not yet at a level that makes me feel confortable (last week I discovered that Bob has pSimonlems with .tar.gz files, and Samba support is something that David kludged up by screaming for help). Hardware support in the following areas: o Console support for clusters of IVR systems. I can track at least two Vindaloo run arounds that would have been avoided if we would have had a console hanging of the machine. o The Call Centre NT workstations are not here yet. James and I will pSimonably need a couple of days to prepare those systems for testing. The sooner we get them, the better." 10/8: PSimonably the single biggest brake-pedal f9/19/18: Jack is now handled, but we'll see how it works out. 1/15: Ultimately no problems, but we probably lost about a day.9/18: Nope. Probably won't make it. 1/15: The whole risk-based backbone test effort was cancelled. See a previous record.0827: Waiting for 0831 delivery, allegedly. 9/3: Nope. 9/18: See change record 126.0827: Nope, no translation. They will run this, not us.9/3: Beta was never removed, but it was taken down for this training, which didn't impact us.9/3: Still open as to whether we should test this. 9/18: Still open. 10/2: Definitively deleted.10/8: Not clear where this stands... 1/15: Never happened. 3/19: At some point, the Montreal Call Center will happen, and the testing will happen at that time.LVAL  :The 2.6/2.11 touchpoint for the first cycle of integration testing, BB[0], never showed up.None of the third integration cycle BB[2] items were The 2.6/2.11 touchpoint for the first cycle of integration testing, BB[0], never showed up.SWint was unable to spend time keeping 5A/5B in sync with the telephony driver and telephony API. As a result, we now find that 5A/5B--and the stress/capacity telephony test suite based on them--do not run.We needed SWint's help debugging the Telephony Stress and Capacity suite, since they had the only working IVR server besides the development server here. We sent it to them in late May. (See pages 110 and 111 in log book, and change mgmt report 1.) By early June it became clear that they would not help us, primarily because they were overcommitted already and being forced to take on additional duties.Apparently, James and Frank are going to arrange for more, and better-trained, test environment support staff.I'm not clear that this was ever *on* test's plate, but cut-over/transition is--for now--a marketing/call center issue.Security testing was *removed* from Test's scope.Because of the amount of work required, we have pushed the IVR Telephony testing back on TeleVendor. We will audit their work.The 2.11 piece of the 2.6/2.11 touchpoint will not make the first integration cycle BB[0] cutoff, and will instead land in the third integration cycle BB[2]. This means that our recovery plan for the first integration cycle BB0] slip didn't work out.We are going to lose Beta to SWint so they can wrap up their telephony API development and testing.We are not getting sufficient test environment support for the Platform System Test, and the support we receive does not have sufficient skills or authority.It looks like we may have to do localization testing, in terms of French support, for all the applications that may end up running in Montreal.f LVALbξ 11/5: It ended up showing up at 5:00 PM.10/22: See issue 149. PRI ISDN is now the only one. 11/5: this has ongoing consequences (see change record 155).10/8: The whole debate continues to play out. 10/22: Resolved. We retain the use of the server for testing.9/18: Jack is now handled, but we'll see how it works out. 1/15: Ultimately no pSimonlems, but we pSimonably lost about a day.0827: We filed a bug against the tiny partition issue, because this will happen in the field, too.9/3: Continues to impose significant delays and ove11/5: It ended up showing up at 5:00 PM.10/22: See issue 149. PRI ISDN is now the only one. 11/5: this has ongoing consequences (see change record 155).10/8: The whole debate continues to play out. 10/22: Resolved. We retain the use of the server for testing.9/18: Jack is now handled, but we'll see how it works out. 1/15: Ultimately no pSimonlems, but we pSimonably lost about a day.0827: We filed a bug against the tiny partition issue, because this will happen in the field, too.9/3: Continues to impose significant delays and overheads. 9/18: See ID 126.9/18: We're about to wrap up the third integration cycle BB[2] and the 2.11 components are still not here. 10/8: BB[2] is over. No single 2.11 component has yet entered pre-integration test. 11/20: The risk-based backbone integration has be11/5: It ended up showing up at 5:00 PM.10/22: See issue 149. PRI ISDN is now the only one. 11/5: this has ongoing conseque11/5: It ended up showing up at 5:00 PM.10/22: See issue 149. PRI ISDN is now the only one. 11/5: this has ongoing consequences (see change record 155).10/8: The whole debate continues to play out. 10/22: Resolved. We retain the use of the server for testing.0827: We filed a bug against the tiny partition issue, because this will happen in the field, too.9/3: Continues to impose significant delays and overheads. 9/18: See ID 126.hLVAL<n $ ȞZBThis has the inevitable schedule-delay impacts--I anticipate This has the inevitable schedule-delay impacts--I anticipate that we won't get this worked out for at least a week, delaying Shaheen's access to the box for test development. In addition, I worry about the message this kind this miscommunication sends to a team of individual contributors all of whom are working overtime.If we accept it, we lay the foundations for a Tower of Babel in the test system architecture. If we don't, we lose the ability to test the telephony API. Strictly speaking, this testing belongs ot SWint anyhow.We are losing the 2.11 risk-mitigation from the risk-based backbone testing, as well as increasing risk in 2.11 proper and in 2.6 proper. (2.6 is tested primarily through integration testing.)None, really. Having Beta out of commission for a couple days doesn't matter, because we're in the post-BB[0]/pre-BB[2] (unplanned) hiatus.No risk-mitigation here for BB[0].Could delay BB[2] exit by up to two weeks.The impact came around 7/20, when, with Vindaloo set up, we found our tests in need of significant debugging. The Telephony test especially needs a lot of help. We are about two weeks behind schedule now (8/3) on script debugging because of this.This is very good. We need this.This is a good thing, because, if the plan had assumed I would handle this, it would involve my whole staff for a month. Just the data transition is going to be very tricky.Saves us some work, but a pretty high-risk move in my opinion.This saves us a tremendous amount of person*days of work.If we get the CDR server set up as the cooperating host for stress and capacity, the impact should be minimal. If not, we will take a serious hit. Also, integration may get hurt if the baby-alpha system gets on the critical path.This could impact our ability to deliver a maintainable, integrated test system after the project.4LVAL D L1/16: When all was said and done, we probaby lost a of schedule week and a half-person-month of effort.10/8: An on-going issue. Support team learning curve continually costs my team--especially Rotem--productive time. 1/15: See change record 109.9/3: I'm not clear that this has improved. 9/21: See change ID 10. 10/22: Bob is apparently getting better. 1/15: See previously reports for total impact, but quite serious. 3/19: As of the end of the project, the improvements that were needed did not materialize. The issue is not so much the test environment support staff people, but rather a project-wide underestimation of the complexity (technical and logistical) of what we're about. It may be that my expectations are too high, also, and that I should have built more slack and resilience into my estimates.9/3: Ooops, now it looks like we will have some sort of p1/16: When all was said and done, we probaby lost a of schedule week and a half-person-month of effort.10/8: An on-going issue. Support team learning curve continually costs my team--especially Rotem--productive time. 1/15: See change record 109.9/3: I'm not clear that this has improved. 9/21: See change ID 10. 10/22: Bob is apparently getting better. 1/15: See previously reports for total impact, but quite serious. 3/19: As of the end of the project, the improvements that were needed did not materialize. The issue is not so much the test environment support staff people, but rather a project-wide underestimation of the complexity (technical and logistical) of what we're about. It may be that my expectations are too high, also, and that I should have built more slack and resilience into my estimates.9/3: Ooops, now it looks like we will have some sort of participation. 9/18: We have to test data conversion. 10/8: Situation unclear. 10/22: Data transition in System Test for IVR App, CSA, and CDR. 1/15: No direct involvement. 3/19: We did end up doing some work in this area.zLVALЦ 9/18: This is apparently now handled. 3/19. But it wasn't handled in September. This pSimonlem cropped up again in Integration Test and must have caused at least a dozen or so outages of an hour or two each.9/3: We should have planned originally to write all our own test tools, because this created a dependency that, on top of all the other dependencies,exposed us to way too much schedule risk. 1/15: Impact assessment captured in change record 117.9/3: I'm not clear where this is going, and I need to spend time with Rotem to w9/18: This is apparently now handled. 3/19. But it wasn't handled in September. This pSimonlem cropped up again in Integration Test and must have caused at least a dozen or so outages of an hour or two each.9/3: We9/18: This is apparently now handled. 3/19. But it wasn't handled in September. This pSimonlem cropped up again in Integration Test and must have caused at least a dozen or so outages of an hour or two each.9/3: We should have planned originally to write all our own test tools, because this created a dependency that, on top of all the other dependencies,exposed us to way too much schedule risk. 1/15: Impact assessment captured in change record 117.9/3: I'm not clear where this is going, and I need to spend time with Rotem to work on this. 9/18: Remains critical and totally unresolved. Milos is unable to commit to dates for detailed specs. 10/8: 2.11 remains outside of BB testing, the risks totally unmitigated.9/3: We really need to revisit the plan/schedule for all the risk-based backbones. 9/18: Reschedule of BBs is critical, but now dropping the BBs is on the table. 10/8: BB[2] has exitted, the dropping of the backbone testing has occurred, but we have little to show for the resources invested. 3/19: The omission of the backbone testing lead to a large number of hard-to-isolate yet avoidable pSimonlems in Integration Test.LVAL 2X \ ~ FTIBCO licenses expire frequently. It was hourly, then daily, now it's weekly.The new story is that the telephony test suite will be done by 8/31. Per Hitesh's e-mail: "We will complete the 5A/5B Benchmarks and the telephony API by Aug. 31st. Formal System Test can begin with the telephony infrastructure on that date."The root partition on Vindaloo became full, due to it's tiny (190 MB) size. We needed to get our accounts moved over to /usr/home (from /export/home). Test environment support was an insurmountable issue in the time frame. Ultimately I had to write a script and do it myself.We have a hardware allocation plan, of sorts, but I'm not convinced that we can make it work. It's not clear to me how we manage the logistics of all this stuff.The 2.4 piece for BB[2] just showed up today, which was originally the day to exit BB[2].The 5A/5B situation remains without any commitment to complete on SWint's part.SWint is now slipping 5A/5B day-for-day. They are overbooked with other commitments.SWint did not deliver the repaired 5A/5B benchmark on Friday as planned.The processor board HWint received for Production Seven is apparently fubared. It will take a while to fix it.The lack of a unified hardware allocation/configuration/transition plan continues to generate urgent goat rodeos. Today, I discovered that both Linda and I are planning to use the CSA Server. It's double booked, just like Beta Bob two weeks ago. Also, we have no updated Production IVR delivery schedule.SWint intends to deliver a test script for telephony API in perl, not TCL.Starting with BB[0] and continuing into BB[2], 2.11 is not delivering its touchpoints with 2.6, 2.4, and 2.5.The latest plan for Beta is that it will not return to SWint, but will remain here. Larry Abraham will come up from HWint the week of 8/10 to make this "production ready" and to train the network team in the operation of the unit.t+i  J 2 <8b(|ZEٜ@@@(Platform (2.2)2@ Late Deliverable@@#@"~~rrRF*3(g@@@Integration@ Late Deliverablef@@z@!xxllL@*2i@@?CSA (2.5)`@ Late Deliverable@We need written specs.Z@!tthhH<*3q2I6@@@>IVR App & Tele (2.4)F@ Test Deletion@ None needed.J@Szz`T*30b2@ @@Integration@ DeliverableT@ None possible.@!nnbbL@*3Q1@@?Platform (2.2)@ DependencyH@ None possible.@!rrffRF*3QΠ@@?All$@ Dependencyz@ f@@th\\PP<0*3 @ @IVR App & Tele (2.4)@ Test Deletion"SWint must run 5A/5B themselves.@`@Szz`T*3w: @@@Platform (2.2)@ Dependencyf@ P@(@~rrffRF*0[0 @ @@ Integration@ Dependencyh@:We need to get new licenses.@Sll``L@*34/@@Platform (2.2)@ Late Deliverablej@x@@~~rrRF*0ϊF@ϊF@Platform (2.2)@ Late Deliverabler@None possible.n@~~rrRF*7'T@@@Platform (2.2)*@ Dependency@\@@~rrffRF*7bT@@AllD@ Schedule Shortened@@ @xll``<0*0W@W@Integration@DeliverableThree week slip for BB[2].@ @bbL@*0Y܉@`@Platform (2.2)@Dependency@None possible.@rrffRF*7@@@Platform (2.2)@Dependency@None possible.@rrffRF*0~+@@Platform (2.2)@Late Deliverable@2.2.xNone possible.@~rrRF*?}[E@@Platform (2.2)@Test Change/Deletion@No SUT, no test.H@zzRF*7|@@@$@ Allf@Dependency@ @ @th\\PP<0*7{ᐖm@ᐖm@IVR App & Tele (2.4)@Test Change/Deletion@ V@ r@`T*0zm@@Content Management (2.11)@Late Deliverable@ 9.9.x@ "@h\*8LLVALH (f0827: Looking for 0831. 9/18: See ID 126. The total hit was actually about four months when all was said and done. 1/15: I have rolled all these impacts up in change record 117.0827: Still working this out, but a single test environment is the plan. 9/18: I have finished the hardware allocation plan. 10/8: The hardware plan is in place, but we may have trouble meeting dates. 10/22: The plan is solid, but logistics remain an issue. 11/20: Overloading of the network support team in general and the complexity of the logistics leads to on-going delays in the delivery and configuration of hardware. 1/16: See issue 124 for an impact assessment.9/18: Testing almost complete now. 10/8: BB[2] exitted last week, only partially run. 1/16: See earlier reports for an estimate of the impact of what happened to the risk-based backbone testing.0827: Allegedly to deliver on 0831. We'll see. 9/3: Nope. 9/18: 5A/5B out of the loop. We have written our own babyivr/caller program pair. 10/8: See previous notes for the pSimonlems associated with telephony software testing.827: Actually, Prod 7 is a single CPU system. This has no impact. 10/2: Now I'm hearing that multiple CPUs are completely off the plate. 1/15: Never an impact.0827: We are now going with a single EPCOT test environment for both System and Integration Testing. This should make the logistics of setting up the network much easier. 9/3: Planning this involves creating an Access database to understand the moving parts. I'm working on it right now. 9/21: The test hardware plan is out. Once we get availability dates worked out for hardware, I'll move on to logistics. 10/8: I have put together the plan. Now it remains to Jenna to execute. 10/22: Plan is solid. Need to put the people-pieces in place today. 3/19: Hardware availability and logistics remained a significant hurdle throughout, but would have been an insurmountable obstacle without the hardware planning database.LVAL ^ p B r \x&We have offered the project manager half a loaf, which is switching off, one week to the next, the evening shift on the system. This puts Hitesh in a difficult situation, but it's pWe have offered the project manager half a loaf, which is switching off, one week to the next, the evening shift on the system. This puts Hitesh in a difficult situation, but it's pSimonably the best we can do.The test team has spent at least 40 person-hours so far reproducing and isolation failed boards, then installing the replacement boards. We have also engaged a consultant to do root cause analysis on the failures.Jennifer lost about a day, total, on two incidents.As an example, Hitesh lost a whole day's worth of test development because the verbal communication about how something was supposed to work turned out to be wrong. Hitesh needs a written oracle, not verbal comments from a developer.This will free David up to work with Jack on developing IVR application system tests.This reset the tests and cost us two days.Another day of time and effort lost.One test engineer may not be able to come up here after 9/30.This confuses the test results of a system lock-up.We don't even know yet. Obviously, a serious impact.We have now suffered a two month hit, which is significant even in the context of a two year project. Had Beta arrived on schedule in early June, we would have taken a three month hit.The test techs lost about three or four hours of run time, and a weekend test run was aborted.We could get blind-sided by missing hardware at random times, leaving testing dead in the water for days or even weeks.We have no idea when about 50% of the Platform System Test Suite will become unblocked.Day-for-day slip on System Test, Stress/Capacity, MTBF and Burn-inSlipping day-for-day on Stress/Capacity and Platform System Test.Production Seven is U/A for 9/7, which slips the second cycle of the environmental test out by at least a couple weeks.jLVAL nnξ t.2 There is some question whether we will receive all the T1 lines we need.Failure to anticipate hardware needs in oneThere is some question whether we will receive all the T1 lines we need.Failure to anticipate hardware needs in one project may cost the test team one server from the test environment.These telephony board failures are creating serious inefficiencies. They claim an MTBF of eight years, but we are seeing one board fail every few weeks.The CSA team was not keeping track of the versions of the various communicating components, so we had no way of predicting in advance whether a collection of software loaded on the agent desktop would work.Dolores has not delivered detailed specs on CSA.Due to lack of test envronment support, a confusion about work environments, and other logistical details, we are disengaging from work with the IVR 2.4 test team.Dolores delivered 2.4/2.5 software in the middle of BB[2] testing.John made no forward progress on his test development last night because: 1) SWint didn't return support calls; and, 2) Jeremy was mistaken when he said that the telephony cards had an internal loopback mode.There is no process for getting employment visa extensions for people, and some of the team is having trouble getting their work permits extended.SWint re-engineering of 5A/5B in Perl makes it not runnable by my team. They will own it. Also, they have apparently moved the timing loop from the external NT box to the SUT. This is called self-generation of load, which is a serious no-no for performance testing (per Beizer's discussion in Software System Testing and Quality Assurance). The measured data will be distorted by the load generators. This loss of the NT box could also affect other performance testing.SWint missed the new 5A/5B date. It turns out that a key player on their team is leaving. The test is incomplete. What is done is written in Perl, which is incompatible with our TCL test environment.LVAL^ v 01/16: We have a cool new tool, CSiTCL, so good came out of this. However, it was painful at the time.10/8: As an end note, the time Jennifer lost was the only time remaining, after development of BB[2] tests, for their execution. So, we ran one (out of a dozen or so) tests total for BB[2]. 3/19: Ultimately, the test team had to define--in the test plans--the version control and configuration management processes9/18: Remains open. 10/2: Finally receive the 3.2 download last week. Hitesh is off and running. 1/16: Yes, but the lack of frozen meta1/16: We have a cool new tool, CSiTCL, so good came out of this. However, it was painful at the time.10/8: As an end note, the time Jennifer lost was the only time remaining, after development of BB[2] tests, for their execution. So, we ran one (out of a dozen or so) tests total for BB[2]. 3/1/16: We have a cool new tool, CSiTCL, so good came out of this. However, it was painful at the time.10/8: As an end note, the time Jennifer lost was the only time remaining, after development of BB[2] tests, for their execution. So, we ran one (out of a dozen or so) tests total for BB[2]. 3/19: Ultimately, the test team had to define--in the test plans--the version control and configuration management processes9/18: Remains open. 10/2: Finally receive the 3.2 download last week. Hitesh is off and running. 1/16: Yes, but the lack of frozen metadata is still impeding progress.9/18: Testing here almost complete. 10/8: This also contributed to reduction in scope of the third cycle of integration testing BB[2]. 1/15: In the grand scheme of things, time lost on a given backbone is insignificant compared to time lost overall when the risk-based backone tesitng got scrubbed. See change record 112.9/18: Celia's one and a half day on-site seems to have helped. 10/8: John turned out to be not entirely competent to the task. Jason and David, however, were very competent. 1/16: See change record 117..LVALP > L11/23: It looks like VME bus damage only at this point. 1/16: Ultimately, we lost about three schedule days, and ten person-days due to wasted time trying to figure out what happened.1/16: Jennifer's lost time on this became an issue later in that it delayed Platform test (which was also cancelled). Together, this rippled into Formal Integration Test and contributed to the automated testing (the fake pipe piece) not being done on time.11/20: This apparently is a continuing problem. Beta is still not talking to the telephony network, and Kefetew is spending time isolating the problem with Lee.11/5: I should have a date for delivery tomorrow. 11/20: Finally here, but status unclear. Jennifer can test when he gets back.11/20: Here's a nice wrinkle. System Test starts on 11/23 and we have only one PRI ISDN line available for test. What is the sound of one line winking?10/22: Need to get Purify here for testing the demon for memory leaks. 11/5: We reengineered the test program. 11/20: We anticipate the load tests will fail. 1/16: This became the crisis-du-jour that turned into the crisis-du-project, because the telephony servers never could scale to the design-specification number of calls10/22: I believe we will get the PRI ISDN lines we need (see change records 147& 149). 11/5: This is still unclear. 11/20: Insufficient T1 lines are wasting time and delaying test development. We may well not be ready Telephony System Test on 11/23, in no small part because of a scarcity of T1 lines. 3/19: Ultimately, this was just another facet of the complex hardware procurement, configuration, and support logistics of the project.10/8: The saga continues. We have to investigate this further. 10/15: We may have had another die today. 10/22: Need to follow upwith the vendor on status today. 11/5: The vendor remains uncertain as to why we're seeing all these failures. 11/20: Unreliable hardware, especially these boards, remains remains a source of inefficiency.LVALb : R(>$0Must fix the fanout script to resume testing.Again, configuration management and release/version control is the weak link.Configuration management and version control--hand-offs to testing, basically--need a more robust process.Again, this relocation would have been less painful with a more robust plan, more carefully executed.The move appears unavoidable, but better planning could have allowed us to do it during a lull in testing.Jason, David, and, most of all, Jack are to figure out what's going on with this tomorrow, establish checkpoints, and put a plan in place to resolve this.Implement the configuration management tools for CSA ASAP.We really need either a smoke test or some other clear entry criteria that are abided to in advanced to prevent this.Institute a higher caliber of system administration support.None possible. Delete tests that rely on it from ST, move to formal integration testing.None possible. The quality of the CSA, CM, and AM tests will suffer.May have to come back to get the stress/capacity tests running before the operational test.None possible. Revise the plan, soldier forth.None possible. Retest with PRI ISDN.Fix the demon, live with testing only 400 calls, or reengineer the test program.Someone must make a decision immediately, and that decision must be communicated clearly, upwards (to Joe/Larry), sideways (Susie/PLs), and downwards (into Test). The time lost is unrecoverable. Also, if the CDR server goes away, and the CSA server goes away, we lose a week of CSA development time, which is also unrecoverable. In general, this kind of fuzzy, jerky management will cause slips on our tight schedule. 10/22: We have CDR now.If Susie won't go for that, then the test team will need to disengage from SOB System Test.Steve Malmstrom needs to help the telephony board manufacturer get to a stable point in their manufacturing process.'j1R b j -0BteY@Y@Network (2.3)@(DeliverableJ@'None needed.@)rrffPD*0qY@qY@Allt@(Quality Problem@'!The genie is out of the bottle.ffZZ<0*lY@@,@(IVR App & Tele (2.4)r@(Quality Problem@'4@ #@!~~`T*3l@l@@CSA (2.5)\@(DeliverableJ@'None possible.jj^^H<*t@t@@CSA (2.5)j@(Deliverable@%t@ #4@)vjj^^H<*3@@?CSA (2.5)@(Dependencyp@%None possible.hh\\H<*+G@+G@@Platform (2.2)@@(DeliverableN@'None possible.@)tthhRF*3!N&@!N&@?Platform (2.2)@(Quality Problem@%@ #||ppRF*@@@ Platform (2.2)\@ &Quality Problem@'r@"|||ppRF*#/@/@,@Integration@ &Test Deletion@ %Time machine?@"rrffL@*3Πȅ١@Πȅ١@Platform (2.2)@ &Quality Problem@ %None possible.||ppRF*١@١@Platform (2.2),@ &Dependency@ %x@ #D@"~rrffRF*2@@Platform (2.2)x@&Deliverableh@ %Need to understand issue.tthhRF*r@r@CSA (2.5)x@&Deliverable\@ @#vvjj^^H<*9@9@@All@&Late Deliverable(@%@#@thh\\<0*3Uk@Uk@?Platform (2.2)@&Deliverablef@%@#R@tthhRF*3zj@zj@IVR App & Tele (2.4)D@&Deliverable@%None possible.@"vv`T*0ot?U@ot?U@?Platform (2.2)>@&Test Deletion@%^@#xxllRF*4/Q@4/Q@IVR App & Tele (2.4)D@&Test Addition@%J@#4@"zz`T*0NW@NW@@(Platform (2.2)@&Deliverable@%@#@"tthhRF*3V?W@V?W@Platform (2.2)PRI ISDN is now in scope.Test Addition@%Get hardware, scope changes.@mF*0@Pٕ@@Pٕ@All@ Dependency@%Get those T1 lines installed.x@"\\PP<0*01u}@1u}@@SOB (2.8)\@&Test Deletion@%@#zznnbbH<*.*ڜ@.*ڜ@SOB (2.8)@ Dependency@@#@thh\\H<*0LVAL*F . Ŏ zUp to a two day hit (schedule) is possible wheUp to a two day hit (schedule) is possible when bad installation happen on the test environment.At least two person-days (Hitesh and Kate) fixing tests.We lost about four hours of Kefetew's time, a couple hours of Jennifer's, and an hour of mine.Since we didn't get any risk amelioration from the testing we did--no bugs were fixed--we did not achieve any ROI on all the time Jennifer and I spent on this. That's at least a person-month, plus whatever time Jason lost.I had a whole series of problems that cost me about three or four hours.Jack, Jennifer, and David probably lost a person-day between them.This could invalidate a significant number of tests.CM and AM have to move out to the new offices in 11/23. This creates problems because we may not be able to move CSA and CDR out there. This dislocates the test teams, which will create logistical problems. We also spent two person days (me and Hitesh) following up on this.I sat on my hands most of the day. Because we have to get the stress/capacity tests running on this box, we may not get it running in time for it to go into the chamber tomorrow.We probably won't be able to scale to the necessary number of calls.Probably about four hours spent in replanning and crisis management.The two or three days Jennifer and Kefetew spent testing T1 is now sunk time.Well, the caller/answer simulation programs don't work, because swap fills up before we hit 768 calls. This is delaying many of our platform tests and has probably cost Jennifer a good 40 person*hours.Unclear. Need additional hardware. Probably adds to test time.We may end up having to run a bunch of tests at the POP, which is not an eventuality the test team is staffed for.Hitesh, Jennifer, and Jack have lost 25 person*hours this week trying to rework the plan on this effort. I've burned another two or three hours of my own (copious free) time trying to get Linda and Simon to decide one way or another.LVAL $ ^ ^DThree significant miscommunications/failures to perform occured at HWLab on Environmental Test: 1) No phone line made available to us for remote supervision (by Jennifer) of tests while I was on-site. 2) Chamber mis-set for 40C (should have been 25C) and not caught until system was cooked. 3) Chamber turned off in response to Jennifer's request, but door not opened, causing chamber to reach 50C with the system operating.The risk-based backbone testing was dropped after about 20% of the effort was expended.Jennifer and Bob didn't adequate test out the media with the stress test before we took it to HWLab.Someone misconfigured the telephony card and telephony API library configuration when trying to change over the PRI ISDN. This left the system hosed.We may have to change the platform in terms of the WAN link.Apparently the RM link *won't* be available for System Test.The hardware management plan contained some bad assumptions about using the development PBX switch for System Test.Prod 7 was supposed to show up at HWLab this morning. It wasn't here at of 2:20.SWint committed to providing a stable, interim telephony driver for use in Test. However, configuratoin management issues are makin that impossible at this time.The day before the environmental testing was to start, the hardware allocation plan changed, causing us to have to rethink a large number of logistical issues.We are moving from T1 to PRI ISDN.The telephony demon consumes about 1 MB per instance, on average (over ~400 processes). This causes our caller/answer simulation to choke. 1.0*400 (demon memoy demands) 0.3*400 (caller/answer private data needs) 5 (caller/answer shared code needs) 30 (OS needs) ------------- +30+5>512)Because Susie Greenfield wants the CDR production box for his development--Jenna having been unable to provide the hard disk space Susie needs--2.8 may drop off Test's plate.LVAL| . 4 F |,l0Jennifer, Jason, David, Jack, Kefetew, and I put about three or four man-months of test development plus a couple man-months of execution into these phases. No ROI will be achieved on this time.Three days out of CSA testing effort.Probably at least a two week slip, if not three. Also, impact on the test development is not clear, but presumably not trivial.Down on test development for about a day.We're getting further away from where we need to go, not closer.CM test development is moving slowly, if at all.Loss of 1/2 day for Deepak, probably an hour for Hitesh.Day-for-day slip on a project already way behind.Emma has lost a bunch of time on this today, as have David and Jason.This has blocked us from running tests for the last four or five days.Because this is a major nexus between 2.8 and every other component, changes here ripple through all the components, as well as affecting all the testing.Five person hours lost for three people.I've spent about two hours today discussing the plan with Jenna.We weren't ready to use these anyway.None, because we need to catch up with our automated test tool development, so downtime due to blocked testing is okay.We are slipping starting test day-for-day while we are waiting to get this tool working.Test development/maintenance blocked.A fairly signficant amount of confusion and lost time associated with trying to make this happen, all for the want of telephony cards. I'd guess that I personally have spent most of today trying to rework this plan. Jack, Jennifer, and the techs are working at significantly reduced efficiency.1) Loss of about three schedule days and about one person-day in terms of confusion, scrambling to get back on schedule. 2) Damage to the VME backplane or the bus controller (not clear which at this time). Possible damage to one or more of the eight operating telephony cards. 3) Same as two, but *further* damage.LVAL h ( 6hjpPapadum is broke and blocks proceeding with automated testing.Reference data continuous to change.Hitesh has suffered repeated problems related to getting the right software and hardware set up for testing CDR. Every problem seems to eat up a whole day after it comes up.Test environment not ready at new location.The relocation of the test team in the midst of system test.The delivery of PRI lines remains unclear. We don't know exactly when, but the date now is 12/4. This is well after the 11/23 deadline.Signficant violations of system test entry criteria were approved in order to hit the 11/23 system test date. Only CDR were actually met all of the entry criteria for System Test today.Problems with legacy test tools have come home to roost. David and Jason, bless their hearts, are on the critical path now. The caller_sim program is fubared, and has been for a while.CSA doesn't perform basic functions right now.Install process for CSA is manual and not documented.The CSA interface changed five weeks ago and Hitesh found out today.We had a plan to receive our remaining five IVR boxes by Monday 11/16. This didn't happen. We'll get two on 11/23 (D-Day), one of which will be three telephony cards short of a full load. The others won't show up until HWint gets telephony cards. This is supposed to happen next week.Jennifer and Kefetew became involved in debugging the configuration management tool. Biryani got totally scrogged, costing us a system when we were already short four IVR units. SWint didn't respond to our reports of problems. This circus went on for about five days, at which time I put an end to our participation by pointing out that we don't do debugging or unit test, we do system test, and the configuration management tool was clearly not ready for system test. Linda agreed and disengaged us. The circus continues, but without us dressed up as the crying-on-the-inside clown.LVAL H  x 1/16: This may work out well, though, because the CSiTCL tool is starting to come together. 2/24: The quality of the CSiTCL is signficantly below what is needed for effective and efficient testing.12/28: We are still struggling with this. 1/16: Still struggling with significant inefficiencies. 1/26: Still open.1/16: There's a lesson to be learned here. When a test phase misses its original schedule by some large factor (who knows, 200% or so?), insist that it be cancelled unless the schedules are changing along with it. Otherwise, some effort is bound get squeezed in the middle as these did. Utlimately, something had to "pop out." Visualize squeezing on a water bottle from both sides. If some water isn't let out deliberately, the bottle will burst and more water will spray out the hole.1/16: CDR system test remains very slow, and subject to losses.12/9: Forward progress still blocked by various other problems related to releasing a working CM product into the test environment.1/16: Ultimately, we lost about three weeks of Gordon, Luis, and Jamie's time because of this.12/5: Ultimate one day lost for each team member.1/16: Whoops, this did kill us. PRI doesn't work with the telephony boards. No impact on test yet, but it could happen.3/19: Ultimately, configuration management was a large issue for formal Integration Test, but this CSA piece was only part of that larger set of problems.11/23: Impacts continue. We had an incident today where Bob began reconfiguring the telephony hardware while we were trying to fix tests, which ended up confusing everyone. 1/16: We probably lost about a person-month total. This event would not have occurred had our hardware allocation plan been more successfully executed. Once again, a testament to the logistical and technical problems that arose in terms of test environment procurement, configuration, and support.Fv'q % m 7 ?G8Tij@@@>@Content Management (2.11)@/Late Deliverablex@-None possible.@.h\*1:@:@?Content Management (2.11)V@/Late Deliverablej@-v@ +@.h\*3;A@@?Integration@,Dependency~@-@ +xxll``L@*@@@N@All2@ ,Quality Problem@-L@ +@.~rffZZ<0*3p@p@SOB (2.8)@ ,DependencyH@-None possible.hh\\H<*ߦX@@IVR App & Tele (2.4)Z@ ,Feature Change/Deletion@-None possible.N@.`T*0ųQ@ @IVR App & Tele (2.4)R@ ,Late Deliverablev@-P@ +1/16: See change record 191.`T*0\;@@IVR App & Tele (2.4)@ ,Test Additionh@-None possible.@ )zz`T*0 ;@ ;@N@IVR App & Tele (2.4)@,Quality Problem@-None possible.@)~~`T*30;@0;@xPlatform (2.2)z@,Test Deletion@'*@+@)xxllRF*29:@@@CSA (2.5)@,DependencyJ@'t@+tthh\\H<*@@@>@ SOB (2.8)@,Deliverable@'@+@)vjj^^H<*3ej-5@ej-5@?Content Management (2.11)@,DeliverableR@' @+@)~~h\*3K~$5@K~$5@IVR App & Tele (2.4)@,Test Addition@'@+zz`T*$ט@$ט@Content Management (2.11)@,Dependency`@ '@+||h\**% _@*% _@?SOB (2.8)@,Deliverablep@ '@+vvjj^^H<*QS@QS@5@?Content Management (2.11)j@,Deliverableb@ '@+@)~~h\*3$S@$S@?IVR App & Tele (2.4)|@ (Dependency@ '@+tt`T*kS@kS@@SOB (2.8)Fanout script for RDT broke.Deliverable@ 'Z@#||f<*wzS@wzS@SOB (2.8)H@ (Deliverable4@'@#1/16: See change record 171.vjj^^H<*0V46@@@SOB (2.8)\@ (DeliverableWe've lost about a week@#^^H<*Qa@Qa@?Content Management (2.11)V@ (DependencyP@'@#||h\*$ku@@?Allx@(Dependency@'@ #b@)th\\PP<0*3LVALN   |  B ZdJennifer will direct the techs towards writing scripts that have an Integration Test usage as well this week. That does delay System Test by about a week, but the scripts had to be written anyway. The debugging of the CSiTCL tool should not be affected.This has to get fixed, or we will miss our 2/1 target.Hitesh/Gordon to work with Jennifer on Integration scripting and data.We need contact information for the whole SWint team.Re-allocate Madras from Beta One to telephony system test development. Share one of the integration test platforms for the other pair. This is another symptom of the hardware allocation challenges on this project.I assume they worked on other things, but less efficiently.The escalation plan should probably include redundancy in each area of test environment support.None possible at the Test level. CM, CSA, and CDR got much better over the first month of system test. IVR system and software remains, at this date, entirely hosed.None possible, just keep coding scripts.Recovery's not in the cards, but a more realistic consensus on critical quality risks up front could have prevent so much time sunk into this effort.The move is necessary, but again probably not good timing.There is no way at this point to accelerate the development effort.Institute more robust configuration management and version/release engineering. A smoke test would have helped us avoid this problem.Make sure that the test toolsmiths focus on finalizing this tool. It's on the critical path for our testing effort.This is partially a test deliverable issue, partly a configuration management issue.This was another manifestation of the configuration management problems on this project.As with the expensive servers, redundancy is not in the cards for this element of the test environment.It's hard, when testing with servers this expensive, to have redundancy in place to avoid these kinds of problems.tLVAL Z l &zd2Test environment support was not over the holiday as previously arranged.None of the components satisfied their entry criteria. Many wouldn't even survive the stopping critieria, esp. IVR Telephony, but the project management team decided to waive the criteria in an attempt to meet the schedule. I do not believe this will work, but we are continuing.We had management communcation problems that resulted in Vincent and Simon not completing a number of tasks that I had scheduled them to doGetDigit is broken, returning extraneous digits and sometimes losing digits. This should result in (valid) failures from the CSiTCL tool when the wrong brand is encountered. However, we have been instructed to code the tool to kludge around this, returning pass when a failure has actually occurred.We were to receive the CSiTCL tool on Christmas, but it's slipping now. The current delivery target is "tomorrow" (12/29), but tomorrow is always a suspect date to me.I spent three days putting together a tool design spec that we really should have had done months ago.Jack and Kefetew have spent the last two days lost in debugging various telephony problems which could have been caught by component testing.The MTBF and Platform System Test phases have been cancelled.We will be down at least three days due to the CSA move 12/28-12/29 plus having to set up our own systems on 12/30.CDR 3 will not show up until 12/23, which is not only two weeks late (scheduled 12/11) it is smack in the middle of the Christmas slowdown.It looks like the delivery of the second build o the CM app broke something, and now we can't develop tests.We keep discovering new features that we need on the CallSim program.The CM test development/execution environment remains unavailable for the most part, due to Tibco licenses and other problems.Jack and Bob accidentloy deleted some test scripts that Deepak and Hitesh had put together.The new location agent workstations are still not up.LVALx  |J>Jennifer is probably at 25% efficiency on her data development, so we've lost some amount of time now, and will continue to loose time.We have slipped another week from last week.I'd say that Kate has worked at about 50%, Hitesh's new engineer and Larry were out to lunch, and Hitesh is, of course, doing nothing.Probably an hour here, and hour there, adding up to a person-day or so of time spent creating update scripts that stuff appropriate values into fields that really ought to be in external tables.Jack's development team is probably operating at 50% efficiency on test development while they have to share systems with Jennifer. Jennifer, meanwhile, estimates a one-day (two person-day) hit on setup time, plus a three to four day (6-8 person-day) hit per cycle.It remains to be seen, but how could it be less than 5 person*hours per week?Hitesh and Mary have burned about a person-day trying to get this going so far.We have had a delay of three weeks based on the original deadline.This is a three week slip off the already-late initial date.This cost us a whole day of the CM test group's time.Probably lost about a person-day of time due to ineffeciencies.Difficult to say, but preobably three to four person-weeks of lost tech and engineer time, as well as over a dozen calendar-weeks summed over all the projects, due to inefficiencies of testing unready systems..Four to six man-days lost this week.Major risk of "test escapes" when the time comes to remove this kludge. I have asked the project management team and the developers to make sure that this kludge is not accessible to my team, whether by command line option, config file, or otherwise. I don't want us to be responsible or appear to be responsible for any of these test escapes.Day-for-day slip on starting to debug our new test scripts.I was distracted, which got me sucker-punched later.Four more person-days lost due to test escapes from earlier efforts.LVAL8z \ dvd^B3/13: The final note in this story is that the component development managers took over all bug triage, with the project management team no longer having any role.2/24: This turns out to be related to some problems with T1, PRI, and loopback used together on a system.2/24: Was moved back to its original configuration on schedule.2/24: Now all the formal Integration Test has been cancelled, too.1/26: Jenna says we're on track for a 3/1 cut-over. 2/24: Again, a recommit from Jenna, but no specific list of hardware or dates yet.2/24: We *did* miss our 2/1 target. This is supposed to be fixed now.1/26: Still no contact info from SWint. 2/24: Still no contact info from SWint.1/26: The plan is for freeze of all data by end of January. 2/24: Here is is, end of February, and changes are still happening, albeit few.1/26: AM system test may start this week. 2/24: AM system test never happened.1/15: Biryani and Madras were reallocated to Jack, but, as of this point, he still doesn't have them. Jennifer is to have Vindaloo and Jalfrezi by COB 1/20, but we'llsee if that happens. 1/16: We now have them up and running, so this is closed. We lost about a day and about 5 person-days.1/16: No major impact yet, so maybe this is no issue. 1/26: Ultimately, it turned out to be less an issue than it seemed to me it would be.1/26: Still open? 2/24: Abandoned upgrade and returned to old verion.1/16: We are finally up and running on this, having lost about three weeks and about five person-days in terms of productivity. 1/26: We are moving forward, but are short-staffed due to illness. 2/24: The tool still doesn't scale, but appears stable in small blocks.1/16: One month late and counting. 1/26: This week? 2/24: Never delivered for system test.1/16: Database change in general is a huge issue. 1/26: Scheduled for completion by end of January.1/16: This is my guess, at this time, on best case schedule and resource impact to date.1/26: This continues as an open issue.LVAL z L VT DAll the System Test efforts got scrubbed.The Tibco licensing issue is back.I found out today that the production hardware and underlying software (System Test) environment for CM testing has never been present. Jenna's team is still installing the last pieces. This is why Larry keeps having all these problems with system stability. Also, we couldn't even have run System Tests because of this.There was some sort of mis-cue on a release of RDT data to Hitesh for CSA, which confused everyone and cost Hitesh about a day.We will lose the EPCOT environment when Hamilton goes live.Jack got sick and will be out all week.The CM/RDT data is incompatible on the latest version, which blocks both CM and CSA testing.We have had multiple instances this weekend where phone support from development has not come through.The "shifting brand bug" is worse than we thought. Instead of just shifting one digit, it can shift up to three (or maybe more).The underlying data model remains in flux.Gordon has just now received the first piece of AM test development materials, a screen shot. It was scheduled originally for 12/14 or so.With Hitesh out sick and Dolores at his father's funeral, life has pretty much come to a halt on CSA this week.The denormalization of the data tables confuses the DataTect test data generation tool.Jack's team was temporarily but without noticed stripped of its SUT resources, Roganjosh and Halva, which are in use to support IVR software debugging.Kefetew is no longer the point man on supporting debugging of 2.4 any more. The test team is now to provide 24x7 support involving Jason and David, too, including cell phones.The latest version of QAP, which works with CM, is apparently partially incompatible with Vantive.The CSiTCL tool is still not working.In case I haven't noted this before, AM still isn't delivered, and won't show up until 1/6 or so.The database underlying CM changed (again).u6  n ; ? ~Hg@@?Integration~@4Late Deliverable@19.9.xNone possiblexllL@*@@,@All@4Budget Reduced@3AllF@2udXX<0*@@,AllR@/Test Change/Deletion$@3AllNone@ .pdd<0*?@@Content Management (2.11)D@/Dependency@12.11.x, 2.5.xLet's get the licenses fixed!"2/24: This issue is now closed.||h\*?@@@@@Content Management (2.11)@/Dependencyp@12.11.xd@2||h\* @ @?CSA (2.5)@/Late Deliverable See above.2.5.xNone possible.hhH<* @`@@-@Allv@/Budget Reduced@1All@2@ .udXX<0*? @ @@IVR App & Tele (2.4)N@ /Dependency@12.4.x@+tt`T*@@@IVR App & Tele (2.4)"Customer registration is broken.Quality Problem@12.4.xl@+@ .T*?@@@Content Management (2.11)@ /Quality Problemb@12.11.x, 2.5.x@+1/26: Now resolved.h\*? @@?IVR App & Tele (2.4)@ /Dependencyr@12.4.xj@+@ .tt`T*?@@IVR App & Tele (2.4)@ /Quality PSimonlem@12.4.xMust get fix!1/26: Still kludged around.`T*?l@@ AllT@ /Feature Change/Deletion@- This stuff just has to freeze.@ .vvjj<0*7ų@ų@Content Management (2.11)@/DependencyX@-None possible.@.||h\*0 @@@ CSA (2.5)@/Dependency @-None possible--one week slip.hh\\H<*Y򋥮@Y򋥮@?Integration@/Quality Problem@ -None possible.vvjjL@*@@@?IVR App & Tele (2.4).@/Budget Reduced@ -@ +P@.||`T*3mҘ@@@IVR App & Tele (2.4)`@/Test Addition@ -None possible.@.zz`T*0 %@@@?CSA (2.5)@/Dependency@ -None possible.@.hh\\H<*3*0ب@*0ب@5@IVR App & Tele (2.4)J@/Late Deliverable@ -None possible.@.`T*3LVAL    B,This cost the 2.4 test team a whole day of "guerrilla" testing of the call flow.2.4 script debugging brought to a halt for a day.Jennifer has been blocked on a proof-of-concept since last Friday.Jennifer lost about half-a-day trying to figure out where the system was.It's interfering with CM and CSA testing, and will block Integration Test if we don't resolve it. A background irritant for now, wasting a little bit of Hitesh, Gordon, Catherine, and Veronika's time every day.Probably a few person-weeks lost in productivity. In terms of schedule, who can say? Had all the hardware and the configuration software been in place on 11/23, we'd probably be a week ahead of where we are now, and, with all the scripts written, who knows what kind of slip is happening until this gets fixed.This is serious. We will take a one week hit on the cut-over to the new test environment (based on past performance at best). All results gathered in the new environment are questionable. Since I'm not convinced we'll get a full run of every test suite prior to the cut-over, this means test escapes. I'd estimate that we're looking at 5 serious bugs missed, and, using an order-of-magnitude estimate, I come up with a 5,000 hit for what would have been 500 to find and fix during Test.We will probably loose momentum and quality of work, and miss our 2/1 first cycle of System Test target. See recovery plan below.The ability to debug scripts is hampered, and procedes at 50% efficiency.At least a week while they get this squared away.This has resulted in many instances where the whole team--David, Jason, Kefetew, Chris, Scott, and Jack--were thrashing while waiting for someone to solve their problems. I'd say we worked at 50% efficiency this weekend, which is a problem after all the effort and expense of having everyone on deck all weekend.This totally blocked test execution, and impedes test debugging.LVALF T 4 l  PI need to talk this over with Jenna. We need a higher caliber of support people, we need more people, we need faster response, and we need people empowered to solve problems for us without getting extremely busy people like Jenna in the sign-off loop.We should have had a plan in place for a run-time test environment from the start. That's as much my fault as anyone's.This is all really momentum of past project issues accelerated through multiple missed opportunities to resolve quality problems during earlier testing phases.Not clear what the root cause is on this problem.A more robust plan might have caught this.Robust and consistently executed configuration management.None possible. Triage will ultimately be needed to remove test cases from the plan to hit the current schedule.Need better change control, configuration management, etc.Institute better test environment management.Fortunately, Lucy has plenty of other stuff to work on.Configuration management and release engineering processes that work.Again, better configuration management and release control is what we need.Institute better version/release engineering and configuration management processes.Jennifer is focusing on other things.This fits in with the quote in the Software Engineering Institute's Capability Maturity Model book on cancellation of test activities when schedule pressure looms.Again, test environment configuration and support.Use a development-type environment for testing. Again, difficulties with procuring, configuring, and supporting the test environment.LVAL & :,nThis equates to a loss of staff for at least the next six weeks (until 1C), at least in terms of the original scope.This is causing delays (of at least two days) in Lucy turning around bugs.Lucy, the data czar, loses two days. Others invovled in the test effort lose about a day each.System supports out about four hours. The whole day is shot for Larry and Cheri. Hitesh lost a good three or four hours of work, too.Many bugs have probably already been missed. Assume that we will only have ten serious field failures that could have been detected in System Test, at a cost of $1,000 each to detect and fix. This means we have a $10,000 hit from not being able to figure this out. Alternatively, if we find these bugs in Integration Test, they will result in schedule slips.First, all the tests which resided on the box must move to Beta. This will result in an hour or so of lost time. Second, all testing on CDR is now blocked. (Hence the six-week hit on CDR.) This reassigns Lou to CSA, which probably is an hour or so of his time. Finally, Jennifer, Jack, Hitesh, and I have spent an hour in a lather about this, so all considered, we're out about two person-days.We lost four technicians to the "DDT" effort, which may or may not be productive in finding bugs. (They're gone for at least eight weeks, so 8*5*4=160.) My guess is the loss of these four resources on test is about two weeks in schedule time--most of the loss test will absorb by reducing scope. It took me four whole days to work out the plan.Let's assume that every skipped test involves eight hours of work, of which half is wasted and half well-spent because we can re-use in integration. This is seven person-days, mostly CM. However, the whole IVR App effort is a bigger waste. This also comes from the MCF changes. On the upside, this may accelerate the testing by a couple weeks, since we're not requiring the same level of quality.<LVALHP , j ` B "NljWe started testing against the IVR system in the basement (Vindaloo) rather than an installed IVR due to the need for 0-out work.For some reason, the test team had to move on Monday morningManual testing was scheduled to start tomorrow morning, but now it's delayed at least one day.We lost two servers without advance notice.Linda told us to proceed with test development based on James's scripts. Well, it turns out--now that Dolores has decided to re-engineer this decision--that we really should be proceeding from his list of end-to-end test cases. Based on a perusal of his list, the coverage overlap is less than 50%.We had two reboot incidents this week where someone on Jenna's team rebooted a test computer without talking to anyone in test first. One happened on the CSA server. The other happened on the two IVR boxes Jack is using for testing.All automated testing is on hold for the moment, with our team directed to perform scripted and exploratory manual tesitng.CM delivered a client first, but no CM server or indexed database. We don't know when that will show up, but it will be out of step with the client.There was a change to a field definition in the member table.Something changed on CSiTCL w/o anyone telling Jack or the 2.4 test team about ti.The CM team installed a version of the CM desktop on the clients that was incompatible with the CM server and other back-end systems.The other integration system was to be delivered to Jennifer last Friday. It's still not set up.The PRI ISDN "incompatibility" that has plagued us ever since System Test started way back in November turned out to be merely a misconfiguration.The Curry server has been pre-empted "for a few days" to use in the IVR resource debugging. Pieces will be installed in an IVR system to see if it will scale with this additional resource.Naan, Jennifer's other integration test system, showed up late.There was a large test re-organization instituted that took a week-and-a-half to hammer out.L=n   I ! ;Ht8@@@2Integration@8Quality Problemb@ 6None possible.@7vvjjL@*7@@Integration@8Quality Problem@6t@ 2vvjjL@*@@,Integration@8Test Additiont@6None possible.rrffL@*@@Integration.@8Late Deliverable Hopefully none, but we'll see.None needed?@7llL@*7@@?-Integration@8Quality ProblemOne day, everyone blocked.@ 2@7jjL@*7@@?#IVR App & Tele (2.4)L@8Quality Problem@6t@ 23/13: Still no fix.~~`T*7@@Integration@4Late Deliverablev@6None needed.z@7xxllL@*7@@?IVR App & Tele (2.4)x@4Dependency@6None possible.tt`T*`@@?-All@ 4Late Deliverable:@69.9.xNone possible.@7{h\\<0*?@@IVR App & Tele (2.4)V@ 4Dependency@62.4.xNone possible.tt`T*@ @?IntegrationX@ 4Test AdditionF@69.9.xWe have to go and backfill.@7rffL@*?@@?All@ 4Dependencyz@62.4.x, 2.5.xZ@2v\PP<0*@@>@,All@ 4Budget Reduced@3AllNone possible.H@.udXX<0*?@@@Content Management (2.11)*@4Late Deliverable@32.11.xn@2h\* @ @?Integrationz@4Quality Problem@39.9.x@2vjjL@*@@?IVR App & Tele (2.4)@4Quality Problem@ 12.4.x@2@.~~`T*?@@?IVR App & Tele (2.4) Someone pulled Jack's T1 line.Dependencyb@ 12.4.x, 9.9.xGet this line back.T*@@@@?Content Management (2.11) @4Quality Problem@32.11.x@2h\*@@@@Integration@4Dependency@ 19.9.xJ@2l``L@*@@@@IVR App & Tele (2.4)$@4Quality Problem@32.4.xNone possible.~~`T* @ @E@SOB (2.8)z@4Dependency@3 2.8.x, allMove stuff to Beta.@.h\\H<*?DLVAL @ : ^ x*jJennifer is spending about four hours managing something that wasn't initially scheduled for us.Just confusion at the management level.We should have been testing since Tuesday, right?We were dead in the water for 1.5 hours. Assume 20 people, so that's 30 person-hours.I spent two hours dealing with this issue. The client is out for bonuses and meals. Each tech gets $100 (by five techs), each temp gets $50 (by 7 temps), each CSTA/CMTA gets $100 (by 15), and Hitesh gets a week vacation ($1,000). Figure 30 people getting four meals at $10 a meal, so that's $1,200. Plus the six hours of overtime (mininum) for each tech ($300) and the consultants ($1,800). So, this change costs the client around $7,000.All the data created by all our tests so far must be flushed, including CM and CSA to stay consistent. So, the testers must all recreate their data. The last 1.5 days have been nothing but. Assuming that .5 days involved reporting bugs and learning process--which we needn't repeat--that adds up to a loss of about 30-35 person days. The schedule hit is at least one day.Mostly just confusion, but we appear to have dealt with it.Everyone involved lost 3-4 hours in the middle of our furious manual test crunch.Since the schedule will not change, undetected bugs means larger quality issues. Also, 30 or so people will be underutilized tomorrow, so that's money lost.It's hard to say right now. Possibly none, unless we need to add automated testing/load generation to the Integration Test effort.Based on the fact that about 50% of the coverage seems to have been achieved accidently, let's assume that the other 50% of the time is non-ROI, at least in terms of management priorities. We had ten people writing test cases Friday and all weekend, about ten hours per day, so that comes to 300 person hours total, 150 hours of which were not spent on target by this formula, and three schedule days, 1 1/2 days lost.Jack's team lost about 10 person-hours. Hitesh lost about 4.b LVALB8 ~ 3/18: Such blockages continue. 3/19: And again today.3/14: Downtime last night and this morning (two hours during test) means that we only got a few hours in yesterday (system down most of the morning) and will probably have limited testing today. 3/14: Scheduled downtime of 1 hour at lunch became 1.5 hours.3/13: It didn't work, and the smoke test was omitted, so we got hosed. See change record 230.3/19: We prioritized, but the schedule slips made triage unnecessary.3/13: See change record 230 for the further results of this.3/11: Ended up delayed another half day, and started afternoon of 3/10.3/13: We ended up getting all the coverage we needed anyway. Hats off to the techs and engrs.LVAL $ v DBL.None of the third integration cycle BB[2] items were ready to go on 8/3. As of 8/11, we probably won't receive all of them until Friday, 8/14.No telephony system is available at SWint to help Rotem debug the stress/capacity test scripts. We will probably have to wait until we receive Beta, then Rotem will have to debug and tune them here.Testing again scheduled to be here over the weekend.The test environment probably won't be ready after 1C. Too much other stuff going on.Now we are extending manual testing for at least three or four more days. The system remains unstable and the conversion process is failing in test.We spent the whole day (until about 4:30 PM) blcoked due to data conversion and a router misconfiguration.We only ended up getting in about six or eight hours of execution all day yesterday. This was due to various bugs we found, among them a problem in terms of WAN bandwidth.Tibco went down from 1:45 to 4:15 (and counting).The data migration into testing fell through the cracks until today, and we had to scramble to come up with a plan.We had some requests for test automation, but then those were all dropped to do more manual testing over the weekend.Downtime since we started testing has allowed only about two days of testing in so far.Smoke testing was omitted and the IVR system was hosed by the Vindaloo to TOR_IVR move.Project managemeent decreed an expansion of the weekend test coverage at 4:30 PM Friday.We're going to move from Vinedaloo to TOR_IVR tonight between 10:00 PM and 7:30 AM and will--theoretically--be back up tomorrow morning with no impact.The system has been down since 9:00 AM this morning until now (3:30 PM) and we have no ETA, so I'm assuming the whole day is gone.Somebody did something and "melted" the IVR database beyond repair. The real problem is that someone *could* do something like this. (The system should prevent it.) N G q `@`@@Integrationh@ 8Schedule Shortened@:None possible.||ppL@*@@All@ 8Late Deliverable@:@2tthh\\<0*@@@xzIntegration*@ 8Quality Problem|@:>@2vvjjL@*@@?Integration@ 8Quality ProblemJ@:None possible.l@7vvjjL@*3@`@IntegrationX@ 8Quality Problemn@:None possible.vvjjL@*`@`@Integrationb@8Quality Problem Testing blocked all afternoon.9.9.xb@ 2jjL@* @ @Integration@8Test Addition@ 6T@ 2~~rrffL@*@@Integration@8Test Change/DeletionN@ 6None possible.ttL@*`LVALHH8 d ~When this happens, tests die and must be restarted. It's probably chewed up twenty or so test hours, and maybe ten or so person*hours. It also reduces confidence in test results.1. Bob received a list of tasks necessary to getting Vindaloo up and running on Friday. He left and took the weekend off, in spite of the criticality. 2. Lisa Burns, the other person, is out for a week or so. 3. Bob can't set up a second server to help us run and debug the network load scripts. 4. Bob, according to Rotem, knows "more about Unix that us (Test), but just barely." This means that his learning curve is on our critical path. 5. Bob has lots of other work, so, when we need urgent support, we have to queue up for it. 6. When we ask Bob for help, someone else--Jenna, Phil, etc.--often ends up on the critical path for permission purposes. This can add days to the response time on a trivial, fifteen-minute task. I would guess that, calendar-wise, this has impacted the network load tests by a week. The lack of urgency on Vindaloo cost us three days. The other problems are harder to estimate, but are costing us time.Probably only slips us a day or so. Debugging of the Stress/Capacity/Burn-in scripts will start tomorrow.Each tech got 12 paid hours--by 10 techs--at an average of US$10. Also, the automated testing doesn't get worked on while this happens.Test execution will be block until a test environment is provided.30 people a day, four days. Add meals and such, another $800.No testing in the second-to-last day.50% efficiency, making it impossible to finish testing.  @@@@@ @ @ @ @ @@ @ @ @ @@@@@@@ @@LoOUQmiQOoMQO0LoOUQmiQOoMQO0 LoOUQmiQOoMQO0LoOUQmiQOoMQO5OQ^YqQiJL^Q OQ^YqQiJL^Q OQ^YqQiJL^QOQ^YqQiJL^QOQ^YqQiJL^Q$OQ^YqQiJL^Q$OQ^YqQiJL^Q$OQ^YqQiJL^Q$ OQ^YqQiJL^Q$ OQ^YqQiJL^Q$OQ^YqQiJL^Q$OQ^YqQiJL^Q$OQ^YqQiJL^Q$OQ^YqQiJL^Q*OQ^YqQiJL^Q*OQ^YqQiJL^Q*OQ^YqQiJL^Q*OQ^YqQiJL^Q*OQ^YqQiJL^Q* OQ^YqQiJL^Q* OQfQbOQbMvOQfQbOQbMv OQfQbOQbMv OQfQbOQbMvOQfQbOQbMvOQfQbOQbMvOQfQbOQbMvOQfQbOQbMvOQfQbOQbMvOQfQbOQbMv OQfQbOQbMv OQfQbOQbMv OQfQbOQbMvOQfQbOQbMvOQfQbOQbMv$OQfQbOQbMv$OQfQbOQbMv$ OQfQbOQbMv$OQfQbOQbMv*OQfQbOQbMv*OQfQbOQbMv*OQfQbOQbMv*OQfQbOQbMv* OQfQbOQbMv*OQfQbOQbMv*OQfQbOQbMv0OQfQbOQbMv0OQfQbOQbMv0OQfQbOQbMv0 OQfQbOQbMv0 OQfQbOQbMv0OQfQbOQbMv0OQfQbOQbMv5OQfQbOQbMv5OQfQbOQbMv5OQfQbOQbMv5 OQfQbOQbMv5 OQfQbOQbMv5 SQJmoiQMWJbUQOQ^QmYdb*SQJmoiQMWJbUQOQ^QmYdb0^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q ^JmQOQ^YqQiJL^Q ^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q^JmQOQ^YqQiJL^Q$ ^JmQOQ^YqQiJL^Q*^JmQOQ^YqQiJL^Q*^JmQOQ^YqQiJL^Q*^JmQOQ^YqQiJL^Q0^JmQOQ^YqQiJL^Q0^JmQOQ^YqQiJL^Q0^JmQOQ^YqQiJL^Q5^JmQOQ^YqQiJL^Q5 ^JmQOQ^YqQiJL^Q5^JmQOQ^YqQiJL^Q5^JmQOQ^YqQiJL^Q9hoJ^YmvfidL^Q`$ hoJ^YmvfidL^Q`$hoJ^YmvfidL^Q`$hoJ^YmvfidL^Q`$hoJ^YmvfidL^Q`$hoJ^YmvfidL^Q`*hoJ^YmvfidL^Q`*hoJ^YmvfidL^Q`0hoJ^YmvfidL^Q`0 hoJ^YmvfidL^Q`0 hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`5hoJ^YmvfidL^Q`9hoJ^YmvfidL^Q`9hoJ^YmvfidL^Q`9hoJ^YmvfidL^Q`9hoJ^YmvfkY`db^Q`0kMWQOo^QkMWQOo^QkWdimQbQOkMWQOo^QkWdimQbQO9mQkmJOOYmYdbmQkmJOOYmYdb$mQkmJOOYmYdb$mQkmJOOYmYdb* mQkmJOOYmYdb*mQkmJOOYmYdb0mQkmJOOYmYdb5 mQkmJOOYmYdb5mQkmJOOYmYdb9mQkmMWJbUQOQ^QmYdbmQkmMWJbUQOQ^QmYdbmQkmMWJbUQOQ^QmYdb0mQkmMWJbUQOQ^QmYdb9mQkmOQ^QmYdb mQkmOQ^QmYdbmQkmOQ^QmYdbmQkmOQ^QmYdbmQkmOQ^QmYdbmQkmOQ^QmYdb$mQkmOQ^QmYdb$mQkmOQ^QmYdb$mQkmOQ^QmYdb* 7@ @   @  @ @@  @ @ @ @  @   @ @ @ @ @ @ @ @ df^JmSdi`::ef^JmSdi`::ff^JmSdi`::gf^JmSdi`::hf^JmSdi`::iYbmQUiJmYdbjf^JmSdi`::kf^JmSdi`::lJ^^mf^JmSdi`:: nJ^^ oYbmQUiJmYdb pYbmQUiJmYdb qYqiJffmQ^Q:> rJ^^sYbmQUiJmYdbtJ^^uf^JmSdi`::vf^JmSdi`::wYbmQUiJmYdbxYbmQUiJmYdbyYbmQUiJmYdbzMdbmQbm`JbJUQ`Qbm:88{YqiJffmQ^Q:>|J^^}f^JmSdi`::~f^JmSdi`::f^JmSdi`::f^JmSdi`::YbmQUiJmYdbJ^^f^JmSdi`:: f^JmSdi`:: f^JmSdi`:: YbmQUiJmYdb f^JmSdi`:: YqiJffmQ^Q:>J^^f^JmSdi`::YbmQUiJmYdbYqiJffmQ^Q:>MkJ:@YbmQUiJmYdbf^JmSdi`::kdL:F$kdL:F$J^^$f^JmSdi`::$f^JmSdi`::$YqiJffmQ^Q:>$f^JmSdi`::$YqiJffmQ^Q:>$f^JmSdi`::$J^^$ MkJ:@$ f^JmSdi`::$ f^JmSdi`::$ f^JmSdi`::$ YbmQUiJmYdb$f^JmSdi`::$f^JmSdi`::$f^JmSdi`::$MkJ:@$MkJ:@$MkJ:@$YqiJffmQ^Q:>$J^^$bQmsdi\:<$J^^*MdbmQbm`JbJUQ`Qbm:88*kdL:F*kdL:F*kdL:F*YqiJffmQ^Q:>*MdbmQbm`JbJUQ`Qbm:88*kdL:F*MdbmQbm`JbJUQ`Qbm:88*YqiJffmQ^Q:>* MdbmQbm`JbJUQ`Qbm:88* kdL:F* MkJ:@* f^JmSdi`::* YqiJffmQ^Q:>*YqiJffmQ^Q:>*YqiJffmQ^Q:>*YqiJffmQ^Q:>*kdL:F*J^^*YbmQUiJmYdb*MdbmQbm`JbJUQ`Qbm:88*MdbmQbm`JbJUQ`Qbm:88*YqiJffmQ^Q:>0MkJ:@0YqiJffmQ^Q:>0YqiJffmQ^Q:>0YbmQUiJmYdb0MkJ:@0MdbmQbm`JbJUQ`Qbm:880J^^0YqiJffmQ^Q:>0YqiJffmQ^Q:>0 MdbmQbm`JbJUQ`Qbm:880 YqiJffmQ^Q:>0 YqiJffmQ^Q:>0 J^^0 MkJ:@0MdbmQbm`JbJUQ`Qbm:880MdbmQbm`JbJUQ`Qbm:880J^^0J^^0YbmQUiJmYdb0kdL:F5YqiJffmQ^Q:>5YbmQUiJmYdb5MdbmQbm`JbJUQ`Qbm:885YqiJffmQ^Q:>5YqiJffmQ^Q:>5YbmQUiJmYdb5MdbmQbm`JbJUQ`Qbm:885J^^5J^^5 YbmQUiJmYdb5 YqiJffmQ^Q:>5 J^^5 YqiJffmQ^Q:>5 YbmQUiJmYdb5YqiJffmQ^Q:>5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9J^^9YbmQUiJmYdb9@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ @ @ J^^J^^ J^^J^^J^^J^^J^^J^^$J^^$ J^^$J^^*J^^*J^^0J^^0 J^^0J^^0J^^5J^^5 J^^5 J^^9MdbmQbm`JbJUQ`Qbm:88MdbmQbm`JbJUQ`Qbm:88*MdbmQbm`JbJUQ`Qbm:88*MdbmQbm`JbJUQ`Qbm:88*MdbmQbm`JbJUQ`Qbm:88* MdbmQbm`JbJUQ`Qbm:88*MdbmQbm`JbJUQ`Qbm:88*MdbmQbm`JbJUQ`Qbm:880MdbmQbm`JbJUQ`Qbm:880 MdbmQbm`JbJUQ`Qbm:880MdbmQbm`JbJUQ`Qbm:880MdbmQbm`JbJUQ`Qbm:885MdbmQbm`JbJUQ`Qbm:885MkJ:@MkJ:@$ MkJ:@$MkJ:@$MkJ:@$MkJ:@* MkJ:@0MkJ:@0MkJ:@0YbmQUiJmYdbYbmQUiJmYdb YbmQUiJmYdb YbmQUiJmYdbYbmQUiJmYdbYbmQUiJmYdbYbmQUiJmYdbYbmQUiJmYdbYbmQUiJmYdb YbmQUiJmYdbYbmQUiJmYdbYbmQUiJmYdb$YbmQUiJmYdb*YbmQUiJmYdb0YbmQUiJmYdb0YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5 YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb5YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YbmQUiJmYdb9YqiJffmQ^Q:> YqiJffmQ^Q:>YqiJffmQ^Q:>YqiJffmQ^Q:>YqiJffmQ^Q:>$YqiJffmQ^Q:>$YqiJffmQ^Q:>$YqiJffmQ^Q:>*YqiJffmQ^Q:>* YqiJffmQ^Q:>*YqiJffmQ^Q:>*YqiJffmQ^Q:>*YqiJffmQ^Q:>*YqiJffmQ^Q:>0YqiJffmQ^Q:>0YqiJffmQ^Q:>0YqiJffmQ^Q:>0YqiJffmQ^Q:>0 YqiJffmQ^Q:>0 YqiJffmQ^Q:>0 YqiJffmQ^Q:>5YqiJffmQ^Q:>5YqiJffmQ^Q:>5YqiJffmQ^Q:>5 YqiJffmQ^Q:>5 YqiJffmQ^Q:>5bQmsdi\:<$f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`:: f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`::f^JmSdi`:: f^JmSdi`:: f^JmSdi`:: f^JmSdi`:: f^JmSdi`::f^JmSdi`::f^JmSdi`::$f^JmSdi`::$f^JmSdi`::$f^JmSdi`::$f^JmSdi`::$ f^JmSdi`::$ f^JmSdi`::$ f^JmSdi`::$f^JmSdi`::$f^JmSdi`::$f^JmSdi`::* kdL:F$kdL:F$kdL:F*kdL:F*kdL:F*kdL:F*kdL:F* kdL:F*kdL:F5 Y N?? Y Y ZoDataIDropab?IZoYAOIndexv1K>[v@@@ @ l0`EoctLLS@LLpaia GoHoGoPFoHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaHoaLoNo`Io@Ko@oaK>[ࡱ> ;<   37 !"#$%&'()*+,-./01245689:Root EntryVVBAVVVBAProjectVVVBAVVdir_VBA_PROJECTPROJECT'FPROJECTwm-  !"#$%&()*+,123456789:;<=>?@EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdef0* pHd3Case Study IVR Network Test Change Management (new)l@  =    n=( %<9stdole>stdole f%\*\G{00020430-;C 0046}#2.0#0#C:\WINDOWS\SYSTEM\StdOle2.Tlb#OLE Automation/gADODB> ADOBC CD1C10-8CAA006D2(EA4C1CPROGRAM FILES\COMMONL9\msado21.tNMicrosoft ActiveX Data Objects 2.1 Libra,ryH"U am  *\G{000204EF-0000-0000-C000-000000000046}#4.0#9#C:\PROGRAM FILES\COMMON FILES\MICROSOFT SHARED\VBA\VBA6\VBE6.DLL#Visual Basic For Applications*\G{4AFFC9A0-5F99-101B-AF4E-00AA003F0F07}#9.0#0#C:\Program Files\Microsoft Office\Office\MSACC9.OLB#Microsoft Access 9.0 Object Library*\G{00020430-0000-0000-C000-000000000046}#2.0#0#C:\WINDOWS\SYSTEM\StdOle2.Tlb#OLE Automation*\G{00000201-0000-0010-8000-00AA006D2EA4}#2.1#0#C:\PROGRAM FILES\COMMON FILES\SYSTEMK>[\ADO\msado21.tlb#Microsoft ActiveX Data Objects 2.1 Libraryn= Ux 4%AccessVBAWin16~Win32MacVBA6#Project1 stdole`3Case Study IVR Network Test Change Management (new)EADODBs<  ID="{56898E9D-2DC0-4C3F-B649-8EB24A54D010}" Name="Case Study IVR Network Test Change Management (new)" HelpContextID="0" VersionCompatible32="393222000" CMG="4240A6E369E769E769E769E7" DPB="090BEDEEEEEEEEEE" GC="D0D234B735B73548" [Host Extender Info] &H00000001={3832D640-CF90-11CF-8E43-00A0C911005A};VBE;&H00000000 AcessVBAData.Forms VV0 VVBlob  0=Babc+e ghij3kg &;@b DID>j{N Ut1+IM8{Aa3@ChangesDC&yt1+IM8{IDgYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date Noted{nU5uB&f Et1+IM8{Impact Date̍ 'L}L;6t1+IM8{Change TypexWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description5N7:K& t1+IM8{Recovery Plan)>*@%0t1+IM8{Cross-References :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/Savings@ dLpÄtt1+IM8{Test Impactoc@qѻmt1+IM8{StatusChanges,Changes Affecting Test Ariald2e12f5g237hghMS Sans Serifi1ghj1ghk1l23:m4n2o2p5r35zijMS Sans Serif`FormHeaderu#K׊` DetailA4fM7_Zm7U`taxbdceIDIDi;O: aVd235U9`<axbcID LabelID:ID_Labelxx]G[CSA (2.5)";"Call Centre (2.6)";"900 Integration (2.7)";"SOB (2.8)";"Content Management (2.11)";"Integration";"All"RName of project in which change occurred.Project_Named235U9`@ axbc$Project Name LabelProject:$Project_Name_Label1DCm7U`axbce9kDate NotedDate NotedShort DateRDate on which the change was first noted.Date_Noted,JK@.5d235U9`axbc Date Noted LabelDate Noted: Date_Noted_Labelg`ddHLrm7U``'axb8ce9kImpact DateImpact DateShort DatefDate on which the change will first affect testing.Impact_DateN(D"*d235U9`(#axbc"Impact Date LabelImpact Date:"Impact_Date_Label7CMlJo05Uactde fh n}2bqN_bChange TypeChange TypeValue List"Late Deliverable";"Early Deliverable";"Feature Addition";"Feature Change/Deletion";"Test Addition";"Test Change/Deletion";"Schedule Slipped";"Schedule Shortened";"Dependency";"Budget Increased";"Budget Reduced";"Quality Problem";"Quality Exceptional"TType of change which is affecting testing.Change_Typed235U9`<abc"Change Type Label Type:"Change_Type_LabelJ02=BP&qZLm 27U`taXb$'c8e k$Change Description$Change Description[24&Other Costs/Savings*$#,##0.00;($#,##0.00)"& NARd235U9`!abcLabel25.Other Cost/Savings ($):XGgb3GS͹m7U`tahb$'ck Text26Test ImpactR`.JAm3Zd235U9`<ahbcLabel27Test Impact:)O?B5Cm 7U`tapb$'c(k Text28 Status9<ʻAd{Ķd235U9`<apbcLabel29Status Log:X;H όPz:`FormFooter":FZPropData /TypeInfo0&DirData APropDataBͬ%'E,MP'zFormHeaderDetailmIDd ID_LabelID LabeloProject_NameProject Named Project_Name_LabelProject Name LabelmDate_NotedDate Notedd Date_Noted_LabelDate Noted LabelmImpact_DateImpact Dated Impact_Date_LabelImpact Date Labelo Change_TypeChange Typed Change_Type_LabelChange Type Labelm Change_DescriptionChange Descriptiond Change_Description_LabelChange Description LabelmImpact_DescriptionImpact Descriptiond Impact_Description_LabelImpact Description LabelmRecovery_PlanRecovery Pland Recovery_Plan_LabelRecovery Plan LabelmText18d Label19mText20d Label21mText22d Label23mText24d Label25mText26d Label27mText28d Label29FormFooterCross-References Schedule Impact!Resource Impact"Other Costs/Savings#Test Impact$StatusChanges 0CB0ͬ;cj֟vF)7{ReportHeaderdLabel41fLine44PageHeaderdChange_Type_LabelChange Type LabeldImpact_Date_LabelImpact Date LabeldProject_Name_LabelProject Name LabeldDate_NoCmdbarsVVModules VVReportsVV0VVBlob s(PropDataCTypeInfoDDirDatag0:<=Babc2e g/hXiJj:k6$^I|@5-CG/ Ut1K>[+IM8{ |10@Changes̍ 'L}L;6t1+IM8{Change Type{nU5uB&f Et1+IM8{Impact DategYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date NotedxWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/SavingsAJQ5-CG/Change TypeX !ORlK5-CG/Schedule Impact\+'^G"5-CG/Resource Impact(jWد-KVenF`5-CG/Other Costs/SavingsChangesChanges Arial82JhHP OfficeJet LX@f,,@MSUD$HP OfficeJet LXdT!winspoolHP OfficeJet LX\\PORSCHE\OfficeJetd278d eTimes New Romane24fg37jghm5C Arialn= Arialo3A Arialp4Change TypeImpact Date`ReportHeader|k(B~ cd2`<a<bcdLabel41Changes@ mHG^f2a<b2 Line44o5nAnZ(qC`PageHeader[o0cefnK< ON:Project NameProject NameValue ListL"Godzilla Phase I";"Godzilla Phase II"RName of project on which change occurred.Project_Namem;` bHckDate NotedDate Notedm/d/yyyyRDate on which the change was first noted.Date_NotedcJLhm`bP ck$Change Description$Change Description[0.00)4Sum_Of_Other_Costs_Savings7PGV<:Ym 0`-a:bck 4Avg Of Other Costs/Savings6=Avg([Other Costs/Savings])*$#,##0.00;($#,##0.00)4Avg_Of_Other_Costs_Savings;gujO(P9@m 0`-aWbck 4Min Of Other Costs/Savings6=Min([Other Costs/Savings])*$#,##0.00;($#,##0.00)4Min_Of_Other_Costs_Savingsq}2Cxr!W@m 0`-atbck 4Max Of Other Costs/Savings6=Max([Other Costs/Savings])*$#,##0.00;($#,##0.00)4Max_Of_Other_Costs_SavingsT'u%DpPd^v`PageFooterAׄOLm ;<`<abc,i j Text42 =Now()Long DateTimes New RomanL)]Jzͱ:m ;<`abc,i jk Text43H="Page " & [Page] & " of " & [Pages]Times New Roman$@*Ϗ O.WG+-\`hReportFooter<I3 m 5<`4&bc,i >Schedule Impact Grand Total Sum.=Sum([Schedule Impact])Times New Roman>Schedule_Impact_Grand_Total_SumM-j]C胨4d`<bGcd Label24Grand Total9 @Y& HLm 5<`)bc,i k>Resource Impact Grand Total Sum.=Sum([Resource Impact])Times New Roman>Resource_Impact_Grand_Total_Sum*`5JLY@?d`<bGcd Label30Grand TotalvEm(/Ny'm 05<`-bc,i kFOther Costs/Savings Grand Total Sum6=Sum([Other Costs/Savings])*$#,##0.00;($#,##0.00)Times New RomanFOther_Costs_Savings_Grand_Total_Sum<7X$KD1Iu&d`<bGcd Label36Grand Total7M˘) ted_LabelDate Noted LabeldChange_Description_LabelChange Description Labeld Impact_Description_LabelImpact Description Labeld Schedule_Impact_LabelSchedule Impact Labeld Resource_Impact_LabelResource Impact Labeld Other_Costs_Savings_LabelOther Costs/Savings Labelf Line45fLine46GroupHeader0mChange_TypeChange TypeDetailmImpact_DateImpact DateoProject_NameProject NamemDate_NotedDate NotedmChange_DescriptionChange DescriptionmImpact_DescriptionImpact DescriptionmSchedule_ImpactSchedule ImpactmResource_ImpactResource ImpactmOther_Costs_SavingsOther Costs/SavingsGroupFooter1mText18dLabel19dLabel20dLabel21dLabel22m Sum_Of_Schedule_ImpactSum Of Schedule Impactm!Avg_Of_Schedule_ImpactAvg Of Schedule Impactm"Min_Of_Schedule_ImpactMin Of Schedule Impactm#Max_Of_Schedule_ImpactMax Of Schedule Impactm$Sum_Of_Resource_ImpactSum Of Resource Impactm%Avg_Of_Resource_ImpactAvg Of Resource Impactm&Min_Of_Resource_ImpactMin Of Resource Impactm'Max_Of_Resource_ImpactMax Of Resource Impactm(Sum_Of_Other_Costs_SavingsSum Of Other Costs/SavK>[ingsm)Avg_Of_Other_Costs_SavingsAvg Of Other Costs/Savingsm*Min_Of_Other_Costs_SavingsMin Of Other Costs/Savingsm+Max_Of_Other_Costs_SavingsMax Of Other Costs/Savings,PageFooterm-Text42m.Text43/ReportFooterm0Schedule_Impact_Grand_Total_SumSchedule Impact Grand Total Sumd1Label24m2Resource_Impact_Grand_Total_SumResource Impact Grand Total Sumd3Label30m4Other_Costs_Savings_Grand_Total_SumOther Costs/Savings Grand Total Sumd5Label366ID7Cross-References8Test Impact9Recovery Plan:StatusChangesPropDatahScriptsVVPropDataiDatabasesVV 0CB0ijMSysDb0VVBlob DirDatajCustomGroups VVDataAccessPages VVRoot Entryʹ2VBAVVVBAProjectVVVBAVV   =@9>?7ABCD8FGHIJKLMNOPQRSTUVWXYCmdbarsVVModules > @ @ABCDEFGH Z X [ YgsForeignNameIdLvLvExtraLvModule LvPropName OwnerParentIdRmtInfoLongRmtInfoShortTypeniYYIdParentIdName        CvcP=*z @ -  } j C 0  m Z G p]I6#r_K8%ftrosanx?Ir CoYAOIndex         % &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp $ % &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp $ % T&tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp $ T&tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp $ >% >&tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp >$ % &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp $  %  &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp  $  %  &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp  $  $  &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp  %  $  &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp  % % % &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp % $ % &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp $ % % $ &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp % % $ &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp % % $ &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp % % &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp $ % % $ &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp % % $ &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp % $ % $ &tLfz'~X/. lBS"dҜ gW,' c~jCYti` t֢-&=±=9'w@OZ!_NIXqp % *LVAL> @ @ @ @ʾChangesCreate query in Design viewChangesChanges^3ʾChangesCreate query in Design viewChangesChanges^3DLVAL Ɉ`z @ -  } j C 0  m Z G MR2 GUIDNameMap 5-CG/  Ut1+IM8{Aa3@Changes̍ 'L}L;6t1+IM8{Change Type{nU5uB&f Et1+IM8{Impact DategYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date NotedxWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/SavingsAJQ5-CG/Change TypeX !ORlK5-CG/Schedule Impact\+'^G"5-CG/Resource Impact(jWد-KVenF`5-CG/Other Costs/SavingsMR2 GUIDNameMap 5-CG/  Ut1+IM8{Aa3@Changes̍ 'L}L;6t1+IM8{Change Type{nU5uB&f Et1+IM8{Impact DategYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date NotedxWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/SavingsAJQ5-CG/Change TypeX !ORlK5-CG/Schedule Impact\+'^G"5-CG/Resource Impact(jWد-KVenF`5-CG/Other Costs/SavingsMR2 GUIDNameMap b DID>j{N  Ut1+IM8{Aa3@ChangesDC&yt1+IM8{IDgYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date Noted{nU5uB&f Et1+IM8{Impact Date̍ 'L}L;6t1+IM8{Change TypexWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description5N7:K& t1+IM8{Recovery Plan)>*@%0t1+IM8{Cross-References :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/Savings@ dLpÄtt1+IM8{Test Impactoc@qѻmt1+IM8{StatusMR2J ReplicateProject Copyright Notice r jCopyright (c) 1998-2002 Rex Black All Rights ReservedMR20 Title AuthorCompanyb ZCase Study IVR Network Test Change Management Rex BlackN FRex Black Consulting Services, Inc.MR2.AccessVersion Build* 08.50 LVAL>^NMR2OrderByOnColumnWidthColumnOrderColumnHiddenDescription FormatDefaultValueRequiredValidationRuleAllowZeroLengthDisplayControlRowSourceTypeRowSourceBoundColumnColumnCountColumnHeadsListRowsListWidthLimitToListDecimalPlacesGUID FilterOrderByNameMap!  t1+IM8{  Ut1+IM8{ |10@ChangesDC&yt1+IM8{IDgYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date Noted{nU5uB&f Et1+IM8{Impact DatexWD.o]Nt1+IM8{Change Description̍ 'L}L;6t1+IM8{Change Type)>*@%0t1+IM8{Cross-References BGƎ\t1+IM8{Impact Description :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/Savings@ dLpÄtt1+IM8{Test Impact5N7:K& t1+IM8{Recovery Planoc@qѻmt1+IM8{Status ID   J BUnique identifier for the change. DC&yDate Noted   Z RDate on which the change was first noted. m/d/yyyy* "=DateValue(Now())  >#5/15/98# ց (Er|#0Impact Date   n fDate on which the change will first affect testing. m/d/yyyy* "=DateValue(Now())  >#5/15/98# {nU5uB&f EϿ*$Change Description   D <Short summary of what changed.    xWD.o]N Recovery Plan    |Short description of how we intend to accommodate this change.    5N7:K& Project Name   Z RName of project on which change occurred.  ""     o Value ListT L"Godzilla Phase I";"Godzilla Phase II"       0twip  gYJtbcT Status   r jLog of progress towards recovering from this problem.    oc@qѻm$Schedule Impact   ~ vNumber of calendar-days lost (slippage) due to this change.   0   m  :pwANYn[$Resource Impact   z rNumber of person-days lost (overtime) due to this change.   0   m o(#JUFtTest Impact   | tA short list of the test case IDs affected by LVAL N  this change.    @ dLpÄtM& Cross-References    A list of the specific test documents that are affected by the change, with page and paragraph numbers.    )>*@%0!,&Other Costs/Savings   h `Other costs/savings associated with this change.2 *$#,##0.00;($#,##0.00)   0  _qE ^A*$Impact Description    xShort description of what this change will mean for testing.     BGƎ\=Change Type   \ TType of change which is affecting testing.  ""     o Value List "Late Deliverable";"Early Deliverable";"Feature Addition";"Feature Change/Deletion";"Test Addition";"Test Change/Deletion";"Schedule Slipped";"Schedule Shortened";"Dependency";"Budget Increased";"Budget Reduced";"Quality Problem";"Quality Exceptional"       0twip  ̍ 'L}L;6LVALPFD ^   0|?0^  ^ ^ ^  @^ h^ ^ ^ ^ ^ H^ ^ ^ ^ ^ P^ x^ ^ 0^ X^ ^ ^ ^ ^ 8^ p^ ^ ^ ^ @^ h^ ^ H^ H^ H^ H^a H^ H^ H^ H^ H^ H^ H^ H^ H^ H^ @^ h^ ^ ^ ^ ^ H^ ^ ^ ^ ^ P^ x^ ^   d      d                   Changes.ID Changes'Changes.Project Name#Changes.Date Noted%Changes.Impact Date3Changes.Change Description%Changes.Change Type/Changes.Cross-References3Changes.Impact Description-Changes.Schedule Impact-Changes.Resource Impact5Changes.Other Costs/Savings%Changes.Test Impact)Changes.Recovery PlanChanges.Status^ x^ ^ X^  ^ w3@x^ ^ @^  h^  ^ ^ ^   ^  H^  ^ ^ ^  ^  P^  x^  ^  Changes ^  ^  ^ ^ ^ Aa3@~sq_fChanges^  0^ X ^  X^ ` ^  ^ h ^  ^ p ^  ^ x ^  ^ ^  8^ ^  p^ ^  ^ ^   ^ ^   ^ ^   @^ ^   h^ ^   ^ ^ 0^ X^ ^ ^ ^ ^ 8^ p^ ^ ^ ^ @^ h^ ^ @^ Sa;h^ Te^ ac^ ve^ n ^ usH^  ^  ^ ^  ^  P^  x^  ^  Changes @^ ^  h^ ^  ^ ^  ^ ^  ^ ^  ^ ^  H^ ^  ^ ^  ^ ^  ^ ^  ^ ^  P^ ^  x^ ^  ^ ^ 0^ X^ ^ ^ ^ ^ 8^ p^ ^ ^ ^ @^ h^ ^  ^ ^ ^  ^ 8^ p^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^  ^ ^ ^ ^ `^ ^  ^ ^  ^ ^  @^ ^  $^ ^  @^ ^  $@^ ^  $^ ^  ^ ^  ^ ^  @^ ^  $@^ ^  $@^ ^  $^ z ^ ^ ^ ^ ^ ^ ^ 8^ ^ p^ ^  ^ ^ ^ 0^  ^ @^  P^ P^ ^ `^ ^ p^ ^ ^  0^ ^  h^ ^  ^ ^ ^  d^ 8^ p^  ^  d^  ^  P^ ^ ^ ^  0^  h^  ^ `8^ ^ (^ 0^ 8^ @^ H^ P^ X^ `^ h^ p^ x^ ^ ^ ^ 8^ p^ ^ ^ ^ P^ ^ ^ ^ 0^ h^ ^ ^ Changes d^ ^  d^ Project NamePrimaryKeyChange Type8^ ^ v LVAL N P^ (^ 8^ ^ ^ ^q ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^  ^ ^ ChangesPrimaryKey^ ^ P^ ^ P^ ^ ^ ^ ^ ^ ^ ^ (LVAL|:0730: I'm not sure yet. 0825: Probably won't happen until end of January. 10/2: Now I'm pushing to gain engineers. 10/8: I should have a hiring requisition tomorrow, or the whole engagement may end. 10/22: Additional engineer is (almost) approved. 11/20: I now have about a dozen people working in the test team. Who'd have guessed by what was going on in July? 1/15: Never happened. 3/19: Well, the transition plan is upon us, but it's interesting to reflect that during manual testing something like 50 people were working for me!0827: This issue remains open. 0921: A snapshot: Per Rotem, "System administration support on the IVR boxes is not yet at a level that makes me feel confortable (last week I discovered that Bob has problems with .tar.gz files, and Samba support is something that David kludged up by screaming for help). Hardware support in the following areas: o Console support for clusters of IVR systems. I can track at least two Vindaloo run arounds that would have been avoided if we would have had a console hanging of the machine. o The Call Centre NT workstations are not here yet. James and I will probably need a couple of days to prepare those systems for testing. The sooner we get them, the better." 10/8: Probably the single biggest brake-pedal for the whole Test operation. 10/22: Looking better now, but issues remain. 1/15: Over the course of the project, I'd have to estimate--conservatively--the losses shown above. 3/19: As much as I like support team members personally, these folks were just overwhelmed by the complexity (logistical and technical) of what they were asked to do. I would rank the lack of adequate system administration resources to support the test environment as one of the top five inhibitors of testing progress, right after excessively optimistic schedules and the changes in project scope and definition.LVAL {tmf_XQJC<5.'  xqjc\UNG@92+$ | u n g ` Y R K D = 6 / ( !    y r k d ] V O H A : 3 , 9/3: The quality of TeleVendor's test effort remains an open issue. 9/18: Both quality and schedule of TeleVendor's test effort remains unaligned with ours. 10/2: I have major concerns about TeleVendor's ability to perform an adequate system test. 10/8: SWint has now accepted the responsibility to test and catch any test escapes that occur in terms of TeleVendor testing. 10/22: I need to audit IVR Telephony testing. Susie is noncommital and Chitra doesn't seem to understand the testing issues. 11/20: As the status log trend foretold, the telephony supporting infrastructure will show up almost entirely untested. 1/15: The lack of adequate testing of some of the components that make up the IVR system lead to significant problems in IVR system quality and subsequently in the ability to test the IVR system. We never completed all the planned IVR system tests.9/18: We're about to wrap up the third integration cycle BB[2] and the 2.11 components are still not here. 10/8: BB[2] is over. No single 2.11 component has yet entered pre-integration test. 11/20: The risk-based backbone integration has been dropped due to lack of interest. Most of the time spent to now was creating tools and instracture, so that's a scope cut that cost us considerable time. 1/15: We're about to pay the price for omitting the backbone integration testing in formal the Integration Test phase, because the quality isn't there and we now have to run all the same tests, but with a lessened ability to isolate bugs. 3/19: Many of the delays and quality problems in the formal Integration Test phase could be traced, at least partially, to the deletion of the backbone integration testing.LVAL  z1/16: I'm guessing that we probably saved about a month (non-critical path) of David's time, and gained back 15 person days. 3/19: The quality of the telephony software was problematic during System and Integration testing. This lead to significant delays and blockages in test progress.9/18: I confirmed that the timing loop is inside the SUT. This is bad. I escalate to project management. 1/15: See change record 117. This specific incident saved us about a week, because we didn't have to run tests. I lost about a day reworking the plan, though. 3/19: It's important to point out that the performance pSimonlems with the IVR system that plagued the project from November onwards could have been detected and resolved in September when they were easier to isolate that the self-generation of load for performance testing not been done.9/18: This is apparently now handled. 3/19. But it wasn't handled in September. This problem cropped up again in Integration Test and must have caused at least a dozen or so outages of an hour or two each.9/3: Basically, more confusion and problems associated with all the moving parts and quality issues in the telephony servers. 1/15: The previous record tracks all the time James and David spent developing CallSim.9/3: Never received any kind of support from SWint in this area. 9/18: SWint finally sent Len for a couple days to get Sunny going on our replacement for 5A/5B. The support was sufficient, finally, but about four months too late. 10/8: The telephony services won't scale up to 768 processes. SWint is claiming this is a test artifact. 1/15: I've estimated the schedule and resource impact for all the goat rodeos associated with having to replace 5A/5B (not including developing the new test tool). They are shown in this record.LVAL UD k  ` ?0k  k k k  @k hk k k k k Hk k k k k Pk xk k 0k Xk k k k k 8k pk k k k @k hk k Hk Hk Hk Hka Hk Hk Hk Hk Hk Hk Hk Hk Hk Hk @k hk k k k k Hk k k k k Pk xk k   d      d                   Changes.ID Changes'Changes.Project Name#Changes.Date Noted%Changes.Impact Date3Changes.Change Description%Changes.Change Type/Changes.Cross-References3Changes.Impact Description-Changes.Schedule Impact-Changes.Resource Impact5Changes.Other Costs/Savings%Changes.Test Impact)Changes.Recovery PlanChanges.Statusk xk k Xk  k w3@xk k @k `j hk `j k `j k `j k `j  k `j Hk `j k `j k `j k `j k `j Pk `j xk `j k `j Changes k  k  k k k Aa3@~sq_fChangesk  0k X k  Xk ` k  k h k  k p k  k x k  k k  8k k  pk k  k k   k k   k k   @k k   hk k   k k 0k Xk k k k k 8k pk k k k @k hk k @k `j hk `j k `j k `j k `j  k `j Hk `j k `j k `j k `j k `j Pk `j xk `j k `j Changes @k k  hk k  k k  k k  k k  k k  Hk k  k k  k k  k k  k k  Pk k  xk k  k k 0k Xk k k k k 8k pk k k k @k hk k  k k k  k 8k pk k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k  k k Pk k `k k  Pk k  Pk k  k k  $k k  k k  $k k  $Pk k  Pk k  Pk k  k k  $k k  $k k  $k z k k k lk k k k 8k k pk k  k k k 0k  k @k  Pk Pk k `k k pk k k  0k k  hk k  k k k  dk 8k pk  k  dk  k  Pk k k k  0k  hk  k `8k k (k 0k 8k @k Hk Pk Xk `k hk pk xk k k k 8k pk k k k Pk k k k 0k hk k k Changes dk k  dk Project NamePrimaryKeyChange Type8k k v LVAL N__Agg14,Sum([Resource Impact])S 7__Agg13,Sum([Schedule Impact])S 7__Agg124Max([Other Costs/Savings])[ 7__Agg114Min([Other Costs/Savings])[ 7__Agg104Avg([Other Costs/Savings])[ 7__Agg94Sum([Other Costs/Savings])Y 7__Agg8,Max([Resource Impact])Q 7__Agg7,Min([Resource Impact])Q 7__Agg6,Avg([Resource Impact])Q 7__Agg5,Sum([Resource Impact])Q 7__Agg4,Max([Schedule Impact])Q 7 __Agg3,Min([Schedule Impact])Q 7 __Agg2,Avg([Schedule Impact])Q 7 __Agg1,Sum([Schedule Impact])Q 7 __Agg0Count(*)5 7 *[Other Costs/Savings]C 3"[Resource Impact]; 3"[Schedule Impact]; 3([Impact Description]A 3([Change Description]A 3[Date Note Pk (k 8k k k kq k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k  k k ChangesPrimaryKeyPk k Pk k Pk Pk k k k k k k K>[ `NUt1+IM8{Aa3@Changes̍ 'L}L;6t1+IM8{Change Type{nU5uB&f Et1+IM8{Impact DategYJtbcTt1+IM8{Project Nameց (Er|#t1+IM8{Date NotedxWD.o]Nt1+IM8{Change Description BGƎ\t1+IM8{Impact Description :pwANYn[t1+IM8{Schedule Impacto(#JUFtt1+IM8{Resource Impact_qE ^At1+IM8{Other Costs/SavingsAJQ5-CG/Change TypeX !ORlK5-CG/Schedule Impact\+'^G"5-CG/Resource Impact(jWد-KVenF`5-CG/Other Costs/SavingsChangesChanges Arial82JhAcrobat PDFWriter$ odXX9#winspoolAcrobat PDFWriterLPT1:d278d eTimes New Romane24fg37jghm5C Arialn= Arialo3A Arialp4Change TypeImpact Date`ReportHeader|k(B~ cd2`<a<bcdLabel41Changes@ mHG^f2a<b2 Line44o5nAnZ(qC`PageHeader[d _ Z g  ~  k 1wV<:Ym 0`-a:bck 4Avg Of Other Costs/Savings6=Avg([Other Costs/Savings])*$#,##0.00;($#,##0.00)4Avg_Of_Other_Costs_Savings;gujO(P9@m 0`-aWbck 4Min Of Other Costs/Savings6=Min([Other Costs/Savings])*$#,##0.00;($#,##0.00)4Min_Of_Other_Costs_Savingsq}2Cxr!W@m 0`-atbck 4Max Of Other Costs/Savings6=Max([Other Costs/Savings])*$#,##0.00;($#,##0.00)4Max_Of_Other_Costs_SavingsT'u%DpPd^v`PageFooterAׄOLm ;<`<abc,i j Text42 =Now()Long DateTimes New RomanL)]Jzͱ:m ;<`abc,i jk Text43H="Page " & [Page] & " of " & [Pages]Times New Roman$@*Ϗ O.WG+-\`hReportFooter<I3 m 5<`4&bc,i >Schedule Impact Grand Total Sum.=Sum([Schedule Impact])Times New Roman>Schedule_Impact_Grand_Total_SumM-j]C胨4d`<bGcd Label24Grand Total9 @Y& HLm 5<`)bc,i k>Resource Impact Grand Total Sum.=Sum([Resource Impact])Times New Roman>Resource_Impact_Grand_Total_Sum*`5JLY@?d`<bGcd Label30Grand TotalvEm(/Ny'm 05<`-bc,i kFOther Costs/Savings Grand Total Sum6=Sum([Other Costs/Savings])*$#,##0.00;($#,##0.00)Times New RomanFOther_Costs_Savings_Grand_Total_Sum<7X$KD1Iu&d`<bGcd Label36Grand Total7M˘)  VVDataAccessPages VVon_LabelTkXA%sd7`4&a<bc,*Schedule Impact LabelScheduleDetachedLabel*Schedule_Impact_Label%,&i5SI5 d7`)a<bc,*Resource Impact LabelResourceDetachedLabel*Resource_Impact_Labellq3BI_ejZd7`-a<bc,2Other Costs/Savings Label&Other Costs/SavingsDetachedLabel2Other_Costs_Savings_Labelsx9CT)Tf2b2 Line45ljhwA%f2ab2 Line46ӵJA4`GroupHeader0=RXB%*)m 0<`<bcJi Change TypeChange TypeTType of change which is affecting testing.Times New RomanChange_TypeAJQ`J Detail-jdnJTJlm;`bHcImpact DateImpact Datem/d/yyyyfDate on which the change will first affect testing.Impact_Date=~NL;$e K>[ VVReportsVʹ20Vʹ2DataAccessPages VVPropDataCopyBlobCopyTypeInfoCopyBlob E (PropDataCTypeInfoDDirDatag#Test Impact$StatusChanges 0CB0ͬ;cj֟vF)7{ReportHeaderdLabel41fLine44PageHeaderdChange_Type_LabelChange Type LabeldImpact_Date_LabelImpact Date LabeldProject_Name_LabelProject Name LabeldDate_Noted_LabelDate Noted LabeldChange_Description_LabelChange Description Labeld Impact_Description_LabelImpact Description Labeld Schedule_Impact_LabelSchedule Impact Labeld Resource_Impact_LabelResource Impact Labeld Other_Costs_Savings_LabelOther Costs/Savings Labelf Line45fLine46GroupHeader0mChange_TypeChange TypeDetailmImpact_DateImpact DateoProject_NameProject NamemDate_NotedDate NotedmChange_DescriptionChange DescriptionmImpact_DescriptionImpact DescriptionmSchedule_ImpactSchedule ImpactmResource_ImpactResource ImpactmOther_Costs_SavingsOther Costs/SavingsGroupFooter1mText18dLabel19dLabel20dLabel21dLabel22m Sum_Of_Schedule_ImpactSum Of Schedule Impactm!Avg_Of_Schedule_ImpactAvg Of Schedule Impactm"Min_Of_Schedule_ImpactMin Of Schedule Impactm#Max_Of_Schedule_ImpactMax Of Schedule Impactm$Sum_Of_Resource_ImpactSum Of Resource Impactm%Avg_Of_Resource_ImpactAvg Of Resource Impactm&Min_Of_Resource_ImpactMin Of Resource Impactm'Max_Of_Resource_ImpactMax Of Resource Impactm(Sum_Of_Other_Costs_SavingsSum Of Other Costs/Savingsm)Avg_Of_Other_Costs_SavingsAvg Of Other Costs/Savingsm*Min_Of_Other_Costs_SavingsMin Of Other Costs/Savingsm+Max_Of_Other_Costs_SavingsMax Of Other Costs/Savings,PageFooterm-Text42m.Text43/ReportFooterm0Schedule_Impact_Grand_Total_SumSchedule Impact Grand Total Sumd1Label24m2Resource_Impact_Grand_Total_SumResource Impact Grand Total Sumd3Label30m4Other_Costs_Savings_Grand_Total_SumOther Costs/Savings Grand Total Sumd5Label366ID7Cross-References8Test Impact9Recovery Plan:StatusChanges0:<=Babc2e g/hXiJj:k!^I|@5-CG/  K>[t NameValue ListL"Godzilla Phase I";"Godzilla Phase II"RName of project on which change occurred.Project_Namem;` bHckDate NotedDate Notedm/d/yyyyRDate on which the change was first noted.Date_NotedcJLhm`bP ck$Change Description$Change Description