case Solution template

Teaching Case

Bank SolutionsDisasterRecoveryand Business

Continuity:ACaseStudyfor CSIA 485

SteveCamara

Senior Manager, KPMG LLP

1021 E CaryStreet, Suite2000

Richmond, VA23219 scamara@kpmg.com

Robert Crossler Vishal Midha Assistant Professor

ComputerInformation Systems

TheUniversityof Texas– Pan American recrossler@utpa.edu, vmidha@utpa.edu

Linda Wallace

AssociateProfessor

AccountingandInformation Systems VirginiaTech wallacel@vt.edu

ABSTRACT

DisasterRecovery andBusinessContinuity(DR/BC) planningis anissue thatstudents willlikely come incontactwithas they enterindustry.  Many differentfieldsrequirethisknowledge,whetheremployeesareadvisingacompanyimplementinganew DR/BCprogram,auditingacompany’sexistingprogram,orimplementingand/orservingasakeyparticipantinacompany program.  Oftentimesintheclassroom itisdifficulttofindrealworldpracticeforstudentstoapply thetheoriestaught.  The informationinthiscase providesstudentswithrealworlddatatopracticewhattheywoulddoif theywereonanengagement teamevaluatingaDR/BCplan.  Providingstudentswiththisopportunitybetterpreparesthemforoneofthejobstheycould perform aftergraduation.  

Keywords: Casestudy,Computer security,Criticalthinking,Experientiallearning& education,Informationassurance and security,Role-play, Security,Teamprojects

2.CASE TEXT

2.1CompanyBackground

BankSolutions,Inc.(apseudonym),foundedin1973bythe

First  Presidential  Bank,  a  major  bank  of  its  time,  is  a providerofitem processingservicesitocommunitybanks, savingsandloanassociations,Internetbanks,andsmall-to mid-sizecreditunions.  It offersafullrangeof services, includingin-clearingand Proof ofDeposit(POD) processing, itemcapture,returnandexceptionitem processing,image archive storageandretrieval,andcustomerstatement rendering.

Bank Solutions wasformedin1973whenthe Chief OperatingOfficerof First PresidentialBank,amajor commercial bank, recognizedanopportunity.  Since item processingfunctionsarestandardized(they havetobein orderfor originating andreceiving financialinstitutionsto clearcustomertransactions) andscalablewithincreases in item processingvolumes,theywereabletoofferthese servicestootherfinancialinstitutionswishing to reduce operating expenseandfocus on growthstrategiesandother core business functions.  FirstPresidentialmarketedthese services underthe BankSolutionsbrandname.

Overthe next15years,Bank Solutionsenjoyedmodest growth.  By1988,itserved41small-tomid-sizefinancial institutions.     It had not, however,  developed  a  market

presenceoutsideoftheNorthwesternRegionoftheUnited States,asmanagement hadhoped.   Thiswas primarily because Bank Solutionswasunabletocompetewithother item-processing   service   providers   that   had   developed

proprietarysoftwaresystemsconsidered“topoftheline.” Tomakemattersworse,atthe timealmost one quarter of BankSolutions‟clientbasewassavingandloanassociations (savingandloans).   AsaresultoftheSavingsandLoan crisis,60%ofBankSolutions‟savingsandloancustomer base failedoverthe sixyears spanning 1985–1991,thus stuntingtheoutsourcer‟sgrowth.  Therelatedslowdownof the financialservicesandrealestateindustries andthe recessionof1990–1991presentedfurtherheadwindstothe growth  objectives  of  First  Presidential  management.      In

1994,FirstPresidentialsoldoffBankSolutions.

Undernewmanagement,BankSolutionsthrived.  Keys

tothe company‟s renewedsuccess includedthe following:

·    The  development  of  key  strategic  partnerships  with other  industry  participants,   including   data   clearing housesandfinancialinstitutioncore processing system outsourcers.ii

·    Theintroductionofanewcompanyculturethatfocused onopen doormanagement,mentoring,andenhanced employee benefits.

·    Thedevelopmentofaproprietary,stateoftheartitem processingsystem thatusesstate-of-the-artOptical CharacterRecognition(OCR)technology toachieve characterrecognitionaccuraciesthat were previously unheardof.

·    Theimplementationof“remotecapture”technologiesiii

to  meetelectronicbankinginitiativesand  regulations suchas“Check21.”

·    The  upgrade  or  replacement  of  other  administrative informationsystems,includingthecompany‟sfinancial reporting system.  Thishelpedtoincrease operational effectivenessandefficiencies.

From  1995–2008,  Bank  Solutions  enjoyed unprecedentedgrowth.  During thattimeframe,the company expanded  operations  to  18  item  processing  facilities,  two

datacentersinwhichtheitem processing systemwashosted, and345financial institutions.

2.2Current Scenario(2011)

DouglasSmith,theChief InformationOfficerforBank Solutions,wasoneof theoriginalmembersof“new management”andresponsibleformanyofBankSolutions‟ pastsuccesses.    A solid,middle-sizedcompanywith continuedgrowthpotential,BankSolutionshasbecomea

targetfora leveragedcorporatebuyout.   Thisisanattractive situationforDouglasandothermembersof executive management.   Severalof theseindividualsarecloseto retirement;andinitialindicationsarethatthepriceofthe

buyoutwillbeveryfavorableformembersof executive management.

TheCEOand other influentialmembersof executive managementwantBankSolutionsto  remain  an  attractive

purchase optionand,asaresult,havecontractedtheservices ofyourteamasanoutsideconsultanttoidentifyoperating andregulatory risksandadvisethem oncontrolmeasuresto mitigate the risks.

2.3RiskAssessmentTask

Asmembersoftheengagementteamperformingtherisk

assessment,yourteamhasbeengiventhetaskofassessing

BankSolutions‟incidenthandling,businesscontinuity,and disasterrecoverystrategy.

Inordertoperform theassessment,preliminary interviewswithDouglasSmith,theDataCenterManagers,

Systems  Engineers   and  Network  Architect  in  each  of BankingSolutions‟datacenters,andtheITManagersand Day  and  Night  Operations  Managers  from  seven  of  the largest    item    processing    facilities    were    conducted.

Additionally,the following documentationrelatedtoBank Solutions‟securityincidentmanagement,DR/BCplanning activitieswas reviewed:

·   Flowchartsthatdiagram theitemprocessingoperations anddataflow betweenBankSolutionsitem processing facilities  and  data  centers  and  outside  entities  (see

AppendixA)

·   AdiagramofBankSolutions‟network architecture

·   Bank  Solutions‟Data  Center  Disaster  Recovery  and

BusinessContinuityPlan(DRBCP)

·   Policies,procedures,guidelines,andstandardsrelated tosecurityincidentresponse

·   ItemProcessingFacilityDRBCPs

·   Results  from  the  most  recently  completed  DRBCP

test/exercise

·   Distributionlist forthe DRBCP

·   BankSolutions‟BackupandRecoveryPolicy.

·   Screen   prints   of   the   configurations   from   Bank

Solutions‟backup  utility  (these  configurations  show

what serversharesaresubject toautomated backupand the frequencyofthosebackups)

·   Contracts withtheoff-site storageprovider

·  A system-generatedlisting of accesstoeventlogging servers

·   Alistofindividualswhohavebeenprovidedaccessto recall backuptapes fromthe off-site storage vendor.

·  ScreenshotsoftheIntrusionDetectionSystem (IDS), firewall,and othereventlogging capability configurations

·  Excerptsfrom theIDSandfirewalleventlogsand management‟s manuallymaintainedincidenttracking log.

2.4 Facts: RiskAssessmentFindings

Based onthe discussionsheldwiththe managementanda

reviewofthe documentationprovided,younote the followingfacts:

1.     With  the  assistance  of  an  external  consultant,  Bank Solutions wrote its current data center DRBCPin2007. Itwas last updatedinJanuary2009.

2.     AccordingtoDouglas,thedatacenterDRBCPwaslast

testedin 2007.   Testingactivitiesconsistedof a conceptual,table-topwalkthroughof theDRBCP conductedbyDouglaswiththeDataCenterManagers andNetworkandSystemsEngineers.  Itemprocessing facilityDRBCPs have notyet beentested.

3.     Site-specificDRBCPshavebeenwrittenforthefive largestitemprocessingfacilities.    Theremainingitem processing  facilities  have  a  generic  “small  center”

DRBCPtemplate thatwas distributedtoandcustomized by facility managementinJune 2010.   Fouritem processing facilities have notyetcompletedthe customizationexercise.

4.     DRBCPs   contain   several   sections,   including   the following:

·   Emergency/crisis responseprocedures

·   Businessrecoveryprocedures

·   “Returnto normal” procedures

·   Various appendices

RecoveryTimeObjectivesandRecovery Point Objectivesiv   for  each  critical  business  process  and system   were   not   identified   in   the   DRBCP.   The

following details,mostofwhichareincludedinthe DRBCPappendices,are also documentedinthe text of the DRBCP:

·   Criticalsystems,includingdetailedhardwareand software inventories

·   Critical businessprocesses andprocessowners

·   Alternative   processing   facility   addresses   and

directions

·   “CallingTrees” (notificationlistings)

·   Critical  plan  participant  roles,  responsibilities,

andrequirements

·   Criticalvendorcontactlistings

·   Keybusinessforms

·   Specific recoveryprocedures forkeysystems

·   Procedures  for  managing  public  relations  and

communications

5.     Based  on  a  review  of  DRBCP  distribution  lists,  it appearsthatnotallkeyplanparticipantshaveacopyof

theplan.Whenthiswasdiscussed  withDouglas,he

respondedthatcopiesof allDRBCPsarestoredonthe network(whichisreplicatedacrossbothdatacenters

andvia backuptape).

6.     Criticalplanparticipantshavenotbeentrainedtouse

DRBCPs.

7.     BankSolutionshasimplemented  arobusthost-based

IDS,including detailedeventlogging andreporting capabilities.    However,  neither  the  DRBCP  nor  any otherpolicy,standard,guideline,or procedure addresses security incident handlingsteps,including escalation pointsof contactand proceduresforpreservingthe forensic qualities oflogicalevidence.

8.     Event  logging  is  also  performed  when  power  users perform  specific  privileged  activities  on  production

serversandselectedadministrative back office systems. Interestingly,  it  was  noted  that  several  of  the  same poweruserswhose actionsarerecorded ontoeventlogs also have write accesstothe logsthemselves.

9.     A  review  of  the  network  diagram  and  conversations withthe Network Architectrevealthatredundancies have beenimplementedatthe network perimeter (e.g., routers,firewalls,IDS,loadbalancers,etc.).

10.  BankingSolutionshasorganizedtheirDR/BCprogram

according toa“sistercenter”format;thatis,eachdata center  serves  as  the  other‟s  “hot  site”  processing locationandeachitem processingfacility hasbeen assignedacorrespondingitem processingfacility to serve asa backupprocessing location.   Neitherthe DRBCPsnoranyotherdocumentationoutlinespecific processingresponsibilities for backupfacilities.

11.  Onadailybasis,transactiondetailanditemimagefiles

fromthecurrentday‟s processingoperationsare uploadedfromeachitem processingfacility totheir regional data center (see AppendixA).

12.  At   the  data  centers,   electronic  vaulting  has  been

establishedwhereby alle-mail,file,andapplication serversand databases at the datacenter arecontinuously backedupto the other data centervia dual dedicated fiber optic lines.

13.A  data  backup  and  recovery  utility  has  been implemented  in  each  data  center  and  the  item processingfacilities.  Fullbackupsofcriticaldatafiles, softwareprograms,  and  configurations  are  performed

onceaweekandincrementalbackupsareperformedon a dailybasis MondaythroughFriday.

14.  At  one  item  processing  facility,  backup  jobs  have

routinely failed due tounknown causes.  Whenthe topic was  discussed  with  the  IT  Manager  on  duty,  he shruggedthefailuresoffnotingthatthecorefinancial institutiontransactiondataandimagesaretransmitted toandarchivedatthe BankSolutionsDataCenterEast onadailybasis.

15.Attheitemprocessingfacilities,themanagementhas beentaskedwithcontracting the off-sitestorage of backuptapes.  Atoneoftheitemprocessingfacilities, management has contractedthe bank across the streetto store  its  backup  tapes  in  a  safety  deposit  box.    At anotheritem processingfacility,thenightOperations Managerstoresthebackuptapesinasafeathishome. Atathirditem processingcenter,tapesarestoredina shedatthe backofthe building.

ii

Thisisindividualproject. Asa memberofanengagementteamincharge of performingthe incident handling,  DR/BC  risk  assessment  for  Bank Solutions.youshouldreadthecase backgroundand the facts identifiedinthe interviews.

IndividualWork:For allofthe facts/findings,preparea writtenreportthatliststhecondition(s)that presentrisksto Bank Solutionsaswellas proposedrecommendationsfor addressingthoseconditions.

JournalofInformationSystems Education,Vol.22(2)

Appendix A

Thiscasewasdevelopedsolelyforclassdiscussion.Whilethesituationdescribedinthiscaseisbasedonrealisticevents,theBankSolutionsisafictionalorganization. Further,thenames,product/serviceofferings,andthenamesofallindividualsinthecasearefictional.Anyresemblancetoactualcompanies,offerings,orindividualsis accidental.

122