Orig­i­nal source pub­li­ca­tion: Capeça, G. e F. de Sá-Soares (2015). Avali­ação da Resil­iên­cia de Sis­temas de Infor­mação. Pro­ceed­ings of the 12^th Inter­na­tional Con­fer­ence on Infor­ma­tion Sys­tems and Tech­nol­ogy Man­age­ment—CON­TECSI 2015. São Paulo (Brasil).
The final pub­li­ca­tion is avail­able here.

Eval­u­a­tion of Infor­ma­tion Sys­tems Resilience

Gilberto Capeça and Fil­ipe de Sá-Soares

Cen­ter ALGO­RITMI, Uni­ver­sity of Minho, Guimarães, Por­tu­gal

1. Introduction

In recent times, orga­ni­za­tions have been faced with an increas­ingly com­plex envi­ron­ment in which their sur­vival is highly depen­dent on their abil­ity to deal with uncer­tain­ties and dis­rup­tions of vary­ing mag­ni­tudes. Among the chal­lenges that orga­ni­za­tions face are tech­no­log­i­cal obso­les­cence, geopo­lit­i­cal shocks, reg­u­la­tory and leg­isla­tive changes and the emer­gence of new busi­ness mod­els. Although orga­ni­za­tions are required to demon­strate reac­tive capa­bil­i­ties in the face of this diver­sity of chal­lenges, just as impor­tant, if not more so, is the need to develop capa­bil­i­ties to antic­i­pate and mit­i­gate risks in an increas­ingly unpre­dictable and volatile busi­ness envi­ron­ment [Oh and Teo 2009]. This paper argues that one way for orga­ni­za­tions to reduce their vul­ner­a­bil­i­ties to expected or unex­pected adverse events is to become resilient orga­ni­za­tions. Orga­ni­za­tional resilience is under­stood here as the abil­ity of an orga­ni­za­tion to recover its basic func­tions after expe­ri­enc­ing any inter­rup­tion or dis­rup­tion to its oper­a­tions [Neaga 2010].

Among the var­i­ous aspects that con­trib­ute to an orga­ni­za­tion’s resilience, it is worth con­sid­er­ing the role played by Infor­ma­tion Tech­nol­ogy (IT). In fact, the use of IT by orga­ni­za­tions has inten­si­fied and these tech­nolo­gies have become one of the pil­lars on which orga­ni­za­tions base their busi­ness processes. Today, IT sup­ports almost every busi­ness strat­egy, and it is essen­tial that its appli­ca­tion be prop­erly aligned with the orga­ni­za­tion’s strat­egy [Croteau and Begeron 2001].

Rec­og­niz­ing the poten­tial of IT to achieve suc­cess in orga­ni­za­tions, whether by improv­ing the flow of infor­ma­tion between processes, reduc­ing bureau­cracy or cre­at­ing com­par­a­tive advan­tages [Silva and Fer­reira 2006], it is there­fore impor­tant to ensure that these tech­nolo­gies remain oper­a­tional even in the pres­ence of adverse sit­u­a­tions. How­ever, it is argued that the issue of IT resilience should not be approached in iso­la­tion from the resilience of Infor­ma­tion Sys­tems (IS). This under­stand­ing stems from the fact that IS are con­ceived as sys­tems which, being made up of peo­ple, pro­ce­dures, data and IT, col­lect, store, process and dis­trib­ute infor­ma­tion with a view to meet­ing spe­cific objec­tives [Tur­ban et al. 1999], embody­ing an orga­ni­za­tion’s inter­nal and exter­nal infor­ma­tion flows [Gou­veia and Ran­ito 2004, p. 24].

To ensure that the infor­ma­tion sys­tem is prop­erly mon­i­tored, pro­tected, secure and avail­able to pro­vide its ser­vices and sup­port the orga­ni­za­tion’s busi­ness processes, even in the pres­ence of adverse sit­u­a­tions, it is nec­es­sary to ensure that it is resilient. Just as IT resilience in an orga­ni­za­tion should not be seen in iso­la­tion from infor­ma­tion sys­tem resilience, it is also argued that infor­ma­tion sys­tem resilience is not some­thing iso­lated within the orga­ni­za­tion. The resilience of the infor­ma­tion sys­tem must be framed by the orga­ni­za­tion’s strat­egy and the deci­sions arisen there­from, par­tic­u­larly with regard to choos­ing the most appro­pri­ate tech­nol­ogy; defin­ing, design­ing and redesign­ing processes, choos­ing and hir­ing the best staff; defin­ing and mon­i­tor­ing the desired lev­els of IS resilience, so that IT and IS help to improve busi­ness processes, deliver ser­vices effec­tively and effi­ciently, mit­i­gate ambi­gu­ity and uncer­tainty in deci­sion-mak­ing, and pro­vide ongo­ing sup­port for the orga­ni­za­tion’s oper­a­tions, both in the course of nor­mal sit­u­a­tions and in the face of dis­rup­tive sit­u­a­tions, whether inter­nal or exter­nal to the orga­ni­za­tion.

For an orga­ni­za­tion to prop­erly man­age its resilience efforts, both orga­ni­za­tion­ally and in terms of its IS, it needs to be able to assess its resilience at any given time. In this way, it will first be able to under­stand the degree to which its capa­bil­ity to be resilient is present, and from there draw ele­ments on which to sup­port ini­tia­tives aimed at main­tain­ing or improv­ing that degree. Sub­se­quently, a new assess­ment will allow it to know how effec­tive those ini­tia­tives have been, by under­tak­ing inter­nal bench­mark­ing exer­cises or even exter­nal bench­mark­ing, com­par­ing itself with part­ner orga­ni­za­tions or com­peti­tors. Specif­i­cally, this assess­ment of IS resilience would have to take the form of a process for mea­sur­ing that orga­ni­za­tional capa­bil­ity. How­ever, although there are stud­ies that address the issue of IS resilience and the need to assess IS resilience, none of them address the cre­ation of an instru­ment to mea­sure IS resilience. In fact, by hav­ing such a tool, orga­ni­za­tions will be able to take pre­cau­tions in order to antic­i­pate the res­o­lu­tion of issues that have to do with their nor­mal func­tion­ing, namely by iden­ti­fy­ing aspects that require inter­ven­tion in order to strengthen their resilience capa­bil­i­ties. Sim­i­larly, the exis­tence of such a tool would also help orga­ni­za­tions to take cer­tain mea­sures after an inci­dent or dis­as­ter has occurred, point­ing out ways in which orga­ni­za­tions can learn to recover their IS more effec­tively.

The instru­ment for mea­sur­ing IS resilience should bring together the dimen­sions con­sid­ered crit­i­cal to the orga­ni­za­tion’s sur­vival. This instru­ment should, based on pre­vi­ously iden­ti­fied objec­tives, estab­lish spe­cific mea­sur­able and achiev­able tar­gets, as well as defin­ing indi­ca­tors that pro­vide a man­age­ment vision to sup­port efforts to make the orga­ni­za­tion resilient from an IS point of view.

Hence, it is con­sid­ered that the con­struc­tion of an instru­ment to mea­sure the resilience of IS is extremely impor­tant at the present time, since through this instru­ment it will be pos­si­ble to define met­rics to deter­mine the extent to which the IS of a given orga­ni­za­tion are resilient and, con­se­quently, to mea­sure the resilience of the orga­ni­za­tion itself, open­ing up per­spec­tives for its improve­ment.

The pur­pose of this study will there­fore be to cre­ate and val­i­date an instru­ment for mea­sur­ing IS resilience. The cre­ation of such an instru­ment should help orga­ni­za­tions to mea­sure, i.e., assess the resilience of their IS, iden­tify their crit­i­cal sys­tems and ver­ify the weak­nesses of their IS. To a cer­tain extent, by apply­ing this tool, orga­ni­za­tions should be bet­ter equipped to pre­dict the occur­rence of adverse sit­u­a­tions and try to antic­i­pate the res­o­lu­tion of these sit­u­a­tions.

Method­olog­i­cally, this goal will be achieved by meet­ing the fol­low­ing spe­cific objec­tives:

1. Crit­i­cal analy­sis of the exist­ing lit­er­a­ture on resilience, orga­ni­za­tional resilience and infor­ma­tion sys­tems resilience;

2. Char­ac­ter­i­za­tion of the con­cept of resilience, orga­ni­za­tional resilience and infor­ma­tion sys­tems resilience;

3. Devel­op­ment of an instru­ment to mea­sure the resilience of infor­ma­tion sys­tems;

4. Val­i­da­tion of the mea­sur­ing instru­ment;

5. Explo­ration of the eval­u­a­tion of the mea­sure­ment instru­ment in dif­fer­ent orga­ni­za­tional and cul­tural con­texts.

This ongo­ing research aims to under­stand and describe the rela­tion­ship between orga­ni­za­tional resilience and the resilience of infor­ma­tion sys­tems and the way in which infor­ma­tion sys­tems recover in the pres­ence of a sit­u­a­tion that impairs their nor­mal func­tion­ing. It also aims to con­trib­ute to increas­ing knowl­edge about the resilience of infor­ma­tion sys­tems. In par­tic­u­lar, the aim is to develop a tool to help assess and improve the resilience of infor­ma­tion sys­tems in orga­ni­za­tions.

Struc­turally, this arti­cle is orga­nized into four sec­tions. After this intro­duc­tion, the lit­er­a­ture on resilience and infor­ma­tion sys­tems is reviewed, focus­ing on the con­cept of infor­ma­tion sys­tem resilience, the com­pet­ing fac­tors for resilience and the assess­ment of infor­ma­tion sys­tem resilience. This is fol­lowed by an expla­na­tion of the process of build­ing an instru­ment to mea­sure the resilience of infor­ma­tion sys­tems. Finally, a set of con­clud­ing remarks on this study is included.

2. Resilience and Information Systems

Resilience is a rel­a­tively new topic in the aca­d­e­mic world and has sparked var­i­ous debates, as shown by the exis­tence of sev­eral stud­ies on resilience [Bar­lach et al. 2008; Bhamra et al. 2011; Cata­lan and Robert 2011; Car­pen­ter et al. 2001; Chabot 2008; Coutu 2002; Erol et al. 2010; Evans and Steven 2009; Folke 2006; Holling 1973; Park and Shar­man 2008; Robert 2010; Seville 2008; Starr et al. 2003; Wang et al. 2010].

The need to inves­ti­gate and deepen stud­ies on resilience, fun­da­men­tally on the resilience of infor­ma­tion sys­tems, results from the grow­ing impor­tance that IS assume in the strat­egy of orga­ni­za­tions, given that they are a key ele­ment in improv­ing busi­ness processes. These sys­tems, gen­er­ally made up of tech­no­log­i­cal infra­struc­tures, data, appli­ca­tions and peo­ple, have the daily task of ensur­ing that the orga­ni­za­tion’s infor­ma­tion needs are met in the man­ner and within the time­frames defined, thus enabling these sys­tems to meet the objec­tives for which they were designed.

Although orga­ni­za­tions use IS to improve their oper­a­tions and obtain ben­e­fits from exploit­ing the ser­vices pro­vided by those sys­tems, they are also sub­ject to a wide range of risks, such as those aris­ing from the use of mali­cious pro­grams, human error, intru­sions, inad­e­quate secu­rity poli­cies, denial of ser­vice attacks, phys­i­cal and nat­ural acci­dents, sys­tem mal­func­tions or out­dated soft­ware [Ahmed and Hus­sain 2007, p. 7]. Given the role that IS plays in orga­ni­za­tions, which is to sup­port the busi­ness strat­egy, it is essen­tial that these sys­tems are resilient in order to help the orga­ni­za­tion sur­vive in the face of adverse con­di­tions.

Given the pur­pose of this study, it is impor­tant to first look at the con­cept of infor­ma­tion sys­tems resilience.

2.1 Concept of Information Systems Resilience

Resilience has been stud­ied in dif­fer­ent con­texts, such as orga­ni­za­tions, com­mu­ni­ties, ecol­ogy, engi­neer­ing, and infor­ma­tion sys­tems. How­ever, there are few stud­ies on IS resilience.
The term resilience is used in var­i­ous areas of knowl­edge to address flex­i­bil­ity, the capac­ity for self-renewal, adap­ta­tion to change, among other char­ac­ter­is­tics related to the apti­tude needed to pos­i­tively over­come adverse and risky sit­u­a­tions [Car­mona et. al. 2013]. In the con­text of IS, there are stud­ies that address the resilience of infor­ma­tion sys­tems, such as those devel­oped by Ahmed and Hus­sain [2007], But­ler and Gray [2006], Dalziell and Mac­manus [2004], Haimes [2009], Leve­son [2006], Leve­son et al. [2006] Madni and Jack­son [2009]; Park and Shar­man [2008], Riolli and Sav­icki [2003] and Wang et al. [2010]. Based on the stud­ies reviewed, var­i­ous def­i­n­i­tions of the con­cept of IS resilience were iso­lated and con­densed in Table 1.

Table 1: Def­i­n­i­tions of Infor­ma­tion Sys­tems Resilience

Although the def­i­n­i­tions pre­sented by the var­i­ous authors are dif­fer­ent, they all con­verge on the abil­ity to recover from inci­dents in order to min­i­mize losses or dam­age. In a com­pi­la­tion of def­i­n­i­tions sug­gested by var­i­ous researchers con­ducted by Erol et. al [2010], the main char­ac­ter­is­tics of IS resilience are the abil­ity of the sys­tem to respond adap­tively to a dis­turb­ing event in order to avoid losses; the abil­ity to recover quickly in a period of time and at an accept­able cost, allow­ing the sys­tem to con­tinue to func­tion in order to achieve its objec­tives, main­tain­ing con­trol over its oper­a­tion and struc­ture, exhibit­ing a capac­ity for self-orga­ni­za­tion, learn­ing and adap­ta­tion. In this way, Bar­lach et. al. [2008] argue that, regard­less of the con­text in which it is used, resilience is related to the abil­ity of an ele­ment to return to its nor­mal state after suf­fer­ing a dis­rup­tion in its func­tion­ing. Thus, although the scope of resilience may vary, its fun­da­men­tal prin­ci­ple remains the same: the adjust­ment of an ele­ment or sys­tem fol­low­ing a dis­tur­bance or dis­rup­tion.

As men­tioned, IS resilience is not some­thing iso­lated within the orga­ni­za­tion, but must be linked to the orga­ni­za­tion’s strat­egy. Given that struc­ture, cul­ture, pol­icy, pro­ce­dures, the sur­round­ing envi­ron­ment and man­age­ment deci­sions are aspects of the orga­ni­za­tion that can have a direct impact and con­di­tion the way in which IT is inte­grated into the orga­ni­za­tion [de Sá-Soares 1998, p. 46], and bear­ing in mind that tech­nolo­gies alone do not add value or con­trib­ute to the suc­cess of the orga­ni­za­tion if they are not sup­port­ing the orga­ni­za­tional sys­tems in an inte­grated and coher­ent way with the orga­ni­za­tion’s objec­tives and strat­egy [Foina 2009, p. 19], it is impor­tant that the resilience of infor­ma­tion sys­tems is seen as one of the con­stituent and con­trib­u­tory aspects of orga­ni­za­tional resilience. For this to hap­pen, a pre­req­ui­site is the exis­tence of strate­gic align­ment between Infor­ma­tion Sys­tems and Tech­nol­ogy (IST) and the orga­ni­za­tion’s busi­ness strat­egy.

The align­ment between IT and the busi­ness refers to the appli­ca­tion and exploita­tion of IT, in an appro­pri­ate and timely man­ner, in har­mony with the objec­tives, needs and strat­egy of the busi­ness. To achieve align­ment, the orga­ni­za­tion needs to con­sider how IT is aligned with the busi­ness, as well as how the busi­ness could or should be aligned with IT [Lufman 2000]. The search for align­ment is a con­tin­u­ous process of adjust­ments that orga­ni­za­tions under­take to achieve the link between busi­ness objec­tives and strate­gies and IS objec­tives and strate­gies, in order to obtain com­pet­i­tive advan­tages [Affeldt and Vanti 2009].

It fol­lows that it is not enough for IST to be aligned with the busi­ness, it is impor­tant for the busi­ness to under­stand the impor­tance of IST in help­ing the orga­ni­za­tion achieve its objec­tives. In other words, the busi­ness and IST must adapt their strate­gies together. There­fore, as far as resilience is con­cerned, it is well-founded that IS resilience will only be use­ful to the extent that IST is aligned with busi­ness strat­egy, because there would be lit­tle or no point in recov­er­ing IS fol­low­ing an adverse sit­u­a­tion if it con­trib­uted lit­tle or noth­ing to the value cre­ated by the orga­ni­za­tion. How­ever, it is also essen­tial to rec­og­nize that orga­ni­za­tional resilience is largely a func­tion of the resilience of its IS. It is believed that few orga­ni­za­tions will be able to show high lev­els of resilience if they are not able to have an orga­nized aggre­gate of peo­ple, tech­nolo­gies, data and pro­ce­dures, in other words, an infor­ma­tion sys­tem, which allows them to respond to inter­rup­tions or dis­rup­tions in their oper­a­tion.

Given that the resilience of Infor­ma­tion Sys­tems is the abil­ity of IS to adapt to dis­tur­bances or adverse con­di­tions and return to their ini­tial state offer­ing the min­i­mum ser­vices required for the con­ti­nu­ity of the orga­ni­za­tion’s busi­ness, and since the resilience of IS should not be treated in iso­la­tion within the orga­ni­za­tion, or should not be the sole ini­tia­tive or con­cern of IT man­agers, but should be aligned with the busi­ness strat­egy, it is essen­tial that top man­age­ment is com­mit­ted to the suc­cess of the IS within the orga­ni­za­tion.

2.2 Contributing Factors for Resilience

As pre­vi­ously pointed out, this paper argues that IS resilience is not some­thing iso­lated from the resilience of the orga­ni­za­tion itself. In fact, it is advo­cated that IS resilience is inte­grated into orga­ni­za­tional resilience, since an orga­ni­za­tion aims to be resilient in order to guar­an­tee the unin­ter­rupt­ibil­ity of its busi­ness oper­a­tions.

The Aus­tralian National Audit Office [2009, p. 3] lists inter­re­lated activ­i­ties that work together to pre­vent and man­age a sig­nif­i­cant busi­ness inter­rup­tion event for an orga­ni­za­tion, namely:

• Busi­ness Con­ti­nu­ity Man­age­ment (which includes IT dis­as­ter recov­ery);

• Risk Man­age­ment;

• Emer­gency Response Man­age­ment and

• Inci­dent Man­age­ment.

The inte­gra­tion of these four activ­i­ties is seen as a suc­cess fac­tor for build­ing orga­ni­za­tional resilience, as they offer a strate­gic, tac­ti­cal and oper­a­tional response to a busi­ness inter­rup­tion. The rela­tion­ship between these fun­da­men­tal activ­i­ties is illus­trated in Fig­ure 1.

Fig­ure 1: Rela­tion­ship between Risk, Emer­gency Response, Inci­dent Man­age­ment and Busi­ness Con­ti­nu­ity Man­age­ment
Source: Aus­tralian National Audit Office [2009, p. 2]

Busi­ness Con­ti­nu­ity Man­age­ment (BCM) is about devel­op­ing, imple­ment­ing and main­tain­ing frame­works, poli­cies, pro­ce­dures and pro­grams to help an orga­ni­za­tion man­age a busi­ness inter­rup­tion, thereby con­tribut­ing to strength­en­ing its resilience. The tasks included in BCM assist in pre­vent­ing, prepar­ing for, respond­ing to, man­ag­ing and recov­er­ing from the impact of the dis­rup­tion event. BCM focuses on deal­ing with the neg­a­tive con­se­quences of an event for the orga­ni­za­tion and cre­ates oppor­tu­ni­ties for orga­ni­za­tional ben­e­fits and gains, in that enti­ties that respond sat­is­fac­to­rily to dis­rup­tion events can posi­tion them­selves to recover quickly in the short term and improve their busi­ness per­for­mance in the medium to long term.

One of the com­po­nents of an orga­ni­za­tion’s busi­ness con­ti­nu­ity strat­egy is IT dis­as­ter recov­ery. This term is used to describe the oper­a­tional responses asso­ci­ated with the recov­ery of IT-based resources. Typ­i­cally, these resources include infor­ma­tion pro­cess­ing com­puter sys­tems and tele­com­mu­ni­ca­tions. IT dis­as­ter recov­ery involves defin­ing an over­all strat­egy for the recov­ery of these resources and the activ­i­ties required to imple­ment that strat­egy, includ­ing the recov­ery time for each spe­cific tech­nol­ogy com­po­nent as required by the busi­ness, the avail­abil­ity of suit­ably qual­i­fied per­son­nel and the pro­vi­sion of spe­cial­ized equip­ment.

Emer­gency Response Man­age­ment is an activ­ity that is car­ried out imme­di­ately after an inci­dent has occurred, and can be thought of as the tac­ti­cal man­age­ment of the sit­u­a­tion. The pri­mary con­cern of emer­gency response is the safety of peo­ple. This can include evac­u­at­ing build­ings, liais­ing with the emer­gency ser­vices, start­ing to assess the dam­age that has occurred and the impli­ca­tions for man­age­ment.

Inci­dent Man­age­ment cor­re­sponds to the gen­eral man­age­ment of the inci­dent and includes the strate­gic deci­sion-mak­ing process, involv­ing obtain­ing infor­ma­tion about the inci­dent, decid­ing that the inci­dent is esca­lated to busi­ness inter­rup­tion and trig­ger­ing the pro­ce­dures set out in the busi­ness con­ti­nu­ity plan, when this proves nec­es­sary. It also involves man­ag­ing com­mu­ni­ca­tion with stake­hold­ers, staff and other inter­ested par­ties.

All orga­ni­za­tions face a vari­ety of risks. The best orga­ni­za­tional risk man­age­ment prac­tices are based on adopt­ing a struc­tured and sys­tem­atic process to iden­tify and treat risks and imple­ment appro­pri­ate con­trols that act to mit­i­gate the effects of dis­rup­tive events.

Fig­ure 1 shows that the four activ­i­ties men­tioned are orga­nized to form a response cycle to busi­ness inter­rup­tions, con­sist­ing of inci­dent pre­ven­tion and oper­a­tional, tac­ti­cal and strate­gic response actions.

An alter­na­tive view of the fac­tors that con­trib­ute to orga­ni­za­tional resilience was put for­ward by Stephen­son [2010], who pre­sented a model of inte­grated func­tions that sug­gests that orga­ni­za­tional resilience is the result of the com­bi­na­tion of the activ­i­ties in Fig­ure 2.

Fig­ure 2: Inte­grated Func­tions Model
Adapted from Stephen­son [2010, p. 41]

From this fig­ure, it can be inferred that the func­tions of secu­rity man­age­ment, busi­ness con­ti­nu­ity man­age­ment, emer­gency man­age­ment and cri­sis man­age­ment are largely inter­re­lated, and that the effec­tive man­age­ment of these four func­tions pro­vides the orga­ni­za­tion with bet­ter risk man­age­ment. More­over, an orga­ni­za­tion that improves its risk man­age­ment becomes more resilient.

The stud­ies reviewed in this sec­tion focus on orga­ni­za­tional resilience, specif­i­cally on the activ­i­ties that con­trib­ute to the devel­op­ment of an orga­ni­za­tion’s resilience capa­bil­ity. Under­ly­ing these expo­si­tions is the idea that an orga­ni­za­tion’s level of resilience will depend on how well it car­ries out those activ­i­ties, either each one per se or by artic­u­lat­ing them together.

In view of the above, the pro­posal for the cur­rent study is to look for empir­i­cal evi­dence of the rel­e­vance of these activ­i­ties to the resilience of infor­ma­tion sys­tems. In other words, we will try to ascer­tain whether, in the spe­cific case of IS resilience, the activ­i­ties men­tioned by the authors con­trib­ute to this capa­bil­ity, whether these activ­i­ties take dif­fer­ent forms in the case of IS or whether there are other activ­i­ties or fac­tors that can estab­lish the degree of resilience of an orga­ni­za­tion’s IS. It is thought that the con­clu­sions of this search could be a first impor­tant con­tri­bu­tion to this work, namely by cre­at­ing the basis for the process of eval­u­at­ing the resilience of infor­ma­tion sys­tems.

2.3 Assessing the Resilience of Information Systems

The resilience of infor­ma­tion sys­tems is per­ceived in this study as an orga­ni­za­tional capa­bil­ity. Since the orga­ni­za­tion is a dynamic entity, inter­act­ing with its envi­ron­ment and with the poten­tial to under­take change, it is argued that this capa­bil­ity could (indeed, should) be man­aged by the orga­ni­za­tion itself, with the aim of main­tain­ing or strength­en­ing it. In either case, the orga­ni­za­tion will have to make it pos­si­ble to assess its degree of IS resilience, because only then will it be able to diag­nose the value of that capa­bil­ity at a given time, iso­late areas or com­pe­ten­cies that it will need improve­ment to increase that value and check to what extent those improve­ment ini­tia­tives have been effec­tive or not.

For an orga­ni­za­tion to be able to assess itself in terms of IS resilience, it will need to know how to mea­sure this capa­bil­i­tyc­ity. In fact, good admin­is­tra­tive prac­tice dic­tates that all activ­i­ties car­ried out should be mea­sured and com­pared with pre-estab­lished tar­gets. The mea­sure­ment of any com­plex char­ac­ter­is­tic usu­ally involves the appli­ca­tion of a mea­sur­ing instru­ment.

Since the aim of this research is to cre­ate an instru­ment to mea­sure the resilience of infor­ma­tion sys­tems, fol­low­ing Madni and Jack­son [2009], the devel­op­ment of resilience met­rics and indi­ca­tors is rec­og­nized as a fun­da­men­tal aspect.

The cre­ation of a mea­sur­ing instru­ment implies knowl­edge of the rules for devel­op­ing this instru­ment. It also implies knowl­edge of the sub­ject of mea­sure­ment.

Accord­ing to Nun­nally [1967, p. 2], mea­sure­ment con­sists of rules for assign­ing num­bers to objects to rep­re­sent quan­ti­ties of attrib­utes. The term ‘rules’ indi­cates that the pro­ce­dure for assign­ing num­bers can be explic­itly for­mu­lated. In some instances, the rules are so obvi­ous that detailed for­mu­la­tions are not nec­es­sary. In that def­i­n­i­tion of mea­sure­ment, the term ‘attribute’ indi­cates that mea­sure­ment always relates to some char­ac­ter­is­tics of objects, i.e., one does not mea­sure the object—one mea­sures its attrib­utes. In fact, Nun­nally notes that mea­sure­ment pre­sup­poses a process of abstrac­tion. An attribute refers to the rela­tion­ships between objects in a par­tic­u­lar dimen­sion. For exam­ple, a red stone and a white stone can have the same weight, and two white stones have dif­fer­ent weights. Thus, the attribute weight is an abstrac­tion that can­not be con­fused with all the par­tic­u­lar char­ac­ter­is­tics of the object.

Another rea­son for empha­siz­ing that mea­sure­ment always con­cerns a par­tic­u­lar attribute is to force us to care­fully con­sider the nature of an attribute before attempt­ing to mea­sure it. One pos­si­bil­ity is that the attribute does not exist. Another pos­si­bil­ity is that the mea­sure­ment may con­cern a mix­ture of attrib­utes rather than just one attribute. This often occurs in mea­sure­ment adjust­ments for ques­tion­naires, which tend to count items related to a num­ber of sep­a­ra­ble attrib­utes.

Con­sid­er­ing the def­i­n­i­tion of mea­sure­ment, it should be noted that num­bers are used to rep­re­sent quan­ti­ties. Quan­tifi­ca­tion is con­cerned with how much of an attribute is present in an object, and num­bers are used to com­mu­ni­cate quan­ti­ties. In the same vein, Viswanathan [2005] notes that sci­en­tific mea­sure­ment is made up of rules that allow num­bers to be assigned to objects in order to rep­re­sent quan­ti­ties of attrib­utes. Or that mea­sure­ment con­sists of rules for assign­ing sym­bols that (1) rep­re­sent numer­i­cally scaled quan­ti­ties or (2) define whether objects fall into the same or dif­fer­ent cat­e­gories in rela­tion to a given attribute (clas­si­fi­ca­tion). One can see here that when we are mea­sur­ing some­thing, we are assign­ing num­bers to rep­re­sent a cer­tain quan­tity and this assign­ment has to be made accord­ing to pre­de­ter­mined rules. In other words, when it comes to mea­sur­ing, one has to take into account (i) the attrib­utes of the object and (ii) the rules that will guide the mea­sure­ment.
Firstly, the attrib­utes of objects are mea­sured, not the objects them­selves. In turn, rules refer to every­thing that needs to be done to mea­sure some­thing. The cen­tral­ity of num­bers in the def­i­n­i­tion of mea­sure­ment results from sev­eral rea­sons: (a) num­bers are stan­dard­ized media and facil­i­tate com­mu­ni­ca­tion in sci­ence; (b) num­bers can be sub­jected to sta­tis­ti­cal analy­sis, and (c) num­bers are pre­cise.

For Zanolli [2007], the best approach for select­ing appro­pri­ate mea­sures is to first define what the orga­ni­za­tion needs or wants to know, and then choose the appro­pri­ate mea­sures. Based on the strate­gic objec­tives, it is pos­si­ble to set spe­cific, mea­sur­able, achiev­able and time-bound tar­gets and define indi­ca­tors and mea­sures that will give man­age­ment vis­i­bil­ity to achieve these objec­tives. Each indi­ca­tor must have a defined pur­pose and must answer the fol­low­ing ques­tions:

• What is the need for this infor­ma­tion?

• What do I want to achieve with this infor­ma­tion?

Accord­ing to that author, mea­sure­ments should be used to com­pare the cur­rent sit­u­a­tion with the desired sit­u­a­tion. Through a com­par­i­son, the trend of these mea­sure­ments is ana­lyzed and the nec­es­sary actions are defined to achieve the estab­lished objec­tive. In fact, with­out facts it is prac­ti­cally impos­si­ble to develop an improve­ment process and make deci­sions in line with the orga­ni­za­tion’s objec­tives. Indi­ca­tors and mea­sure­ments can tell you how far the orga­ni­za­tion is from achiev­ing its goals.

It is impor­tant to bear in mind that the ben­e­fits obtained from tak­ing mea­sure­ments are the result of deci­sions made based on the analy­sis of the data and not just the action of col­lect­ing the data. In addi­tion to col­lect­ing and ana­lyz­ing mea­sure­ments, they must also be dis­sem­i­nated.

An effec­tive mea­sure­ment and analy­sis process is of cru­cial impor­tance for obtain­ing knowl­edge about the orga­ni­za­tion’s real per­for­mance. Through mea­sure­ment, it is pos­si­ble to obtain answers that are expected to be suf­fi­ciently well founded regard­ing pre­vi­ously iden­ti­fied prob­lems, as well as to dis­cover and seek solu­tions to new prob­lems in fairly short peri­ods of time.

Hence, to find out to what extent infor­ma­tion sys­tems are resilient, it is essen­tial to build a tool to mea­sure this resilience.

3. Construction of the Information Systems Resilience Measurement Tool

From the lit­er­a­ture review, we iden­ti­fied the research prob­lem of not know­ing how to assess the resilience of infor­ma­tion sys­tems, and for­mu­lated the research ques­tion “How can we mea­sure the resilience of infor­ma­tion sys­tems?” and stated as a the­sis the fea­si­bil­ity of con­struct­ing a valid mea­sur­ing instru­ment for the resilience of infor­ma­tion sys­tems.

In order to con­tinue this work, it is impor­tant to plan the research from a method­olog­i­cal point of view. This study will there­fore adopt the method for devel­op­ing mea­sures pro­posed by Churchill [1979]. Fig­ure 3 shows a sequence dia­gram of the spe­cific steps to be fol­lowed in devel­op­ing mea­sures, which embody that method.

Fig­ure 3: Method for Devel­op­ing Mea­sures
Adapted from Churchill [1979, p. 66]

Accord­ing to this method, the process of devel­op­ing a mea­sur­ing instru­ment begins by spec­i­fy­ing the domain of the con­struct, in this case the resilience of infor­ma­tion sys­tems, char­ac­ter­iz­ing the con­cept and dis­tin­guish­ing it from other related con­cepts. Next, a sam­ple of items is gen­er­ated, which should be orga­nized accord­ing to the dimen­sions (attrib­utes) that are sup­posed to make up the con­struct. Sub­se­quently, data is col­lected on the items and the mea­sures are puri­fied based on data analy­sis. This activ­ity may require revis­it­ing pre­vi­ous steps in the process (clar­i­fy­ing the con­struct, dis­card­ing dimen­sions, merg­ing dimen­sions, adding new dimen­sions or rework­ing items, for exam­ple).

Once the mea­sure­ments have been puri­fied, more data will have to be col­lected to assess the reli­a­bil­ity and valid­ity of the mea­sure­ments, steps that, again, could lead to a review of pre­vi­ous phases of the process. Finally, stan­dards or rules are devel­oped for the mea­sure­ment process.

From a philo­soph­i­cal point of view, a post-pos­i­tivist per­spec­tive is adopted, as it will allow two dif­fer­ent research tech­niques to be used - quan­ti­ta­tive and qual­i­ta­tive - but which are believed to com­ple­ment each other when col­lect­ing data. These two tech­niques offer dif­fer­ent ways of val­i­dat­ing results and using them in par­al­lel makes the results more reli­able.

To this end, a ques­tion­naire will be used as a data col­lec­tion tool to gauge the orga­ni­za­tion’s per­cep­tion of the attrib­utes that could char­ac­ter­ize the resilience of the orga­ni­za­tion’s infor­ma­tion sys­tems. The ques­tion­naire is a data col­lec­tion instru­ment, con­sist­ing of an ordered series of pre-pre­pared ques­tions, sys­tem­at­i­cally and sequen­tially arranged into items that con­sti­tute the sub­ject of the research, which must be answered in writ­ing and with­out the pres­ence of the researcher, thus deal­ing with a planned inter­locu­tion [Moresi 2003, p. 65]. The ques­tion­naire is being devel­oped based on the results of the lit­er­a­ture review car­ried out, par­tic­u­larly with regard to the dimen­sions that poten­tially char­ac­ter­ize the “infor­ma­tion sys­tems resilience” con­struct, such as those related to the fac­tors seen as con­cern­ing orga­ni­za­tional resilience (risk man­age­ment, busi­ness con­ti­nu­ity man­age­ment, etc.) and the com­po­nents of infor­ma­tion sys­tems, namely IT (com­put­ing and com­mu­ni­ca­tions infra­struc­tures, as well as com­puter appli­ca­tions), peo­ple, data and poli­cies/pro­ce­dures.

Once we have a first ver­sion of the ques­tion­naire, a pre-test will be car­ried out to try to assess the valid­ity of the instru­ment, namely whether the ques­tions pre­sented in the ques­tion­naire are suf­fi­cient to guar­an­tee an effec­tive mea­sure­ment of Infor­ma­tion Sys­tems Resilience. We will also try to assess whether the ques­tions are clear, whether the con­tent of each ques­tion is suf­fi­cient and what rec­om­men­da­tions there are for improve­ment, if any. The pre-test will be val­i­dated by a group of IS researchers and a panel of senior pro­fes­sion­als from the IS, Risk Man­age­ment, Orga­ni­za­tion and Meth­ods and Inter­nal Audit areas. The ques­tion­naire will then be val­i­dated by pro­fes­sion­als who have respon­si­bil­ity for IT gov­er­nance and man­age­ment, as well as by IS, IT, Risk Man­age­ment and Com­pli­ance and Audit pro­fes­sion­als.

At the same time, semi-struc­tured inter­views will be used to col­lect nar­ra­tives from pre­vi­ously selected groups of indi­vid­u­als on the sub­ject under study. Given that this is also a qual­i­ta­tive study, it is felt that it would be more advan­ta­geous to carry out a sur­vey of a restricted group of indi­vid­u­als who man­age orga­ni­za­tions on a day-to-day basis and use IS for deci­sion-mak­ing pur­poses.

The semi-struc­tured inter­view method is based on draw­ing up a script, with a set of ques­tions, which is used in the inter­views and thus guar­an­tees some con­sis­tency between them (since the same script is always used). On the other hand, the script also gives the inter­viewer and the inter­vie­wee free­dom to bring up other sub­jects dur­ing the inter­view, or to ask other ques­tions. The inter­view script is being drawn up based on the lit­er­a­ture review, and is a key ele­ment in gath­er­ing the opin­ion of a group of pro­fes­sion­als work­ing in this field in what con­cerns the cre­ation of an instru­ment for mea­sur­ing the resilience of infor­ma­tion sys­tems. For this pur­pose, a pur­po­sive sam­ple, or non-prob­a­bilis­tic sam­ple by judg­ment, will be formed, since in this type of sam­ple the choice can be restricted to indi­vid­u­als who have knowl­edge and expe­ri­ence on the research topic.

4. Final Considerations

Given the increased sup­port pro­vided by infor­ma­tion sys­tems and tech­nol­ogy for busi­ness processes and the rapid pace at which these tech­nolo­gies are evolv­ing, it can be con­cluded that infor­ma­tion and the sys­tems that sup­port it have become vital ele­ments for orga­ni­za­tions. Con­se­quently, a fail­ure in their oper­a­tion could result in con­sid­er­able losses for the orga­ni­za­tion. Orga­ni­za­tions must there­fore be aware of the risks asso­ci­ated with their infor­ma­tion sys­tems and tech­nol­ogy and look for ways to guar­an­tee the oper­a­tional con­ti­nu­ity of their orga­ni­za­tions in the pres­ence of adverse con­di­tions.

This guar­an­tee will depend on the orga­ni­za­tion’s capa­bil­ity in terms of the resilience of its infor­ma­tion sys­tems and, at the same time, in terms of orga­ni­za­tional resilience. The cur­rent research iden­ti­fied the need to develop a valid instru­ment to assess the degree of resilience of an orga­ni­za­tion’s infor­ma­tion sys­tems. To this end, the lit­er­a­ture was reviewed and the work to be car­ried out was orga­nized with a view to cre­at­ing this instru­ment.

It is hoped that the met­rics defined for each of the dimen­sions on which the instru­ment is struc­tured will make it pos­si­ble to assess the resilience of infor­ma­tion sys­tems and help imple­ment­ing poli­cies, meth­ods and pro­ce­dures that make infor­ma­tion sys­tems more resilient. To this end, we will try to apply the instru­ment for mea­sur­ing the resilience of infor­ma­tion sys­tems in dif­fer­ent orga­ni­za­tional and cul­tural con­texts.

Acknowledgements

This work was sup­ported by FCT—Foun­da­tion for Sci­ence and Tech­nol­ogy under Pro­ject Scope UID/CEC/00319/2013.

References

• Ahmed, A. and S. Hus­sain (2007). Meta-Model of Resilient Infor­ma­tion Sys­tem. Mas­ter The­sis in Com­puter Sci­ence. Depart­ment of Inter­ac­tion and Sys­tem Design. School of Engi­neer­ing. Blekinge Insti­tute of Tech­nol­ogy. Swe­den.

  Affeldt, F. and A. Vanti (2009). Alin­hamento Estratégico e Tec­nolo­gia da Infor­mação: Análise de Mod­e­los e Pro­postas para Pesquisas Futuras. Revista de Gestão da Tec­nolo­gia e Sis­tema de Infor­mação 6(2), 203–226.

  Aus­tralian National Audit Office (2009). Busi­ness Con­ti­nu­ity Man­age­ment: Build­ing Resilience in Pub­lic Sec­tor Enti­ties, Best Prac­tice Guide. http://www.anao.gov.au/~/media/Uploads/doc­u­ments/busi­ness_con­ti­nu­ity_man­age­ment_.pdf

  Bar­lach, R., P. Cur­tis, J. Allen, D. White and L. Young (2008). Improv­ing Oper­a­tional Resilience Process. IEEE Inter­na­tional Con­fer­ence on Social Com­put­ing.

  Bhamra, R., S. Dani and K. Burnard (2011). Resilience: The Con­cept, a Lit­er­a­ture Review and Future Direc­tions. Inter­na­tional Jour­nal of Pro­duc­tion Research 49(18), 5375–5393.

  But­ler, B. and P. Gray (2006). Reli­a­bil­ity, Mind­ful­ness and Infor­ma­tion Sys­tems. MIS Quar­terly 30(2), 211–224.

  Car­mona, V., Y. Guimarães and L. Rodrigues (2013). Resil­iên­cia Orga­ni­za­cional: Uma Meta-Análise da Pro­dução Cien­tí­fica Nacional. Uni­ver­si­dade Nove de Julho. Brasil.

  Car­pen­ter, S., B. Walker, J. Anderies and N. Abel (2001). From Metaphor to Mea­sure­ment: Resilience of What to What? Ecosys­tems 4, 765–781.

  Cata­lan, C. and B. Robert (2011). Eval­u­a­tion of Orga­ni­za­tional Resilience: Appli­ca­tion in Que­bec. Pro­ceed­ings of the Fourth Resilience Engi­neer­ing Sym­po­sium, 50–57. Sophia Antipo­lis(France).

  Chabot, P. (2008). An His­tor­i­cal Case Study of Orga­ni­za­tional Resiliency within the Arel­lano-Felix Drug Traf­fick­ing Orga­ni­za­tion. PhD Dis­ser­ta­tion. Wash­ing­ton Uni­ver­sity.

  Churchill, G. A. (1979). A Par­a­digm for Devel­op­ing Bet­ter Mea­sures of Mar­ket­ing Con­structs. Jour­nal of Mar­ket­ing Research XVI, 64–73.

  Coutu, D. L. (2002). How Resilience Works. Har­vard Busi­ness Review 80(5), 50–52.

  Croteau, A. and F. Berg­eron (2001). An Infor­ma­tion Tech­nol­ogy Tril­ogy: Busi­ness Strat­egy, Tech­no­log­i­cal Deploy­ment and Orga­ni­za­tional Per­for­mance. Jour­nal of Strate­gic Infor­ma­tion Sys­tems 10, 77–79.

  Dalziell, E. P. and S. T. McManus (2004). Resilience, Vul­ner­a­bil­ity and Adap­tive Capac­ity: Impli­ca­tions for Sys­tems Per­for­mance. Inter­na­tional Forum for Engi­neer­ing Deci­sion Mak­ing (IFED). Stoos (Switzer­land).

  Erol, O., J. Sauser and M. Man­souri (2010): A Frame­work for Inves­ti­ga­tion into Extended Enter­prise Resilience. Enter­prise Infor­ma­tion Sys­tem 4(2), 111–136.

  Evans, A. and D. Steven (2009). Risks and Resilience in the New Global Era. Renewal 17(1), 44–52.

  Foina, P. (2009). Tec­nolo­gias de Infor­mação: Plane­ja­mento e Gestão, 2ª edição. São Paulo: Atlas.

  Folke, C. (2006). Resilience: The Emer­gence of a Per­spec­tive for Social-Eco­log­i­cal Sys­tems Analy­ses. Global Envi­ron­men­tal Change 16(3), 253–267.

  Gou­veia, L. and J. Ran­ito (2004). Sis­temas de Infor­mação de Apoio à Gestão. SPI - Sociedade Por­tuguesa de Ino­vação. Porto (Por­tu­gal).

  Haimes, Y. Y. (2009). On the Def­i­n­i­tion of Resilience in Sys­tems. Risk Analy­sis 29(4), 498–501.

  Holling, C. (1973). Resilience and Sta­bil­ity of Eco­log­i­cal Sys­tems. Insti­tute of Resource Ecol­ogy. Uni­ver­sity of Colum­bia, Van­cou­ver (Canada).

  Leve­son, N., N. Dulac, D. Zip­kin, J. Cutcher-Ger­snen­feld, J. Car­rol and B. Bar­rett (2006). Engi­neer­ing Resilience into Safety-Crit­i­cal Sys­tems. Mass­a­chu­setts Insti­tute of Tech­nol­ogy.

  Luft­man, J. (2000). Assess­ing Busi­ness-IT Align­ment Matu­rity. Com­mu­ni­ca­tions of Asso­ci­a­tion of Infor­ma­tion Sys­tems 4, Arti­cle 14.

  Madni, A.M. and S. Jack­son (2009). Towards a Con­cep­tual Frame­work for Resilience Engi­neer­ing. IEEE Sys­tems Jour­nal 3(2), 181–191.

  Moresi, E. (2000). Delin­e­ando o Valor do Sis­tema de Infor­mação de uma Orga­ni­za­ção. Ciên­cia da Infor­mação 29(1), 14–24.

  Neaga, I. (2010). Man­ag­ing Knowl­edge, Com­plex­ity and Resilience in Global Enter­prise. In Tome, E. (Ed.), Pro­ceed­ings of 11^th Euro­pean Con­fer­ence on Knowl­edge Man­age­ment, 717–723. Famal­icão (Por­tu­gal).

  Nun­nally, J. (1967). Psy­cho­me­t­ric The­ory. McGraw-Hill.

  Oh, L. and H. Teo (2009). An Empir­i­cal Study of IT-Enabled Enter­prise Risk Man­age­ment and Orga­ni­za­tional Resilience. CONF-IRM—Inter­na­tional Con­fer­ence on Infor­ma­tion Resources Man­age­ment.

  Park, I. and R. Shar­man (2008). Per­ceived Risk and Resilience in the Face of Nat­ural Dis­as­ters: A Study of Hos­pi­tal. AMCIS—Amer­i­cas Con­fer­ence on Infor­ma­tion Sys­tems.

  Riolli, L. and V. Sav­icki (2003). Infor­ma­tion Sys­tem Orga­ni­za­tional Resilience. Omega 31(3), 227–233.

  Robert, B. (2010). Orga­ni­za­tional Resilience—Con­cepts and Eval­u­a­tion Method. Presses Inter­na­tionales Poly­tech­nique, Québec(Canada).

  de Sá-Soares, D. (2008). Planea­mento de Sis­temas de Infor­mação: Estudo de Var­iáveis que Condi­cionam a sua Estraté­gia de Exe­cução. Mas­ter The­sis. Uni­ver­sity of Minho. Por­tu­gal.

  Seville, E. (2008). Resilience: Great Con­cept but What Does it Mean? Coun­cil on Com­pet­i­tive­ness—Risk Intel­li­gence and Resilience Work­shop, Novem­ber.

  Silva, W. and L. Fer­reira (2006). Importân­cia da Audi­to­ria dos Sis­temas de Infor­mação na Con­formi­dade das Demon­strações Con­tábeis.

  Starr, R., J. Newfrock and M. Dulurey (2003). Enter­prise Resilience: Man­ag­ing Risk in Net­worked Econ­omy. Strat­egy+Busi­ness Mag­a­zine.

  Stephen­son, A. (2010). Bench­mark­ing the Resilience of Orga­ni­za­tions. PhD Dis­ser­ta­tion. Uni­ver­sity of Can­ter­bury.

  Tur­ban, E., E. McLean and J. Wetherbe (1999). Infor­ma­tion Tech­nol­ogy for Man­age­ment—Mak­ing Con­nec­tions for Strate­gic Advan­tage, sec­ond edi­tion. New York: John Wiley.

  Viswanathan, M. (2005). Mea­sure­ment Error and Research Design. Thou­sand Oaks: Sage Pub­li­ca­tions.

  Wang, J. W., F. Gao and W. H. Ip (2010). Mea­sure­ment of Resilience and Its Appli­ca­tion to Enter­prise Infor­ma­tion Sys­tem. Enter­prise Infor­ma­tion Sys­tems 4(2), 215–223.

  Zanolli, J. (2007). Importân­cia das Medições para a Orga­ni­za­ção. Admin­is­tração e Negó­cios, 8 de Março.